Early warning and storage method, system, equipment and medium based on infrared video monitoring

Abstract

The present invention discloses an early warning and storage method, system, device and medium based on infrared video surveillance, belonging to the field of image processing and video surveillance early warning technology. The method comprises: obtaining a video shot by an infrared camera, extracting an infrared image using the video; using the Yolov12 network as a feature extraction network, improving the feature extraction network, and extracting a feature map of the infrared image through the improved feature extraction network; performing classification on the candidate area and outputting early warning information of the target object; and storing the infrared image containing the early warning information as a video in chronological order. The present invention adds an optional strip convolution attention layer in the feature extraction stage, and constructs a difference-guided multi-scale feature fusion pyramid in the feature fusion stage, which is helpful for infrared video surveillance early warning; the present invention performs screening storage on the surveillance video, and only saves the key video clips containing the warning object according to the detection results, effectively reducing the storage space occupied by invalid video.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and video surveillance early warning, and in particular to an early warning and storage method, system, equipment and medium based on infrared video surveillance. Background Art

[0002] Thermal infrared imaging technology has garnered widespread attention in recent years due to its advantages, including high concealment, low power consumption, all-weather operation, and strong anti-interference capabilities. This has been particularly evident in computer vision applications such as intelligent surveillance and autonomous driving. Using infrared cameras to capture images within a surveillance area and perform real-time detection of specific objects, such as people, ships, and vehicles, helps promptly detect accidents and quickly locate specific objects, significantly improving the efficiency of search and rescue missions and possessing significant application value in ensuring regional safety.

[0003] The features learned by convolutional neural networks cannot fully capture tiny and complex details, resulting in the inability of convolutional neural networks to distinguish small objects from the background or other objects with similar appearances during object detection and classification in images. Direct application of existing convolution-based image detection algorithms makes it difficult to distinguish small objects from background noise, resulting in a large number of false detections and missed detections. Its detection performance is insufficient to support the actual application scenarios of infrared small object monitoring and early warning.

[0004] Furthermore, infrared cameras continuously record the monitored area 24 hours a day, generating a massive amount of video data. Storing all of this data would require significant storage space. However, most of the time, the objects requiring warnings do not appear in these videos, resulting in wasted resources. This also makes subsequent playback of warning videos difficult, significantly reducing the efficiency of monitoring and warning operations. Summary of the Invention

[0005] Purpose of the invention: In response to the above problems, the purpose of the present invention is to provide an early warning and storage method, system, equipment and medium based on infrared video surveillance, which significantly enhances the ability to extract image features in an infrared environment, optimizes the feature fusion process, and improves the accuracy of identifying target objects in infrared images.

[0006] Technical solution: In the first aspect, the present invention provides an early warning and storage method based on infrared video monitoring, comprising the following steps:

[0007] Obtain a video captured by an infrared camera, extract infrared images from the video, preprocess the infrared images, construct a dataset of the preprocessed infrared images, and divide them into a training set and a validation set according to the ratio;

[0008] The Yolov12 network is used as the feature extraction network, and an optional strip convolution attention layer and a difference feature fusion network are constructed to improve the feature extraction network. The improved feature extraction network is used to extract the feature map of the infrared image, which contains the candidate area of ​​the target object.

[0009] Perform classification on the candidate area and output warning information of the target object;

[0010] The infrared images containing warning information are stored as videos in chronological order.

[0011] Furthermore, the improved feature extraction network includes a feature extraction backbone network, a difference feature fusion network and a detection head;

[0012] The feature extraction backbone network includes the first convolutional layer, the second convolutional layer, the first feature extraction layer, the third convolutional layer, the second feature extraction layer, the fourth convolutional layer, the third feature extraction layer, the fifth convolutional layer, the fourth feature extraction layer and an optional strip convolution attention layer connected in sequence.

[0013] Furthermore, the difference feature fusion network includes a first upsampling layer, a first difference fusion layer, a first feature fusion layer, a second upsampling layer, a second difference fusion layer, a second feature fusion layer, a sixth convolutional layer, a first connection layer, a third feature fusion layer, a seventh convolutional layer, a second connection layer, and a fourth feature fusion layer, which are sequentially connected;

[0014] Among them, the second feature extraction layer is connected to the second difference fusion layer, the third feature extraction layer is connected to the first difference fusion layer, the strip convolution attention layer can be optionally connected to the first upsampling layer and the second connection layer respectively, and the first feature fusion layer is connected to the first connection layer; the second feature fusion layer, the third feature fusion layer, and the fourth feature fusion layer are all connected to the detection head.

[0015] Furthermore, the input feature map of the optional strip convolution attention layer is denoted as X in , the output feature map of the optional strip convolution attention layer is recorded as X out , get the feature map X out The process includes:

[0016] Feature Map X in First, horizontal strip convolution and vertical strip convolution are performed to generate optional strip receptive fields, and then 1×1 convolution is performed to obtain the feature map F. H 、F V ;

[0017] The feature map F H 、F V Perform channel splicing to obtain the feature map F;

[0018] The spatial relationship is extracted from the feature map F using average pooling and maximum pooling, the spatial pooling features are connected through convolution, and the Sigmoid activation function is applied to the feature map obtained after spatial selection. The expression is:

[0019] ([ ]) ,

[0020] Among them, MaxPool is the maximum pooling, AvgPool is the average pooling, is the convolution operation, is the Sigmoid activation function;

[0021] Next, the output feature map They are weighted according to their corresponding spatial selections and fused using convolution to obtain the attention feature map S. The formula is:

[0022] ,

[0023] Finally, the output feature map X of the strip convolution attention layer can be selected out From the input feature map X in The element-wise product between and the attention feature map S is obtained as follows:

[0024] .

[0025] Furthermore, the feature map X in First, the steps of generating optional strip receptive fields through horizontal strip convolution and vertical strip convolution respectively include:

[0026] First, create horizontal and vertical convolution kernels using asymmetric padding P(L, R, T, B), where L, R, T, and B represent the number of padded pixels in the left, right, top, and bottom directions, respectively.

[0027] The input tensor is padded with asymmetric zeros in the horizontal direction P(k,0,1,0), P(0,k,0,1) and asymmetric zeros in the vertical direction P(0,1,k,0), P(1,0,0,k) to obtain the output tensor 、 、 、 ;

[0028] Then perform parallel convolution on the output tensor to obtain the tensor , the formula is:

[0029] ,

[0030] ,

[0031] ,

[0032] ,

[0033] ,

[0034] ,

[0035] in, Indicates that convolution is performed using a convolution kernel with a height of 1 and a width of k. Indicates that convolution is performed using a convolution kernel with a height of k and a width of 1. is the number of convolution kernel output channels, s is the convolution step size, and is the height and width of the output tensor; feature map X in The height, width and number of channels are 、 、 The input tensor of

[0036] The tensor and To splice, the tensor and Splice and get a new tensor. The formula is:

[0037] ,

[0038] ,

[0039] Among them, Cat represents the splicing operation;

[0040] Use a height of 2, width of 2, and output channels of The convolution pair and Normalize them separately to get the final horizontal strip convolution output tensor and vertical strip convolution tensors , the formula is:

[0041] ,

[0042] ,

[0043] ,

[0044] = ,

[0045] Where h2 and w2 represent the height and number of channels respectively.

[0046] Furthermore, the working process of the difference feature fusion network includes:

[0047] The feature maps of adjacent layers are respectively used and Represents pixel-by-pixel subtraction, absolute value operation and 3 × 3 convolution. , generate the difference feature map D, the formula is:

[0048] ,

[0049] Where, i∈{1,2}, Represent shallow features, middle features, and deep features respectively;

[0050] After that, the difference feature map D is activated by the Sigmoid function and then compared with the feature map and Multiply to get the weighted feature map and , the feature map and Add, and Add to generate a feature map containing multi-scale information and , the formulas are:

[0051] ,

[0052] ,

[0053] ,

[0054] ,

[0055] Feature Map and Make channel connections, then apply 3 3 convolution fusion, the fused features are multiplied by the difference features, and then 3 3 After convolution, the final fusion difference feature map is obtained , the formula is:

[0056] ,

[0057] in, Indicates channel feature connection.

[0058] Furthermore, the step of preprocessing the infrared image includes:

[0059] Randomly flip the image horizontally, randomly scale the image size, randomly crop the image area, randomly change the brightness, contrast, color saturation of the image, and perform Mosaic data enhancement on the image.

[0060] Furthermore, the warning information includes object category, confidence level and detection frame coordinates.

[0061] In a second aspect, the present invention provides an early warning and storage system based on infrared video surveillance, comprising:

[0062] An image acquisition module is used to acquire the video captured by the infrared camera and extract the infrared image using the video;

[0063] An image processing module, used for preprocessing infrared images;

[0064] Feature extraction module, used to extract feature maps from preprocessed images;

[0065] The warning and storage module is used to classify the candidate areas and output warning information of the target objects, and store the infrared images containing the warning information as videos in chronological order.

[0066] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0067] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect above.

[0068] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0069] 1. The improved Yolov12 network architecture of the present invention adds an optional strip convolution attention layer in the feature extraction stage and constructs a difference-guided multi-scale feature fusion pyramid in the feature fusion stage, which improves the accuracy of infrared image detection and facilitates infrared video surveillance and early warning.

[0070] 2. The present invention performs screening storage on surveillance videos and only saves key video clips containing warning targets based on the detection results, effectively reducing the storage space occupied by invalid videos, making it easier for users to quickly locate and find objects that need warnings, and improving the efficiency of users' playback of surveillance videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a flow chart of the early warning and storage method based on infrared video monitoring;

[0072] Figure 2 This is a schematic diagram of the structure of the improved feature extraction network;

[0073] Figure 3 Schematic diagram of the structure of the optional strip convolution attention layer;

[0074] Figure 4 It is a structural diagram of the difference feature fusion network;

[0075] Figure 5 Flowchart for recording warning videos;

[0076] Figure 6 This is a schematic diagram of early warning video storage. DETAILED DESCRIPTION

[0077] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0078] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0079] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0080] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0081] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized.

[0082] Example 1

[0083] Combine Figure 1 The early warning and storage method based on infrared video monitoring described in this embodiment includes the following steps:

[0084] Step 1: Obtain a video shot by an infrared camera, extract infrared images from the video, preprocess the infrared images, construct a dataset of the preprocessed infrared images, and divide them into a training set and a validation set according to the ratio.

[0085] In this example, an infrared camera captures video within a surveillance area and transmits the video stream to a computer connected to the camera. An SRS streaming server is set up on the local computer. The infrared camera video stream is encoded in H.264 using the open-source computer program FFmpeg. The video is then pushed to the SRS streaming server using the RTMP protocol, which generates a video stream access address. For the infrared video to be detected, the OpenCV server uses the RTMP video stream address as a parameter to establish a connection with the streaming server. The server then begins pulling video data. Based on the video stream encoding type, an appropriate decoder is selected to decode the video data into frame-by-frame image data. After selecting infrared images containing target objects such as people, cars, and boats, a total of 30,000 images are generated. These images are divided into training and validation datasets, for example, with an 8:2 ratio. Image annotation tools are used to annotate the target objects, such as people, cars, and boats, in the images. Data augmentation is then performed on the infrared images, including random horizontal flipping, random image scaling, random image cropping, random image brightness, contrast, and color saturation changes, and mosaic data augmentation.

[0086] Step 2: Use the Yolov12 network as the feature extraction network, construct an optional strip convolution attention layer and a difference feature fusion network, improve the feature extraction network, and extract the feature map of the infrared image through the improved feature extraction network. The feature map contains the candidate area of ​​the target object.

[0087] In this example, the Yolov12 network is selected as the feature extraction network, and the Yolov12 network is improved. Specifically, an optional strip convolution attention layer is set to be added to the feature extraction backbone network, and the difference feature fusion network architecture is used to optimize the feature fusion network.

[0088] Combine Figure 2 ,Furthermore, the improved feature extraction network includes a feature ,extraction backbone network, a difference feature fusion network, and a ,detection head;

[0089] The feature extraction backbone network includes the first convolutional layer, the second convolutional layer, the first feature extraction layer, the third convolutional layer, the second feature extraction layer, the fourth convolutional layer, the third feature extraction layer, the fifth convolutional layer, the fourth feature extraction layer and an optional strip convolution attention layer connected in sequence.

[0090] Furthermore, the difference feature fusion network includes a first upsampling layer, a first difference fusion layer, a first feature fusion layer, a second upsampling layer, a second difference fusion layer, a second feature fusion layer, a sixth convolutional layer, a first connection layer, a third feature fusion layer, a seventh convolutional layer, a second connection layer, and a fourth feature fusion layer, which are sequentially connected;

[0091] Among them, the second feature extraction layer is connected to the second difference fusion layer, the third feature extraction layer is connected to the first difference fusion layer, the strip convolution attention layer can be optionally connected to the first upsampling layer and the second connection layer respectively, and the first feature fusion layer is connected to the first connection layer; the second feature fusion layer, the third feature fusion layer, and the fourth feature fusion layer are all connected to the detection head.

[0092] In infrared images, people are typically distributed as vertical stripes, while boats and vehicles are distributed as horizontal stripes. In this example, a stripe convolution with a stripe-shaped receptive field is designed to simulate the target grayscale distribution, expanding the receptive field and enhancing feature extraction. The optional stripe convolution attention layer includes horizontal stripe convolution, vertical stripe convolution, and a spatial selection module. The input feature map is passed through horizontal and vertical stripe convolutions to generate a selectable stripe-shaped receptive field. To enhance the network's ability to focus on the most relevant spatial context, a spatial selection module is introduced to improve target object classification and localization.

[0093] Combine Figure 3 , further, the input feature map of the optional strip convolution attention layer is denoted as X in , the output feature map of the optional strip convolution attention layer is recorded as X out , get the feature map X out The process includes:

[0094] Feature Map X in First, horizontal strip convolution and vertical strip convolution are performed to generate optional strip receptive fields, and then 1×1 convolution is performed to obtain the feature map F. H and F V ;

[0095] The feature map F H and F V Perform channel splicing to obtain the feature map F;

[0096] The spatial relationship is extracted from the feature map F using average pooling and maximum pooling, the spatial pooling features are connected through convolution, and the Sigmoid activation function is applied to the feature map obtained after spatial selection. The expression is:

[0097] ([ ]) ,

[0098] Among them, MaxPool is the maximum pooling, AvgPool is the average pooling, is the convolution operation, is the Sigmoid activation function;

[0099] Next, the output feature map They are weighted according to their corresponding spatial selections and fused using convolution to obtain the attention feature map S. The formula is:

[0100] ,

[0101] Finally, the output feature map X of the strip convolution attention layer can be selected out From the input feature map X in The element-wise product between and the attention feature map S is obtained as follows:

[0102] .

[0103] Furthermore, the feature map X in First, the steps of generating optional strip receptive fields through horizontal strip convolution and vertical strip convolution respectively include:

[0104] First, create horizontal and vertical convolution kernels using asymmetric padding P(L, R, T, B), where L, R, T, and B represent the number of padded pixels in the left, right, top, and bottom directions, respectively.

[0105] The input tensor is padded with asymmetric zeros in the horizontal direction P(k,0,1,0), P(0,k,0,1) and asymmetric zeros in the vertical direction P(0,1,k,0), P(1,0,0,k) to obtain the output tensor 、 、 、 ;

[0106] Then perform parallel convolution on the output tensor to obtain the tensor , the formula is:

[0107] ,

[0108] ,

[0109] ,

[0110] ,

[0111] ,

[0112] ,

[0113] in, Indicates that convolution is performed using a convolution kernel with a height of 1 and a width of k. Indicates that convolution is performed using a convolution kernel with a height of k and a width of 1. is the number of convolution kernel output channels, s is the convolution step size, and is the height and width of the output tensor; feature map X in The height, width and number of channels are 、 、 The input tensor of

[0114] The tensor and To splice, the tensor and Splice and get a new tensor. The formula is:

[0115] ,

[0116] ,

[0117] Among them, Cat represents the splicing operation;

[0118] Use a height of 2, width of 2, and output channels of The convolution pair and Normalize them separately to get the final horizontal strip convolution output tensor and vertical strip convolution tensors , the formula is:

[0119] ,

[0120] ,

[0121] ,

[0122] = ,

[0123] Where h2 and w2 represent the height and number of channels respectively.

[0124] Shallow features typically contain richer details and edge information, while deep features contain more high-level semantic information. To fuse information from different scales and improve the model's understanding of both details and global information, deep features are upsampled and then subtracted pixel by pixel from shallow features to generate difference features. This subtraction allows the model to learn the differences between deep and shallow features, preventing loss of shallow features and retaining more detailed information.

[0125] Combine Figure 4 The structural diagram of the difference feature fusion network shown in FIG. 1 is further used to fuse features obtained from feature maps of adjacent layers and generate a difference feature map. The working process includes:

[0126] The feature maps of adjacent layers are respectively used and Represents pixel-by-pixel subtraction, absolute value operation and 3×3 convolution. , generate the difference feature map D, the formula is:

[0127] ,

[0128] Where, i∈{1,2}, Represent shallow features, middle features, and deep features respectively; the second feature extraction layer outputs shallow features F1, the third feature extraction layer outputs middle features F2, and the strip convolution attention layer can be selected to output deep features F3;

[0129] After that, the difference feature map D is activated by the Sigmoid function and then compared with the feature map and Multiply to get the weighted feature map and , the feature map and Add, and Add to generate a feature map containing multi-scale information and , the formulas are:

[0130] ,

[0131] ,

[0132] ,

[0133] ,

[0134] Feature Map and Make channel connections, then apply 3 3 convolution fusion, the fused features are multiplied by the difference features, and then 3 3 After convolution, the final fusion difference feature map is obtained , the formula is:

[0135] ,

[0136] in, Indicates channel feature connection.

[0137] Step 3: Classify the candidate areas and output warning information of the target objects.

[0138] For example, the second feature fusion layer outputs a feature map of size 80×80, the third feature fusion layer outputs a feature map of size 40×40, and the fourth feature fusion layer outputs a feature map of size 20×20. The feature maps of these three scales are subjected to the convolution operation of the detection head to obtain the target category, confidence and detection frame coordinates. All detection frames from the three scales will be merged, and then preliminarily screened according to the confidence threshold. Finally, redundant frames are removed by non-maximum suppression. The target category, confidence and detection frame coordinates corresponding to each remaining detection frame are the final warning information.

[0139] In this example, the improved feature extraction network is first trained using the image data and label data in the training set. The training process includes:

[0140] The training set images were resized to 640×640 and fed into the feature extraction backbone network for feature extraction. The three features of different scales output by the feature extraction backbone network were then fused through a differential feature fusion network, resulting in three feature maps of different scales: 80×80, 40×40, and 20×20. The three detection heads generated the object category, confidence score, and coordinates for each bounding box. The model's predictions were compared with the ground-truth annotated values, and a loss function, such as CIoU loss, was calculated. Based on the calculated loss, the gradient of the loss function with respect to the model parameters was calculated using the backpropagation algorithm. The model parameters were then updated using the SGD optimizer based on the calculated gradients. Training was performed on an NVIDIA GeForce RTX 4090D GPU and an Intel(R) Xeon(R) CPU E5-2680 v4. The software environment used was the Pytorch deep learning framework running on Ubuntu. The initial learning rate was set to 0.01, the number of training epochs was 200, and the batch size was 32.

[0141] After training, the images in the test set are fed into the improved feature extraction network. Each feature map at each scale outputs a set of predictions, including the target object category, confidence level, and bounding box coordinates. The outputs of the three detection heads are concatenated and confidence filtered, filtering out bounding boxes with confidence levels below a threshold of 0.2. Finally, non-maximum suppression is performed to eliminate redundant bounding boxes, retaining only the highest-confidence bounding boxes at the same target location to obtain the final detection results.

[0142] By traversing and reading each frame of image data and passing it as input to the improved feature extraction network, specific infrared objects can be detected, and the final detection results are obtained. Warning information such as object category, confidence, detection box coordinates, etc. are sent to the client server in JSON format.

[0143] Step 4: Store the infrared images containing the warning information as a video in chronological order.

[0144] Combine Figure 5 When the present invention detects the presence of one or more target objects in the current frame, if the system was not previously recording, it automatically starts recording, creates a new video file, and begins writing the current and subsequent frames to the file. If the system was previously recording, it continues writing the current frame to the current video file. If the system does not detect any target objects in the current frame, if it was previously recording, it terminates the writing operation, completes, and saves the current video file, thereby automatically storing the video clip containing the warning target.

[0145] like Figure 6 As shown in the figure, in the total detection video, warning video segments A, B, and C contain the target object, while the remaining video segments are ordinary videos. The system automatically records and stores warning videos A, B, and C, while ordinary video segments without the target object are not saved. This method effectively reduces the storage of irrelevant video data and significantly reduces the waste of storage resources. Furthermore, by retaining only key videos containing the warning target, the playback efficiency of the warning video is improved, allowing users to quickly locate and query relevant objects, thereby enhancing the system's responsiveness and ease of use in practical applications.

[0146] In one example, an infrared camera was used to continuously shoot a fixed area for 10 minutes. The video storage results are shown in Table 1. A total of 6 videos were saved, with a total length of 61 seconds. The video clips containing the warning objects were separated and saved from the original video, which greatly reduced the space wasted in storing invalid background videos and improved the efficiency of users in playing back surveillance videos and viewing warning targets.

[0147] Table 1 Video storage results

[0148]

[0149] Example 2

[0150] The infrared video surveillance-based early warning and storage system described in this embodiment includes:

[0151] An image acquisition module is used to acquire the video captured by the infrared camera and extract the infrared image using the video;

[0152] An image processing module, used for preprocessing infrared images;

[0153] Feature extraction module, used to extract feature maps from preprocessed images;

[0154] The warning and storage module is used to classify the candidate areas and output warning information of the target objects, and store the infrared images containing the warning information as videos in chronological order.

[0155] Example 3

[0156] This embodiment provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiment when executing the computer program.

[0157] This embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method described in the above method embodiment are implemented.

Claims

1. The early warning and storage method based on infrared video monitoring is characterized in that: The steps include: Obtain a video captured by an infrared camera, extract infrared images from the video, preprocess the infrared images, construct a dataset of the preprocessed infrared images, and divide them into a training set and a validation set according to the ratio; The Yolov12 network is used as the feature extraction network, and an optional strip convolution attention layer and a difference feature fusion network are constructed to improve the feature extraction network. The improved feature extraction network is used to extract the feature map of the infrared image, which contains the candidate area of ​​the target object. Perform classification on the candidate area and output warning information of the target object; The infrared images containing the warning information are stored as videos in chronological order; The improved feature extraction network includes a feature extraction backbone network, a difference feature fusion network, and a detection head; The feature extraction backbone network includes a first convolutional layer, a second convolutional layer, a first feature extraction layer, a third convolutional layer, a second feature extraction layer, a fourth convolutional layer, a third feature extraction layer, a fifth convolutional layer, a fourth feature extraction layer and an optional strip convolution attention layer connected in sequence; The input feature map of the optional strip convolution attention layer is denoted as X in , the output feature map of the optional strip convolution attention layer is recorded as X out , get the feature map X out The process includes: Feature Map X in First, horizontal strip convolution and vertical strip convolution are performed to generate optional strip receptive fields, and then 1×1 convolution is performed to obtain the feature map F. H 、F V ; The feature map F H 、F V Perform channel splicing to obtain the feature map F; The spatial relationship is extracted from the feature map F using average pooling and maximum pooling, the spatial pooling features are connected through convolution, and the Sigmoid activation function is applied to the feature map obtained after spatial selection. , the expression is: ([ ]) ， Among them, MaxPool is the maximum pooling, AvgPool is the average pooling, is the convolution operation, is the Sigmoid activation function; Next, the output feature map They are weighted according to their corresponding spatial selections and fused using convolution to obtain the attention feature map S. The formula is: ， Finally, the output feature map X of the strip convolution attention layer can be selected out From the input feature map X in The element-wise product between and the attention feature map S is obtained as follows: 。 2. The early warning and storage method based on infrared video monitoring according to claim 1 is characterized in that: The difference feature fusion network includes a first upsampling layer, a first difference fusion layer, a first feature fusion layer, a second upsampling layer, a second difference fusion layer, a second feature fusion layer, a sixth convolutional layer, a first connection layer, a third feature fusion layer, a seventh convolutional layer, a second connection layer, and a fourth feature fusion layer, which are connected in sequence; Among them, the second feature extraction layer is connected to the second difference fusion layer, the third feature extraction layer is connected to the first difference fusion layer, the strip convolution attention layer can be optionally connected to the first upsampling layer and the second connection layer respectively, and the first feature fusion layer is connected to the first connection layer; the second feature fusion layer, the third feature fusion layer, and the fourth feature fusion layer are all connected to the detection head.

3. The early warning and storage method based on infrared video monitoring according to claim 2 is characterized in that: Feature Map X in First, the steps of generating optional strip receptive fields through horizontal strip convolution and vertical strip convolution respectively include: First, create horizontal and vertical convolution kernels using asymmetric padding P(L, R, T, B), where L, R, T, and B represent the number of padded pixels in the left, right, top, and bottom directions, respectively. The input tensor is padded with asymmetric zeros in the horizontal direction P(k,0,1,0), P(0,k,0,1) and asymmetric zeros in the vertical direction P(0,1,k,0), P(1,0,0,k) to obtain the output tensor 、 、 、 ; Then perform parallel convolution on the output tensor to obtain the tensor , the formula is: ， ， ， ， ， ， in, Indicates that convolution is performed using a convolution kernel with a height of 1 and a width of k. Indicates that convolution is performed using a convolution kernel with a height of k and a width of 1. is the number of convolution kernel output channels, s is the convolution step size, and is the height and width of the output tensor; feature map X in The height, width and number of channels are 、 、 The input tensor of The tensor and To splice, the tensor and Splice and get a new tensor. The formula is: ， ， Among them, Cat represents the splicing operation; Use a height of 2, width of 2, and output channels of The convolution pair and Normalize them separately to get the final horizontal strip convolution output tensor and vertical strip convolution tensors , the formula is: ， ， ， = ， Where h2 and w2 represent the height and number of channels respectively.

4. The early warning and storage method based on infrared video monitoring according to claim 3 is characterized in that: The working process of the difference feature fusion network includes: The feature maps of adjacent layers are respectively used and Represents pixel-by-pixel subtraction, absolute value operation and 3 × 3 convolution. , generate the difference feature map D, the formula is: ， Where, i∈{1,2}, Represent shallow features, middle features, and deep features respectively; After that, the difference feature map D is activated by the Sigmoid function and then compared with the feature map and Multiply to get the weighted feature map and , the feature map and Add, and Add to generate a feature map containing multi-scale information and , the formulas are: ， ， ， ， Feature Map and Make channel connections, then apply 3 3 convolution fusion, the fused features are multiplied by the difference features, and then 3 3 After convolution, the final fusion difference feature map is obtained , the formula is: ， in, Indicates channel feature connection.

5. The early warning and storage method based on infrared video monitoring according to claim 1 is characterized in that: The steps of preprocessing infrared images include: Randomly flip the image horizontally, randomly scale the image size, randomly crop the image area, randomly change the brightness, contrast, color saturation of the image, and perform Mosaic data enhancement on the image.

6. The early warning and storage method based on infrared video monitoring according to claim 1 is characterized in that: The warning information includes object category, confidence level, and detection frame coordinates.

7. A system for implementing the early warning and storage method based on infrared video monitoring according to claim 1, characterized in that: include: An image acquisition module is used to acquire the video captured by the infrared camera and extract the infrared image using the video; An image processing module, used for preprocessing infrared images; Feature extraction module, used to extract feature maps from preprocessed images; The warning and storage module is used to classify the candidate areas and output warning information of the target objects, and store the infrared images containing the warning information as videos in chronological order.

8. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images