Infrared target detection method and model based on convolution attention

Abstract

The invention discloses an infrared target detection method and model based on convolution attention. The model comprises a backbone network, a neck network and a detection layer. Generating feature maps of multiple scales through repeated feature extraction and attention adjustment of the backbone network; performing multi-scale feature fusion on the multi-layer feature map through a neck network module to generate a final feature map suitable for detection; and converting the final feature map into a final target detection result through a detection layer. According to the method, small targets in the image can be more effectively concerned by introducing an attention mechanism, so that the detection precision is improved, detail information and context information of the image can be captured at the same time by utilizing a multi-scale feature fusion technology, the feature expression ability is enhanced, and the content of the image can be more carefully understood.

Description

Technical Field

[0001] The present invention belongs to the field of computer detection technology, and in particular relates to an infrared target detection method and model based on convolutional attention. Background Art

[0002] Object detection is a cornerstone of computer vision applications, including identifying, localizing, and segmenting objects in images or videos captured by visual cameras. It is the fundamental technique for countless visual object detection applications, even replacing other sensing or detection methods. Powerful detection algorithms are essential in areas such as autonomous driving and missile guidance, which require recovery from a variety of lighting and environmental conditions such as day, night, rain, and fog. However, traditional visible light systems often fail to perform under adverse conditions, rendering some tasks unachievable. In contrast, infrared imaging systems demonstrate robust adaptability to complex weather and lighting scenarios by capturing thermal radiation images of targets. They operate seamlessly day and night and have an extended detection range. In the field of infrared object detection, methods can be divided into traditional and deep learning-based methods. Traditional methods, especially single-frame detectors, have historically dominated due to their simplicity, high real-time performance, and ease of hardware implementation. These methods include target feature-based, background feature-based, and morphology-based techniques, each of which is tuned to enhance object detection under specific conditions. However, in environments characterized by complex backgrounds and low signal-to-noise ratios, traditional methods often struggle to implement, rely heavily on manual parameter tuning, and produce sub-par detection performance. At present, traditional methods are gradually being replaced by deep learning-based techniques. In recent years, deep learning-based detection algorithms have attracted much attention due to their excellent learning and characterization capabilities. These algorithms have been widely used in various target detection tasks, showing superior adaptability and accuracy. Notable research works include methods for detecting small vehicle targets and addressing the challenges posed by weak infrared targets. Although deep learning-based methods are effective, they may still make false predictions when detecting weak targets in complex backgrounds. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the defects of poor texture information, low resolution and high noise of infrared images in the prior art, and to provide an infrared target detection method based on convolutional attention for improving the accuracy and efficiency of infrared small target detection.

[0004] The technical solution adopted to solve the above technical problems is: an infrared target detection method based on convolutional attention, comprising the following steps:

[0005] Step 1. Generate feature maps of multiple scales through repeated feature extraction and attention adjustment of the backbone network

[0006] Step 1.1. Perform a two-dimensional convolution operation on the input infrared image to obtain a primary feature map;

[0007] Step 1.2. The primary feature map is normalized, then nonlinearly transformed through a rectified linear unit ReLU and output to the CBAM attention module;

[0008] Step 1.3. The CBAM attention module adjusts the input feature map and generates an adjusted feature map;

[0009] Step 1.4. Repeat steps 1.1 to 1.3 four times to generate four adjusted feature maps to form a multi-layer feature map;

[0010] Step 2: Perform multi-scale feature fusion on the multi-layer feature map through the neck network module to generate a final feature map suitable for detection

[0011] Step 2.1. Perform multi-scale feature fusion on the multi-layer feature map through the feature pyramid network FPN, and output the fused feature map;

[0012] Step 2.2. Further integrate the fused feature graph through a path aggregation network PAN and output it;

[0013] Step 2.3. Perform a convolution operation on the output of step 2.2 to generate a final feature map suitable for detection;

[0014] Step 3: Convert the final feature map into the final target detection result through the detection layer

[0015] Step 3.1. Process the final feature map using the optimized YOLO detection head structure to extract more refined features for target detection;

[0016] Step 3.2. Apply the Anchor mechanism, select the Anchor scale suitable for small objects, and adjust the bounding box position;

[0017] Step 3.3. Calculate the predicted class probability and bounding box offset;

[0018] Step 3.4. Introduce kernel weighted distance loss L NWD And the distance between the predicted box and the real box is obtained according to the following formula to reduce the positioning error of small targets;

[0019]

[0020] In the formula, is a weight function that calculates the distance or difference between the predicted box and the true box. is the predicted box, P is the real box, and Z is the normalization factor;

[0021] Step 3.5. Output the final detection results, including the category and location coordinates of the target.

[0022] As a preferred technical solution, the convolution operation in step 1.1 is performed according to the following formula:

[0023]

[0024] Where padding[·] is the padding operation, dilation[·] is the spacing of the convolution kernel points, kernel_size[·] is the size of the convolution kernel, stride[·] is the convolution step size, and H in is the pixel height of the input infrared image, W in is the pixel width of the input infrared image, H out is the pixel height of the output primary feature map, W out is the pixel width of the output primary feature map.

[0025] As a preferred technical solution, the normalization process in step 1.2 is:

[0026]

[0027] In the formula, BatchNorm2d(x) n,c,h,w is a two-dimensional batch normalization operation on the four-dimensional tensor x, n,c,h,w is the element of the input tensor x at the nth sample, cth channel, hth row, and wth column, H is the pixel height of the primary feature map, W is the pixel width of the primary feature map, and γ c is the normalized scaling factor, β c is the normalized offset, and ε is a constant not equal to 0.

[0028] As a preferred technical solution, the method for adjusting the attention of the CBAM attention module in step 1.3 is:

[0029] 1) Perform global average pooling and global maximum pooling on the input feature map according to the following formula;

[0030]

[0031] In the formula, F avg is the global average pooling result, which represents the average value of each channel in the input feature map, F max is the global maximum pooling result, which represents the maximum value of each channel in the input feature map. 1 is the input feature map;

[0032] 2) Use the shared fully connected layer to obtain the channel weight according to the following formula and use the sigmoid activation function to obtain the channel attention weight M c ;

[0033] M c =σ(W 1 δ(W 0 +F avg )+W 1 δ(W 0 F max ))

[0034] Where σ(·) is the Sigmoid activation function, δ is the ReLU activation function, and W 0 ,W 1 are all weight matrices of the fully connected layers;

[0035] 3) Perform average pooling and maximum pooling operations on the input feature map in the channel dimension according to the following formula, and stack the results to obtain the pooled feature map F pool , F pool =[AvgPool(F 1 );MaxPool(F 1 )];

[0036] 4) Feature map F after pooling pool Generate a spatial attention map M through a 7×7 convolution operation s ,

[0037] M s =δ(f (7x7) (F pool ))

[0038] In the formula, f 7×7 It is a 7×7 convolution operation used to generate a spatial attention map;

[0039] 5) Set the channel attention weight M c and the spatial attention map M s Applied to the input feature map F as follows 1 , get the adjusted feature map F out , F out =F 1 ×M c ×M s .

[0040] As a preferred technical solution, in step 2.1, the multi-scale feature fusion of the multi-layer feature map is performed by the feature pyramid network FPN. The specific operation is: the low-resolution feature map in the multi-layer feature map is upsampled to a high resolution by bilinear interpolation to obtain an upsampled feature map F up , the upsampled feature map Fup And the corresponding high-resolution feature map F high Add and connect horizontally to get the fused feature map.

[0041] As a preferred technical solution, the path aggregation network PAN in step 2.2 further integrates the operation of the fused feature map as follows: using the maximum pooling layer to downsample the high-resolution feature map in the fused feature map to maintain the same resolution as other feature maps, and by performing lateral connections between feature maps of different scales, low-level details and high-level semantic features are fully combined.

[0042] A model of an infrared target detection method based on convolutional attention, including: a backbone network, a neck network, and a detection layer;

[0043] The backbone network includes an input layer, a convolution layer, a normalization layer, a convolution block attention module, and an output layer connected in sequence; the input layer is used to receive an infrared image as input; the convolution layer is used to extract local features of the infrared image; the normalization layer is used to stabilize the learning process and reduce internal covariate shift; the convolution block attention module is used to enhance the network's attention to important features; the output layer is used to output feature maps processed by the backbone network to the neck network;

[0044] The neck network includes a feature pyramid network and a path aggregation network. The feature pyramid network is used to fuse feature maps of different scales to facilitate the detection of objects of different sizes. The path aggregation network is used to further integrate the feature maps output by the feature pyramid network to enhance the expressiveness of the features.

[0045] The detection layer is used to convert the fused feature map output by the neck network into a final target detection result.

[0046] The beneficial effects of the present invention are as follows:

[0047] By introducing the attention mechanism, the present invention can more effectively focus on small targets in the image, thereby improving detection accuracy. By utilizing multi-scale feature fusion technology, it can simultaneously capture the detailed information and contextual information of the image, enhance the expressiveness of the features, and help to understand the image content more carefully.

[0048] The lightweight network design and optimized algorithm of the present invention reduce the computational burden of the model, making the training and reasoning processes more efficient. By batch normalization and the introduction of loss function optimization, good performance can be maintained in different data sets and environments, thereby enhancing robustness and generalization capabilities.

[0049] The optimized loss function and detection layer design of the present invention help to reduce the error between the predicted bounding box and the true bounding box, and the positioning accuracy is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the model structure of the infrared target detection method based on convolutional attention of the present invention.

[0051] Figure 2 It is a flow chart of the infrared target detection method based on convolutional attention of the present invention.

[0052] Figure 3 It is a schematic diagram of the structure of the backbone network of the present invention.

[0053] Figure 4 It is a structural schematic diagram of the neck network mold of the present invention. DETAILED DESCRIPTION

[0054] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

[0055] exist Figure 1 In the embodiment, the model of the infrared target detection method based on convolutional attention includes: a backbone network, a neck network, and a detection layer.

[0056] The backbone network includes an input layer, a convolutional layer, a normalization layer, a convolutional block attention module, and an output layer connected in sequence; the input layer is used to receive the infrared image as input; the convolutional layer is used to extract the local features of the infrared image; the normalization layer is used to stabilize the learning process and reduce the internal covariate shift; the convolutional block attention module is used to enhance the network's attention to important features; the output layer is used to output the feature map processed by the backbone network to the neck network.

[0057] The neck network includes a feature pyramid network and a path aggregation network. The feature pyramid network is used to fuse feature maps of different scales to facilitate the detection of targets of different sizes; the path aggregation network is used to further integrate the feature maps output by the feature pyramid network to enhance the expressiveness of the features.

[0058] The detection layer is used to convert the fused feature map output by the neck network into the final target detection result.

[0059] exist Figure 2 In this example, the infrared target detection method based on convolutional attention includes the following steps:

[0060] Step 1. Generate feature maps of multiple scales through repeated feature extraction and attention adjustment of the backbone network, such as Figure 3 ;

[0061] Step 1.1. Perform a two-dimensional convolution operation on the input infrared image according to the following formula to obtain a primary feature map;

[0062]

[0063] Where padding[·] is the padding operation, dilation[·] is the spacing of the convolution kernel points, kernel_size[·] is the size of the convolution kernel, stride[·] is the convolution step size, and H in is the pixel height of the input infrared image, W in is the pixel width of the input infrared image, H out is the pixel height of the output primary feature map, W out is the pixel width of the output primary feature map;

[0064] Step 1.2. The primary feature map is normalized according to the following formula, and then nonlinearly transformed through the rectified linear unit ReLU and output to the CBAM attention module;

[0065]

[0066] In the formula, BatchNorm2d(x) n,c,h,w is a two-dimensional batch normalization operation on the four-dimensional tensor x, n,c,h,w is the element of the input tensor x at the nth sample, cth channel, hth row, and wth column, H is the pixel height of the primary feature map, W is the pixel width of the primary feature map, and γ c is the normalized scaling factor, β c is the normalized offset, ε is a constant not equal to 0;

[0067] Step 1.3. The CBAM attention module adjusts the input feature map to generate an adjusted feature map. The specific operations are as follows:

[0068] 1) Perform global average pooling and global maximum pooling on the input feature map according to the following formula;

[0069]

[0070] In the formula, F avg is the global average pooling result, which represents the average value of each channel in the input feature map, F max is the global maximum pooling result, which represents the maximum value of each channel in the input feature map, F 1 is the input feature map;

[0071] 2) Use the shared fully connected layer to obtain the channel weight according to the following formula and use the sigmoid activation function to obtain the channel attention weight M c ;

[0072] M c =σ(W1 δ(W 0 +F avg )+W 1 δ(W 0 F max ))

[0073] Where σ(·) is the Sigmoid activation function, δ is the ReLU activation function, and W 0 ,W 1 are all weight matrices of the fully connected layers;

[0074] 3) Perform average pooling and maximum pooling operations on the input feature map in the channel dimension according to the following formula, and stack the results to obtain the pooled feature map F pool , F pool =[AvgPool(F 1 );MaxPool(F 1 )];

[0075] 4) Feature map F after pooling pool Generate a spatial attention map M through a 7×7 convolution operation s ,

[0076] M s =δ(f (7x7) (F pool ))

[0077] In the formula, f 7×7 It is a 7×7 convolution operation used to generate a spatial attention map;

[0078] 5) Set the channel attention weight M c and the spatial attention map M s Applied to the input feature map F as follows 1 , get the adjusted feature map F out , F out =F 1 ×M c ×M s ;

[0079] Step 1.4. Repeat steps 1.1 to 1.3 four times to generate four adjusted feature maps to form a multi-layer feature map;

[0080] Step 2: Perform multi-scale feature fusion on the multi-layer feature map through the neck network module to generate a final feature map suitable for detection, such as Figure 4 ;

[0081] Step 2.1. Perform multi-scale feature fusion on the multi-layer feature map through the feature pyramid network FPN, and output the fused feature map. The specific operation is:

[0082] The low-resolution feature map in the multi-layer feature map is upsampled to high resolution through bilinear interpolation to obtain the upsampled feature map F up , the upsampled feature map F up And the corresponding high-resolution feature map F high Add and connect horizontally to obtain the fused feature map;

[0083] Step 2.2. Further integrate the fused feature graph through the path aggregation network PAN and output it. The specific operation is:

[0084] Use the maximum pooling layer to downsample the high-resolution feature map in the fusion feature map to maintain the same resolution as other feature maps. By making lateral connections between feature maps of different scales, low-level details and high-level semantic features are fully combined.

[0085] Step 2.3. Perform a convolution operation on the output of step 2.2 to generate a final feature map suitable for detection;

[0086] Step 3: Convert the final feature map into the final target detection result through the detection layer

[0087] Step 3.1. Use the optimized YOLO detection head structure to process the final feature map to extract more refined features for target detection and obtain the feature map F for detection. det ;

[0088] Step 3.2. Apply the Anchor mechanism, select the Anchor scale suitable for small targets based on the prior information of the target, and adjust the bounding box position;

[0089] Step 3.3. Calculate the predicted class probability and bounding box offset;

[0090] P class =σ(Conv(F det ,W class ))

[0091] Δx, Δy, Δw, Δh=Conv(F det ,W bbox )

[0092] Where P class is the predicted target category probability, σ is the sigmoid activation function used to calculate the category probability, W class is the weight of the category prediction convolutional layer, Δx, Δy are the offsets of the center coordinates of the predicted bounding box relative to the anchor box, Δw, Δh are the offsets of the width and height of the predicted bounding box relative to the anchor box, respectively, and W bbox are the weights of the convolutional layer for bounding box prediction;

[0093] Step 3.4. Introduce kernel weighted distance loss L NWD And the distance between the predicted box and the real box is obtained according to the following formula to reduce the positioning error of small targets;

[0094]

[0095] In the formula, is a weight function that calculates the distance or difference between the predicted box and the true box. is the predicted box, P is the real box, and Z is the normalization factor;

[0096] Step 3.5. Output the final detection results, including the category and location coordinates of the target.

[0097] experiment

[0098] In order to verify the beneficial effects of the present invention, the inventors used the infrared target detection method based on convolutional attention of the present invention to compare with the classic infrared small target detection models YOLOv3 and YOLOv5, as follows:

[0099] The FLIR dataset was used, and 9019 infrared images were selected as the data for this experiment, of which 7659 were taken as the training set and 1360 were taken as the test set.

[0100] The epoch times of the YOLOv3 and YOLOv5 models are both 200. After the training is completed, the best results in the test are taken to calculate the mAP, precision and recall, as shown in Table 1.

[0101] The infrared target detection method based on convolutional attention is carried out in a Linux environment, based on the Pytorch framework (CUDA 11.2, Torchvision>=0.8.2, Python 3.8), using NVIDIA RTX 3090GPU, and training for a total of 200 epochs with a batch size of 16. The Adam optimizer is used during training, and the β 1 =0.9,β 2 = 0.999, and the learning rate is set to 0.001. After training, the best result in the test is taken to calculate mAP, precision, and recall, as shown in Table 1.

[0102] Table 1 Test results of different models on the FLIR dataset

[0103]

[0104] As can be seen from Table 1, in the test results on the FLIR dataset, the mAP, precision and recall of the YOLOv3 model are 54.4, 63.5 and 51.6 respectively, showing a relatively balanced detection performance. In contrast, the mAP of YOLOv5 has slightly improved to 55, but the precision and recall are 62.6 and 49.5 respectively, and the recall rate has decreased, resulting in an increase in missed detections. The present invention has significantly improved in all indicators, with mAP reaching 58.5, precision reaching 70.7, and recall reaching 52, showing a strong target detection capability. Especially in terms of precision, this method is significantly better than other benchmark models, indicating that this method can more accurately identify targets in infrared images and has excellent comprehensive performance.

Claims

1. A convolutional attention-based infrared target detection method, characterized in that: The following steps are involved: Step 1. Generate feature maps of multiple scales through repeated feature extraction and attention adjustment of the backbone network Step 1.

1. Perform a two-dimensional convolution operation on the input infrared image to obtain a primary feature map; Step 1.

2. The primary feature map is normalized, then nonlinearly transformed through a rectified linear unit ReLU and output to the CBAM attention module; Step 1.

3. The CBAM attention module adjusts the input feature map and generates an adjusted feature map; Step 1.

4. Repeat steps 1.1 to 1.3 four times to generate four adjusted feature maps to form a multi-layer feature map; Step 2: Perform multi-scale feature fusion on the multi-layer feature map through the neck network module to generate a final feature map suitable for detection Step 2.

1. Perform multi-scale feature fusion on the multi-layer feature map through the feature pyramid network FPN, and output the fused feature map; Step 2.

2. Further integrate the fused feature graph through a path aggregation network PAN and output it; Step 2.

3. Perform a convolution operation on the output of step 2.2 to generate a final feature map suitable for detection; Step 3: Convert the final feature map into the final target detection result through the detection layer Step 3.

1. Process the final feature map using the optimized YOLO detection head structure to extract more refined features for target detection; Step 3.

2. Apply the Anchor mechanism, select the Anchor scale suitable for small objects, and adjust the bounding box position; Step 3.

3. Calculate the predicted class probability and bounding box offset; Step 3.

4. Introduce kernel weighted distance loss L NWD And the distance between the predicted box and the real box is obtained according to the following formula to reduce the positioning error of small targets; In the formula, is a weight function that calculates the distance or difference between the predicted box and the true box. is the predicted box, P is the real box, and Z is the normalization factor; Step 3.

5. Output the final detection results, including the category and location coordinates of the target.

2. The infrared target detection method based on convolutional attention according to claim 1 is characterized in that: The convolution operation described in step 1.1 is performed as follows; Where padding[·] is the padding operation, dilation[·] is the spacing of the convolution kernel points, kernel_size[·] is the size of the convolution kernel, stride[·] is the convolution step size, and H in is the pixel height of the input infrared image, W in is the pixel width of the input infrared image, H out is the pixel height of the output primary feature map, W out is the pixel width of the output primary feature map.

3. The infrared target detection method based on convolutional attention according to claim 1 is characterized in that: The normalization process described in step 1.2 is: In the formula, BatchNorm2d(x) n,c,h,w is a two-dimensional batch normalization operation on the four-dimensional tensor x, n,c,h,w is the element of the input tensor x at the nth sample, cth channel, hth row, and wth column, H is the pixel height of the primary feature map, W is the pixel width of the primary feature map, and γ c is the normalized scaling factor, β c is the normalized offset, and ε is a constant not equal to 0.

4. The infrared target detection method based on convolutional attention according to claim 1, characterized in that: The method for the CBAM attention module to adjust attention in step 1.

3. is: 1) Perform global average pooling and global maximum pooling on the input feature map according to the following formula; In the formula, F avg is the global average pooling result, which represents the average value of each channel in the input feature map, F max It is the global maximum pooling result, which indicates the maximum value of each channel in the input feature map. F1 is the input feature map. 2) Use the shared fully connected layer to obtain the channel weight according to the following formula and use the sigmoid activation function to obtain the channel attention weight M c ; M c =σ(W1δ(W0+F avg )+W1δ(W0F max )) Where σ(·) is the Sigmoid activation function, δ is the ReLU activation function, and W0 and W1 are the weight matrices of the fully connected layer; 3) Perform average pooling and maximum pooling operations on the input feature map in the channel dimension according to the following formula, and stack the results to obtain the pooled feature map F pool , F pool =[AvgPool(F1);MaxPool(F1)]; 4) Feature map F after pooling pool Generate a spatial attention map M through a 7×7 convolution operation s , M s =δ(f (7x7) (F pool )) In the formula, f 7×7 It is a 7×7 convolution operation used to generate a spatial attention map; 5) Set the channel attention weight M c and the spatial attention map M s According to the following formula applied to the input feature map F1, the adjusted feature map F is obtained out , F out =F1×M c ×M s .

5. The infrared target detection method based on convolutional attention according to claim 1, characterized in that: In step 2.1, the multi-scale feature fusion of the multi-layer feature map is performed by the feature pyramid network FPN. The specific operation is: the low-resolution feature map in the multi-layer feature map is upsampled to a high resolution by bilinear interpolation to obtain the upsampled feature map F up , the upsampled feature map F up And the corresponding high-resolution feature map F high Add and connect horizontally to get the fused feature map.

6. The infrared target detection method based on convolutional attention according to claim 1, characterized in that: The path aggregation network PAN described in step 2.2 further integrates the operation of the fused feature map as follows: using the maximum pooling layer to downsample the high-resolution feature map in the fused feature map to maintain the same resolution as other feature maps, and by performing lateral connections between feature maps of different scales, low-level details and high-level semantic features are fully combined.

7. The model of the infrared target detection method based on convolutional attention according to claim 1 is characterized in that: include: Backbone network, neck network, detection layer; The backbone network includes an input layer, a convolutional layer, a normalization layer, a convolutional block attention module, and an output layer connected in sequence; The input layer is used to receive the infrared image as input; the convolution layer is used to extract the local features of the infrared image; the normalization layer is used to stabilize the learning process and reduce the internal covariate shift; the convolution block attention module is used to enhance the network's attention to important features; the output layer is used to output the feature map processed by the backbone network to the neck network; The neck network includes a feature pyramid network and a path aggregation network. The feature pyramid network is used to fuse feature maps of different scales to facilitate the detection of objects of different sizes. The path aggregation network is used to further integrate the feature maps output by the feature pyramid network to enhance the expressiveness of the features. The detection layer is used to convert the fused feature map output by the neck network into a final target detection result.

Images