Infrared small target detection method based on multi-scale feature fusion

Through the attention mechanism of multi-scale feature fusion and injection module, combined with ELAN and spatial pyramid pooling improvement layer, the problems of low feature information transmission efficiency and information loss in infrared small object detection are solved, and the efficiency of infrared small object detection is achieved. Improved accuracy.

CN118918308APending Publication Date: 2024-11-08西安中科立德红外科技有限公司

Patent Information

Application Number
CN202410894900.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The detection of infrared small object in the image has a small proportion, missing texture details and a large amount of noise, which leads to the problems of low efficiency and loss of information in the transmission and computing speed of feature information, making it difficult to effectively detect small infrared objects.

Method used

Using a multi-scale feature fusion method, the feature information at different levels is collected through the path aggregation network framework, and the injection module is used to inject global information and local features through the attention mechanism, combining the ELAN module and the spatial pyramid pooling improvement layer to enhance feature extraction and expression ability to avoid cross-layer information loss, and adopt anchor-free method for target position regression.

Benefits of technology

It significantly improves the accuracy and robustness of infrared small object detection, reduces the missed detection and false alarm rate of the model, improves detection performance and computing efficiency, adapts to targets of different scales and resolutions, and enhances the generalization ability of the model. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918308A_ABST
    Figure CN118918308A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of infrared target detection, and particularly relates to an infrared small target detection method based on multi-scale feature fusion, and the method comprises the steps: inputting an infrared image to a backbone network, and carrying out the deep feature extraction; collecting feature information of different levels to a collection module by using a path aggregation network framework, and unifying the scale of a feature map by using average pooling and bilinear interpolation operation; the features of all levels are fused through a fusion module to generate global information, feature weights are adjusted in a self-adaptive mode through an SE module, important features are enhanced, and secondary features are restrained; global information and local features are combined through an injection module, so that the detection capability of a small target is enhanced; and an anchor-free method based on a central point is adopted to regress the original image and output target position information. According to the method, spatial and semantic feature information of each level is fully reserved, cross-layer information loss is avoided, and the accuracy and robustness of infrared small target detection are remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of infrared target detection technology, and specifically relates to an infrared small target detection method based on multi-scale feature fusion. Background Art

[0002] The task of infrared target detection is to quickly and accurately locate the target in the image. However, the scene changes in infrared images are large and the observation distance is usually far, resulting in the target occupying a small proportion in the image, low contrast, and difficulty in distinguishing from the surrounding noise. These reasons make infrared small target detection challenging in the field of target detection.

[0003] Existing infrared small target detection methods are divided into traditional infrared target detection methods and infrared target detection methods based on deep learning. The target detection method based on deep learning uses training to extract the features of each layer of image data to identify the position and size of the target. The method based on deep learning has become the mainstream method due to its excellent performance. In recent years, with the development of artificial intelligence technology, research related to deep learning has developed rapidly. At present, many mature algorithm models have been applied to the field of image processing. Advanced typical models such as YOLO, fast regional convolutional neural network (FAST-R-CNN), and faster regional convolutional neural network (FASTER-R-CNN) have good performance in the field of target detection. However, these algorithms are mainly aimed at general visible light data sets, and further improvement and optimization are needed to face the problem of infrared small target detection. The detection performance of the network is constantly improving. When the model faces infrared small targets, the complex network design and multiple downsampling processes easily cause the feature information of infrared small targets to be lost, resulting in detection failure.

[0004] Infrared small target detection is challenging because the target occupies a small proportion in the image, texture details are missing, and there is a lot of noise. Feature fusion is another strategy to solve the problem of infrared small target detection. Most of the current feature fusion methods work based on the structure of pyramid networks. Such methods have problems such as low computing speed and information loss due to too frequent path connections and indirect connection methods. Although advanced feature fusion methods have improved this problem, the cross-layer connection method makes the feature information transmission efficiency inefficient and the feature information is easily lost. In view of the above problems, it is also necessary to design targeted strategies for the problem of easy loss of infrared small target features to improve the accuracy and robustness of the algorithm. Summary of the invention

[0005] To solve the above problems, the present invention provides an infrared small target detection method based on multi-scale feature fusion, the method comprising the following steps:

[0006] Step 1: Input infrared image to the deep feature extraction of backbone network;

[0007] Step 2: Using the path aggregation network framework, feature information at different levels is collected into the collection module;

[0008] Step 3: The features at each level are fused to generate global information through the fusion module;

[0009] Step 4: Use the injection modules distributed at each layer to inject the global information and the local features representing the layer and the adjacent layers into the layer through the attention mechanism;

[0010] Step 5: Collect feature information at different high levels and perform the same operation to generate the output of the multi-scale feature fusion module;

[0011] Step 6: Regress the original image using an anchor-free method and output an image with target location information.

[0012] In the algorithm proposed in the present invention, after the image is input into the network and features are extracted by the backbone network, the feature information of different levels is collected into the collection module in the framework of the path aggregation network, and then the features of each level are fused through the fusion module to generate global information. The global information and the local features representing the level and its adjacent layers are injected into the layer through the attention mechanism using the injection modules distributed at each level, thereby improving the ability to detect small targets. After that, feature information of different high levels is collected to perform the same operation, thereby improving the model's ability to extract high-level features. The present invention fully retains the spatial and semantic feature information of each level, avoids cross-layer information loss, and makes the fusion of feature information between different levels more complete, significantly improving the detection performance of the model.

[0013] Furthermore, in step 1, the backbone network includes a convolution layer, a batch normalization layer, and a Silu activation function layer (i.e., a CBL layer), and feature enhancement is performed through an ELAN module and a spatial pyramid pooling improvement layer (Sppcspc). Through the combination of convolution layers, batch normalization layers, and Silu activation function layers, the features of the image can be extracted and represented at a deeper level. The Silu activation function layer can provide stronger nonlinear representation capabilities and enhance the network's ability to capture complex features. The ELAN module is an efficient network structure that enables the network to learn more features and has stronger robustness by controlling the shortest and longest gradient paths. Specifically, the module is divided into two branches, one branch changes the number of channels through a convolution layer, and the other branch extracts features through multiple convolution layers. Through the cascading and splicing between different layers, the feature expression ability of the network is enhanced. The spatial pyramid pooling improvement layer (Sppcspc) increases the receptive field through the maximum pooling layers of different scales, so that the model can adapt to images of different resolutions. The Sppcspc module is divided into two branches. Through convolutional layers and maximum pooling layers of different sizes, it can better distinguish targets with large size differences, while reducing the amount of calculation and improving speed and accuracy.

[0014] Furthermore, if Figure 1As shown, the multi-scale feature fusion module includes a low feature collection module, a low feature fusion module, a high feature collection module, a high feature fusion module and an injection module. The low feature collection module performs average pooling and bilinear interpolation operations on the input feature map. In order to solve the problem of information loss and low computational efficiency in the process of cross-layer feature information transmission in the traditional model, a multi-scale feature fusion module is proposed. After the image is input to the network and the features are extracted by the backbone network, the feature information of different levels is collected to the collection module in the framework of the path aggregation network, and then the features of each level are fused to generate global information through the fusion module. The injection modules distributed at each level are used to inject the global information and the local features representing the level and its adjacent layers into the layer through the attention mechanism, thereby improving the ability to detect small targets. After that, the feature information of different high levels is collected for fusion and injected into each level using the injection module. The multi-scale feature fusion module fully retains the spatial and semantic feature information of each level, avoids cross-layer information loss, makes the feature information fusion between different levels more sufficient, and significantly improves the detection performance of the model. Through average pooling and bilinear interpolation operations, feature maps of different scales are adjusted to the same size, which is convenient for subsequent feature fusion processing. Average pooling can retain the important information of each feature map and reduce information loss, while bilinear interpolation helps to maintain the continuity and spatial distribution of features. The unified feature map has the same size, which facilitates the low feature fusion module and the high feature fusion module to effectively fuse features and make full use of multi-level information. In addition, through the fusion of multi-scale features, the model can better adapt to targets of different sizes and resolutions, and improve the accuracy and robustness of detection. In short, this design makes the model more efficient in processing features of different scales, significantly improving the performance of infrared small target detection.

[0015] Furthermore, the low-feature fusion module performs feature fusion through convolutional layers, batch normalization layers and SE modules, and adaptively extracts the feature weights of each channel. The convolutional layer extracts important features in the image through convolution operations, and the batch normalization layer can stabilize the training process, accelerate convergence, and improve the robustness of the model; the SE module adaptively learns the weights of each channel through compression and excitation operations, enhances the model's use of the relationship between feature channels, thereby highlighting key features and suppressing irrelevant features; through the joint action of the above levels, the low-feature fusion module can more efficiently fuse feature information from different scales and levels, and improve the richness and accuracy of the overall feature expression; by adaptively adjusting the weights of each channel feature, it can effectively improve the detection capability of infrared small targets, especially in complex backgrounds and with more noise, significantly improving the detection performance and robustness of the model. In short, this design makes full use of the spatial and semantic feature information of each level, avoids cross-layer information loss, makes feature information fusion more sufficient, and improves the detection effect of the model.

[0016] Furthermore, the input of the injection module is global features and local features. The global features are processed separately through two branches, multiplied with local features and then spliced, and finally the feature map is output through the SE module. By multiplying and splicing global features and local features, information at different levels can be effectively combined. Global features provide overall context information, while local features contain detailed information. This combination enables the model to understand the input image more comprehensively and improve detection accuracy. The SE module can adaptively assign weights to each channel through the Squeeze-and-Excitation mechanism, thereby enhancing the expression of important features and suppressing irrelevant or minor features. This adaptive feature enhancement mechanism can significantly improve the performance of the model. The injection module injects global information into local features, and through the attention mechanism, the model can keep its attention on small targets at different levels. This can improve the model's detection ability for infrared small targets, especially when the target is small and the background is complex. The traditional cross-layer connection method easily leads to information loss during the transmission process, while the injection module can fully retain and utilize the feature information of each level through multi-level fusion and injection, avoiding cross-layer information loss and making the transmission of feature information more efficient and complete. Through this multi-level feature fusion and attention mechanism, the model can better cope with detection tasks in various complex environments, improving the robustness and generalization ability of the model.

[0017] Furthermore, the high-feature collection module has the same structure as the low-feature collection module, and produces a high-level global feature map through splicing operations, which is then fused through the high-feature fusion module. Through the splicing operation of the high-feature collection module, feature maps of different scales are adjusted to the same size to ensure the consistency of the high-level feature map, which is conducive to subsequent processing; the high-feature fusion module performs feature fusion through convolutional layers, batch normalization layers, and SE modules, adaptively extracts the feature weights of each channel, and enhances the feature expression capability; the high-feature collection module effectively collects and retains high-level spatial and semantic feature information, avoids cross-layer information loss, and makes feature fusion more sufficient; by fusing high-dimensional feature information from different levels, the model can more comprehensively understand and characterize the input image, thereby improving the accuracy and robustness of infrared small target detection.

[0018] Furthermore, in step 6, an anchor-free method based on the center point is used to determine the target position by predicting the offset distance between the four edges and the center point of the grid on each grid in the feature map. This method does not rely on the clustering algorithm to obtain the target anchor box size, but directly learns the anchor box shape of the target during training, so that the network has a better expression of the target shape and size. When encountering extreme samples, the prediction method based on the anchor box makes it difficult for the network to learn these samples, while the anchor-free method enhances the versatility and robustness of the network, especially when facing targets with large multi-scale changes. This method can detect occluded objects more flexibly because it does not need to rely on predefined anchor boxes and can more accurately predict the boundaries of the target. In addition, such a design avoids the process of performing a large amount of calculation and matching on the anchor box, simplifies the structure and calculation of the model, and improves the detection speed and efficiency.

[0019] Furthermore, it also includes regressing and updating the network parameters through the loss function.

[0020] Furthermore, the loss functions include binary cross entropy loss, full intersection-over-union loss, and distributional focal loss.

[0021] Furthermore, the final loss score is obtained by weighted calculation of binary cross entropy loss, complete intersection-over-union loss, and distribution focal loss, which is used to regress and update network parameters. The combination of multiple loss functions such as binary cross entropy loss, complete intersection-over-union loss, and distribution focal loss can more comprehensively measure the difference between the predicted results and the true labels, thereby improving the detection accuracy of the model. The distribution focal loss function optimizes the probability of bounding box distribution, enabling the network to locate the target boundary more accurately, especially when dealing with complex target shapes and multi-scale targets. The final loss score obtained by weighted calculation balances different types of errors, making the network more robust during training, effectively reducing overfitting, and improving the generalization ability of the model in different scenarios. In addition, the joint application of multiple loss functions accelerates the convergence of network parameters and improves training efficiency by comprehensively optimizing category loss and position loss.

[0022] Beneficial effects of the present invention:

[0023] (1) Compared with the traditional infrared small target detection algorithm, this method does not rely on prior information such as target saliency and background consistency. The backbone network of the present invention has a strong feature extraction capability and has strong robustness and generalization in complex environments.

[0024] (2) Compared with the general small target detection method, this method uses a multi-scale feature fusion module to collect global information and uses an injection module to inject the global information and local features representing the layer and its adjacent layers into the layer through an attention mechanism, which is beneficial to improving the accuracy of infrared small target detection. The multi-scale feature fusion module adopted by the present invention has lower information transmission loss than the traditional pyramid network, which makes the network converge faster when facing small targets, and the detection of small targets is more stable, reducing the missed detection and false alarm rate of the model.

[0025] (3) Compared with specific infrared small target detection methods, the anchor-free method adopted by this method does not rely on clustering algorithms to obtain the target anchor box size, but directly learns the target anchor box shape during training. When encountering extreme samples, the prediction method based on anchor boxes makes it difficult for the network to learn these samples. The method adopted by the present invention does not rely on prior knowledge of the data set, so that the network has better expression ability for the target shape and size, and is more versatile and robust, especially when facing targets with large multi-scale changes. At the same time, the detection of occluded objects can also be more flexible. Compared with the dense nesting method, the multi-scale feature fusion module adopted by the present invention reduces redundant information, has fewer network parameters, and accelerates calculation efficiency, ensuring accuracy while improving real-time performance.

[0026] Based on the above beneficial effects, the present invention has good application prospects in the field of infrared target detection technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a network model diagram of infrared small target detection based on multi-scale feature fusion in the present invention.

[0028] Figure 2 It is the operation flow chart of the low-level feature collection and fusion module.

[0029] Figure 3 It is a model diagram of the injection module of the present invention.

[0030] Figure 4 It is an operation flow chart of the high-level feature collection and fusion module of the present invention. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and examples.

[0032] The present invention provides an infrared small target detection method based on multi-scale feature fusion, which includes 6 main steps. The implementation steps of the present invention are described in detail below in conjunction with the accompanying drawings:

[0033] Step 1: Input infrared image to the deep feature extraction of backbone network. This step includes 6 sub-steps.

[0034] Step 1.1: Input an infrared image of size 640×640, and after a CBS layer (convolution layer + batch normalization layer + Silu activation function layer), the number of image channels is increased to 32 to obtain the feature map y1. The dimension increase is to enable the model to learn richer features.

[0035] Step 1.2: Feature map y1 passes through a CBS layer and an ELAN (Efficient Aggregation Network Layer) module to obtain feature map y2. The ELAN module is an efficient network structure that controls the shortest and longest gradient paths, allowing the network to learn more features and have stronger robustness. The specific implementation is that the module is divided into two branches, one branch changes the number of channels through the CBS layer, and the other branch changes the number of channels through a CBS layer, and then extracts features through four CBS layers. The cascade between different layers is spliced ​​with the feature map of the first branch and passed through a layer of CBL to obtain feature map y2.

[0036] Step 1.3: Feature map y2 is downsampled through the maximum pooling layer and then passed through the ELAN module to obtain feature map y3.

[0037] Step 1.4: Feature map y3 is downsampled through the maximum pooling layer and then passed through a layer of ELAN module to obtain feature map y4.

[0038] Step 1.5: After this, the feature map y4 is downsampled through the maximum pooling layer and then passes through a layer of ELAN module to obtain the feature map y5.

[0039] Step 1.6: Feature map y5 passes through Sppcspc (spatial pyramid pooling improvement layer) to obtain feature map y6. The function of the Sppcspc module is to increase the receptive field so that the model can adapt to images of different resolutions. The module is divided into two branches. The feature map in the first branch passes through three layers of CBL and enters four branches with different scales of maximum pooling layers. The maximum pooling layers of different sizes represent that the module can have four receptive fields. Then the feature maps of the four branches are spliced ​​and then spliced ​​with the feature maps of the second branch through two layers of CBL. The second branch has only one layer of CBL to adjust the size of the feature map. The spliced ​​feature map passes through a layer of CBL to obtain feature map y7. Because the Sppcspc module has four different maximum pooling layers, it can increase the receptive field in disguise and better distinguish targets with large size differences. At the same time, only one branch performs pyramid pooling operations, which reduces the amount of calculation by half, making the speed faster and the accuracy higher.

[0040] Step 2: Use the path aggregation network framework to collect feature information at different levels into the collection module.

[0041] The feature maps y2, y3, y4, and y6 are input into the low feature collection module for splicing. The size of the feature map y3 is used as the standard, and the remaining feature maps are average pooled and bilinear interpolated to keep the scale of each feature map consistent. Average pooling can fairly retain information at all levels.

[0042] Step 3: Fusion modules are used to fuse features at each level to generate global information. This step includes the following sub-steps:

[0043] Step 3.1: Input the collected feature maps into the feature fusion module to fuse low-level global features. The specific operation is that the feature map first passes through a layer of CBL and then uses the channel attention mechanism SE (Squeeze-and-Excitation) module for adaptive feature extraction. The SE module can adaptively learn the weights of each channel through compression and excitation operations, and enhance the model to utilize the relationship between feature channels. After that, the feature map size is adjusted through the CBL layer and the split operation is used to divide the feature map into feature maps suitable for injecting feature map y3 and feature map y4. The collection module and the fusion module are as follows Figure 2 The calculation process is shown in the following formula:

[0044] F align =Low_gather([y2,y3,y4,y6]) (1)

[0045] F fuse =SE(F align ) (2)

[0046] F P3 ,F P4 =Split(F fuse ) (3)

[0047] where F align is the low-level global feature, F fuse is the low-level global feature fused by the SE module. P3 With F P4 They are feature maps suitable for injecting feature map y3 and feature map y4 respectively.

[0048] Step 3.2: After the feature map y3 passes through a CBL layer, local feature fusion is performed with the feature maps y2 and y3. The specific operation is to perform average pooling and bilinear interpolation operations on different feature maps to unify the size of the feature maps for splicing, and then adjust the number of channels through the CBL layer to output a local feature map suitable for the injection module input.

[0049] Step 4: Use the injection modules distributed at each layer to inject the global information and the local features representing the layer and the adjacent layers into the layer through the attention mechanism.

[0050] The local feature map and the low-level global feature map are used as input to the injection module. The output feature map is passed through the ELAN module to obtain the feature map P3. The specific operation of the injection module is as follows: Figure 3 As shown: The global feature has two branches. One branch passes through the CBL layer and then adjusts the feature map size through average pooling or bilinear interpolation and multiplies it with the local features that have passed the CBL layer, and then splices it with the global features that have passed the CBL layer and average pooling or bilinear interpolation operations. After that, the SE module is used to output the feature map P3. The width and height of the feature map P3 are the same as those of the feature map y3 (the feature map P3 does not participate in the fusion of high-level features and the injection module is directly output to the detection head part, referred to as the feature map N3). The injection module can effectively inject local and global features into the current level and efficiently utilize information. The calculation process is shown in the following formula:

[0051] F global1 = resize(sigmod(Conv(F Pi ))) (4)

[0052] F global2 = resize(Conv(F Pi )) (5)

[0053] F fuse =Conv(F part )*F globol1 +F global2 (6)

[0054] P i =SE(F fuse ) (7)

[0055] Among them, sigmod is the activation function, resize is the size setting operation, and F global1 is the global feature 1, F global2 is the global feature 2, F part It is a local feature.

[0056] Step 5: Collect different high-level feature information and perform the same operation to generate the output of the multi-scale feature fusion module. This step includes the following sub-steps:

[0057] Step 5.1: Feature map y6 passes through the CBL layer and the ELAN module to perform local feature fusion with feature map y6 and feature map y6, and then enters the injection module as input together with the low-level global feature to output a feature map with the same width and height as feature map y4. After passing through the ELAN module, feature map P4 is obtained.

[0058] Step 5.2: Feature map y6 passes through the CBL layer to obtain feature map P6. At this time, the high-level feature collection module is used to collect feature maps P4, P3, and P6. The high-level feature collection module has the same structure as the low-level feature collection module, and the input is three feature maps. The fusion module is the same as the low-level feature fusion module, such as Figure 4 As shown in Figure 2, the final output is a high-level global feature map suitable for injection into feature map P6 and feature map P4.

[0059] Step 5.3: Feature map P4 is fused with feature map P3 for local features. The operation is similar to the low-level local feature fusion. Feature map P3 is resized by average pooling and concatenated with feature map P4. Then the number of channels is adjusted by convolution to output high-level local features. The high-level global features and high-level local features are input into the injection module to output the feature map, and then the feature map N4 is output through the ELAN module.

[0060] Step 5.4: After the local features of feature map P6 are fused with the high-level global features, they are input into the injection module and the feature map N6 is obtained through the ELAN module.

[0061] Step 6: Regress the original image in an anchor-free way and output an image with target position information. This step includes the detection head related step and the prediction head related step.

[0062] The present invention adopts model parameterization technology, which can combine multiple computing modules into one in the inference stage to improve the efficiency and performance of the model. The detection head related steps include the following 6 steps:

[0063] Step 6.1.1: Feature map N3 passes through the Rep (re-parameterized convolution) module. During the training process, the Rep module adds a 1*1 convolution from a 3*3 convolution. If the input and output channels are consistent in width and height, a BN (batch normalization) branch is added. When deployed, the branch parameters are re-parameterized to the main branch, and the 3*3 convolution output is taken. Use a multi-branch convolution layer during training, and then re-parameterize the branch parameters to the main branch during inference, thereby reducing the amount of calculation and memory consumption.

[0064] Step 6.1.2: Feature map N4 passes through the Rep module.

[0065] Step 6.1.2: Feature map N6 passes through the Rep module.

[0066] Step 6.1.3: Feature map N3 is passed through the CBL module to obtain feature map R3.

[0067] Step 6.1.4: Feature map N4 is passed through the CBL module to obtain feature map R4.

[0068] Step 6.1.5: Feature map N6 is passed through the CBL module to obtain feature map R6.

[0069] Taking the input image size as the standard, the image size of feature map R3 is 1 / 8 of the input image; feature map R4 is 1 / 16 of the input image; and feature map R6 is 1 / 32 of the input image. Feature maps of different sizes contain feature information at different levels, where feature map R3 contains more position information from the lower layers, while feature map R6 contains more semantic information from the higher layers.

[0070] The prediction head is an anchor-free method that uses center-based methods. This method does not rely on clustering algorithms to obtain the target anchor box size, but directly learns the target anchor box shape during training. When encountering extreme samples, the anchor box-based prediction method makes it difficult for the network to learn these samples. The anchor-free method does not rely on prior knowledge of the data set, so that the network has better expression ability for the target shape and size, and is more versatile and robust, especially when facing targets with large multi-scale changes. At the same time, it can also be more flexible in detecting occluded objects.

[0071] The specific method is to predict the offset distance between the four sides of the box and the center point of the grid on each grid in the obtained feature map. The offset of the four sides is obtained by integrating 16 grids and then multiplying by the downsampling multiple. Taking the input image of 640*640 as an example in the network of the present invention, there are 8400 prediction boxes calculated. The prediction part includes the following 5 steps:

[0072] Step 6.2.1: During training, update the parameters using the loss function for the small target.

[0073] During the training process, after the detection head obtains the prediction results of feature maps of three different scales, the network parameters are updated through loss function regression according to the obtained parameters (whether the feature points contain the target, the category of the target, etc.). The loss functions used are divided into category loss and position loss. The category loss function uses the BEC (Binary Cross Entropy) function, which directly outputs the confidence scores of each category and then finds the maximum value; the position loss uses the CIOU (Complete-IoU) function and the DFL (Distribution Focal Loss) function. First, the DFL is used to calculate the loss of the "bounding box distribution probability" and the "distribution probability" of the label, so as to optimize each edge. Then the "bounding box distribution probability" is restored to the prediction box, and the loss of the prediction box and the "actual box" of the label is calculated through the CIoU loss, so as to optimize the prediction box as a whole. The formulas of the three loss functions are shown as follows:

[0074]

[0075] L DFL (S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ) (10)

[0076] The BEC function p i Indicates the probability that the target is of this class, and N is the number of categories; IoU in CIoU represents the intersection over union function, where α is the weight coefficient and v is used to measure the consistency of the relative proportions of the two rectangular boxes; DFL optimizes the probability of the two positions closest to the label in the form of cross entropy, so that the network can focus on the target position and the distribution of the adjacent area more quickly.

[0077] Step 6.2.2: Finally, the three losses are weighted according to a certain weight ratio to obtain the final loss score. During the detection process, the feature map prediction result is obtained and the prediction box is drawn in the original image.

[0078] Step 6.2.3: Find the prediction box that is greater than the threshold function and loop for different categories.

[0079] Step 6.2.4: Sort the targets of this category according to the confidence scores, and use the NMS (non-maximum suppression) algorithm to filter the redundant detection boxes to obtain the final detection results.

[0080] Step 6.2.5: Output the infrared small target detection result. This network is an end-to-end network, and the final output is:

[0081] output = net(input).

[0082] In summary, the present invention first uses the backbone network to extract target feature information and uses the ELAN module and the Sppcspc module to enhance the robustness of the network, making the network extract features more flexible and effective. The collection and injection modules in the multi-scale feature fusion module are then used to prevent the network from losing information in the process of transmitting feature information at different levels, and to ensure that the network parameters remain within a reasonable range under the premise of more effectively utilizing feature information at different levels. A reparameterization module is introduced in the detection head part, and a multi-branch convolutional layer is used during training. Then, the parameters of the branch are reparameterized to the main branch during reasoning, thereby reducing the amount of calculation and memory consumption. In the prediction part, the Anchor-free method is used to enhance the robustness and generalization of the network in the face of multi-scale changes and occlusion of the target. The loss function uses three loss functions to weight the final loss, and the prediction anchor box confidence score is obtained. Finally, the final output result is obtained through the NMS algorithm.

[0083] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for detecting small infrared targets based on multi-scale feature fusion, characterized in that: The method comprises the following steps: Step 1: Input infrared image to the deep feature extraction of backbone network; Step 2: Using the path aggregation network framework, feature information at different levels is collected into the collection module; Step 3: The features at each level are fused to generate global information through the fusion module; Step 4: Use the injection modules distributed at each layer to inject the global information and the local features representing the layer and the adjacent layers into the layer through the attention mechanism; Step 5: Collect feature information at different high levels and perform the same operation to generate the output of the multi-scale feature fusion module; Step 6: Regress the original image using an anchor-free method and output an image with target location information.

2. The infrared small target detection method based on multi-scale feature fusion as claimed in claim 1, characterized in that: In step 1, the backbone network includes a convolutional layer, a batch normalization layer, and a Silu activation function layer, and feature enhancement is performed through an ELAN module and a spatial pyramid pooling improvement layer.

3. The infrared small target detection method based on multi-scale feature fusion as claimed in claim 1, characterized in that: The multi-scale feature fusion module includes a low feature collection module, a low feature fusion module, a high feature collection module, a high feature fusion module and an injection module. The low feature collection module performs average pooling and bilinear interpolation operations on the input feature map.

4. The infrared small target detection method based on multi-scale feature fusion as claimed in claim 3, characterized in that: The low-feature fusion module performs feature fusion through a convolution layer, a batch normalization layer and an SE module, and adaptively extracts the feature weights of each channel.

5. The infrared small target detection method based on multi-scale feature fusion as claimed in claim 4, characterized in that: The input of the injection module is global features and local features. The global features are processed separately through two branches, multiplied with the local features and then spliced, and finally the feature map is output through the SE module.

6. The infrared small target detection method based on multi-scale feature fusion as claimed in claim 5, characterized in that: The high-feature collection module has the same structure as the low-feature collection module, produces a high-level global feature map through a splicing operation, and is fused through the high-feature fusion module.

7. The infrared small target detection method based on multi-scale feature fusion as claimed in claim 1, characterized in that: In step 6, an anchor-free method based on a center point is used to determine the target position by predicting the offset distances between the four edges and the center point of the grid on each grid in the feature map.

8. The infrared small target detection method based on multi-scale feature fusion as claimed in claim 1, characterized in that: It also includes regression updating of network parameters through loss function.

9. The infrared small target detection method based on multi-scale feature fusion as claimed in claim 8, characterized in that: The loss functions include binary cross entropy loss, complete intersection-over-union loss, and distribution focal loss.

10. The infrared small target detection method based on multi-scale feature fusion according to claim 9, characterized in that: The final loss score is obtained by weighted calculation of binary cross entropy loss, complete intersection-over-union loss, and distribution focus loss, which is used for regression to update network parameters.

Citation Information

Patent Citations

  • Target detection algorithm based on improved YOLOv8s

    CN117456167A

  • Remote sensing image ship small target detection method and system

    CN117789030A

  • Infrared small target detection and classification method based on multi-scale feature fusion

    CN118038152A

Cited By

  • Infrared small target detection method

    CN120070849A

  • Infrared small target detection method

    CN120070849B

  • Aviation infrared small target detection optimization method based on neural network

    CN121883799A