Infrared small target detection method and system

Abstract

The invention discloses an infrared small target detection method and system, and belongs to the technical field of target detection. The method comprises the following steps: obtaining an original image, and carrying out scaling processing and normalization processing on the original image to obtain a preprocessed image F; inputting the preprocessed image F into a hyper-fusion backbone network, and performing multi-level multi-scale feature fusion processing to obtain a hyper-fusion feature map; inputting the hyper-fusion feature map into a convolution attention fusion module, performing local feature enhancement processing and global feature enhancement processing, and respectively obtaining a local feature map RJ1 and a global feature map RQ1; and carrying out splicing fusion on the local feature map RJ1 and the global feature map RQ1, and then carrying out decoding processing to obtain a detection result. According to the invention, the identification capability of the system on the infrared target can be effectively enhanced on the whole, so that the detection precision on the infrared small target is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a small infrared target detection method and system. Background Art

[0002] Infrared small target detection is mainly used to identify and locate tiny objects in infrared images. It has broad application prospects and important value in many fields such as security, maritime rescue, and fire monitoring. However, in most cases, the infrared imaging environment is relatively complex, resulting in low differentiation between the background and the target, and the features of small targets are also relatively weak. Traditional detection methods mainly include filter-based methods and methods based on the human visual system, but the detection accuracy of these methods is usually low. Although the detection method based on convolutional neural networks can realize target detection by extracting the features of infrared images, the U-Net backbone network it uses has insufficient feature fusion and insufficient subsequent feature processing, which also leads to poor model detection accuracy. Summary of the invention

[0003] The main purpose of the present invention is to provide an infrared small target detection method and system, aiming to solve the technical problem of poor infrared small target detection accuracy in the related art.

[0004] To achieve the above object, the present invention provides a method for detecting small infrared targets, the method comprising the following steps:

[0005] S1, obtaining an original image of a small infrared target, performing scaling and normalization processing on the original image of the small infrared target, and obtaining a preprocessed image F;

[0006] S2, input the preprocessed image F into the hyper-fusion backbone network, perform multi-level multi-scale feature fusion processing, and obtain a hyper-fusion feature map; wherein the hyper-fusion backbone network includes multiple fusion processing modules, the fusion processing module includes at least one nonlinear convolution unit, the nonlinear convolution unit includes a first convolution layer, a second convolution layer and a first nonlinear attention layer; the multi-scale feature fusion processing includes splicing fusion processing and nonlinear convolution processing, and the steps of nonlinear convolution processing specifically include:

[0007] A1, the feature map input to the nonlinear convolution unit is first processed by the first convolution layer and then by the second convolution layer to obtain the feature map J1;

[0008] A2, input the feature map J1 into the first nonlinear attention layer for feature weighting processing to obtain the feature map J2;

[0009] A3, after concatenating and fusing the feature map J2 with the feature map input to the nonlinear convolution unit, outputs it through the activation function to complete the nonlinear convolution processing;

[0010] S3, input the super-fused feature map into the convolutional attention fusion module, perform local feature enhancement processing and global feature enhancement processing, and obtain the local feature map RJ1 and the global feature map RQ1 respectively;

[0011] S4, concatenates and fuses the local feature map RJ1 and the global feature map RQ1, and then performs decoding processing to obtain the detection result.

[0012] Optionally, the hyper-converged backbone network includes five fusion processing modules, namely a first fusion processing module, a second fusion processing module, a third fusion processing module, a fourth fusion processing module and a fifth fusion processing module;

[0013] The first fusion processing module includes five nonlinear convolution units with the same structure, namely a first nonlinear convolution unit (0,0), a second nonlinear convolution unit (1,0), a third nonlinear convolution unit (2,0), a fourth nonlinear convolution unit (3,0) and a fifth nonlinear convolution unit (4,0);

[0014] The second fusion processing module includes four nonlinear convolution units with the same structure, namely a first nonlinear convolution unit (0, 1), a second nonlinear convolution unit (1, 1), a third nonlinear convolution unit (2, 1) and a fourth nonlinear convolution unit (3, 1);

[0015] The third fusion processing module includes three nonlinear convolution units with the same structure, namely a first nonlinear convolution unit (0, 2), a second nonlinear convolution unit (1, 2) and a third nonlinear convolution unit (2, 2);

[0016] The fourth fusion processing module includes two nonlinear convolution units with the same structure, namely a first nonlinear convolution unit (0, 3) and a second nonlinear convolution unit (1, 3);

[0017] The fifth fusion processing module includes a nonlinear convolution unit, which is a first nonlinear convolution unit (0,4);

[0018] S2 specifically includes:

[0019] The preprocessed image F is input into the first nonlinear convolution unit (0,0) for processing to obtain a 16-dimensional feature map P00;

[0020] After downsampling the feature map P00 by two times, it is input into the second nonlinear convolution unit (1,0) for processing to obtain a 32-dimensional feature map P10;

[0021] After downsampling the feature map P10 by two times, it is input into the third nonlinear convolution unit (2,0) for processing to obtain a 64-dimensional feature map P20;

[0022] After downsampling the feature map P20 by two times, it is input into the fourth nonlinear convolution unit (3,0) for processing to obtain a 128-dimensional feature map P30;

[0023] After downsampling the feature map P30 by two times, it is input into the fifth nonlinear convolution unit (4,0) for processing to obtain a 256-dimensional feature map P40;

[0024] The feature maps P00, P10, P20, P30 and P40 are concatenated and fused, and input into the first nonlinear convolution unit (0,1) for processing to obtain a 16-dimensional feature map P01;

[0025] After downsampling the feature map P01 by two times, it is concatenated with the feature maps P10, P20, P30 and P40, and then input into the second nonlinear convolution unit (1,1) for processing to obtain a 32-dimensional feature map P11;

[0026] After downsampling the feature map P11 by two times, it is concatenated with the feature maps P00, P20, P30, and P40, and then input into the third nonlinear convolution unit (2,1) for processing to obtain a 64-dimensional feature map P21;

[0027] After downsampling the feature map P21 by two times, it is concatenated with the feature maps P00, P10, P30 and P40, and then input into the fourth nonlinear convolution unit (3,1) for processing to obtain a 128-dimensional feature map P31;

[0028] The feature maps P01, P11, P21, P31 and P00 are concatenated and fused, and input into the first nonlinear convolution unit (0, 2) for processing to obtain a 16-dimensional feature map P02;

[0029] After downsampling the feature map P02 by two times, it is concatenated with the feature maps P11, P21, P31 and P10, and then input into the second nonlinear convolution unit (1, 2) for processing to obtain a 32-dimensional feature map P12;

[0030] After downsampling the feature map P12 by two times, it is concatenated with the feature maps P01, P21, P31 and P20, and then input into the third nonlinear convolution unit (2,2) for processing to obtain a 64-dimensional feature map P22.

[0031] The feature maps P02, P12, P22, P00 and P01 are concatenated and fused, and input into the first nonlinear convolution unit (0,3) for processing to obtain a 16-dimensional feature map P03;

[0032] After downsampling the feature map P03 by two times, it is concatenated and fused with the feature maps P12, P22, P10, and P11, and then input into the second nonlinear convolution unit (1,3) for processing to obtain a 32-dimensional feature map P13;

[0033] The feature maps P03, P13, P00, P01 and P02 are concatenated and fused, and input into the first nonlinear convolution unit (0,4) for processing to obtain a 16-dimensional feature map P04;

[0034] The feature maps P40, P31, P22, P13 and P04 are concatenated and fused to obtain a 16-dimensional super-fused feature map.

[0035] Optionally, in A2, the step of feature weighting processing specifically includes:

[0036] A21, obtains the channel attention weight w1 of the feature map J1 through the nonlinear attention mechanism, and uses the channel attention weight w1 to perform channel weighted processing on the feature map J1 to obtain the feature map J11;

[0037] A22, obtains the spatial attention weight w2 of the feature map J11 through the spatial attention mechanism, uses the spatial attention weight w2 to perform spatial weighted processing on the feature map J11, and obtains the feature map J2.

[0038] Optionally, A21 specifically includes:

[0039] A21-1, uses the ReLU activation function to activate the input feature map J1, enhances its nonlinear characteristics, and obtains the activated feature map;

[0040] A21-2, using the channel attention mechanism, performs maximum pooling and average pooling on the activated feature maps to obtain feature maps i1 and i2 respectively;

[0041] A21-3, fully connect feature map i1 and feature map i2 respectively, map them to a feature space with a size of 1 / 16 of the input dimension, then use the ReLU activation function to activate the fully connected feature maps, and fully connect the activated feature maps again to restore them to the same dimension as the input feature map, and obtain feature maps i11 and i21 respectively;

[0042] A21-4, fuse the feature map i11 and the feature map i21, and then use the Sigmoid logic function to output the channel attention weight w1;

[0043] A21-5, use the channel attention weight w1 to perform channel weighted processing on the feature map J1 to obtain the feature map J11.

[0044] Optionally, A22 specifically includes:

[0045] A22-1, perform average pooling and maximum pooling processing on the feature channels of feature map J11 respectively to obtain two single-channel feature maps;

[0046] A22-2, concatenates and fuses the two single-channel feature maps, performs convolution operation after fusion, and obtains a single-channel spatial attention weight map;

[0047] A22-3, use the Sigmoid logic function to process the single-channel spatial attention weight map to obtain the spatial attention weight w2;

[0048] A22-4, use the spatial attention weight w2 to perform spatial weighted processing on the feature map J11 to obtain the feature map J2.

[0049] Optionally, the convolutional attention fusion module includes a second nonlinear attention layer and a cascaded dilated convolution submodule, the cascaded dilated convolution submodule includes a first average pooling unit, a third convolution layer, a cascaded dilated convolution channel unit, and a fourth convolution layer, the second nonlinear attention layer has the same structure as the first nonlinear attention layer, and S3 specifically includes:

[0050] Local feature enhancement processing:

[0051] Based on the second nonlinear attention layer, perform feature weighting processing on the super-fusion feature map to obtain a local feature map RJ1;

[0052] Global feature enhancement processing:

[0053] Based on the cascaded dilated convolution channel unit, four dilated convolutions with different dilation rates are used to process the super-fused feature maps respectively to obtain feature maps of four receptive field information with different scales;

[0054] The first average pooling unit and the third convolutional layer are used to perform average pooling and one convolution on the super-fused feature map in turn, and then it is concatenated and fused with the feature maps of four receptive field information of different scales. Then, the fourth convolutional layer is used to perform one convolution to obtain the global feature map RQ1.

[0055] In addition, to achieve the above object, the present invention also provides an infrared small target detection system, the system comprising:

[0056] An image preprocessing module is used to obtain an original image of a small infrared target, perform scaling and normalization processing on the original image of the small infrared target, and obtain a preprocessed image F;

[0057] The hyper-fusion backbone network is used to perform multi-level and multi-scale feature fusion processing on the pre-processed image F to obtain a hyper-fusion feature map; the hyper-fusion backbone network includes five fusion processing modules, and the fusion processing module includes at least one nonlinear convolution unit, which is used to perform nonlinear convolution processing on the input feature map;

[0058] The convolutional attention fusion module is used to perform local feature enhancement and global feature enhancement on the super-fusion feature map to obtain the local feature map RJ1 and the global feature map RQ1 respectively;

[0059] The detection result output module is used to splice and fuse the local feature map RJ1 and the global feature map RQ1, and then perform decoding processing to obtain the detection result.

[0060] Optionally, the nonlinear convolution unit includes:

[0061] The first convolution layer is used to perform the first convolution processing on the feature map of the input nonlinear convolution unit;

[0062] The second convolution layer is used to perform a second convolution on the feature map after the first convolution;

[0063] The first nonlinear attention layer is used to perform channel attention weighted processing on the feature map after the second convolution processing, and then perform spatial attention weighted processing.

[0064] Optionally, the convolutional attention fusion module includes:

[0065] The second nonlinear attention layer is used to perform local feature enhancement processing on the super-fusion feature map to obtain the local feature map RJ1;

[0066] The cascaded hole convolution sub-module is used to perform global feature enhancement processing on the super-fused feature map to obtain the global feature map RQ1.

[0067] The present invention performs multi-level and multi-scale feature fusion processing on the pre-processed image F through a hyper-fusion backbone network, which can greatly increase the feature extraction details while ensuring that features of different dimensions are fully fused. On this basis, the convolutional attention fusion module simultaneously performs global feature enhancement processing and local feature enhancement processing on the hyper-fusion feature map, thereby improving the system's capture effect of global and local information in image features. Therefore, the present invention can effectively enhance the system's ability to recognize infrared targets as a whole, thereby improving the detection accuracy of small infrared targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a schematic diagram of the flow chart of the first embodiment of the infrared small target detection method of the present invention;

[0069] Figure 2 This is a schematic diagram of the hyper-converged backbone network processing flow;

[0070] Figure 3 It is a schematic diagram of the structure of the nonlinear convolution unit;

[0071] Figure 4 Schematic diagram of the structure of the first nonlinear attention layer;

[0072] Figure 5 Schematic diagram of the processing flow of the nonlinear attention mechanism;

[0073] Figure 6 Schematic diagram of the processing flow of the spatial attention mechanism;

[0074] Figure 7 Schematic diagram of the structure of the convolutional attention fusion module.

[0075] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0076] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0077] The inventive concept of the present application is further described below in conjunction with some specific embodiments and implementation methods.

[0078] The embodiment of the present invention provides a method for detecting small infrared targets. Figure 1 , Figure 1 The figure is a flow chart of a first embodiment of a method for detecting small infrared targets according to the present invention.

[0079] In this embodiment, the infrared small target detection method includes:

[0080] Step S1: Acquire the original image of the small infrared target, perform scaling and normalization processing on the original image of the small infrared target, and obtain a preprocessed image F.

[0081] Step S1 specifically includes:

[0082] Step S11: scaling the original image of the small infrared target to convert it into an image of target resolution.

[0083] Step S12: normalize the R, G, and B channels of the scaled image to obtain a preprocessed image F.

[0084] Specifically, the original image of the small infrared target is scaled and converted into an image with a resolution of 320*320 (target resolution), which can reduce the burden of computational processing while retaining enough details.

[0085] Each RGB channel of the scaled image is normalized separately to avoid the problem of large differences in image brightness in different areas of the infrared image caused by lighting changes. This allows the system to focus more on the relative structural information of the image rather than the absolute brightness value, thereby improving the robustness of the system under different lighting conditions.

[0086] During the normalization process, for the red channel, the mean value was 0.485 and the standard deviation was 0.229; for the green channel, the mean value was 0.456 and the standard deviation was 0.224; for the blue channel, the mean value was 0.406 and the standard deviation was 0.225.

[0087] In this process, using specific means and standard deviations (such as means of 0.485, 0.456, and 0.406 and standard deviations of 0.229, 0.224, and 0.225) to normalize the RGB channels can unify the distribution of color channels and make the value range of each channel more consistent during input; at the same time, it can reduce the influence of lighting and other external factors, thereby better extracting the structural features of the image.

[0088] Step S2: Input the preprocessed image F into the hyper-fusion backbone network, perform multi-level multi-scale feature fusion processing, and obtain a hyper-fusion feature map.

[0089] Figure 2 The following is a schematic diagram of the hyper-converged backbone network processing flow. Figure 2As shown, the hyper-converged backbone network includes five fusion processing modules, namely, a first fusion processing module, a second fusion processing module, a third fusion processing module, a fourth fusion processing module and a fifth fusion processing module; the first fusion processing module includes five nonlinear convolution units with the same structure, namely, a first nonlinear convolution unit (0,0), a second nonlinear convolution unit (1,0), a third nonlinear convolution unit (2,0), a fourth nonlinear convolution unit (3,0) and a fifth nonlinear convolution unit (4,0); the second fusion processing module includes four nonlinear convolution units with the same structure, namely, a first nonlinear convolution unit (0,1 ), a second nonlinear convolution unit (1,1), a third nonlinear convolution unit (2,1) and a fourth nonlinear convolution unit (3,1); the third fusion processing module includes three nonlinear convolution units with the same structure, namely the first nonlinear convolution unit (0,2), the second nonlinear convolution unit (1,2) and the third nonlinear convolution unit (2,2); the fourth fusion processing module includes two nonlinear convolution units with the same structure, namely the first nonlinear convolution unit (0,3) and the second nonlinear convolution unit (1,3); the fifth fusion processing module includes one nonlinear convolution unit, which is the first nonlinear convolution unit (0,4).

[0090] Step S2 specifically includes:

[0091] The processing process of the first fusion processing module is as follows: the preprocessed image F is input into the first nonlinear convolution unit (0,0) for processing to obtain a 16-dimensional feature map P00; the feature map P00 is downsampled twice and then input into the second nonlinear convolution unit (1,0) for processing to obtain a 32-dimensional feature map P10; the feature map P10 is downsampled twice and then input into the third nonlinear convolution unit (2,0) for processing to obtain a 64-dimensional feature map P20; the feature map P20 is downsampled twice and then input into the fourth nonlinear convolution unit (3,0) for processing to obtain a 128-dimensional feature map P30; the feature map P30 is downsampled twice and then input into the fifth nonlinear convolution unit (4,0) for processing to obtain a 256-dimensional feature map P40.

[0092] The processing process of the second fusion processing module is as follows: the feature map P00, the feature map P10, the feature map P20, the feature map P30 and the feature map P40 are concatenated and fused, and input into the first nonlinear convolution unit (0,1) for processing to obtain a 16-dimensional feature map P01; after the feature map P01 is doubled downsampled, it is concatenated and fused with the feature map P10, the feature map P20, the feature map P30 and the feature map P40, and then input into the second nonlinear convolution unit (1,1) for processing to obtain a 32-dimensional feature map P11 ; After the feature map P11 is downsampled by two times, it is concatenated and fused with the feature map P00, the feature map P20, the feature map P30 and the feature map P40, and then input into the third nonlinear convolution unit (2,1) for processing to obtain a 64-dimensional feature map P21; After the feature map P21 is downsampled by two times, it is concatenated and fused with the feature map P00, the feature map P10, the feature map P30 and the feature map P40, and then input into the fourth nonlinear convolution unit (3,1) for processing to obtain a 128-dimensional feature map P31.

[0093] The processing process of the third fusion processing module is as follows: the feature map P01, the feature map P11, the feature map P21, the feature map P31 and the feature map P00 are spliced ​​and fused, and input into the first nonlinear convolution unit (0,2) for processing to obtain a 16-dimensional feature map P02; after the feature map P02 is downsampled by two times, it is spliced ​​and fused with the feature map P11, the feature map P21, the feature map P31 and the feature map P10, and then input into the second nonlinear convolution unit (1,2) for processing to obtain a 32-dimensional feature map P12; after the feature map P12 is downsampled by two times, it is spliced ​​and fused with the feature map P01, the feature map P21, the feature map P31 and the feature map P20, and then input into the third nonlinear convolution unit (2,2) for processing to obtain a 64-dimensional feature map P22.

[0094] The processing process of the fourth fusion processing module is as follows: the feature map P02, the feature map P12, the feature map P22, the feature map P00 and the feature map P01 are concatenated and fused, and input into the first nonlinear convolution unit (0,3) for processing to obtain a 16-dimensional feature map P03; after the feature map P03 is downsampled by two times, it is concatenated and fused with the feature map P12, the feature map P22, the feature map P10 and the feature map P11, and then input into the second nonlinear convolution unit (1,3) for processing to obtain a 32-dimensional feature map P13.

[0095] The processing process of the fifth fusion processing module is as follows: the feature map P03, the feature map P13, the feature map P00, the feature map P01 and the feature map P02 are concatenated and fused, and input into the first nonlinear convolution unit (0,4) for processing to obtain a 16-dimensional feature map P04.

[0096] Hyper-converged backbone network output process: Feature map P40, feature map P31, feature map P22, feature map P13 and feature map P04 are concatenated and fused to obtain a 16-dimensional hyper-converged feature map.

[0097] In this process, multi-level and multi-scale feature fusion of the pre-processed image F is performed through five fusion processing modules, which can greatly increase the feature extraction details and ensure the full fusion of features of different dimensions, thereby improving the system's detection performance for small infrared targets.

[0098] Furthermore, the structure of each nonlinear convolution unit in the above five fusion processing modules is the same. The processing process of the nonlinear convolution unit is described below by taking the first nonlinear convolution unit (0,0) in the first fusion processing module as an example.

[0099] Figure 3 It is a schematic diagram of the structure of the nonlinear convolution unit. Figure 3 As shown, the nonlinear convolution unit includes a first convolution layer, a second convolution layer, and a first nonlinear attention layer.

[0100] The processing process of the nonlinear convolution unit specifically includes the following steps:

[0101] Step A1: The feature map input to the nonlinear convolution unit is first processed by the first convolution layer and then by the second convolution layer to obtain the feature map J1.

[0102] Step A2: Input the feature map J1 into the first nonlinear attention layer for feature weighting processing to obtain the feature map J2.

[0103] Step A3: After the feature map J2 is concatenated and fused with the feature map input to the nonlinear convolution unit, the nonlinear convolution processing is completed through the activation function output.

[0104] That is, after the preprocessed image F is input into the first nonlinear convolution unit (0,0), the first convolution layer and the second convolution layer perform convolution processing in sequence to extract the features of the preprocessed image F. After the feature extraction is completed, the first nonlinear attention layer is used to perform feature weighting processing on the extracted features.

[0105] Furthermore, Figure 4 is a schematic diagram of the structure of the first nonlinear attention layer, as shown in Figure 4 As shown, the feature weighting processing of the first nonlinear attention layer specifically includes the following steps:

[0106] Step A21: Obtain the channel attention weight w1 of the feature map J1 through the nonlinear attention mechanism, and use the channel attention weight w1 to perform channel weighted processing on the feature map J1 to obtain the feature map J11.

[0107] Step A22: Obtain the spatial attention weight w2 of the feature map J11 through the spatial attention mechanism, and use the spatial attention weight w2 to perform spatial weighted processing on the feature map J11 to obtain the feature map J2.

[0108] Furthermore, Figure 5 This is a schematic diagram of the processing flow of the nonlinear attention mechanism, such as Figure 5 As shown, step A21 specifically includes the following steps:

[0109] Step A21-1: Use the ReLU activation function to activate the input feature map J1 to enhance its nonlinear characteristics and obtain the activated feature map.

[0110] Step A21-2: Use the channel attention mechanism to perform maximum pooling and average pooling on the activated feature maps to obtain feature maps i1 and i2 respectively.

[0111] In this process, the activated feature map is processed by maximum pooling, and the maximum value in the pooling window is selected to focus on the most significant features. The activated features are processed by average pooling, and the average value of the pooling window is calculated to obtain smoother and more holistic features.

[0112] Step A21-3: Perform full connection processing on feature map i1 and feature map i2 respectively, and map them to a feature space with a size of 1 / 16 of the input dimension to reduce the dimension of the features while enhancing the abstraction level of information; then use the ReLU activation function to activate the fully connected feature maps, and perform full connection processing on the activated feature maps again to restore them to the same dimension as the input feature map, and obtain feature maps i11 and feature maps i21 respectively.

[0113] Step A21-4: Fuse the feature map i11 and the feature map i21, and then use the Sigmoid logic function to output the channel attention weight w1;

[0114] Step A21-5: Use the channel attention weight w1 to perform channel weighted processing on the feature map J1 to obtain the feature map J11.

[0115] In this process, the nonlinear attention mechanism is used to perform channel-weighted processing on the feature map, which can strengthen the nonlinear characteristics of the feature map and enhance the system's ability to recognize complex features in the feature map, thereby improving the system's detection performance.

[0116] Furthermore, Figure 6 This is a schematic diagram of the processing flow of the spatial attention mechanism, such as Figure 6 As shown, step A22 specifically includes the following steps:

[0117] Step A22-1: Perform average pooling and maximum pooling processing on the feature channels of the feature map J11 to obtain two single-channel feature maps;

[0118] Step A22-2: concatenate and fuse the two single-channel feature maps, perform a convolution operation after fusion, and obtain a single-channel spatial attention weight map;

[0119] Step A22-3: Use the Sigmoid logic function to process the single-channel spatial attention weight map to obtain the spatial attention weight w2;

[0120] Step A22-4: Use the spatial attention weight w2 to perform spatial weighted processing on the feature map J11 to obtain the feature map J2.

[0121] That is to say, in the processing process taking the first nonlinear convolution unit (0,0) in the first fusion processing module as an example, the first nonlinear attention layer in the first nonlinear convolution unit (0,0) outputs the feature map J2, and then the preprocessed image F input into the first nonlinear convolution unit (0,0) and the feature map J2 are spliced ​​and fused through the residual connection, and the feature map P00 is output through the activation function, which is the feature map output by the first nonlinear convolution unit (0,0).

[0122] In this process, the feature-weighted image is concatenated and fused with the input image through residual connection, which can retain the original information of the input image as much as possible and avoid the loss of useful information after multi-layer attention mechanism and convolution operation.

[0123] Step S3: Input the super-fused feature map into the convolutional attention fusion module to perform local feature enhancement processing and global feature enhancement processing to obtain the local feature map RJ1 and the global feature map RQ1 respectively.

[0124] Specifically, Figure 7 This is a schematic diagram of the structure of the convolutional attention fusion module. Figure 7 As shown, the convolutional attention fusion module includes a second nonlinear attention layer and a cascaded hole convolution submodule, the cascaded hole convolution submodule includes a first average pooling unit, a third convolution layer, a cascaded hole convolution channel unit and a fourth convolution layer, the second nonlinear attention layer has the same structure as the first nonlinear attention layer, and step S3 specifically includes:

[0125] Local feature enhancement processing:

[0126] Step S31: Based on the second nonlinear attention layer, perform feature weighting processing on the super-fusion feature map to obtain the local feature map RJ1.

[0127] Among them, the second nonlinear attention layer performs the same processing steps as the first nonlinear attention layer, that is, the input is replaced by the feature map J1 with the super-fused feature map to obtain the local feature map RJ1, which will not be repeated here.

[0128] In this process, the second nonlinear attention layer is used to perform weighted processing on the hyper-fused features through the nonlinear attention mechanism and the spatial attention mechanism, which can improve the system's attention to the key feature channels in the hyper-fused feature map, thereby achieving the purpose of enhancing local features.

[0129] Global feature enhancement processing:

[0130] Step S32: Based on the cascaded dilated convolution channel unit, the super-fused feature map is processed respectively using dilated convolutions with four different dilation rates (the dilation rates are 1, 4, 8, and 12, respectively) to obtain feature maps of receptive field information of four different scales.

[0131] Step S33: After using the first average pooling unit and the third convolutional layer to perform average pooling processing and one convolution processing on the super-fused feature map in turn, it is spliced ​​and fused with the feature maps of four different scale receptive field information, and then the fourth convolutional layer is used to perform one convolution processing to obtain the global feature map RQ1.

[0132] In this process, by performing dilated convolution processing with different dilation rates (such as 1, 4, 8, and 12), the receptive field can be expanded, a wider range of contextual information can be captured, and the system's ability to capture global features can be improved. On this basis, fusion with feature maps after average pooling and convolution processing can achieve a better balance between multi-scale contextual information and local details.

[0133] Step S4: The local feature map RJ1 and the global feature map RQ1 are concatenated and fused, and then decoded to obtain the detection result.

[0134] Specifically, the local feature map RJ1 and the global feature map RQ1 obtained in step S3 are spliced ​​and fused, and the fused image is input into the decoder for decoding. The decoder can specifically use a convolution layer, and the channel of the feature map output by the convolution layer is 1, which is used to represent the existence or location information of the target to obtain the detection result.

[0135] In this embodiment, the pre-processed image F is subjected to multi-level multi-scale feature fusion processing through the hyper-fusion backbone network, which can greatly increase the feature extraction details while ensuring that the features of different dimensions are fully fused. On this basis, the hyper-fusion feature map is simultaneously subjected to global feature enhancement processing and local feature enhancement processing through the convolution attention fusion module, thereby improving the system's capture effect of global and local information in image features. Therefore, the present invention can effectively enhance the system's ability to recognize infrared targets as a whole, thereby improving the detection accuracy of small infrared targets.

[0136] Furthermore, to achieve the above objectives, the present invention also provides an infrared small target detection system, which may include an image preprocessing module, a hyper-fusion backbone network, a convolutional attention fusion module, and a detection result output module.

[0137] Among them, the image preprocessing module is used to obtain the original image of the infrared small target, scale and normalize the original image of the infrared small target, and obtain the preprocessed image F; the hyper-fusion backbone network is used to perform multi-level multi-scale feature fusion processing on the preprocessed image F to obtain a hyper-fusion feature map; the hyper-fusion backbone network may include five fusion processing modules, and the fusion processing module includes at least one nonlinear convolution unit, which is used to perform nonlinear convolution processing on the input feature map; the convolution attention fusion module is used to perform local feature enhancement processing and global feature enhancement processing on the hyper-fusion feature map to obtain a local feature map RJ1 and a global feature map RQ1 respectively; the detection result output module is used to splice and fuse the local feature map RJ1 and the global feature map RQ1, and then perform decoding processing to obtain the detection result.

[0138] Furthermore, the nonlinear convolution unit may include a first convolution layer, a second convolution layer, and a first nonlinear attention layer;

[0139] Among them, the first convolution layer is used to perform the first convolution processing on the feature map of the input nonlinear convolution unit; the second convolution layer is used to perform the second convolution processing on the feature map after the first convolution processing; the first nonlinear attention layer is used to perform channel attention weighted processing and then spatial attention weighted processing on the feature map after the second convolution processing.

[0140] Furthermore, the convolutional attention fusion module may include a second non-linear attention layer and a cascaded dilated convolution sub-module.

[0141] Among them, the second nonlinear attention layer is used to perform local feature enhancement processing on the super-fusion feature map to obtain the local feature map RJ1; the cascaded hole convolution sub-module is used to perform global feature enhancement processing on the super-fusion feature map to obtain the global feature map RQ1.

[0142] It should be noted that the functions that can be realized by each module in the infrared small target detection system provided in this embodiment and the corresponding technical effects achieved can refer to the description of the specific implementation methods in each embodiment of the infrared small target detection method of the present invention. For the sake of brevity of the specification, they will not be repeated here.

[0143] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0144] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting small infrared targets, characterized in that: The method comprises: S1, obtaining an original image of a small infrared target, performing scaling and normalization processing on the original image of the small infrared target, and obtaining a preprocessed image F; S2, inputting the preprocessed image F into the hyper-fusion backbone network, performing multi-level multi-scale feature fusion processing, and obtaining a hyper-fusion feature map; wherein the hyper-fusion backbone network includes multiple fusion processing modules, the fusion processing module includes at least one nonlinear convolution unit, the nonlinear convolution unit includes a first convolution layer, a second convolution layer and a first nonlinear attention layer; the multi-scale feature fusion processing includes splicing fusion processing and nonlinear convolution processing, and the steps of the nonlinear convolution processing specifically include: A1, the feature map input to the nonlinear convolution unit is first processed by the first convolution layer and then by the second convolution layer to obtain the feature map J1; A2, input the feature map J1 into the first nonlinear attention layer for feature weighting processing to obtain the feature map J2; A3, after concatenating and fusing the feature map J2 with the feature map input to the nonlinear convolution unit, outputs it through the activation function to complete the nonlinear convolution processing; S3, inputting the super-fused feature map into a convolutional attention fusion module, performing local feature enhancement processing and global feature enhancement processing, and obtaining a local feature map RJ1 and a global feature map RQ1 respectively; S4, concatenate and fuse the local feature map RJ1 and the global feature map RQ1, and then perform decoding processing to obtain a detection result.

2. The infrared small target detection method according to claim 1, characterized in that: The hyper-converged backbone network includes five fusion processing modules, namely a first fusion processing module, a second fusion processing module, a third fusion processing module, a fourth fusion processing module and a fifth fusion processing module; The first fusion processing module includes five nonlinear convolution units with the same structure, namely a first nonlinear convolution unit (0, 0), a second nonlinear convolution unit (1, 0), a third nonlinear convolution unit (2, 0), a fourth nonlinear convolution unit (3, 0) and a fifth nonlinear convolution unit (4, 0); The second fusion processing module comprises four nonlinear convolution units with the same structure, namely a first nonlinear convolution unit (0, 1), a second nonlinear convolution unit (1, 1), a third nonlinear convolution unit (2, 1) and a fourth nonlinear convolution unit (3, 1); The third fusion processing module comprises three nonlinear convolution units with the same structure, namely a first nonlinear convolution unit (0, 2), a second nonlinear convolution unit (1, 2) and a third nonlinear convolution unit (2, 2); The fourth fusion processing module comprises two nonlinear convolution units with the same structure, namely a first nonlinear convolution unit (0, 3) and a second nonlinear convolution unit (1, 3); The fifth fusion processing module includes one nonlinear convolution unit, which is a first nonlinear convolution unit (0, 4); The S2 specifically includes: Input the preprocessed image F into the first nonlinear convolution unit (0,0) for processing to obtain a 16-dimensional feature map P00; After downsampling the feature map P00 by a factor of two, the feature map P00 is input into the second nonlinear convolution unit (1,0) for processing to obtain a 32-dimensional feature map P10; After downsampling the feature map P10 by a factor of two, the feature map P10 is input into the third nonlinear convolution unit (2,0) for processing to obtain a 64-dimensional feature map P20; After downsampling the feature map P20 by a factor of two, the feature map P20 is input into the fourth nonlinear convolution unit (3,0) for processing to obtain a 128-dimensional feature map P30; After downsampling the feature map P30 by two times, the feature map P30 is input into the fifth nonlinear convolution unit (4,0) for processing to obtain a 256-dimensional feature map P40; The feature map P00, the feature map P10, the feature map P20, the feature map P30 and the feature map P40 are concatenated and fused, and input into the first nonlinear convolution unit (0, 1) for processing to obtain a 16-dimensional feature map P01; After downsampling the feature map P01 by a factor of two, the feature map P01 is concatenated with the feature map P10, the feature map P20, the feature map P30 and the feature map P40, and then input into the second nonlinear convolution unit (1,1) for processing to obtain a 32-dimensional feature map P11; After downsampling the feature map P11 by a factor of two, the feature map P11 is concatenated with the feature map P00, the feature map P20, the feature map P30 and the feature map P40, and then input into the third nonlinear convolution unit (2,1) for processing to obtain a 64-dimensional feature map P21; After downsampling the feature map P21 by a factor of two, the feature map P21 is concatenated with the feature map P00, the feature map P10, the feature map P30 and the feature map P40, and then input into the fourth nonlinear convolution unit (3,1) for processing to obtain a 128-dimensional feature map P31; The feature map P01, the feature map P11, the feature map P21, the feature map P31 and the feature map P00 are concatenated and fused, and input into the first nonlinear convolution unit (0, 2) for processing to obtain a 16-dimensional feature map P02; After downsampling the feature map P02 by a factor of two, the feature map P02 is concatenated with the feature map P11, the feature map P21, the feature map P31 and the feature map P10, and then input into the second nonlinear convolution unit (1, 2) for processing to obtain a 32-dimensional feature map P12; After downsampling the feature map P12 by a factor of two, the feature map P12 is concatenated with the feature map P01, the feature map P21, the feature map P31 and the feature map P20, and then input into the third nonlinear convolution unit (2,2) for processing to obtain a 64-dimensional feature map P22; The feature map P02, the feature map P12, the feature map P22, the feature map P00 and the feature map P01 are concatenated and fused, and input into the first nonlinear convolution unit (0,3) for processing to obtain a 16-dimensional feature map P03; After downsampling the feature map P03 by a factor of two, the feature map P03 is concatenated and fused with the feature map P12, the feature map P22, the feature map P10 and the feature map P11, and then input into the second nonlinear convolution unit (1, 3) for processing to obtain a 32-dimensional feature map P13; The feature map P03, the feature map P13, the feature map P00, the feature map P01 and the feature map P02 are concatenated and fused, and input into the first nonlinear convolution unit (0, 4) for processing to obtain a 16-dimensional feature map P04; The feature map P40, the feature map P31, the feature map P22, the feature map P13 and the feature map P04 are spliced ​​and fused to obtain the 16-dimensional super-fused feature map.

3. The infrared small target detection method according to claim 1, characterized in that: In A2, the feature weighting processing step specifically includes: A21, obtains the channel attention weight w1 of the feature map J1 through the nonlinear attention mechanism, and uses the channel attention weight w1 to perform channel weighted processing on the feature map J1 to obtain the feature map J11; A22, obtains the spatial attention weight w2 of the feature map J11 through the spatial attention mechanism, uses the spatial attention weight w2 to perform spatial weighted processing on the feature map J11, and obtains the feature map J2.

4. The infrared small target detection method according to claim 3, characterized in that: The A21 specifically includes: A21-1, uses the ReLU activation function to activate the input feature map J1, enhances its nonlinear characteristics, and obtains the activated feature map; A21-2, using the channel attention mechanism, performs maximum pooling and average pooling on the activated feature maps to obtain feature maps i1 and i2 respectively; A21-3, fully connect feature map i1 and feature map i2 respectively, map them to a feature space with a size of 1 / 16 of the input dimension, then use the ReLU activation function to activate the fully connected feature maps, and fully connect the activated feature maps again to restore them to the same dimension as the input feature map, and obtain feature maps i11 and i21 respectively; A21-4, fuse the feature map i11 and the feature map i21, and then use the Sigmoid logic function to output the channel attention weight w1; A21-5, use the channel attention weight w1 to perform channel weighted processing on the feature map J1 to obtain the feature map J11.

5. The infrared small target detection method according to claim 3, characterized in that: The A22 specifically includes: A22-1, perform average pooling and maximum pooling processing on the feature channels of feature map J11 respectively to obtain two single-channel feature maps; A22-2, concatenates and fuses the two single-channel feature maps, performs convolution operation after fusion, and obtains a single-channel spatial attention weight map; A22-3, use the Sigmoid logic function to process the single-channel spatial attention weight map to obtain the spatial attention weight w2; A22-4, use the spatial attention weight w2 to perform spatial weighted processing on the feature map J11 to obtain the feature map J2.

6. The infrared small target detection method according to claim 3, characterized in that: The convolutional attention fusion module includes a second nonlinear attention layer and a cascaded hole convolution submodule, the cascaded hole convolution submodule includes a first average pooling unit, a third convolution layer, a cascaded hole convolution channel unit and a fourth convolution layer, the second nonlinear attention layer has the same structure as the first nonlinear attention layer, and S3 specifically includes: Local feature enhancement processing: Based on the second nonlinear attention layer, performing the step of feature weighting processing on the super-fusion feature map to obtain the local feature map RJ1; Global feature enhancement processing: Based on the cascaded dilated convolution channel unit, the super-fused feature map is processed respectively using dilated convolutions with four different dilation rates to obtain feature maps of receptive field information of four different scales; The first average pooling unit and the third convolutional layer are used to perform average pooling processing and one convolution processing on the super-fused feature map in turn, and then the super-fused feature map is spliced ​​and fused with the feature maps of the four receptive field information of different scales, and then the fourth convolutional layer is used to perform one convolution processing to obtain the global feature map RQ1.

7. An infrared small target detection system, characterized in that: The system comprises: An image preprocessing module is used to obtain an original image of a small infrared target, perform scaling and normalization processing on the original image of the small infrared target, and obtain a preprocessed image F; A hyper-fusion backbone network, used for performing multi-level and multi-scale feature fusion processing on the pre-processed image F to obtain a hyper-fusion feature map; the hyper-fusion backbone network includes five fusion processing modules, and the fusion processing module includes at least one nonlinear convolution unit, used for performing nonlinear convolution processing on the input feature map; A convolutional attention fusion module is used to perform local feature enhancement processing and global feature enhancement processing on the super-fusion feature map to obtain a local feature map RJ1 and a global feature map RQ1 respectively; The detection result output module is used to splice and fuse the local feature map RJ1 and the global feature map RQ1, and then perform decoding processing to obtain the detection result.

8. The infrared small target detection system according to claim 7, characterized in that: The nonlinear convolution unit comprises: A first convolutional layer, used for performing a first convolution process on the feature map input into the nonlinear unit; A second convolution layer, used for performing a second convolution process on the feature map after the first convolution process; The first nonlinear attention layer is used to first perform channel attention weighted processing on the feature map after the second convolution processing, and then perform spatial attention weighted processing.

9. The infrared small target detection system according to claim 7, characterized in that: The convolutional attention fusion module includes: The second nonlinear attention layer is used to perform local feature enhancement processing on the super-fusion feature map to obtain a local feature map RJ1; The cascaded hole convolution sub-module is used to perform global feature enhancement processing on the super-fusion feature map to obtain the global feature map RQ1.

Images