A method, system, and computer storage medium for detecting small infrared targets in the air.

Abstract

This invention discloses a method, system, and computer storage medium for detecting weak infrared targets in the air. The method includes: determining learnable candidate boxes; generating learnable candidate features in the target image corresponding to the learnable candidate boxes; inputting the target image into a backbone network, which employs a weighted bidirectional recurrent feature pyramid network and includes a switchable dilated convolution module. The backbone network is used to extract features of different resolutions in the target image and fuse the extracted features to obtain fused features; inputting the learnable candidate features and fused features into a dynamic detection module to extract regional features from each learnable candidate box; and classifying and predicting the regional features to obtain detection results. This invention uses a weighted bidirectional recurrent feature pyramid network to improve feature extraction capabilities and introduces switchable dilated convolution to increase the receptive field with less image information loss, thereby improving detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a method, system, and computer storage medium for detecting small infrared targets in the air. Background Technology

[0002] Infrared imaging relies on the difference in radiation intensity between the target and the background to capture clear target outline information. It is unaffected by harsh environments such as rain, snow, and frost, and provides clear, accurate imaging. It can also identify camouflage and resist interference, making it an essential means of target detection under complex conditions.

[0003] In the early explorations of infrared target detection, many scholars focused on methods based on filtering, the human visual system, and low-rank sparse recovery to achieve infrared target detection through background suppression and target enhancement. With the rapid development of deep neural networks, researchers are now dedicated to applying deep neural networks to infrared multi-scale target detection.

[0004] However, the accuracy of current infrared image-based aerial weak target detection technology still needs to be further improved. Summary of the Invention

[0005] This invention provides a method, system, and computer storage medium for detecting weak aerial infrared targets, which addresses the problem that the accuracy of existing infrared image-based aerial target detection technologies still needs further improvement.

[0006] On one hand, embodiments of the present invention provide a method for detecting weak infrared targets in the air, including:

[0007] Identify learnable candidate boxes;

[0008] Generate learnable candidate features in the target image that correspond to learnable candidate boxes;

[0009] The target image is input into the backbone network, which adopts a weighted bidirectional recurrent feature pyramid network containing a switchable dilated convolution module. The backbone network is used to extract features of different resolutions in the target image and fuse the extracted features to obtain fused features.

[0010] The learnable candidate features and fused features are input into the dynamic detection module to extract the region features of each learnable candidate box;

[0011] The regional features are classified and predicted to obtain the detection results.

[0012] On the other hand, embodiments of the present invention provide an aerial infrared weak target detection system, comprising:

[0013] The candidate box selection module is used to determine learnable candidate boxes;

[0014] The candidate feature generation module is used to generate learnable candidate features in the target image that correspond to learnable candidate boxes;

[0015] The backbone network adopts a weighted bidirectional recurrent feature pyramid network, which includes a switchable dilated convolution module. The backbone network is used to extract features of different resolutions in the target image and fuse the extracted features to obtain fused features.

[0016] The dynamic detection module is used to extract the regional features of each learnable candidate box from the learnable candidate features and fused features, and to classify and predict the regional features to obtain the detection results.

[0017] On the other hand, embodiments of the present invention provide a computer storage medium storing a plurality of computer instructions, which are used to cause a computer to execute the above-described method.

[0018] The method, system, and computer storage medium for detecting weak infrared targets in the air as described in this invention have the following advantages:

[0019] A sparse prior model is used to eliminate the influence of dense priors on detection results, reducing computational complexity. Simultaneously, a weighted bidirectional recurrent feature pyramid network is employed to improve feature extraction capabilities, and switchable dilated convolutions are introduced to increase the receptive field with minimal image information loss, thereby improving detection accuracy. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of an aerial infrared weak target detection method provided in an embodiment of the present invention;

[0022] Figure 2 This is a diagram illustrating the composition of an aerial infrared weak target detection system provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Figure 1 This is a flowchart illustrating a method for detecting weak infrared targets in the air, provided as an embodiment of the present invention. The present invention provides a method for detecting weak infrared targets in the air, comprising:

[0025] S100, identify learnable candidate boxes.

[0026] For example, a small set (100 sets) of learnable candidate boxes (sparse regions) can be selected to replace the Region of Interest (ROI) predicted by the RPN (Region Proposal Network).

[0027] S110, Generate learnable candidate features in the target image corresponding to learnable candidate boxes.

[0028] For example, after selecting learnable candidate boxes, a high-dimensional vector can be randomly initialized based on the learnable candidate boxes. As the network training iterates and updates continuously, learnable candidate features of the target image can be generated.

[0029] S120, The target image is input into the backbone network. The backbone network adopts a weighted bidirectional recurrent feature pyramid network, which includes a switchable dilated convolution module. The backbone network is used to extract features of different resolutions in the target image and fuse the extracted features to obtain fused features.

[0030] For example, the backbone network is trained before the target image is input into it. Training the backbone network includes: inputting the training image into the backbone network to obtain feature extraction results; and updating the parameters of the backbone network using a backpropagation algorithm based on the feature extraction results. Since the parameters of the backbone network are continuously updated during network training, their initial values ​​have a relatively small impact on the final detection results, thus solving the problem of dependence on prior knowledge in conventional dense detection algorithms.

[0031] The process of extracting features from the backbone network and fusing those features mainly includes the following steps:

[0032] (1) The backbone network uses a fast normalization fusion method to fuse the extracted features. The output features obtained by using the fast normalization fusion method are shown below:

[0033]

[0034] in, w m and w n Representing the input features respectively x m and x n Learnable weight parameters. To normalize the weight parameters... w And to ensure weight w m ≥0, use the ReLU function to adjust the weight parameters. w m Make adjustments and set... =0.0001 to avoid numerical instability. Taking the sixth layer of the backbone network as an example, the feature fusion representation of the sixth layer of the backbone network is as follows:

[0035]

[0036] in, It is an intermediate feature of the 6th layer from top to bottom. This represents the input features of the 6th layer from top to bottom. Resize() resizes the features to accommodate convolution calculations. The input features are from the 7th layer of the top-down path. This represents the output features of the 5th layer in the bottom-up path. It is the output feature of the 6th layer in the bottom-up path. These are the learnable weight parameters for the output features.

[0037] In an embodiment of the present invention, in order to improve the detection accuracy of small targets, the backbone network performs a standard normalization operation after each convolution process of the extracted features using a switchable dilated convolution module.

[0038] (2) Add a switchable dilated convolution module to the backbone network. The switchable dilated convolution module consists of two global context modules and one switchable dilated convolution component, with the two global context modules added before and after the switchable dilated convolution component, respectively. The switchable dilated convolution component is then added to the ordinary convolution. r -1 zero is formed, which is equivalent to needing to kn × kn The convolution kernel is expanded to kn = kn +( kn -1)( r -1), where r The parameter represents the void ratio, and different settings can be used. rDifferent receptive fields can be obtained, effectively acquiring multi-scale receptive fields without increasing computational complexity. In this embodiment of the invention, a configuration is set... r =3 dilated convolution component and r A regular convolution with a value of 1 is used to fill in any missing pixels using standard convolution.

[0039] The convolution operation in a switchable dilated convolution component can be represented as: y out =Conv( x , w , r ),in x Indicates input features, w This represents the weights, consistent with the weight values ​​used in the feature pyramid structure. The complete switchable dilated convolution component is then represented as follows:

[0040]

[0041] Wherein, the transformation function S ( x It consists of a 5×5 average pooling layer and a 1×1 convolutional layer, with a void ratio of r The default setting is 3. and Shared weights using a locking mechanism w However, for a void ratio of r The convolutional layers add additional trainable weights This is because pre-trained weights are often used to initialize network parameters when training the backbone network. However, when converting ordinary convolutions to switchable dilated convolutions, convolutional weights with higher dilation rates are lost. Since targets of different scales can be trained with the same weights but different dilation rates... r The convolutional layers perform coarse detection, so the weights that should have been lost are initialized with the weights from the pre-trained model. This means copying the weights of the ordinary convolutional kernel to the convolutional kernel with a larger dilation rate in the switchable dilated convolution.

[0042] S130: Input the learnable candidate features and fused features into the dynamic detection module to extract the region features of each learnable candidate box.

[0043] For example, there are multiple dynamic detection modules, and these modules use the ROI Align operation to extract the region features of each learnable candidate box from the learnable candidate features and the fused features (i.e., Figure 2 After extracting the region features, the dynamic detection module interacts with the region features to filter out invalid feature blocks, outputs the target features, then uses a perceptron to perform regression prediction on the target features, and finally uses a fully connected layer to perform classification prediction on the regression prediction results.

[0044] The number of dynamic detection modules is the same as the number of learnable candidate boxes. The features of each learnable candidate box are input into multiple independent dynamic detection modules for target localization and classification. Each dynamic detection module is configured based on specific region features, where there is a one-to-one correspondence between region features and learnable candidate boxes. N There are 1 learnable candidate box. N Each region has its own features. In specific processing, the dynamic detection module first considers the features of each region... fi ( S × S The system interacts with the target features to filter out invalid feature blocks and outputs the final target features. Then, a 3-layer perceptron with ReLU activation function and hidden dimension C is used to perform regression prediction on the target features. Finally, a fully connected layer is used to perform classification prediction on the regression prediction results.

[0045] In embodiments of the present invention, the interaction process is implemented using an attention mechanism to focus on the parts that need to be prioritized when selecting learnable candidate boxes. First, kernel parameters for convolution are generated using region features. Then, the generated convolution processes the region features to obtain more differentiated features, thereby enabling blocks with more foreground information to contribute significantly to the final target location and category. Furthermore, an iterative structure is employed, using newly generated target regions and target features as learnable candidate boxes and region features for the next iteration to improve network performance. Simultaneously, to reduce network computational complexity, consecutive 1×1 convolutions are used after the ReLU activation function to implement the interaction process.

[0046] S140 classifies and predicts regional features to obtain detection results.

[0047] This invention also provides an aerial infrared weak target detection system, such as... Figure 2 As shown, the system includes:

[0048] The candidate box selection module is used to determine learnable candidate boxes;

[0049] The candidate feature generation module is used to generate learnable candidate features in the target image that correspond to learnable candidate boxes;

[0050] The backbone network adopts a weighted bidirectional recurrent feature pyramid network, which includes a switchable dilated convolution module. The backbone network is used to extract features of different resolutions in the target image and fuse the extracted features to obtain fused features.

[0051] The dynamic detection module is used to extract the regional features of each learnable candidate box from the learnable candidate features and fused features, and to classify and predict the regional features to obtain the detection results.

[0052] This invention also provides a computer storage medium storing a plurality of computer instructions for causing a computer to execute the above-described method.

[0053] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0054] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for detecting weak infrared targets in the air, characterized in that, include: Identify learnable candidate boxes: Select a set of learnable candidate boxes to replace the regions of interest predicted by the RPN; Generate learnable candidate features in the target image that correspond to the learnable candidate bounding boxes; The target image is input into a backbone network, which employs a weighted bidirectional recurrent feature pyramid network and includes a switchable dilated convolution module. This backbone network extracts features of different resolutions from the target image and fuses these extracted features to obtain fused features. The convolution operation in the switchable dilated convolution module is represented as follows: y out =Conv( x , w , r ),in x Indicates input features, w The weights are consistent with those used in the feature pyramid structure. The complete switchable dilated convolution module is represented as follows: Wherein, the transformation function S ( x It consists of a 5×5 average pooling layer and a 1×1 convolutional layer, with a void ratio of r The default setting is 3. and Shared weights using a locking mechanism w For a void ratio of r The convolutional layers add additional trainable weights. ; The learnable candidate features and fused features are input into the dynamic detection module to extract the region features of each learnable candidate box; The region features are classified and predicted to obtain the detection results.

2. The method for detecting weak infrared targets in the air according to claim 1, characterized in that, The backbone network is trained before the target image is input into it.

3. The method for detecting weak infrared targets in the air according to claim 2, characterized in that, The training of the backbone network includes: The training images are input into the backbone network to obtain feature extraction results; Based on the feature extraction results, the parameters of the backbone network are updated using the backpropagation algorithm.

4. The method for detecting weak infrared targets in the air according to claim 1, characterized in that, The backbone network uses a fast normalization fusion method to fuse the extracted features.

5. The method for detecting weak infrared targets in the air according to claim 1, characterized in that, After each convolution process of the extracted features using a switchable dilated convolution module, the backbone network also performs a standard normalization operation.

6. An aerial infrared weak target detection system, characterized in that, include: The candidate box selection module is used to determine learnable candidate boxes: a set of learnable candidate boxes are selected to replace the region of interest predicted by the RPN; A candidate feature generation module is used to generate learnable candidate features in the target image that correspond to the learnable candidate boxes; The backbone network employs a weighted bidirectional recurrent feature pyramid network, which includes a switchable dilated convolution module. This backbone network extracts features at different resolutions from the target image and fuses these extracted features to obtain fused features. The convolution operation in the switchable dilated convolution module is represented as follows: y out =Conv( x , w , r ),in x Indicates input features, w The weights are consistent with those used in the feature pyramid structure. The complete switchable dilated convolution module is represented as follows: Wherein, the transformation function S ( x It consists of a 5×5 average pooling layer and a 1×1 convolutional layer, with a void ratio of r The default setting is 3. and Shared weights using a locking mechanism w For a void ratio of r The convolutional layers add additional trainable weights. ; The dynamic detection module is used to extract the region features of each learnable candidate box from the learnable candidate features and fused features, and to classify and predict the region features to obtain the detection results.

7. The aerial infrared weak target detection system according to claim 6, characterized in that, The switchable dilated convolution module includes a switchable dilated convolution component and two global context modules, with the two global context modules located at the front and rear of the switchable dilated convolution component, respectively.

8. The aerial infrared weak target detection system according to claim 7, characterized in that, The switchable dilated convolution component is formed by adding multiple zeros to a normal convolution.

9. The aerial infrared weak target detection system according to claim 6, characterized in that, The number of dynamic detection modules is multiple. The multiple dynamic detection modules use the ROI Align operation to extract the region features of each learnable candidate box in the learnable candidate features and fused features respectively. After the region features are extracted, the dynamic detection modules also interact with the region features to filter out invalid feature blocks and output the target features. Then, a perceptron is used to perform regression prediction on the target features. Finally, a fully connected layer is used to perform classification prediction on the regression prediction results.

10. A computer storage medium, characterized in that, The computer storage medium stores a plurality of computer instructions, which are used to cause the computer to perform the method described in any one of claims 1-5.

Images