Scale and position sensitive infrared weak and small target detection method

Abstract

The invention discloses a scale and position sensitive infrared weak and small target detection method. A hybrid architecture combining a feature pyramid and LSTM time sequence modeling is constructed. Multi-scale features {P2-P5} fused with semantic information and spatial details are generated through feature extraction from bottom to top and feature enhancement paths from top to bottom, and the scale features are respectively input into a special LSTM module for time sequence modeling. According to the method, collaborative perception of a space dimension and a time dimension is realized, so that the model not only can utilize the accurate positioning capability of shallow layer features, but also can capture a target motion track through time sequence context reasoning, and the problem of insufficient robustness of single-frame detection under a complex background is effectively solved. A scale and position sensitive loss function specially designed for a weak and small target is provided. The defect that a traditional loss function is insufficient in optimization of weak and small targets is overcome, physical imaging characteristics of the targets are explicitly introduced to serve as optimization constraints, and the scale adaptability and positioning precision of a detection frame are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, specifically to a method for detecting small infrared targets that is sensitive to scale and location. Background Technology

[0002] Infrared target detection plays a crucial role in military early warning, guidance, and civilian security and disaster monitoring. However, due to the extremely small pixel ratio, lack of texture and shape details, low signal-to-noise ratio, and susceptibility to interference from complex backgrounds such as clouds and waves, detection is extremely difficult. The core problem with existing infrared target detection methods lies in the extremely small pixel ratio and lack of texture and shape features, resulting in a very low signal-to-noise ratio. Traditional methods rely on manually designed prior knowledge, which has insufficient generalization ability under complex background interference, and their robustness and real-time performance in complex combat environments still need improvement.

[0003] Current technology addresses this problem in the following ways:

[0004] Multi-scale spatiotemporal feature detection architecture: To address the problem that single-frame images are easily interfered with by image noise in complex real-world situations, a multi-scale spatiotemporal feature detection architecture is proposed. Modeling is performed on infrared image sequences, enabling the model to use the target's movement trajectory as a strong prior for detection and judgment, which can greatly enhance the robustness of the model in the face of extremely complex environments.

[0005] Scale and position sensitive loss: The loss function generally used in current infrared weak target detection methods is the detection box loss, which does not take into account the objective scale and position information of weak targets. This application proposes a scale and position sensitive loss function, which enables the model to optimize the detection box more accurately.

[0006] Existing technologies have the following shortcomings:

[0007] The application of inherent target characteristics is insufficient: Infrared targets with low resolution have extremely small pixel ratios, extremely low signal-to-noise ratios, and lack identifiable textures and shapes, making their essential features highly scarce, which constitutes a fundamental detection challenge. Traditional methods attempt to solve this problem through artificially designed prior knowledge, but this approach lacks generalization ability and struggles to cope with ever-changing and complex background interference such as clouds and ocean waves.

[0008] System performance bottlenecks in practical applications: Although deep learning methods have alleviated the difficulty of feature extraction to some extent, existing technologies still face severe application challenges overall. These challenges mainly manifest in two aspects: robustness and real-time performance. In complex and ever-changing real-world environments, the stability and reliability of algorithms still need improvement; simultaneously, processing speed cannot yet meet the stringent requirements of critical tasks such as precision guidance and high-speed early warning. Summary of the Invention

[0009] The purpose of this invention is to provide a scale- and position-sensitive infrared weak target detection method, the specific steps of which are as follows:

[0010] Step 1: Image preprocessing: The input is an infrared grayscale image, and the output is a list [cls,conf,x1,y1,x2,y2], where cls represents the category, conf is the confidence score, and the following are the coordinates of the top left corner and the bottom right corner of the detection box. Before building the network, the infrared image needs to be acquired and the detection box needs to be labeled.

[0011] Step 2: Constructing the Feature Pyramid: Construct the multi-scale feature pyramid backbone network. First, construct a bottom-up feature extraction path based on the feedforward convolutional neural network ResNet-50. Select the outputs of the four stages from conv2 to conv5 as the basic feature maps {C2, C3, C4, C5}, with their spatial resolution decreasing sequentially from 1 / 4 to 1 / 16 of the input. Then, construct a top-down feature enhancement path, starting from the deepest layer C5, and perform upsampling by 2 times sequentially. Then, fuse it with the basic feature maps of the corresponding scale through horizontal connections.

[0012] Step 3: Spatiotemporal Feature Module: After obtaining multi-scale features, it is necessary to model these features at the sequence level so that the model can make good use of prior information. Specifically, the multi-scale features {P2, P3, P4, P5} obtained after feeding each image in the image sequence into the backbone network are fed into the corresponding LSTM modules {L2, L3, L4, L5}. The results calculated by each layer are fed into the detection head. Since the size of each LSTM module is different, after training, the model will pay attention to the subtle changes in local features while focusing on the macro motion trend.

[0013] Step 4: Detection Head: After performing detailed feature extraction on the image using the multi-scale spatiotemporal feature module, the features obtained from each layer are input into the detection head. In the detection head module, the features from each layer undergo a convolution operation with a kernel size of 1 to adjust the feature size. Then, the sigmoid function is used for activation, resulting in {a,b,c,d}. Next, the smaller features are fused into the higher-level features, outputting {a+b,a+b+c,a+b+c+d}. Finally, a+b+c+d is fed into the conv+sigmoid module to obtain another output e.

[0014] Step 5: Scale and Position Sensitive Loss Function: The scale and position sensitive loss function aims to address the problem that existing loss functions are insensitive to changes in target scale and position. This loss function consists of two parts: scale-sensitive loss and position-sensitive loss. The scale-sensitive loss is implemented by assigning weights to the intersection-union (IU) loss, as follows:

[0015]

[0016]

[0017] in, and The predicted pixel set and the true labeled pixel set;

[0018] Position-sensitive loss is calculated based on the center point coordinates of the predicted target and the real target. Given a set of predicted pixels... With the set of real labeled pixels The coordinates of their corresponding center points can be obtained by calculating the average of their respective pixel coordinates, denoted as . = ( , ) and = ( , Then, the Cartesian coordinates of these two center points are converted to polar coordinates to predict the center point. For example, its corresponding distance in the polar coordinate system With angle The calculation formula is as follows:

[0019]

[0020]

[0021] Therefore, the formula for the location-sensitive loss function is as follows:

[0022]

[0023] in, and They represent the true center points of the target. Distance and angle in polar coordinates.

[0024] Preferably, image enhancement and preprocessing are performed when loading the dataset, namely image scaling, random rotation, random brightness, and normalization.

[0025] Preferably, the fusion of the horizontal connection with the basic feature map of the corresponding scale is as follows: First, a 1×1 convolution is applied to the Ci layer features to unify the number of channels, then the features of the upsampled Pi+1 layer are added element by element, and finally the aliasing effect is eliminated by a 3×3 convolution to generate a fused feature pyramid {P2,P3,P4,P5}.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] 1. Significant Improvement in Detection Accuracy and Positioning Precision: Through innovative design of scale- and position-sensitive loss functions, the problem of insufficient sensitivity of traditional detection methods to scale changes and positional shifts in small targets is effectively solved. The scale-sensitive loss enhances adaptability to targets of different sizes through a dynamic weight adjustment mechanism, while the position-sensitive loss accurately captures center point deviations using polar coordinate transformation. Experiments show that this loss function improves the positioning accuracy of the detection box, especially significantly improving the detection rate of extremely small targets smaller than 3×3 pixels, effectively solving the core problems of easy missed detection and inaccurate positioning of small targets.

[0028] 2. Enhanced Robustness and Reduced False Alarm Rate in Complex Environments: By constructing a multi-scale spatiotemporal feature detection architecture, an effective combination of spatial features and temporal context is achieved. The feature pyramid network ensures the ability to extract features from targets at different scales, while the hierarchical LSTM temporal modeling utilizes the consistent features of target motion trajectories to effectively distinguish real targets from random noise. Tests in extreme scenarios such as strong noise and complex cloud interference show that this architecture effectively reduces the false alarm rate while maintaining over 90% detection stability under low signal-to-noise ratio conditions, significantly improving the system's reliability in real-world environments.

[0029] 3. Optimization of Multi-Scale Feature Utilization Efficiency and System Practicality: Through a multi-level feature fusion mechanism of the feature pyramid, efficient integration of shallow spatial details and deep semantic information is achieved. The multi-scale feature fusion strategy of the detection head further enhances the feature preservation capability for small targets, avoiding the feature vanishing problem in deep networks. Actual deployment tests show that the system maintains high accuracy while meeting real-time processing speed requirements, is compatible with mainstream hardware platforms, and provides reliable technical support for practical applications such as military early warning and security monitoring, demonstrating significant engineering application value. Attached Figure Description

[0030] Figure 1 This is a diagram of the overall architecture of the present invention;

[0031] Figure 2 This is the backbone network diagram of the present invention;

[0032] Figure 3 This is a diagram of the detection head system of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0034] Example 1

[0035] A scale- and position-sensitive infrared weak target detection method, characterized by the following specific steps:

[0036] Step 1: Image Preprocessing: The input is an infrared grayscale image, and the output is a list [cls,conf,x1,y1,x2,y2], where cls represents the class, conf is the confidence score, and the following are the coordinates of the top left and bottom right corners of the detection box. Before building the network, the infrared image needs to be acquired and the detection box needs to be labeled. When loading the dataset, image enhancement and preprocessing are performed, namely image scaling, random rotation, random brightness, and normalization.

[0037] Step 2: Constructing the Feature Pyramid: A multi-scale feature pyramid backbone network is constructed. First, a bottom-up feature extraction path is built based on a feedforward convolutional neural network ResNet-50. The outputs of four stages, conv2 to conv5, are selected as the basic feature maps {C2, C3, C4, C5}, with their spatial resolution decreasing sequentially from 1 / 4 to 1 / 16 of the input. Then, a top-down feature enhancement path is constructed, starting from the deepest layer C5, performing a 2x upsampling step by step. The features are then fused with the corresponding scale's basic feature maps through lateral connections: first, a 1×1 convolution is applied to the Ci layer features to unify the number of channels; then, they are added element-wise with the upsampled Pi+1 layer features; finally, a 3×3 convolution is used to eliminate aliasing, generating the fused feature pyramid {P2, P3, P4, P5}. This method innovatively achieves cross-level feature interaction, enabling each pyramid level to simultaneously contain shallow, precise localization information and deep, strong semantic information, significantly improving the model's ability to detect weak targets in complex scenes. The backbone network architecture is as follows: Figure 2 As shown.

[0038] Step 3: Spatiotemporal Feature Module: After obtaining multi-scale features, it is necessary to model these features at the sequence level so that the model can make good use of prior information. Specifically, the multi-scale features {P2, P3, P4, P5} obtained after feeding each image in the image sequence into the backbone network are fed into the corresponding LSTM modules {L2, L3, L4, L5}. The results calculated at each layer are fed into the detector head. Since the size of each LSTM module is different, after training, the model will pay attention to the subtle changes in local features while focusing on the macroscopic motion trend. This hierarchical temporal modeling mechanism enables the model to effectively utilize the target motion trajectory as strong prior knowledge, significantly improving the ability to track and recognize the trajectory of small targets in complex backgrounds. The combination method is as follows: Figure 1 As shown.

[0039] Step 4: Detection Head: After detailed feature extraction of the image using the multi-scale spatiotemporal feature module, the features from each layer are input into the detection head. In the detection head module, each layer's features undergo a convolution operation with a kernel size of 1 to adjust the feature size. Then, the sigmoid function is used for activation, yielding {a,b,c,d}. Next, the smaller features are fused to the higher-level features, outputting {a+b,a+b+c,a+b+c+d}. Finally, a+b+c+d is fed into the conv+sigmoid module to obtain another output e. A 1*1 convolution operation is performed on each output to predict [cls,conf,x1,y1,x2,y2]. The architecture of the detection head is as follows: Figure 3 As shown.

[0040] Step 5: Scale and Position Sensitive Loss Function: The scale and position sensitive loss function aims to address the problem that existing loss functions are insensitive to changes in target scale and position. This loss function consists of two parts: scale-sensitive loss and position-sensitive loss. The scale-sensitive loss is implemented by assigning weights to the intersection-union (IU) loss, as follows:

[0041]

[0042]

[0043] in, and The predicted pixel set and the true labeled pixel set;

[0044] Position-sensitive loss is calculated based on the center point coordinates of the predicted target and the real target. Given a set of predicted pixels... With the set of real labeled pixels The coordinates of their corresponding center points can be obtained by calculating the average of their respective pixel coordinates, denoted as . = ( , ) and = ( , Then, the Cartesian coordinates of these two center points are converted to polar coordinates to predict the center point. For example, its corresponding distance in the polar coordinate system With angle The calculation formula is as follows:

[0045]

[0046]

[0047] Therefore, the formula for the location-sensitive loss function is as follows:

[0048]

[0049] in, and They represent the true center points of the target. Distance and angle in polar coordinates.

[0050] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A scale- and position-sensitive infrared target detection method, characterized in that: The specific steps of this scale- and location-sensitive infrared weak target detection method are as follows: Step 1: Image preprocessing: The input is an infrared grayscale image, and the output is a list [cls,conf,x1,y1,x2,y2], where cls represents the category, conf is the confidence score, and the following are the coordinates of the top left corner and the bottom right corner of the detection box. Before building the network, the infrared image needs to be acquired and the detection box needs to be labeled. Step 2: Constructing the Feature Pyramid: Construct the multi-scale feature pyramid backbone network. First, construct a bottom-up feature extraction path based on the feedforward convolutional neural network ResNet-50. Select the outputs of the four stages from conv2 to conv5 as the basic feature maps {C2, C3, C4, C5}, with their spatial resolution decreasing sequentially from 1 / 4 to 1 / 16 of the input. Then, construct a top-down feature enhancement path, starting from the deepest layer C5, and perform upsampling by 2 times sequentially. Then, fuse it with the basic feature maps of the corresponding scale through horizontal connections. Step 3: Spatiotemporal Feature Module: After obtaining multi-scale features, it is necessary to model these features at the sequence level so that the model can make good use of prior information. Specifically, the multi-scale features {P2, P3, P4, P5} obtained after feeding each image in the image sequence into the backbone network are fed into the corresponding LSTM modules {L2, L3, L4, L5}. The results calculated by each layer are fed into the detection head. Since the size of each LSTM module is different, after training, the model will pay attention to the subtle changes in local features while focusing on the macro motion trend. Step 4: Detection Head: After performing detailed feature extraction on the image using the multi-scale spatiotemporal feature module, the features obtained from each layer are input into the detection head. In the detection head module, the features from each layer undergo a convolution operation with a kernel size of 1 to adjust the feature size. Then, the sigmoid function is used for activation, resulting in {a,b,c,d}. Next, the smaller features are fused into the higher-level features, outputting {a+b,a+b+c,a+b+c+d}. Finally, a+b+c+d is fed into the conv+sigmoid module to obtain another output e. Step 5: Scale and Position Sensitive Loss Function: The scale and position sensitive loss function aims to address the problem that existing loss functions are insensitive to changes in target scale and position. This loss function consists of two parts: scale-sensitive loss and position-sensitive loss. The scale-sensitive loss is implemented by assigning weights to the intersection-union (IU) loss, as follows: in, and The predicted pixel set and the true labeled pixel set; Position-sensitive loss is calculated based on the center point coordinates of the predicted target and the real target, given a set of predicted pixels. With the set of real labeled pixels The coordinates of their corresponding center points can be obtained by calculating the average of their respective pixel coordinates, denoted as . = ( , ) and = ( , Then, the Cartesian coordinates of the two center points are converted to polar coordinates to predict the center point. For example, its corresponding distance in the polar coordinate system With angle The calculation formula is as follows: Therefore, the formula for the location-sensitive loss function is as follows: in, and They represent the true center points of the target. Distance and angle in polar coordinates.

2. The method for detecting small infrared targets that is sensitive to scale and position according to claim 1, characterized in that: Image enhancement and preprocessing are performed when loading the dataset, namely image scaling, random rotation, random brightness, and normalization.

3. The method for detecting small infrared targets that is sensitive to scale and position according to claim 1, characterized in that: The fusion of lateral connections and corresponding scale base feature maps is as follows: First, a 1×1 convolution is applied to the Ci layer features to unify the number of channels. Then, the features of the upsampled Pi+1 layer are added element by element. Finally, a 3×3 convolution is used to eliminate the aliasing effect and generate a fused feature pyramid {P2,P3,P4,P5}.

Images