A method for detecting small infrared targets

Abstract

The present invention relates to the field of target detection technology, and specifically provides an infrared small target detection method. Data enhancement is performed on two acquired infrared small target images A and B, and features are extracted respectively to obtain feature layers A1, A2, A3 and B1, B2, and B3. A final feature layer C3 is obtained through splicing, local contrast calculation, asymmetric fusion, and other means. Anchor-free frame detection is performed on feature layer C3 to obtain target detection results. The method has low computational complexity, can quickly extract infrared small targets in the air under complex backgrounds, accurately and quickly obtain their number and position information, effectively reduce the false alarm rate, and provide a feasible method for detecting infrared small targets in the air.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and specifically provides a method for detecting infrared dim small targets. Background Art

[0002] Early methods for infrared small target detection assumed that the background region in which the target resides changes gently, while the target is an object with high and drastically varying radiation intensity that breaks the background correlation. Based on this assumption, the target is enhanced and detected by filtering or suppressing the background. The basic idea is that a complete infrared small target image can be divided into a gently varying background image and a prominent small target image. These two parts are independent of each other and can be detected by a simple difference method, subtracting the estimated background image from the original image. Representative methods include high-pass filtering, median filtering, and morphological filtering algorithms. However, these methods cannot effectively distinguish between real targets and background clutter, resulting in low accuracy, high false positive rates, low robustness, and weak generalization capabilities. They are not suitable for infrared image detection in real scenes.

[0003] Compared with background estimation methods, low-rank sparse decomposition has better background suppression ability and detection performance. The essential idea of ​​low-rank sparse decomposition is to utilize the low-rank characteristics of the infrared background and the sparse characteristics of infrared weak targets in infrared images. It is assumed that the infrared image is a superposition of low-rank background and sparse targets, and the infrared weak target detection problem is converted into the solution of an unconstrained optimization problem to obtain the target detection result. However, due to the increase in the computational complexity of the iterative solution of the objective function, the algorithm is very time-consuming and does not meet the real-time requirements of actual engineering applications.

[0004] The development of deep learning has provided new data-driven solutions to these problems. Convolutional neural networks can effectively learn the characteristics of infrared small targets from data, enabling end-to-end detection. Many existing target detection methods utilize advanced general-purpose image detection models, such as Faster R-CNN, UNet, and YOLO. However, these image detection models primarily target general-purpose, traditional targets. Existing infrared small target models suffer from structural redundancy, leading to targets disappearing into the background. Furthermore, general-purpose models run slowly, failing to meet real-time and accuracy requirements, resulting in low detection efficiency.

[0005] MDvsFA-cGAN is an algorithm based on conditional generative adversarial networks (cGANs). Its core concept is to train two generators separately to suppress the missed detection rate (MD) and false alarm rate (FA) in the small object segmentation task. A generative consistency loss is used to encourage the two generators to communicate, balancing MD and FA, thereby improving the accuracy and robustness of small object segmentation. The ACM algorithm focuses on feature fusion, generating feature maps through an encoder-decoder structure. Based on the information contained in the features, an asymmetric structure is used to fuse low-level and deep semantics to obtain more effective feature representations. These methods approach infrared dim small target detection from different perspectives, but they still have many shortcomings. First, the stacked convolution operation limits the network's perceptual field, often requiring global information to distinguish the target's location. Furthermore, some methods are limited to single-scale measurements when acquiring global information, which limits the network's detection accuracy. Furthermore, the feature fusion process constrains low-level and deep semantics separately, resulting in feature mismatch and reducing the network's feature representation capability.

[0006] Since 2018, some detection algorithms have incorporated deep learning for single-frame detection. Traditional detection algorithms build mathematical models based on the patterns of targets in images. Deep learning, on the other hand, uses a data-driven approach, allowing the model to automatically learn network parameters based on the input image to achieve optimal values, thus avoiding the limitations of manually summarizing patterns and building models. However, the following problems still exist in the current algorithms: In terms of detection, the actual targets of traditional detection algorithms differ from the theoretical values, and they are unable to summarize all background features. In addition, the presence of noise interference in the image leads to a high number of false alarms during detection. Deep learning algorithms do not require a large number of preset hyperparameters and are more robust than traditional detection algorithms, with a lower false alarm rate. However, there is still a gap in performance compared to conventional target detection. Therefore, when designing algorithms, it is necessary to explore algorithm structures that are more suitable for the characteristics of weak and small targets.

[0007] During long-distance detection, targets occupy few pixels and are often noisy. Single-frame detection makes it difficult to pinpoint the target, and manual identification requires multi-frame temporal information to determine the target's location. However, deep learning detection algorithms currently rely primarily on single-frame detection, which fails to effectively utilize inter-frame information during algorithm design, leading to missed detections and false detections in complex backgrounds.

[0008] In airborne infrared search and tracking (IRST) applications, achieving high-precision and efficient detection of small infrared targets is a core task. These targets typically occupy only a few pixels in the image, exhibit weak signal strength, and lack significant texture and shape features. Detection algorithms for these small infrared targets face the following key challenges:

[0009] 1. Low-quality input constraints: The infrared images processed by the algorithm inherently have low resolution, poor contrast, and blurred visual effects. This is due to limitations in infrared imaging technology. This inherent limitation makes it extremely difficult to accurately distinguish a small set of target pixels from the background.

[0010] 2. Feature scarcity: To meet long-range early warning requirements, algorithms must effectively identify targets even when they appear as extremely small areas within an image. In these cases, the target contains little usable shape, texture, or structural information, making traditional algorithms based on feature or template matching prone to failure.

[0011] 3. Robustness Challenges in Low Signal-to-Noise Ratio Environments: Long-range detection results in extremely weak target infrared radiation signals. Simultaneously, images are subject to strong interference from various noise sources (such as thermal noise, readout noise, and quantization noise) and complex background clutter (such as cloud edges and ground heat sources). The algorithm must possess strong noise immunity and weak signal extraction capabilities under extremely low signal-to-noise ratio (SNR) conditions to prevent the target from being completely overwhelmed by noise or clutter.

[0012] 4. Complex dynamic background suppression: In air-to-air scenarios, the background is highly dynamic and uncertain. The algorithm must accurately model and suppress this dynamically changing background in real time. Small, dim targets can easily be confused with strong interference sources such as cloud edges, solar flares, and hot objects, significantly increasing false alarm rates. Distinguishing targets from these bright background edges or transient interference is a core challenge for the algorithm.

[0013] 5. The dual pressures of real-time performance and accuracy: Faced with high-speed, highly maneuverable, small infrared targets, detection algorithms must complete processing and output accurate results in an extremely short time. However, high-precision small target detection algorithms often have high computational complexity and a large number of model parameters, making them difficult to meet the limited computing resources and stringent real-time requirements of airborne platforms. Designing lightweight, high-speed algorithms while maintaining high detection rates and low false alarm rates is a key bottleneck in meeting these requirements.

[0014] In summary, it is of great research significance to improve the target detection accuracy in passive detection, study the detection algorithm of infrared weak targets in the air, establish a model suitable for infrared weak target detection, and design a detection algorithm for infrared weak targets. Summary of the Invention

[0015] To address the above issues, the present invention provides an infrared small target detection method. This method performs data enhancement on two acquired infrared small target images A and B, performs feature extraction on each, and then obtains the final feature layer C3 through splicing, local contrast calculation, and asymmetric fusion. Anchor-free frame detection is performed on feature layer C3 to obtain target detection results. This method has low computational complexity and effectively reduces the false alarm rate.

[0016] The infrared small target detection method provided by the present invention includes:

[0017] S1: Acquire two frames of infrared dim target images, marked as image A and image B;

[0018] S2: Perform data enhancement on images A and B;

[0019] S3: Perform feature extraction on image A and image B respectively, and obtain feature layers A1, A2, A3 and feature layers B1, B2, B3 respectively;

[0020] S4: Concatenate feature layer A3 and feature layer B3, and obtain feature layer C1 through 3D convolution operation;

[0021] S5: Calculate the local contrast of feature layer A1 and feature layer A2 respectively to obtain feature layer and feature layer ;

[0022] S6: For feature layer C1 and feature layer Perform asymmetric fusion to obtain feature layer C2;

[0023] S7: For feature layer C2 and feature layer Perform asymmetric fusion to obtain feature layer C3;

[0024] S8: Perform anchor-free frame detection on feature layer C3 to obtain target detection results.

[0025] Preferably, image A and image B are two adjacent frames of infrared weak target images.

[0026] Preferably, the data enhancement method includes operations such as changing the size of the infrared small target image, performing color gamut transformation on the infrared small target image, and flipping the infrared small target image.

[0027] Preferably, the splicing method is: directly superimposing the feature layers of feature layer A3 and feature layer B3.

[0028] Preferably, the 3D convolution operation process is:

[0029] S4.1: For P3D-A, spatial calculation is performed first, followed by time domain calculation in a cascade manner, and the final time domain calculation is connected to the output port;

[0030] S4.2: For P3D-B, use a parallel approach and act on the results simultaneously;

[0031] S4.3: For P3D-C, a compromise approach is adopted, performing the calculation in parallel.

[0032] Preferably, the local contrast calculation method is: obtain the feature points on the feature layer A1 or the feature layer A2 , will collect points With this feature point Contrast calculation of the surrounding 8 feature points:

[0033] ;

[0034] ;

[0035] Among them, x and y represent the horizontal and vertical offset sizes respectively. Representation feature layer midpoint The result of contrast calculation (after multiplication with the neighborhood point), express Channel length is Width The feature layer, express Feature points at the location, Representation feature layer The minimum eigenvalue calculated in .

[0036] Preferably, in step S6, the asymmetric fusion method is:

[0037] S6.1: For low-level semantic branches Using the point attention mechanism, the generated weights can be expressed as:

[0038] ;

[0039] in, Denotes point convolution, subscripts 1 and 2 denote the serial numbers of the point convolution, and the sizes of the two point convolutions are C×1×1 and C / 4×1×1 respectively. represents the rectified linear unit (ReLU), represents batch normalization (BN), Represents the Sigmoid function;

[0040] S6.2: For high-level semantic branches First, we use global pooling to traverse the features, change the number of network channels through two convolutions, and finally use the excitation operation to reshape the high-level features and minimize the network parameters;

[0041] The calculation formula is:

[0042] ;

[0043] in, represents global average pooling, Represents the feature layer The feature layer output after high-dimensional processing, and Represents two convolution kernels of size 1×1.

[0044] Preferably, in step S6, the asymmetric fusion method is the same as step S6, only As high-dimensional input, C2 as low-dimensional input.

[0045] Preferably, the total loss function for anchor-free box detection includes a heat map loss function, a center point offset loss function, and an object length and width loss function.

[0046] Preferably, the total loss function expression is:

[0047] ;

[0048] in, represents the heat map loss value, and Both represent the balance coefficient for correcting the center point offset loss and the target length and width loss.

[0049] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0050] The present invention utilizes image sequence to simultaneously input two adjacent images to increase the information source, extracts intra-frame and inter-frame information from the image pair, adopts a dual-channel feature extraction method, utilizes three-dimensional convolution to extract inter-frame information under the premise of controlling the parameter amount, and superimposes multiple 3D convolutions to ensure the diversity of feature extraction. At the same time, an asymmetric fusion decoding structure is proposed to enhance the retention of texture information and position information in decoding, and then a local contrast method is used for feature layer enhancement to capture the relationship between the target and the adjacent background context, enhance the nonlinearity of the feature layer, and realize long-range feature association. In the design of the detection frame, Heatmap, center point offset loss function and target length and width loss function are used for network regression respectively, and finally the detection result is output. The present invention has simple parameter settings and low computational complexity. It can quickly extract infrared weak targets in the air under complex backgrounds, accurately and quickly obtain the number and position information of infrared weak targets, effectively reduce the false alarm rate, and provide a feasible method for the detection of infrared weak targets in the air.

[0051] This paper designs a comprehensive set of aerial infrared small target detection technologies that can be adapted to various complex scenarios. Compared to traditional methods, it is more robust and generalizable, effectively reducing false alarms. Furthermore, compared to existing deep learning algorithms, the proposed method has a lightweight network structure and fast detection speed, meeting real-time requirements.

[0052] In model design, the present invention has the following advantages:

[0053] 1. Compared with directly processing 8-bit images, this algorithm uses raw 14-bit data as input, which is more convenient for end-to-end processing of aerial images;

[0054] 2. The image data enhancement process increases the robustness of the algorithm;

[0055] 3. The dual-channel input detection method can utilize the target timing information in multiple frames of images;

[0056] 4. Combining multiple 3D convolution methods to increase feature diversity;

[0057] 5. Designed a feature extraction backbone network that combines 2D convolution with 3D convolution. 2D convolution extracts spatial information, while 3D convolution extracts inter-frame information, integrating spatial and temporal feature correlations.

[0058] 6. An asymmetric fusion decoding method is proposed, which preserves the target texture information while enhancing the model's retention of spatial position information during feature fusion;

[0059] 7. The idea of ​​local contrast is adopted in model design to capture the relationship between the target and the adjacent background context and enhance the nonlinearity of the feature layer;

[0060] 8. The network has few calculation parameters, short image processing time and strong real-time performance, which meets the requirements of practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 2 is a schematic diagram of a small infrared target detection process according to an embodiment of the present invention;

[0062] Figure 2 is a schematic diagram of a backbone network structure provided according to an embodiment of the present invention;

[0063] Figure 3 3D convolution operation process according to an embodiment of the present invention;

[0064] Figure 4 3D convolution operation process according to an embodiment of the present invention;

[0065] Figure 5 is a schematic diagram of local contrast calculation provided by an embodiment of the present invention;

[0066] Figure 6 It is a schematic diagram of an asymmetric fusion process provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention. Similar elements in different embodiments use associated similar element numbers. In the following embodiments, many detailed descriptions are intended to enable the present invention to be better understood. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, or methods. In some cases, some operations related to the present invention are not shown or described in the specification. This is to avoid the core part of the present invention being overwhelmed by too much description. For those skilled in the art, it is not necessary to describe these related operations in detail. They can fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0068] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other to form various implementation methods. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various orders in the description and the drawings are only for the purpose of clearly describing a certain embodiment and are not intended to be a required order, unless otherwise specified that a certain order must be followed.

[0069] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0070] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0071] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0072] Aiming at the situation that after the infrared weak target in the air is imaged, the infrared weak target occupies few pixels in the image, the quality is low, the background is complex, and the single-frame detection algorithm has missed detection and false detection, a method is proposed such as Figure 1 The infrared small target detection method shown is as follows:

[0073] S1: Acquire two frames of infrared dim target images, marked as image A and image B.

[0074] This embodiment of the present invention uses dual-channel raw data input. This detection method can utilize more data information, thereby improving detection accuracy and robustness. To meet the dual-channel input detection data requirements, two frames of infrared dim target images are required. This embodiment of the present invention labels these two frames as Image A and Image B, respectively.

[0075] Because there is relative motion between the infrared target and the device capturing the infrared target image, the features of two adjacent infrared target images in a series of captured infrared target images are most similar. For this reason, embodiments of the present invention preferably capture two adjacent infrared target images as raw data input, i.e., Image A and Image B are two adjacent frames in the series of infrared target images.

[0076] The characteristics of the infrared small target images captured by this embodiment are as follows: Both infrared small target images A and B are 255×320 pixels in size, with the infrared small target occupying between 2×2 and 3×3 pixels. The background is mostly cloud, with the infrared small target distributed outside, at the edge of, or within the cloud layer. Distant buildings can be seen in a small portion of the background. Both images A and B feature small targets and complex backgrounds.

[0077] It should be noted that the detection device of the embodiment of the present invention needs to recognize jpg format images. If the original image is in raw format, format conversion is required before detection. The 14-bit image is converted from integer to floating point to obtain an 8-bit displayable jpg format. If the original image is in jpg format, it can be directly applied.

[0078] Figure 1 In the figure, LCM represents the local contrast structure, 3D represents the convolutional feature extraction structure, AFSU represents the asymmetric fusion decoding structure, and Conv represents the convolution processing.

[0079] S2: Perform data augmentation on images A and B.

[0080] Image data augmentation is a method of generating more training data by performing various transformations and processing on existing images. This technique can increase the diversity of the dataset and improve the generalization ability and robustness of the model.

[0081] The image data enhancement method of the embodiment of the present invention includes operations such as resizing the infrared small target image, adding gray bars to the excess portion of an image with unequal length and width, performing color gamut conversion on the infrared small target image, and flipping the infrared small target image. When resizing the infrared small target image, the image is scaled to 512×512.

[0082] S3: Perform feature extraction on image A and image B respectively.

[0083] The embodiment of the present invention uses a deep learning network based on multi-frame input to extract inter-frame information from input images A and B.

[0084] like Figure 2 As shown, features of image A and image B are extracted respectively through the backbone network. The backbone network is a multi-frame backbone network structure, which extracts inter-frame information from image A and image B respectively. Each feature extraction part adopts a residual structure, and skip-layer connection is used to avoid network degradation caused by increased depth. Figure 2 In the figure, 3*3Conv2d represents a 2D convolution with a kernel size of 3×3, 7*7Conv2d represents a 2D convolution with a kernel size of 7×7, BN represents a batch normalization layer, ReLU is the activation function, and 3*3Maxpool represents a 3×3 pooling layer. During network design, a residual structure with 1×1 convolutional connections is used to change the network dimension, while a residual structure with direct connections is used to deepen the network. In the backbone network, the number of feature map channels is transformed four times, ultimately outputting a 16×16 feature layer. The extracted feature maps are stacked channel-wise as the output of the intra-frame feature extraction stage.

[0085] Through feature extraction, feature layers A1, A2, and A3 of image A, and feature layers B1, B2, and B3 of image B are obtained accordingly.

[0086] S4: Concatenate feature layer A3 and feature layer B3, and obtain feature layer C1 through 3D convolution operation.

[0087] The feature layer A3 of the image A and the feature layer B3 of the image B are spliced ​​together. The splicing method adopted in the embodiment of the present invention is to directly superimpose the feature layers of the feature layer A3 and the feature layer B3.

[0088] Perform 3D convolution operation on the concatenated feature layer to obtain feature layer C1. The 3D convolution operation process is as follows: Figure 3 As shown, it includes three parts: P3D-A, P3D-B, and P3D-C. Figure 3 Starting from the left arrow, it represents the input data. P3D-A, P3D-B, and P3D-C represent convolution form A, convolution form B, and convolution form C, respectively, which correspond to the three stages of data processing. The structures of P3D-A, P3D-B, and P3D-C are as follows: Figure 4 shown.

[0089] Figure 4 In the figure, ResNet3D represents the standard structure of 3D convolution, 1×1×1Conv3d represents a three-dimensional convolution with a convolution kernel size of 1×1×1, 3×3×3Conv3d represents a three-dimensional convolution with a convolution kernel size of 3×3×3, 1×3×3Conv3d represents a three-dimensional convolution with a convolution kernel size of 1×3×3, and 3×1×1Conv3d represents a three-dimensional convolution with a convolution kernel size of 3×1×1.

[0090] The acquisition process of feature layer C1 specifically includes:

[0091] S4.1: For P3D-A, it is the first processing stage of the 3D convolution operation. The spatial calculation is performed first, then the time domain calculation is performed in a cascade manner, and the final time domain calculation is connected to the output. The calculation method is as follows:

[0092] ,

[0093] in, represents a convolution operation of size 3×1×1, represents a convolution operation of size 1×1×3, represents the input features, represents the output features, represents the original input, .

[0094] S4.2: For P3D-B, a parallel approach is used. This means that the two convolution operations are performed independently. The two parts of the convolution do not affect each other, but act on the result at the same time. The calculation method is as follows:

[0095] .

[0096] S4.3: For P3D-C, a compromise is adopted, and the calculation is performed in parallel. The two can affect each other but both directly affect the results. The calculation method is as follows:

[0097] .

[0098] Features obtained after calculating P3D-C The final feature layer C1 is formed.

[0099] It should be noted that, in the process of calculating P3D-A, P3D-B, and P3D-C one by one, As the input feature calculated at the current moment, represent The calculated output value, obtained in the previous calculation As input for the next calculation , obtained after P3D-C calculation That is the feature layer C1

[0100] S5: Perform local contrast calculation on feature layer A1 and feature layer A2 respectively to obtain feature layers and feature layer .

[0101] Local contrast calculation is an important operation in image processing and analysis. It is used to measure the degree of difference in brightness and darkness in local areas of an image and can highlight details such as edges and textures in the image.

[0102] like Figure 5 As shown, the local contrast calculation method adopted in the embodiment of the present invention is:

[0103] The embodiment of the present invention is used Represents any feature point in the feature layer, that is, Figure 5 The center point o in the feature point Respectively with the surrounding 8 Figure 5 The feature points shown are calculated, for example Figure 5 Point a in and b Each feature point The contrast calculation method with surrounding feature points can be expressed as:

[0104] ,

[0105] Among them, x and y represent the horizontal and vertical offset sizes respectively. Representation feature layer midpoint The result of contrast calculation (after multiplication with the neighborhood point), express Channel length is Width The feature layer, express Feature points at the location.

[0106] Finally, the minimum value is selected to obtain the feature point The eigenvalues ​​of:

[0107] ,

[0108] in, Representation feature layer The minimum eigenvalue calculated in .

[0109] S6: For high-dimensional input feature layer C1 and low-dimensional input feature layer Perform asymmetric fusion to obtain feature layer C2.

[0110] Different feature layers may contain information of different types or importance. Through asymmetric fusion, different weights can be assigned to different feature layers according to needs, so that the fused features can better highlight important information while retaining diverse feature representations.

[0111] The specific process of asymmetric fusion in the embodiment of the present invention is as follows Figure 6 As shown, it is divided into low-level semantic branches and high-level semantic branches. The low-level semantic branches are Figure 6 The lower half of the high-level semantic branch is Figure 6 The upper half of the asymmetric fusion corresponds to C1, which is a high-dimensional input, i.e., a high-level semantic branch. It is a low-dimensional input, that is, a low-level semantic branch, and finally the feature layer C2 is obtained.

[0112] Figure 6 In the figure, C×H×W is the original feature layer size, the number of channels is C×height H×width W, and C / 4×1×1 is the size of the feature layer during processing, the number of channels is C / 4, the height is 1, and the width is 1. is the output feature layer, Input feature maps for low-level semantic branches, Input feature maps to the high-level semantic branch.

[0113] The specific steps include the following:

[0114] S6.1: For low-level semantic branches Using the point attention mechanism, it aggregates the channel feature context of each spatial position separately, passing spatial information in a bottom-up manner, and the generated weights can be expressed as:

[0115] ,

[0116] in, Denotes point convolution, subscripts 1 and 2 denote the serial numbers of the point convolution, and the sizes of the two point convolutions are C×1×1 and C / 4×1×1 respectively. represents the rectified linear unit (ReLU), represents batch normalization (BN), represents the Sigmoid function, Represents the feature layer The feature layer output after low-dimensional processing.

[0117] S6.2: For high-level semantic branches First, global pooling is used to traverse the features, then the number of network channels is changed through two convolutions, and finally the excitation operation is used to reshape the high-level features and minimize the network parameters.

[0118] ,

[0119] in, represents global average pooling, Represents the feature layer The feature layer output after high-dimensional processing, and Represents two convolution kernels of size 1×1.

[0120] S6.3: Finally, the two weights and It is superimposed with the original input to enhance the details and semantic information without losing any information. The final output is:

[0121] ,

[0122] in, Indicates the feature layer C1 and feature layer The result of asymmetric fusion, that is, the feature layer output after asymmetric fusion, is feature layer C2. in Representation feature layer, and feature layer or correspond.

[0123] S7: The feature layer C2 and feature layer obtained by asymmetric fusion Perform asymmetric fusion to obtain feature layer C3. The asymmetric fusion method in this step is the same as step S6. is the high-dimensional input, i.e., the high-level semantic branch, C2 is the low-dimensional input, i.e., the low-level semantic branch, and finally the feature layer C3 is obtained.

[0124] S8: Perform anchor-free frame detection on feature layer C3 to obtain target detection results.

[0125] The total loss function of anchor-free box detection in this embodiment of the present invention is divided into three parts: heatmap loss function, center point offset loss function, and target length and width loss function.

[0126] The calculation formula of the heatmap loss function is as follows:

[0127] ,

[0128] in, represents the heat map loss value, Indicates the number of key points in the image, used for normalization, Represents the coordinate points on the heat map loss function , represents the calculated target prediction value, To mark the true value, and represents a hyperparameter used to balance positive and negative samples and difficult and easy samples. In the embodiment of the present invention, α =2, β =4.

[0129] The focal loss method is used and the heat map loss function is rewritten. In order to suppress the loss of negative sample points near the Gaussian center point in the heat map, the , .

[0130] The center point offset loss function and the target length and width loss function are calculated using the L2 loss method. The center point offset loss function is used to correct the position deviation of each point in the feature layer. The calculation formula is as follows:

[0131] ,

[0132] in, represents the number of key points in the image, represents the offset loss value, represents the predicted offset value, represents the coordinates of the target center point, represents the zoom scale, Indicates the rounded coordinates of the center point after scaling.

[0133] The target length and width loss function is used to calculate the regression loss of the target box size. The calculation formula is as follows:

[0134] ,

[0135] in, Indicates the length and width loss value, represents the sum index, represents the number of key points in the image, Indicates true size. Represents the predicted size.

[0136] Total loss function As follows:

[0137] ,

[0138] in, represents the heat map loss value, and It represents the balance coefficient for correcting the center point offset loss and the target length and width loss. Experimental results show that 、 Achieve optimal results.

[0139] The detection result obtained by the embodiment of the present invention includes target location information, target type and confidence level of the detection result.

[0140] Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0141] The above specific embodiments of the present invention do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for detecting small infrared targets, characterized in that: include: S1: Acquire two frames of infrared dim target images, marked as image A and image B; S2: performing data enhancement on the image A and the image B; S3: Perform feature extraction on the image A and the image B respectively, and obtain feature layers A1, A2, A3 and feature layers B1, B2, B3 accordingly; S4: Concatenate the feature layer A3 and the feature layer B3, and obtain the feature layer C1 through a 3D convolution operation; S5: Calculate the local contrast of the feature layer A1 and the feature layer A2 respectively to obtain the feature layer and feature layer ; S6: The characteristic layer C1 and the characteristic layer Perform asymmetric fusion to obtain feature layer C2; S7: The characteristic layer C2 and the characteristic layer Perform asymmetric fusion to obtain feature layer C3; S8: Perform anchor-free frame detection on the feature layer C3 to obtain target detection results.

2. The infrared small target detection method according to claim 1, wherein: The image A and the image B are two adjacent frames of infrared weak target images.

3. The infrared small target detection method according to claim 1, wherein: The data enhancement method includes changing the size of the infrared small target image, performing color gamut transformation on the infrared small target image, and performing a flip operation on the infrared small target image.

4. The infrared small target detection method according to claim 1, wherein: The splicing method is: directly superimposing the feature layers of the feature layer A3 and the feature layer B3.

5. The infrared small target detection method according to claim 1, wherein: The 3D convolution operation process is: S4.1: For P3D-A, spatial calculation is performed first, followed by time domain calculation in a cascade manner, and the final time domain calculation is connected to the output port; S4.2: For P3D-B, a parallel approach is used, acting on the results simultaneously; the calculation method is as follows: ； in, represents a convolution operation of size 3×1×1, represents a convolution operation of size 1×1×3, represents the input features, represents the output features, Represents the original input; S4.3: A compromise approach is adopted for P3D-C, which is to perform the calculation in parallel. The calculation method is as follows: ； Among them, the previous calculation obtained As input for the next calculation .

6. The infrared small target detection method according to claim 1, wherein: The local contrast calculation method is to obtain the feature points on the feature layer A1 or the feature layer A2: , the feature points With this feature point Contrast calculation of the surrounding 8 feature points: ； ； Among them, x and y represent the horizontal and vertical offset sizes respectively. Representation feature layer midpoint The contrast calculation result is: express Channel length is Width The feature layer, express Feature points at the location, Representation feature layer The minimum eigenvalue calculated in , finally the minimum value is selected to obtain the feature point The eigenvalue of .

7. The infrared small target detection method according to claim 1, wherein: In step S6, the asymmetric fusion method is: S6.1: For low-level semantic branches Using the point attention mechanism, the generated weights can be expressed as: ； in, Denotes point convolution, subscripts 1 and 2 denote the serial numbers of the point convolution, and the sizes of the two point convolutions are C×1×1 and C / 4×1×1 respectively. represents the linear unit of correction, represents batch normalization, Represents the Sigmoid function; S6.2: For high-level semantic branches First, we use global pooling to traverse the features, change the number of network channels through two convolutions, and finally use the excitation operation to reshape the high-level features and minimize the network parameters; The calculation formula is: ； in, represents global average pooling, Represents the feature layer The feature layer output after high-dimensional processing, and Represents two convolution kernels of size 1×1.

8. The infrared small target detection method according to claim 7, wherein: In step S6, the asymmetric fusion method is the same as step S6, only As high-dimensional input, C2 as low-dimensional input.

9. The infrared small target detection method according to claim 1, wherein: The total loss function of the anchor-free box detection includes a heat map loss function, a center point offset loss function, and an object length and width loss function.

10. The infrared small target detection method according to claim 9, wherein: The total loss function expression is: ； in, represents the heat map loss value, and Both represent the balance coefficient for correcting the center point offset loss and the target length and width loss.

Images