Infrared small target detection method in complex background

Abstract

The application provides an infrared small target detection method in a complex background, image preprocessing is performed through a morphological adaptive reconstruction method to highlight the target center feature; through label conversion, a mask mask reflecting the target contour is constructed according to the labeled target rectangular frame, and the center position of the target is calculated to construct the mask mask; a double-branch U-shaped network is used to learn the contour feature and the center feature of the target through an independent contour branch and a center area branch respectively; the morphological adaptive reconstruction method performs morphological reconstruction through structural elements of different scales and different shapes, and the best reconstruction results are selected and integrated together as the reconstruction results; the double-branch U-shaped network adopts a wavelet decomposition and a skip connection combined mode to extract multi-scale features, and introduces a channel attention mechanism to enhance the expression of key features; through a mask fusion module, the predicted masks of the two branches are fused to generate a target predicted mask as the final output.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of infrared imaging detection, and particularly relates to an infrared small target detection method under complex background. BACKGROUND

[0002] Infrared imaging detection has good concealment, anti-interference and all-weather working advantages, and is widely used in early warning and air defense fields. Since small targets are often submerged in complex background, not only will they be missed due to weak signal, low resolution, lack of shape, color and texture information, but also a large number of clutter false alarms will be generated, and the small target detection capability and scene adaptability need to be improved. The traditional infrared small target detection method mainly utilizes the intensity difference between the target and the background, enhances the signal-to-noise ratio of the target through background suppression, and achieves good performance in uniform background; the human visual system method enhances the contrast between the small target and the background to detect the target, and the utilization efficiency of the complex background characteristics of fluctuation is low; the low-rank sparse matrix models the background as a low-rank signal, and separates the target image from it, and part of the background will be misjudged as a target to generate false alarms. However, the existing methods mainly extract shallow features of the target and the background, and cannot effectively mine deep features, so the adaptability to infrared small target detection under complex scenes is poor. SUMMARY

[0003] In view of the above-mentioned deficiencies of the existing methods, the present application proposes an infrared small target detection method under complex background according to the characteristics that the infrared small target presents obvious central region features but not obvious contour features. First, in view of the problems of noise interference and unobvious target features in the actually collected images, a morphological adaptive reconstruction method is designed for image preprocessing to highlight the target central features. Label conversion is performed, a mask mask reflecting the target contour is constructed according to the labeled target rectangular frame, and the center position of the target is calculated to construct the mask mask. A double-branch U-shaped network is designed, which learns the contour features and central features of the target through independent contour branch and central region branch, captures the global morphology of the target, and focuses on the fine features of the central region. The network adopts the combination of wavelet decomposition and jump connection to extract multi-scale features, and introduces a channel attention mechanism to enhance the expression of key features. Through a mask fusion module, the predicted masks of the two branches are fused to generate a target predicted mask as the final output, which improves the detection accuracy and anti-interference ability of small targets in complex background. The main design includes:

[0004] (1) Image preprocessing. In view of the problem of noise interference in the actual collected image, a multi-scale and multi-structure morphological reconstruction method is designed, an adaptive morphological operation is constructed, morphological reconstruction is carried out through different scale and different shape structural elements, the best part of the reconstruction effect is selected and integrated together as the reconstruction result, the influence of potential noise pixels in the image is eliminated, and the center point features of the preprocessed image are more obvious, so that the accuracy of subsequent target positioning is improved.

[0005] (2) Label conversion, the normalized label value is converted into the actual value in the image coordinate system, and the segmentation mask is generated in the corresponding target area, wherein the pixel value of the target area is 1, and the pixel value of the background area is 0. According to the labeled target rectangular frame, the mask mask reflecting the target contour is constructed, and the mask mask of the target center area is constructed according to the position of the target center point.

[0006] (3) A double-branch U-shaped network is designed, which is a target contour branch and a target center area branch. The double-branch U-shaped network uses two independent U-shaped networks, which have the same architecture, and the original image and the preprocessed image are input at the same time. The target contour branch takes the original image as input, learns the contour features of the target, and outputs the target contour prediction mask; the target center area branch takes the preprocessed image as input, learns the center area features, and outputs the target center area prediction mask. When each branch is connected, the features output by the encoder are wavelet decomposed to extract low-frequency information, and the low-frequency features are combined with the features of the corresponding scale of the decoder, so as to enhance the target features in the decoding process. At the same time, the channel attention mechanism is introduced after the output of the decoder, and the adaptive weight is given to the decoded features, so as to further enhance the target feature expression ability. Finally, the target contour and the center area prediction mask are combined through the mask fusion module to generate the final target prediction mask.

[0007] (4) Performance test, the test image is input into the network model trained to verify the output mask mask, and a rectangular frame is circumscribed as the target detection result, the IoU of the test image is calculated, the experimental results are counted, and the infrared small target detection accuracy is calculated.

[0008] The technical scheme adopted by the present application to solve the technical problem is:

[0009] An infrared small target detection method in a complex background: image preprocessing is carried out through a morphological adaptive reconstruction method to highlight the target center features; through label conversion, according to the labeled target rectangular frame, the mask mask reflecting the target contour is constructed, and the mask mask is constructed by calculating the center position of the target; the double-branch U-shaped network is used to learn the contour features and center features of the target through independent contour branch and center area branch respectively;

[0010] The morphological adaptive reconstruction method performs morphological reconstruction using structural elements of different scales and shapes, and selects the part with the best reconstruction effect to integrate as the reconstruction result.

[0011] The annotation process transforms the normalized annotation values ​​into actual values ​​in the image coordinate system, generating a segmentation mask in the corresponding target region, where the pixel value of the target region is 1 and the pixel value of the background region is 0.

[0012] The dual-branch U-shaped network extracts multi-scale features by combining wavelet decomposition and skip connections, and introduces a channel attention mechanism to enhance the expression of key features. The prediction masks of the two branches are fused through a mask fusion module to generate a target prediction mask as the final output.

[0013] Furthermore, the dual-branch U-shaped network uses two independent U-shaped networks with the same architecture, simultaneously inputting the original image and the preprocessed image of the same size. The dual branches include a target contour branch and a target center region branch. The target contour branch takes the original infrared image as input, learns the contour features of the target, and outputs a target contour prediction mask. The target center region branch takes the preprocessed infrared image as input, learns the features of the target center region, and outputs a target center region prediction mask. In each branch, at each hop connection, the features output by the encoder are decomposed by a wavelet decomposition module to extract low-frequency features. The low-frequency features are then combined with the features of the corresponding scale of the decoder to enhance the target features during the decoding process. After feature combination, the local context information is further extracted by a dual convolution module. After the decoder output, the channel attention module assigns adaptive weights to the features of different channels to further enhance the target feature representation capability. After completing the prediction of the two branches, the target contour mask and the target center region mask are fused by a mask fusion module, and then the output layer generates a complete target prediction mask.

[0014] Furthermore, in the morphological adaptive reconstruction method, two grayscale images f and g represent the marker image as the starting point of the transformation and the template image as the constraint condition, respectively; the erosion operation reconstruction obtained from structuring elements of different scales is represented as follows:

[0015]

[0016] in, S represents the morphological erosion operation. i This represents a structuring element with different scales, where the scale of the structuring element is i, 1≤i≤n, i∈N. + ∨ indicates that the maximum value is taken for each pixel in both images;

[0017] The reconstruction of the dilation operation obtained from structuring elements of different scales is as follows:

[0018]

[0019] in, S represents the morphological erosion operation. i This represents a structuring element with different scales, where the scale of the structuring element is i, 1≤i≤n, i∈N. + , ∧ represents the minimum value taken pixel by pixel in both images;

[0020] Using the infrared image as the template image G, morphological reconstruction of the labeled image F is performed, which is represented as:

[0021]

[0022] In the formula, i,j (1≤i,j≤n,i,j∈N) + ) represents the size of the structuring element. This indicates a reconstruction of the expansion operation. This indicates a reconstruction based on the erosion operation;

[0023] The results of multi-scale and multi-structure operations are synthesized using the maximum value synthesis method. The final adaptive processing result is expressed as follows:

[0024]

[0025] in, It is an adaptive morphological reconstruction operator that increases with the structuring element scale, where m and n are the start and end points of the structuring element scale selection, and k is the selection of the structuring element shape.

[0026] Furthermore, in the wavelet decomposition module: first, a wavelet decomposition operation is performed:

[0027] cA,(cH,cV,cD)=DWT2(x,wavelet)

[0028] Where x represents the current input feature map, cA represents the extracted low-frequency component used to represent the main target features, cH, cV, cD represent the extracted high-frequency components containing noise or irrelevant information, and DWT2 represents the two-dimensional discrete wavelet transform.

[0029] To match the upsampled decoded feature map, an interpolation operation is performed:

[0030] cA resized =Interpolate(cA,target) size )

[0031] Among them, target size cA represents the interpolated target size, matched with the upsampled decoder feature map.resized This represents the low-frequency feature map after interpolation, used for skip connections.

[0032] Furthermore, in the dual convolution module: through two consecutive 3×3 convolutions, and activated by BN and ReLU: the first convolution layer is represented as:

[0033] y1 = ReLU(BN(Conv2D(x,W1)))

[0034] Where x represents the input feature map, W represents the convolution kernel weights, and y1 represents the intermediate features after the first convolution; the second convolution is represented as:

[0035] y2 = ReLU(BN(Conv2D(y1,W2)))

[0036] Here, W2 represents the weight of the second convolutional kernel.

[0037] Furthermore, the downsampling of the dual-branch U-shaped network first undergoes a max-pooling operation, expressed as:

[0038] x pooled =MaxPool2D(x,kernel) size =2)

[0039] Where x represents the current input feature map, kernel size =2 indicates that the pooling window size is 2×2;

[0040] Then, after a double convolution operation, it can be represented as:

[0041] y = DoubleConv(x) pooled )

[0042] The upsampling of the dual-branch U-shaped network first involves a transposed convolutional upsampling operation, expressed as:

[0043] x upsampled =ConvTranspose2D(x,W up )

[0044] Where x represents the current input feature map, W up Indicates the weights of the transposed convolution kernel;

[0045] Then, wavelet low-frequency feature extraction and interpolation are performed, represented as follows:

[0046] x wavelet =WaveletDecomposition(x skip ,target size =x upsampled .size[2:])

[0047] Where, x skip This represents the feature map used by the encoder for skip connections;

[0048] Then perform a jump connection operation, expressed by the formula:

[0049] x concat =Concat(x) upsampled ,x wavelet )

[0050] Finally, after the double convolution operation, the formula is:

[0051] y = DoubleConv(x) concat ).

[0052] Furthermore, the channel attention module weights the output features through a channel attention mechanism:

[0053] First, the channel context is generated through global pooling, and is represented as follows:

[0054] avg=GlobalAvgPool(x),max=GlobalMaxPool(x)

[0055] Then, the channel weights are generated, represented as:

[0056] out=σ(Conv2D(ReLU(Conv2D(avg+max))))

[0057] Where σ represents the Sigmoid activation function, which generates channel attention weights;

[0058] Finally, feature weighting is performed, represented as:

[0059] x att =x·out.

[0060] Furthermore, the mask fusion module fuses the masks of the contour and the central region, fusing the positional information provided by the central region mask with the contour information provided by the contour mask, as shown below:

[0061] x fused =x contour +x center

[0062] Where, x contour Represented as a contour region mask, x center This is represented as the central area mask.

[0063] Furthermore, the loss functions of the two branches of the dual-branch U-shaped network are the contour cross-entropy loss L1 and the central region cross-entropy loss L2, respectively, expressed as:

[0064]

[0065] Where N is the number of pixels, y i It's a real tag, p i It is a predicted probability;

[0066] Joint loss function L t It is a weighted combination of the contour cross-entropy loss L1 and the center region cross-entropy loss L2, expressed as:

[0067] L t =αL1+L2

[0068] Where α is the training weight.

[0069] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of an infrared small target detection method in a complex background as described above.

[0070] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an infrared small target detection method in a complex background as described above.

[0071] Compared to existing technologies, this invention and its preferred embodiment first employ morphological reconstruction techniques to preprocess the image, significantly enhancing the features of the target's center point. Then, annotation transformation is performed, and a mask reflecting the target's contour is constructed based on the annotated target bounding box. Simultaneously, a mask for the target's central region is constructed based on the target's center point position. Next, the preprocessed image and the original image are input into a dual-branch U-shaped network for training, jointly learning the features of the target contour and the central region. Finally, test images are selected for performance verification. The IoU of each test image is calculated, and the experimental results are statistically analyzed to determine the accuracy of infrared small target detection. Attached Figure Description

[0072] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0073] Figure 1 This is a flowchart illustrating the implementation of an embodiment of the present invention;

[0074] Figure 2 As an example of preprocessing in this embodiment of the invention, in the figure: (a) is the original image of the dataset, and (b) is the infrared image after image preprocessing;

[0075] Figure 3The following is an example of constructing a target mask according to an embodiment of the present invention. In the figure: (a) is the original image of the dataset, (b) is a mask that reflects the outline of the target based on the labeled target rectangle, (c) is the infrared image after image preprocessing, and (d) is a mask that reflects the central region of the target after preprocessing.

[0076] Figure 4 This is a diagram of a dual-branch U-shaped network structure according to an embodiment of the present invention. Detailed Implementation

[0077] To make the features and advantages of this patent more apparent and understandable, specific embodiments are provided below for detailed explanation:

[0078] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0079] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0080] This invention discloses an infrared small target detection method under complex backgrounds, mainly addressing the need for refined detection of infrared small targets under complex backgrounds and overcoming the problem of low detection performance and difficulty in accurately detecting target contours in existing methods when infrared small target features are not obvious. The implementation steps are as follows: (1) Image preprocessing: To address the problem of noise interference and unclear target features in the actual acquired images, a morphological adaptive reconstruction method is designed to preprocess the images, highlighting the target center features and improving the accuracy of subsequent target localization; (2) Labeling transformation: Based on the labeled target rectangle, a mask reflecting the target contour is constructed; at the same time, a mask for the target center region is constructed based on the target center position; (3) Design of a dual-branch U-shaped network, namely a target contour branch and a target center region branch. The dual-branch U-shaped network uses two independent U-shaped networks with the same architecture, inputting the original image and the preprocessed image simultaneously. The target contour branch takes the original image as input, learns the contour features of the target, and outputs the target contour prediction mask; the target center region branch takes the preprocessed image as input, learns the center region features, and outputs the target center region prediction mask. In each branch, wavelet decomposition is performed on the features output by the encoder during skip connections to extract low-frequency information. These low-frequency features are combined with the features of the corresponding scale of the decoder to enhance the target features during the decoding process. At the same time, a channel attention mechanism is introduced after the decoder output to assign adaptive weights to the decoded features, further enhancing the target feature expression ability. Finally, the target prediction mask is generated by combining the target contour and the center region prediction mask through the mask fusion module; (4) The test image is input into the trained network model for verification, the IoU of the test image is calculated, the experimental results are statistically analyzed, and the accuracy of infrared small target detection is calculated. Compared with the existing methods, this invention designs a morphological adaptive reconstruction method to preprocess the image, which can effectively remove noise interference and highlight the target center features; a dual-branch U-shaped network is designed, which learns the contour features and center features of the target through independent contour branches and center region branches, respectively, capturing both the global information of the target and focusing on the fine features of the center region. Contour features help enhance the ability to distinguish between the target and the background, while center region features focus on the position information of the target and improve the accuracy of target localization. By designing a joint loss function to jointly learn two features, efficient detection of small infrared targets can be achieved in complex environments.

[0081] Reference Figure 1 The steps for implementing this invention are as follows:

[0082] Step 1, Image Preprocessing

[0083] Infrared images often contain numerous black noise pixels, which can interfere with target detection. Morphological preprocessing aims to filter out this noise. However, using structuring elements of a single shape and scale may incorrectly filter out meaningful information needed for image segmentation, leading to the loss of some morphological and spatial features and hindering high segmentation accuracy. To address this issue, this embodiment designs a multi-scale, multi-structure morphological reconstruction method. It constructs adaptive morphological operations, performing morphological reconstruction using structuring elements of different scales and shapes. The image with the best reconstruction result is selected, eliminating the influence of potential noise pixels. The preprocessed image exhibits more prominent center point features, improving the accuracy of subsequent target localization. Figure 2 As shown.

[0084] Let two grayscale images f and g represent the marker image as the starting point of the transformation and the template image as the constraint condition, respectively. The erosion operation reconstruction obtained from structuring elements at different scales is represented as follows:

[0085]

[0086] in, S represents the morphological erosion operation. i This represents structuring elements of different scales, where the scale of the structuring element is i (1≤i≤n, i∈N). + ), ∨ represents taking the maximum value for each pixel in both images.

[0087] Accordingly, the dilation operation reconstructions obtained from structuring elements of different scales are as follows:

[0088]

[0089] in, S represents the morphological erosion operation. i This represents structuring elements of different scales, where the scale of the structuring element is i (1≤i≤n, i∈N). + ), ∧ represents the minimum value taken for each pixel of the two images.

[0090] In this invention, the infrared image is used as the template image G to perform morphological reconstruction on the labeled image F, as follows:

[0091]

[0092] In the formula, i,j (1≤i,j≤n,i,j∈N) + ) represents the size of the structuring element. This indicates a reconstruction of the expansion operation. This indicates a reconstruction based on the erosion operation.

[0093] As a preferred embodiment, morphological operations are performed using structural elements of five different scales and three different shapes (square, circle, and rhombus). To obtain the optimal reconstruction result, a maximum value synthesis method is used to synthesize the results of multi-scale and multi-structure operations. Therefore, the final adaptive processing result can be expressed as:

[0094]

[0095] in, It is an adaptive morphological reconstruction operator that increases with the scale of the structuring element. i and j are the scale of the structuring element, m and n are the start and end points of the structuring element scale selection, respectively, and k is the selection of the structuring element shape. For example, in the experimental code, k=1 can be set as a square structuring element, k=2 as a circular structuring element, k=3 as a rhombus structuring element, m=1, and n=5.

[0096] Step 2, label and convert

[0097] The YOLO-formatted annotation file contains the target's class ID, center coordinates (x_center, y_center), and width and height (width, height). These coordinates and dimensions are normalized values ​​relative to the image's width and height. These normalized annotation values ​​are converted to actual values ​​in the image coordinate system to reflect the actual target location and size. Next, a mask is generated on the image for each target region. Pixels within the target region are assigned a value of 1 to explicitly identify the target's presence, while pixels in the background region are uniformly set to 0 to distinguish the target from its surroundings.

[0098] The mask for the target contour is generated directly based on the precise position of the bounding box in the original infrared image, ensuring the model can learn the target contour features. Furthermore, to further enhance the model's sensitivity to the target center point, a mask for the target center region is constructed, such as... Figure 3 As shown.

[0099] Step 3, Design a dual-branch U-shaped network

[0100] This embodiment designs a dual-branch U-shaped network consisting of two independent U-shaped networks, used for predicting the target contour and the target center region, respectively. Both branches have the same architecture, with input being grayscale infrared images of the same size. The target contour branch uses the original infrared image as input, learns the features of the target contour region, and generates a target contour prediction mask. The target center region branch uses the preprocessed infrared image as input, learns the features of the target center region, and generates a center region prediction mask. To enhance network performance, a wavelet decomposition module is designed. At each skip connection, wavelet decomposition is performed on the encoder features to extract low-frequency information, which is then combined with features of the corresponding scale in the decoder. After combination, the features are further processed by a dual convolution module to extract local contextual information. Furthermore, each branch uses a channel attention module to assign adaptive weights to features from different channels, enhancing target features and suppressing irrelevant information. After completing the predictions of both branches, a mask fusion module fuses the target contour mask and the target center region mask to form a more refined target prediction feature. Finally, the output layer generates a complete target prediction mask. The following is a detailed description of the network modules, such as... Figure 4 As shown:

[0101] (a) Wavelet Decomposition Module

[0102] In small target detection, wavelet decomposition, by extracting low-frequency information from the input features, can better preserve the global information and main features of the target, while reducing the impact of noise interference and irrelevant information. Combining low-frequency features during the decoding stage supplements the global information that may be lost during the encoding stage, improving the integrity of small target features.

[0103] First, after wavelet decomposition, the formula is expressed as:

[0104] cA,(cH,cV,cD)=DWT2(x,wavelet)

[0105] Where x represents the current input feature map, cA represents the extracted low-frequency component used to represent the main target features, cH, cV, cD represent the extracted high-frequency components containing noise or irrelevant information, and DWT2 represents the two-dimensional discrete wavelet transform.

[0106] To match the upsampled decoded feature map, an interpolation operation is required, expressed by the formula:

[0107] cA resized =Interpolate(cA,target) size )

[0108] Among them, target sizecA represents the interpolated target size, matched with the upsampled decoder feature map. resized This represents the low-frequency feature map after interpolation, used for skip connections.

[0109] (b) DoubleConvolution Module

[0110] By performing two consecutive 3×3 convolutions, the receptive field is expanded, local features are extracted, and the network's ability to capture details of small targets is enhanced. At the same time, Batch Normalization (BN) and ReLU activation are used to improve training stability and the model's nonlinear expressive ability.

[0111] The formula for the first layer of convolution is expressed as:

[0112] y1 = ReLU(BN(Conv2D(x,W1)))

[0113] Where x represents the input feature map, W represents the convolution kernel weights, and y1 represents the intermediate features after the first convolution.

[0114] The formula for the second layer of convolution is expressed as follows:

[0115] y2 = ReLU(BN(Conv2D(y1,W2)))

[0116] Here, W2 represents the weight of the second convolutional kernel.

[0117] (c) Downsampling module

[0118] Feature map downsampling is achieved through max pooling, and the receptive field is expanded through double convolution to extract deep features, enabling the model to capture the difference information between the target and the background.

[0119] First, perform max pooling, as shown in the formula:

[0120] x pooled =MaxPool2D(x,kernel) size =2)

[0121] Where x represents the current input feature map, kernel size =2 indicates that the pooling window size is 2×2.

[0122] Then, after the double convolution operation, the formula is expressed as:

[0123] y = DoubleConv(x) pooled )

[0124] (d) Upsampling module

[0125] The resolution of the feature map is restored, allowing the model to gradually refine the positional information of small targets; the low-frequency information from wavelet decomposition enhances the ability to restore details and target shapes.

[0126] First, perform a transposed convolution upsampling operation, expressed by the formula:

[0127] x upsampled =ConvTranspose2D(x,W up )

[0128] Where x represents the current input feature map, W up This represents the weights of the transposed convolution kernel.

[0129] Then, wavelet low-frequency feature extraction and interpolation are performed, expressed by the following formula:

[0130] x wavelet =WaveletDecomposition(x skip ,target size =x upsampled .size[2:])

[0131] Where, x skip This represents the feature map used by the encoder for skip connections.

[0132] Then perform a jump connection operation, expressed by the formula:

[0133] x concat =Concat(x) upsampled ,x wavelet )

[0134] Finally, after the double convolution operation, the formula is expressed as:

[0135] y = DoubleConv(x) concat )

[0136] (e) Channel Attention Module

[0137] In the final stage of the decoder, the output features are weighted through a channel attention mechanism to highlight the features of the target region while suppressing background or irrelevant parts, thereby improving the accuracy of target detection.

[0138] First, channel contexts are generated through global pooling, as expressed by the formula:

[0139] avg=GlobalAvgPool(x),max=GlobalMaxPool(x)

[0140] Then, the channel weights are generated, expressed by the formula:

[0141] out=σ(Conv2D(ReLU(Conv2D(avg+max))))

[0142] Where σ represents the Sigmoid activation function, which generates channel attention weights.

[0143] Finally, feature weighting is performed, expressed by the formula:

[0144] x att =x·out

[0145] (f) Fusion Module

[0146] The masking of the contour and the central region is fused. By combining the positional information provided by the central region mask with the contour information provided by the contour mask, the detection accuracy is improved. The formula is expressed as:

[0147] x fused =x contour +x center

[0148] Where, x contour Represented as a contour region mask, x center This is represented as the central area mask.

[0149] (g) Loss Function

[0150] The loss functions for the two branch networks are contour cross-entropy loss L1 and center region cross-entropy loss L2, respectively. Cross-entropy loss measures the pixel-level difference between the predicted mask and the true mask, and is expressed by the formula:

[0151]

[0152] Where N is the number of pixels, y i It's a real tag, p i It is a predicted probability.

[0153] Joint loss function L t It is a weighted combination of the contour cross-entropy loss L1 and the center region cross-entropy loss L2, expressed by the formula:

[0154] L t =αL1+L2

[0155] Where α is the training weight, and the preferred setting for the test experiment is α = 2.

[0156] Step 4, Model Testing

[0157] One hundred infrared images were selected for testing to obtain target prediction results. Specifically, a mask of the same size as the target outline was generated on the original image based on the central region mask. A rectangular bounding box was then enclosed on the target prediction result to represent the target detection result. The IoU of the target detection was calculated, and the experimental results were statistically analyzed to calculate the accuracy of infrared small target detection.

[0158] The effects of this invention can be further illustrated by the following simulation experiments.

[0159] 1. Simulation conditions

[0160] This invention is based on experiments conducted using PyCharm software on an Intel(R) Xeon(R) Gold 5220R CPU@2.20GHz, 224G of memory, and Ubuntu 20.04 operating system.

[0161] The image database used in the experiment was an infrared image dataset with complex backgrounds. The infrared images were 256×256 grayscale images. The dataset was divided into training and testing sets, each containing 100 different infrared images and corresponding annotation files in YOLO format. The batch size was 4, the epochs were 600, the optimizer was Adam, and the initial learning rate was set to 10%. -4 .

[0162] 2. Simulation Content

[0163] The algorithm of this invention was tested on a dataset of infrared images of UAV targets under a complex background.

[0164] The metric is calculated as follows: Detection accuracy Q = Number of correctly detected samples / Total number of samples. For each test sample, if there is one and only one predicted bounding box that intersects with the true target bounding box, and the intersection-union ratio (IU) is greater than the threshold (0.5), it is recorded as "detection successful"; otherwise, it is recorded as "detection failed". The final detection accuracy on 100 infrared images is 72%.

[0165] To further verify the effectiveness of the proposed method, a comparison was made with the traditional U-shaped network. In the dataset used, the traditional U-shaped network achieved a detection accuracy of only 42% on 100 infrared images, while the method used in this invention achieved a 30% improvement, demonstrating superior performance.

[0166] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0168] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0169] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0170] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0171] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

[0172] This patent is not limited to the above-described preferred embodiment. Anyone can derive other forms of infrared small target detection methods in complex backgrounds based on the inspiration of this patent. All equivalent changes and modifications made within the scope of this patent application shall fall within the scope of this patent.

Claims

1. A method for detecting small infrared targets under complex backgrounds, characterized in that: Image preprocessing is performed using a morphological adaptive reconstruction method to highlight the central features of the target. Through annotation transformation, a mask reflecting the target contour is constructed based on the labeled target bounding box, and the center position of the target is calculated to construct the mask; using a two-branch U-shaped network, the contour features and center features of the target are learned through independent contour branches and center region branches respectively. The morphological adaptive reconstruction method performs morphological reconstruction using structural elements of different scales and shapes, and selects the part with the best reconstruction effect to integrate as the reconstruction result. The annotation process transforms the normalized annotation values ​​into actual values ​​in the image coordinate system, generating a segmentation mask in the corresponding target region, where the pixel value of the target region is 1 and the pixel value of the background region is 0. The dual-branch U-shaped network extracts multi-scale features by combining wavelet decomposition and skip connections, and introduces a channel attention mechanism to enhance the expression of key features. The mask fusion module fuses the prediction masks of the two branches to generate the target prediction mask as the final output. The dual-branch U-shaped network uses two independent U-shaped networks with the same architecture, simultaneously inputting original and preprocessed images of the same size. The dual branches include a target contour branch and a target center region branch. The target contour branch takes the original infrared image as input, learns the contour features of the target, and outputs a target contour prediction mask. The target center region branch takes the preprocessed infrared image as input, learns the features of the target center region, and outputs a target center region prediction mask. Within each branch, at each skip connection, the wavelet decomposition module performs wavelet decomposition on the features output by the encoder to extract low-frequency features. These low-frequency features are then combined with features at the corresponding scale of the decoder to enhance the target features during the decoding process. After feature combination, a dual convolution module further extracts local contextual information. After the decoder output, a channel attention module assigns adaptive weights to the features of different channels to further enhance the target feature representation capability. After the prediction of the two branches is completed, the target contour mask and the target center region mask are fused through the mask fusion module, and then the output layer generates a complete target prediction mask. The mask fusion module fuses the masks of the contour and the central region, and merges the position information provided by the central region mask with the contour information provided by the contour mask, represented as: x fused =x contour +x center Where, x contour Represented as a contour region mask, x center Represented as the central region mask; The loss functions of the two branches of the bi-branch U-shaped network are the contour cross-entropy loss L1 and the central region cross-entropy loss L2, respectively, expressed as: Where N is the number of pixels, y i It's a real tag, p i It is a predicted probability; Joint loss function L t It is a weighted combination of the contour cross-entropy loss L1 and the center region cross-entropy loss L2, expressed as: L t =αL1+L2 Where α is the training weight.

2. The infrared small target detection method under complex background according to claim 1, characterized in that: The morphological adaptive reconstruction method uses two grayscale images, f and g, to represent the marker image as the starting point of the transformation and the template image as the constraint condition, respectively; the erosion operation reconstruction obtained from structuring elements at different scales is represented as follows: in, S represents the morphological erosion operation. i This represents a structuring element with different scales, where the scale of the structuring element is i, 1≤i≤n, i∈N. + ∨ indicates that the maximum value is taken for each pixel in both images; The reconstruction of the dilation operation obtained from structuring elements of different scales is as follows: in, S represents the morphological erosion operation. i This represents a structuring element with different scales, where the scale of the structuring element is i, 1≤i≤n, i∈N. + , ∧ represents the minimum value taken pixel by pixel in both images; Using the infrared image as the template image G, morphological reconstruction of the labeled image F is performed, which is represented as: In the formula, i,j (1≤i,j≤n,i,j∈N) + ) represents the size of the structuring element. This indicates a reconstruction of the expansion operation. This indicates a reconstruction based on the erosion operation; The results of multi-scale and multi-structure operations are synthesized using the maximum value synthesis method. The final adaptive processing result is expressed as follows: in, It is an adaptive morphological reconstruction operator that increases with the structuring element scale, where m and n are the start and end points of the structuring element scale selection, and k is the selection of the structuring element shape.

3. The infrared small target detection method under complex background according to claim 1, characterized in that: In the wavelet decomposition module: the wavelet decomposition operation is first performed. cA,(cH,cV,cD)=DWT2(x,wavelet) Where x represents the current input feature map, cA represents the extracted low-frequency component used to represent the main target features, cH, cV, cD represent the extracted high-frequency components containing noise or irrelevant information, and DWT2 represents the two-dimensional discrete wavelet transform. To match the upsampled decoded feature map, an interpolation operation is performed: cA resized =Interpolate(cA,target size ) Among them, target size cA represents the interpolated target size, matched with the upsampled decoder feature map. resized This represents the low-frequency feature map after interpolation, used for skip connections.

4. The infrared small target detection method under complex background according to claim 1, characterized in that: In the dual convolution module: two consecutive 3×3 convolutions are performed, followed by BN and ReLU activation. The first convolutional layer is represented as: y1 = ReLU(BN(Conv2D(x,W1))) Where x represents the input feature map, W represents the convolution kernel weights, and y1 represents the intermediate features after the first convolution; The second convolutional layer is represented as: y2 = ReLU(BN(Conv2D(y1,W2))) Here, W2 represents the weight of the second convolutional kernel.

5. The infrared small target detection method under complex background according to claim 1, characterized in that: The downsampling of the dual-branch U-shaped network first undergoes a max pooling operation, expressed as: x pooled =MaxPool2D(x,kernel size =2) Where x represents the current input feature map, kernel size =2 indicates that the pooling window size is 2×2; Then, after a double convolution operation, it can be represented as: y=DoubleConv(x pooled ) The upsampling of the dual-branch U-shaped network first involves a transposed convolutional upsampling operation, expressed as: x upsampled =ConvTranspose2D(x,W up ) Where x represents the current input feature map, W up Indicates the weights of the transposed convolution kernel; Then, wavelet low-frequency feature extraction and interpolation are performed, represented as follows: x wavelet =WaveletDecomposition(x skip ,target size =x upsampled .size[2:]) Where, x skip This represents the feature map used by the encoder for skip connections; Then perform a jump connection operation, expressed by the formula: x concat =Concat(x upsampled ,x wavelet ) Finally, after the double convolution operation, the formula is: y=DoubleConv(x concat )。 6. The infrared small target detection method under complex background according to claim 1, characterized in that: The channel attention module weights the output features through a channel attention mechanism: First, the channel context is generated through global pooling, and is represented as follows: avg=GlobalAvgPool(x),max=GlobalMaxPool(x) Then, the channel weights are generated, represented as: out=σ(Conv2D(ReLU(Conv2D(avg+max)))) Where σ represents the Sigmoid activation function, which generates channel attention weights; Finally, feature weighting is performed, represented as: x att =x·out。 7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the infrared small target detection method under complex background as described in any one of claims 1-6.

Images