Infrared small target detection method under complex background

Abstract

The invention provides an infrared small target detection method under a complex background, and the method comprises the steps: carrying out the image preprocessing through a morphological adaptive reconstruction method, so as to highlight the center features of a target; through label conversion, according to the labeled target rectangular frame, constructing a mask which reflects the contour of the target, and calculating the center position of the target to construct the mask; respectively learning contour features and center features of a target through independent contour branches and center area branches by using a double-branch U-shaped network; according to the morphological self-adaptive reconstruction method, morphological reconstruction is carried out through structural elements with different scales and different shapes, and parts with the best reconstruction effect are selected and integrated together to serve as a reconstruction result; the dual-branch U-shaped network extracts multi-scale features by combining wavelet decomposition and jump connection, and introduces a channel attention mechanism to enhance key feature expression; and fusing the prediction masks of the two branches through a mask fusion module to generate a target prediction mask as final output.

Description

Technical Field

[0001] The invention belongs to the technical field of infrared imaging detection, and in particular relates to a method for detecting small infrared targets under complex backgrounds. Background Art

[0002] Infrared imaging detection has the advantages of good concealment, anti-interference and all-weather operation, and is widely used in early warning and air defense. Since small targets are often submerged in complex backgrounds, they will not only be missed due to weak signals, low resolution, lack of information such as shape, color and texture, but also generate a large number of clutter false alarms. The small target detection capability and scene adaptability need to be improved urgently. Traditional infrared small target detection methods mainly use the intensity difference between the target and the background to enhance the target signal-to-noise ratio through background suppression, and achieve good performance in uniform backgrounds; the human visual system-based method detects the target by enhancing the contrast between the small target and the background, and has low efficiency in utilizing the fluctuating complex background features; the low-rank sparse matrix models the background as a low-rank signal, from which the target image is separated, and part of the background will be misjudged as the target and produce false alarms. However, the existing methods mainly extract shallow features of the target and background, and cannot effectively mine deep features, thus having poor adaptability to infrared small target detection in complex scenes. Summary of the invention

[0003] In view of the shortcomings of the above-mentioned existing methods, the present invention proposes a method for detecting infrared small targets under complex backgrounds based on the characteristics that the central area features of infrared small targets are obvious, but the contour features are not obvious. First, in view of the problems of noise interference and unclear target features in the actual collected images, a morphological adaptive reconstruction method is designed to perform image preprocessing to highlight the central features of the target; annotation conversion is performed, and a mask reflecting the target contour is constructed according to the annotated target rectangular frame, and the center position of the target is calculated to construct a mask; a dual-branch U-shaped network is designed, and the contour features and center features of the target are learned respectively through independent contour branches and center area branches, which not only captures the global morphology of the target, but also pays attention to the fine features of the center area. The network extracts multi-scale features by combining wavelet decomposition with jump connections, and introduces a channel attention mechanism to enhance the expression of key features. Through the mask fusion module, the predicted masks of the two branches are fused to generate a target prediction mask as the final output, which improves the detection accuracy and anti-interference ability of small targets under complex backgrounds. Its main designs include:

[0004] (1) Image preprocessing. In order to solve the problem of noise interference in the actual collected images, a multi-scale and multi-structure morphological reconstruction method is designed. An adaptive morphological operation is constructed to perform morphological reconstruction through structural elements of different scales and shapes. The parts with the best reconstruction effect are selected and integrated as the reconstruction result to eliminate the influence of potential noise pixels in the image. The center point features of the preprocessed image are more obvious, which improves the accuracy of subsequent target positioning.

[0005] (2) Annotation transformation: convert the normalized annotation value into the actual value in the image coordinate system, and generate a segmentation mask in the corresponding target area, where the pixel value of the target area is 1 and the pixel value of the background area is 0. Based on the annotated target rectangle, a mask reflecting the target outline is constructed, and at the same time, a mask of the target center area is constructed based on the position of the target center point.

[0006] (3) Design a dual-branch U-shaped network, which is a target contour branch and a target center region branch. The dual-branch U-shaped network uses two independent U-shaped networks with the same architecture, which input the original image and the preprocessed image at the same time. The target contour branch takes the original image as input, learns the contour features of the target, and outputs the target contour prediction mask; the target center region branch takes the preprocessed image as input, learns the center region features, and outputs the target center region prediction mask. When each branch jumps, the features output by the encoder are wavelet decomposed to extract low-frequency information. These low-frequency features are combined with the features of the corresponding scale of the decoder to enhance the target features in the decoding process. At the same time, the channel attention mechanism is introduced after the decoder output to give adaptive weights to the decoded features to further enhance the target feature expression ability. Finally, the target contour and the center region prediction mask are combined through the mask fusion module to generate the final target prediction mask.

[0007] (4) Performance test: Input the test image into the trained network model to verify the output mask, add a rectangular box as the result of target detection, calculate the IoU of the test image, statistically analyze the experimental results, and calculate the accuracy of infrared small target detection.

[0008] The technical solution specifically adopted by the present invention to solve the technical problem is:

[0009] A method for detecting small infrared targets under complex backgrounds: image preprocessing is performed through a morphological adaptive reconstruction method to highlight the target center feature; a mask reflecting the target outline is constructed based on the annotated target rectangular frame through annotation conversion, and the target center position is calculated to construct a mask; a dual-branch U-shaped network is used to learn the target outline feature and center feature respectively through independent outline branches and center area branches;

[0010] The morphological adaptive reconstruction method performs morphological reconstruction through structural elements of different scales and shapes, and selects the parts with the best reconstruction effect and integrates them together as the reconstruction result;

[0011] The annotation conversion is to convert the normalized annotation value into the actual value in the image coordinate system, and generate a segmentation mask in the corresponding target area, wherein the pixel value of the target area is 1 and the pixel value of the background area is 0;

[0012] The dual-branch U-shaped network extracts multi-scale features by combining wavelet decomposition with skip connections, and introduces a channel attention mechanism to enhance the expression of key features; through the mask fusion module, the prediction masks of the two branches are fused to generate a target prediction mask as the final output.

[0013] Furthermore, the dual-branch U-type network uses two independent U-type networks, which have the same architecture and simultaneously input the original image and the preprocessed image of the same size; the dual branches include a target contour branch and a target center area branch; the target contour branch takes the original infrared image as input, learns the contour features of the target, and outputs a target contour prediction mask; the target center area branch takes the preprocessed infrared image as input, learns the features of the target center area, and outputs a target center area prediction mask; in the branch, each time a jump connection is made, the features output by the encoder are wavelet decomposed by a wavelet decomposition module to extract low-frequency features, and the low-frequency features are then combined with the features of the corresponding scale of the decoder to enhance the target features in the decoding process, and after the features are combined, the local context information is further extracted by a dual convolution module; after the decoder output, the features of different channels are given adaptive weights by a channel attention module to further enhance the target feature expression capability; after completing the prediction of the two branches, the target contour mask and the target center area mask are fused by a mask fusion module, and then a complete target prediction mask is generated by the output layer.

[0014] Furthermore, the morphological adaptive reconstruction method assumes that two grayscale images f and g represent the marker image as the starting point of the transformation and the template image as the constraint condition respectively; the erosion operation reconstruction obtained by the structural elements of different scales is expressed as:

[0015]

[0016] in, represents the morphological corrosion operation, S i Represents structural elements of different scales, the scale of the structural element is i, 1≤i≤n, i∈N + , ∨ means taking the maximum value of each pixel of the two images;

[0017] The dilation operation reconstruction obtained by structural elements of different scales is expressed as:

[0018]

[0019] in, represents the morphological corrosion operation, S i Represents structural elements of different scales, the scale of the structural element is i, 1≤i≤n, i∈N + , ∧ means taking the minimum value of the two images pixel by pixel;

[0020] Taking the infrared image as the template image G, the marked image F is morphologically reconstructed and closed, which can be expressed as:

[0021]

[0022] In the formula, i,j(1≤i,j≤n,i,j∈N + ) represents the size of the structural element, represents the dilation operation reconstruction, Represents the reconstruction of the erosion operation;

[0023] The maximum value synthesis method is used to synthesize the multi-scale and multi-structure operation results, and the final adaptive processing result is expressed as:

[0024]

[0025] in, It is an adaptive morphological reconstruction operator, which increases with the increase of the scale of the structure element. m and n are the start and end of the structure element scale selection, respectively, and k is the selection of the structure element shape.

[0026] Furthermore, in the wavelet decomposition module: firstly, a wavelet decomposition operation is performed:

[0027] cA,(cH,cV,cD)=DWT2(x,wavelet)

[0028] Among them, x represents the feature map of the current input, cA represents the extracted low-frequency component, which is used to represent the main target features, cH, cV, cD represent the extracted high-frequency components, which contain noise or irrelevant information, and DWT2 represents two-dimensional discrete wavelet transform;

[0029] In order to match the upsampled decoded feature map, an interpolation operation is performed:

[0030] cA resized =Interpolate(cA,target size )

[0031] Among them, target size represents the interpolated target size, matching the upsampled decoder feature map, cAresized Represents the interpolated low-frequency feature map, which is used for skip connection.

[0032] Furthermore, in the dual convolution module: through two consecutive 3×3 convolutions and activation by BN and ReLU: the first layer of convolution is expressed as:

[0033] y1 = ReLU(BN(Conv2D(x,W1)))

[0034] Among them, x represents the input feature map, W represents the convolution kernel weight, and y1 represents the intermediate feature after the first convolution; the second layer of convolution is expressed as:

[0035] y2=ReLU(BN(Conv2D(y1,W2)))

[0036] Among them, W2 represents the second convolution kernel weight.

[0037] Furthermore, the downsampling of the dual-branch U-type network first performs a maximum pooling operation, which is expressed as:

[0038] x pooled =MaxPool2D(x,kernel size =2)

[0039] Among them, x represents the current input feature map, kernel size =2 means the size of the pooling window is 2×2;

[0040] Then after double convolution operation, it is expressed as:

[0041] y = DoubleConv(x pooled )

[0042] The upsampling of the dual-branch U-type network first performs a transposed convolution upsampling operation, which is expressed as:

[0043] x upsampled =ConvTranspose2D(x,W up )

[0044] Among them, x represents the current input feature map, W up Represents the transposed convolution kernel weight;

[0045] Then the wavelet low-frequency feature extraction and interpolation operation are performed, which is expressed as:

[0046] x wavelet = WaveletDecomposition(x skip ,target size =x upsampled .size[2:])

[0047] Among them, x skip Represents the feature map used by the encoder for skip connections;

[0048] Then perform the skip connection operation, the formula is expressed as:

[0049] x concat =Concat(x upsampled ,x wavelet )

[0050] Finally, after double convolution operation, the formula is:

[0051] y = DoubleConv(x concat ).

[0052] Furthermore, the channel attention module weights the output features through the channel attention mechanism:

[0053] First, the channel context is generated through global pooling, which is expressed as:

[0054] avg=GlobalAvgPool(x),max=GlobalMaxPool(x)

[0055] Then, the channel weight is generated, which is expressed as:

[0056] out=σ(Conv2D(ReLU(Conv2D(avg+max))))

[0057] Among them, σ represents the Sigmoid activation function, which generates the channel attention weight;

[0058] Finally, feature weighting is performed, expressed as:

[0059] x att =x·out.

[0060] Furthermore, the mask fusion module fuses the masks of the contour and the center area, and fuses the position information provided by the center area mask with the contour information provided by the contour mask, which is expressed as:

[0061] x fused =x contour +x center

[0062] Among them, x contour Represented as a contour region mask, x center Represented as a center region mask.

[0063] Furthermore, the loss functions of the two branch networks of the dual-branch U-shaped network are the contour cross entropy loss L1 and the center area cross entropy loss L2, respectively, which are expressed as:

[0064]

[0065] Where N is the number of pixels, y i is the true label, p i is the predicted probability;

[0066] Joint loss function L t It is a weighted combination of the contour cross entropy loss L1 and the center area cross entropy loss L2, expressed as:

[0067] L t =αL1+L2

[0068] Among them, α is the training weight.

[0069] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a method for detecting small infrared targets in a complex background as described above are implemented.

[0070] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for detecting small infrared targets in a complex background as described above.

[0071] Compared with the prior art, the present invention and its preferred solution firstly use morphological reconstruction technology to preprocess the image to significantly enhance the features of the target center point, then perform annotation conversion, construct a mask reflecting the target outline based on the annotated target rectangular frame, and construct a mask of the target center area based on the target center point position, then input the preprocessed image and the original image into a dual-branch U-shaped network for training, and jointly learn the target outline and center area features. Finally, select test images for performance verification, calculate the IoU of each test image, and statistically analyze the experimental results to calculate the accuracy of infrared small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0073] Figure 1 is a flowchart of an implementation of an embodiment of the present invention;

[0074] Figure 2 This is a preprocessing example of an embodiment of the present invention, in which: (a) is the original image of the data set, and (b) is the infrared image after image preprocessing;

[0075] Figure 3This is an example of constructing a target mask according to an embodiment of the present invention. In the figure: (a) is the original image of the data set, (b) is a mask reflecting the target outline based on the marked target rectangular frame, (c) is an infrared image after image preprocessing, and (d) is a mask reflecting the target center area after preprocessing;

[0076] Figure 4 This is a diagram of a dual-branch U-shaped network structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail as follows:

[0078] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0079] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0080] The present invention discloses a method for detecting infrared small targets under complex backgrounds, which mainly solves the demand for refined detection of infrared small targets under complex backgrounds, and overcomes the problem that the existing method has low detection performance and is difficult to accurately detect the target contour when the features of the infrared small targets are not obvious. The implementation steps are as follows: (1) image preprocessing, in view of the problems of noise interference and unclear target features in the actually collected images, a morphological adaptive reconstruction method is designed to preprocess the image, highlight the target center features, and improve the accuracy of subsequent target positioning; (2) annotation conversion, according to the annotated target rectangular frame, a mask reflecting the target contour is constructed; at the same time, a mask of the target center area is constructed according to the target center position; (3) a dual-branch U-type network is designed, which is a target contour branch and a target center area branch. The dual-branch U-type network uses two independent U-type networks with the same architecture, and inputs the original image and the preprocessed image at the same time. The target contour branch takes the original image as input, learns the contour features of the target, and outputs the target contour prediction mask; the target center area branch takes the preprocessed image as input, learns the center area features, and outputs the target center area prediction mask. In each branch, the features output by the encoder are subjected to wavelet decomposition to extract low-frequency information during the jump connection. These low-frequency features are combined with the features of the corresponding scale of the decoder to enhance the target features in the decoding process. At the same time, a channel attention mechanism is introduced after the decoder output to give adaptive weights to the decoded features, further enhancing the target feature expression capability. Finally, the target contour and the center area prediction mask are combined through the mask fusion module to generate the final target prediction mask; (4) The test image is input into the trained network model for verification, the IoU of the test image is calculated, the experimental results are statistically analyzed, and the infrared small target detection accuracy is calculated. Compared with the existing methods, the present invention designs a morphological adaptive reconstruction method to preprocess the image, which can effectively remove noise interference and highlight the target center features; a dual-branch U-shaped network is designed to learn the contour features and center features of the target through independent contour branches and center area branches, respectively, which not only captures the global information of the target, but also pays attention to the fine features of the center area. The contour features help to enhance the ability to distinguish between the target and the background, while the center area features focus on the location information of the target and improve the accuracy of target positioning. By designing a joint loss function, the two features are jointly learned, thereby achieving efficient detection of small infrared targets in complex environments.

[0081] Reference Figure 1 , the steps implemented by the present invention are as follows:

[0082] Step 1: Image preprocessing

[0083] There are many black noise points on infrared images, which may interfere with target detection. The purpose of image preprocessing using morphological operations is to filter out black noise points. Using structural elements of a single shape and scale may incorrectly filter out meaningful information required for image segmentation, resulting in the loss of some morphological and spatial features of the image, making it difficult to obtain high segmentation accuracy. In order to solve this problem, this embodiment designs a multi-scale and multi-structure morphological reconstruction method, constructs an adaptive morphological operation, performs morphological reconstruction through structural elements of different scales and shapes, selects the image with the best reconstruction effect as the reconstruction result, eliminates the influence of potential noise pixels in the image, and makes the center point features of the preprocessed image more obvious, thereby improving the accuracy of subsequent target positioning. Figure 2 shown.

[0084] Assume that two grayscale images f and g represent the marker image as the starting point of the transformation and the template image as the constraint condition respectively. The erosion operation reconstruction obtained by structural elements of different scales is expressed as:

[0085]

[0086] in, represents the morphological corrosion operation, S i Represents structural elements of different scales, and the scale of the structural element is i (1≤i≤n, i∈N + ), ∨ means taking the maximum value of the two images pixel by pixel.

[0087] Correspondingly, the dilation operation reconstruction obtained by structural elements of different scales is expressed as:

[0088]

[0089] in, represents the morphological corrosion operation, S i Represents structural elements of different scales, and the scale of the structural element is i (1≤i≤n, i∈N + ), ∧ means taking the minimum value of the two images pixel by pixel.

[0090] In the present invention, the infrared image is used as the template image G, and the marker image F is morphologically reconstructed and closed, which is expressed as:

[0091]

[0092] In the formula, i,j(1≤i,j≤n,i,j∈N + ) represents the size of the structural element, represents the dilation operation reconstruction, Represents the reconstruction of the erosion operation.

[0093] As a preferred embodiment, morphological operations are performed using structural elements of five scales and three different shapes (square, circle, diamond). In order to obtain the best reconstruction result, the maximum synthesis method is used to synthesize the multi-scale and multi-structure operation results. Therefore, the final adaptive processing result can be expressed as:

[0094]

[0095] in, It is an adaptive morphological reconstruction operator, which increases with the increase of the scale of the structure element. i, j are the scales of the structure element, m and n are the start and end of the structure element scale selection, and k is the shape of the structure element. For example, the experimental code can set k=1 for a square structure element, k=2 for a circular structure element, k=3 for a diamond structure element, m=1, n=5.

[0096] Step 2: Mark the conversion

[0097] The annotation file in YOLO format contains the target's category ID, center coordinates (x_center, y_center), and width and height (width, height). These coordinates and dimensions are normalized values ​​relative to the image width and height. These normalized annotation values ​​are converted to actual values ​​in the image coordinate system to reflect the actual target position and size. Next, a mask is generated for each target area on the image, where the pixels in the target area are assigned a value of 1 to clearly identify the existence of the target, while the pixels in the background area are uniformly set to 0 to distinguish the target from the surrounding environment.

[0098] The mask of the target contour is generated directly based on the precise position of the annotation box on the original infrared image, ensuring that the model can learn the target contour features. In addition, in order to further improve the model's sensitivity to the target center point, a mask of the target center area is constructed based on the target center area, such as Figure 3 shown.

[0099] Step 3: Design a dual-branch U-shaped network

[0100] The dual-branch U-shaped network designed in this embodiment consists of two independent U-shaped networks, which are used for the prediction of the target contour and the target center area respectively. The two-branch network has the same architecture, and the input is a grayscale infrared image of the same size. The target contour branch takes the original infrared image as input, learns the features of the target contour area, and generates the target contour prediction mask. The target center area branch takes the preprocessed infrared image as input, learns the features of the target center area, and generates a center area prediction mask. In order to enhance the network performance, a wavelet decomposition module is designed. At each jump connection, the features of the encoder are wavelet decomposed to extract the low-frequency information of the features, and then combined with the features of the corresponding scale of the decoder. After the features are combined, the local context information is further extracted through a dual convolution module. In addition, each branch uses a channel attention module to assign adaptive weights to the features of different channels to enhance the target features and suppress irrelevant information. After completing the prediction of the two branches, the target contour mask and the target center area mask are fused through the mask fusion module to form a more refined target prediction feature. Finally, a complete target prediction mask is generated through the output layer. The following is a detailed introduction to the network modules, such as Figure 4 As shown:

[0101] (a) Wavelet Decomposition

[0102] In small target detection, wavelet decomposition can better preserve the global information and main features of the target by extracting low-frequency information of the input features, while reducing the influence of noise interference and irrelevant information. In the decoding stage, low-frequency features are combined to supplement the global information that may be lost in the encoding stage, thereby improving the integrity of small target features.

[0103] First, after wavelet decomposition operation, the formula is expressed as:

[0104] cA,(cH,cV,cD)=DWT2(x,wavelet)

[0105] Among them, x represents the feature map of the current input, cA represents the extracted low-frequency component, which is used to represent the main target features, cH, cV, cD represent the extracted high-frequency components, which contain noise or irrelevant information, and DWT2 represents two-dimensional discrete wavelet transform.

[0106] In order to match the upsampled decoded feature map, an interpolation operation is required, and the formula is expressed as:

[0107] cA resized =Interpolate(cA,target size )

[0108] Among them, target sizerepresents the interpolated target size, matching the upsampled decoder feature map, cA resized Represents the interpolated low-frequency feature map, which is used for skip connection.

[0109] (b) Double Convolution Module (DoubleConv)

[0110] Through two consecutive 3×3 convolutions, the receptive field is expanded, local features are extracted, and the network’s ability to capture small target details is enhanced. At the same time, BatchNormalization (BN) and ReLU activation are used to improve training stability and the nonlinear expression ability of the model.

[0111] The first layer of convolution formula is expressed as:

[0112] y1 = ReLU(BN(Conv2D(x,W1)))

[0113] Among them, x represents the input feature map, W represents the convolution kernel weight, and y1 represents the intermediate feature after the first convolution.

[0114] The second layer convolution formula is expressed as:

[0115] y2=ReLU(BN(Conv2D(y1,W2)))

[0116] Among them, W2 represents the second convolution kernel weight.

[0117] (c) Downsampling module (Down)

[0118] The feature map is down-sampled through maximum pooling, and the receptive field is expanded and deep features are extracted through double convolution operations, so that the model can capture the difference information between the target and the background.

[0119] First, perform the maximum pooling operation, the formula is expressed as:

[0120] x pooled =MaxPool2D(x,kernel size =2)

[0121] Among them, x represents the current input feature map, kernel size =2 means the size of the pooling window is 2×2.

[0122] Then, after double convolution operation, the formula is expressed as:

[0123] y = DoubleConv(x pooled )

[0124] (d) Upsampling module (Up)

[0125] Restore the resolution of the feature map so that the model can gradually refine the location information of small targets; enhance the ability to restore details and target shapes through low-frequency information decomposed by wavelet.

[0126] First, perform the transposed convolution upsampling operation, the formula is expressed as:

[0127] x upsampled =ConvTranspose2D(x,W up )

[0128] Among them, x represents the current input feature map, W up Represents the transposed convolution kernel weights.

[0129] Then, wavelet low-frequency feature extraction and interpolation operations are performed, and the formula is expressed as:

[0130] x wavelet = WaveletDecomposition(x skip ,target size =x upsampled .size[2:])

[0131] Among them, x skip Represents the feature map used by the encoder for skip connections.

[0132] Then perform the skip connection operation, the formula is expressed as:

[0133] x concat =Concat(x upsampled ,x wavelet )

[0134] Finally, after double convolution operation, the formula is expressed as:

[0135] y = DoubleConv(x concat )

[0136] (e) Channel Attention Module

[0137] In the final stage of the decoder, the output features are weighted through the channel attention mechanism to highlight the features of the target area while suppressing the background or irrelevant parts, thereby improving the accuracy of target detection.

[0138] First, the channel context is generated through global pooling, and the formula is expressed as:

[0139] avg=GlobalAvgPool(x),max=GlobalMaxPool(x)

[0140] Then, the channel weight is generated, and the formula is expressed as:

[0141] out=σ(Conv2D(ReLU(Conv2D(avg+max))))

[0142] Among them, σ represents the Sigmoid activation function, which generates the channel attention weight.

[0143] Finally, feature weighting is performed, and the formula is expressed as:

[0144] x att = x·out

[0145] (f) Mask Fusion Module (FusionModule)

[0146] The mask of the fusion contour and the center area is used to improve the detection accuracy by fusing the position information provided by the center area mask with the contour information provided by the contour mask. The formula is expressed as:

[0147] x fused =x contour +x center

[0148] Among them, x contour Represented as a contour area mask, x center Represented as a center region mask.

[0149] (g) Loss function

[0150] The loss functions of the two branch networks are the contour cross entropy loss L1 and the center area cross entropy loss L2. The cross entropy loss is used to measure the difference between the predicted mask and the true mask at the pixel level, and the formula is expressed as:

[0151]

[0152] Where N is the number of pixels, y i is the true label, p i is the predicted probability.

[0153] Joint loss function L t It is a weighted combination of the contour cross entropy loss L1 and the center area cross entropy loss L2, and the formula is expressed as:

[0154] L t =αL1+L2

[0155] Here, α is the training weight, and the preferred setting for the test experiment is α=2.

[0156] Step 4: Model testing

[0157] 100 infrared images were selected as tests to obtain the target prediction results. Specifically, a mask of the same size as the target outline was generated based on the central area mask on the original image. A rectangular box was added to the target prediction result as the target detection result, and the IoU of the target detection was calculated. The experimental results were statistically analyzed to calculate the accuracy of infrared small target detection.

[0158] The effect of the present invention can be further illustrated by the following simulation experiment.

[0159] 1. Simulation conditions

[0160] The present invention is an experiment conducted using Pycharm software on an Intel(R) Xeon(R) Gold 5220R CPU@2.20GHz, 224G memory, and Ubuntu20.04 operating system.

[0161] The image database used in the experiment is a dataset of infrared images under complex backgrounds. The infrared images are grayscale images of size 256×256. The dataset is divided into a training set and a test set. The training set and the test set contain 100 different infrared images and corresponding annotation files, which are annotated in YOLO format. The batch size is 4, the epoch is 600, the optimizer is the Adam optimizer, and the initial learning rate is set to 10 -4 .

[0162] 2. Simulation content

[0163] The experiment of the algorithm of the present invention is completed on a dataset of infrared images of UAV targets in a complex background.

[0164] Indicator calculation method: Detection accuracy Q = number of correctly detected samples / total number of samples. For each test sample, if there is only one predicted box and the true target box have an intersection, and it is greater than the intersection ratio threshold (0.5), it is recorded as "detection success", otherwise "detection failure". The final detection accuracy on 100 infrared images is 72%.

[0165] In order to further verify the effectiveness of the proposed method, a comparison was made with the traditional U-shaped network. In the data set used, the detection accuracy of the traditional U-shaped network on 100 infrared images was only 42%, while the method used in the present invention increased it by 30%, showing excellent performance.

[0166] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0167] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0170] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0171] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

[0172] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive other various forms of infrared small target detection methods under complex backgrounds under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should be covered by this patent.

Claims

1. A method for detecting small infrared targets under complex backgrounds, characterized by: The image is preprocessed by morphological adaptive reconstruction method to highlight the central features of the target; Through annotation conversion, a mask reflecting the target contour is constructed based on the annotated target rectangular frame, and the center position of the target is calculated to construct the mask; using a dual-branch U-shaped network, the contour features and center features of the target are learned respectively through independent contour branches and center area branches; The morphological adaptive reconstruction method performs morphological reconstruction through structural elements of different scales and shapes, and selects the parts with the best reconstruction effect and integrates them together as the reconstruction result; The annotation conversion is to convert the normalized annotation value into the actual value in the image coordinate system, and generate a segmentation mask in the corresponding target area, wherein the pixel value of the target area is 1 and the pixel value of the background area is 0; The dual-branch U-shaped network extracts multi-scale features by combining wavelet decomposition with skip connections, and introduces a channel attention mechanism to enhance the expression of key features; Through the mask fusion module, the prediction masks of the two branches are fused to generate the target prediction mask as the final output.

2. The infrared small target detection method based on a complex background according to claim 1 is characterized in that: The dual-branch U-shaped network uses two independent U-shaped networks, which have the same architecture and input the original image and the preprocessed image of the same size at the same time; the dual branches include a target contour branch and a target center area branch; the target contour branch takes the original infrared image as input, learns the contour features of the target, and outputs the target contour prediction mask; the target center area branch takes the preprocessed infrared image as input, learns the features of the target center area, and outputs the target center area prediction mask; in the branch, at each jump connection, the features output by the encoder are wavelet decomposed by a wavelet decomposition module to extract low-frequency features, and then the low-frequency features are combined with the features of the corresponding scale of the decoder to enhance the target features in the decoding process, and the local context information is further extracted by a dual convolution module after the features are combined; after the decoder output, the features of different channels are given adaptive weights through a channel attention module to further enhance the target feature expression capability; After completing the prediction of the two branches, the target contour mask and the target center area mask are fused through the mask fusion module, and then the complete target prediction mask is generated through the output layer.

3. The method for detecting small infrared targets under complex background according to claim 1, characterized in that: The morphological adaptive reconstruction method assumes that two grayscale images f and g represent the marker image as the starting point of the transformation and the template image as the constraint condition respectively; the erosion operation reconstruction obtained by the structural elements of different scales is expressed as: in, represents the morphological corrosion operation, S i Represents structural elements of different scales, the scale of the structural element is i, 1≤i≤n,i∈N + , ∨ means taking the maximum value of the two images pixel by pixel; The dilation operation reconstruction obtained by structural elements of different scales is expressed as: in, represents the morphological corrosion operation, S i Represents structural elements of different scales, the scale of the structural element is i, 1≤i≤n, i∈N + , ∧ means taking the minimum value of the two images pixel by pixel; Taking the infrared image as the template image G, the marked image F is morphologically reconstructed and closed, which can be expressed as: In the formula, i,j(1≤i,j≤n,i,j∈N + ) represents the size of the structural element, represents the dilation operation reconstruction, Represents the reconstruction of the erosion operation; The maximum value synthesis method is used to synthesize the multi-scale and multi-structure operation results, and the final adaptive processing result is expressed as: in, It is an adaptive morphological reconstruction operator, which increases with the increase of the scale of the structure element. m and n are the start and end of the structure element scale selection, respectively, and k is the selection of the structure element shape.

4. The method for detecting small infrared targets under complex background according to claim 2, characterized in that: In the wavelet decomposition module: firstly, the wavelet decomposition operation is performed: cA,(cH,cV,cD)=DWT2(x,wavelet) Among them, x represents the feature map of the current input, cA represents the extracted low-frequency component, which is used to represent the main target features, cH, cV, cD represent the extracted high-frequency components, which contain noise or irrelevant information, and DWT2 represents two-dimensional discrete wavelet transform; In order to match the upsampled decoded feature map, an interpolation operation is performed: cA resized =Interpolate(cA,target size ) Among them, target size represents the interpolated target size, matching the upsampled decoder feature map, cA resized Represents the interpolated low-frequency feature map, which is used for skip connection.

5. The method for detecting small infrared targets under complex background according to claim 2, characterized in that: In the double convolution module: through two consecutive 3×3 convolutions, and through BN and ReLU activation: The first convolution layer is expressed as: y1 = ReLU(BN(Conv2D(x,W1))) Among them, x represents the input feature map, W represents the convolution kernel weight, and y1 represents the intermediate feature after the first convolution; The second convolution layer is expressed as: y2=ReLU(BN(Conv2D(y1,W2))) Among them, W2 represents the second convolution kernel weight.

6. The method for detecting small infrared targets under complex background according to claim 2, characterized in that: The downsampling of the dual-branch U-type network first performs a maximum pooling operation, which is expressed as: x pooled =MaxPool2D(x,kernel size =2) Among them, x represents the current input feature map, kernel size =2 means the size of the pooling window is 2×2; Then after double convolution operation, it is expressed as: y=DoubleConv(x pooled ) The upsampling of the dual-branch U-type network first performs a transposed convolution upsampling operation, which is expressed as: x upsampled =ConvTranspose2D(x,W up ) Among them, x represents the current input feature map, W up Represents the transposed convolution kernel weight; Then the wavelet low-frequency feature extraction and interpolation operation are performed, which is expressed as: x wavelet =WaveletDecomposition(x skip ,target size =x upsampled .size[2:]) Among them, x skip Represents the feature map used by the encoder for skip connections; Then perform the skip connection operation, the formula is expressed as: x concat =Concat(x upsampled ,x wavelet ) Finally, after double convolution operation, the formula is: y=DoubleConv(x concat )。 7. The method for detecting small infrared targets under complex background according to claim 2, characterized in that: The channel attention module weights the output features through the channel attention mechanism: First, the channel context is generated through global pooling, which is expressed as: avg=GlobalAvgPool(x),max=GlobalMaxPool(x) Then, the channel weight is generated, which is expressed as: out=σ(Conv2D(ReLU(Conv2D(avg+max)))) Among them, σ represents the Sigmoid activation function, which generates the channel attention weight; Finally, feature weighting is performed, expressed as: x att =x·out。 8. The method for detecting small infrared targets under complex backgrounds according to claim 2, characterized in that: The mask fusion module fuses the mask of the contour and the center area, and fuses the position information provided by the center area mask with the contour information provided by the contour mask, which is expressed as: x fused =x contour +x center Among them, x contour Represented as a contour region mask, x center Represented as a center region mask.

9. The method for detecting small infrared targets under complex background according to claim 2, characterized in that: The loss functions of the two branch networks of the dual-branch U-shaped network are the contour cross entropy loss L1 and the center area cross entropy loss L2, respectively, expressed as: Where N is the number of pixels, y i is the true label, p i is the predicted probability; Joint loss function L t It is a weighted combination of the contour cross entropy loss L1 and the center area cross entropy loss L2, expressed as: L t =αL1+L2 Among them, α is the training weight.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of a method for detecting small infrared targets under a complex background as described in any one of claims 1 to 9 are implemented.

Images