Infrared small target detection method in complex scene based on linear attention

Through the infrared small object detection method based on linear attention, using sequence noise modeling and negative sample enhancement, combined with feature extraction network and eight-connected neighborhood clustering algorithm, the noise misjudgment and noise detection of infrared small object detection in the existing technology is solved. Feature loss problem improves detection accuracy and generalization ability.

CN119006780AActive Publication Date: 2024-11-22NORTHEASTERN UNIV AT QINHUANGDAO

Patent Information

Application Number
CN202411021763.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-22
Estimated Expiration
2044-07-29

AI Technical Summary

Technical Problem

The existing infrared small-object detection methods lack global information perception capabilities and are prone to detect noise as targets. The deep network is prone to lose small-object characteristics, resulting in poor infrared small-object detection performance in complex backgrounds.

Method used

Using infrared small object detection method in complex scenarios based on linear attention, noise data sets are generated through serial noise modeling technology, data fusion and negative sample enhancement are carried out, and feature extraction networks are designed including convolution modules, linear attention modules and hollow volumes. The integrated module combines the eight-connected neighborhood clustering algorithm to improve feature extraction capabilities and detection accuracy.

Benefits of technology

Effectively suppress background noise interference, improve the accuracy and generalization ability of infrared small object detection, reduce the false detection rate, and enhance the object detection ability in complex backgrounds.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0004967481930000021
    Figure BDA0004967481930000021
  • Figure BDA0004967481930000052
    Figure BDA0004967481930000052
  • Figure BDA0004967481930000093
    Figure BDA0004967481930000093
Patent Text Reader

Abstract

The invention provides an infrared small target detection method in a complex scene based on linear attention. The method comprises the following steps: firstly, processing and dividing an original data set, and applying noise data to pictures in a training set by using a sequence noise modeling technology so as to obtain small target data in a complex scene and improve the detection generalization ability; carrying out negative sample enhancement on a small target area in the data set, and expanding the data set; secondly, a small target segmentation network is invented, a coding part obtains high-level semantic information and extracts features by reducing spatial dimensions, and a decoding stage recovers the spatial dimensions and reconstructs an original image structure; a linear attention module in the network aims to improve the feature extraction capability and reduce the space complexity at the same time; the cavity convolution module expands a receptive field and weakens small target information loss caused by too many down-sampling times; wherein high-resolution information is reserved by using jump connection, and information of different scales is supervised by using depth; finally, an eight-connected neighborhood clustering module is used for ensuring matching accuracy, and false detection is reduced; according to the method, target detection can be carried out on the infrared image under the complex background, interference of background noise is effectively suppressed, and the accuracy of infrared small target detection is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of infrared small target detection, and in particular to an infrared small target detection method in a complex scene based on linear attention. Background Art

[0002] With the vigorous development of science and technology, target detection has gradually developed into an important part of the field of computer vision. Target detection is an image segmentation based on the geometric and statistical characteristics of the target, and its purpose is to find the position and size of the target object in the image. In recent years, the aerospace field has developed rapidly, and timely and accurate detection of high-heat targets at a long distance is an important research to ensure the safety of people's lives and property. Among them, target detection in the field of infrared images can detect infrared objects in dim and harsh environments, improve target detection performance at night and in invisible weather, and capture the thermal radiation signals of target objects, thereby ensuring the continuity and effectiveness of detection activities, and therefore has received great attention.

[0003] Compared with ordinary visible light target detection, infrared small target detection faces the following difficulties: 1. The imaging distance of infrared small targets is relatively long, and the target pixels account for a very small proportion of the entire image pixels; 2. The background is complex and is interfered by many atmospheric clouds. The infrared radiation energy decays with distance, causing the target to be easily submerged in background clutter and sensor noise; 3. The image of infrared small target detection is very sparse, resulting in a serious imbalance between the target and background areas; therefore, when conducting relevant research on infrared small target detection, it is necessary to fully consider the impact of these unfavorable factors;

[0004] Existing infrared small target detection methods can be divided into four categories according to different algorithmic ideas: infrared small target detection methods based on filters, infrared small target detection methods based on contrast, infrared small target detection methods based on low-rank sparse models, and infrared small target detection methods based on deep learning. The first three methods often rely heavily on prior knowledge, are sensitive to hyperparameters, and perform poorly on complex backgrounds and noisy images; while the deep learning-based methods are limited by the local receptive field of convolution operations, lack the ability of global perception, and are prone to detecting noise in infrared images as targets. Since the small targets in the infrared small target detection data set are small, multiple downsamplings can easily cause target loss, and the process is irreversible;

[0005] From the above analysis, it can be seen that the existing methods lack the ability to perceive global information, easily detect noise as targets, and deep networks are prone to lose small target features, resulting in poor performance in infrared small target detection tasks. Summary of the invention

[0006] The purpose of the present invention is to provide a method for detecting small infrared targets in complex scenes based on linear attention, so as to solve the problems raised in the above-mentioned background technology.

[0007] A method for detecting infrared small targets in complex scenes based on linear attention, characterized by comprising the following steps:

[0008] Construction and preprocessing of data sets: obtaining infrared small target images and preprocessing them, including adjusting image size, image standardization, and adjusting annotation format, to obtain preprocessed infrared small target detection data;

[0009] The infrared noise dataset is generated using sequence noise modeling technology;

[0010] The images in the infrared small target dataset contain objects such as aircraft, airplanes, and floating objects on the sea surface. The scenes include towns, fields, sky, and sea surface. The images are classified and sorted according to the scenes.

[0011] Select no less than 800 images as noise sampling images, and give a noise sample U with a noise sampling area size of 32×32 i , i represents the i-th noise sampling area in a given noise sampling image, where i∈[1,64]; in order to ensure that the texture in the extracted noise sequence is as uniform as possible, the noise sampling area N is selected p for:

[0012]

[0013] Among them, Cut(·) is the cropping operation, Mean(·) and Var(·) are the mean and variance respectively, μ is the maximum value of the mean of all sampling areas in a given noise sampling image, and θ is the maximum value of the variance of all sampling areas in a given noise sampling image.

[0014] Fusing the generated infrared noise data with the original image;

[0015] Resize the noise sampling region to be equal to the size of the noise sampling image input to accommodate real-world noise:

[0016] N real =Resize(N p )

[0017] Resize(·) means adjusting the size of the noise-prone area to match the input image I input The size of the infrared sensor data is sampled with real-world noise; the input training image I input Replaced by:

[0018] I mix =βNreal +(1-β)I input

[0019] Among them I mix represents the training samples mixed with noise, and β∈[0,1] represents the hyperparameter of noise replacement.

[0020] Perform negative sample enhancement on the fused data;

[0021] 850 to 950 images are selected for negative sample enhancement, and the small target anchor area in the image is denoted as P target ∈[c,p,p]; where p represents the length and width of the small target anchor area, c represents the number of channels, and P target Perform center rotation to generate negative samples:

[0022] I R =Rot(P target ,γ),γ∈[0°,90°,180°,270°]

[0023] in, represents the samples after negative enhancement by rotating 0°, 90°, 180°, and 270° respectively, γ represents the rotation angle; Rot(·) represents the random center rotation operation.

[0024] Design and training of detection models;

[0025] Firstly, a feature extraction network is established, and high-level abstract features in infrared small target image data are extracted through the feature extraction network;

[0026] The feature extraction network is mainly composed of convolution module, linear attention module and dilated convolution module;

[0027] The input tensor passes through the convolution module 1 to obtain the feature tensor a;

[0028] The feature tensor a is pooled and then passed through the linear attention module 1 to obtain the feature tensor b.

[0029] The feature tensor b undergoes a pooling operation and then passes through the linear attention module 2 to obtain the feature tensor c;

[0030] The feature tensor c undergoes a pooling operation and then passes through a linear attention module 3 to obtain the feature tensor d;

[0031] The feature tensor d is pooled and then passes through the dilated convolution module 1 to obtain the feature tensor e.

[0032] The feature tensor e is upsampled to obtain the feature tensor D, which is concatenated with the feature tensor d and then passed through the dilated convolution module 2 to obtain the feature tensor f.

[0033] The feature tensor f is upsampled to obtain the feature tensor C; the feature tensor C is concatenated with the feature tensor c, and then passed through the convolution module 2 to obtain the feature tensor g;

[0034] The feature tensor g is upsampled to obtain the feature tensor B; the feature tensor B is concatenated with the feature tensor b, and then passed through the convolution module 3 to obtain the feature tensor h;

[0035] The feature tensor h is upsampled to obtain the feature tensor A; the feature tensor A is concatenated with the feature tensor a, and then passed through the convolution module 4 to obtain the feature tensor i;

[0036] The architecture of convolution module 1, convolution module 2, convolution module 3, and convolution module 4 is as follows: the input is output after passing through convolution layer 1, batch normalization, convolution layer 2, batch normalization, and activation function in sequence. The convolution layer uses a convolution kernel of size 3×3, and the activation function is the Relu function:

[0037] f(x)=max(0,x)

[0038] The architecture of linear attention module 1, linear attention module 2 and linear attention module 3 are as follows: first, the input is sequentially regularized, fully connected layer 1, depthwise separable convolution and two-dimensional selective scanning, and the regularized output feature tensor 1 is obtained; secondly, the input is regularized, fully connected layer 1 and fully connected layer 2 output feature tensor 2; then feature tensor 1 and feature tensor 2 are multiplied by the corresponding position elements to obtain feature tensor 3; feature tensor 3 is weighted and output with the input through fully connected layer 3; among them, the two-dimensional selective scanning consists of three parts: scanning expansion operation, selective scanning space state sequence block operation and scanning merging operation; the scanning expansion operation expands the input image into sequences along four different directions; then the sequences are feature extracted through the selective scanning space state sequence block, and the scanning merging operation sums and merges the sequences from the four directions to restore the output image to the same size as the input image; by adjusting the parameters of the dimension selective scanning module according to the input

[0039] The architecture of dilated convolution module 1 and dilated convolution module 2 is as follows: the input first passes through convolution layer 1, then enters convolution layer 2, and then enters convolution layer 3, and is concatenated with the output of convolution layer 2, and then input into convolution layer 4, and then concatenated with the output of convolution layer 1, and finally output through convolution layer 5; among them, convolution layers 1 and 5 are 3×3 convolution kernels, convolution layers 2 and 4 use convolution kernels with a dilation rate of 2 and a size of 3×3, and convolution layer 3 is a convolution kernel with a dilation rate of 4 and a size of 3×3;

[0040] The feature tensor i is output through convolution layer 1;

[0041] The feature tensor i is output through convolution layer 2;

[0042] The feature tensor h passes through the convolution layer 3 and then is upsampled by bilinear interpolation to obtain the output;

[0043] The feature tensor g passes through the convolution layer 4 and is then upsampled by bilinear interpolation to obtain the output;

[0044] The feature tensor f passes through the convolution layer 5 and is then upsampled by bilinear interpolation to obtain the output;

[0045] The feature tensor e passes through the convolution layer 6 and is then upsampled by bilinear interpolation to obtain the output;

[0046] Finally, the eight-connected neighborhood clustering algorithm is used to reduce false detections and output results. Pixels belonging to the same target are clustered together, and the centroid of each target is calculated. Areas with a centroid distance less than a threshold are matched and added to the target list, while those with a centroid distance greater than a threshold are not added, thereby reducing the number of predicted areas that have no real corresponding objects. The successfully matched areas are counted in the PD (Probability of detection), while the unmatched areas are counted in the FA (False rate).

[0047] Use the trained target detection model to detect small infrared targets.

[0048] The infrared small target detection dataset is divided into training set and test set in the ratio of 8:2. The image data in the training set is input into the neural network, the output is calculated through forward propagation, and then the gradient is calculated and the network parameters are updated through back propagation. The set rounds are iterated, and the training status of the current model is fed back through the validation set after each set round to prevent the model from overfitting during training. After the training, the test set is used to evaluate the performance indicators of the model.

[0049] The beneficial effects of adopting the above technical solution are:

[0050] The invention provides an infrared small target detection method in complex scenes based on linear attention. Firstly, the original data set is processed and divided, and noise data is applied to the pictures in the training set by using the sequence noise modeling technology to obtain the small target data in the complex scenes, so as to improve the detection generalization ability. Then, the small target area in the data set is enhanced by negative samples to expand the data set. Secondly, a small target segmentation network is invented, the encoding part obtains high-level semantic information and extracts features by reducing the spatial dimension, and the spatial dimension is restored in the decoding stage to reconstruct the original image structure. The linear attention module in the network aims to improve the feature extraction ability while reducing the spatial complexity. The atrous convolution module expands the receptive field and reduces the loss of small target information caused by too many downsampling times. Among them, the skip connection is used to retain the high-resolution information, and the information of different scales is supervised by depth. Finally, the eight-connected neighborhood clustering module is used to ensure accurate matching and reduce false detection. The invention can perform target detection on infrared images under complex backgrounds, effectively suppress the interference of background noise, and improve the accuracy of infrared small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is the overall flow chart of the example of the present invention;

[0052] Figure 2 It is the flow chart of S5 in the example of the present invention;

[0053] Figure 3 This is a diagram of the infrared small target detection network structure in an example of the present invention;

[0054] Figure 4 It is a structural diagram of the convolution module in an example of the present invention;

[0055] Figure 5 It is a structural diagram of the linear attention module in the example of the present invention;

[0056] Figure 6 This is a structural diagram of a dilated convolution module in an example of the present invention;

[0057] Figure 7 This is a diagram showing the detection and recognition results of an infrared small target in an example of the present invention;

[0058] Figure 8 This is a diagram showing the detection and recognition results of an infrared small target in an example of the present invention; DETAILED DESCRIPTION

[0059] The specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings and examples; the following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention;

[0060] A method for detecting small infrared targets in complex scenes based on linear attention, such as Figure 1 As shown, the following steps are included:

[0061] S1: Construction and preprocessing of the data set: obtaining infrared small target images and preprocessing them, including adjusting the image size, image standardization, and adjusting the annotation format, to obtain the preprocessed infrared small target detection data;

[0062] The images in the infrared small target dataset contain objects such as aircraft, airplanes, and floating objects on the sea. The scenes include towns, fields, sky, and sea. The images are classified and sorted according to the scenes.

[0063] S2: Generate infrared noise dataset using sequence noise modeling technology;

[0064] Negative samples are samples that do not belong to the target category and are used to enhance the discriminative ability of the target model; no less than 800 images are selected as noise sampling images I noi :

[0065]

[0066] Given a noise sample U with a noise sampling area size of 32×32 i , i represents the i-th noise sampling area in the noise sampling image, where i∈[1,64]; usually, the noise sampling area with large variance has a wide distribution of pixel values, and the light and dark parts of the image are significantly different, containing rich textures, which often cover the noise itself, and are usually called noise-free areas; on the contrary, the smooth area with small gradient variance has a concentrated distribution of pixel values, and the image looks more uniform and smooth, and the noise features are stronger; this low gradient variance area is called a noise-prone area; in order to ensure that the texture in the extracted noise sequence is as uniform as possible, the noise sampling area is selected as:

[0067]

[0068] Among them, Cut(·) is the cropping operation, Mean(·) and Var(·) are the mean and variance respectively, μ is the maximum value of the mean of all sampling areas in a given noise sampling image, and θ is the maximum value of the variance of all sampling areas in a given noise sampling image; by transferring the unique distribution of noise to different data, the model can learn more discriminative features.

[0069] S3: Fusion of the generated infrared noise data with the original image;

[0070] Data augmentation refers to the process of increasing the amount of data by creating synthetic data from existing data; however, basic data augmentation methods may change the semantic information of the image, resulting in image quality degradation and overfitting problems; the present invention generates new images by linear interpolation and using weights to mix the pixel values ​​of two different images, which can improve the generalization ability and robustness of the model;

[0071] The size of the noise feature is adjusted to be equal to the size of the noise sampled image input to accommodate real-world noise;

[0072] N real =Resize(N p )

[0073] Resize(·) means adjusting the size of the noise-prone area to match the input image I input The size of the infrared data is sampled with real-world noise; the input training image I input Replaced by:

[0074] I mix =βN real +(1-β)I input

[0075] Among them I mix ∈R n×c×h×w represents the training samples mixed with noise, and β∈[0,1] represents the hyperparameter of noise replacement.

[0076] S4: Perform negative sample enhancement on the fused data;

[0077] 850 to 950 images are selected for negative sample enhancement, and the small target anchor area in the image is denoted as P target ∈[c,p,p]; where p represents the length and width of the small target anchor area, and c represents the number of channels; for P target Perform center rotation to generate negative samples, which helps the model learn more diverse features from challenging negative samples:

[0078] I R =Rot(P target ,γ),γ∈[0°,90°,180°,270°]

[0079] in, represents the samples after negative enhancement by rotating 0°, 90°, 180°, and 270° respectively, γ represents the rotation angle; Rot(·) represents the random center rotation operation.

[0080] S5: Design and training of detection models;

[0081] The overall process is as follows Figure 2 As shown;

[0082] S501: Establish a feature extraction network, and use the feature extraction network to extract high-level abstract features from infrared small target image data, such as Figure 3 As shown;

[0083] The feature extraction network is mainly composed of convolution module, linear attention module and hole convolution module. Figure 4 As shown, the linear attention module is Figure 5 As shown in Figure 6 As shown;

[0084] The overall process is as follows:

[0085] The input tensor [4, 3, 256, 256] is passed through convolution module 1 to obtain the feature tensor a: [4, 64, 256, 256];

[0086] The feature tensor a is pooled to obtain the feature tensor: [4, 128, 128, 128], and the feature tensor b is obtained after the linear attention module 1: [4, 128, 128, 128];

[0087] The feature tensor b is pooled to obtain the feature tensor: [4, 256, 64, 64], and the feature tensor c is obtained after the linear attention module 2: [4, 256, 64, 64];

[0088] The feature tensor c is pooled to obtain the feature tensor: [4, 512, 32, 32], and the feature tensor d is obtained after the linear attention module 3: [4, 512, 32, 32];

[0089] The feature tensor d is pooled to obtain the feature tensor: [4,512,16,16], and the feature tensor e is obtained through the dilated convolution module 1: [4,1024,16,16];

[0090] The feature tensor e is upsampled to obtain the feature tensor D: [4, 512, 32, 32]; the feature tensor D is concatenated with the feature tensor d to become a tensor of [4, 1024, 32, 32], and the feature tensor f is obtained after the dilated convolution module 2: [4, 512, 32, 32];

[0091] The feature tensor f is upsampled to obtain the feature tensor C: [4, 256, 64, 64]; the feature tensor C and the feature tensor c are concatenated to become a tensor of [4, 512, 64, 64], and the feature tensor g is obtained after the convolution module 2: [4, 128, 64, 64];

[0092] The feature tensor g is upsampled to obtain the feature tensor B: [4, 128, 128, 128]; the feature tensor B and the feature tensor b are concatenated to become a tensor of [4, 256, 128, 128], and the feature tensor h is obtained after the convolution module 3: [4, 128, 128, 128];

[0093] The feature tensor h is upsampled to obtain the feature tensor A: [4, 64, 256, 256]; the feature tensor A is concatenated with the feature tensor a to become a tensor of [4, 128, 256, 256], and the feature tensor i is obtained after the convolution module 4: [4, 64, 256, 256];

[0094] The architecture of S5011, convolution module 1, convolution module 2, convolution module 3, and convolution module 4 is as follows: the input is sequentially output after passing through convolution layer 1, batch normalization, convolution layer 2, batch normalization, and activation function. The convolution layer uses a convolution kernel of size 3×3, and the activation function is the Relu function:

[0095] f(x)=max(0,x)

[0096] Among them, Batch Normalization (BN) is a commonly used technique in deep learning. It normalizes each small batch of data during the training process to solve some problems when training deep neural networks. Batch normalization can speed up training, improve model stability, and has a certain regularization effect to reduce overfitting. The main idea of ​​batch normalization is to keep the input data of each layer in a relatively stable distribution during the training process.

[0097] S5012: The architecture of the linear attention module is as follows: First, the input is sequentially regularized, fully connected layer 1, depthwise separable convolution, and two-dimensional selective scanning, and the regularized output feature tensor 1 is obtained; secondly, the input is regularized, fully connected layer 1, and fully connected layer 2 output feature tensor 2; then feature tensor 1 and feature tensor 2 are multiplied by the corresponding position elements to obtain feature tensor 3; feature tensor 3 is weighted and output with the input through fully connected layer 3; wherein, the two-dimensional selective scanning consists of three parts: scanning expansion operation, selective scanning space state sequence block operation and scanning merging operation; the scanning expansion operation expands the input image into sequences along four different directions; then the selective scanning space state sequence block is used to extract features from these sequences to ensure that the information in each direction is thoroughly scanned, thereby capturing different features; subsequently, the scanning merging operation sums and merges the sequences from the four directions to restore the output image to the same size as the input image; by adjusting the parameters of the selective scanning space state sequence block according to the input, the model can distinguish and retain relevant information while filtering out irrelevant information, which can effectively enhance the target features and suppress noise;

[0098] S5013: The architecture of the dilated convolution module is as follows: the input first passes through convolution layer 1, and the number of channels changes from 512 to 1024; then enters convolution layer 2, and the number of channels changes from 1024 to 512; then enters convolution layer 3, the number of output channels remains unchanged, and the output of convolution layer 2 is concatenated with features to become 1024 channels, and then input into convolution layer 4, the number of output channels remains unchanged, and then concatenated with the output of convolution layer 1, the number of channels becomes 2048, and finally outputs through convolution layer 5, the number of channels becomes 1024; among them, convolution layers 1 and 5 are 3×3 convolution kernels, convolution layers 2 and 4 use convolution kernels with a dilation rate of 2 and a size of 3×3, and convolution layer 3 is a convolution kernel with a dilation rate of 4 and a size of 3×3; after the input feature map is processed by the linear attention module, most of the noise is filtered out, and the target feature is more significant when it is passed to the deeper layers of the network; at this time, the resolution of the feature map is very small, and the receptive field of the convolution operation is relatively large; in order to better process these deep features, The hole convolution module is designed based on the characteristics of the network. Unlike conventional convolution, the hole convolution separates the convolution kernels by introducing gaps between them. This process expands the receptive field without increasing the number of parameters, allowing the network to capture a wider range of context from the input data without increasing complexity. The dilation rate determines how many pixels are skipped between each step of the convolution. A dilation rate of 1 represents a conventional convolution, while a higher dilation rate will separate more pixels. Since deep features contain less semantic information, skip connections are used for feature compensation to prevent information loss caused by convolution operations. The hole convolution module makes full use of the deep characteristics of the network. The first three layers have a gradually increasing receptive field, which can process a larger range of pixels and effectively filter out the noise around small infrared targets. Since the resolution of the feature map is very small, after being processed by the first three layers, the target in the feature map is extremely small. Therefore, by gradually increasing the receptive field and using the hole convolution module, a clean and refined feature map can be obtained.

[0099] Among them, regularization is a technique designed to prevent the model from overfitting the training data, thereby improving the model's generalization ability on unseen data. Regularization reduces the complexity of the model by introducing some constraints or penalty terms during the model training process.

[0100] S5014: The jump connection operation between the encoder and the decoder realizes feature fusion. Specifically, after the features of the decoder part pass through a module, they will be spliced ​​with the features of the same level of the encoder, and then upsampled to send the features to the next module until the final output out; the feature pyramid fusion module upsamples from the top-level features downward through a top-down path, and at the same time performs cross-scale fusion from the bottom-level features upward through a bottom-up path, thereby generating a multi-scale feature pyramid at each level; it plays the role of supplementing information, allowing the model to rely on more information while improving the problem of insufficient upsampling information, thereby improving segmentation accuracy; the bottom-level features generally contain more detailed information, while the top-level features contain higher-level semantic information; this helps the network to obtain rich information at different scales, so as to better handle targets of different scales.

[0101] S502: Perform multi-scale deep supervision on the result of feature fusion;

[0102] The feature tensor i passes through the convolution layer 1 to get the output out: [4, 1, 256, 256];

[0103] The feature tensor i passes through the convolution layer 2 to obtain the output out1: [4, 1, 256, 256];

[0104] The feature tensor h is passed through convolution layer 3 to obtain the output out2: [4, 1, 128, 128];

[0105] The feature tensor g passes through the convolution layer 4 to obtain the output out3: [4, 1, 64, 64];

[0106] The feature tensor f passes through the convolution layer 5 to obtain the output out4: [4, 1, 32, 32];

[0107] The feature tensor e passes through convolution layer 6 to obtain the output out5: [4, 1, 16, 16];

[0108] out2, out3, out4, out5, and out1 are upsampled by bilinear interpolation to obtain feature tensors of the same size and dimension as out and out1: [4, 1, 256, 256];

[0109] The features of each layer of the feature extraction network are retained, and they are adjusted to the output size through upsampling and convolution operations for deep supervision. During the training process, deep supervision usually introduces additional loss functions, and combines the loss functions of each output layer to update the parameters of the network, outputting results of different scales from shallow to deep: out1, out2, out3, out4, out5, and then fused with the feature pyramid result out for supervision. In the test phase, usually only the prediction results of the final output layer are used.

[0110] Deep supervision adds additional output layers at different levels of the decoder and combines these output layers with corresponding loss functions to supervise the network to make predictions on features at different levels; features at different levels capture information at different scales, which can better fuse global and local information and improve the model's ability to perceive the target; by adding loss functions at multiple levels, gradients can be better propagated and the convergence process of the model can be accelerated; deep supervision can alleviate the problem of gradient vanishing or exploding, and help train deeper and more stable networks.

[0111] S503: reducing false detections by using an eight-connected neighborhood clustering algorithm and outputting a result;

[0112] During the feature extraction network training process, an eight-connected neighborhood clustering module is introduced to cluster pixels belonging to the same target and calculate the centroid of each target; if any two pixels (k0, r0) and (k1, r1) in the feature map have overlapping areas in their eight-neighborhoods:

[0113]

[0114] Then (k0, r0) and (k1, r1) are determined to be adjacent pixels; then, if these two pixels have the same value (0 or 1):

[0115]

[0116] Where a(k 0 ,r 0 ) and a(k 1 ,r 1 ) represent pixels (k 0 ,r 0 ) and (k 1 ,r 1 ), then the two pixels are considered to be in a connected area, and the pixels in the connected area belong to the same target; once all the targets in the image are determined, the center of mass can be calculated according to their coordinates. By matching the areas where the center of mass distance is less than the threshold, false alarms can be reduced, that is, those predicted areas with no real corresponding objects can be reduced; the successfully matched areas are counted in PD (Probability of detection), while the unmatched areas are counted in FA (False rate).

[0117] S6: Use the trained target detection model to detect small infrared targets;

[0118] The evaluation indicators used in the present invention are as follows:

[0119] The loss function Soft_IoU (Soft Intersection over Union Loss) measures the accuracy of the prediction results in the segmentation task by calculating the ratio of the intersection and union between the predicted mask and the target mask. The formula is as follows:

[0120]

[0121] Among them, smooth is a smoothing factor used to prevent the denominator from being zero; intersection and union are the intersection and union of the prediction results and label pixels; Soft_IoU is smoothed to better optimize the training process; compared with the traditional IoU loss, Soft_IoU can better handle the situation of category imbalance and can update the gradient more smoothly, thereby avoiding the oscillation phenomenon during training;

[0122] The interaction ratio IOU, normalized interaction ratio nIOU, and F1 score formulas are:

[0123]

[0124]

[0125] Among them, TP (True Positive): true positive example, the model predicts it to be a positive example, but it is actually a positive example; FP (False Positive): false positive example, the model predicts it to be a positive example, but it is actually a negative example;

[0126] After data enhancement, the data is divided into a training set and a test set in a ratio of 8:2; the training set is used to train the network model, and the model adjusts parameters by learning the features and labels of small infrared targets in the training set; the test set is used to verify whether the trained model can accurately detect small infrared targets in real scenes;

[0127] The image data in the training set is input into the target detection network, the output is calculated through forward propagation, and then the gradient is calculated and the network parameters are updated through back propagation; the set rounds are iterated, and 1500 rounds are set in this embodiment. The training status of the current model is fed back every 10 rounds to prevent the model from overfitting during training; after the training, the test set is used to evaluate the performance indicators of the model, such as interaction ratio, normalized interaction ratio, F1 score, etc.

[0128] Figure 7 , Figure 8 To use the infrared small target detection network to detect the effect diagram, this network can accurately detect infrared small targets and accurately segment them;

[0129] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. A method for detecting small infrared targets in complex scenes based on linear attention, characterized in that: The following steps are involved: S1: Construction and preprocessing of the data set: obtaining infrared small target images and preprocessing them, including adjusting the image size, image standardization, and adjusting the annotation format, to obtain the preprocessed infrared small target detection data; S2: Generate infrared noise dataset using sequence noise modeling technology; S3: Fusion of the generated infrared noise data with the original image; S4: Perform negative sample enhancement on the fused data; S5: Design and training of detection models; S6: Use the trained target detection model to detect small infrared targets.

2. According to the method of detecting small infrared targets in complex scenes based on attention in claim 1, it is characterized in that: The step 2 specifically includes the following: The images in the infrared small target dataset contain objects such as aircraft, airplanes, and floating objects on the sea surface. The scenes include towns, fields, sky, and sea surface. The images are classified and sorted according to the scenes. Select no less than 800 images as noise sampling images, and give a noise sample U with a noise sampling area size of 32×32 i , i represents the i-th noise sampling area in a given noise sampling image, where i∈[1,64]; in order to ensure that the texture in the extracted noise sequence is as uniform as possible, the noise sampling area N is selected p for: Among them, Cut(·) is the cropping operation, Mean(·) and Var(·) are the mean and variance respectively, μ is the maximum value of the mean of all sampling areas in a given noise sampling image, and θ is the maximum value of the variance of all sampling areas in a given noise sampling image.

3. According to the method of infrared small target detection in complex scenes based on attention as described in claim 1, it is characterized in that: The step 3 specifically includes the following steps: Resize the noise sampling region to be equal to the size of the noise sampling image input to accommodate real-world noise: N real =Resoze(N p ) Resize(·) means adjusting the size of the noise-prone area to match the input image I input The size of the infrared data is sampled with real-world noise; the input training image I input Replaced by: I mix =βN real +(1-β)I input Among them I mix represents the training samples mixed with noise, and β∈[0,1] represents the hyperparameter of noise replacement.

4. According to the method of detecting small infrared targets in complex scenes based on attention in claim 1, it is characterized in that: The step 4 specifically comprises the following steps: 850 to 950 images are selected for negative sample enhancement, and the small target anchor area in the image is denoted as P target ∈[c,p,p]; where p represents the length and width of the small target anchor area, c represents the number of channels, and P target Perform center rotation to generate negative samples: AND R =Rot(P target ,γ),γ∈[0°,90°,180°,270°] in, represents the samples after negative enhancement by rotating 0°, 90°, 180°, and 270° respectively, γ represents the rotation angle; Rot(·) represents the random center rotation operation.

5. According to the method of detecting small infrared targets in complex scenes based on attention in claim 1, it is characterized in that: The step 5 specifically comprises the following steps: S501: Establish a feature extraction network, and extract high-level abstract features from infrared small target image data through the feature extraction network; The feature extraction network is mainly composed of convolution module, linear attention module and hole convolution module. The overall process is as follows: The input tensor passes through the convolution module 1 to obtain the feature tensor a; The feature tensor a is pooled and then passed through the linear attention module 1 to obtain the feature tensor b. The feature tensor b undergoes a pooling operation and then passes through the linear attention module 2 to obtain the feature tensor c; The feature tensor c undergoes a pooling operation and then passes through a linear attention module 3 to obtain the feature tensor d; The feature tensor d is pooled and then passes through the dilated convolution module 1 to obtain the feature tensor e. The feature tensor e is upsampled to obtain the feature tensor D, which is concatenated with the feature tensor d and then passed through the dilated convolution module 2 to obtain the feature tensor f. The feature tensor f is upsampled to obtain the feature tensor C; the feature tensor C is concatenated with the feature tensor c, and then passed through the convolution module 2 to obtain the feature tensor g; The feature tensor g is upsampled to obtain the feature tensor B; the feature tensor B is concatenated with the feature tensor b, and then passed through the convolution module 3 to obtain the feature tensor h; The feature tensor h is upsampled to obtain the feature tensor A; the feature tensor A is concatenated with the feature tensor a, and then passed through the convolution module 4 to obtain the feature tensor i; The architecture of convolution module 1, convolution module 2, convolution module 3, and convolution module 4 is as follows: the input is output after passing through convolution layer 1, batch normalization, convolution layer 2, batch normalization, and activation function in sequence. The convolution layer uses a convolution kernel of size 3×3, and the activation function is the Relu function: f(x)=max(0,x) The architecture of linear attention module 1, linear attention module 2 and linear attention module 3 are as follows: first, the input is sequentially regularized, fully connected layer 1, depthwise separable convolution and two-dimensional selective scanning, and the regularized output feature tensor 1 is obtained; secondly, the input is regularized, fully connected layer 1 and fully connected layer 2 output feature tensor 2; then feature tensor 1 and feature tensor 2 are multiplied by the corresponding position elements to obtain feature tensor 3; feature tensor 3 is weighted and output with the input through fully connected layer 3; among them, the two-dimensional selective scanning consists of three parts: scanning expansion operation, selective scanning space state sequence block operation and scanning merging operation; the scanning expansion operation expands the input image into sequences along four different directions, and then extracts features from these sequences through the selective scanning space state sequence block, and the scanning merging operation sums and merges the sequences from the four directions to restore the output image to the same size as the input image; the parameters of the dimension selective scanning module are adjusted according to the input; The architecture of dilated convolution module 1 and dilated convolution module 2 is as follows: the input first passes through convolution layer 1, then enters convolution layer 2, and then enters convolution layer 3, and is concatenated with the output of convolution layer 2, and then input into convolution layer 4, and then concatenated with the output of convolution layer 1, and finally output through convolution layer 5; among them, convolution layers 1 and 5 are 3×3 convolution kernels, convolution layers 2 and 4 use convolution kernels with a dilation rate of 2 and a size of 3×3, and convolution layer 3 is a convolution kernel with a dilation rate of 4 and a size of 3×3; S502: Perform multi-scale deep supervision on the result of feature fusion; The feature tensor i is output through convolution layer 1; The feature tensor i is output through convolution layer 2; The feature tensor h passes through the convolution layer 3 and then is upsampled by bilinear interpolation to obtain the output; The feature tensor g passes through the convolution layer 4 and is then upsampled by bilinear interpolation to obtain the output; The feature tensor f passes through the convolution layer 5 and is then upsampled by bilinear interpolation to obtain the output; The feature tensor e passes through the convolution layer 6 and is then upsampled by bilinear interpolation to obtain the output; S503: reducing false detections by using an eight-connected neighborhood clustering algorithm and outputting a result; Among them, the pixels belonging to the same target are clustered together, and the centroid of each target is calculated; by matching the areas with a centroid distance less than a threshold, those less than a threshold are added to the target list, and those greater than a threshold are not added, thereby reducing the predicted areas that have no real corresponding objects; the successfully matched areas are counted in the PD (Probability of detection), and the unmatched areas are counted in the FA (False rate).

6. According to the method of detecting small infrared targets in complex scenes based on attention in claim 1, it is characterized in that: The step 6 specifically comprises the following steps: The infrared small target detection dataset is divided into training set and test set in the ratio of 8:

2. The image data in the training set is input into the neural network, the output is calculated through forward propagation, and then the gradient is calculated and the network parameters are updated through back propagation. The set rounds are iterated, and the training status of the current model is fed back through the validation set after each set round to prevent the model from overfitting during training. After the training, the test set is used to evaluate the performance indicators of the model.

Citation Information

Patent Citations

  • Lightweight small target detection method in combination with attention mechanism

    CN113065558A

  • Salient target detection method based on RGB-T multi-source image data

    CN114898106A

  • Infrared small target detection method based on attention fusion feature pyramid network

    CN116524312A

  • Forest fire intelligent identification method and related device

    CN118052967A

  • Infrared weak and small target detection method based on feature fusion

    CN118279707A

Cited By

  • Infrared weak and small target detection method for complex space scene

    CN121564323A