An infrared small target detection method in complex scenes based on linear attention

Abstract

The application provides an infrared small target detection method in a complex scene based on linear attention; first, the original data set is processed and divided, noise data is applied to the pictures in the training set by using sequence noise modeling technology to obtain small target data in a complex scene and improve the detection generalization ability; the small target region in the data set is subjected to negative sample enhancement to expand the data set; second, a small target segmentation network is invented; the encoding part obtains high-level semantic information by reducing the spatial dimension, extracts features, and the decoding stage restores the spatial dimension and reconstructs the original image structure; the linear attention module in the network aims to improve the feature extraction capability while reducing the spatial complexity; the hollow convolution module enlarges the receptive field and reduces the loss of small target information caused by too many down-sampling times; the high-resolution information is retained by using the jump connection, and the information of different scales is supervised by using the depth; finally, the eight-connected neighborhood clustering module is used to ensure the matching accuracy and reduce the false detection; the application can detect the target in the infrared image in the complex background, effectively suppress the interference of background noise, and improve the accuracy of infrared small target detection.

Description

Technical Field

[0001] This invention relates to the field of infrared small target detection technology, and in particular to an infrared small target detection method based on linear attention in complex scenes. Background Technology

[0002] With the rapid development of science and technology, target detection has gradually become an important part of the field of computer vision. Target detection is an image segmentation based on the geometric and statistical features of targets, with the aim of finding the location and size of target objects in an image. In recent years, the aerospace field has developed rapidly, and timely and accurate detection of high-heat targets at long distances is a major research area to protect people's lives and property. Among them, target detection in the field of infrared imaging can detect infrared objects in dark and harsh environments, improve the target detection performance at night and in no-visibility weather, and capture the thermal radiation signals of target objects, thereby ensuring the continuity and effectiveness of detection activities. Therefore, it has received great attention.

[0003] Compared to ordinary visible light target detection, infrared weak target detection faces the following challenges: 1. Infrared small targets are imaged at long distances, and the target pixels account for a very small percentage of the total image pixels; 2. The background is complex and subject to interference from many atmospheric clouds, and the infrared radiation energy attenuates with distance, making the target easily submerged in background clutter and sensor noise; 3. Infrared weak target detection images are very sparse, resulting in a severe imbalance between the target and background areas. Therefore, when conducting research on infrared weak target detection, the impact of these unfavorable factors needs to be fully considered.

[0004] Existing infrared small target detection methods can be divided into four main categories based on their algorithmic approaches: filter-based infrared small target detection methods, contrast-based infrared small target detection methods, low-rank sparse model-based infrared small target detection methods, and deep learning-based infrared small target detection methods. The first three methods often rely heavily on prior knowledge, are sensitive to hyperparameters, and perform poorly on complex backgrounds and noisy images. Deep learning-based methods are limited by the local receptive field of convolution operations, lack global perception capabilities, and are prone to detecting noise in infrared images as targets. Since the small targets in the infrared small target detection dataset are small, multiple downsampling can easily lead to target loss, and this process is irreversible.

[0005] The above analysis shows that existing methods lack global information perception capabilities, easily misdetect noise as targets, and deep networks are prone to losing small target features, resulting in poor performance in infrared small target detection tasks. Summary of the Invention

[0006] The purpose of this invention is to provide an infrared small target detection method based on linear attention in complex scenes, so as to solve the problems mentioned in the background art.

[0007] A method for detecting small infrared targets in complex scenes based on linear attention, characterized by the following steps:

[0008] The dataset is constructed and preprocessed by acquiring infrared small target images and preprocessing them, including adjusting image size, image standardization, and adjusting annotation format, to obtain preprocessed infrared small target detection data.

[0009] Infrared noise datasets were generated using sequence noise modeling techniques.

[0010] The infrared small target dataset contains images of objects such as aircraft, airplanes, and floating objects on the sea surface. The scenes include towns, fields, sky, and sea. The images are classified and sorted according to the scenes.

[0011] Select no fewer than 800 images as noise sampling images, and give a noise sampling region U of size 32×32. i Let i represent the i-th noise sampling region in a given noise sampling image, where i∈[1,64]; to ensure that the texture in the extracted noise sequence is as uniform as possible, select noise sampling region N. p for:

[0012]

[0013] Where Cut(·) is the cropping operation, Mean(·) and Var(·) are the mean and variance, respectively, μ is the maximum value of the mean of all sampling regions in the given noisy sampled image, and θ is the maximum value of the variance of all sampling regions in the given noisy sampled image.

[0014] The generated infrared noise data is then fused with the original image;

[0015] Adjust the size of the noise sampling region to match the size of the noise sampling image input, in order to adapt to real-world noise:

[0016] N real =Resize(N) p )

[0017] Where Resize(·) represents adjusting the size of the noise-prone region to match the input image I. input The size of the infrared sensor data was used to sample real-world noise; the input training image I was then used. input Replace with:

[0018] I mix =βNreal +(1-β)I input

[0019] Where I mix Let represent a training sample mixed with noise, and β∈[0,1] represent the hyperparameters of the noise permutation.

[0020] Negative sample augmentation is performed on the fused data;

[0021] 850 to 950 images were selected for negative sample enhancement, and the small target anchor point region in the image was denoted as P. target ∈[c,p,p]; where p represents the length and width of the small target anchor region, c represents the number of channels, and for P target Perform a center rotation to generate negative samples:

[0022] I R =Rot(P target ,γ),γ∈[0°,90°,180°,270°]

[0023] in, This represents the samples after negative enhancement by rotating them by 0°, 90°, 180°, and 270°, respectively, where γ represents the rotation angle; Rot(·) represents the random center rotation operation.

[0024] Design and training of the detection model;

[0025] First, a feature extraction network is established, and high-level abstract features are extracted from infrared small target image data through the feature extraction network;

[0026] The feature extraction network mainly consists of convolutional modules, linear attention modules, and dilated convolutional modules;

[0027] The input tensor is processed by convolution module 1 to obtain the feature tensor a;

[0028] Feature tensor a undergoes pooling operation, and then passes through linear attention module 1 to obtain feature tensor b;

[0029] The feature tensor b undergoes a pooling operation, and then passes through the linear attention module 2 to obtain the feature tensor c;

[0030] The feature tensor c undergoes a pooling operation, and then passes through the linear attention module 3 to obtain the feature tensor d;

[0031] The feature tensor d is pooled and then passed through dilated convolution module 1 to obtain the feature tensor e.

[0032] The feature tensor e is upsampled to obtain the feature tensor D. The feature tensor D and the feature tensor d are concatenated and then passed through the dilated convolution module 2 to obtain the feature tensor f.

[0033] The feature tensor f is upsampled to obtain the feature tensor C; the feature tensor C and the feature tensor c are concatenated, and then passed through the convolution module 2 to obtain the feature tensor g;

[0034] The feature tensor g is upsampled to obtain the feature tensor B; the feature tensor B and the feature tensor b are concatenated, and then passed through the convolution module 3 to obtain the feature tensor h;

[0035] The feature tensor h is upsampled to obtain the feature tensor A; the feature tensor A and the feature tensor a are concatenated, and then passed through the convolution module 4 to obtain the feature tensor i;

[0036] The architecture of convolutional modules 1, 2, 3, and 4 is as follows: the input passes through convolutional layer 1, batch normalization, convolutional layer 2, batch normalization, and activation function sequentially before outputting. The convolutional layers use 3×3 kernels, and the activation function is the ReLU function.

[0037] f(x) = max(0,x)

[0038] The architecture of linear attention modules 1, 2, and 3 is as follows: First, the input sequentially passes through regularization, fully connected layer 1, depthwise separable convolution, and 2D selective scanning, with regularization outputting feature tensor 1. Second, the input passes through regularization, fully connected layer 1, and fully connected layer 2, outputting feature tensor 2. Then, feature tensor 1 and feature tensor 2 are multiplied element-wise at corresponding positions to obtain feature tensor 3. Feature tensor 3 is weighted with the input through fully connected layer 3 and output. The 2D selective scanning consists of three parts: scan expansion, selective scan spatial state sequence block operation, and scan merging operation. The scan expansion operation expands the input image into a sequence along four different directions. Then, features are extracted from these sequences using selective scan spatial state sequence blocks. The scan merging operation sums and merges the sequences from the four directions, restoring the output image to the same size as the input image. The parameters of the 2D selective scanning module are adjusted according to the input.

[0039] The architecture of dilated convolutional modules 1 and 2 is as follows: The input first passes through convolutional layer 1, then convolutional layer 2, then convolutional layer 3, where it is concatenated with the output of convolutional layer 2. The input is then fed into convolutional layer 4, where it is concatenated with the output of convolutional layer 1, and finally output through convolutional layer 5. Convolutional layers 1 and 5 use 3×3 kernels, convolutional layers 2 and 4 use kernels with a dilation rate of 2 and a size of 3×3, and convolutional layer 3 uses a kernel with a dilation rate of 4 and a size of 3×3.

[0040] The feature tensor i is passed through convolutional layer 1 to obtain the output;

[0041] The feature tensor i is passed through convolutional layer 2 to obtain the output;

[0042] The feature tensor h passes through convolutional layer 3, and then is upsampled by bilinear interpolation to obtain the output;

[0043] The feature tensor g is passed through convolutional layer 4, and then upsampled by bilinear interpolation to obtain the output;

[0044] The feature tensor f is passed through convolutional layer 5, and then upsampled by bilinear interpolation to obtain the output;

[0045] The feature tensor e is passed through convolutional layer 6, and then upsampled by bilinear interpolation to obtain the output;

[0046] Finally, an 8-connected neighborhood clustering algorithm is used to reduce false detections and output results. Pixels belonging to the same target are clustered together, and the centroid of each target is calculated. Regions with centroid distances less than a threshold are added to the target list if the distance is less than the threshold, and not added if the distance is greater than the threshold, thus reducing the number of predicted regions without corresponding objects. Successfully matched regions are counted in PD (Probability of detection), while unmatched regions are counted in FA (False alarm rate).

[0047] The trained target detection model is used to detect small infrared targets.

[0048] The infrared small target detection dataset is divided into a training set and a test set in a ratio of 8:2. The image data in the training set is input into the neural network, and the output is calculated through forward propagation. Then, the gradient is calculated and the network parameters are updated through backpropagation. The model is iterated for a set number of rounds. After each set number of rounds, the training status of the current model is fed back through the validation set to prevent the model from overfitting during training. After training, the performance metrics of the model are evaluated using the test set.

[0049] The beneficial effects of adopting the above technical solution are as follows:

[0050] This invention provides a method for detecting small infrared targets in complex scenes based on linear attention. First, the original dataset is divided, and sequential noise modeling techniques are used to add noise data to the images in the training set to obtain small target data in complex scenes, improving detection generalization ability. Then, negative sample enhancement is performed on the small target regions in the dataset to expand the dataset. Second, a small target segmentation network is invented. The encoding part obtains high-level semantic information by reducing spatial dimensions and extracts features. The decoding stage restores spatial dimensions and reconstructs the original image structure. The linear attention module in the network aims to improve feature extraction ability while reducing spatial complexity. The dilated convolution module expands the receptive field and reduces the loss of small target information caused by excessive downsampling. Skip connections are used to retain high-resolution information, and depth supervision is used to monitor information at different scales. Finally, an eight-connected neighborhood clustering module is used to ensure accurate matching and reduce false detections. This invention can perform target detection on infrared images with complex backgrounds, effectively suppressing background noise interference and improving the accuracy of infrared small target detection. Attached Figure Description

[0051] Figure 1 This is an overall flowchart in an example of the present invention;

[0052] Figure 2 This is a flowchart of step S5 in an example of the present invention;

[0053] Figure 3 This is a diagram of the infrared small target detection network structure in an example of the present invention;

[0054] Figure 4 This is a structural diagram of the convolution module in an example of the present invention;

[0055] Figure 5 This is a structural diagram of the linear attention module in an example of the present invention;

[0056] Figure 6 This is a structural diagram of the dilated convolution module in an example of the present invention;

[0057] Figure 7 This is an image showing the infrared small target detection and recognition results in an example of the present invention;

[0058] Figure 8 This is an image showing the infrared small target detection and recognition results in an example of the present invention; Detailed Implementation

[0059] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples; the following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention;

[0060] A method for detecting small infrared targets in complex scenes based on linear attention, such as... Figure 1 As shown, it includes the following steps:

[0061] S1: Dataset construction and preprocessing: acquire infrared small target images and preprocess them, including adjusting image size, image standardization and adjusting annotation format, to obtain preprocessed infrared small target detection data;

[0062] The infrared small target dataset contains images of objects such as aircraft, airplanes, and floating objects on the sea surface. The scenes include towns, fields, sky, and sea. The images are classified and sorted according to the scene.

[0063] S2: Infrared noise dataset is generated using sequential noise modeling techniques;

[0064] Negative samples are those that do not belong to the target category, used to enhance the discriminative ability of the target model; at least 800 images are selected as noise sampling images I. noi :

[0065]

[0066] Given a noise sample U with a noise sampling region size of 32×32 i Let i represent the i-th noise sampling region in the noise sampling image, where i∈[1,64]. Generally, noise sampling regions with large variance have a wide distribution of pixel values, significant differences between bright and dark areas in the image, and rich textures. These textures often cover the noise itself and are usually called noise-free regions. Conversely, regions with small gradient variance and gentle pixel distribution have concentrated pixel values, making the image appear more uniform and smooth, and the noise characteristics are stronger. This low gradient variance region is called a noise-prone region. To ensure that the texture in the extracted noise sequence is as uniform as possible, the noise sampling region is selected as:

[0067]

[0068] Where Cut(·) is the cropping operation, Mean(·) and Var(·) are the mean and variance, respectively, μ is the maximum value of the mean of all sampled regions in a given noisy sampled image, and θ is the maximum value of the variance of all sampled regions in a given noisy sampled image; by passing the unique distribution of noise to different data, the model can learn more discriminative features.

[0069] S3: Fuse the generated infrared noise data with the original image;

[0070] Data augmentation refers to the process of increasing the amount of data by creating synthetic data from existing data; however, basic data augmentation methods may alter the semantic information of an image, leading to image quality degradation and overfitting problems; this invention generates new images by linear interpolation and using weighted mixing of pixel values ​​from two different images, which can improve the generalization ability and robustness of the model.

[0071] Adjust the size of the noise feature to match the size of the noise sampled image input to adapt to real-world noise;

[0072] N real =Resize(N) p )

[0073] Where Resize(·) means adjusting the size of the noise-prone region to match the input image I. input The size of the infrared data was used to sample real-world noise; the input training image I was... input Replace with:

[0074] I mix =βN real +(1-β)I input

[0075] Where I mix ∈R n×c×h×w Let represent a training sample mixed with noise, and β∈[0,1] represent the hyperparameters of the noise permutation.

[0076] S4: Perform negative sample augmentation on the fused data;

[0077] 850 to 950 images were selected for negative sample enhancement, and the small target anchor point region in the image was denoted as P. target ∈[c,p,p]; where p represents the length and width of the small target anchor point region, and c represents the number of channels; for P target A center rotation is performed to generate negative samples, thereby helping the model learn more diverse features from challenging negative samples:

[0078] I R =Rot(P target ,γ),γ∈[0°,90°,180°,270°]

[0079] in, This represents the samples after negative enhancement by rotating them by 0°, 90°, 180°, and 270°, respectively, where γ represents the rotation angle; Rot(·) represents the random center rotation operation.

[0080] S5: Design and training of the detection model;

[0081] The overall process is as follows Figure 2 As shown;

[0082] S501: Establish a feature extraction network and use the feature extraction network to extract high-level abstract features from infrared small target image data, such as... Figure 3 As shown;

[0083] The feature extraction network mainly consists of convolutional modules, linear attention modules, and dilated convolutional modules. The convolutional modules, for example... Figure 4 As shown, the linear attention module is as follows Figure 5 As shown, the dilated convolution module is as follows: Figure 6 As shown;

[0084] The overall process is as follows:

[0085] The input tensor [4,3,256,256] is processed by convolution module 1 to obtain the feature tensor a: [4,64,256,256].

[0086] Feature tensor a is obtained by pooling operation to obtain feature tensor [4,128,128,128], and then by linear attention module 1 to obtain feature tensor b [4,128,128,128].

[0087] The feature tensor b is obtained by pooling operation to obtain the feature tensor: [4,256,64,64], and then the feature tensor c is obtained by linear attention module 2: [4,256,64,64].

[0088] The feature tensor c is obtained by pooling operation: [4,512,32,32], and then the feature tensor d is obtained by linear attention module 3: [4,512,32,32].

[0089] The feature tensor d is obtained by pooling operation to obtain the feature tensor: [4,512,16,16], and then obtained by dilated convolution module 1 to obtain the feature tensor e: [4,1024,16,16].

[0090] The feature tensor e is upsampled to obtain the feature tensor D: [4,512,32,32]; the feature tensor D and the feature tensor d are concatenated to become the tensor [4,1024,32,32], which is then passed through the dilated convolution module 2 to obtain the feature tensor f: [4,512,32,32].

[0091] The feature tensor f is upsampled to obtain the feature tensor C: [4,256,64,64]; the feature tensor C and the feature tensor c are concatenated to form the tensor [4,512,64,64], which is then passed through the convolution module 2 to obtain the feature tensor g: [4,128,64,64];

[0092] The feature tensor g is upsampled to obtain the feature tensor B: [4,128,128,128]; the feature tensor B and the feature tensor b are concatenated to form a tensor [4,256,128,128], which is then processed by the convolution module 3 to obtain the feature tensor h: [4,128,128,128].

[0093] The feature tensor h is upsampled to obtain the feature tensor A: [4,64,256,256]; the feature tensor A and the feature tensor a are concatenated to form the tensor [4,128,256,256], which is then passed through the convolution module 4 to obtain the feature tensor i: [4,64,256,256].

[0094] The architecture of S5011, Convolutional Module 1, Convolutional Module 2, Convolutional Module 3, and Convolutional Module 4 is as follows: The input passes through Convolutional Layer 1, batch normalization, Convolutional Layer 2, batch normalization, and activation function in sequence before outputting. The convolutional layers use 3×3 kernels, and the activation function is the ReLU function.

[0095] f(x) = max(0,x)

[0096] Batch Normalization (BN) is a commonly used technique in deep learning. It normalizes each mini-batch of data during training to address some issues encountered when training deep neural networks. Batch normalization can accelerate training, improve model stability, and has a certain regularization effect, reducing overfitting. The main idea behind batch normalization is to maintain a relatively stable distribution of input data for each layer during training.

[0097] The architecture of the S5012 linear attention module is as follows: First, the input passes through regularization, fully connected layer 1, depthwise separable convolution, and 2D selective scanning in sequence, with regularization outputting feature tensor 1. Second, the input passes through regularization, fully connected layer 1, and fully connected layer 2, outputting feature tensor 2. Then, feature tensor 1 and feature tensor 2 are multiplied element-wise at corresponding positions to obtain feature tensor 3. Feature tensor 3 is weighted with the input through fully connected layer 3 and output. The 2D selective scanning consists of three parts: scan expansion operation, selective scan spatial state sequence block operation, and scan merging operation. The scan expansion operation expands the input image into a sequence along four different directions. Then, the selective scan spatial state sequence block extracts features from these sequences, ensuring that information in each direction is thoroughly scanned, thereby capturing different features. Subsequently, the scan merging operation sums and merges the sequences from the four directions, restoring the output image to the same size as the input image. By adjusting the parameters of the selective scan spatial state sequence block according to the input, the model can distinguish and retain relevant information while filtering out irrelevant information, effectively enhancing target features and suppressing noise.

[0098] S5013: The architecture of the dilated convolution module is as follows: The input first passes through convolutional layer 1, where the number of channels increases from 512 to 1024; then it enters convolutional layer 2, where the number of channels decreases from 1024 to 512; next, it enters convolutional layer 3, where the number of output channels remains unchanged, and is concatenated with the output of convolutional layer 2 to obtain 1024 channels. This input then enters convolutional layer 4, where the number of output channels remains unchanged, and is concatenated with the output of convolutional layer 1 to obtain 2048 channels. Finally, it is output through convolutional layer 5, where the number of channels becomes 1024. Convolutional layers 1 and 5 use 3×3 kernels, convolutional layers 2 and 4 use kernels with a dilation rate of 2 and a size of 3×3, and convolutional layer 3 uses a kernel with a dilation rate of 4 and a size of 3×3. After the input feature map is processed by the linear attention module, most of the noise is filtered out, and the target features become more significant when passed to deeper layers of the network. At this point, the resolution of the feature map is very small, while the receptive field of the convolution operation is relatively large. To better handle these deeper features... The design incorporates a dilated convolution module. Unlike regular convolution, dilated convolution separates the kernels by introducing gaps between them. This process expands the receptive field without increasing the number of parameters, allowing the network to capture a wider range of context from the input data without adding complexity. The dilation rate determines how many pixels are skipped between each step of the convolution; a dilation rate of 1 represents a regular convolution, while a higher dilation rate separates more pixels. Since deep features contain less semantic information, skip connections are used for feature compensation to prevent information loss caused by convolution operations. The dilated convolution module fully utilizes the deep nature of the network. The first three layers have progressively larger receptive fields, which can handle a wider range of pixels and effectively filter out noise around small infrared targets. Due to the small resolution of the feature map, the target in the feature map is extremely small after the first three layers. Therefore, by progressively increasing the receptive field and using the dilated convolution module, a clean and detailed feature map can be obtained.

[0099] Regularization is a technique designed to prevent models from overfitting training data, thereby improving the model's generalization ability on unseen data. Regularization reduces model complexity by introducing constraints or penalty terms during model training.

[0100] S5014: Skip connections between the encoder and decoder enable feature fusion. Specifically, features from the decoder are concatenated with features from the same level in the encoder after passing through a module, then upsampled before being fed into the next module, until the final output is reached. The feature pyramid fusion module upsamples from top-level features downwards via a top-down path, while simultaneously performing cross-scale fusion from bottom-level features upwards via a bottom-up path, generating multi-scale feature pyramids at each level. This serves to supplement information, allowing the model to rely on more information while addressing the problem of insufficient upsampled information, thereby improving segmentation accuracy. Bottom-level features generally contain more detailed information, while top-level features contain higher-level semantic information. This helps the network acquire rich information at different scales, thus better handling targets at different scales.

[0101] S502: Perform multi-scale deep supervision on the results of feature fusion;

[0102] The feature tensor i passes through convolutional layer 1 to obtain the output out: [4,1,256,256];

[0103] The feature tensor i passes through convolutional layer 2 to obtain output out1: [4,1,256,256];

[0104] The feature tensor h passes through convolutional layer 3 to obtain output out2: [4,1,128,128];

[0105] The feature tensor g passes through convolutional layer 4 to obtain output out3: [4,1,64,64];

[0106] The feature tensor f is passed through convolutional layer 5 to obtain output out4: [4,1,32,32];

[0107] The feature tensor e passes through convolutional layer 6 to obtain output out5: [4,1,16,16];

[0108] out2, out3, out4, out5, and out1 are upsampled using bilinear interpolation to obtain feature tensors of the same size and dimension as out and out1: [4,1,256,256].

[0109] Features from each layer of the feature extraction network are preserved and adjusted to the output size through upsampling and convolution operations for deep supervision. During training, deep supervision typically introduces an additional loss function and combines it with the loss function of each output layer to update the network parameters, outputting results at different scales from shallow to deep: out1, out2, out3, out4, out5, which are then fused with the feature pyramid result out for supervision. During the testing phase, only the prediction results of the final output layer are typically used.

[0110] Deep supervision involves adding additional output layers at different levels of the decoder and combining these output layers with corresponding loss functions to supervise the network's predictions on features at different levels. Features at different levels capture information at different scales, which can better integrate global and local information and improve the model's ability to perceive the target. By adding loss functions at multiple levels, gradients can be propagated better, accelerating the model's convergence process. Deep supervision can alleviate the problems of gradient vanishing or exploding, which helps to train deeper and more stable networks.

[0111] S503: Reduce false detections and output results using an eight-connected neighborhood clustering algorithm;

[0112] During the training of the feature extraction network, an 8-connected neighborhood clustering module is introduced to cluster pixels belonging to the same target together and calculate the centroid of each target. If any two pixels (k0, r0) and (k1, r1) in the feature map have an intersection region in their eight neighborhoods:

[0113]

[0114] Then (k0, r0) and (k1, r1) are determined to be adjacent pixels; then, if these two pixels have the same value (0 or 1):

[0115]

[0116] Where a(k0,r0) and a(k1,r1) represent the gray values ​​of pixels (k0,r0) and (k1,r1) respectively, these two pixels are considered to be in a connected region, and pixels in the connected region belong to the same target. Once all targets in the image are determined, the centroid can be calculated based on their coordinates. By matching regions where the centroid distance is less than a threshold, false alarms can be reduced, that is, the predicted regions that do not correspond to real objects can be reduced. Successfully matched regions are counted in PD (Probability of detection), while unmatched regions are counted in FA (False alarm rate).

[0117] S6: Use the trained target detection model to detect small infrared targets;

[0118] The evaluation metrics used in this invention are as follows:

[0119] The loss function Soft_IoU (Soft Intersection over Union Loss) measures the accuracy of prediction results in a segmentation task by calculating the ratio of the intersection to the union between the predicted mask and the target mask. The formula is as follows:

[0120]

[0121] Here, smooth is a smoothing factor used to prevent the denominator from being zero; intersection and union are the intersection and union of the predicted result and the label pixels; Soft_IoU performs smoothing processing to better optimize the training process; compared with the traditional IoU loss, Soft_IoU can better handle the case of class imbalance and can update the gradient more smoothly, thus avoiding oscillations during the training process.

[0122] The formulas for Interaction Ratio (IOU), Normalized Interaction Ratio (nIOU), and F1 score are as follows:

[0123]

[0124]

[0125] Wherein, TP (True Positive): a true positive example, which is predicted by the model to be a positive example and is actually a positive example; FP (False Positive): a false positive example, which is predicted by the model to be a positive example and is actually a negative example.

[0126] After data augmentation, the data is divided into training and testing sets in an 8:2 ratio. The training set is used to train the network model, which adjusts its parameters by learning the features and labels of small infrared targets in the training set. The testing set is used to verify whether the trained model can accurately detect small infrared targets in real-world scenarios.

[0127] Image data from the training set is input into the object detection network. The output is calculated through forward propagation, and the gradient is calculated and the network parameters are updated through backpropagation. The number of iterations is set to 1500 in this embodiment. The training status of the current model is fed back every 10 iterations to prevent the model from overfitting during training. After training, the test set is used to evaluate the model's performance metrics, such as interaction ratio, normalized interaction ratio, and F1 score.

[0128] Figure 7 , Figure 8 The image shows the detection results using the infrared small target detection network. This network can accurately detect and precisely segment infrared small targets.

[0129] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. An infrared small target detection method in a complex scene based on linear attention, characterized in that, Includes the following steps: S1: Dataset construction and preprocessing: acquire infrared small target images and preprocess them, including adjusting image size, image standardization and adjusting annotation format, to obtain preprocessed infrared small target detection data; S2: Infrared noise dataset is generated using sequential noise modeling techniques; S3: Fuse the generated infrared noise data with the original image; S4: Perform negative sample augmentation on the fused data; S5: Design and training of the detection model; S501: Establish a feature extraction network and extract high-level abstract features from infrared small target image data through the feature extraction network; The feature extraction network mainly consists of convolutional modules, linear attention modules, and dilated convolutional modules. The overall process is as follows: The input tensor is processed by convolution module 1 to obtain the feature tensor a; Feature tensor a undergoes pooling operation, and then passes through linear attention module 1 to obtain feature tensor b; The feature tensor b undergoes a pooling operation, and then passes through the linear attention module 2 to obtain the feature tensor c; The feature tensor c undergoes a pooling operation, and then passes through the linear attention module 3 to obtain the feature tensor d; The feature tensor d is pooled and then passed through dilated convolution module 1 to obtain the feature tensor e. The feature tensor e is upsampled to obtain the feature tensor D. The feature tensor D and the feature tensor d are concatenated and then passed through the dilated convolution module 2 to obtain the feature tensor f. The feature tensor f is upsampled to obtain the feature tensor C; the feature tensor C and the feature tensor c are concatenated, and then passed through the convolution module 2 to obtain the feature tensor g; The feature tensor g is upsampled to obtain the feature tensor B; the feature tensor B and the feature tensor b are concatenated, and then passed through the convolution module 3 to obtain the feature tensor h; The feature tensor h is upsampled to obtain the feature tensor A; the feature tensor A and the feature tensor a are concatenated, and then passed through the convolution module 4 to obtain the feature tensor i; The architecture of convolutional modules 1, 2, 3, and 4 is as follows: the input passes through convolutional layer 1, batch normalization, convolutional layer 2, batch normalization, and activation function sequentially before outputting. The convolutional layers use 3×3 kernels, and the activation function is the ReLU function. The architecture of linear attention modules 1, 2, and 3 is as follows: First, the input sequentially passes through regularization, fully connected layer 1, depthwise separable convolution, and 2D selective scanning, with regularization outputting feature tensor 1. Second, the input passes through regularization, fully connected layer 1, and fully connected layer 2, outputting feature tensor 2. Then, feature tensor 1 and feature tensor 2 are multiplied element-wise at corresponding positions to obtain feature tensor 3. Feature tensor 3 is weighted with the input through fully connected layer 3 and output. The 2D selective scanning consists of three parts: scan expansion, selective scan spatial state sequence block operation, and scan merging. The scan expansion operation expands the input image into a sequence along four different directions, and then extracts features from these sequences using selective scan spatial state sequence blocks. The scan merging operation sums and merges the sequences from the four directions, restoring the output image to the same size as the input image. The parameters of the 2D selective scanning module are adjusted according to the input. The architecture of dilated convolutional modules 1 and 2 is as follows: The input first passes through convolutional layer 1, then convolutional layer 2, then convolutional layer 3, where it is concatenated with the output of convolutional layer 2. The input is then fed into convolutional layer 4, where it is concatenated with the output of convolutional layer 1, and finally output through convolutional layer 5. Convolutional layers 1 and 5 use 3×3 kernels, convolutional layers 2 and 4 use kernels with a dilation rate of 2 and a size of 3×3, and convolutional layer 3 uses a kernel with a dilation rate of 4 and a size of 3×3. S502: Perform multi-scale deep supervision on the results of feature fusion; S503: Reduce false detections and output results using an eight-connected neighborhood clustering algorithm; S6: Use the trained target detection model to detect small infrared targets.

2. The infrared small target detection method based on linear attention in complex scenes according to claim 1, characterized in that, S2 specifically includes the following: The images in the infrared small target dataset contain aircraft, airplanes, and floating objects on the sea surface. The scenes include towns, fields, sky, and sea. The images are classified and sorted according to the scene. Select no fewer than 800 images as noise sampling images, and provide a noise sampling region of size 32×32. , In a given noisy sampled image, the first... There are noise sampling areas, among which To ensure that the texture in the extracted noise sequence is as uniform as possible, the noise sampling region is selected. for: in, For the cropping operation, and These are the mean and variance, respectively. The maximum value of the mean of all sampled regions in a given noisy sampled image. It represents the maximum variance of all sampled regions in a given noisy sampled image.

3. The infrared small target detection method based on linear attention in complex scenes according to claim 1, characterized in that, S3 specifically includes the following steps: Adjust the size of the noise sampling region to match the size of the noise sampling image input, in order to adapt to real-world noise: in, This indicates adjusting the size of the noise-prone area to match the input image. The size of the infrared data is used to sample real-world noise; the input training image is then used. Replace with: in This represents training samples mixed with noise. The hyperparameters representing noise permutation.

4. The infrared small target detection method based on linear attention in complex scenes according to claim 1, characterized in that, S4 specifically includes the following steps: 850 to 950 images were selected for negative sample augmentation, and the small target anchor point regions in the images were denoted as... Where p represents the length and width of the small target anchor point region, and c represents the number of channels. Perform a center rotation to generate negative samples: in, Indicates that by rotating respectively , , , Samples after negative enhancement Indicates the rotation angle; This indicates a random center rotation operation.

5. The infrared small target detection method based on linear attention in complex scenes according to claim 1, characterized in that, S502 specifically includes the following steps: The feature tensor i is passed through convolutional layer 1 to obtain the output; The feature tensor i is passed through convolutional layer 2 to obtain the output; The feature tensor h passes through convolutional layer 3, and then is upsampled by bilinear interpolation to obtain the output; The feature tensor g is passed through convolutional layer 4, and then upsampled by bilinear interpolation to obtain the output; The feature tensor f is passed through convolutional layer 5, and then upsampled by bilinear interpolation to obtain the output; The feature tensor e is passed through convolutional layer 6, and then upsampled by bilinear interpolation to obtain the output; The S503 specifically Includes the following steps: In this process, pixels belonging to the same target are clustered together, and the centroid of each target is calculated. By matching regions whose centroid distance is less than a threshold, regions that are less than the threshold are added to the target list, while regions that are greater than the threshold are not added, thus reducing the number of predicted regions that do not actually correspond to objects. Regions that are successfully matched are counted in PD (Probability of detection), while regions that are not matched are counted in FA (False alarm rate).

6. The infrared small target detection method in complex scenes based on linear attention as described in claim 1, characterized in that, S6 specifically includes the following steps: The infrared small target detection dataset is divided into a training set and a test set in a ratio of 8:

2. The image data in the training set is input into the neural network, and the output is calculated through forward propagation. Then, the gradient is calculated and the network parameters are updated through backpropagation. The model is iterated for a set number of rounds. After each set number of rounds, the training status of the current model is fed back through the validation set to prevent the model from overfitting during training. After training, the performance metrics of the model are evaluated using the test set.

Images