Infrared small target detection method

Abstract

The invention relates to the technical field of target detection, in particular to an infrared small target detection method, which comprises the following steps of: acquiring an infrared image data set, and filtering noise of an image in the infrared image data set by using a basic feature map extractor to obtain a basic feature map; performing feature enrichment on the basic feature map through a semantic mark converter to obtain an enriched feature map, and then performing block enhancement on the enriched feature map by using a local space semantic attention module to obtain an enhanced feature map; and performing model training based on the enhanced feature map to obtain a small target detection model, and performing small target detection on the to-be-detected infrared image to obtain a detection result. The small target detection model in the invention can inhibit complex infrared background information, and effectively improves the detection accuracy and detection efficiency of the infrared small target.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a small infrared target detection method. Background Art

[0002] Regarding infrared small target detection, infrared sensors can detect the thermal radiation of objects, so they can also monitor at night or in bad weather conditions. In other words, infrared small target detection can be used to identify small and low-contrast targets in infrared images. Traditional infrared small target detection methods, such as manual recognition, mainly rely on expert experience and are affected by the characteristics of small target objects, such as different object sizes, high object density, weak features, and complex backgrounds. They are often time-consuming and easily affected by subjective factors, resulting in low efficiency and accuracy in small target detection.

[0003] In order to solve the defects of manual recognition of small targets, a model algorithm based on deep convolutional neural network has been proposed. However, during the learning process, the model algorithm may also take certain patterns in the complex background as features, resulting in false positives or false negatives. False positives refer to the possibility of misidentifying noise or debris in the background as targets, while false negatives refer to the omission of actual targets. Therefore, the existing model algorithm has poor detection accuracy for infrared small targets in complex backgrounds. Summary of the invention

[0004] In order to solve the technical problem that the detection accuracy of the above-mentioned existing model algorithm for infrared small targets with complex backgrounds is poor, the purpose of the present invention is to provide an infrared small target detection method, and the technical solution adopted is as follows:

[0005] An embodiment of the present invention provides a method for detecting small infrared targets, the method comprising the following steps:

[0006] Obtain several public infrared image datasets;

[0007] Using the several public infrared image data sets to train a model, a small target detection model is obtained;

[0008] Performing small target detection on the infrared image to be detected by using the small target detection model to obtain a detection result;

[0009] The small target detection model includes a basic feature map extractor, a semantic tag converter and a local space semantic attention module;

[0010] The basic feature map extractor is used to filter noise from the input infrared image to obtain a basic feature map;

[0011] The semantic tag converter is used to perform feature enrichment on the input basic feature map to obtain an enriched feature map;

[0012] The local spatial semantic attention module is used to perform block enhancement on the input enriched feature map to obtain an enhanced feature map.

[0013] Further, the basic feature map extractor includes a residual block and a dilated convolution, and the residual block is a Bottleneck;

[0014] The expression of the basic feature map extractor is:

[0015] In the formula, O represents the basic feature map output by the basic feature map extractor, N represents the number of layers, σ represents the ReLu activation function, and D n represents the extended convolution, f(x) represents the feature of the input infrared image at the nth layer, x represents the input infrared image, It means that the first layer has not undergone dilated convolution operation, and the dilated convolution from the second layer is overlapped with the Bottleneck.

[0016] Further, the semantic tag converter includes a semantic tagger, a Transformer encoder and a Transformer decoder;

[0017] The semantic tagger uses point-by-point convolution to obtain a spatial attention map for the input basic feature map, and then calculates the weighted average of pixels through the spatial attention map to obtain a semantic tag set;

[0018] Introducing multi-head self-attention and multi-layer perceptron into the Transformer encoder, performing feature encoding processing on the semantic tag set, and obtaining encoded features;

[0019] A Siamese Transformer decoder is used in the Transformer decoder, and multi-head cross attention is introduced to decode the encoded features to obtain an enriched feature map.

[0020] Furthermore, the expression of the semantic tagger is:

[0021] Where Y represents the semantic tag set output by the semantic tagger, A represents the spatial attention map, and A∈R HW×N , R represents the expansion factor in the spatial resolution dimension, N represents the number of layers, H represents the image height, W represents the learnable kernel, W∈R C×N , C represents the number of feature map channels, A T represents the transpose of A, softmax represents the normalization function, represents a point-by-point convolution operation, and O represents the basic feature map output by the basic feature map extractor.

[0022] Furthermore, the expression of the multi-head self-attention is:

[0023] MSA(Y N-1 )=Concat(head′1,...,head′ h )W O ;In the formula, MSA(Y N-1 ) represents the output result of multi-head self-attention, Y N-1 represents the semantic tag set of the N-1th layer output by the semantic tagger, Concat represents tensor concatenation, head′1,...,head′ h represents the features from the first multi-head self-attention output to the features of the h-th head self-attention output, represents the feature of the j-th self-attention output, Att represents the attention mechanism, represents the learnable parameters of the three linear projection layers, d represents the channel dimension of the triplet, and W O ∈R C×hd represents the matrix of the linear projection layer, q, k, and v represent the triplets obtained from the semantic tag set Y, and h represents the number of self-attention heads.

[0024] Furthermore, the expression of the multi-head cross attention is:

[0025] MCA(Y N-1 ,Y new )=Concat(head1,...,head h )W O ; Where MCA represents the enriched feature map output by multi-head cross attention, Y N-1 represents the semantic tag set of the N-1th layer output by the semantic tagger, Concat represents tensor concatenation, and Y new represents the new semantic tag set, Y new =(head1,...,head h ), head1,...,head h represents the features from the first head cross attention output to the hth head cross attention output, W O Share weights with multi-head self-attention MSA.

[0026] Furthermore, the local spatial semantic attention module includes:

[0027] The local block technology is used to divide the enriched feature map into several feature blocks;

[0028] Based on the feature map, spatial attention is used to aggregate semantic information and spatial information to obtain an enhanced feature map.

[0029] Furthermore, the expression of the spatial attention is:

[0030] S = Sigmoid(f[AvgPooling(SP),MaxPooling(SP)])×SE+SE; where S represents the enhanced feature map output by the spatial attention, Sigmoid represents the activation function, f represents the 7×7 convolutional layer, SP represents the spatial information path feature map, SE represents the semantic feature map, AvgPooling(SP) represents the average pooling of each pixel in the spatial information path feature map along the channel axis, and MaxPooling(SP) represents the maximum pooling of each pixel in the spatial information path feature map along the channel axis.

[0031] Further, various training parameters in the process of training the model are set;

[0032] The various training parameters include neuron activation function, connection layer activation function, optimization function, loss function, number of network iterations, batch size and learning rate.

[0033] The present invention has the following beneficial effects:

[0034] The present invention provides an infrared small target detection method, which uses several public infrared image data sets to train models, and is superior to other existing partial models in detection accuracy and efficiency; a basic feature map extractor filters noise from infrared images, which can reduce the interference of complex backgrounds and obtain basic feature maps with high image quality; a semantic tag converter enriches the basic feature map, which can better represent the features of small targets in infrared images, and the obtained enriched feature map can realize contextual high-level semantic tagging, which helps to avoid missing small targets during detection; a local spatial semantic attention module performs block enhancement on the enriched feature map, which helps to realize the combination of spatial and semantic information, enhances the local features of the target, and helps to further solve the class imbalance problem of target and background pixels. Therefore, the present invention overcomes the defects of small target detection in complex environments to a certain extent through a small target detection model, and improves the accuracy and reliability of infrared small target detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1This is a flow chart of an infrared small target detection method of the present invention;

[0037] Figure 2 is a structural block diagram of a small target detection model in an embodiment of the present invention;

[0038] Figure 3 is a structural diagram of a semantic tag converter in an embodiment of the present invention;

[0039] Figure 4 is a flowchart of the implementation of the semantic tag converter in the embodiment of the present invention;

[0040] Figure 5 is a structural diagram of a Transformer encoder and a Transformer decoder in an embodiment of the present invention;

[0041] Figure 6 4 is a structural diagram of a local spatial semantic attention module in an embodiment of the present invention;

[0042] Figure 7 Implementation flow chart of the local spatial semantic attention module in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the technical solutions proposed by the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0044] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0045] The embodiment of the present invention provides a method for detecting small infrared targets. Figure 1 As shown, the following steps are included:

[0046] S1, obtain several public infrared image datasets.

[0047] In this embodiment, in order to facilitate the subsequent training of the model, several public infrared image data sets are obtained. The public infrared image data set refers to a collection of existing publicly available infrared image data. The number of data sets can be set to three. The three public infrared image data sets can be three different types of public infrared image data sets, such as: ROSI (Remote Sensing Object Detection and Tracking). It should be noted that the current test set is one of the three data sets. The implementer can set the test order according to the actual situation without specific limitation.

[0048] S2, use several public infrared image datasets to train the model and obtain a small target detection model.

[0049] In this embodiment, three public infrared image data sets are used as training data of the model, that is, as input data sets of the model; various training parameters of the model are set during the training process, and each training parameter includes: neuron activation function, fully connected layer activation function, optimization function, loss function, number of network iterations, batch size and learning rate. Specifically, the configuration of each training parameter is shown in Table 1:

[0050] Table 1

[0051]

[0052] In Table 1, the feature dimensions of the model input are 3, which are the number of channels, the highest temperature at height, and the width;

[0053] The neuron activation function is the Relu activation function;

[0054] The network training optimization function is the Adam function, and the loss function uses binary cross entropy to solve the regression problem;

[0055] The model iteration number epochs = 150, and 20 samples are used to update the weights in each batch, that is, the batch_size value is 20;

[0056] The model learning rate is 0.001.

[0057] It should be noted that the specific training process of the small target detection model is a prior art and is not within the scope of protection of the present invention, and will not be elaborated here.

[0058] So far, this implementation has obtained a small target detection model for detecting small infrared targets.

[0059] Among them, the structural diagram of the small target detection model is as follows Figure 2As shown, it includes a basic feature map extractor 201, a semantic tag converter 202 and a local space semantic attention module 203.

[0060] The basic feature map extractor 201 is used to filter noise from the input infrared image to obtain a basic feature map, which effectively suppresses complex background information.

[0061] Here, since the size of the target object in the infrared image is relatively small and the size of the small target in the image is inconsistent, the proportion of the target and the background is extremely unbalanced. At this time, the detection effect of using only the convolutional network to detect small targets is poor. In order to overcome the defect of the poor small target detection effect of the existing convolutional network, this embodiment proposes a Bottleneck residual block with dilated convolution, and extracts basic information through the interaction of the residual block and the dilated convolution.

[0062] Specifically, the basic feature map extractor includes a residual block and a dilated convolution, the residual block is a Bottleneck, and the expression of the basic feature map extractor here is:

[0063] In the formula, O represents the basic feature map output by the basic feature map extractor, N represents the number of layers, σ represents the ReLu activation function, and D n represents the extended convolution, f(x) represents the feature of the input infrared image at the nth layer, x represents the input infrared image, It means that the first layer has not undergone dilated convolution operation, and the dilated convolution from the second layer is overlapped with the Bottleneck.

[0064] It should be noted that the Bottleneck residual block usually contains three convolutional layers, namely a 1×1 convolutional layer, a 3×3 convolutional layer and another 1×1 convolutional layer. It maintains the expressiveness of the network by reducing and increasing dimensions, which helps to reduce the computational cost. The Bottleneck structure residual block and the subsequent captured comprehensive contextual information can reduce the missed detection rate of small targets.

[0065] The semantic tag converter 202 is used to enrich the input basic feature map to obtain an enriched feature map.

[0066] Here, the structure of the semantic tag converter is as follows Figure 3As shown in the figure, small targets in infrared images usually appear as dim image features, which are easily disturbed by complex backgrounds during detection, resulting in a weakened small target feature representation capability. Since context modeling in space and time is of great significance to improving the small target feature representation capability, this embodiment first uses a semantic tagger to refine the original features of small targets through high-level semantic tags, and then uses a Transformer encoder to model the context in space and time based on compact tags. Finally, the learned context-rich tags are fed back to the pixel space to refine the original features through a Transformer decoder.

[0067] The semantic tag converter 202 can be Figure 4 The steps shown achieve:

[0068] S301, the semantic tagger uses point-by-point convolution to obtain a spatial attention map for the input basic feature map, and then calculates the weighted average of pixels through the spatial attention map to obtain a semantic tag set.

[0069] In this embodiment, the idea of ​​word segmentation in natural language processing is adopted to segment the basic feature map into several visual words and represent them with word vectors. In order to obtain compact semantic tags, the semantic tagger spatially pools the basic feature map into a set of features, namely, the semantic tag set, by learning a set of spatial attention maps.

[0070] Specifically, by applying point-by-point convolution to each pixel on the basic feature map, groups representing semantic concepts are obtained; the HW dimension of each semantic group is calculated using a normalization function to obtain a spatial attention map; in order to obtain a semantic tag set, the attention map is used to calculate the weighted average sum of pixels to obtain a semantic tag set, that is, a compact vocabulary set.

[0071] The expression of the semantic marker in this embodiment may be:

[0072] Where Y represents the semantic tag set output by the semantic tagger, A represents the spatial attention map, and A∈R HW×N , R represents the expansion factor in the spatial resolution dimension, N represents the number of layers, H represents the image height, W represents the learnable kernel, W∈R C×N , C represents the number of feature map channels, A T represents the transpose of A, softmax represents the normalization function, represents a point-by-point convolution operation, and O represents the basic feature map output by the basic feature map extractor.

[0073] S302, introducing multi-head self-attention and multi-layer perceptron into the Transformer encoder, performing feature encoding processing on the semantic tag set, and obtaining encoded features.

[0074] In this embodiment, a Transformer encoder is designed to perform context modeling on the obtained semantic tag set, generate rich tag representations for each moment, that is, the semantic tag set is used as the input of the Transformer encoder, and finally a new semantic tag set is obtained. The Transformer encoder consists of several layers of multi-head self-attention (MSA) and multi-layer perceptron (MLP). According to the ViT design, normalization is used before MSA / MLP, for example, using a pre-norm residual unit, which is more stable and capable than other means.

[0075] Among them, the expression of multi-head self-attention in this embodiment is:

[0076] MSA(Y N-1 )=Concat(head′1,...,head′ h )W O ;In the formula, MSA(Y N-1 ) represents the output result of multi-head self-attention, Y N-1 represents the semantic tag set of the N-1th layer output by the semantic tagger, Concat represents tensor concatenation, head′1,...,head′ h represents the features from the first multi-head self-attention output to the features of the h-th head self-attention output, represents the feature of the j-th self-attention output, Att represents the attention mechanism, represents the learnable parameters of the three linear projection layers, d represents the channel dimension of the triplet, and W O ∈R C×hd represents the matrix of the linear projection layer, q, k, and v represent the triplets obtained from the semantic tag set Y, and h represents the number of self-attention heads.

[0077] It should be noted that in the Transformer encoder, the same configuration of PerNorm and MLP is used; position information helps to model the context, so the learnable position needs to be embedded or added to the semantic tag set to encode the relative or absolute position of the elements in the token sequence, and make full use of the relevant context information to obtain a semantically rich tag set.

[0078] S303, using the Siamese Transformer decoder in the Transformer decoder, and introducing multi-head cross attention, decoding the encoded features to obtain an enriched feature map.

[0079] In this embodiment, the Transformer encoder obtains a context-rich semantic tag set, which can well represent the location information of small objects. In order to obtain processable pixel-level features, the abstract semantic tags need to be remapped back to the pixel space. Specifically, the Siamese Transformer decoder is used to refine the pixel features at each time. The basic feature map and the new semantic tag set are used as the input of the Transformer decoder to obtain a rich and refined feature map, that is, an enriched feature map. Each pixel in the enriched feature map can be represented by a combination of compact semantic tags. In the Transformer decoder, the pixels in Y are regarded as queries and the tags are regarded as keys. The structure diagram of the Transformer encoder and the Transformer decoder is shown in FIG. Figure 5 shown.

[0080] The expression of multi-head cross attention in this embodiment can be:

[0081] MCA(Y N-1 ,Y new )=Concat(head1,...,head h )W O ; Where MCA represents the enriched feature map output by multi-head cross attention, Y N-1 represents the semantic tag set of the N-1th layer output by the semantic tagger, Concat represents tensor concatenation, and Y new represents the new semantic tag set, Y new =(head1,...,head h ), head1,...,head h represents the features from the first head cross attention output to the hth head cross attention output, W O Share weights with multi-head self-attention MSA.

[0082] It should be noted that, unlike the MSA in the Transformer encoder, the multi-head cross-attention query in the Transformer decoder comes from the basic feature image, and the keys and values ​​come from the new semantic tag set. Therefore, the Transformer decoder can map the enriched contextual semantic information back to the original space to enhance the small object representation capability.

[0083] The local spatial semantic attention module 203 is used to perform block enhancement on the input enriched feature map to obtain an enhanced feature map.

[0084] Here, due to the sparsity of small targets and the extreme class imbalance between small targets and background, this embodiment adds a local spatial semantic attention module (Patch Space Attention Module, PSAM) to the small target detection model to improve the detection performance by using the spatial semantic information of small targets. The structure of the local spatial semantic attention module is shown in FIG. Figure 6 .

[0085] The local space semantic attention module 203 can be Figure 7 The steps shown achieve:

[0086] S401, using local block technology to divide the enriched feature map into a number of feature blocks.

[0087] In this embodiment, the enriched feature map is divided into several feature blocks in a block manner to obtain multi-scale small target features, which helps to reduce the imbalance between the target and background pixel proportions.

[0088] S402, based on the feature block, using spatial attention to aggregate semantic information and spatial information to obtain an enhanced feature map.

[0089] In this embodiment, the local context information of the small target is obtained by aggregating semantic information and spatial information through spatial attention. The spatial attention draws on the idea of ​​the Convolutional Block Attention Module (CBAM) to aggregate semantic information and spatial information to enhance the target features. Among them, paying attention to spatial information helps to improve the performance of small target detection.

[0090] Specifically, through average pooling and maximum pooling, the spatial information of the spatial information path feature map is combined to obtain the spatial position information feature; the Sigmoid activation function is used to obtain the spatial weight, and the spatial weight is multiplied by the corresponding spatial position information to obtain the spatial attention feature map; the spatial attention feature map is summed with the corresponding semantic feature information to obtain the spatial semantic attention map, and the spatial semantic attention map is used as the enhanced feature map.

[0091] The expression of spatial attention in this embodiment can be:

[0092] S = Sigmoid(f[AvgPooling(SP),MaxPooling(SP)])×SE+SE; where S represents the enhanced feature map output by the spatial attention, Sigmoid represents the activation function, f represents the 7×7 convolutional layer, SP represents the spatial information path feature map, SE represents the semantic feature map, AvgPooling(SP) represents the average pooling of each pixel in the spatial information path feature map along the channel axis, and MaxPooling(SP) represents the maximum pooling of each pixel in the spatial information path feature map along the channel axis.

[0093] S3, performing small target detection on the infrared image to be detected through the small target detection model to obtain the detection result.

[0094] In this embodiment, the infrared image to be detected is input into a trained small target detection model, and the small target detection model outputs the detection result of the infrared image to be detected.

[0095] Thus, this embodiment achieves more accurate infrared small target detection under complex background.

[0096] The present invention provides an infrared small target detection method, which uses several public infrared image data sets to train models, and is superior to other existing partial models in detection accuracy and efficiency; a basic feature map extractor filters noise from infrared images, which can reduce the interference of complex backgrounds and obtain basic feature maps with high image quality; a semantic tag converter enriches the basic feature map, which can better represent the features of small targets in infrared images, and the obtained enriched feature map can realize contextual high-level semantic tagging, which helps to avoid missing small targets during detection; a local spatial semantic attention module performs block enhancement on the enriched feature map, which helps to realize the combination of spatial and semantic information, enhances the local features of the target, and helps to further solve the class imbalance problem of target and background pixels. Therefore, the present invention overcomes the defects of small target detection in complex environments to a certain extent through a small target detection model, and improves the accuracy and reliability of infrared small target detection results.

[0097] In order to verify the detection effect of the infrared small target detection method provided in this embodiment, the following experiment is performed.

[0098] (1) Determine the evaluation indicators.

[0099] Here, target-level evaluation indicators and deep learning pixel-level evaluation indicators are used as evaluation indicators, among which the target-level evaluation indicators include target-level accuracy (Target Precision, TarPrec), target-level recall (TargetRecall, TarRec), target-level F1 value (Target F1, TarF1T ) ; Pixel-level evaluation indicators include pixel-level recall rate (Pixel Recall, PxlRec), pixel-level F1 value (Pixel F1, PxlF1 P ), detection probability, and false alarm rate.

[0100] (2) Prepare the dataset.

[0101] Here, the dataset comes from three publicly released infrared image small target datasets, namely the SIRST dataset, the MFIRST dataset, and the NUDT-SIRST dataset.

[0102] Among them, the SIRST dataset is a widely used public dataset for single-frame infrared small target detection, with 480 instances of different scenes, including 427 representative images from real-world videos; most of the small targets in the SIRST dataset are very dim, hidden in extremely complex backgrounds, and are also subject to clutter interference.

[0103] The MFIRST dataset contains 10,978 images, consisting of a large number of real and synthetic infrared images. The real infrared images come from 11 real infrared sequences (a total of 2,098 frames) and real single infrared images of 100 different small objects; the background of the synthetic infrared image is generated by cropping different areas from infrared high-resolution natural scene images, and the small targets are separated from the real infrared images or synthesized using a two-dimensional Gaussian function; the MFIRST dataset has a large number of heterogeneous scenes, a wide coverage, and a larger number of samples than other datasets. It is currently the largest dataset and has strong representativeness.

[0104] The NUDT-SIRST dataset contains targets of various sizes, targets of various categories, cluttered and diverse backgrounds, and accurate annotations. It is a newly developed synthetic infrared small target dataset with a total of 1,327 images; each image is synthesized from real backgrounds and various targets under various shooting machines and rich poses. The NUDT-SIRST dataset contains five main background scenes: city, field, highlight, ocean, and cloud, creating more challenging scenes of multiple targets, point targets, and dim targets.

[0105] (3) Set up the experiment.

[0106] Here, the experiment is conducted on a computer with 40GB of memory on the PyTorch platform. The programming language and version are Python 3.9, the framework and corresponding version are Pytorch 2.0.1, and the development tool is PyCharm Community Edition 2023.1.4.

[0107] (4) Comparative experiment

[0108] Here, mathematical morphology filtering TopHat, maximum and maximum median filtering methods MaxMean\MaxMedian, average absolute grayscale difference algorithm AAGD, absolute directional average difference algorithm ADMD, generalized structure tensor GST, scale block contrast algorithm MPCM, multi-scale patch-based contrast method MPCM, infrared patch image IPI, local intensity gradient algorithm LIG, iterative linear convolution matching algorithm ILCM, three-layer linear convolution matching algorithm TLLCM, local energy fraction algorithm LEF, asymmetric context modulation method ACM, global and local multi-scale detection of small targets method LPNet are used as comparison methods. Among them, TopHat, MaxMean\MaxMedian, AAGD, ADMD, GST, MPCM, MPCM, IPI, LIG, ILCM, TLLCM and LEF are traditional methods, and MDvsFA-cGAN, ACM and LPNet are deep learning methods.

[0109] On the SIRST dataset, the comparative experimental model evaluation results are shown in Table 2:

[0110] Table 2

[0111]

[0112] In Table 2, the model-driven method uses specific expert knowledge to show good results on this dataset; the AUC value is reduced due to the large difference in data distribution between the designed test set and the training set; the high complexity of the MDvsFA-cGAN model cannot be applied to the limited SIRST dataset and no reasonable evaluation value can be obtained; the data-driven method has a strong feature expression ability, and its detection results are significantly higher than the model-driven method.

[0113] The STPSA-Net model (small target detection model) provided in this embodiment is superior to other comparison methods and obtains the best value detection in small targets in infrared images, such as F1 T Improved by 2.06%, F1 P Improved by 0.10%, P d Improved by 2.60%; for the small-scale real SIRST dataset, STPSA-Net enhances the feature expression ability, improves the contrast between small background targets, and reduces the missed detection rate of small target detection.

[0114] On the MFIRST dataset, the comparative experimental model evaluation results are shown in Table 3:

[0115] Table 3

[0116]

[0117] In Table 3, due to the large scale of the MFIRST dataset, the model-driven method has poor robustness to a large number of different scenarios and performs much worse than the data-driven method.

[0118] The STPSA-Net model provided in this embodiment improves AUC by 1.53% compared with LPNet. d The STPSA-Net model has a 5.71% improvement over LPNet, which shows that the STPSA-Net model has good performance for a large number of different scenes and has a high ability to detect small targets. In addition, the STPSA-Net model has 300 epochs, which is half the training efficiency of LPNet. Although the F1 T 、F1 P Similar to LPNet, but with lower AUC and P d It is still relatively high, which shows that the STPSA-Net model has strong generalization ability.

[0119] On the NUDT-SIRST dataset, the comparative experimental model evaluation results are shown in Table 4:

[0120] Table 4

[0121]

[0122] In Table 4, the NUDT-SIRST dataset has images of different sized targets and cluttered backgrounds; the model-driven method did not obtain reasonable values ​​on this dataset; the STPSA-Net model achieved significant improvements compared to the LPNet method; overall, F1 T 、F1 P , P d and AUC were improved by 0.05%, 2.48%, 37.41% and 1.47% respectively.

[0123] Among them, the AUC value is significantly higher than that of LPNet, because STPSA-Net focuses on the local small target area of ​​interest and has strong learning and feature representation capabilities, which to a certain extent produces better detection performance and lower missed detection rate.

[0124] Therefore, through Table 2, Table 3 and Table 4, it can be concluded that the infrared small target detection method of the present invention can better detect small targets in infrared images under complex backgrounds and has stronger expression and feature extraction capabilities.

[0125] (5) Ablation experiment

[0126] Here, in order to explore the contribution of each component of the STPSA-Net model to the infrared small target detection performance, an ablation experiment was conducted. The STPSA-Net model consists of a basic feature extractor, a semantic tag converter, and a local spatial semantic attention module. These components will be gradually removed to evaluate their impact on the model performance. Specifically:

[0127] The small target detection model that does not include the local spatial semantic attention module removes the processing steps of the PSAM (local spatial semantic attention module) algorithm in the STPSA-Net model, uses the normalization method for processing, and retains the semantic tag converter technology; the small target detection model that does not include the semantic tag converter removes the STT (semantic tag converter) technology in the STPSA-Net model and retains the local spatial semantic attention module.

[0128] The evaluation results of the ablation experiment model are shown in Table 5:

[0129] Table 5

[0130]

[0131] In Table 5, on the SIRST dataset, without the PSAM module, the F1T value decreased by 5.92%, the F1P value decreased by 2.6%, the AUC value decreased by 4.28%, and the Pd value decreased by 6.06%. Similar results were also seen on the MFIRST and NUDT-SIRST datasets, and the values ​​of the four evaluation indicators all decreased. This is mainly because the lack of PSAM blocks makes it impossible to effectively combine space and semantics, and it is difficult to deal with the extreme imbalance between target and background pixels.

[0132] On the SIRST, MFIRST and NUDT-SIRST datasets, the values ​​of the four indicators without the STT module are all reduced by about 2%, which proves that the STT technology has a good ability to capture the fine-grained features of small targets and has a good segmentation effect on the shapes of small targets.

[0133] The ablation experiment shows the effectiveness of each component of the STPST-Net module in this embodiment, which means that it is very necessary to combine the semantic tag converter with the local spatial semantic attention module for infrared small target detection.

[0134] This is the end of the present invention.

[0135] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting small infrared targets. It is characterized in that The following steps are involved: Obtain several public infrared image datasets; Using the several public infrared image data sets to train a model, a small target detection model is obtained; Performing small target detection on the infrared image to be detected by using the small target detection model to obtain a detection result; The small target detection model includes a basic feature map extractor, a semantic tag converter and a local space semantic attention module; The basic feature map extractor is used to filter noise from the input infrared image to obtain a basic feature map; The semantic tag converter is used to perform feature enrichment on the input basic feature map to obtain an enriched feature map; The local spatial semantic attention module is used to perform block enhancement on the input enriched feature map to obtain an enhanced feature map.

2. A method for detecting small infrared targets according to claim 1, It is characterized in that The basic feature map extractor includes a residual block and a dilated convolution, and the residual block is a Bottleneck; The expression of the basic feature map extractor is: In the formula, O represents the basic feature map output by the basic feature map extractor, N represents the number of layers, σ represents the ReLu activation function, and D n represents the extended convolution, f(x) represents the feature of the input infrared image at the nth layer, x represents the input infrared image, It means that the first layer has not undergone dilated convolution operation, and the dilated convolution from the second layer is overlapped with the Bottleneck.

3. The infrared small target detection method according to claim 1, It is characterized in that The semantic tag converter includes a semantic tagger, a Transformer encoder and a Transformer decoder; The semantic tagger uses point-by-point convolution to obtain a spatial attention map for the input basic feature map, and then calculates the weighted average of pixels through the spatial attention map to obtain a semantic tag set; Introducing multi-head self-attention and multi-layer perceptron into the Transformer encoder, performing feature encoding processing on the semantic tag set, and obtaining encoded features; A Siamese Transformer decoder is used in the Transformer decoder, and multi-head cross attention is introduced to decode the encoded features to obtain an enriched feature map.

4. A method for detecting small infrared targets according to claim 3, It is characterized in that The expression of the semantic tagger is: Where Y represents the semantic tag set output by the semantic tagger, A represents the spatial attention map, and A∈R HW×N , R represents the expansion factor in the spatial resolution dimension, N represents the number of layers, H represents the image height, W represents the learnable kernel, W∈R C×N , C represents the number of feature map channels, A T represents the transpose of A, softmax represents the normalization function, represents a point-by-point convolution operation, and O represents the basic feature map output by the basic feature map extractor.

5. The infrared small target detection method according to claim 3, It is characterized in that The expression of the multi-head self-attention is: MSA(Y N-1 ) = Concat(head′ 1 ,...,head′ h )W O ;In the formula, MSA(Y N-1 ) represents the output result of multi-head self-attention, Y N-1 represents the semantic tag set of the N-1th layer output by the semantic tagger, Concat represents tensor concatenation, head′ 1 ,...,head′ h represents the features from the first multi-head self-attention output to the features of the h-th head self-attention output, head′ j =Att(Y N-1 W j q ,Y N-1 W j k ,Y N-1 W j v ) represents the feature of the j-th self-attention output, Att represents the attention mechanism, W j q ,W j k ,W j v ∈R C×d represents the learnable parameters of the three linear projection layers, d represents the channel dimension of the triplet, and W O ∈R C ×hd represents the matrix of the linear projection layer, q, k, and v represent the triplets obtained from the semantic tag set Y, and h represents the number of self-attention heads.

6. A method for detecting small infrared targets according to claim 3, It is characterized in that The expression of the multi-head cross attention is: MCA(Y N-1 ,Y new )=Concat(head 1 ,...,head h )W O ; Where MCA represents the enriched feature map output by multi-head cross attention, Y N-1 represents the semantic tag set of the N-1th layer output by the semantic tagger, Concat represents tensor concatenation, and Y new represents the new semantic tag set, Y new =(head 1 ,...,head h ), head 1 ,...,head h represents the features from the first head cross attention output to the hth head cross attention output, W O Share weights with multi-head self-attention MSA.

7. The infrared small target detection method according to claim 1, It is characterized in that The local spatial semantic attention module includes: The local block technology is used to divide the enriched feature map into several feature blocks; Based on the feature map, spatial attention is used to aggregate semantic information and spatial information to obtain an enhanced feature map.

8. A method for detecting small infrared targets according to claim 7, It is characterized in that The expression of the spatial attention is: S = Sigmoid(f[AvgPooling(SP),MaxPooling(SP)])×SE+SE; where S represents the enhanced feature map output by the spatial attention, Sigmoid represents the activation function, f represents the 7×7 convolutional layer, SP represents the spatial information path feature map, SE represents the semantic feature map, AvgPooling(SP) represents the average pooling of each pixel in the spatial information path feature map along the channel axis, and MaxPooling(SP) represents the maximum pooling of each pixel in the spatial information path feature map along the channel axis.

9. The infrared small target detection method according to claim 1, It is characterized in that Set various training parameters in the process of training the model; The various training parameters include neuron activation function, connection layer activation function, optimization function, loss function, number of network iterations, batch size and learning rate.

Images