Infrared small target detection method, system, equipment and medium based on IST-DETR

Abstract

The present invention discloses an infrared small target detection method, system, device and medium based on IST-DETR, which uses a backbone network to extract a multi-scale feature map of an infrared small target, uses a hybrid encoder including an MSF module to adapt to the multi-scale feature map of the infrared small target, outputs an infrared image code of feature expression, and uses a decoder with an auxiliary prediction head to determine the position and category of the infrared small target according to the infrared image code of feature expression. The present invention obtains the global features of the infrared target through the backbone network, uses a hybrid encoder including an MSF module to enhance the detection capability of the model, fully considers the actual features of the infrared image, and improves the detection effect. The model of the present invention significantly reduces the complexity of infrared small target detection and improves the detection efficiency.

Description

Technical Field

[0001] The present invention relates to the field of infrared target detection technology, and in particular to an infrared small target detection method, system, equipment and medium based on IST-DETR. Background Art

[0002] In recent years, infrared target detection technology has been widely used. This technology is mainly used in vehicle assisted driving, electrical equipment thermal fault detection, disaster relief, real-time monitoring of war, etc. Infrared imaging technology has strong anti-interference and all-weather working characteristics. When the target is in complex environments such as night, fog, and rain, the visible light image method cannot obtain clear imaging. However, compared with visible light, infrared images often have the characteristics of lack of color information, less texture details, low resolution and small targets, and the accuracy of target detection technology depends in part on this. Therefore, target detection in infrared images is a challenging and attractive topic.

[0003] To overcome the limitations, the Transformer model architecture from the field of natural language processing can capture the connection between remote information through self-attention, thereby making better use of contextual information. In 2020, Carion et al. proposed a new Transformer target detection framework DETR (detection transformer), which laid an important foundation for the application of Transformer in target detection tasks. DETR is implemented end-to-end on the target detection model, which eliminates the non-maximum suppression component (NMS), greatly simplifies the target detection pipeline, and better adapts to targets of different scales, shapes, and numbers. Due to the large amount of computation required by DETRs, slow training convergence, and high optimization difficulty, the application of DETRs in infrared target detection is limited. RT-DETR is superior to YOLOV8 of the same scale in speed and accuracy due to its designed hybrid encoder and IoU-aware query selection.

[0004] The YOLO series is an efficient, accurate, and easy-to-deploy target detection algorithm that is widely used in industry. Therefore, it is an indispensable part of infrared target detection research. Zhang et al. used the YOLOv3 model to detect vehicles in infrared targets and achieved satisfactory detection results with a small network. In order to solve the interference factors around the infrared detection target, Li et al. first obtained the zoom area of ​​the target from the background and then determined the position of the target, which enhanced the robustness and effectiveness of the algorithm. In order to further reduce the interference of the background, Wei et al. improved the YOLOv5 algorithm and used the attention mechanism to ignore irrelevant information and focus on key information to enhance the target detection accuracy. Facing the same problem, Guo et al. tried to introduce the CBAM attention mechanism in the backbone network and the Neck network to enhance the small target positioning ability, and replaced the original module with the ASPP module in the Neck network, which had a significant improvement in detection accuracy. Because the infrared detection distance will not remain unchanged, when the observation distance is far, the infrared target detection performance will drop sharply. To solve this kind of problem, Yang et al. proposed an improved YOLOv4 algorithm My_YOLOv4. However, the problems of insufficient spatial resolution, low signal-to-noise ratio, and lack of texture details of small infrared targets are still difficult to solve. Firdiantika et al. combined Coordinate Attention (CA) from a lightweight network with YOLOv7 to reduce the instability of small target detection. Min et al. improved the backbone of YOLOv4 to CSPDarknet53-Deeper to enhance the feature extraction effect of small targets because of the slow detection speed and low accuracy of traditional target detection methods. When there are no clear visual clues, Zhao et al. tried to use the improved UNet and YOLO networks to share visible light information to achieve the effect of improving detection accuracy. In order to reduce the computational complexity of the model and improve the inference speed of the model, Yin et al. replaced the backbone of YOLOv5 with the EMO architecture and added the Cotention module, providing an efficient and accurate infrared image recognition method. To address the problem of difficulty in real-time detection of infrared targets due to the large temperature difference between day and night in infrared images, Song et al. proposed three full-time infrared pedestrian detection models based on CSP (Center and Scale Prediction) networks, which successfully reduced the false detection rate during daytime and nighttime detection.

[0005] At present, the research on infrared target detection mainly uses convolutional neural networks (CNNs) to extract features and then locate and classify targets. This method is affected by the convolution operator and cannot learn the global features of infrared images, which hinders the accurate positioning and classification of infrared targets. In addition, the detection efficiency of infrared images is significantly limited by the following challenges: compared with visible light images, the resolution of infrared images is lower, combined with the usage scenarios of infrared imaging technology, which leads to insufficient features of infrared targets; in different usage scenarios, the sizes of infrared targets are different, resulting in scale differences between infrared targets; when training infrared image samples, the accuracy is easily reduced due to uneven sample types and numbers. Therefore, although these research methods are effective in some aspects of infrared target detection, they do not fully consider the actual characteristics of infrared images and do not consider infrared small target detection from a real-time perspective, which limits the detection effect of the model. Summary of the invention

[0006] The purpose of the present invention is to provide an infrared small target detection method, system, device and medium based on IST-DETR, which fully considers the actual characteristics of infrared images and improves the detection effect of infrared small target detection.

[0007] To achieve the above purpose, the technical solution provided by the present invention is:

[0008] An infrared small target detection method based on IST-DETR comprises the following steps:

[0009] The backbone network is used to extract multi-scale feature maps of small infrared targets;

[0010] A hybrid encoder including the MSF module is used to adapt to the multi-scale feature map of small infrared targets and output the infrared image encoding of feature expression;

[0011] A decoder with an auxiliary prediction head is used to determine the position and category of small infrared targets based on the infrared image encoding expressed by features.

[0012] In order to optimize the above technical solutions, the specific measures / limitations taken also include:

[0013] The ResNet-18 network is adopted as the backbone network, and the output features of the last three stages {P3, P4, P5} of the backbone are used as the input of the hybrid encoder.

[0014] The MSF module includes a feature selection module and a feature fusion module. The feature selection module is weighted by a CA module: the CA module performs global maximum pooling and global average pooling on the multi-scale feature map, calculates the maximum value and average value of each channel respectively, adds the generated feature maximum value and average value, and then uses an activation function to obtain the weight of each channel; the channel weight generated by the CA module is multiplied by the feature map of the corresponding scale to obtain a filtered feature map, and the number of channels of each layer of the feature map is ensured to be consistent through convolution.

[0015] The feature fusion module performs feature fusion on the multi-scale feature maps of the infrared small target: after the feature maps of different scales are weighted by the feature selection module, the feature fusion module is used to perform top-down feature fusion, share the subtle features of the infrared small target, and then perform bottom-up information sharing to collect the position information of the infrared small target; the screening feature fusion mechanism is as follows: high-level semantic features are used as weights for filtering low-level features, the position information of low-level features is used to filter high-level semantic features, and the filtered image features are added point by point to the original image features to achieve multi-scale feature fusion.

[0016] The hybrid coding has an internal feature map interaction module for learning self-attention features, and only the high-level semantic feature P5 output by the backbone network is fed into the internal feature map interaction module.

[0017] After the MSF module outputs the feature-fused infrared image, the image features are selected from the infrared image sequence in the feature map output by the MSF module as the initial query object of the decoder through the improved IoU loss function; the formula of the improved IoU loss function is as follows:

[0018] L final =(1-α)L Inner-MPDIoU +αL nwd ;

[0019] Among them, L final represents the improved IoU loss function, L Inner-MPDIoU represents the bounding box loss function calculated by combining Inner-Iou and MPDIoU, L nwd represents the similarity function calculated by the NWD algorithm, and α represents the loss ratio adjustment constant.

[0020] The effectiveness of the detection method is verified using the public infrared dataset FLIR. When selecting image features through the improved IoU loss function, the EMA weighting function is used to emphasize the samples at the boundary of the public infrared dataset FLIR. The formula of the EMA weighting function is as follows:

[0021] Δ t =μ·Δt-1 +(1-μ)·θ t

[0022]

[0023] In the formula, θ t Represents the size of the improved IoU loss function at time t, μ is a constant, representing the previous The average of the IoU loss function values.

[0024] The present invention also provides an infrared small target detection system based on IST-DETR, comprising:

[0025] A feature map extraction module is used to extract multi-scale feature maps of small infrared targets using a backbone network;

[0026] A feature expression module is used to adopt a hybrid encoder including an MSF module to adapt to the multi-scale feature map of small infrared targets and output an infrared image of feature expression;

[0027] A loss function calculation module, used for selecting image features from the infrared image sequence expressed by the features output by the encoder as an initial query object of the decoder through the loss function;

[0028] The detection result acquisition module is used to determine the position and category of the small infrared target according to the infrared image expressed by the features by using a decoder with an auxiliary prediction head.

[0029] The present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the infrared small target detection method based on IST-DETR as described above is implemented.

[0030] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the infrared small target detection method based on IST-DETR as described above.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention provides an infrared small target detection method, system, device and medium based on IST (Infrared Small Target)-DETR (Detection Transformer), which obtains the global features of the infrared target through a backbone network, encodes the image using a hybrid encoder including an MSF module, and obtains the position and category of the infrared target using a decoder, wherein the detection capability of the model is enhanced by using the MSF module, the actual features of the infrared image are fully considered, and the detection effect is improved.

[0033] The present invention also introduces an improved IoU loss function, and emphasizes samples at the boundaries of the image feature data set through the EMA weighting function, thereby enhancing the detection rate of the model for small infrared targets.

[0034] The infrared small target detection method based on IST-DETR of the present invention significantly reduces the complexity of infrared small target detection and improves the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 : Schematic diagram of the overall architecture of the infrared small target detection system based on IST-DETR of the present invention.

[0036] Figure 2 : A schematic diagram of the structure of the Mutual Screening-feature (MSF) module of the present invention.

[0037] Figure 3 : A schematic diagram of the structure of the screening feature fusion mechanism (SFF) of the present invention.

[0038] Figure 4 : A schematic structural diagram of the Exponential moving average (EMA) weighted function of the present invention. DETAILED DESCRIPTION

[0039] The above contents of the present invention are further described in detail below in the form of embodiments, but this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the above contents of the present invention belong to the scope of the present invention.

[0040] The present invention proposes an infrared small target detection method based on IST-DETR, comprising the following steps:

[0041] The backbone network is used to extract multi-scale feature maps of small infrared targets;

[0042] A hybrid encoder including the MSF module is used to adapt to the multi-scale feature map of small infrared targets and output the infrared image encoding of feature expression;

[0043] A decoder with an auxiliary prediction head is used to determine the position and category of small infrared targets based on the infrared image encoding expressed by features.

[0044] The overall system architecture of the infrared small target detection method based on IST-DETR of the present invention is as follows: Figure 1As shown in the figure, it consists of three key components: a backbone network, a hybrid encoder containing a Mutual Screening-feature (MSF) module, and a decoder with an auxiliary prediction head. The auxiliary prediction head is responsible for predicting the bounding box and category of the target.

[0045] The backbone network is mainly used to extract multi-scale image features of infrared targets to facilitate feature fusion in subsequent processing; MSF is a feature fusion module designed and improved to adapt to the characteristics of infrared targets, solving the challenges brought by limited features and size differences of infrared targets in infrared target images; this is achieved through the Channel Attention (CA) module in the MSF module, which can use high-level semantic features as weights to filter low-level features, and can also use the position information of low-level features to filter high-level semantic features; then these filtered features are added point by point to the original features to achieve multi-scale feature fusion, which ultimately improves the feature expression ability of the model; then a fixed number of image features are selected from the encoder output sequence using improved slideloss and IOU as the initial object query for the decoder; finally, the decoder with an auxiliary prediction head is used to determine the location and category of the infrared target.

[0046] Backbone network: Considering the size and computational efficiency of the algorithm model, the present invention does not use a larger backbone network, but uses ResNet-18 as the backbone network for the IST-DETR feature extraction process; ResNet-18 uses residual connections to alleviate the problem of gradient disappearance; the present invention uses the output features of the last three stages {P3, P4, P5} of the backbone as the input of the encoder, where P5 is a high-level semantic feature and P3 is a low-level semantic feature.

[0047] Hybrid Encoder (including LPE-AIFI, Mutual Screening-feature Module): In order to accelerate training convergence and enhance the detection performance of small infrared targets, Zhu et al. introduced multi-scale features and proposed a deformable attention module to reduce the amount of computation; however, although the improvement of the attention mechanism reduced the amount of computation, the large increase in the length of the input sequence still hindered the real-time implementation of DETR; Lin et al. found that the encoder accounted for 49% of GFLOPs, but only contributed 11% of AP in Deformable-DETR; Wen et al. did research on this and designed the Attention-based Intrascale Feature Interaction (AIFI), which only input the highest-level features into the encoder, which can significantly reduce the computational cost.

[0048] Different from the above methods, the present invention adopts Learned Position Encodeing (LPE) to replace the original sine function hard coding, and in order to solve the challenge of less color information inherent in infrared image datasets, a hierarchical scale feature fusion module (MSF) is developed in conjunction with the LPE-AIFI module to obtain higher accuracy and faster training convergence speed; the structure of the MSF module is as follows Figure 2 As shown in FIG. 1 , the MSF module proposed in the present invention mainly consists of two parts: (1) feature selection module; (2) feature fusion module. Initially, after the feature maps of different scales are weighted by the feature selection module, the feature fusion mechanism (SFF) is first used to perform top-down feature fusion to share the subtle features of small infrared targets, and then bottom-up information sharing is performed to collect the position information of small infrared targets, thereby enhancing the detection capability of the model.

[0049] LPE-AIFI: This invention uses a different encoder form than the previous one. The hybrid encoding has an LPE-AIFI module (internal feature map interaction module) for learning self-attention features, and changes its internal encoding form, replacing the original sinusoidal encoding form with a learnable position encoding. This is a design of a random position encoding, the formula is:

[0050] pos_emb=Random() (1)

[0051] emb=emb+pos_emb (2)

[0052] Among them, pos_emb represents the position encoding, Random() represents a random value, and emb represents a learnable position encoding.

[0053] In the preferred solution of the present invention, only P5 is sent to the LPE-AIFI module. This is not just a trade-off between accuracy and efficiency. Considering that the rich semantic information in the high-level features contributes the most AP value, although the low-level features have the position information and local details of the infrared small target, they also lead to information redundancy. This redundancy lacking semantic information has the risk of interactive duplication and confusion with high-level features; therefore, the present invention applies self-attention operation to high-level features to capture conceptual information in infrared images; and the present invention combines learnable position encoding with AIFI, assigning a unique vector to each position in the input sequence so that the model can distinguish information at different positions to make up for the lack of information in the high-level features, and then adding the learned position encoding to the sequence. This process is described as follows:

[0054] Q=K=V=Flatten(P5) (3)

[0055] F5=Reshape(Attn(Q,K,V)) (4)

[0056] Among them, Q, K, V represent Q(uery), K(ey), and V(alue) in the encoder respectively. The attention value is obtained by first calculating the similarity between Q and K and then multiplying them with V; Flatten represents flattening the image, such as a 3*3 image becomes 1*9; F5 is what P5 becomes after passing through the AIFI module, see Figure 1 ; Attn(Q,K,V) represents the calculation of the attention value of the input P5 image; Reshape represents restoring the image after Flattening to its original size, such as 1*9 becomes 3*3; where Attn represents multi-head self-attention, Reshape represents restoring the shape of the feature to the same as P5, which is the inverse operation of Flatten.

[0057] Mutual Screening-feature Module: The present invention uses a feature selection module and a feature fusion module in the mutual screening feature module. The feature selection module is mainly composed of a CA module. The CA module first processes the input feature map (including all P3, P4, and P5), where C represents the number of input channels, H represents the height of the feature map, and W represents the width of the feature map. The CA module first performs two pooling layers on the feature map - global maximum pooling and global average pooling - and then adds the generated features. Then the Sigmoid activation function is used to obtain the weight of each channel. The purpose of selecting the pooling layer in the present invention is (1) the information in the image is too large, and the main infrared target features need to be retained while reducing parameters; (2) to ensure the scale invariance of the features, hoping that only some irrelevant information is removed when the infrared image is compressed. The CA module uses global maximum pooling and global average pooling to calculate the maximum value and average value of each channel respectively. The main purpose of maximum pooling is to reduce the offset of the estimated mean and retain more texture information, while average pooling can evenly obtain all data to reduce some image losses. Therefore, the CA module combines the two to extract the most relevant information from each channel while reducing information loss. Finally, the channel weights generated by the CA module are multiplied by the feature map of the corresponding scale to obtain the filtered feature map, and finally a 1×1 convolution is performed to ensure that the number of channels of each layer of the feature map is consistent. The feature fusion module has two steps. The present invention first fuses the filtered features from top to bottom, and then fuses them from bottom to top; high-level features have rich semantic information, but target positioning information is relatively scarce; on the contrary, low-level features can provide accurate location information, but the semantic information is very limited. Faced with this problem, a common method is to directly sum the high-level features adopted from above with the low-level features, and share the semantic information to each layer. However, this method ignores the fact that not every piece of information of the high-level features is effective, and simply adds the pixel values ​​of multiple feature layers. In the face of this problem, the present invention chooses to use the SFF module. The module can use the features of one layer as weights to filter and embed features of another layer, thereby strategically fusing features, such as Figure 3 As shown, given a feature map f used as weight weight ∈R C ×H×W and a feature map to be filtered The present invention first uses a transposed convolution (T-Conv) with a convolution kernel of 1 and a step size of 2 to perform convolution on f weight Expanded, the size becomes Because the size of each layer of feature maps set by the backbone network is twice that of each other, the image dimensions of the upper and lower layers have been unified, and then After the weights are obtained through the CA module and fin Finally, the filtered features are combined with Fusion gets f out Formula (5) and formula (6) represent the process of feature screening and fusion:

[0058]

[0059] in, The definition of is the feature map after convolution and transposition; f weight is defined as a feature map in {S3,S4,S5}.

[0060] The MFS module designed in the present invention is not limited by the unidirectional information flow like the traditional top-down fusion FPN. While ensuring the performance and efficiency of the model, the scheme achieves better feature fusion in infrared target detection by using feature selection modules and feature fusion modules.

[0061] Regression Loss: Due to the image characteristics of small infrared targets, when the predicted bounding box is completely covered by the real bounding box, IoU easily loses its effectiveness. Some scholars have proposed some solutions after considering this problem. Although they effectively solve this kind of problem, they limit the convergence speed and accuracy. In the face of this kind of problem, the present invention designs an IoU loss function, which combines MPD-IoU and Inner-IoU, and uses Normalized GaussianWasserstein Distance (NWD) to enhance the detection ability of small infrared targets. The MPD-IoU formula is as follows:

[0062]

[0063] In the formula, B represents the predicted bounding box, B gt represents the ground truth box, w and h represent the width and height of the input image, d1 represents the L2 distance between the upper left corner coordinates of the two bounding boxes, and d2 represents the L2 distance between the lower right corner coordinates of the two bounding boxes. The purpose of this invention is to enhance the convergence speed of the model in infrared detection. It introduces a scaling factor ratio, where the center points of the predicted bounding box and the ground truth box are represented by (x c ,y c )and The areas of the predicted bounding box and the ground truth box are represented by S and S gtIn the experiment, the present invention found that the best effect was achieved when ratio = 0.7, and the formula used was as follows:

[0064]

[0065] Among them, b left b right b bottom b top The definition of is the left, right, up, and down offset position of the center point of the auxiliary detection frame;

[0066] The definition of is the up, down, left, and right offset position of the center point of the real detection box. The center points of the predicted bounding box and the real bounding box are represented by (x c ,y c )and The definition of ratio is that the size of the auxiliary bounding box is variable, and the fixed value 0.7 can be selected; the definition of inter is the size of the auxiliary prediction box; IoU Inner The definition of is the intersection-over-union ratio of the predicted bounding box obtained using the auxiliary prediction box and the true bounding box. The areas of the predicted box and the true bounding box are denoted by S and S gt express.

[0067] The present invention applies the Inner-IoU loss to MPD-IoU to obtain:

[0068]

[0069] Among them, L Inner-MPDIoU It is a bounding box loss function calculated by combining Inner-Iou and MPDIoU.

[0070] NWD is an algorithm for small target detection. This algorithm models two boxes as Gaussian distribution and uses Wasserstein distance to measure the similarity of the two boxes. The formula is as follows:

[0071]

[0072] Where N B and Represents the Gaussian distribution of the predicted bounding box and the Gaussian distribution of the ground truth box respectively. 2 Represents the Wasseresteion distance between two box Gaussian distributions. C represents a constant that is related to the data set.

[0073] The above algorithms are integrated. The present invention uses α as the loss ratio adjustment and selects a fixed value of 0.5 to finally obtain the regression loss used in this article:

[0074] L final =(1-α)L Inner-MPDIoU +αL nwd (17)

[0075] Sample weighting function: This paper not only uses feature fusion and regression loss to improve the accuracy of infrared small target recognition, but also pays attention to the public dataset FLIR used. It is noted that in actual use, uneven sample types in the dataset are relatively common, and uneven sample types will affect the robustness of the model and the accuracy of individual types. In the FLIR dataset, there are many simple samples and large targets with high recognition rates, while there are few difficult samples and small targets that are difficult to recognize. This is a recognized difficulty in the public dataset FLIR and an important reason why the mAP value is not high.

[0076] The method proposed by the present invention to solve this problem is to design a sliding sample weighting function, and introduce the Exponential moving average (EMA) mechanism to iteratively update the threshold Δ of difficult samples and simple samples by reading the IoU value of the sample prediction boundary box and the ground truth box. The present invention regards samples less than Δ as negative samples and samples greater than Δ as positive samples. In order to enable the model to better distinguish small infrared targets, the weighting function is used to emphasize samples at the boundary, and a higher weight is assigned to difficult samples, such as Figure 4 As shown. The EMA weighted function formula is as follows:

[0077] Δ t =μ·Δ t-1 +(1-μ)·θ t (18)

[0078]

[0079] In the formula, θ t Represents the IoU size at time t, μ is a constant, representing the previous The average of the IoU values ​​is 0.999 in this embodiment. t The definition of is the threshold Δ, and the definition of f(x) is the weight given to difficult samples.

[0080] Using the above method, the present invention carried out practical application tests and comparative tests:

[0081] In the training environment and test environment, Ubuntu 20.04 is selected as the operating system, and NVIDIA RTX A6000 is used as the hardware configuration. In the embodiment, 100 epochs are set during training, and the AamW optimizer is used, where learning rate = 0.0001, weight_decay = 0.0001, warmup_momentum = 0.8, warmup_epochs = 2000. Data enhancement is used, and it is randomly {expand, flip, color distort, resize, overlap_mask}. When performing FPS testing, batch = 1, image_size = [640, 640].

[0082] The IST-DETR proposed in this invention achieves 44.1% AP and 74FPS on the FLIR dataset. In the bicycle class that we pay special attention to, an accuracy of 68.7% is obtained, and the other two classes also achieve high accuracy: people (86.1%) and cars (91.4%). To demonstrate the performance of the proposed method, Table 1 compares the proposed IST-DETR with several commonly used target detection algorithms. It is compared with end-to-end detectors and real-time detectors respectively (in the comparison, pre-trained weights are not used to ensure fairness).

[0083] The present invention also provides an infrared small target detection system based on IST-DETR, comprising:

[0084] A feature map extraction module is used to extract multi-scale feature maps of small infrared targets using a backbone network;

[0085] A feature expression module is used to adopt a hybrid encoder including an MSF module to adapt to the multi-scale feature map of small infrared targets and output an infrared image of feature expression;

[0086] A loss function calculation module, used for selecting image features from the infrared image sequence expressed by the features output by the encoder as an initial query object of the decoder through the loss function;

[0087] The detection result acquisition module is used to determine the position and category of the small infrared target according to the infrared image expressed by the features by using a decoder with an auxiliary prediction head.

[0088] The present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the above-mentioned infrared small target detection method based on IST-DETR is implemented.

[0089] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the above-mentioned infrared small target detection method based on IST-DETR.

[0090] Compared with real-time detectors:

[0091] Because the sizes of the models are different, when selecting the comparison models, the embodiment selects two models for each model, one is a model larger than IST-DETR Mem (GB), and the other is a model smaller than IST-DETR Mem (GB) to reflect the advantages of the model of the present invention. YOLOv5, YOLOv6, YOLOv7, YOLOv, PPYOLOE, and RMTDET are selected for comparison.

[0092] Compared with YOLOv5-n, YOLOv6-n, YOLOv7-tiny, YOLOv8-s, YOLOv8-n, YOLOX-tiny, RMTDET-tiny, and PPYOLOE-tiny models, the AP of IST-DETR is improved by 14.9%, 12.0%, 10%, 4%, 5%, 17.3%, 0.2%, and 5.2%, respectively.

[0093] Compared with end-to-end detectors:

[0094] The present invention uses Resnet-18, while the minimum backbone used by most end-to-end detectors is Resnet-50, such as CO-DETR, DAB-DETR, DN-DETR, DINO, Conditional-DETR, and RT-DETR. However, the model of the present invention has a higher accuracy.

[0095] Compared with DINO, CO-DINO, CO-Deformable-DETR, and RT-DETR models, IST-DETR improves its AP by 7.8% / 6.7% / 8.1% / 3.5% respectively.

[0096] Table 1 Comparison of the measurement results of each model

[0097]

[0098] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any technician familiar with the profession, without departing from the scope of the technical solution of the present invention, based on the technical essence of the present invention, may make any simple modification, equivalent substitution and improvement to the above embodiment, such as combining the units and algorithm steps of each example described in the embodiments disclosed in this application, and using different methods to implement the described functions for each specific application with electronic hardware or a combination of computer software and electronic hardware, which still falls within the protection scope of the technical solution of the present invention.

Claims

1. An infrared small target detection method based on IST-DETR, characterized in that: The following steps are involved: The backbone network is used to extract multi-scale feature maps of small infrared targets; A hybrid encoder including the MSF module is used to adapt to the multi-scale feature map of small infrared targets and output the infrared image encoding of feature expression; A decoder with an auxiliary prediction head is used to determine the position and category of small infrared targets based on infrared image coding expressed by features; The MSF module includes a feature selection module and a feature fusion module. The feature selection module is weighted by a CA module: the CA module performs global maximum pooling and global average pooling on the multi-scale feature map, calculates the maximum value and average value of each channel respectively, adds the generated feature maximum value and average value, and then uses an activation function to obtain the weight of each channel; the channel weight generated by the CA module is multiplied by the feature map of the corresponding scale to obtain a filtered feature map, and the number of channels of each layer of the feature map is ensured to be consistent through convolution; The feature fusion module performs feature fusion on the multi-scale feature maps of the infrared small target: after the feature maps of different scales are weighted by the feature selection module, the feature fusion module performs top-down feature fusion using the screening feature fusion mechanism to share the subtle features of the infrared small target, and then performs bottom-up information sharing to collect the position information of the infrared small target; the screening feature fusion mechanism is: using high-level semantic features as weights for filtering low-level features, using the position information of low-level features to filter high-level semantic features, adding the filtered image features to the original image features point by point, and realizing multi-scale feature fusion; After the MSF module outputs the infrared image after feature fusion, the image features are selected from the infrared image sequence in the feature map output by the MSF module as the initial query object of the decoder through the improved IoU loss function; the formula of the improved IoU loss function is as follows: L final =(1-α)L Inner-MPDIoU +αL nwd ; Among them, L final represents the improved IoU loss function, L Inner-MPDIoU represents the bounding box loss function calculated by combining Inner-Iou and MPDIoU, L nwd represents the similarity function calculated by the NWD algorithm, and α represents the loss ratio adjustment constant.

2. The infrared small target detection method based on IST-DETR according to claim 1, characterized in that: The ResNet-18 network is adopted as the backbone network, and the output features of the last three stages {P3, P4, P5} of the backbone are used as the input of the hybrid encoder.

3. The infrared small target detection method based on IST-DETR according to claim 2, characterized in that: The hybrid coding has an internal feature map interaction module for learning self-attention features, and only the high-level semantic feature P5 output by the backbone network is fed into the internal feature map interaction module.

4. The infrared small target detection method based on IST-DETR according to claim 1, characterized in that: The effectiveness of the detection method is verified using the public infrared dataset FLIR. When selecting image features through the improved IoU loss function, the EMA weighting function is used to emphasize the samples at the boundary of the public infrared dataset FLIR. The formula of the EMA weighting function is as follows: D t =μ·D t-1 +(1-μ)·θ t In the formula, θ t Represents the size of the improved IoU loss function at time t, μ is a constant, representing the previous The average of the IoU loss function values.

5. The infrared small target detection system based on IST-DETR adopted by the method described in any one of claims 1 to 4 is characterized in that: include: A feature map extraction module is used to extract multi-scale feature maps of small infrared targets using a backbone network; A feature expression module is used to adopt a hybrid encoder including an MSF module to adapt to the multi-scale feature map of small infrared targets and output an infrared image of feature expression; A loss function calculation module, used for selecting image features from the infrared image sequence expressed by the features output by the encoder as an initial query object of the decoder through the loss function; The detection result acquisition module is used to determine the position and category of the small infrared target according to the infrared image expressed by the features by using a decoder with an auxiliary prediction head.

6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the infrared small target detection method based on IST-DETR as described in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the infrared small target detection method based on IST-DETR as described in any one of claims 1 to 4.

Images