Deep learning based thermal infrared image small target detection method
By generating high-quality thermal infrared simulation data and improving the YOLOv8 model, the problems of false detection, missed detection, and data scarcity in thermal infrared small target detection are solved, improving detection accuracy and robustness, and adapting to complex scenarios.
Patent Information
- Application Number
- CN202610123309.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-29
- Publication Date
- 2026-07-24
AI Technical Summary
Existing thermal infrared small target detection methods suffer from severe false detections and false negatives in complex backgrounds, and the datasets are limited, lacking robustness and generalization ability across different scenarios.
By generating simulation datasets and improving the YOLOv8 model, high-quality thermal infrared simulation data is generated using recurrent adversarial networks and point spread models. Combined with improved convolutional networks and feature pyramid networks, feature extraction and fusion are enhanced, and a dedicated detection head is designed to improve the detection accuracy of small targets.
It effectively alleviates the problem of data scarcity, improves the accuracy and robustness of thermal infrared small target detection, reduces the false alarm rate, and enhances the model's ability to adapt to complex scenarios.
Smart Images
Figure CN122454129A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and target detection technology, specifically relating to a method for detecting small targets in thermal infrared images based on deep learning. Background Technology
[0002] Thermal infrared imaging technology can overcome the limitations of day and night illumination, enabling all-weather detection at night and in harsh environments such as rain, fog, and smoke. It has wide application value in fields such as military early warning, drone reconnaissance, security monitoring, autonomous driving, and search and rescue. Target detection based on thermal infrared images is currently a hot research direction.
[0003] Traditional thermal infrared small target detection methods primarily improve target detection rates by enhancing responses to weak signals. However, these methods are largely model-driven, with limited ability to distinguish targets from complex background interference, leading to high false alarm rates in complex scenes. Furthermore, these algorithms are typically designed for specific scenarios, lacking robustness and generalization capabilities across different scenarios. In recent years, with the development of deep learning technology, target detection methods based on convolutional neural networks (CNNs) have significantly improved detection accuracy and real-time performance, and are gradually being applied to thermal infrared small target detection tasks. These methods automatically learn target features through data-driven approaches, outperforming traditional methods in suppressing false alarms. However, due to the inherent information gaps and complex backgrounds in thermal infrared images, existing deep learning models still exhibit false detections, missed detections, or overdetections in complex environments. Moreover, compared to the visible light domain, the number and representativeness of publicly available datasets in the thermal infrared small target detection field are limited, severely restricting further optimization and performance improvement of data-driven models.
[0004] Therefore, a thermal infrared small target detection scheme that balances dataset effectiveness and algorithm adaptability can improve the detection accuracy of small targets in thermal infrared images. Summary of the Invention
[0005] In view of the shortcomings of the prior art, the present invention provides a method for small target detection in thermal infrared images based on deep learning, comprising the following steps: S1. Organizing the thermal infrared small target dataset and generating the simulation dataset, wherein the simulation dataset generates a thermal infrared simulation background through a recurrent adversarial network, generates thermal infrared simulation targets through a convolutional network based on a point spread model, and synthesizes the simulation background and simulation targets into a thermal infrared small target simulation dataset through an automatic mapping algorithm. S2. Construct an improved YOLOv8 thermal infrared small target detection model. The improved YOLOv8 detection model includes a backbone network, a neck network, and a detection head. The backbone network uses a windmill-shaped convolution module to replace the C2f convolution module and introduces a polarization attention mechanism module. The neck network replaces the original path aggregation network-feature pyramid network structure with a weighted bidirectional feature pyramid network. The detection head module adds a tiny target detection head corresponding to a higher resolution feature map at the end of the original detection head and deletes the large target detection head corresponding to the lowest resolution feature map. S3. After mixing the real dataset and the simulation dataset, divide them into training set, validation set and test set, and train the constructed YOLOv8 model parameters; S4. Input the thermal infrared image to be detected into the trained model to detect small targets.
[0006] Furthermore, the recurrent adversarial network includes a prior module and a recurrent adversarial module. The prior module uses paired visible light-thermal infrared image pairs to learn the initial mapping relationship through supervised learning pre-training. The recurrent adversarial module transfers the mapping relationship of the prior module through knowledge distillation, converting visible light images without corresponding thermal infrared images into thermal infrared simulation background images of the target. The loss function is a weighted sum of adversarial loss and recurrent consistency loss.
[0007] Furthermore, the point diffusion model is approximated by the Airy disk model. Small target regions are cropped from the real thermal infrared dataset, their grayscale characteristics are statistically analyzed, the maximum grayscale value of the target is obtained, and the Airy disk model is fitted with a two-dimensional Gaussian function according to the target characteristics to generate a basic target. Through a convolutional network and the addition of Poisson noise to simulate detector characteristics, a thermal infrared simulation target containing diffraction-limited effects, spatial sampling distortion, and radiation noise characteristics is generated.
[0008] Furthermore, the automatic texture synthesis algorithm includes a physical-level fusion module, a sensor simulation module, and a label generation module. The physical-level fusion module simulates the energy decay of the target based on the exponential decay model and dynamically adjusts the target intensity according to the background radiation. The sensor simulation module simulates the point diffusion model of the optical system through mean filtering and adds Poisson noise and Gaussian noise to increase the realism of the image. The label generation module automatically detects the bounding box of the target region and generates a label file.
[0009] Furthermore, the windmill-shaped convolution employs asymmetric padding and grouped convolution, creating horizontal and vertical convolution kernels for different regions of the image. After different padding and convolution operations, the feature maps are stitched together and convolutional normalization is performed to output the final feature map.
[0010] Furthermore, the polarization attention mechanism module is added between the first three C2f modules and the convolutional layer of the backbone network, and is deployed in parallel in the channel dimension and spatial dimension without significant dimensionality compression.
[0011] Furthermore, the weighted bidirectional feature pyramid network retains the original feature nodes of the backbone network, reduces feature loss through bidirectional cross-scale connections that transmit semantic information from top to bottom and detailed information from bottom to top, learns the weights of features at different scales through training, and adopts a weighted feature fusion mechanism to combine features at different levels, thereby improving the fusion efficiency of small target features.
[0012] Based on the above technical solution, the present invention has the following beneficial effects:
[0013] (1) It solves the problem of insufficient sample quantity and incomplete representativeness of thermal infrared small targets. By improving the recurrent adversarial network, physical model-driven target generation and automatic texture fusion, it can generate thermal infrared small target simulation data with diverse backgrounds and realistic target physical characteristics in batches, and automatically generate annotations, which greatly alleviates the problem of data scarcity and reduces the cost of data acquisition and annotation.
[0014] (2) The improved YOLOv8 algorithm optimizes the “weak, small and textureless” characteristics of thermal infrared small targets from feature extraction, feature fusion to detection output. Windmill convolution and polarization attention enhance local feature extraction and context awareness, weighted bidirectional features strengthen multi-scale information fusion, and the dedicated detection head design makes the network more focused on small targets. Attached Figure Description
[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0016] Figure 1 Flowchart of the method of this invention.
[0017] Figure 2 Diagram of a recurrent adversarial network.
[0018] Figure 3 The network structure diagram of the YOLOv8 algorithm of this invention. Detailed Implementation
[0019] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be described in detail below with reference to the accompanying drawings and embodiments. The following embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.
[0020] A deep learning-based method for small target detection in thermal infrared images, such as... Figure 1As shown, it includes the following steps:
[0021] 1. Data Preparation
[0022] Using drones in low-altitude environments as the main detection targets, the selected datasets include the SIRST-V2 dataset and the Infrared Small Aircraft Target Detection and Tracking Dataset (IDSAT) in ground / air background. Image frames that do not meet the definition of small targets, have blurred images, or are highly repetitive are removed. A semi-automatic bounding box generation method based on dynamic window adjustment, contrast enhancement, adaptive threshold segmentation, and contour analysis is used to generate YOLO format labels from the point annotations in the IDSAT dataset.
[0023] 2. Generate a thermal infrared background based on a recurrent adversarial network.
[0024] Recurrent adversarial networks divide the training process into two stages: prior modules and recurrent adversarial modules.
[0025] The prior module trains the generator using visible light data labeled with real thermal infrared images through supervised learning, enabling it to accurately convert visible light images into corresponding thermal infrared images. Input A t Generate thermal infrared simulation image B in generator G1 t ′ At this point, generator G1 is mainly responsible for learning the potential mapping relationship from visible light to thermal infrared images, while discriminator D1 receives the thermal infrared simulation image B. t ′ and true thermal infrared image B t Discriminant image B t The authenticity of ′.
[0026] The recurrent adversarial model uses knowledge distillation to convert visible light images without corresponding thermal infrared images into target thermal infrared images by leveraging the mapping relationships of prior modules. Generator G1 is shared with the generator in the prior modules, and its input is a visible light image A without corresponding infrared information. r Output the corresponding infrared simulation image B r The generator G2 takes image B as input. r ′, convert it into a visible light simulation image A r ′.
[0027] like Figure 2 As shown, the input data includes paired visible light-thermal infrared image datasets, A t Represents an RGB image, B t This represents the corresponding thermal infrared image; Single visible light image dataset A r .
[0028] The loss function is a weighted sum of adversarial loss and cycle consistency loss.
[0029] 3. Generate thermal infrared target data based on the point spread model
[0030] Target images were generated using the SIRST dataset, which contains 427 thermal infrared images. The images have five different labeling formats for the targets. Cropping was performed using instance segmentation labels to obtain the original thermal infrared small targets. Infrared small targets typically appear as dots in the images, with a bright spot in the center that gradually darkens outwards. The grayscale characteristics of the infrared small targets were statistically analyzed, and the maximum grayscale value was obtained. Based on these characteristics, a two-dimensional Gaussian function was used to fit an Airy disk model to generate a basic Airy spot. Building upon the generated spot kernel, a convolution module was designed, and Poisson noise was added to simulate an actual detector, increasing the realism of the generated targets. The final simulated target image simultaneously incorporates diffraction-limited effects, spatial sampling distortion, and radiated noise characteristics.
[0031] 4. Thermal infrared data synthesis based on automatic mapping algorithm
[0032] The algorithm automatically combines the simulated thermal infrared background image obtained in step 2 with the simulated thermal infrared target obtained in step 3, generating corresponding annotation files. The algorithm consists of three modules: the first module is a physical-level fusion model, which simulates the attenuation of target energy with distance based on an exponential decay model, and designs a radiation adaptive model, dynamically adjusting the target intensity according to the radiation of the background image; the second module simulates sensor effects, using mean filtering to simulate the point spread model of the optical system, and combining Poisson noise and Gaussian noise to increase image realism; the third module generates annotations, automatically detecting the target region bounding box and generating JSON format conforming to the COCO standard.
[0033] 5. Construct an improved YOLOv8 thermal infrared small target detection model
[0034] The improved YOLOv8 network model structure is as follows: Figure 3 As shown, the method specifically includes the following steps:
[0035] (1) Backbone network improvement
[0036] The C2f convolution operation in CSPDarknet53 is replaced with the windmill-shaped convolution PConv to enhance low-level feature extraction and expand the receptive field. Small thermal infrared targets exhibit a Gaussian feature distribution. The PConv module uses asymmetric padding to create horizontal and vertical convolution kernels for different regions of the image. The outward diffusion of the convolution kernels resembles a Gaussian distribution, and the effectiveness of the receptive field gradually decreases outwards, making the convolution module more consistent with the imaging characteristics of small thermal infrared targets. The smaller the target, the more concentrated its feature information, highlighting the importance of central features. After different padding and convolution operations, the feature maps are stitched together to form a higher-dimensional feature map. The stitched feature map is then normalized through a final convolution, ultimately outputting a single feature map. PConv utilizes grouped convolution to significantly increase the receptive field while minimizing the number of parameters.
[0037] A Polarized Attention (PSA) module is introduced to enhance the model's ability to capture long-range contextual information. The PSA module is added between the first three C3 layers and the convolutional layers of the Backbone module to improve the model's perception of small objects and complex scenes. PSA employs two branches, using self-attention mechanisms in the channel and spatial dimensions respectively, ultimately outputting a fused result. Furthermore, minimal compression is applied to either the spatial or channel dimensions, reducing information loss associated with dimensionality reduction.
[0038] (2) Neck Improvement
[0039] In the feature fusion module, a bidirectional feature pyramid network (BiFPN) is introduced to extract features from multiple network layers of the backbone network and generate a feature pyramid. Two additional lateral connection paths are introduced between adjacent levels of the feature pyramid, allowing semantic information to flow in both top-down and bottom-up directions, better preserving the original feature information from the backbone network and reducing information loss. A full feature fusion mechanism is adopted to combine features at different levels, and the fusion weights are learned during training.
[0040] (3) Improvement of the detection head
[0041] Typically, YOLOv8 performs detection at different scales, usually using three detection heads to handle large, medium, and small targets respectively. Thermal infrared small targets are typically within the 9x9 pixel range, and given the YOLOv8 detection head configuration, the algorithm's detection performance for this size is not ideal.
[0042] To design a small target detection layer, an Upsample-Concat-C2f layer is added after the Head layer output from the original 15th layer, with a detection feature map size of 160×160. Then, a CBS-Concat-C2f structure is added to change the 160×160 convolution to 80×80. A Head structure is output on the added C2f structure, which is the same as other detection head structures, with a smaller receptive field size of 4×4. This allows the network to focus more on small targets and improves detection accuracy.
[0043] The detection layer for large targets does not improve the detection accuracy of small targets; instead, it increases the computational load and inference speed. Therefore, the last CBS-Concat-C2f structure and its corresponding Head detection head in the original network are removed to reduce computational redundancy and improve the detection speed of small targets.
[0044] 6. Model Training
[0045] The real dataset was mixed with the simulation dataset generated in the first step and divided into training, validation, and test sets in an approximately 8:1:1 ratio. The AdamW optimizer was used with 300 training epochs and an initial learning rate of 0.01. The batch size was 4 with an initial learning rate of 0.01. The overall performance of the model was evaluated using the metrics precision (P), recall (R), and average detection precision (mAP) to obtain the optimal parameter combination.
[0046] 7. Thermal Infrared Small Target Detection
[0047] Input the thermal infrared image to be detected into the trained YOLOv8 model, and output the coordinates of the small thermal infrared target.
[0048] The method for generating thermal infrared small target datasets and the improved YOLOv8 detection method provided by this invention form a complete solution from data to model, effectively solving key problems in thermal infrared small target detection, and has high practical value and promising prospects for promotion.
[0049] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for detecting small targets in thermal infrared images based on deep learning, characterized in that, Includes the following steps: S1. Organizing the thermal infrared small target dataset and generating the simulation dataset, wherein the simulation dataset generates a thermal infrared simulation background through a recurrent adversarial network, generates thermal infrared simulation targets through a convolutional network based on a point spread model, and synthesizes the simulation background and simulation targets into a thermal infrared small target simulation dataset through an automatic mapping algorithm. S2. Construct an improved YOLOv8 thermal infrared small target detection model. The improved YOLOv8 detection model includes a backbone network, a neck network, and a detection head. The backbone network uses a windmill-shaped convolution module to replace the C2f convolution module and introduces a polarization attention mechanism module. The neck network replaces the original path aggregation network-feature pyramid network structure with a weighted bidirectional feature pyramid network. The detection head module adds a tiny target detection head corresponding to a higher resolution feature map at the end of the original detection head and deletes the large target detection head corresponding to the lowest resolution feature map. S3. After mixing the real dataset and the simulation dataset, divide them into training set, validation set and test set, and train the constructed YOLOv8 model parameters; S4. Input the thermal infrared image to be detected into the trained model to detect small targets.
2. The method for detecting small targets in thermal infrared images based on deep learning according to claim 1, characterized in that, The recurrent adversarial network includes a prior module and a recurrent adversarial module. The prior module learns an initial mapping relationship through supervised learning pre-training using pairs of visible light and thermal infrared images. The recurrent adversarial module transfers the mapping relationship of the prior module through knowledge distillation, converting visible light images without corresponding thermal infrared images into thermal infrared simulation background images of the target. The loss function is a weighted sum of adversarial loss and recurrent consistency loss.
3. The method for detecting small targets in thermal infrared images based on deep learning according to claim 1, characterized in that, The point diffusion model is approximated by the Airy disk model. Small target regions are cropped from the real thermal infrared dataset, their grayscale characteristics are statistically analyzed, the maximum grayscale value of the target is obtained, and the Airy disk model is fitted with a two-dimensional Gaussian function according to the target characteristics to generate a basic target. Through a convolutional network and the addition of Poisson noise to simulate detector characteristics, a thermal infrared simulation target containing diffraction-limited effects, spatial sampling distortion and radiation noise characteristics is generated.
4. The method for detecting small targets in thermal infrared images based on deep learning according to claim 1, characterized in that, The automatic texture synthesis algorithm includes a physical-level fusion module, a sensor simulation module, and a label generation module. The physical-level fusion module simulates the energy decay of the target based on the exponential decay model and dynamically adjusts the target intensity according to the background radiation. The sensor simulation module simulates the point diffusion model of the optical system through mean filtering and adds Poisson noise and Gaussian noise to increase the realism of the image. The label generation module automatically detects the bounding box of the target area and generates a label file.
5. The method for detecting small targets in thermal infrared images based on deep learning according to claim 1, characterized in that, The windmill-shaped convolution employs asymmetric padding and grouped convolution, creating horizontal and vertical convolution kernels for different regions of the image. After different padding and convolution operations, the feature maps are stitched together and convolutional normalization is performed to output the final feature map.
6. The method for detecting small targets in thermal infrared images based on deep learning according to claim 1, characterized in that, The polarization attention mechanism module is added between the first three C2f modules and the convolutional layer of the backbone network, and is deployed in parallel in the channel dimension and spatial dimension without significant dimensionality compression.
7. The method for detecting small targets in thermal infrared images based on deep learning according to claim 1, characterized in that, The weighted bidirectional feature pyramid network retains the original feature nodes of the backbone network, reduces feature loss through bidirectional cross-scale connections that transmit semantic information from top to bottom and detailed information from bottom to top, learns the weights of features at different scales through training, and adopts a weighted feature fusion mechanism to combine features at different levels, thereby improving the fusion efficiency of small target features.