An unmanned aerial vehicle infrared small target detection method based on thermal response fragmentation shielding estimation and background heat source inhibition gating

Abstract

This invention discloses a method for detecting small infrared targets on UAVs based on thermal response fragmentation occlusion estimation and background heat source suppression gating, belonging to the fields of computer vision, infrared image processing, and UAV airborne intelligent perception technology. Addressing the problems of low pixel count for distant small targets in UAV infrared images, discontinuous thermal response due to local occlusion, susceptibility to false enhancement of background heat sources from complex terrain features, and the difficulty of general attention modules in distinguishing real targets from background hotspots, this invention proposes a restricted recalibration method for deep semantic features of occluded small targets. This method first performs normalization preprocessing on the UAV infrared image and extracts shallow detail features, mid-level semantic features, and deep semantic features using a target detection network. Then, before the deep semantic features enter the feature fusion network, a thermal response fragmentation occlusion estimation map, a background heat source suppression map, and occlusion condition channel semantic weights are constructed. The three types of weights are then used to jointly gating the lightweight residual enhancement features, and the enhanced deep semantic features are obtained through restricted residual injection. Finally, the enhanced deep semantic features are fused with other scale features, and the detection head outputs the target category, confidence level, and bounding box position. This invention does not simply employ channel attention or spatial attention enhancement features. Instead, it uses a joint modeling approach of thermal response continuity deficit, local contrast difference, and background heat source suppression to model the occlusion of small infrared target regions. This improves the robustness of detection for long-range, weak thermal radiation, and partially occluded targets with only a slight increase in computational load. It is suitable for real-time airborne infrared target detection scenarios such as UAV nighttime search and rescue, low-altitude security, disaster inspection, and complex terrain backgrounds.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary fields of computer vision deep learning, infrared thermal imaging image processing, UAV-borne embedded intelligent sensing, and low-altitude target detection. Specifically, it addresses all-weather imaging scenarios using UAVs equipped with infrared thermal imagers, providing a deep semantic feature-controlled recalibration detection scheme to solve the challenges of identifying long-distance, small-scale, partially occluded, and weakly thermally radiating targets. This invention can be widely applied to various airborne infrared small target detection tasks with stringent requirements for real-time performance and detection accuracy, such as nighttime search and rescue of personnel in the wild, urban low-altitude security vehicle monitoring, infrared patrol for forest fires, search and rescue of people trapped in floods / earthquakes, long-range infrared early warning at borders, and unmanned safety monitoring in mining areas. Background Technology

[0002] Unmanned aerial vehicles (UAVs) equipped with infrared thermal imaging sensors can perform target detection. Utilizing the principle of thermal radiation imaging, they overcome the limitations of visible light and can capture ground targets such as pedestrians and vehicles with temperature differences in environments such as night, dense fog, smoke, and low light. This makes them an irreplaceable core technology for low-altitude security and emergency rescue. However, UAV infrared imaging itself has inherent imaging defects, which are further amplified by long-distance shooting and obstructed scenes. Long-distance imaging scale degradation: When UAVs take high-altitude photos, the number of pixels in the ground target image is very small. After multiple downsampling layers of the backbone network, the effective thermal features of the target are greatly compressed and diluted in the deep features, and the deep network has difficulty distinguishing between weak targets and random thermal noise. Occlusion causes thermal feature defects: Vegetation, buildings, and mountains can cut off the continuous thermal radiation area of ​​the target. Deep semantic features can only obtain fragmented target responses. The model is very likely to judge the incomplete target as background clutter, resulting in a large number of missed detections. Infrared background interference is complex: temperature differences exist in ground rocks, roads, water bodies, and building exteriors, which generate a large number of background hot spots with intensity close to the target, highly confusing them with the characteristics of weak targets; The contrast of weakly heated targets is extremely low: the temperature difference between pedestrians and stationary vehicles and the background is slight, the target boundaries are blurred, there is no obvious texture gradient, and conventional feature enhancement methods are difficult to distinguish them.

[0003] Current mainstream detection solutions in the industry are based on YOLO series single-stage detectors and Faster R-CNN two-stage detectors. They rely on the backbone network to extract multi-scale features in layers, and combine Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) to achieve cross-scale fusion, ultimately outputting the detection results. The industry's general logic is: shallow features retain complete spatial pixel information, suitable for spatial localization of small targets; deep features possess global abstract semantics, used to distinguish targets from the background. However, existing technical solutions have four inherent defects that cannot be avoided: The deep representation of weak targets fails: After multiple downsampling, the effective response of targets in deep semantic features of distant and occluded targets is greatly reduced. The deep features output by the original backbone network alone cannot complete the classification and recognition of weak targets. General attention modules have a single dimension and are prone to amplifying background noise: Existing attention modules such as SE, CBAM, and CA only use single channel or single space weighting and have not designed a dedicated perception mechanism for infrared occlusion and incomplete thermal features. While enhancing target features, they simultaneously amplify background heat sources and ground object clutter, reduce the separability of target and background features, and significantly increase the false detection rate. Global multi-scale context expansion has huge computational overhead: Traditional solutions expand the global context by stacking dilated convolutions and large receptive field convolutions, which requires adding a large number of convolutional layers. The number of model parameters and floating-point computation GFLOPs increase exponentially, making it unsuitable for low-computing hardware such as UAV onboard embedded boards and edge computing boxes. Unconstrained feature enhancement disrupts the original feature distribution: Existing context enhancement modules lack amplitude limiting mechanisms, and excessive amplification of feature responses can distort the native semantic distribution of the backbone network, interfere with the downstream multi-scale feature fusion process, and reduce the localization accuracy of small targets.

[0004] Current methods mainly improve the performance of small target detection from the perspectives of channel attention, spatial attention, multi-scale fusion, or lightweight convolutional structures. However, there is still a lack of an explicit modeling mechanism for the discontinuous thermal response of small targets under infrared occlusion by UAVs. In particular, there is a lack of detection methods that simultaneously utilize thermal response continuity loss, local contrast difference, and background heat source suppression to constrain the enhancement of deep semantic features. Summary of the Invention

[0005] To overcome the problems of existing UAV infrared small target detection methods, such as attenuation of deep response of small targets at long distances, discontinuous thermal response of occluded targets, false enhancement of heat sources from complex terrain features, and the inability of general attention modules to distinguish between real targets and background hotspots, this invention provides a UAV infrared small target detection method based on thermal response fragmentation occlusion estimation and background heat source suppression gating. This method does not rely on simply adding detection heads, stacking attention modules, or expanding the receptive field. Instead, it addresses the fragmentation of thermal response of infrared occluded small targets by performing a restricted recalibration before the deep semantic features enter the feature fusion network.

[0006] The technical solution adopted in this invention is as follows: S1. Acquire raw infrared images of the UAV and corresponding labels, and perform size normalization, grayscale normalization and label matching processing on the infrared images to obtain standardized input images.

[0007] S2. Construct an infrared small target detection network, which includes a backbone feature extraction network, an SCE module (i.e., a thermal response fragmentation occlusion estimation and background heat source suppression gating module), a feature fusion network, and a detection head.

[0008] S3. Feed the input image into the backbone feature extraction network to obtain shallow detail features, mid-level semantic features, and deep semantic features, and denote the deep semantic features as... .

[0009] S4. Deep semantic features Average aggregation and maximum aggregation are performed along the channel dimension to obtain the deep thermal response map. Further calculations The difference between the local and neighborhood mean is used to obtain a local contrast difference map. Simultaneously, a thermal response continuity defect map is constructed based on the local response area, the degree of response fracture, and the neighborhood contrast. .

[0010] S5. Develop a deep thermal response map. Local contrast difference map Thermal response continuous loss diagram Following the input convolutional mapping layer and the sigmoid function, a thermal response fragmented occlusion estimation map is obtained. This figure is used to highlight small target candidate regions with localized shading, weak thermal response, and discontinuous thermal radiation.

[0011] S6. Based on the deep thermal response diagram The local diffusion response, local maximum response, and edge gradient response generate a background heat source suppression map. For heat source regions that are large in area, have slowly changing boundaries, and lack local abrupt changes, Assign lower weights to candidate regions that exhibit characteristics of local mutations in small targets. Retain higher weights.

[0012] S7, Utilization and Commonly constrained deep semantic features Obtain occlusion condition features and based on Generate occlusion conditional channel semantic weights .

[0013] S8. Lightweight residual enhancement branches are constructed using depthwise separable convolutions and 1×1 convolutions to obtain lightweight residual enhancement features. .

[0014] S9. Imposing semantic weights on occlusion conditional channels. Thermal response fragmentation occlusion estimation map Background heat source suppression diagram and lightweight residual enhancement features Joint gating is performed, and recalibrated deep semantic features are obtained through restricted residual injection. The calculation process is as follows: Wherein, β is the restricted residual injection coefficient, which is restricted to a preset range [0, β_max] to avoid excessive perturbation of the original deep semantic distribution by the enhanced features.

[0015] S10. Re-label the deep semantic features The features, along with shallow detail features and mid-level semantic features, are fed into a feature fusion network to obtain multi-scale detection features. The detection head then outputs the target category, confidence level, and bounding box location.

[0016] The beneficial effects of this invention are as follows: First, this invention takes the "thermal response fragmentation" of infrared occlusion small targets as an explicit modeling object, and generates an occlusion estimation map by combining a deep thermal response map, a local contrast difference map, and a thermal response continuity loss map, which is different from ordinary feature enhancement methods that only use channel attention or spatial attention.

[0017] Second, the present invention introduces a background heat source suppression map to suppress large-area background heat spots such as roads, buildings, rocks, and water edges, avoiding the situation where traditional context enhancement modules simultaneously amplify background heat sources while enhancing the target area, thereby reducing the risk of false detection in complex terrain backgrounds.

[0018] Third, this invention uses occlusion conditional channel semantic weights instead of directly generating channel weights from the original deep features; the channel weights are constrained by both the occlusion estimation map and the background suppression map, making them more suitable for detecting infrared weak targets, partially occluded targets, and low-contrast targets.

[0019] Fourth, this invention controls the enhancement magnitude by limiting the residual injection coefficient β, which compensates for the local context information of small targets while preserving the original deep semantic feature distribution, thus avoiding excessive enhancement that could damage the stability of the subsequent feature fusion network.

[0020] Fifth, this invention inserts a lightweight module only before deep semantic features enter the feature fusion network, without changing the detection head form, label format, and training process. It is applicable to YOLO-like detection networks and other target detection networks with multi-scale feature fusion structures, and is easy to deploy on UAV airborne edge computing platforms.

[0021] To make the technical solution of the present invention clearer, the present invention will be described below with reference to embodiments. The described embodiments are only used to explain the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, equivalent substitutions or conventional modifications made to module names, network backbones, detection head forms, convolutional kernel sizes, and training strategies without departing from the concept of the present invention should all fall within the scope of protection of the present invention. Attached Figure Description

[0022] Figure 1 This is an overall flowchart of the UAV infrared small target detection method of the present invention; Figure 2 This is a diagram illustrating the deployment of the SCE module in the YOLOv11 detection network; Figure 3 This is a schematic diagram of the structure of the thermal response fragmentation occlusion estimation and background heat source suppression gating module; Figure 4 This is a schematic diagram of the infrared small target detection training and inference process based on the module. Detailed Implementation

[0023] To make the technical solution of the present invention clearer, the present invention will be described below with reference to embodiments. The described embodiments are only used to explain the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, equivalent substitutions or conventional modifications made to module names, network backbones, detection head forms, convolutional kernel sizes, and training strategies without departing from the concept of the present invention should all fall within the scope of protection of the present invention.

[0024] In this embodiment, a YOLO-like single-stage detection network is used as the basic detection framework. The input image is an infrared image of a UAV, and the detection targets may include pedestrians, vehicles, bicycles, and other small-scale thermal targets. The network consists of a backbone feature extraction network, a thermal response fragmentation occlusion estimation and background heat source suppression gating module, a feature fusion network, and a detection head. In this embodiment, the deep semantic features output by the backbone feature extraction network are denoted as... For YOLO-like networks, This can correspond to the highest semantic layer features at the end of the backbone network; for other detection networks, This can correspond to the high-level semantic features before entering the feature pyramid. The recalibration module does not change the detection head type or the training label format, but rather performs recalibration before feature fusion. Perform a controlled enhancement.

[0025] The SCE module, namely the thermal response fragmentation occlusion estimation and background heat source suppression gating module, first processes the deep semantic features. Perform average channel aggregation and maximum channel aggregation to generate a deep thermal response map. Then based on Calculate the local contrast difference map Thermal response continuous loss diagram This is used to characterize the response breaks, area degradation, and weak thermal discontinuities of small infrared-masked targets in deep features. Subsequently, and After convolution mapping and the Sigmoid function, a thermal response fragmented occlusion estimation map is generated. Furthermore, the module is based on The local diffusion response, local maximum response, and edge gradient response generate a background heat source suppression map. This is used to reduce the false enhancement of deep semantic features by large-area background heat sources such as roads, buildings, rocks, and water edges. Finally, the module utilizes... and Commonly constrained deep semantic features Obtain occlusion condition features and based on Generate occlusion conditional channel semantic weights Then With lightweight residual enhancement features Joint gating is performed, and recalibrated deep semantic features are obtained through restricted residual injection. .

[0026] During training, the detection loss function can include bounding box regression loss, classification loss, and target confidence loss. The bounding box regression loss can use CIoU, DIoU, SIoU, NWD, or other loss functions suitable for small target localization; the classification loss and confidence loss can use cross-entropy, binary cross-entropy, or focus loss.

[0027] Example 2: Method for generating thermal response fragmentation occlusion estimation map In this embodiment, deep semantic features The dimensions are C×H×W, where C is the number of channels, and H and W are the spatial dimensions. First, along the channel dimension... Average response aggregation and maximum response aggregation were performed, and the aggregation results were normalized to obtain the deep thermal response map. Deep thermal response map It is used to reflect the overall response distribution of small infrared targets in deep semantic features.

[0028] Then, the deep thermal response map Perform local average pooling to obtain the local mean response map. and calculate The absolute difference between the local mean response map and the local contrast difference map is obtained. . It can characterize the difference in local thermal response between weak targets and the surrounding background.

[0029] Furthermore, based on the deep thermal response map A thermal response continuity defect map is constructed using the response area, response fragmentation degree, and neighborhood contrast within a local neighborhood. When a local area exhibits characteristics of small response area, discontinuous response, and high local contrast, it indicates that the area may correspond to the incomplete thermal response of small infrared targets obscured by vegetation, buildings, or ground features. The corresponding location is... It is given a relatively large weight. Finally, The images are stitched together, and a thermal response fragmented occlusion estimation map is generated using convolutional mapping and the Sigmoid function. The calculation process is as follows: Where σ represents the Sigmoid function, Conv represents the convolution mapping operation, and [ ] represents the feature concatenation operation. (Thermal response fragmented occlusion estimation map) Used to highlight small target candidate regions with local occlusion, weak thermal response, and discontinuous thermal radiation.

[0030] Example 3: Method for generating background heat source suppression map and occlusion condition channel weights In this embodiment, the background heat source suppression map This is used to reduce the interference of large-area hot spots in complex terrain backgrounds on the deep semantic feature enhancement process. Specifically, based on the deep thermal response map... Calculate the local diffusion response, local maximum response, and edge gradient response. When a region exhibits a large area, wide response diffusion range, slow boundary changes, and a lack of local abrupt changes in small targets, this region is more likely to correspond to roads, buildings, rocks, water edges, or other background heat sources. Background heat source suppression map. A lower weight is assigned to this region; when a region exhibits concentrated local responses, significant neighborhood contrast, and small-target abrupt change characteristics, This region retains a higher weight.

[0031] In obtaining the thermal response fragmentation occlusion estimation map Background heat source suppression map Then, the two are applied together to deep semantic features. Obtain occlusion condition features The calculation process is as follows: Here, ⊙ represents element-wise multiplication. Subsequently, the occlusion condition features are discussed. Global average pooling and global max pooling are performed, and occlusion conditional channel semantic weights are generated through a lightweight mapping network. Compared to directly derived from original deep semantic features... Different channel weights are generated. The channel weights are obtained under the joint constraints of thermal response fragmentation occlusion estimation and background heat source suppression. Therefore, they can more effectively enhance the semantic channels related to small infrared occlusion targets and suppress irrelevant channel responses corresponding to complex background heat sources.

[0032] Finally, lightweight residual enhancement branches are constructed using depthwise separable convolutions and 1×1 convolutions to obtain lightweight residual enhancement features. and will Joint gating is performed to obtain recalibrated deep semantic features through restricted residual injection. Wherein, β is the restricted residual injection coefficient, which is restricted to a preset range [0, β_max] to avoid excessive perturbation of the original deep semantic distribution by the enhanced features.

[0033] Example 4: Comparison of detection performance on the HIT-UAV dataset To verify the effectiveness of the method of this invention, experiments were conducted on the HIT-UAV UAV infrared target detection dataset. The original data was divided into three sets: a training set containing 2008 images, a validation set containing 287 images, and a test set containing 571 images. Using YOLO11n as the baseline model, the detection performance before and after adding thermal response fragmentation occlusion estimation and background heat source suppression gating modules was compared under the same training strategy. Experimental metrics included Precision, Recall, mAP50, mAP50-95, number of parameters, and computational cost. The comparison results are shown in Table 1.

[0034] Table 1 Comparison of detection results of the benchmark model and the model after adding the module of this invention on the HIT-UAV dataset. YOLO11n Baseline 0.822 0.702 0.771 0.486 2.58 6.3 YOLO11n + This invention module Add thermal response fragmentation occlusion estimation and background heat source suppression gating 0.850 0.764 0.800 0.497 2.59 6.4 Increase relative to Baseline +0.028 +0.062 +0.029 +0.011 +0.01 +0.1 As shown in Table 1, on the HIT-UAV validation set, after adding the thermal response fragmentation occlusion estimation and background heat source suppression gating module of this invention, Precision increased from 0.822 to 0.850, Recall increased from 0.702 to 0.764, mAP50 increased from 0.771 to 0.800, and mAP50-95 increased from 0.486 to 0.497. Experimental results show that this method can improve the recall capability and overall detection accuracy of infrared UAV small target detection while keeping the number of parameters basically the same and increasing the computational load slightly.

[0035] Example 5: Module Deployment Location and Engineering Application The core of this invention lies in the thermal response fragmentation occlusion estimation and background heat source suppression gating process, rather than being limited to a fixed layer name. For YOLO-like networks, this module can be set after the deep semantic feature output at the end of the backbone network and before the feature fusion network sampling or path aggregation operation; for other detection networks with feature pyramid structures, this module can be set before the highest semantic layer features enter the pyramid fusion.

[0036] Since the module only performs a single post-recalibration of the deep semantic features, the increase in computation is minimal, making it suitable for UAV-borne computing platforms, edge devices, or real-time infrared monitoring systems. After training, the model can be deployed to UAV search and rescue systems for nighttime pedestrian searches, disaster area target detection, low-altitude patrols, and thermal target detection against complex backgrounds.

[0037] This invention is not limited to the above embodiments. For those skilled in the art, without departing from the principle of this invention, the thermal response fragmentation occlusion estimation and background heat source suppression gating module can be used in combination with different backbone networks, different detection heads, different loss functions or different infrared target detection datasets. All such modifications and improvements should fall within the protection scope of this invention.

Claims

1. A method for detecting small infrared targets on unmanned aerial vehicles (UAVs) based on thermal response fragmentation occlusion estimation and background heat source suppression gating, characterized in that, Includes the following steps: S1. Acquire infrared images collected by the UAV platform, and perform size normalization, grayscale normalization and label matching processing on the infrared images to obtain the input image to be detected; S2. Construct an infrared small target detection network, which includes a backbone feature extraction network, an SCE module (i.e., thermal response fragmentation occlusion estimation and background heat source suppression gating module), a feature fusion network, and a detection head. S3. Input the input image to be detected into the backbone feature extraction network to extract shallow detail features, mid-level semantic features, and deep semantic features, and select the deep semantic features as the features to be recalibrated. ; S4. For the deep semantic features... Channel aggregation is performed to generate a deep thermal response map. And based on the deep thermal response map Calculate the local mean response plot, the local contrast difference plot, and the thermal response continuity deficit plot; S5. Based on the deep thermal response map The local contrast difference map and the thermal response continuity loss map generate a fragmented occlusion estimation map of the thermal response. The thermal response fragmentation occlusion estimation map Used to characterize areas of local shading, weak thermal response, or discontinuity in target thermal radiation; S6. Based on the deep thermal response map The local high response region, neighborhood diffusion response, and edge gradient response generate a background heat source suppression map. The background heat source suppression map It is used to reduce the false enhancement of deep semantic features by large-area ground object heat spots, road heat spots, building heat sources or rock heat sources; S7. Fragment the thermal response occlusion estimation map Background heat source suppression map Together they act on deep semantic features Obtain occlusion condition features And based on the occlusion condition features Generate occlusion conditional channel semantic weights ; S8. Utilizing lightweight convolutional residual branches for deep semantic features Local context enhancement is performed to obtain lightweight residual enhancement features. ; S9. The thermal response fragmentation occlusion estimation map Background heat source suppression diagram Occlusion condition channel semantic weights and lightweight residual enhancement features Joint gating is performed, and recalibrated deep semantic features are obtained through restricted residual injection. ; S10. The recalibrated deep semantic features The features are input into a feature fusion network along with shallow detail features and mid-level semantic features to obtain multi-scale detection features; S11. During the training phase, the infrared small target detection network is trained based on bounding box regression loss, classification loss, and target confidence loss. During the inference phase, the multi-scale detection features are classified and predicted and bounding box regressed by the detection head to output the category, confidence, and location information of the infrared target. The trained infrared small target detection network is then used to detect small targets in the UAV infrared image.

2. The method according to claim 1, characterized in that, The deep thermal response map From deep semantic features The result is obtained by normalizing after aggregating the average response and the maximum response along the channel dimension. The calculation process includes: in, This indicates average aggregation of channels. This indicates the maximum aggregation of the channels, and Norm represents the normalization operation.

3. The method according to claim 1, characterized in that, The local contrast difference map According to the deep thermal response diagram The absolute difference between the response and the mean response of its neighborhood is obtained, and the calculation process includes: in, This represents a local average pooling operation with a window size of k, where the local contrast difference map is... Used to characterize the difference in local thermal response between weak targets and their surrounding background.

4. The method according to claim 1, characterized in that, The thermal response continuous loss diagram According to the deep thermal response diagram The continuity of the response within a local neighborhood is obtained. When a local region simultaneously satisfies the conditions of small response area, response discontinuity, and high neighborhood contrast, the corresponding location is in the thermal response continuity defect map. It is given a relatively large weight.

5. The method according to claim 1, characterized in that, The thermal response fragmentation occlusion estimation map According to the deep thermal response diagram Local contrast difference map Thermal response continuous loss diagram The concatenated data is obtained through convolution mapping and the Sigmoid function. Its calculation process includes: Where σ represents the Sigmoid function, Conv represents the convolution mapping operation, and [ ] represents the feature concatenation operation.

6. The method according to claim 1, characterized in that, The background heat source suppression map According to the deep thermal response diagram The local diffusion response, local maximum response, and edge gradient response are generated to suppress background heat source regions that are large in area, have slowly changing boundaries, and lack local abrupt changes in small targets.

7. The method according to claim 1, characterized in that, The occlusion condition features satisfy: The occlusion condition channel semantic weight Depend on It is obtained through global average pooling, global max pooling and lightweight mapping network, and is used to enhance the target-related semantic channel under the joint constraints of occlusion estimation region and background suppression region.

8. The method according to claim 1, characterized in that, The lightweight residual enhancement feature It is generated sequentially by depthwise separable convolution and 1×1 convolution, and is used to compensate for the local context information of small targets in deep semantic features without significantly increasing the number of model parameters.

9. The method according to claim 1, characterized in that, The recalibrated deep semantic features satisfy: Wherein, β is the restricted residual injection coefficient, which is restricted to a preset range [0, β_max] to avoid excessive perturbation of the original deep semantic distribution by the enhanced features.

10. A UAV infrared small target detection device based on thermal response fragmentation occlusion estimation and background heat source suppression gating, characterized in that, It includes an image acquisition unit, a backbone feature extraction unit, a thermal response fragmentation occlusion recalibration unit, a feature fusion unit, and a detection output unit; wherein, the thermal response fragmentation occlusion recalibration unit is used to perform the thermal response fragmentation occlusion estimation, background heat source suppression, occlusion condition channel semantic weight generation, and restricted residual injection operations as described in any one of claims 1 to 9.

Images