Infrared target detection method and device based on dynamic sparse perception and storage medium

Abstract

The application discloses an infrared target detection method based on dynamic sparse perception and belongs to the technical field of computer vision and artificial intelligence. First, infrared image data sets are collected and preprocessed. Then, an infrared target detection model EHD-RTDETR is constructed, and the preprocessed infrared image data sets are input into the infrared target detection model for training and evaluation, so that the infrared target detection model after training and evaluation is obtained. The infrared target detection model comprises a dynamic sparse perception module, an intensity perception module and a high-resolution adaptive fusion network. Finally, the infrared image to be detected is input into the infrared target detection model after training and evaluation for end-to-end detection, and the detection result is output and the model performance is evaluated. The application can effectively suppress complex background thermal noise interference and significantly improve the detection precision of infrared targets in complex scenes. The application also discloses an electronic device and a storage medium for realizing the above method.

Description

Technical Field

[0001] This invention relates to an infrared target detection method, device, and storage medium, specifically to an infrared target detection method, device, and storage medium based on dynamic sparse perception, belonging to the fields of computer vision and artificial intelligence technology. Background Technology

[0002] With the development of infrared thermal imaging technology, it has become a core technology for environmental perception in fields such as intelligent transportation, urban security, industrial defect detection, and drone inspection, thanks to its inherent advantages such as all-weather operation, strong resistance to visible light interference, and ability to penetrate smoke and fog. In these complex application scenarios, infrared target detection is a key link in achieving situational awareness, risk warning, and autonomous decision-making. However, due to the limitations of the physical mechanism of infrared imaging, infrared images generally suffer from low resolution, lack of texture details, and low signal-to-noise ratio. Especially in complex environments such as nighttime, rain, and fog, targets often exhibit weak feature representation and low contrast. Coupled with complex background interference such as ground thermal reflection and dense heat sources, accurate and efficient detection of infrared targets has become a severe technical challenge.

[0003] Currently, deep learning-based object detection algorithms have become the mainstream technology in this field, mainly divided into two categories: two-stage and single-stage. While two-stage detection algorithms, represented by Faster R-CNN, offer high detection accuracy, their complex processing flow, high computational cost, and significant inference latency make them unsuitable for the real-time and lightweight requirements of practical engineering scenarios. In contrast, single-stage algorithms, represented by the YOLO and DETR series, significantly improve detection speed through end-to-end network structures, making them more suitable for deployment on edge devices with limited computing power. The YOLO series algorithms, in particular, have been widely studied for their efficiency; however, these algorithms rely excessively on the local receptive field of convolutional operations, making it difficult to capture long-range global dependencies. When processing infrared images with weak texture features and low contrast, they are highly susceptible to interference from complex backgrounds, leading to increased false positive and false negative rates. Furthermore, their reliance on non-maximum suppression post-processing often results in false suppression when dealing with dense targets, further limiting detection performance.

[0004] In recent years, the Transformer architecture has made breakthroughs in the field of object detection. Its representative model, the Detection Transformer (DETR), achieves end-to-end matching and localization, eliminating steps such as candidate box generation and non-maximum suppression, and possesses a natural advantage in global modeling. The Real-Time Detection Transformer (RT-DETR), as a real-time object detection model based on the DETR architecture, achieves a better balance between detection speed and accuracy by integrating an efficient hybrid encoder and an uncertainty-guided query mechanism, providing a more advantageous structural foundation for infrared object detection in complex scenes.

[0005] However, existing RT-DETR models still have significant limitations when directly applied to infrared target detection tasks, mainly in the following three aspects: First, the traditional residual structure of the backbone network has a large amount of computational redundancy in the infrared feature extraction process, which not only increases the number of model parameters and computational overhead, restricting the deployment efficiency of edge platforms, but also fails to optimize the extraction path for the sparse features of infrared images and lacks the ability to actively suppress background thermal clutter; Second, the same-scale interaction module of the encoder only carries out feature interaction in the spatial domain, and cannot distinguish targets from high-energy background clutter based on thermal radiation characteristics, and has insufficient ability to enhance weak target features under low signal-to-noise ratio, which easily leads to model false detection problems; Third, the neck feature fusion module has insufficient ability to represent infrared targets with small size and weak signal, and traditional downsampling strategies are prone to irreversible dissipation of infrared weak target features. Semantic misalignment problems in cross-scale fusion will also further lead to an increase in the model's false detection rate. Summary of the Invention

[0006] To address the core issues in existing technologies, such as the easy dissipation of infrared target features, the difficulty in suppressing background thermal clutter interference, and the inability to simultaneously achieve detection accuracy and model lightweighting, this invention provides a low-complexity, high-precision, and highly robust infrared target detection method. It can effectively suppress complex background thermal clutter interference, significantly improve the detection accuracy of infrared targets in complex scenarios, and at the same time greatly reduce the number of model parameters and computational load, lower the computing power threshold for model deployment, and effectively enhance the all-weather infrared sensing capabilities of various edge devices in complex environments.

[0007] To achieve the above objectives, the present invention provides an infrared target detection method based on dynamic sparse sensing, comprising the following steps:

[0008] Step 1: Acquire infrared image datasets and preprocess them;

[0009] Step 2: Construct an infrared target detection model and input the preprocessed infrared image dataset into the infrared target detection model for training and evaluation to obtain the trained and evaluated infrared target detection model;

[0010] The infrared target detection model includes a dynamic sparse sensing module, an intensity sensing module, and a high-resolution adaptive fusion network.

[0011] Step 3: Input the infrared image to be detected into the trained and evaluated infrared target detection model for end-to-end detection, output the detection results and complete the model performance evaluation.

[0012] Furthermore, in step 2, the specific implementation process of the infrared target detection model includes:

[0013] The preprocessed image is input into the backbone network of the infrared target detection model, and the initial feature map set is obtained through multi-layer feature extraction by the C2f module and the dynamic sparse perception module.

[0014] The initial feature map set output by the backbone network is input into the neck network of the infrared target detection model. The feature is processed by the intensity sensing module and the high-resolution adaptive fusion network to output the final detection feature map.

[0015] The final detection feature map output by the neck network is input into the decoder to generate the final target category and bounding box coordinates.

[0016] Furthermore, in step 2, the implementation process of the dynamic sparse sensing unit in the dynamic sparse sensing module includes:

[0017] The input features are decoupled along the channel dimension to obtain the first channel features and the second channel features.

[0018] The first channel features are feature-mapped to generate prompt features, which are then subjected to global average pooling to obtain the number of retained tokens.

[0019] Sparse attention is calculated on the first channel features, and feature recalibration is performed based on the number of retained tokens to obtain the global enhanced features;

[0020] The global enhancement features and the second channel features are concatenated along the channel dimension and then processed by a linear projection layer to obtain the output features.

[0021] Furthermore, the implementation process of sparse attention computation and feature recorrection includes:

[0022] First, linear projection processing is performed on the features of the first channel to generate a query matrix, a key matrix, and a value matrix, and then the attention score matrix is ​​calculated.

[0023] Then, the attention score matrix is ​​evaluated using the number of retained tokens. Filtering operation to generate a sparse mask;

[0024] Next, the attention score matrix is ​​recalibrated using a sparse mask, and the recalibrated sparse attention output is used as the generated global enhancement feature.

[0025] Furthermore, in step 2, the specific process of the intensity sensing module includes:

[0026] The input feature map is split along the channel dimension using dynamic range convolution, and a global sorting operation based on pixel thermal radiation intensity is performed on the features of one branch.

[0027] The features obtained after global sorting are concatenated with the features from another branch, and then convolutional processing is performed to obtain intermediate features.

[0028] Histogram self-attention is calculated on the intermediate features, and the query vector and key vector are aligned and rearranged based on the intensity scalar of the value vector. A bi-branch attention map is calculated based on both box histogram reorganization and frequency histogram reorganization strategies, as shown below:

[0029] ,

[0030] ,

[0031] in, and These are the box histogram reconstructed branch attention map and the frequency histogram reconstructed branch attention map, respectively. For normalized exponential functions, This is a reshaping operation for box histogram reconstruction. This is a reshaping operation for frequency histogram reconstruction. and These are the query vector and key vector for the box histogram reorganization branch, respectively. and These are the query vector and key vector of the frequency histogram reorganization branch, respectively. For feature channel dimension, For value vectors, This represents the matrix transpose operation.

[0032] Furthermore, the specific process of the high-resolution adaptive fusion network in step 2 is as follows:

[0033] First, the input feature map is fed into the spatial depth convolution module, and the spatial depth layer divides the input feature map into multiple sub-feature maps;

[0034] Then, all sub-feature maps are stitched together along the channel dimension, and the recombined high-dimensional channels are compressed and filtered through convolutional layers;

[0035] Next, the feature map output by the spatial depth convolution module is input into the CSP full-core module for processing to obtain the output features of the high-resolution adaptive fusion network;

[0036] The CSP full-core module includes a full-core network consisting of local branches, large-core branches, and global branches. The local branches use 1×1 convolutions to capture local edge responses. The large-core branches use 33×1, 33×33, and 1×33 convolutions to construct a large receptive field that covers a wide range of context. The global branches introduce dual-domain channel attention and frequency-domain spatial attention to enhance the perception and retention of weak signals.

[0037] Furthermore, in step 3, the performance evaluation metrics for the model include: precision, recall, F1 score, mean precision, number of model parameters, and computational complexity.

[0038] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0039] The present invention also provides a storage medium storing a computer program or instructions that, when the computer program or instructions are run on a computer, execute the steps of the method described.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] (1) To address the core shortcomings of existing technologies, such as the inability to balance detection accuracy and model lightweighting, and the difficulty in deploying on edge devices, this invention reconstructs the backbone network through a dynamic sparse sensing module. Employing a dual-stream decoupling design and a dynamic sparse sensing mechanism, it actively filters background redundancy and thermal clutter at the feature extraction source, performing global attention calculations only on a subset of channels, fundamentally reducing redundant floating-point operations and memory accesses. Experimental results show that using this module alone can improve the model's mAP50 on the FLIR dataset by 1.3%, while reducing the number of model parameters by 32.2% and the computational load by 12.3%. This significantly compresses the model size while achieving active suppression of background thermal clutter and enhanced extraction of effective features.

[0042] (2) To address the core shortcomings of existing technologies, such as the spatial domain attention mechanism being easily dominated by high-energy background thermal clutter and the weak target features being easily submerged by smoothing, this invention extends feature interaction from the traditional spatial domain to the intensity domain through an intensity perception module. It aggregates non-adjacent pixels with similar radiation characteristics in the logical space through thermal radiation intensity ranking and clustering, and combines this with a dual-branch histogram self-attention mechanism to achieve precise enhancement of infrared target features and effective suppression of background interference. Experimental results show that this module can improve the model's mAP50 by 1.5%, significantly enhance the feature separability of distant small targets and occluded targets, greatly reduce false detections and false negatives in complex scenes, and improve the model's detection robustness in low signal-to-noise ratio environments.

[0043] (3) To address the core defect in existing technologies where infrared target features are prone to irreversible dissipation during depth downsampling, this invention utilizes a high-resolution adaptive fusion network to construct a high-fidelity signal transmission link at the physical level using spatial depth convolution. This avoids the direct discarding of infrared target edge and contour information by traditional stride convolution. Combined with the CSP full-core module, it achieves multi-dimensional feature reshaping in the frequency and spatial domains, effectively resolving the structural conflict between deep semantic abstraction and shallow physical information preservation. Experimental results show that introducing only a high-resolution adaptive fusion network can improve the model's mAP50 on the FLIR dataset by 1.6%, significantly improving the feature fidelity and cross-scale fusion consistency of small infrared targets.

[0044] (4) In summary, this invention achieves an excellent balance between detection accuracy and model lightweighting through the synergistic optimization of the three core modules. On the FLIR dataset, the complete model of this invention improves mAP50 and P by 3.4% and 3.3% respectively compared to the baseline model RT-DETR-R18, while significantly reducing the number of model parameters by 25.1%. On the HIT-UAV and M3FD datasets, mAP50 is improved by 2.3% and 1.1% respectively, demonstrating strong generalization ability in different scenarios. This method effectively solves the long-standing technical contradiction between accuracy, speed, and model resource consumption in infrared target detection tasks, providing an effective technical approach for the practical deployment of this technology on edge platforms with limited computing power, and possesses strong engineering application value and practicality. Attached Figure Description

[0045] Figure 1 This is an overall flowchart of the present invention;

[0046] Figure 2 This is a schematic diagram of the overall network structure proposed in this invention;

[0047] Figure 3 This is a schematic diagram of the dynamic sparse sensing module structure designed in this invention;

[0048] Figure 4 This is a schematic diagram of the dynamic sparse sensing unit structure designed in this invention;

[0049] Figure 5 This is a schematic diagram of the intensity sensing module structure designed in this invention;

[0050] Figure 6 This is a schematic diagram of the spatial depth convolution structure designed in this invention;

[0051] Figure 7 This is a schematic diagram of the CSP full-core module structure designed in this invention;

[0052] Figure 8 This is a schematic diagram of the all-core network structure designed in this invention. (a) is the all-core network, (b) is the dual-domain channel attention, (c) is the frequency domain spatial attention, and (d) is the residual set.

[0053] Figure 9 The image shows a comparison of the detection results of the proposed method and mainstream detection methods. (a) is the original image, (b) is the detection result of YOLOv9m, (c) is the detection result of YOLOv11m, (d) is the detection result of RT-DETR-R18, and (e) is the detection result of the model of the present invention. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0055] Example 1:

[0056] This embodiment provides an infrared target detection method based on dynamic sparse sensing, the implementation process of which is as follows: Figure 1 As shown, the specific steps are as follows:

[0057] Step 1: Acquire infrared image dataset and perform preprocessing. Preprocessing includes dividing the data into training, validation, and test sets, with uniform image sizes. The specific steps include:

[0058] Step 1-1: Obtain three publicly available infrared datasets: FLIR, HIT-UAV, and M3FD. The FLIR dataset contains 10,742 vehicle-mounted infrared images, acquired by the Teledyne FLIR Tau2 thermal imaging camera, with a high resolution of 640×512, covering five core categories in road scenes: pedestrians, bicycles, cars, motorcycles, and buses. The HIT-UAV dataset contains 2,898 infrared thermal images taken by drones from high altitude, covering various real-world environments such as campuses, parking lots, roads, and playgrounds, containing five target categories with a total of approximately 25,000 targets. The M3FD dataset contains 4,200 pairs of strictly aligned visible light and infrared images, totaling 34,407 target instances, covering six categories: pedestrians, cars, buses, motorcycles, trucks, and streetlights, comprehensively covering various environments, lighting conditions, and weather conditions.

[0059] Steps 1-2: Randomly divide the acquired datasets into training, validation, and test sets in a 7:2:1 ratio. The FLIR dataset has 7520 training images, 2148 validation images, and 1074 test images; the HIT-UAV dataset has 2029 training images, 579 validation images, and 290 test images; and the M3FD dataset has 2940 training images, 840 validation images, and 420 test images.

[0060] Steps 1-3: Scale all images to 640×640 pixels to complete the standardization preprocessing of the dataset.

[0061] Step 2: Construct an infrared target detection model based on RT-DETR, denoted as EHD-RTDETR, and input the training and validation sets into the improved EHD-RTDETR model for training, finally generating a model weight file. Figure 2 As shown, the model comprises three parts: a backbone network, a neck network, a decoder, and a head. Before the specific steps, the core modules of this invention constitute a collaborative optimization mechanism: First, the dynamic sparse perception module of the backbone network actively filters spatial redundancy to extract features; then, the intensity perception module of the neck network uses thermal radiation characteristics to separate weak targets from background clutter; finally, the high-resolution adaptive fusion network performs cross-scale high-fidelity fusion of the enhanced features. The collaborative optimization of these three components achieves a balance between accuracy and lightweight design. Step 2 specifically includes the following sub-steps:

[0062] Step 2-1: Input the preprocessed image into the backbone network of the infrared target detection model, and extract multi-layer features by passing through the convolutional layer, C2f module and dynamic sparse perception module in sequence. The features are extracted according to the network depth to obtain an initial feature map set {S3,S4,S5} containing three different spatial resolutions and semantic levels.

[0063] The core objective of this step is to actively filter out redundant spatial information at the source of feature extraction, achieve preliminary feature purification and high signal-to-noise ratio calibration, and complete the preliminary extraction and enhancement of infrared key target features while significantly reducing the computational load.

[0064] The dynamic sparse sensing module adopts a cross-stage partial fusion design concept, and its structure is as follows: Figure 3 As shown. First, the input features undergo feature mapping and channel adjustment in the first convolutional layer; then, feature splitting is performed, dividing the features into two parts along the channel dimension. One part serves as an initial bypass branch directly to the end of the module to retain high-frequency infrared physical information and maintain a stable gradient flow; the other part is then sequentially processed... A series of cascaded dynamic sparse sensing units are used for deep feature extraction and background noise filtering. To achieve multi-level feature reuse, each sensing unit's output, except for the initial bypass, leads to an independent bypass branch. Finally, the initial directly connected features and the bypass features from all units are concatenated and then passed through the last second convolutional layer to complete cross-channel feature fusion and dimensionality reduction, outputting a purified feature map with a high signal-to-noise ratio.

[0065] As the core processing unit of the dynamic sparse sensing module, the dynamic sparse sensing unit adopts a dual-stream decoupling strategy. This strategy aims to maximize the preservation of original physical information while reducing computational redundancy in global modeling by constructing differentiated feature processing paths, thereby achieving efficient processing of infrared scenes. Figure 4 As shown.

[0066] Specifically, for input features (Where C is the number of channels, and H and W are the height and width of the feature map, respectively), the module first performs decoupling along the channel dimension, dividing it into two independent processing paths: one is a lightweight global information interaction path, selecting the first... One channel performs only single-head self-attention D to model the overall thermal radiation distribution with low computational cost; the other is a static fidelity path that carries local information, retaining the remaining... Each channel undergoes no attention calculation, thus fully preserving the original pixel-level high-frequency details of the infrared target transmitted from the upstream module. The formal description of this process is as follows:

[0067] ,

[0068] in, Represents a lightweight global information interaction path. Represents a static fidelity path. This indicates a segmentation operation performed along the channel dimension of the feature map.

[0069] Subsequently, the module only interacts with the global branch. To perform single-head self-attention computation and transform infrared pixels in physical space into semantically related feature vectors, the module first generates a query (Query, ), key ) and Value The matrix is ​​used to calculate the initial attention score matrix A.

[0070] ,

[0071] ,

[0072] in, , , These represent the linear projection weight matrices, Scaling factor It characterizes the original thermal radiation correlation intensity between global pixels, and T represents the transpose operation.

[0073] Because infrared images have blurred boundaries, indiscriminate global computation leads to significant feature redundancy. To address this issue, the module designs a cue-guided feature reweighting mechanism. By optimizing feature selection priority through cue features, it significantly reduces computational redundancy while greatly enhancing the ability of sparse attention to capture key thermal radiation features. First, lightweight mapping captures the saliency distribution of the scene, generating cue features. :

[0074] ,

[0075] Subsequently, to adapt to the different interaction granularity requirements of various infrared scenarios, the module adjusted the prompt features. Perform Global Average Pooling (GAP) to extract scene statistical factors. ∈(0,1), this factor serves as an adaptive coefficient for normalization, representing the proportion of effective information in the current image; based on this, the module dynamically calculates the number of retained tokens. (Round down to the nearest integer, minimum value is 1):

[0076] , ,

[0077] in, This represents the function that takes the maximum value.

[0078] Get dynamic After the value is obtained, the module uses it to evaluate the attention score matrix. Recalibrate, specifically by performing the following steps on the attention score matrix: Each row in (i.e., each query vector) Only the highest-scoring players are retained. Elements are used to generate a binary sparse mask matrix. To filter out global redundant interference, its mathematical definition is:

[0079] ,

[0080] Among them, subscript Represents the row index of the matrix (corresponding to the row number). (query vectors), index Represents the column index of the matrix (corresponding to the first column index) (key vectors)

[0081] Finally, the module uses the recalibrated sparse attention output as the generated global augmentation feature. and compared it with high-fidelity streams splicing along the channel dimension, through A linear projection layer, composed of convolutions, is used to process the stitched features. The linear projection layer utilizes a kernel size of... The convolutions perform a cross-channel linear weighted combination on the recombined high-dimensional channels, reducing their dimensionality and mapping them back to the original feature channel dimensions, thereby completing deep cross-domain information fusion.

[0082] ,

[0083] ,

[0084] In this context, ⊙ represents element-wise multiplication. This is the final output projection weight.

[0085] After being processed by the backbone network through multiple layers, the 640×640 pixel input image is transformed into an initial multi-scale feature map set {S3,S4,S5}, which contains three different spatial resolutions and semantic levels, and will serve as the input for the subsequent efficient hybrid encoder.

[0086] Step 2-2: The initial multi-scale feature map set output by the backbone network is fed into the neck network for multi-scale feature fusion. The neck network adopts a bidirectional fusion architecture of top-down and bottom-up. In order to enhance the global semantic representation in the early stage of fusion, the deep feature branches with the smallest resolution are fed into the intensity-aware module for intensity domain feature interaction reconstruction, and the feature map with enhanced global saliency is output.

[0087] Step 2-2 applies the intensity perception module to the deep feature branch with the smallest resolution because this layer of feature maps has the largest receptive field and the richest global abstract semantics. Performing global self-attention calculation here can efficiently capture the global thermal radiation distribution state with minimal computational cost and effectively avoid local clutter noise interference in shallow high-resolution features. This module extends the feature interaction dimension from a single spatial domain to the intensity domain, reconstructing feature sequences based on pixel radiation intensity. By aggregating non-adjacent pixels with consistent radiation characteristics in the logical space, it overcomes the constraints of physical distance to accurately lock specific thermal distribution areas, significantly improving the global saliency and separability of infrared targets in complex backgrounds. The intensity perception module consists of two core components: dynamic range convolution and histogram self-attention, as shown below. Figure 5 As shown.

[0088] During the feature preprocessing stage, the module performs dynamic range convolution operations on the input feature map. The process involves splitting the data along the channel dimension. One branch is retained as an identity mapping, while the other branch undergoes sorting operations based on pixel thermal radiation intensity along both the height (H) and width (W) dimensions of the space. Next, the features sorted along the H and W dimensions are concatenated with the aforementioned identity mapping branch and input into a convolutional module consisting of 1×1 convolutions and 3×3 depthwise separable convolutions (DWConv). This preprocessing operation based on dynamic range convolution forces non-adjacent pixels with similar radiation characteristics to aggregate in the logical space, guiding subsequent convolutional kernels to focus on processing feature subspaces with consistent thermal radiation characteristics. This weakens background interference that does not match the target intensity distribution at the feature level, providing more discriminative feature representations for weak signal extraction.

[0089] Subsequently, the module introduces a histogram self-attention mechanism, based on value vectors. The intensity scalar for the query AND key Alignment and rearrangement are performed, and the features are quantized into B histogram bins. To balance global semantic representation and local detail preservation, the module computes bi-branch attention maps based on bin-based histogram reorganization (BHR) and frequency-based histogram reorganization (FHR) strategies, respectively. and : ,

[0090] ,

[0091] in, and These are the box histogram reconstructed branch attention map and the frequency histogram reconstructed branch attention map, respectively. For normalized exponential functions, This is a reshaping operation for box histogram reconstruction. This is a reshaping operation for frequency histogram reconstruction. and These are the query vector and key vector for the box histogram reorganization branch, respectively. and These are the query vector and key vector of the frequency histogram reorganization branch, respectively. For feature channel dimension, For value vectors, This represents the matrix transpose operation.

[0092] Finally, the outputs of the two branches are fused together through element-wise multiplication to obtain the final attention graph. The system then performs inverse rearrangement using index mapping to restore the original spatial location of the features, outputting a globally salient feature map enhanced in the intensity domain. This enhanced feature is then fed into the bidirectional cross-scale fusion stream of the neck network.

[0093] Specifically, in the top-down fusion process, deep features are upsampled and concatenated with mid-level features of the backbone network, then input into the reparameterization C3 module. Structural reparameterization is used to effectively fuse multi-branch features, improving processing speed and achieving semantic alignment. Meanwhile, the highest-resolution shallow features are processed independently by the subsequent high-resolution adaptive fusion network. Subsequently, in the bottom-up fusion process, the aligned features are downsampled and concatenated with corresponding deeper features, then input into the reparameterization C3 module again for interaction. Through this bidirectional cascaded fusion, full interaction of features from different receptive fields is achieved, thereby constructing a multi-scale feature pyramid.

[0094] Steps 2-3: The highest resolution shallow feature branch directly derived from the backbone network is fed into the high-resolution adaptive fusion network. The spatial depth convolution module and the CSP full-core module complete high-fidelity transmission and cross-scale feature fusion, and output the final detection feature map.

[0095] In steps 2-3, a high-resolution adaptive fusion network is constructed and deployed in the highest-resolution feature branch of the backbone network output. This fully utilizes the most complete fine-grained physical spatial information, such as infrared target edges and contours, preserved in this shallow branch, providing crucial information for the accurate localization of small and weak targets. This network solves the problems of irreversible loss of infrared target geometric structure information and semantic misalignment in cross-scale fusion caused by traditional downsampling by performing high-fidelity transmission and cross-scale fusion of this high-resolution feature, achieving synergistic optimization of deep semantic abstraction and shallow physical information preservation. The high-resolution adaptive fusion network consists of two parts: a spatial depth convolutional high-fidelity downsampling module and a CSP full-kernel semantic reshaping module.

[0096] In this module, the spatial depthwise convolution module replaces the traditional stride convolution, and the specific structure is as follows: Figure 6 As shown, it consists of a spatial-to-depth transform layer and a stride-free convolutional layer, avoiding the loss of infrared target edge and contour information caused by traditional downsampling that directly discards pixels. First, the input feature map is transformed by the SPD layer according to the scaling factor s. Periodic slicing and rearrangement are performed to divide the feature map into... Sub-feature maps (where, The downsampling step size (representing the spatial dimension) is used to transfer pixel information from the spatial dimension to the channel dimension with high fidelity. The spatial decomposition process is defined as follows:

[0097] ,

[0098] in, The generated sub-feature maps are represented by this. After concatenating all sub-feature maps along the channel dimension to generate a high-dimensional intermediate representation, the recombined high-dimensional channels are compressed and filtered through a convolutional layer with a stride of 1. This effectively avoids the physical signal degradation caused by traditional downsampling strategies and achieves high-fidelity downsampling of infrared target edge and contour information.

[0099] Furthermore, to compensate for the lack of high-level semantic information in the downsampled features mentioned above, the high-resolution adaptive fusion network concatenates and fuses the feature map output by the spatial depth convolution module with the upsampled features in the middle layer of the neck network. Then, the concatenated feature map is input into the cascaded CSP full-core module to transform the features that retain physical details into high-level semantic features.

[0100] like Figure 7As shown, the CSP full-core module adopts the CSP architecture to construct a dual-path transmission. Its specific processing flow is as follows: First, the input feature map is processed by a 1×1 convolutional layer for feature mapping and channel adjustment; then, the feature flow is divided into two parallel paths. The main path, which integrates the full-core network, inputs the features into the full-core network, focusing on the deep semantic reshaping of the features. Through multi-dimensional receptive field modeling, the class discriminability of the target features under strong background thermal clutter interference is greatly improved; at the same time, the lightweight path directly transmits the features and is responsible for maintaining stable gradient propagation in the network; next, the output features of the main path and the features of the lightweight path are connected to perform cross-path fusion; finally, the fused features are again processed by a 1×1 convolutional layer for cross-channel information integration, outputting the final high-level semantic features.

[0101] The specific structure of the all-core network is as follows: Figure 8 As shown, multi-scale receptive field modeling from local details to global context is achieved through three parallel branches: the local branch uses 1×1 convolution to focus on the fine-grained geometric features of infrared targets, capture local edge responses, and compensate for the lack of details in infrared images; the large kernel branch uses decomposed 33×1, 33×33 and 1×33 convolutions to construct a large receptive field covering a wide range of context with low computational cost, and uses background information to assist detection in the case of missing texture; the global branch introduces dual-domain channel attention (DCAM) and frequency-domain spatial attention (FSAM) to further enhance the perception and preservation of weak signals from the frequency domain perspective.

[0102] Specifically, DCAM first applies Frequency Channel Attention (FCA), performing channel modulation in the frequency domain through fast FCA to separate the high-frequency abrupt change signal of the infrared target from the low-frequency background clutter. Its expression is as follows:

[0103] ,

[0104] in, and These represent the Fast Fourier Transform (FFT) and its inverse, the Inverse Fourier Transform (IFFT), respectively. GAP stands for Global Average Pooling. This represents element-wise multiplication. This represents the feature map input to the global branch. Indicating frequency channel attention module The weight matrix of the convolutional layer.

[0105] Subsequently, FCA's output The signal is fed into the Spatial Channel Attention (SCA) module, where spatial domain channel attention enhancement is performed to obtain the final output of DCAM:

[0106] ,

[0107] in, Indicating spatial channel attention module The weight matrix of the convolutional layer.

[0108] The output of DCAM is then fed into FSAM, where pixel-level refinement of the spectrum is performed along the spatial dimension, refining the granularity of global frequency domain awareness from the channel level to the spatial pixel level. The operation is defined as follows:

[0109] ,

[0110] in, and These represent the two independent branches used for feature transformation in the dual-branch structure of the frequency domain spatial attention module. The weight matrix of the convolutional layer.

[0111] The feature maps of the local branch, the large kernel branch, and the global branch are complementaryly fused through element-wise addition, achieving joint modeling of the frequency and spatial domains. Based on this, the network flattens and concatenates the high, medium, and low spatial resolution branch features output by the neck network. This multi-branch merging operation aims to complement the advantages of shallow high-resolution features containing rich edge geometric details with deep low-resolution features containing global semantics and radiation intensity priors, thereby constructing an aggregated feature body containing full-scale information.

[0112] Ultimately, this aggregated feature set will provide the most comprehensive cross-scale feature basis for subsequent IoU-aware query selection, ensuring that the model can simultaneously adapt to and accurately detect various weak and conventional targets with drastic scale changes in infrared scenes, and complete the output of high-quality detection features.

[0113] Steps 2-4: Feed the detection feature map output by the high-resolution adaptive fusion network into the Transformer decoder to parse the features and generate the final target category and bounding box coordinates;

[0114] In steps 2-4, the features output by the high-resolution adaptive fusion network are fed into the Transformer decoder. The purpose is to parse the feature maps output by the high-resolution adaptive fusion network and generate the final target category and bounding box coordinates. The decoder employs a Transformer decoder architecture. First, it initializes a set of learnable target queries and receives feature maps output from the high-resolution adaptive fusion network. In the multi-layered structure of the decoder, each target query iteratively interacts with image features through a cross-attention mechanism, thereby gradually and finely locating the target it represents and determining its category. Based on this, each updated query vector output from the last layer of the decoder is fed into independent classification and regression heads to generate the target's category confidence and bounding box coordinates, respectively.

[0115] Steps 2-5: Train the built model on the specified dataset, evaluate its performance on the validation set, and save and obtain the best weight file.

[0116] In this embodiment, the experimental environment was configured as follows: the system environment was Windows 11, the graphics card (GPU) was an NVIDIA GeForce RTX 4090, the deep learning framework was PyTorch 2.5.1, the acceleration environment was CUDA 12.1, and the language was Python 3.10. The input image size used in the experiments was uniformly 640×640, the number of training epochs was set to 300, the batch size was set to 4, no pre-trained weights were used in the models used in the experiments, the initial learning rate during the training phase was set to 0.0001, the optimizer was AdamW, the weight decay coefficient was set to 0.0001, and all experiments used consistent hyperparameters for training and validation.

[0117] Step 3: Based on the trained weights, input the infrared image to be detected into the model for end-to-end detection; use the test set images combined with the trained weight file to detect the target category and bounding box;

[0118] Furthermore, in step 3, the optimal weight file that has been trained is loaded, the infrared image to be detected is input into the model, and end-to-end forward inference is performed without the need for non-maximum suppression post-processing.

[0119] Step 4: Output the detection results and evaluate the model performance.

[0120] In step 4, the model's predicted target class, confidence score, and bounding box coordinates are output. Precision (P), Recall (R), F1 score, mean average precision (mAP50), number of parameters (Params), and computational complexity (GFLOPs) are used as core metrics to quantitatively evaluate model performance. Specifically, precision refers to the proportion of true positive samples among all predicted positive samples; recall measures the proportion of true positive samples correctly identified by the model; the F1 score is the harmonic mean of precision and recall, integrating the two metrics into a single evaluation indicator; mean average precision (mAP) is the average AP value across all classes, used to evaluate the model's overall detection performance on the entire dataset; GFLOPs measures the model's time complexity and computational cost; and Params reflects the model's space complexity. The formulas for calculating the core metrics are as follows:

[0121] , , , , ,

[0122] in, True cases refer to the number of samples that the model correctly predicts as positive. False positives refer to the number of negative samples that are incorrectly predicted as positive by the model. False negatives refer to the number of positive samples that are incorrectly predicted as negative by the model. Represents the total number of target categories. Representing the Average precision for each category.

[0123] To comprehensively evaluate the effectiveness of the model proposed in this invention for infrared target detection, this embodiment conducted comparative experiments with mainstream models, ablation experiments, generalization experiments, and visualization analysis to fully verify the effectiveness of the method of this invention.

[0124] 1. Comparison Experiment of Mainstream Models

[0125] To verify the effectiveness of this invention in infrared target detection tasks, this embodiment quantitatively compares it with current mainstream detection algorithms on the FLIR dataset, including YOLOv11m, YOLOv12m, and baseline models RT-DETR-R18, RT-DETR-R34, YOLOv11-ND, and LRAF-Net. Detailed results are shown in Table 1 below, where "-" in the table indicates that the metric was not reported in relevant literature.

[0126] Table 1. Performance comparison of different models on the FLIR dataset.

[0127]

[0128] As shown in Table 1 above, the model of this invention achieves good overall performance across all key indicators. In terms of detection accuracy, its P-value reaches 89.3%, and its mAP50 reaches 86.9%, outperforming comparative models including YOLOv11m, YOLOv12m, and RT-DETR-R34. By introducing a deep intensity sensing module, the model effectively suppresses background clutter using thermal radiation characteristics; combined with a high-resolution adaptive fusion network to preserve the physical information of shallow feature edges, it further reduces feature loss of small targets during downsampling. Furthermore, in terms of model complexity, the number of parameters in this embodiment is only 14.9M, the lowest among all comparative models. The dynamic sparse sensing module in the backbone network actively filters spatially redundant information during this process, effectively reducing the computational cost of the model for invalid backgrounds. In summary, compared with the benchmark model RT-DETR-R18, this embodiment improves mAP50 by 3.4 percentage points and accuracy by 3.3 percentage points while reducing the number of parameters by 25.1%, which fully verifies the effectiveness of the collaborative optimization mechanism of the three core modules proposed in this invention and achieves an effective balance between detection accuracy and model lightweighting.

[0129] 2. Ablation test

[0130] To verify the independent contributions and synergistic effects of the key technical features (dynamic sparse sensing module, intensity sensing module, and high-resolution adaptive fusion network) in this invention, multiple ablation experiments were conducted using RT-DETR-R18 as the baseline model. The results are shown in Table 2 below. In Table 2, the letters (A to H) in the first column represent different model configuration combinations constructed sequentially in the ablation experiments; "A.Base" represents the baseline model without any improved modules, and "H.This Invention" represents the final model that incorporates all three improved modules; the symbols in the table... This indicates that the corresponding module is not used in this model configuration. This indicates that the corresponding module is used in this model configuration.

[0131] Table 2 Ablation Experiment Results

[0132]

[0133] As shown in Table 2 above, when the dynamic sparse sensing module is introduced alone, the mAP50 improves by 1.3 percentage points, while the number of parameters and computational cost decrease significantly by 32.2% and 12.3%, respectively, significantly reducing model computational redundancy. When the intensity sensing module is introduced alone, the mAP50 improves by 1.5 percentage points, significantly enhancing the model's ability to perceive and recall weak infrared targets. When the high-resolution adaptive fusion network is introduced alone, the mAP50 improves by 1.6 percentage points, effectively solving the problem of feature loss during downsampling of small targets. When all three modules are integrated, the model performance reaches its optimal level, with P reaching 89.3% and mAP50 reaching 86.9%, while the number of parameters is still significantly lower than the baseline model. The results of this embodiment strongly demonstrate that the three core technical features of this invention all contribute positively to the final performance, and their combination can produce a significant synergistic gain effect.

[0134] 3. Generalization test

[0135] To verify the universality and robustness of the method of the present invention under different data distributions, this embodiment uses the HIT-UAV UAV infrared dataset and the M3FD multimodal dataset to test the generalization ability under the same experimental environment. The experimental results are shown in Table 3.

[0136] Table 3 shows the detection performance of the model on different datasets.

[0137]

[0138] As shown in Table 3 above, on the HIT-UAV high-altitude infrared unmanned aerial vehicle dataset, compared with the baseline RT-DETR-R18, the mAP50 of the model of this invention is improved by 2.3 percentage points; on the M3FD dataset, the mAP50 is improved by 1.1 percentage points, and the recall rate is significantly improved by 3.9 percentage points. The results of this embodiment demonstrate that the technical solution proposed in this invention is not only effective for specific datasets, but can still maintain good detection performance under different scenarios and data distributions, and has strong generalization ability.

[0139] 4. Visualized Results Analysis

[0140] This embodiment selects three challenging typical scenarios: multi-scale pedestrian detection in urban streets, fine-grained classification of multiple types of vehicles, and low-light, long-distance, small-scale target detection. The model of this invention is then compared horizontally with YOLOv9m, YOLOv11m, and the baseline model RT-DETR-R18 through visual visualization. Figure 9 As shown.

[0141] The results show that the proposed method can accurately identify multi-scale targets in infrared images against complex backgrounds. Regarding target localization accuracy, the detection box of this invention can more closely fit the target boundary; the high-resolution adaptive fusion network retains fine-grained physical and contour information of shallow features during this process, avoiding boundary blurring caused by downsampling in traditional models. In terms of scene robustness, the model significantly reduces false positives and false negatives. The dynamic sparse perception module and intensity perception module in the backbone and neck networks work together to actively filter redundant backgrounds and suppress clutter using thermal radiation characteristics, thus achieving stable capture of weak targets in extreme environments such as low light and long distances. This result fully demonstrates the effectiveness of this invention in improving the feature representation of weak targets and suppressing background interference, verifying the feasibility and superiority of the technical solution.

[0142] Example 2:

[0143] This invention provides an electronic device, including a processor and a memory. The memory stores program instructions for implementing the infrared target detection method based on dynamic sparse sensing of Embodiment 1. The processor executes the program instructions stored in the memory to perform infrared target detection.

[0144] The processor can also be called a Central Processing Unit (CPU). A processor may be an integrated circuit chip with signal processing capabilities. A processor can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0145] The methods described in Embodiment 1 of this invention are applicable to this electronic device embodiment. The specific functions implemented by this electronic device embodiment are the same as those in the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0146] Example 3:

[0147] This embodiment provides a storage medium storing a computer program or instructions capable of implementing the above-described infrared target detection method based on dynamic sparse sensing. When the computer program or instructions are run on a computer, the steps of the above-described infrared target detection method based on dynamic sparse sensing are executed.

[0148] The computer program or instructions can be stored in the aforementioned storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0149] The methods described in Embodiment 1 of this invention are all applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0150] This invention provides an infrared target detection method, device, and storage medium based on dynamic sparse sensing. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. An infrared target detection method based on dynamic sparse sensing, characterized in that, Includes the following steps: Step 1: Acquire infrared image datasets and preprocess them; Step 2: Construct an infrared target detection model and input the preprocessed infrared image dataset into the infrared target detection model for training and evaluation to obtain the trained and evaluated infrared target detection model; Step 3: Input the infrared image to be detected into the trained and evaluated infrared target detection model for end-to-end detection, output the detection results and complete the model performance evaluation.

2. The infrared target detection method based on dynamic sparse sensing according to claim 1, characterized in that, In step 2, the infrared target detection model includes a dynamic sparse sensing module, an intensity sensing module, and a high-resolution adaptive fusion network. Its specific implementation process includes: The preprocessed image is input into the backbone network of the infrared target detection model, and the initial feature map set is obtained through multi-layer feature extraction by the C2f module and the dynamic sparse perception module. The initial feature map set output by the backbone network is input into the neck network of the infrared target detection model. The feature is processed by the intensity sensing module and the high-resolution adaptive fusion network to output the final detection feature map. The final detection feature map output by the neck network is input into the decoder to generate the final target category and bounding box coordinates.

3. The infrared target detection method based on dynamic sparse sensing according to claim 2, characterized in that, The implementation process of the dynamic sparse sensing unit in the dynamic sparse sensing module includes: The input features are decoupled along the channel dimension to obtain the first channel features and the second channel features. The first channel features are feature-mapped to generate prompt features, which are then subjected to global average pooling to obtain the number of retained tokens. Sparse attention is calculated on the first channel features, and feature recalibration is performed based on the number of retained tokens to obtain the global enhanced features; The global enhancement features and the second channel features are concatenated along the channel dimension and then processed by a linear projection layer to obtain the output features.

4. The infrared target detection method based on dynamic sparse sensing according to claim 3, characterized in that, The implementation process of sparse attention computation and feature recorrection includes: First, linear projection processing is performed on the features of the first channel to generate a query matrix, a key matrix, and a value matrix, and then the attention score matrix is ​​calculated. Then, the attention score matrix is ​​evaluated using the number of retained tokens. Filtering operation to generate a sparse mask; Next, the attention score matrix is ​​recalibrated using a sparse mask, and the recalibrated sparse attention output is used as the generated global enhancement feature.

5. The infrared target detection method based on dynamic sparse sensing according to claim 2, characterized in that, The specific process of the intensity sensing module includes: The input feature map is split along the channel dimension using dynamic range convolution, and a global sorting operation based on pixel thermal radiation intensity is performed on the features of one branch. The features obtained after global sorting are concatenated with the features from another branch, and then convolutional processing is performed to obtain intermediate features. Histogram self-attention is calculated on the intermediate features, and the query vector and key vector are aligned and rearranged based on the intensity scalar of the value vector. A bi-branch attention map is calculated based on both box histogram reorganization and frequency histogram reorganization strategies, as shown below: ， ， in, and These are the box histogram reconstructed branch attention map and the frequency histogram reconstructed branch attention map, respectively. For normalized exponential functions, This is a reshaping operation for box histogram reconstruction. This is a reshaping operation for frequency histogram reconstruction. and These are the query vector and key vector for the box histogram reorganization branch, respectively. and These are the query vector and key vector of the frequency histogram reorganization branch, respectively. For feature channel dimension, For value vectors, This indicates the transpose operation.

6. The infrared target detection method based on dynamic sparse sensing according to claim 2, characterized in that, The specific process of the high-resolution adaptive fusion network includes: First, the input feature map is fed into the spatial depth convolution module, and the spatial depth layer divides the input feature map into multiple sub-feature maps; Then, all sub-feature maps are stitched together along the channel dimension, and the recombined high-dimensional channels are compressed and filtered through convolutional layers; Next, the feature map output by the spatial depth convolution module is input into the CSP full-core module for processing to obtain the output features of the high-resolution adaptive fusion network; The CSP full-core module includes a full-core network consisting of local branches, large-core branches, and global branches. The local branches use 1×1 convolutions to capture local edge responses. The large-core branches use 33×1, 33×33, and 1×33 convolutions to construct a large receptive field that covers a wide range of context. The global branches introduce dual-domain channel attention and frequency-domain spatial attention to enhance the perception and retention of weak signals.

7. The infrared target detection method based on dynamic sparse sensing according to claim 1, characterized in that, In step 3, the performance evaluation metrics for the model include: precision, recall, F1 score, mean precision, number of model parameters, and computational complexity.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the infrared target detection method based on dynamic sparse sensing as described in any one of claims 1 to 7.

9. A storage medium, characterized in that, The device stores a computer program or instructions that, when executed on a computer, perform the steps of the infrared target detection method based on dynamic sparse sensing as described in any one of claims 1 to 7.

Images