Infrared Small Target Detection Method and System Based on Cross-Modal Distillation

Abstract

This invention proposes an infrared small target detection method and system based on cross-modal distillation, belonging to the field of computer vision technology. Registered infrared-visible light images are input into a pre-trained teacher model. Multi-scale feature maps and attention maps are extracted through high- and low-resolution branches containing deformable convolutional modules and multi-head deformable attention modules, respectively, to fine-tune the teacher model. Based on an adaptive scale selection mechanism, the feature maps and attention maps of the corresponding branches of the teacher model are dynamically selected according to the target area. The student model is then distilled using L2 loss and KL divergence loss. The infrared image to be detected is input into the trained student model to obtain a unified detection result, thereby improving the small target detection accuracy by combining adaptive scale selection.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an infrared small target detection method and system based on cross-modal distillation. Background Technology

[0002] Infrared small target detection is a computer vision task aimed at recognizing small, feature-blurred targets in infrared images. It is widely used in security monitoring, remote sensing, and other scenarios. Examples include detecting pedestrians at a distance or small, concealed drones at night in security monitoring, and identifying small ground facilities or trace amounts of leaked gas at low altitudes in remote sensing. Because infrared imaging relies on the thermal radiation characteristics of objects, targets often lack texture details and have blurred edges, posing a natural challenge to accurate identification. Visible light images, on the other hand, provide rich texture and color details. When combined with infrared data, they can effectively compensate for the information limitations of infrared modalities, potentially improving detection performance.

[0003] However, the fundamental differences in imaging principles between infrared and visible light modes lead to vastly different feature distributions and information representations. This modal heterogeneity has become a core obstacle to multimodal data fusion, and how to break down modal barriers and achieve effective information complementarity has always been a key area of ​​exploration in the field.

[0004] Existing technologies have significant limitations in addressing the above problems: on the one hand, multimodal fusion often remains at the level of shallow feature splicing or simple weighting, making it difficult to uncover deep cross-modal correlations and resulting in insufficient transfer of fine-grained features; on the other hand, for infrared small targets of different scales, there is a lack of adaptive feature selection mechanisms, and when the target scale changes drastically in complex scenes, the model cannot accurately match the optimal features, resulting in large fluctuations in detection accuracy and insufficient robustness. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes an infrared small target detection method and system based on cross-modal distillation. By collaboratively training a teacher model and a student model, the advantages of both infrared and visible light dual-modal approaches are integrated to improve the accuracy of infrared small target detection, while ensuring that the student model is lightweight to meet practical application requirements.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides an infrared small target detection method based on cross-modal distillation, comprising:

[0008] Random modal occlusion is performed on the registered infrared-visible training image pairs, which are then input into the teacher model. Cross-modal features are extracted based on deformable convolution and attention mechanisms, and self-supervised pre-training is performed by reconstructing the occluded regions.

[0009] The registered infrared-visible light image is input into the pre-trained teacher model. Multi-scale feature maps and attention maps are extracted through high and low resolution branches, and target detection and segmentation training are performed together to complete model fine-tuning.

[0010] Using the finely tuned teacher model as a fixed teacher, a lightweight student model is trained using only infrared images. An adaptive scale selection mechanism is used to select the output of the corresponding branch of the teacher and perform knowledge distillation of feature maps and attention maps.

[0011] The infrared image to be detected is input into the trained student model, and the target bounding boxes output by its detection branch are fused with the target bounding boxes obtained by post-processing the segmentation branch mask to obtain a unified detection result.

[0012] Secondly, the present invention provides an infrared small target detection system based on cross-modal distillation, comprising:

[0013] The teacher model pre-training module is used to perform random modal occlusion on the registered infrared-visible training image pairs, input them into the teacher model, extract cross-modal features based on deformable convolution and attention mechanisms, and perform self-supervised pre-training by reconstructing the occluded regions.

[0014] The teacher model fine-tuning module is used to input the registered infrared-visible light image into the pre-trained teacher model, extract multi-scale feature maps and attention maps through high and low resolution branches, and jointly perform target detection and segmentation training to complete model fine-tuning;

[0015] The student model distillation training module is used to train a lightweight student model using only infrared images with a fine-tuned teacher model as a fixed teacher. It selects the output of the corresponding branch of the teacher through an adaptive scale selection mechanism to perform knowledge distillation of feature maps and attention maps.

[0016] The target detection template is used to input the infrared image to be detected into the trained student model, and fuse the target bounding boxes output by its detection branch with the target bounding boxes obtained by post-processing the segmentation branch mask to obtain a unified detection result.

[0017] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the infrared small target detection method based on cross-modal distillation described in the first aspect.

[0018] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the infrared small target detection method based on cross-modal distillation described in the first aspect.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] (1) This invention integrates the advantages of infrared and visible light through cross-modal distillation. The teacher model is pre-trained with modal occlusion to improve feature recovery ability. During fine-tuning, deformable attention is used to enhance fine-grained feature extraction. The student model is lightweight and inherits performance through distillation. The adaptive scale selection mechanism adapts to different target scales. Combined with segmentation-assisted supervision, the student model can efficiently detect medium-to-large and small targets using only infrared input. While ensuring the model is compact, it significantly improves detection accuracy and robustness, and achieves high-precision and stable detection of small infrared targets in complex scenes.

[0021] (2) The modal random masking operation of this invention performs local or whole-modal masking on infrared and visible light image pairs, forcing the teacher model to learn cross-modal feature associations and complementary information during pre-training. This mechanism can enhance the model's adaptability to modal loss, improve feature perception and recovery capabilities, lay the foundation for subsequent bimodal feature fusion, enable the model to more fully explore the deep association between the two modalities, effectively compensate for the lack of single-modal information, and enhance the robustness and effectiveness of feature extraction.

[0022] (3) The multi-head deformable attention module of the present invention dynamically adjusts the sampling position through a learnable offset vector, which enables the model to adaptively focus on key areas according to the target shape and scene changes. Compared with fixed sampling, it can accurately capture the blurred features of small infrared targets and the detailed information of visible light, improve the attention to targets in complex backgrounds, enhance the feature extraction capability of multi-scale and irregular targets, make cross-modal feature fusion more accurate, and provide more effective feature support for detection.

[0023] (4) The adaptive scale selection mechanism of this invention dynamically selects the feature map and attention map of the teacher model branch according to the target area, so that small targets are supervised by the fine features of the high-resolution branch, and large targets are guided by the overall features of the low-resolution branch. This adaptive selection allows the student model to learn the features of targets of different scales in a targeted manner during distillation, avoids information loss caused by scale mismatch, improves the adaptability to scale changes, enhances detection accuracy and stability, and especially improves the detection effect of small targets.

[0024] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0025] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.

[0026] Figure 1The main flowchart of an infrared small target detection method based on cross-modal distillation provided in this embodiment of the invention is shown below.

[0027] Figure 2 This is a schematic diagram of the teacher model backbone network structure with deformable convolution and multi-head deformable attention layers, as well as the knowledge distillation process of feature maps and attention maps, provided for embodiments of the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0029] Example 1

[0030] like Figure 1 As shown, this embodiment discloses an infrared small target detection method based on cross-modal distillation, including the following steps:

[0031] S1: Perform random modal occlusion on the registered infrared-visible training image pairs, input them into the teacher model, extract cross-modal features based on deformable convolution and attention mechanisms, and perform self-supervised pre-training by reconstructing the occluded regions;

[0032] S2: Input the registered infrared-visible light image into the pre-trained teacher model, extract multi-scale feature maps and attention maps through high and low resolution branches, and jointly perform target detection and segmentation training to complete model fine-tuning;

[0033] S3: Using the finely tuned teacher model as a fixed teacher, a lightweight student model is trained using only infrared images. The output of the corresponding branch of the teacher is selected through an adaptive scale selection mechanism, and knowledge distillation of feature maps and attention maps is performed.

[0034] S4: Input the infrared image to be detected into the trained student model, fuse the target box output by its detection branch with the target box obtained by post-processing the segmentation branch mask, and obtain a unified detection result.

[0035] Next, combined Figure 1 This embodiment provides a detailed description of an infrared small target detection method based on cross-modal distillation.

[0036] I. Teacher Model Pre-training

[0037] The teacher model backbone is an improvement upon the YOLOv5 CSPDarknet architecture, introducing deformable convolutional modules and multi-head deformable attention mechanisms in multiple feature extraction stages, such as... Figure 2 As shown, specifically, deformable convolutional modules are inserted after the 2nd, 3rd, and 4th C3 layers of the backbone network, and a multi-head deformable attention module is connected to the output of the deformable convolutional module after the last C3 layer.

[0038] Deformable convolution is an improved structure of traditional convolution, consisting of a standard convolutional layer and an offset prediction layer. First, the prediction layer learns the offset of the convolution kernel's sampling position, and then the convolution kernel adaptively adjusts the sampling region based on this offset. Deformable convolution can flexibly adjust the sampling range according to the shape and position of the target, resulting in more accurate feature capture for irregular and small-sized targets.

[0039] Deformable convolutions can enhance the adaptability of feature extraction, while the deformable attention mechanism in teacher networks uses learnable offset vectors. Adaptive adjustment of sampling position, expressed as:

[0040] ;

[0041] in, This represents the attention weight function. For query points, K and V represent key-value pairs, and N represents the number of key-value pairs. For the i-th key vector, Let be the i-th value vector.

[0042] Based on this formula, it can be seen that in the calculation of multi-head deformable attention, the query point Not a fixed sampling key-value pair Instead of the original position, it uses a learnable offset vector. Adaptive adjustment of each key vector The corresponding sampling location, i.e., the actual extraction Key features at the location, combined with attention weight function For the adjusted value vector The weighted sum is used to obtain the final attention output.

[0043] For example, if a small target in the detection scenario, such as a trace amount of leaked gas, undergoes stretching deformation, the offset vector... It will automatically shift towards the target deformation area, so that the attention sampling position is accurately focused on the actual contour area of ​​the gas, rather than fixed at the initial preset regular grid position.

[0044] In this embodiment, the introduction of a deformable convolution module allows the network to adaptively capture the deformation features of the target, overcoming the limitations of fixed convolution kernels; the multi-head deformable attention mechanism can accurately focus on key regions, enhancing the targeting of feature interactions. The synergy of these two mechanisms significantly improves the feature extraction accuracy and generalization ability of the teacher model for targets in complex scenes, laying the foundation for feature alignment between teacher and student models in subsequent knowledge distillation. This also helps the student model maintain detection performance while maintaining a lightweight design, and lowers the computational barrier for model deployment in grassroots work.

[0045] In the initial stage of model training, the teacher model undergoes self-supervised pre-training for dual-modal inputs in infrared and visible light. Specifically, for paired infrared and visible light image samples, random occlusion operations are performed on their modal dimensions, with the occlusion ratio controlled between 30% and 50%. The occlusion methods include local region occlusion and full modal occlusion. The former randomly selects local regions on the infrared or visible light image for occlusion, while the latter temporarily occludes one modality across the entire channel dimension.

[0046] By forcing the model to utilize cross-modal information from unmasked regions, such as using visible light texture features to complete the temperature features of infrared masked regions, the backbone network is forced to learn more robust cross-modal semantic associations. At the same time, random masking of local and global modalities can prevent the model from relying on complete information from a single modality, thereby improving its adaptability to scenarios with missing modalities and blurred local features.

[0047] The occluded image pairs are fed as input into the backbone network of the teacher model, and the original information of the occluded regions is recovered through a lightweight reconstruction head. The reconstruction head used by the model has a lightweight structure containing three upsampled convolutional blocks. Each convolutional block contains two convolutional layers with residual connections and one upsampled convolutional layer. The first convolutional block takes the feature map output by the SPPF of the backbone network as input. The second and third upsampled convolutional blocks concatenate the outputs of the third and second C3 layers of the backbone network with the output of the previous upsampled block along the channel dimension, respectively, as input. For the infrared modality, a grayscale image needs to be reconstructed; for the visible light modality, a three-channel RGB image needs to be reconstructed. The reconstruction loss at this stage is defined as a pixel-level L1 loss function.

[0048] ;

[0049] in, The normalized center point coordinates of the bounding box of the infrared gas target. This represents the set of pixel coordinates of the occluded area. Represents the pixel values ​​of the original image. This is the output for model reconstruction. The loss function optimizes the backbone parameters of the teacher model through backpropagation, improving its cross-modal semantic recovery capability and feature-aware robustness. The reconstruction head is only used during the pre-training phase of the teacher model and is discarded after pre-training is complete. Its main purpose is to enhance the feature extraction capability of the teacher model's backbone network.

[0050] This embodiment enhances the backbone network's cross-modal feature perception and semantic recovery capabilities through occlusion reconstruction, and improves the ability to accurately focus on small targets through a deformable attention mechanism. This enables the teacher model to have robust multi-scale feature representation, laying the foundation for subsequent knowledge distillation to transfer high-quality feature knowledge to the student model. It effectively ensures that the student model can maintain excellent small target detection accuracy even after being lightweight.

[0051] II. Fine-tuning of the teacher model

[0052] After pre-training is completed, the teacher model enters the fine-tuning stage for detection and segmentation tasks, with the input being the registered infrared and visible light image pairs.

[0053] During forward inference, the teacher model infers each fused image using both high-resolution and low-resolution input sizes, obtaining two complete sets of multi-scale feature maps and attention maps. The high-resolution path more clearly captures the edges of small targets and hotspot details, while the low-resolution path is beneficial for representing the overall target structure. In each forward inference iteration, the network simultaneously outputs detection and segmentation results; the segmentation head receives shallow features to generate pixel-level mask predictions. This provides auxiliary supervision, especially for extremely small targets. The detection head outputs the target's bounding box coordinates, classification probability, and confidence score, where the classification loss uses FocalLoss.

[0054] ;

[0055] in The predicted probability for the target category. As a category balance factor, The focus parameter is used to alleviate the training bias caused by sample imbalance.

[0056] The bounding box regression loss uses the CIoU loss form and is defined as follows:

[0057] ;

[0058] in, Intersection over Union (IoU) is used to measure the degree of overlap between the predicted bounding box and the ground truth bounding box. and These are the center points of the predicted bounding box and the ground truth bounding box, respectively. For Euclidean distance, The diagonal distance , This is a shape constraint.

[0059] The segmentation head outputs a pixel-level mask to supplement the spatial information of small targets. Its loss function is a weighted combination of Dice loss and cross-entropy loss.

[0060] ;

[0061] in For model prediction mask, To accurately label the mask, , These are the weighting coefficients. This indicates Dice's loss. This represents the cross-entropy loss.

[0062] The total loss function of the teacher model is defined as:

[0063] ;

[0064] in, where is the weighting coefficient. This loss is used to jointly optimize the detection and segmentation branches during the supervised fine-tuning phase of the teacher model.

[0065] In this embodiment, dual-path inference with high and low resolution is used to take into account both the details of small targets and the overall structure, thereby improving the accuracy of detection and segmentation. FocalLoss is introduced to alleviate sample imbalance, CIoU loss is used to optimize bounding box regression, and Dice and cross-entropy combined loss is used to enhance the segmentation effect. The joint optimization of multiple losses achieves the synergistic improvement of detection and segmentation branches, effectively enhancing the detection and segmentation performance of infrared-visible light images for small and medium-sized targets, adapting to task requirements while improving the robustness of the model.

[0066] III. Student Model Training

[0067] The student model uses only infrared images as input and structurally inherits the teacher model's dual-branch configuration of detection and segmentation, but its backbone network and head structure have fewer channels and fewer parameters compared to the teacher model. During distillation, all trainable parameters of the teacher model are frozen, and only the student model is trained. The student model forms an indirect multimodal learning path through distillation with the teacher network. The feature map distillation loss is expressed as:

[0068] ;

[0069] in, This represents the feature map output by the s-th layer of the student model. This represents the feature map output by the s-th layer of the teacher model. This represents the set of feature layers participating in feature distillation. This represents the MLP layer used to map features from the student model to the feature map scale of the teacher model. The corresponding attention map distillation loss is:

[0070] ;

[0071] in, Denotes KL divergence, and These respectively represent the teacher and student in the first... The attention map of the layer is measured using KL divergence to assess its spatial attention distribution, guiding the student network to learn the teacher model's ability to focus on the target region. Therefore, the total distillation loss can be expressed as:

[0072] ;

[0073] in, This is the total characteristic distillation loss. It is a hyperparameter used to balance the attention map distillation loss and L2 loss.

[0074] Considering the uneven distribution of target sizes, this invention introduces an adaptive scale selection mechanism. For each pre-labeled target region in the training set, a high-resolution or low-resolution path from the teacher model is selected as the distillation source based on its size. The principle of the adaptive scale selection mechanism is as follows: Let the target area be... ,when When a high-resolution path is selected, a low-resolution path is selected otherwise. This is used to construct feature guidance signals from paths of different scales, so that small targets are supervised by high-resolution features and large targets are guided by low-resolution paths, thereby improving distillation stability and perception accuracy.

[0075] The student model also incorporates the same detection and segmentation losses as the teacher model, with the classification loss also using the FocalLoss form. The final total loss function for the student model is:

[0076] ;

[0077] in, These are the weighting coefficients.

[0078] IV. Target Detection

[0079] After training, the student model was used in the actual inference phase, with infrared images as input and unified detection results for medium-to-large and small targets as output.

[0080] The detection bounding boxes for small targets are directly obtained from the mask image output by the segmentation branch of the student model. Specifically, the mask image is first thresholded to retain high-confidence regions as candidate target regions. Then, the minimum bounding rectangle of each candidate region is calculated as the detection bounding box for the small target. The resulting small target detection bounding boxes, together with the medium and large target bounding boxes output by the detection branch, constitute the final detection result, achieving unified detection of targets at different scales.

[0081] In this specific embodiment, the teacher model is pre-trained with modal occlusion to enhance cross-modal feature recovery and association learning capabilities. The fine-tuning stage combines deformable convolution and multi-head deformable attention to enhance fine-grained feature extraction, supplemented by segmentation supervision to provide information on small targets. The student model features a lightweight design that prioritizes efficiency. It inherits teacher performance through feature and attention map distillation, and an adaptive scale selection mechanism matches optimal features to targets of different sizes, ensuring that small targets are guided by high-resolution features and large targets are assisted by low-resolution features. Ultimately, only infrared input is needed to uniformly detect medium-to-large and small targets. While maintaining a compact model, it significantly improves detection accuracy and robustness, effectively addressing the challenges of complex scenes and scale variations.

[0082] Example 2

[0083] This embodiment provides an infrared small target detection system based on cross-modal distillation, including:

[0084] The teacher model pre-training module is used to perform random modal occlusion on the registered infrared-visible training image pairs, input them into the teacher model, extract cross-modal features based on deformable convolution and attention mechanisms, and perform self-supervised pre-training by reconstructing the occluded regions.

[0085] The teacher model fine-tuning module is used to input the registered infrared-visible light image into the pre-trained teacher model, extract multi-scale feature maps and attention maps through high and low resolution branches, and jointly perform target detection and segmentation training to complete model fine-tuning;

[0086] The student model distillation training module is used to train a lightweight student model using only infrared images with a fine-tuned teacher model as a fixed teacher. It selects the output of the corresponding branch of the teacher through an adaptive scale selection mechanism to perform knowledge distillation of feature maps and attention maps.

[0087] The target detection template is used to input the infrared image to be detected into the trained student model, and fuse the target bounding boxes output by its detection branch with the target bounding boxes obtained by post-processing the segmentation branch mask to obtain a unified detection result.

[0088] Example 3

[0089] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the infrared small target detection method based on cross-modal distillation as described in Embodiment 1 above.

[0090] Example 4

[0091] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the infrared small target detection method based on cross-modal distillation as described in Embodiment 1 above.

[0092] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting small infrared targets based on cross-modal distillation, characterized in that, include: Random modal occlusion is performed on the registered infrared-visible training image pairs, which are then input into the teacher model. Cross-modal features are extracted based on deformable convolution and attention mechanisms, and self-supervised pre-training is performed by reconstructing the occluded regions. The registered infrared-visible light image is input into the pre-trained teacher model. Multi-scale feature maps and attention maps are extracted through high and low resolution branches, and target detection and segmentation training are performed together to complete model fine-tuning. Using the finely tuned teacher model as a fixed teacher, a lightweight student model is trained using only infrared images. An adaptive scale selection mechanism is used to select the output of the corresponding branch of the teacher and perform knowledge distillation of feature maps and attention maps. The method of selecting the output of the corresponding branch of the teacher through the adaptive scale selection mechanism specifically includes: selecting the high-resolution or low-resolution branch of the teacher model as the distillation source according to the target area: when the target area is less than the preset value, the high-resolution branch is selected, otherwise the low-resolution branch is selected to construct feature guidance signals from different scale paths. The infrared image to be detected is input into the trained student model, and the target bounding boxes output by its detection branch are fused with the target bounding boxes obtained by post-processing the segmentation branch mask to obtain a unified detection result.

2. The infrared small target detection method based on cross-modal distillation as described in claim 1, characterized in that, The pre-training process of the teacher model specifically includes: Obtain registered infrared-visible training image pairs, randomly occlude local region modes or a certain mode, and obtain occluded image pairs; The occluded image is input into the teacher model. The original information of the occluded region is recovered based on multimodal features through head reconstruction, and the teacher model is pre-trained.

3. The infrared small target detection method based on cross-modal distillation as described in claim 1, characterized in that, The teacher model includes a high-resolution branch and a low-resolution branch; each branch contains a deformable convolutional module and a multi-head deformable attention module; the deformable convolutional module outputs a multi-scale feature map, and the multi-head deformable attention module dynamically adjusts the sampling position through a learnable offset vector to generate an attention map; the feature maps and attention maps output by the two branches are input to the detection head and the segmentation head, and the detection result and segmentation mask are jointly output.

4. The infrared small target detection method based on cross-modal distillation as described in claim 3, characterized in that, The multi-head deformable attention module dynamically adjusts the sampling position using a learnable offset vector to generate an attention map, specifically: ； in, This represents an attention map that incorporates location information. This represents the attention weight function. For query points, K and V represent key-value pairs. The learnable offset vector, where N represents the number of key-value pairs. For the i-th key vector, Let be the i-th value vector.

5. The infrared small target detection method based on cross-modal distillation as described in claim 1, characterized in that, The process involves using a finely tuned teacher model as a fixed teacher, training a lightweight student model using only infrared images, selecting the output of the corresponding branch of the teacher model through an adaptive scale selection mechanism, and performing knowledge distillation on feature maps and attention maps. Specifically, this includes: Infrared images are input into the backbone network of a lightweight student model to extract feature maps and attention maps; The feature map and attention map output by the corresponding branch of the teacher model are matched using an adaptive scale selection mechanism. Calculate the L2 loss between the student model feature map and the corresponding feature map of the teacher model, and simultaneously calculate the KL divergence loss between the student model attention map and the corresponding attention map of the teacher model; update the student model parameters based on the L2 loss and KL divergence loss to complete knowledge distillation training.

6. An infrared small target detection system based on cross-modal distillation, characterized in that, include: The teacher model pre-training module is used to perform random modal occlusion on the registered infrared-visible training image pairs, input them into the teacher model, extract cross-modal features based on deformable convolution and attention mechanisms, and perform self-supervised pre-training by reconstructing the occluded regions. The teacher model fine-tuning module is used to input the registered infrared-visible light image into the pre-trained teacher model, extract multi-scale feature maps and attention maps through high and low resolution branches, and jointly perform target detection and segmentation training to complete model fine-tuning; The student model distillation training module is used to train a lightweight student model using only infrared images with a fine-tuned teacher model as a fixed teacher. It selects the output of the corresponding branch of the teacher through an adaptive scale selection mechanism to perform knowledge distillation of feature maps and attention maps. The method of selecting the output of the corresponding branch of the teacher through the adaptive scale selection mechanism specifically includes: selecting the high-resolution or low-resolution branch of the teacher model as the distillation source according to the target area: when the target area is less than the preset value, the high-resolution branch is selected, otherwise the low-resolution branch is selected to construct feature guidance signals from different scale paths. The target detection template is used to input the infrared image to be detected into the trained student model, and fuse the target bounding boxes output by its detection branch with the target bounding boxes obtained by post-processing the segmentation branch mask to obtain a unified detection result.

7. The infrared small target detection system based on cross-modal distillation as described in claim 6, characterized in that, The pre-training process of the teacher model specifically includes: Obtain registered infrared-visible training image pairs, randomly occlude local region modes or a certain mode, and obtain occluded image pairs; The occluded image is input into the teacher model. The original information of the occluded region is recovered based on multimodal features through head reconstruction, and the teacher model is pre-trained.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the infrared small target detection method based on cross-modal distillation as described in any one of claims 1-5.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the infrared small target detection method based on cross-modal distillation as described in any one of claims 1-5.

Images