Infrared small target detection method, system, detection device, electronic device and medium

Abstract

The application discloses an infrared small target detection method, system, detection equipment, electronic equipment and medium, belongs to the field of image processing, inputs a complete infrared image into a coarse-accurate detection infrared small target detection framework, the coarse-accurate detection infrared small target detection framework first screens a target region from the complete infrared image and then performs accurate target detection to obtain an infrared small target detection result; the complete infrared image is input into a region two-classification network to perform image block level two-classification, determine whether each image block contains a target, generate a multi-scale region feature map based on the determination result, in a lightweight target detection module, utilize a convolutional neural network structure, combine a context-guided knowledge distillation module, perform different level feature extraction and fusion on the multi-scale region feature map, generate an infrared small target region detection result based on the combination of multi-level features; and perform region mapping on the infrared small target region detection result and the complete infrared image to obtain an infrared small target detection result.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an infrared small target detection method, system, detection device, electronic device and medium. BACKGROUND

[0002] The infrared small target detection task mainly faces the problems of small target size, weak target features and complex background interference, which seriously affect the detection performance of small targets.

[0003] The existing target detection method has two types of solutions for small targets. One is to increase the resolution of the input image or an additional high-resolution detection head to improve the detection performance of small targets. However, this method significantly increases the computational complexity and GPU (Graphics Processing Unit) resource consumption, especially in high-resolution images, and most of the computing resources are wasted on non-target background areas. The other is to increase an additional high-resolution shallow detection branch to improve the small target detection performance, but the additional computational cost limits the overall detection performance.

[0004] In addition, the existing coarse detection-fine detection method of first coarsely positioning the target area and then finely detecting the target is mostly used in fixed background monitoring scenes. It uses traditional statistical algorithms to obtain the area where the target does not appear, so as to reduce the redundant calculation of the background. However, in the infrared small target detection task, both the background and the target are moving, and the target area cannot be obtained by using statistical algorithms. There is a lack of a high-precision and high-efficiency infrared small target detection framework that can realize coarse detection-fine detection in a moving background, making the efficient detection of small targets in complex environments still face great challenges. SUMMARY

[0005] The purpose of the present application is to overcome the problem that the detection accuracy and computational efficiency are difficult to balance in the existing infrared small target detection task in a moving background. The present application provides an infrared small target detection method, system, detection device, electronic device and medium, and proposes a coarse detection-fine detection target detection framework of first coarsely positioning the target area and then finely detecting the target. Through the idea of divide and conquer, the accuracy and efficiency of infrared small target detection can be balanced in a moving background.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides an infrared small target detection method, comprising the following steps:

[0008] inputting a complete infrared image into a coarse-fine detection infrared small target detection framework, and the coarse-fine detection infrared small target detection framework first screens a target area from the complete infrared image and then performs accurate target detection to obtain an infrared small target detection result;

[0009] The coarse-to-fine infrared small target detection framework comprises a region binary classification network and a lightweight target detection module.

[0010] The screening target region is to input a complete infrared image into the region binary classification network to perform image block-level binary classification, to determine whether each image block after binary classification contains a target, and to generate a multi-scale region feature map based on the determination result, specifically comprising: in the region binary classification network, after down-sampling the complete infrared image, multi-scale features are extracted through a lightweight feature extraction module; the multi-scale features are input into a denoising reinforcement discrimination module based on pseudo sample guidance to obtain multi-scale enhanced features; the multi-scale enhanced features are fused to obtain a multi-scale feature map and a target region mask; after region slicing of the multi-scale feature map based on the target region mask, a multi-scale region feature map is obtained.

[0011] The accurate target detection specifically comprises: in the lightweight target detection module, a convolutional neural network structure is used, combined with a context-guided knowledge distillation module, to perform different level feature extraction and fusion on the multi-scale region feature map to obtain a combination of multi-level features, and generate an infrared small target region detection result based on the combination of multi-level features; and the infrared small target region detection result is mapped with the complete infrared image to obtain an infrared small target detection result.

[0012] Further, the infrared small target region detection result comprises position information and confidence of the infrared small target.

[0013] Further, the down-sampling uses a single residual bottleneck module combined with pixel de-reconstruction.

[0014] Further, the denoising reinforcement discrimination module based on pseudo sample guidance comprises: introducing Gaussian noise close to the target activation value on the multi-scale feature map, and calculating a multi-scale classification loss combined with the true value.

[0015] Further, after region slicing of the multi-scale feature map based on the target region mask, a multi-scale region feature map is obtained, comprising: using a fully connected layer and a nonlinear activation function to map the target region mask output to a probability distribution graph of 0-1 to obtain the probability of each image block containing a target, and if the probability of the image block containing a target is greater than a probability threshold, the image block contains a target, and based on the determination result of whether each image block contains a target, a multi-scale region feature map is generated.

[0016] Further, the context-guided knowledge distillation module additionally introduces a teacher feature map at the same scale, and the feature map of the lightweight student detection model is adaptively adjusted pixel by pixel by querying the correlation of the teacher feature map at each pixel position and the surrounding pixel positions.

[0017] In a second aspect, the present application provides an infrared small target detection system, comprising:

[0018] The region binary classification network module is configured to: after down-sampling the complete infrared image, extract multi-scale features through the lightweight feature extraction module; input the multi-scale features into the denoising enhancement discrimination module based on the pseudo sample guidance to obtain multi-scale enhanced features; fuse the multi-scale enhanced features to obtain a multi-scale feature map and a target region mask; and after region slicing the multi-scale feature map based on the target region mask, obtain a multi-scale region feature map.

[0019] The lightweight target detection module is configured to: in the lightweight target detection module, utilize a convolutional neural network structure, combine the context-guided knowledge distillation module, extract and fuse multi-scale region feature maps at different levels to obtain a combination of multi-level features, generate an infrared small target region detection result based on the combination of multi-level features, and perform region mapping on the infrared small target region detection result and the complete infrared image to obtain an infrared small target detection result.

[0020] In a third aspect, the present application provides an infrared small target detection device, comprising an infrared camera and an image processing unit, the infrared camera being connected to the image processing unit, and the image processing unit comprising the infrared small target detection system.

[0021] In a fourth aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the infrared small target detection method is implemented when the processor executes the computer program.

[0022] In a fifth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, and the infrared small target detection method is implemented when the computer program is executed by a processor.

[0023] Compared with the prior art, the present application has the following beneficial technical effects:

[0024] The infrared small target detection method provided by the present application is oriented to a moving platform and a moving target, and through a unified coarse detection-precise detection infrared small target detection framework, efficient processing of an infrared image in a complex environment is realized, and the detection accuracy and real-time performance of infrared small target detection are significantly improved. Through a lightweight feature extraction and down-sampling structure and a denoising reinforcement discrimination module based on pseudo sample guidance, the present application can effectively extract a local region containing a target in an infrared image, greatly reducing the calculation cost of a subsequent detection model. The infrared small target features in the local region feature map are more prominent, reducing most of the background interference, providing a higher quality input for the target detection model, thereby improving the accuracy of subsequent detection. In the target detection model, the present application introduces a context-guided knowledge distillation module, which enhances target features through multi-layer feature extraction and attention guidance mechanism, effectively reduces background noise interference while increasing context information. This module can help the network more accurately identify the infrared small target features in the regional features, improving the accuracy and robustness of target positioning, and is particularly suitable for infrared target detection tasks in complex scenes. The unified framework of the present application can adaptively adjust the parameters of the regional binary classification network, and the network adjusts its model parameters through the back propagation of the gradient of the loss function to adapt to different application scenarios. At the same time, the system can autonomously adjust the size of the image region block according to the different target scales, change the size of the region sent to the subsequent target detection module by adjusting the hyperparameters of the customized region block slicing algorithm, and ensure that the complete target is sent to the subsequent target detection module. Compared with the traditional two-stage processing method, the present application reduces the amount of calculation while improving the overall processing effect.

[0025] Further, the present application adopts a lightweight network structure and a multi-level feature extraction strategy, so that the system has strong real-time processing capability while maintaining high precision, and is suitable for real-time detection of infrared targets. In addition, the system can adaptively adjust the block and detection parameters in different scenes, has strong environmental adaptability, and can be applied to infrared small target detection in night, smoke and other harsh conditions.

[0026] Further, through comparative experiments, the present application is superior to existing methods in terms of the comprehensive performance of regional binary classification and target detection. The regional binary classification network realizes higher precision and recall rate on infrared small target images in a complex background; the lightweight detection module realizes higher average precision, recall rate and F1 value in the detection task, the F1 value is the harmonic mean of precision and recall rate, and maintains a high processing frame rate, meeting the demand of real-time detection of infrared targets. BRIEF DESCRIPTION OF DRAWINGS

[0027] The drawings described herein are for purposes of illustration only and are not intended to limit the scope of the present disclosure in any way. Additionally, the shapes and proportions of the various components depicted in the drawings are not intended to be specific, but rather are for purposes of illustration only. In the drawings:

[0028] Figure 1 Flow chart of the infrared small target detection method of the present application.

[0029] Figure 2 Structure diagram of the infrared small target detection system of the present application.

[0030] Figure 3 Electronic device diagram of the infrared small target detection method of the present application.

[0031] Figure 4 Rough-fine detection infrared small target detection framework diagram of the infrared small target detection method of the present application.

[0032] Figure 5 Detection result diagram of the detection of unmanned aerial vehicle targets and ground vehicle targets using the infrared small target detection method in the embodiment of the present application. DETAILED DESCRIPTION

[0033] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0034] Embodiment one

[0035] Referring to Figure 1 , the infrared small target detection method comprises the following steps:

[0036] The complete infrared image is input into the rough-fine detection infrared small target detection framework, which first screens the target region and then performs accurate target detection on the complete infrared image to obtain the infrared small target detection result;

[0037] The rough-fine detection infrared small target detection framework comprises a region binary classification network and a lightweight target detection module.

[0038] The screening target region is to input a complete infrared image into a region binary classification network, perform image block-level binary classification, judge whether each image block after binary classification contains a target, and generate a multi-scale region feature map based on the judgment result, specifically including: in the region binary classification network, after down-sampling the complete infrared image, multi-scale features are extracted through a lightweight feature extraction module; the multi-scale features are input into a denoising enhancement discrimination module based on pseudo sample guidance to obtain multi-scale enhanced features; the multi-scale enhanced features are fused to obtain a multi-scale feature map and a target region mask; after region slicing of the multi-scale feature map based on the target region mask, a multi-scale region feature map is obtained;

[0039] The precise target detection specifically includes: in the lightweight target detection module, a convolutional neural network structure is used, a context-guided knowledge distillation module is combined, different levels of feature extraction and fusion are performed on the multi-scale region feature map, a combination of multi-level features is obtained, and an infrared small target region detection result is generated based on the combination of multi-level features; the infrared small target region detection result is mapped with the complete infrared image to obtain an infrared small target detection result.

[0040] In terms of detection efficiency, the coarse-precision detection framework is adopted, the target region is quickly screened through the region binary classification network, the comprehensive and detailed search on the complete image is avoided, the calculation amount is greatly reduced, the region where the target may exist is quickly located, the detection speed is improved, the real-time detection demand of the moving platform and the moving target is met. In terms of detection accuracy, the region binary classification network uses the denoising enhancement discrimination module based on pseudo sample guidance to enhance the features and generate an accurate multi-scale region feature map; the lightweight target detection module combines the context-guided knowledge distillation module to perform multi-level extraction and fusion on the features, and can more accurately detect small targets. In addition, the method framework structure is reasonable, the modules have clear division and collaborative work, and the lightweight design reduces the requirement for hardware resources, while ensuring efficient detection, the method has good universality and scalability, and can be applied to infrared small target detection tasks in various moving platform scenarios.

[0041] The infrared small target region detection result includes position information and confidence of the infrared small target. Down-sampling is performed using a single residual bottleneck module combined with pixel inverse reconstruction. The denoising reinforcement discrimination module based on pseudo sample guidance includes introducing Gaussian noise close to the target activation value on the multi-scale feature map and calculating a multi-scale classification loss combined with the true value. After region slicing of the multi-scale feature map based on the target region mask, a multi-scale region feature map is obtained, including mapping the target region mask output to a 0-1 probability distribution map using a fully connected layer and a nonlinear activation function to obtain the probability of containing a target in each image block, and if the probability of containing a target in the image block is greater than a probability threshold, the image block contains a target, and a multi-scale region feature map is generated according to the judgment result of whether each image block contains a target. The context-guided knowledge distillation module additionally introduces a teacher feature map at the same scale, and the feature map of the lightweight student detection model is adaptively adjusted pixel by pixel by querying the correlation of the teacher feature map at each pixel position and the surrounding pixel positions.

[0042] The down-sampling in the embodiment uses a single residual bottleneck module combined with pixel inverse reconstruction, which can effectively preserve key image information and reduce information loss, providing a more reliable basis for subsequent detection. The denoising reinforcement discrimination module based on pseudo sample guidance can enhance the robustness of the model to noise, improve feature quality, and make the detection result more accurate by introducing Gaussian noise and calculating the loss combined with the true value. The target region mask is determined and sliced using a fully connected layer, which can accurately filter out regions that may contain targets, reduce irrelevant region interference, and improve detection efficiency. The context-guided knowledge distillation module can make the student model better learn the advantages of the teacher model by adaptively adjusting the pixel-by-pixel values of its own feature map based on the correlation of the teacher model feature map, which can improve detection accuracy and reduce computational cost due to lightweight design, allowing the method to efficiently implement infrared small target detection in resource-limited situations and having high practical value.

[0043] Embodiment Two

[0044] Referring to Figure 2 , the infrared small target detection system includes:

[0045] The region binary classification network module is used to down-sample the complete infrared image, extract multi-scale features through the lightweight feature extraction module, input the multi-scale features into the denoising reinforcement discrimination module based on pseudo sample guidance to obtain multi-scale enhanced features, fuse the multi-scale enhanced features to obtain a multi-scale feature map and a target region mask, and obtain a multi-scale region feature map after region slicing of the multi-scale feature map based on the target region mask.

[0046] The light-weight target detection module is configured to perform different-level feature extraction and fusion on a multi-scale region feature map by using a convolutional neural network structure in combination with a context-guided knowledge distillation module to obtain a combination of multi-level features, and generate an infrared small target region detection result based on the combination of multi-level features; and perform region mapping on the infrared small target region detection result and a complete infrared image to obtain an infrared small target detection result.

[0047] Embodiment three

[0048] The infrared small target detection device comprises an infrared camera and an image processing unit, the infrared camera is connected to the image processing unit, and the image processing unit comprises an infrared small target detection system.

[0049] Embodiment four

[0050] Referring to Figure 3 An electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and an infrared small target detection method implemented when the processor executes the computer program.

[0051] Embodiment five

[0052] A computer-readable storage medium stores a computer program, and an infrared small target detection method when the computer program is executed by a processor.

[0053] Embodiment six

[0054] Referring to Figure 4 The embodiment provides an infrared small target detection method, and the coarse detection and fine detection of the infrared small target are realized through a unified framework based on the coarse detection first and then the fine detection strategy.

[0055] The method mainly comprises a region binary classification network and a light-weight target detection module, and is respectively used for solving the problems of weak features, small scales and large redundant calculation amounts of the infrared small target and the accurate detection of the infrared small target.

[0056] The overall framework comprises an upper region binary classification network and a lower light-weight target detection module. The system input is a complete infrared image, and the region binary classification network generates a multi-scale feature map and a region mask after the input. The multi-scale feature map is sent to the light-weight target detection module after being strictly sliced in the region, the infrared small target in the image is detected, and a detection result is output.

[0057] The region binary classification network is responsible for image block-level binary classification of the complete image, and is used for judging whether each image block contains a target. The specific steps are as follows:

[0058] Image input: the complete infrared image is input into the system as the input of the region binary classification network.

[0059] Feature extraction and down-sampling operation: a single residual bottleneck module is used in combination with pixel de-recomposition to achieve down-sampling of the image. Through convolution and cross-stage connection, the residual bottleneck module extracts multi-scale features, enhances the semantic expression of small targets. Pixel de-recomposition rearranges the spatial information to high-dimensional channels, preserving small target details while improving the network's global modeling ability and reducing background noise interference. The multi-scale of multi-scale features means that the scale of the feature map is different after down-sampling by pixel de-recomposition, so during network propagation, the scale of the features is diverse, while the input scale of a residual bottleneck module and cross-stage connection and pixel de-recomposition is consistent, and the output of pixel de-recomposition is the feature map after down-sampling.

[0060] Denoising reinforcement discrimination module based on pseudo sample guidance: by adding several Gaussian noises similar to the target activation value on the multi-scale feature map, the network directly outputs the classification result at this scale, and combines the true value to calculate the multi-scale classification loss, improving the network's ability to distinguish moving targets and changing backgrounds, and ultimately enhancing the network's classification performance.

[0061] When the deep learning network finally identifies the image block, it is essentially predicting the probability of the image block containing the target, and the probability is represented as a value between 0 and 1 on the feature map. The higher the value, the greater the probability that the image block contains the target, that is, the higher the activation value of the model for the image block; noise is added by analyzing the activation value distribution of the target true value in the feature map, and several noises with similar spatial value distribution to the target activation value are artificially added to simulate background interference noise close to the real target, thereby improving the model's ability to distinguish targets and background noise.

[0062] Since the denoising reinforcement discrimination module based on pseudo sample guidance is added in different stages of the network, that is, the input feature map of the denoising reinforcement discrimination module based on pseudo sample guidance is different in scale. Since the deep and shallow feature maps contain different information, the emphasis is on the classification result at this scale. The denoising reinforcement discrimination module based on pseudo sample guidance assists in training the network model through multi-scale denoising classification loss.

[0063] Classification result output: based on multi-scale feature map fusion, the output is mapped to a probability distribution graph of 0-1 using a fully connected layer and a nonlinear activation function to distinguish the probability of containing the target in each image block. Greater than the probability threshold is considered to contain the target. The output refers to the classification result output by the first stage network. For example, if the entire image is divided into 20x20 image blocks, the output result is a 20x20 binary mask of 0 and 1. The mask is 1, indicating that the image block contains the target, and vice versa. In addition, the output of the region binary classification network also includes multi-scale feature maps as the input of the subsequent lightweight target detection module.

[0064] The core of the region binary classification network is its lightweight feature extraction module and the denoising reinforcement discrimination module based on pseudo sample guidance. The multi-scale denoising auxiliary loss compensates for the weak learning ability of the lightweight feature extraction module, while improving the discrimination ability of the network to the target and the background, effectively reducing the computational complexity and improving the classification performance and efficiency of the network.

[0065] Based on the multi-scale region feature map output by the region binary classification network, the lightweight target detection module only needs to analyze the image through a small amount of convolutional neural network structure to locate and identify the infrared small target. The specific steps are as follows:

[0066] Feature extraction: use the multi-scale region feature map output by the region binary classification network to extract features of different levels through a small number of convolution units and perform feature fusion. Each layer of convolution kernel focuses on extracting target features of different scales, especially the detailed features of infrared small targets.

[0067] The target detection module includes a neck network and a detection head. The neck network is used to fuse the multi-scale feature maps output by the region binary classification. Because the feature maps of different stages have different semantics and resolutions, the feature representation ability of the model is enhanced. After fusing features of different levels, the feature map obtained is a feature map with more information after fusion. The fused feature map is sent to the detection head for fine target detection.

[0068] Different scales are due to the downsampling operation in the network. Small target features are easily lost after downsampling, so small target features are mostly present in shallow networks. Due to the ability of convolution to extract local information, information of medium and large scale targets is mostly present in medium and deep feature maps, so extracting feature maps of different levels is intended to extract target features of different scales.

[0069] Context-guided knowledge distillation module: Since the input of the lightweight target detection module is the region feature map output by the region binary classification network, a large amount of context information will be missing, while the teacher feature map introduced additionally in the same scale has rich context information. By using the context-guided knowledge distillation module, the feature map of the designed lightweight student detection model can be adaptively adjusted by querying the degree of correlation of the teacher feature map at each pixel position and its surrounding pixel positions, to enhance the target while suppressing the background, and to improve the detection performance of the network.

[0070] The same scale means that the scale of the feature map needs to be the same, that is, the student feature map is 4 times down-sampling, and then the teacher feature map also needs to be 4 times down-sampling. The context-guided knowledge distillation module is a feature distillation module, which is used to calculate the loss of the intermediate feature map of the teacher model and the intermediate feature map of the student model, so as to optimize the student model to be more similar to the teacher model with better performance. The loss calculation of the feature map needs to ensure the alignment of the features first, so the intermediate feature map of the teacher and the intermediate feature map of the student need to have the same scale.

[0071] Adaptive adjustment: The teacher feature map and the student feature map can obtain a group of weights between 0 and 1 through the context-guided knowledge distillation module, multiply the weights with the student feature map element by element to achieve the purpose of adaptively adjusting the student feature map, and finally adjust the pixel-by-pixel value of the student feature map.

[0072] Target output: Based on the combination of multi-level features, the target detection result is generated in the output layer, including the position information and confidence of the infrared small target.

[0073] The lightweight target detection module introduces the context-guided distillation module for training, so that the network can effectively distinguish the infrared small target from the background without increasing the additional calculation cost, thereby improving the accuracy and robustness of the infrared small target detection.

[0074] The method of the embodiment can be used for all-weather monitoring of infrared targets, especially for infrared target detection in night or bad weather conditions, and can accurately detect infrared small targets in complex backgrounds. The system can be used in military, security and other scenes with high requirements for target detection accuracy and real-time performance.

[0075] The coarse detection-fine detection target detection method of the divide-and-conquer strategy of the embodiment simplifies the infrared small target detection task into a binary classification problem first, extracts the region containing the target, and then performs a small target detection task on the target region, converts the task into a medium-large scale target detection task, and makes the target detection process more lightweight and efficient. Compared with the existing method which relies on enlarging the resolution of the input image and increasing additional detection branches, the embodiment saves a lot of computing cost, reduces the interference of complex background on detection, makes the detection process of the model more close to the biomimetic principle of top-down attention mechanism in the human visual system, and significantly improves the detection performance and practicability of the method.

[0076] The context information guided knowledge distillation module proposed in the embodiment is an attention guided knowledge distillation, which takes the feature map of the lightweight object detection model as the query, and the additional teacher model feature map as the key and value at the same time, realizes the selective attention of the student model feature to the teacher model feature, captures the discriminative features of the teacher model in the key region, and thus improves the representation ability and target perception ability of the student model.

[0077] The pseudo sample guided denoising reinforcement discrimination module based on the embodiment simulates the false target that may occur in the actual scene by adding noise with a similar response value to the true value target in the multi-scale feature map when training the region binary classification network, so as to improve the discrimination ability of the network for moving targets and changing backgrounds, and improve the classification performance of the network.

[0078] The embodiment adopts a single residual bottleneck module combined with pixel de-reorganization down-sampling as the main component of the network, but other types of modules can also be used to achieve similar feature extraction effects. For example, depth separable convolution can be used as a feature extraction module, and convolution with a step size of 2 can be used for down-sampling. These models can also achieve similar effects when performing region binary classification, thereby replacing the current model.

[0079] In addition to the Gaussian noise used in the current pseudo sample guided denoising reinforcement discrimination module to simulate the true value response value for denoising auxiliary training, other ways of generating pseudo sample noise can be used, such as training a teacher model, extracting the region with high attention of the teacher model outside the target as noise, or using other types of noise instead of Gaussian noise, etc. Similar alternative solutions are all based on pseudo sample denoising training, which further protects the idea of the pseudo sample guided denoising reinforcement discrimination method.

[0080] The target detection module currently used in this embodiment can be replaced by other known detection network architectures, such as the YOLO (You Only Look Once) series, Faster R-CNN (Faster Region-based Convolutional Neural Networks), and various detection methods. By replacing different detection models, the system can select the most suitable detection model according to different scene requirements. These networks can improve adaptability in different scenarios while maintaining detection accuracy.

[0081] In addition to traditional detection networks, infrared images can be fused with other sensor data (such as visible light images) for multi-modal processing to enhance the detection effect of infrared targets. The fused detection network can make decisions based on the features of different modalities, further improving the robustness of detection. This scheme can provide supplementation and alternatives when single-modal detection is not effective.

[0082] The knowledge distillation module can be replaced according to different task scenarios. For example, attention distillation or spatial attention distillation can provide a lighter guidance method. This strategy has low computational overhead and is suitable for resource-constrained deployment scenarios, while still maintaining good distillation effect.

[0083] Although this embodiment uses a two-stage target detection framework of coarse detection and fine detection, an end-to-end approach can be used instead, with an additional branch outputting a binary classification result. For example, in complete target detection, an additional layer is added to output the region binary classification result of the network. This can achieve an end-to-end training method and is suitable for scenarios with lower real-time requirements.

[0084] In existing detection methods based on deep learning models, the solution for small targets is mostly to increase the resolution of the input image or add an additional shallow detection branch to improve detection accuracy. However, this approach significantly increases computational complexity and GPU resource consumption, making it difficult to achieve real-time detection on resource-constrained edge devices. This embodiment uses the divide-and-conquer idea, first uses a lightweight region binary classification network to extract the region containing the target in the image, and then inputs it into the subsequent target detection network. Since the region containing the target is much smaller than the entire image, the subsequent detection network can also be implemented in a lightweight manner, significantly reducing the computational load. This converts the small target detection task into a medium or large scale target detection task, simplifying the task while improving detection efficiency.

[0085] The existing method does not consider the high similarity between the infrared small target and the background, and the network is difficult to distinguish between the two in the training process, which is easy to produce false alarm and seriously affects the detection performance. In the network training, the embodiment adds a denoising reinforcement discrimination module based on pseudo sample guidance, simulates the real target by adding noise, and lets the network learn in the false target and the real target, so as to improve the discrimination ability of the network to the target and the background. At the same time, the module is only used in training, which can improve the detection performance of the model while not affecting the detection efficiency of the network.

[0086] In the existing detection architecture using coarse detection-fine detection, the use scene is a background fixed monitoring system, the traditional statistical method is used to extract the target area in the whole image and then sent to the subsequent detection model, and the task scene is mostly medium-large scale target, which is not applicable to the scene of moving background and moving target. However, for the infrared small target detection task, if the target area cannot be correctly extracted in the coarse detection stage, the subsequent detection model cannot detect the target. Therefore, the embodiment customizes a lightweight region binary classification network to improve the coarse detection accuracy of the moving small scale target and reduce the calculation cost, which provides convenience for the subsequent detection model.

[0087] Through the detection framework of this embodiment, the detection performance of the infrared small target is improved, and the detection efficiency is also improved, so that a more reliable infrared monitoring scheme is provided in the complex actual application scene.

[0088] The embodiment also provides an infrared small target detection system, which can be integrated into the existing infrared small target detection equipment, mainly including an infrared camera and an image processing unit. The region binary classification network and the lightweight target detection module can be realized on a high-performance embedded processor, supporting real-time processing. When processing the infrared image, the system first screens the target area (coarse detection), and then performs accurate target detection (fine detection), to ensure the accuracy of the infrared small target detection.

[0089] The infrared small target detection method of the embodiment is further described below in combination with experiments and experimental results:

[0090] The model is trained and tested on this dataset, and compared with existing target detectors based on Transformer architecture, such as DQ-DETR (Dual Query Detection Transformer), D-Fine (Redefine Regression Task in DETRs as Fine-grained Distribution Refinement), DINO (Distillation with NO labels), Hyper-YOLO (a target detection model based on hypergraph computation), YOLO11 (You Only Look Once version 11), YOLO12 (You Only Look Once version 12), MSHNet (a small infrared target detection network with scale and position sensitivity), YOLOv8-PConv (YOLOv8 with Pyramid Convolution), and ESOD (Enhanced Source-Free Object Detection) and QueryDet (Query-Based Object Detection), which are also two-stage target detection methods. The experimental results are shown in Table 1, where the best performance indicators are bolded and the second best indicators are underlined.

[0091] The model is trained and tested on this dataset, and compared with existing target detectors based on Transformer architecture, such as DQ-DETR (Dual Query Detection Transformer), D-Fine (Redefine Regression Task in DETRs as Fine-grained Distribution Refinement), DINO (Distillation with NO labels), Hyper-YOLO (a target detection model based on hypergraph computation), YOLO11 (You Only Look Once version 11), YOLO12 (You Only Look Once version 12), MSHNet (a small infrared target detection network with scale and position sensitivity), YOLOv8-PConv (YOLOv8 with Pyramid Convolution), and ESOD (Enhanced Source-Free Object Detection) and QueryDet (Query-Based Object Detection), which are also two-stage target detection methods. The experimental results are shown in Table 1, where the best performance indicators are bolded and the second best indicators are underlined.

[0092] Table 1 Comparative experimental results of infrared unmanned aerial vehicle small target detection using the method and other methods

[0093]

[0094] As can be seen from Table 1, the comprehensive performance of the method in area two classification and target detection is better than that of the prior art. The area two classification network achieves higher precision and recall rate on the infrared small target image in a complex background; the lightweight detection module achieves higher average precision, recall rate and F1 value in the detection task, and maintains a high processing frame rate, meeting the needs of real-time detection of infrared targets.

[0095] AP50 in Table 1 represents the average precision when the intersection over union threshold is 0.5, and GFLOPS represents the number of floating point operations per second. LESP (Linear Expenditure System-based Planning System), EFLNet (Enhanced Feature Learning Network), UIU-Net (U-Net in U-Net), ISNet (Image Structure Network), and DANA Net (Dense Nested Attention Network).

[0096] The detection results of the infrared small target detection method of the method of the present application on unmanned aerial vehicle targets and ground vehicle targets are shown in Figure 5 , where car represents a ground vehicle target, UAV represents an unmanned aerial vehicle target, and the numbers represent the confidence of the detection frame.

[0097] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer usable program code.

[0098] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0099] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0100] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0101] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limiting the same. Although the present application is described in detail with reference to the above embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or equivalent replacements without departing from the spirit and scope of the present application, and any modifications or equivalent replacements made to the specific embodiments of the present application without departing from the spirit and scope of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting small infrared targets, characterized in that, The method comprises the following steps: inputting a complete infrared image into a coarse-fine detection infrared small target detection framework, and performing target region screening and accurate target detection on the complete infrared image by the coarse-fine detection infrared small target detection framework to obtain an infrared small target detection result; the coarse-fine detection infrared small target detection framework comprises a region binary classification network and a lightweight target detection module; coarse detection of the coarse-fine detection infrared small target detection framework utilizes the region binary classification network to screen a target region to obtain a multi-scale region feature map, and accurate target detection of the coarse-fine detection infrared small target detection framework utilizes the lightweight target detection module to perform accurate target detection to obtain the infrared small target detection result; the target region screening comprises inputting the complete infrared image into the region binary classification network, performing image block-level binary classification, judging whether each image block after the binary classification contains a target, and generating a multi-scale region feature map based on the judgment result, specifically comprising: in the region binary classification network, after down-sampling the complete infrared image, multi-scale features are extracted through a lightweight feature extraction module; the multi-scale features are input into a denoising reinforcement discrimination module based on pseudo-sample guidance to obtain multi-scale enhanced features; the multi-scale enhanced features are fused to obtain a multi-scale feature map and a target region mask; after region slicing of the multi-scale feature map based on the target region mask, a multi-scale region feature map is obtained; the accurate target detection specifically comprises: in the lightweight target detection module, multi-scale region feature maps are subjected to feature extraction and fusion at different levels by using a convolutional neural network structure combined with a context-guided knowledge distillation module to obtain a combination of multi-level features, and the combination of multi-level features is used to generate an infrared small target region detection result; and the infrared small target region detection result is subjected to region mapping with the complete infrared image to obtain the infrared small target detection result.

2. The method of claim 1, wherein, The infrared small target region detection result comprises position information and confidence of the infrared small target.

3. The method of claim 1, wherein, The down-sampling uses a single residual bottleneck module combined with pixel inverse reconstruction.

4. The method of claim 1, wherein, The denoising reinforcement discrimination module based on pseudo-sample guidance comprises: introducing Gaussian noise close to target activation values on the multi-scale feature map, and calculating a multi-scale classification loss in combination with a true value.

5. The method of claim 1, wherein, After the region slicing of the multi-scale feature map based on the target region mask, the multi-scale region feature map is obtained by using a full connection layer and a nonlinear activation function to map the target region mask output to a probability distribution graph of 0-1, obtaining the probability of each image block containing a target, and if the probability of the image block containing a target is greater than a probability threshold, the image block contains a target, and a multi-scale region feature map is generated according to the judgment result of whether each image block contains a target.

6. The method of claim 1, wherein, The context-guided knowledge distillation module additionally introduces a teacher feature map under the same scale, and the feature map of the lightweight student detection model is adaptively adjusted pixel by pixel by querying the correlation of the teacher feature map at each pixel position and the surrounding pixel positions.

7. An infrared small target detection system using the infrared small target detection method as claimed in any one of claims 1 to 6, characterized by The method comprises the following steps: a region binary classification network module is used to extract multi-scale features through a lightweight feature extraction module after down-sampling a complete infrared image; the multi-scale features are input into a denoising reinforcement discrimination module based on pseudo-sample guidance to obtain multi-scale enhanced features; The multi-scale enhanced features are fused to obtain a multi-scale feature map and a target region mask; After the multi-scale feature map is regionally sliced based on the target region mask, a multi-scale region feature map is obtained; The lightweight target detection module is configured to, in the lightweight target detection module, utilize a convolutional neural network structure, combine a context-guided knowledge distillation module, perform feature extraction and fusion of different levels on the multi-scale region feature map, obtain a combination of multi-level features, generate an infrared small target region detection result based on the combination of multi-level features, and perform regional mapping of the infrared small target region detection result and a complete infrared image to obtain an infrared small target detection result.

8. An infrared small target detection device, characterized in that, The infrared small target detection system includes an infrared camera and an image processing unit, the infrared camera is connected to the image processing unit, and the image processing unit includes the infrared small target detection system as claimed in claim 7.

9. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the infrared small target detection method as claimed in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the infrared small target detection method as claimed in any one of claims 1-6.

Images