An infrared small target detection method based on edge-guided perception

Abstract

The present application relates to an infrared small target detection method based on edge guided perception, which is composed of two branches, namely edge extraction branch and target detection branch. In the encoding stage, a multi-scale feature progressive fusion encoder is adopted to gradually integrate features of different scales and enhance the model's feature representation ability for small targets, so as to better capture details and context information. In the decoding stage, information interaction between edge features and target features is realized through a feature aggregation and separation module. In order to accurately model the target shape, an edge guided image refinement module is designed to refine the shape of the target through local area magnification processing. In addition, a local target magnifier is designed to highlight the visibility of the target in the image and suppress background information. Using the method of the present application can effectively detect small targets in the image and accurately capture their shape details, improving the precision and robustness of infrared small target detection.

Description

Technical Field

[0001] This invention belongs to the field of computer vision target detection, and relates to an infrared small target detection method based on edge-guided perception. Background Technology

[0002] Infrared imagery possesses unique characteristics that make it less susceptible to environmental interference, such as low light and strong sunlight. Even under complex conditions, it can produce relatively clear and visible content. Infrared small target detection is a typical infrared image-based task, aiming to detect small targets within infrared images. This technology has the ability to quickly analyze potential threats or targets, providing timely information and making accurate judgments. This advantage provides substantial support for tasks such as nighttime navigation and fire rescue.

[0003] Unlike general target detection, infrared small target detection faces several challenges: 1) Small size and low contrast: Infrared small targets are typically small in size, resulting in limited information about the target contained in the image. Simultaneously, their contrast is low compared to the thermal radiation of the surrounding background. 2) Complex background interference: Infrared images are usually acquired from a distance, which easily leads to interference from complex backgrounds. Complex backgrounds make infrared small target detection more difficult. 3) Diversity and variability: Infrared small targets exhibit diversity and variability in appearance and shape. This means that targets may have different appearance features and shape variations, requiring algorithms to have strong generalization capabilities to adapt to various situations. These factors increase the possibility of false positives and false negatives in infrared small target detection. Therefore, algorithms for infrared small target detection need to consider these challenges and possess strong generalization capabilities to improve detection accuracy and robustness.

[0004] Thanks to the powerful feature representation capabilities of convolutional neural networks, deep learning-based methods have demonstrated superior performance in adapting to different targets and scenarios. However, existing methods still face some limitations in addressing these challenges. First, small infrared targets are typically much smaller, sometimes occupying only a few pixels in an infrared image. Therefore, many target detection methods suffer significant information loss after multiple downsampling operations, impacting detection accuracy. Second, these methods often use small image patches as input during model training, which may result in the training set not containing the target to be detected. Consequently, the model lacks sufficient target information to accurately learn relevant features. Finally, existing methods often produce smooth target shapes during detection, lacking the ability to model the shape characteristics of small targets. Therefore, further research is needed to detect small targets and accurately capture their shape details. Summary of the Invention

[0005] Technical problems to be solved

[0006] To overcome the shortcomings of existing technologies, this invention proposes an infrared small target detection method based on edge-guided perception, capable of detecting small targets and accurately capturing their shape details. The main idea of ​​this method is to enhance semantic information and contextual association through a multi-scale feature-based progressive fusion encoder. Considering that the detected target shape may produce a smoothing effect, an edge-guided image thinning module is introduced to improve the integrity of the target shape.

[0007] Technical solution

[0008] An infrared small target detection method based on edge-guided sensing, characterized by the following steps:

[0009] Step 1: Use the infrared image dataset as both the training and testing sets;

[0010] Step 2: Extract branching model using edges Obtaining edge information of the target: Extracting image I using a multi-scale feature progressive fusion encoder input Features of small targets; cascaded residual units are used to decode the features obtained from the encoder, and these features are upsampled to the input image I using Bilinear interpolation. input The size of the target is determined, and then a Sigmoid layer is used to limit the output range to between 0 and 1 to obtain the edge information of the target. The process is defined as follows:

[0011]

[0012] Where, θ α This represents the parameter set for edge extraction branches;

[0013] Step 3: Apply the target detection branch to the input image I input Preprocessing is performed to obtain a magnified image of the local target, I. LTA Then, the Sobel edge detection operator is used to process the input image I. LTA Image I is obtained by edge extraction. sobel ;

[0014] Step 4: Using the object detection branch model Obtain the predicted image I pred The constructed multi-scale feature progressive fusion encoder is used for image I. input With two processed images I sobel with I LTA Feature extraction is performed; then, cascaded residual units are used to decode the features obtained from the encoder, and these features are upsampled to the input image I using Bilinear interpolation. inputThe size of the image, under the action of cascaded residual units, is used to obtain the predicted image I through convolution operations. pred ,Right now:

[0015]

[0016] Where, θ β The parameter set representing the target detection branch;

[0017] The edge extraction branch model With the target detection branch model Between them, a separation interaction module is set up in the decoding stage to cascade the intermediate features of the target detection branch and the edge extraction branch. The cascaded features are processed by global average pooling (GAP) and multilayer perceptron (MLP) to obtain the interaction attention vector, which enables the model to suppress noise features in the depth flow.

[0018] The edge extraction branch model With the target detection branch model Between these elements, an edge-guided image thinning module is included, employing an edge extraction branch model. The obtained I edge， The region of interest is located by locating the intermediate feature F of the object detection branch to obtain the region feature F′, so that the model can focus on a specific region in the image;

[0019] Step 5: Train the designed network model using the training set of the infrared image dataset from Steps 2-4; during training, the AdaGrad optimizer is used to train the network parameters by minimizing the loss function;

[0020] The minimized loss function Defined as:

[0021]

[0022]

[0023]

[0024] Where I′ represents I input The corresponding label image, I′ edge This represents the image obtained by processing image I′ using the Sobel edge extraction operator;

[0025] Step 6: The trained network model is used to input images from the test set into the trained model to obtain the detected images I. pred .

[0026] The multi-scale feature progressive fusion encoder consists of four stacked feature extraction modules.

[0027] The feature extraction module is: Let Q i The features output by the (i-1)th feature extraction module, where i = {2, 3, 4}, are the features Q. i Its downsampled features P i The input is combined into the current feature extraction module i; semantic information and contextual relevance are enhanced through content aggregation across multiple scales, as shown in the formula:

[0028]

[0029] Among them, f i This represents a cascaded residual unit, where U(·) and D(·) represent upsampling and downsampling operations, respectively.

[0030] The linked residual unit consists of 3 residual blocks, each of which consists of 2 convolutional layers with 3×3 convolutional kernels and 1 ReLU activation function layer.

[0031] The input image I input Preprocessing is performed using a local target amplifier, and the process is defined as follows:

[0032] N j =Unfold(I input )

[0033] I LTA =Fold(Max(N) j ))

[0034] Among them, blocks By analyzing the input image I input The image is obtained by slicing with a step size of 1. Unfold(·) and Fold(·) are used to change the size of the image patch.

[0035] The image extracted by the edge-guided image thinning module is I. edge Yes: Let F be the intermediate feature of the object detection branch, and I be the image extracted by the edge extraction branch. edge By setting I edge >k Remove some noise from the image and find all connected regions in the image, obtaining the coordinates {X,Y} of the region locations. Calculate the average confidence score within each connected region, sort all confidence scores in descending order, and select the connected regions corresponding to the top three highest confidence scores as candidate regions. Based on this, determine the minimum bounding rectangle of the candidate regions, map it onto the intermediate feature F of the object detection branch, and crop the features at the corresponding positions to obtain the region of interest feature F′.

[0036] The I edge >k uses I edge>0.5.

[0037] An application of the edge-guided sensing-based infrared small target detection method, characterized in that: it is used for edge-guided sensing-based infrared small target detection, and the detection effect is:

[0038]

[0039] An electronic device, characterized in that it includes a processor and a memory, wherein the processor is configured to implement the steps of the data migration method when executing a computer program stored in the memory.

[0040] A readable storage medium, characterized in that a computer program is stored on the readable storage medium, and the computer program, when executed by a processor, implements the steps of the data migration method.

[0041] Beneficial effects

[0042] This invention proposes an infrared small target detection method based on edge-guided perception, consisting of two branches: an edge extraction branch and a target detection branch. In the encoding stage, a multi-scale feature progressive fusion encoder is employed to gradually integrate features from different scales, enhancing the model's ability to represent small targets and thus better capturing details and contextual information. In the decoding stage, information interaction between edge features and target features is achieved through feature aggregation and separation modules. To accurately model the target shape, an edge-guided image refinement module is designed, which refines the target's shape by magnifying local regions. Furthermore, a local target amplifier is designed to highlight the target's visibility in the image and suppress background information. Using this method, small targets in images can be effectively detected and their shape details accurately captured, improving the accuracy and robustness of infrared small target detection.

[0043] This invention detects small targets in infrared images using edge-guided perception. Firstly, the proposed model can independently reconstruct soft edges in the image, providing a priori information on soft edges for high-quality infrared target detection. Simultaneously, the model utilizes an edge-guided strategy to improve the accuracy and reliability of small target detection. Furthermore, a multi-scale feature progressive fusion encoder is proposed, progressively utilizing multi-scale content to enhance semantic information and contextual relevance. A local target amplifier is designed to highlight small targets, improving their visibility in the image while suppressing background information, enabling the model to better locate the target. Attached Figure Description

[0044] To more clearly illustrate the technical solutions implemented in this invention, the various modules required in the embodiments will be briefly described below. Obviously, the accompanying drawings described below are merely flowcharts of this invention. Those skilled in the art can, without any creative effort, extend the scope of these drawings and obtain other accompanying drawings.

[0045] Figure 1 This is a flowchart illustrating the specific implementation of the present invention.

[0046] Figure 2 This is an overall structural diagram of the present invention.

[0047] Figure 3 Specific process for edge-guided image thinning module Detailed Implementation

[0048] The present invention will now be further described in conjunction with the embodiments and accompanying drawings:

[0049] This invention proposes an infrared small target detection method based on edge-guided sensing, which mainly includes the following steps:

[0050] 1. Prepare image data. Download publicly available infrared image datasets and prepare the training and test sets required for the algorithm's training and evaluation.

[0051] 2. Design an edge extraction branch model. Construct a multi-scale feature progressive fusion encoder to extract features of small targets in the image, and simultaneously use a cascaded residual module for decoding to obtain the edge information of the targets.

[0052] 3. Design the target detection branch model. Similarly, construct a multi-scale feature progressive fusion encoder to extract features of small targets in the image, and then decode the feature map through a cascaded residual module to generate the target detection image result.

[0053] 4. Construct a branch interaction module. Edge features and target features are aggregated, and an attention mechanism module is used to suppress noise features in the depth flow. Simultaneously, an edge-guided image thinning module is designed to use the target's edge information to guide the target detection branch to focus on the region of interest.

[0054] 5. Training and Testing the Model. Input the prepared training set into the model and use a loss function to supervise the training of both branches of the model. After training, input infrared images from the test set into the model to obtain the object detection results.

[0055] Reference Figure 1 and Figure 2 The infrared small target detection method of the present invention is implemented in the following steps:

[0056] Step 1: Prepare infrared image data. Download publicly available infrared image datasets and divide them into training and testing sets for algorithm training and evaluation.

[0057] Step 2: Construct a multi-scale feature progressive fusion encoder. This encoder consists of four stacked feature extraction modules. Taking one of these modules as an example, let Q... i The features output by the (i-1)th feature extraction module, where i = {2, 3, 4}, are the features Q. i Its downsampled features P i The combined input is fed into the current feature extraction module i. Through content aggregation across multiple scales, semantic information and contextual relevance are enhanced.

[0058]

[0059] Among them, f i This represents a cascaded residual unit, which consists of three residual blocks. Each residual block comprises two convolutional layers with 3×3 kernels and one ReLU activation function layer. U(·) and D(·) represent the upsampling and downsampling operations, respectively. Note that all sampling operations in this invention use Bilinear interpolation.

[0060] Step 3, Construct the edge extraction branch model Input the infrared image to be trained I input Based on the constructed multi-scale feature progressive fusion encoder, the input image I is processed. input Feature extraction is performed. The features obtained from the encoder are decoded using cascaded residual units, and these features are upsampled to the input image I using Bilinear interpolation. input The size of the target is determined, and then a Sigmoid layer is used to limit the output range to between 0 and 1, thereby obtaining the edge information I of the target. edge The process is defined as follows:

[0061]

[0062] Where, θ α This represents the parameter set for edge extraction branches.

[0063] Step 4, in the object detection branch, for the input image I input Preprocessing is performed. This is to highlight the input infrared image I. input For small targets in an image, a local target amplifier is designed to improve the target's visibility in the image, resulting in image I. LTA The process is defined as follows:

[0064] N j =Unfold(I input )

[0065] ILTA =Fold(Max(N) j ))

[0066] Among them, blocks By analyzing the input image I input The image is obtained by slicing with a step size of 1. `Unfold(·)` and `Fold(·)` are used to change the size of the image patches. Simultaneously, the Sobel edge detection operator is applied to the input image I. LTA Image I is obtained by edge extraction. sobel .

[0067] Step 5, construct the object detection branch model Based on the constructed multi-scale feature progressive fusion encoder, image I input With two processed images I sobel with I LTA Feature extraction is performed. Then, cascaded residual units are used to decode the features obtained from the encoder, and these features are upsampled to the input image I using Bilinear interpolation. input The size of the image, under the action of cascaded residual units, is used to obtain the predicted image I through convolution operations. pred ,Right now

[0068]

[0069] Where, θ β This represents the parameter set of the object detection branch.

[0070] Step 6: Construct the separation and interaction module. During the decoding stage, the intermediate features from the object detection branch and the edge extraction branch are concatenated. Global average pooling (GAP) and a multilayer perceptron (MLP) are used to process the concatenated features, resulting in an interactive attention vector. This allows the model to suppress noisy features in the depth stream. On each branch, the attention vector is multiplied sequentially by each channel of the corresponding branch to obtain a depth representation with less noise.

[0071] Step 7, design the edge-guided image thinning module. For example... Figure 3 As shown, let F be the intermediate feature of the target detection branch, and I be the image extracted by the edge extraction branch. edge By setting I edge >0.5 Remove some noise from the image and find all connected regions in the image, obtaining the coordinates {X,Y} of the region locations. Calculate the average confidence score within each connected region, sort all confidence scores in descending order, and select the connected regions corresponding to the top three highest confidence scores as candidate regions. Based on this, determine the minimum bounding rectangle of the candidate regions, map it onto the intermediate feature F of the object detection branch, and crop the features at the corresponding positions to obtain the region of interest feature F′.

[0072] Step 8: Train the designed network model. Training uses the AdaGrad optimizer, training the network parameters by minimizing the loss function. This loss function... Defined as

[0073]

[0074]

[0075]

[0076] Where I′ represents I input The corresponding label image, I e ′ dge This represents the image obtained by processing image I′ using the Sobel edge extraction operator.

[0077] Step 9: Input the images from the test set into the trained model to obtain the detected images I. pred .

[0078] The effects of this invention can be further illustrated by the following simulation experiments.

[0079] 1. Simulation conditions

[0080] This invention is based on a central processing unit (CPU) The simulation was performed using Python software and the PyTorch deep learning framework on an i7-6800K 3.40GHz CPU, NVIDIA GeForce GTX3090 GPU, and Ubuntu operating system. The simulation used the publicly available infrared small target datasets ISIRST Aug and MDFA. The ISIRST Aug dataset contains 9070 images, each 256×256 pixels, with 8525 images in the training set and 545 images in the test set. The MDFA dataset contains 10078 images, with 9978 images in the training set, each 128×128 pixels, and 100 images in the test set of varying sizes.

[0081] 2. Simulation Content

[0082] To demonstrate the effectiveness of the edge-guided infrared small target detection method of this invention, the Intersection over Union (IoU), Normalized Intersection over Union (nIoU), and Area under Curve (AUC) were used as evaluation criteria. The method was trained and tested on two public datasets, ISIRST Aug and MDFA. Shape-based infrared small target detection method (ISNet), infrared small target detection method based on attention-guided pyramid context network (AGPCNet), and infrared small target detection method based on densely nested attention (DNA-Net) were selected as comparison algorithms. The ISNet algorithm is described in detail in the paper “M. Zhang, R. Zhang, Y. Yang, H. Bai, J. Zhang, and J. Guo. ISNet: Shape Matters for Infrared Small Target Detection. Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2022, 877-886.”; the AGPCNet algorithm is proposed in “T. Zhang, L. Li, S. Cao, T. Pu, and Z. Peng. Attention-Guided Pyramid Context Networks for Detecting Infrared Small Target Under Complex Background. IEEE Transactions on Aerospace and Electronic Systems, 2023, 59(4): 4250-4261.”; the DNA-Net algorithm is proposed in “B. Li, C. Xiao, L. Wang, Y. Wang, Z. Lin, M. Li, W. An, and Y. Guo. Dense nested attention network for infrared small target detection. IEEE Transactions on Image The comparison results are shown in Table 1. (This was proposed in *Processing, 2023, 32: 1745-1758*.)

[0083] Table 1 shows the comparison results of multiple algorithms.

[0084]

[0085] Generally, higher values ​​for IoU, nIoU, and AUC indicate better detection results. As shown in Table 1, on both datasets, the method proposed in this invention significantly outperforms other algorithms in these three metrics.

Claims

1. A method for detecting small infrared targets based on edge-guided sensing, characterized in that... The steps are as follows: Step 1: Use the infrared image dataset as both the training and testing sets; Step 2: Extract branching model using edges Obtaining edge information of the target: Extracting images using a multi-scale feature progressive fusion encoder. Features of small targets; cascaded residual units are used to decode the features obtained from the encoder, and these features are upsampled to the input image using Bilinear interpolation. The size of the target is determined, and then a Sigmoid layer is used to limit the output range to between 0 and 1 to obtain the edge information of the target. The process is defined as follows: in, This represents the parameter set for edge extraction branches; Step 3: Apply the object detection branch to the input image Preprocessing is performed to obtain a magnified image of the local target. Then, the Sobel edge detection operator is used to process the input image. Image obtained by edge extraction ; Step 4: Using the object detection branch model Obtain the predicted image The constructed multi-scale feature progressive fusion encoder is used for image processing. With two processed images and Feature extraction is performed; then, cascaded residual units are used to decode the features obtained from the encoder, and these features are upsampled to the input image using Bilinear interpolation. The size of the image is determined by convolution operations using cascaded residual units to obtain the predicted image. ,Right now: in, The parameter set representing the target detection branch; The edge extraction branch model With the target detection branch model Between them, a separation interaction module is set up in the decoding stage to cascade the intermediate features of the target detection branch and the edge extraction branch. The cascaded features are processed by global average pooling (GAP) and multilayer perceptron (MLP) to obtain the interaction attention vector, which enables the model to suppress noise features in the depth flow. The edge extraction branch model With the target detection branch model Between these elements, an edge-guided image thinning module is included, employing an edge extraction branch model. The obtained I edge， Intermediate features of the target detection branch F Perform region of interest localization to obtain region features. This allows the model to focus on a specific region in the image; The image extracted by the edge-guided image thinning module is Yes: Let the intermediate features of the target detection branch be... F The image obtained by the edge extraction branch is By setting Remove some noise from the image and find all connected regions in the image to obtain the coordinates of the region locations. X , Y }; Calculate the average confidence score within each connected region, sort all confidence scores in descending order, and select the connected regions corresponding to the top three highest confidence scores as candidate regions. Based on this, determine the minimum bounding rectangle of each candidate region and map it to the intermediate features of the object detection branch. F The features of the region of interest are obtained by cropping the features at the corresponding positions. ; Step 5: Train the designed network model using the training set of the infrared image dataset from Steps 2-4; during training, the AdaGrad optimizer is used to train the network parameters by minimizing the loss function; The minimized loss function Defined as: in, express The corresponding label image, This indicates the use of the Sobel edge extraction operator on the image. The processed image; Step 6: Input the images from the test set into the trained network model to obtain the detected images. .

2. The infrared small target detection method based on edge-guided sensing according to claim 1, characterized in that: The multi-scale feature progressive fusion encoder consists of four stacked feature extraction modules.

3. The infrared small target detection method based on edge-guided sensing according to claim 2, characterized in that: The feature extraction module is: Let... For the first i -1 feature extraction module outputs features, where i ={2,3,4}, feature Its downsampled features Combined input to the current feature extraction module i In the middle; by aggregating content across multiple scales, semantic information and contextual relevance are enhanced, as shown in the formula: in, Represents cascaded residual cells. and These represent upsampling and downsampling operations, respectively.

4. The infrared small target detection method based on edge-guided sensing according to claim 3, characterized in that: The linked residual unit consists of 3 residual blocks, each of which consists of 2 convolutional layers with 3×3 convolutional kernels and 1 ReLU activation function layer.

5. The infrared small target detection method based on edge-guided sensing according to claim 1, characterized in that: The input image Preprocessing is performed using a local target amplifier, and the process is defined as follows: Among them, blocks By analyzing the input image Obtained by slicing with a step size of 1. and Used to change the size of image blocks.

6. The infrared small target detection method based on edge-guided sensing according to claim 1, characterized in that: The use .

7. An electronic device, characterized in that, It includes a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of the infrared small target detection method based on edge-guided perception as described in any one of claims 1 to 6.

8. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the infrared small target detection method based on edge-guided perception as described in any one of claims 1 to 6.

Images