Infrared small target detection method and system based on gradient information guided deformable convolution

Abstract

The application discloses an infrared small target detection method and system based on gradient information guided deformable convolution, and belongs to the technical field of computer vision and infrared image processing. The method comprises the following steps: extracting a feature map from an input infrared image; calculating the gradient change intensity and the gradient change direction of each spatial position in the feature map; generating a deformation coefficient based on the gradient change intensity and constructing a scaling matrix to anisotropically compress the sampling grid of a standard convolution kernel; constructing a rotation matrix based on the gradient change direction to rotate the compressed sampling grid to be aligned with the gradient change direction; performing bilinear interpolation sampling and convolution operation according to the transformed sampling offset to output a gradient-guided enhanced feature map; and inputting the enhanced feature map into a target detection network to obtain an infrared small target detection result. The application effectively improves the extraction capability of the edge features of the infrared weak small target by explicitly fusing the gradient information to guide the convolution sampling, significantly improves the detection precision and recall rate under the premise that the calculation overhead is basically unchanged, and can be widely applied to the fields of disaster search and rescue, fire warning, night monitoring and the like.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and infrared image processing technology, specifically to an infrared small target detection method and system based on gradient information-guided deformable convolution. Background Technology

[0002] With the rapid development of target detection algorithms and the widespread deployment of intelligent monitoring equipment, infrared imaging technology has irreplaceable advantages in nighttime drone surveillance, maritime vessel detection, vehicle detection, fire early warning, and military target identification. Compared with visible light imaging, infrared imaging technology is not limited by lighting conditions, can penetrate smoke, dust, and other obstructions, and provides stable thermal radiation information in complex environments.

[0003] However, the inherent characteristics of infrared images, such as low resolution, weak texture features, blurred boundaries, and low signal-to-noise ratio, pose a significant challenge to target detection, especially when detecting small targets at long distances. In infrared images, distant targets typically appear as small bright spots. Imaging distance and thermal limitations mean that these targets occupy a small pixel area in the image, further resulting in a loss of internal texture information. The transition region between the target and the background becomes an important criterion for infrared target detection. However, unlike the clear geometric boundaries in visible light images, the thermal radiation changes between the target and the background in infrared images usually exhibit a continuous and smooth spatial gradient. Under low-resolution and long-distance imaging conditions, infrared images cannot fully reproduce the original continuous thermal radiation gradient, only showing sparse, abrupt, and relatively fragile edge responses.

[0004] Most existing infrared target detection models use standard convolution for feature extraction. When facing weak infrared targets, standard convolution has the following shortcomings: (1) Standard convolution has a fixed sampling structure. When extracting easily damaged and directional edge gradient features, it inevitably introduces background noise, causing a decrease in the signal-to-noise ratio of the target edge features; (2) The discrete values ​​sampled by standard convolution cannot accurately reflect the continuous changing features of the target edge.

[0005] Some methods introduce deformable convolution to adjust the sampling point positions by predicting offsets, thereby enhancing the model's ability to perceive target edges. However, these offset sampling points are usually adaptively generated based on the feature map itself. When infrared targets exhibit weak feature responses, the model lacks sufficient cues to accurately predict the offsets, resulting in unstable sampling positions. In infrared weak feature scenarios, data-driven modeling alone has limitations and needs to be guided by physical prior features. Summary of the Invention

[0006] Purpose of the invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide an infrared small target detection method and system based on gradient information-guided deformable convolution. By explicitly incorporating the physical gradient information of the image into the convolution sampling process, the ability to extract edge features of infrared weak targets is effectively improved.

[0008] Technical solution

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] In a first aspect, the present invention provides an infrared small target detection method based on gradient-guided deformable convolution, comprising: preprocessing an input infrared image to extract a feature map F; calculating gradient information for the feature map F, including the gradient change intensity M and gradient change direction θ at each spatial location; generating a deformation coefficient δ based on the gradient change intensity M and constructing a scaling matrix SM(δ); constructing a rotation matrix RM(θ) based on the gradient change direction θ; transforming the set of regular grid offsets of the standard convolution kernel sequentially through the scaling matrix and the rotation matrix to obtain an adaptive sampling offset; sampling the feature map F using a bilinear interpolation algorithm according to the adaptive sampling offset to obtain feature values ​​at the sampling locations; performing convolution operations based on the sampled feature values ​​and the convolution kernel weights to output a gradient-guided enhanced feature map; and inputting the gradient-guided enhanced feature map into a target detection network to obtain the detection result of the infrared small target.

[0011] Secondly, the present invention provides an infrared small target detection system based on gradient information-guided deformable convolution, comprising: a feature extraction module, a gradient calculation module, a deformation coefficient generation module, a rotation alignment module, a sampling grid transformation module, a feature sampling module, a convolution output module, and a target detection module.

[0012] Compared with existing deformable convolution methods, this invention has the following substantial differences: (1) Existing deformable convolution methods learn the offsets entirely from the data, lacking sufficient clues to accurately predict the offsets in infrared weak feature scenarios; this invention uses the physical gradient information extracted by the Sobel operator to explicitly guide the sampling process, without relying on data-driven offset prediction, making sampling more stable and reliable. (2) Existing methods do not involve anisotropic compression and rotation alignment of the sampling grid; this invention achieves dense sampling in the gradient change direction through scaling and rotation matrices, fully extracting the gradual thermal change features of infrared small target edges.

[0013] Beneficial effects

[0014] The present invention has the following beneficial effects: (1) By explicitly fusing gradient information, the convolution kernel can adaptively adjust the sampling grid according to the intensity and direction of the edge gradient change, and achieve dense sampling in the direction of prominent gradient change, so as to fully extract the heat change characteristics of the gradual change of the edge of the infrared small target; (2) Compared with the data-driven offset prediction method, the present invention uses physical prior features for guidance, and the sampling is more stable and reliable in the weak feature infrared scene; (3) The present invention significantly improves the detection accuracy and recall rate of infrared small targets while keeping the computational overhead basically unchanged. Attached Figure Description

[0015] Figure 1 This is an overall flowchart of the method of the present invention.

[0016] Figure 2 This is a schematic diagram of the structure of GGConv.

[0017] Figure 3 This is a structural diagram of the GG-Net backbone network.

[0018] Figure 4 A schematic diagram of the sampling mesh transformation for anisotropic compression and rotation alignment.

[0019] Figure 5 This is a block diagram of the system of the present invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0021] Example 1: GGConv Module

[0022] The gradient-guided deformable convolution (GGConv) proposed in this invention includes the following steps:

[0023] Step 1: Gradient Calculation. To aggregate global texture information, GGConv uses max pooling to reduce the number of channels in the feature map F and performs normalization. Before the Sobel operation, Gaussian smoothing is performed to suppress high-frequency noise interference, resulting in a single-channel feature map S. GGConv uses the Sobel operator to calculate the gradient components G_x and G_y in the horizontal and vertical directions, respectively. Based on the gradient components, the gradient change intensity M and gradient change direction θ are calculated for each spatial location p_0 in the feature map. GGConv introduces a lightweight multilayer perceptron MLP (Conv→SiLU→Conv→Sigmoid), which receives the gradient change intensity log(1+M) after logarithmic transformation as input and outputs the deformation coefficient δ=σ(MLP(log(1+M))), where σ(·) is the Sigmoid activation function. The deformation coefficient δ reflects the degree of compression of the distance between sampling points in the current region: in the edge region where the gradient is prominent, δ tends to a larger value; in the gradient flat region, δ tends to a smaller value.

[0024] Step 2: Adaptive Sampling. GGConv redetermines the sampling position using the gradient information extracted in Step 1. For a standard convolutional kernel, its set of regular grid offsets is defined as P_k = {(-1,-1), (-1,0), …, (0,0),…, (1,1)}, where k is the number of sampling points (e.g., k=9 for a 3×3 convolutional kernel). GGConv constructs a scaling matrix SM(δ) = [[1, 0], [0, 1-δ]] to compress the regular grid along the x-axis (this direction will later be aligned with the gradient change direction). Based on the gradient direction θ, GGConv constructs a rotation matrix RM(θ) = [[cosθ, -sinθ], [sinθ, cosθ]] to rotate the compressed sampling grid to align with the gradient change direction. GGConv applies the scaling and rotation matrices sequentially to the regular grid offset P_k to obtain the adaptive sampling offset RM(θ)·SM(δ)·P_k.

[0025] Step 3: Bilinear Interpolation and Feature Map Output. Since the sampling locations are usually non-integer coordinates, GGConv uses a bilinear interpolation algorithm to sample the input features F. For position P_0 on the output feature map, the feature value Y(P_0) output after convolution is Σ k=1 K W_k · f(P_0 + RM(θ)·SM(δ)·P_k), where W_k is the convolution kernel weight and f(·) represents the bilinear interpolation function. GGConv outputs a feature map after performing a residual concatenation between the convolution result and the original feature map.

[0026] Example 2: GG-Net Detection Network

[0027] This invention integrates GGConv into the YOLO11 backbone network to obtain GG-Net. Specifically, GGConv replaces the standard convolutions in BottleNeck in the YOLO11 backbone network, forming the C3k2-GG module. Experimental results on the HIT-UAV dataset show that:

[0028] (1) Compared with the original YOLO11, GG-Net improved mAP50 by 1.8 percentage points, from 83.5% to 85.3%; mAP50-95 improved from 55.0% to 56.3%, an improvement of 1.3 percentage points; and Recall improved from 78.5% to 82.3%, an improvement of 3.8 percentage points.

[0029] (2) Ablation experiments show that when anisotropic compression is used alone, mAP50-95 is increased by about 0.6 percentage points, when rotational alignment is used alone, mAP50-95 is increased by about 0.5 percentage points, and when both are used at the same time, mAP50-95 is increased by 1.3 percentage points, which is greater than the sum of the increases of the two mechanisms alone, proving that there is a technical synergistic effect between the two.

[0030] The above experimental results verify the effectiveness of the present invention.

[0031] Example 3: Practical Application Scenarios

[0032] The present invention can be applied to the following scenarios: (1) Disaster search and rescue: using UAVs equipped with infrared imaging equipment to search for and rescue disaster victims at night or in severe weather conditions. The high recall rate of GG-Net can effectively reduce missed detections; (2) Early detection of forest fires: using infrared imaging to detect weak heat source signals to achieve early warning of fires; (3) Nighttime UAV monitoring: accurately detecting and locating low-altitude flying targets; (4) Detection of ships at sea: detecting small-sized ships at long distances in complex sea conditions.

[0033] Industrial applicability

[0034] This invention can be integrated into hardware devices such as infrared monitoring equipment, drone systems, and security monitoring systems, and has good prospects for industrial applications.

Claims

1. A method for detecting small infrared targets based on gradient-guided deformable convolution, characterized in that, include: The input infrared image is preprocessed to extract feature map F; Calculate gradient information for the feature map F, including the gradient change intensity M and gradient change direction θ at each spatial location; Based on the gradient change intensity M, a deformation coefficient δ is generated, and a scaling matrix SM(δ) = [[1, 0], [0, 1-δ]] is constructed; Based on the gradient change direction θ, a rotation matrix RM(θ) = [[cosθ, -sinθ], [sinθ, cosθ]] is constructed; the set of regular grid offsets P_k of the standard convolution kernel is transformed sequentially through the scaling matrix SM(δ) and the rotation matrix RM(θ) to obtain the adaptive sampling offset RM(θ)·SM(δ)·P_k; according to the adaptive sampling offset, the feature map F is sampled using a bilinear interpolation algorithm to obtain the feature values ​​at the sampling positions; convolution operation is performed based on the sampled feature values ​​and the convolution kernel weights, and the convolution operation result is residually concatenated with the feature map F to output the gradient-guided enhanced feature map; the gradient-guided enhanced feature map is input into the target detection network to obtain the detection result of infrared small targets.

2. The method according to claim 1, characterized in that, The calculation process of the gradient change intensity M and gradient change direction θ includes: performing max pooling and normalization on the feature map F, and smoothing it using a Gaussian smoothing filter to obtain a single-channel feature map S; using the Sobel operator to calculate the horizontal gradient component G_x and the vertical gradient component G_y of the single-channel feature map S; and calculating the gradient change intensity M = √(G_x^2 + G_y^2 + ε) and the gradient change direction θ = arctan2(G_x, G_y) based on G_x and G_y.

3. The method according to claim 1, characterized in that, The deformation coefficient δ is generated by a lightweight multilayer perceptron (MLP), which includes convolutional layers, SiLU activation function, and Sigmoid activation function. The MLP receives the gradient change intensity log(1+M) after logarithmic transformation as input and outputs a deformation coefficient δ in the range of (0,1). The deformation coefficient δ reflects the degree of compression of the distance between sampling points in the current region: in the edge region where the gradient is prominent, δ tends to be larger. In regions where the gradient is flat, δ tends to a smaller value.

4. The method according to claim 1, characterized in that, The scaling matrix SM(δ) compresses the x-axis direction of the regular grid offset to reduce the sampling interval along the gradient change direction after subsequent rotation and alignment, thereby achieving dense sampling in the edge region.

5. The method according to claim 4, characterized in that, The rotation matrix RM(θ) rotates the compressed sampling grid to align with the gradient change direction, so that the main axis direction of the convolution kernel is consistent with the edge gradient direction.

6. The method according to claim 1, characterized in that, After sampling the input features F, the bilinear interpolation algorithm outputs the feature value Y(P_0) = Σ for position P_0 on the output feature map. k=1 K W_k · f(P_0 + RM(θ)·SM(δ)·P_k), where W_k is the convolution kernel weight, K is the total number of sampling points, and f(·) represents the pixel value calculated using bilinear interpolation based on the sampling position and the pixel value of the feature map.

7. The method according to claim 1, characterized in that, The feature map output by the convolution operation is residually concatenated with the feature map F to form the final gradient-guided enhanced feature map.

8. The method according to claim 1, characterized in that, The gradient information guides deformable convolutions to replace the standard convolutions in the BottleNeck module of the target detection network backbone, forming a gradient-guided enhancement module; the target detection network is a YOLO series detection network.

9. The method according to claim 1, characterized in that, The infrared small target detection method is applicable to at least one of the following scenarios: nighttime drone surveillance, maritime vessel detection, vehicle detection, fire early warning, military target identification, and disaster search and rescue.

10. An infrared small target detection system based on gradient information-guided deformable convolution, characterized in that, include: The feature extraction module is used to preprocess the input infrared image and extract the feature map F; The gradient calculation module is used to calculate gradient information for the feature map F, including the gradient change intensity M and gradient change direction θ at each spatial location; The deformation coefficient generation module is used to generate the deformation coefficient δ based on the gradient change intensity M, and construct the scaling matrix SM(δ) = [[1, 0],[0, 1-δ]]. The rotation alignment module is used to construct a rotation matrix RM(θ) = [[cosθ, -sinθ], [sinθ, cosθ]] based on the gradient change direction θ; the sampling grid transformation module is used to transform the set of regular grid offsets P_k of the standard convolution kernel sequentially through the scaling matrix SM(δ) and the rotation matrix RM(θ) to obtain an adaptive sampling offset RM(θ)·SM(δ)·P_k; the feature sampling module is used to sample the feature map F according to the adaptive sampling offset using a bilinear interpolation algorithm to obtain the feature values ​​at the sampling positions; the convolution output module is used to perform convolution operations based on the sampled feature values ​​and the convolution kernel weights, and perform residual concatenation between the convolution operation result and the feature map F to output the gradient-guided enhanced feature map; The target detection module is used to input the gradient-guided enhanced feature map into the target detection network to obtain the detection results of infrared small targets.

Images