Multi-direction edge perception feature fusion network for detecting small infrared target and detection method

Abstract

The invention discloses a multi-direction edge perception feature fusion network for detecting an infrared small target and a detection method, and solves the problems that in the prior art, in an infrared small target detection method, target details are easily lost in a deep layer of the network after multiple times of down-sampling, so that the shape of a prediction result is changed; the system specifically comprises an initial feature extraction unit, a backbone feature network unit, a prediction head unit and a loss function calculation unit which are connected in sequence. And the edge feature network unit is respectively connected with the backbone feature network unit and the prediction head unit. An initial feature extraction unit, a backbone feature network unit, an edge feature network unit, a prediction head unit and a loss function calculation unit are combined to detect an infrared small target image, so that a target or target details can be prevented from being lost, the accuracy of a detection result is improved, and meanwhile, target edge detail information is kept to be completed; and the false alarm rate can be greatly reduced.

Description

Technical Field

[0001] This invention relates to a detection system for small infrared targets, and more specifically to a multi-directional edge sensing feature fusion network and detection method for detecting small infrared targets. Background Technology

[0002] In recent years, infrared small target detection has received increasing attention, playing a crucial role in various critical applications such as maritime rescue, fire rescue, and building monitoring. Infrared small target detection refers to the technology of accurately detecting and identifying small targets in infrared images. Compared to visible light images, infrared images are more robust to changes in illumination and environmental interference. Furthermore, compared to synthetic aperture radar (SAR) images, infrared images offer greater concealment. In remote sensing images containing infrared small targets, the detected targets are typically very small due to the long imaging distance, while the background usually occupies most of the image.

[0003] However, unlike typical object detection tasks, IRSTD faces several unique challenges:

[0004] 1) Small target size: Since infrared sensors are generally far from the target, small infrared targets often occupy only a few pixels in the image, such as less than 81 pixels in a 256×256 resolution image.

[0005] 2) Low contrast and signal-to-noise ratio (SNR): Infrared images represent the intensity of thermal radiation received by the sensor from the target. In some scenarios, the contrast difference between the target and the surrounding background is very small, making it difficult for deep learning networks to distinguish them.

[0006] 3) Diverse and variable target shapes: The appearance and shape of small infrared targets vary depending on the background, which further increases the difficulty of detection.

[0007] The existence of the above problems greatly increases the difficulty of detecting small infrared targets. Therefore, how to construct a method for detecting small infrared targets with high accuracy, low false alarm rate and strong robustness has become an urgent problem to be solved.

[0008] In recent years, due to the powerful feature representation capabilities of convolutional neural networks (CNNs), deep learning-based methods have been widely applied in the field of infrared small target detection. However, existing methods face several pressing issues. First, infrared small targets are much smaller than typical small targets. Most current CNN-based methods are simply modified versions of general target detection networks. However, these methods typically rely on multi-layer downsampling for feature extraction, which can lead to the loss of small target information in deeper layers, thus degrading network performance. Second, due to the limited feature information in infrared images, edge information becomes particularly important. However, most existing edge-guided methods rely solely on the unidirectional Sobel operator for feature extraction, often resulting in smooth target shapes that are insufficient to capture complete target edge features and details. Therefore, further research is needed on how to detect small targets and accurately capture details. Summary of the Invention

[0009] To address the technical problem in existing infrared small target detection methods that suffer from target detail loss in deeper layers of the network due to multiple downsampling steps, leading to changes in the shape of the prediction result or even the complete loss of small target information, this invention provides a multi-directional edge perception feature fusion network and detection method for detecting infrared small targets.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] This invention provides a multi-directional edge-aware feature fusion network for detecting small infrared targets, which is unique in that:

[0012] It includes an initial feature extraction unit, a backbone feature network unit, an edge feature network unit, a prediction head unit, and a loss function calculation unit;

[0013] The input end of the initial feature extraction unit is used to input the original infrared image, and its output end is connected to the input end of the backbone feature network unit to perform initial feature extraction on the original infrared image to obtain an initial feature map.

[0014] The first input terminal of the edge feature network unit is used to input the original infrared image, its second input terminal is connected to the first output terminal of the backbone feature network unit, and its output terminal is connected to the first input terminal of the prediction head unit, for edge feature extraction processing of the original infrared image.

[0015] The second output of the backbone feature network unit is connected to the second input of the prediction head unit, and is used to perform feature encoding and decoding processing on the initial feature map.

[0016] The output of the prediction head unit is connected to the loss function calculation unit, and is used to perform feature prediction processing on the feature map after feature encoding and decoding and the feature map after edge feature extraction.

[0017] The loss function calculation unit is used to calculate edge loss and main loss based on the original mask label of the original infrared image and the feature map after feature prediction processing, and then backpropagates to optimize the multi-directional edge perception feature fusion network.

[0018] Furthermore, the backbone feature network unit includes first to fourth encoders, an intermediate layer, and first to fourth decoders;

[0019] The input terminal of the first encoder is connected to the output terminal of the initial feature extraction unit, and its output terminal is connected to the input terminal of the second encoder and the first input terminal of the first decoder, respectively.

[0020] The output of the second encoder is connected to the input of the third encoder and the first input of the second decoder, respectively.

[0021] The output of the third encoder is connected to the input of the fourth encoder and the first input of the third decoder, respectively.

[0022] The output of the fourth encoder is connected to the input of the intermediate layer and the first input of the fourth decoder, respectively.

[0023] The output of the intermediate layer is connected to the second input of the fourth decoder;

[0024] The output of the fourth decoder is connected to the second input of the third decoder;

[0025] The output of the third decoder is connected to the second input of the second decoder;

[0026] The output of the second decoder is connected to the second input of the first decoder;

[0027] The outputs of the first to fourth decoders are also connected to the second input of the prediction head unit, respectively.

[0028] The outputs of the first to third decoders are also connected to the second input of the edge feature network unit, respectively.

[0029] Furthermore, the edge feature network unit includes an MSEE module, a first 1×1 convolutional layer, a first DAGF module, a second DAGF module, a third DAGF module, and a second to fourth 1×1 convolutional layers connected sequentially from the input end to the output end.

[0030] The input terminal of the MSEE module is used to input the original infrared image;

[0031] The output of the first 1×1 convolutional layer is connected to the first input of the first DAGF module;

[0032] The output of the first DAGF module is connected to the first input of the second DAGF module;

[0033] The output of the second DAGF module is connected to the first input of the third DAGF module;

[0034] The input of the second 1×1 convolutional layer is connected to the output of the third decoder, and its output is connected to the second input of the first DAGF module.

[0035] The input of the third 1×1 convolutional layer is connected to the output of the second decoder, and its output is connected to the second input of the second DAGF module.

[0036] The input of the fourth 1×1 convolutional layer is connected to the output of the first decoder, and its output is connected to the second input of the fourth DAGF module.

[0037] The output of the third DAGF module is connected to the first input of the prediction head unit.

[0038] Furthermore, the first DAGF module includes a first depthwise separable convolutional layer, a second pixel stacking module, a sixth 1×1 convolutional layer, a second depthwise separable convolutional layer, a gated convolutional layer, an Alpha module, a second pixel dot product module, and a sixth feature concatenation module;

[0039] The input end of the first depth-separable convolutional layer is connected to the output end of the first 1×1 convolutional layer, and its output end is connected to the first input end of the second pixel overlay module and the input end of the second depth-separable convolutional layer, respectively.

[0040] The second input terminal of the second pixel overlay module is connected to the output terminal of the first 1×1 convolutional layer, and its output terminal is connected to the input terminal of the sixth 1×1 convolutional layer.

[0041] The output of the sixth 1×1 convolutional layer is connected to the first input of the second pixel dot product module and the input of the Alpha module, respectively.

[0042] The output of the second depth separable convolutional layer is connected to the first input of the gated convolutional layer and the first input of the sixth feature stitching module, respectively.

[0043] The second input of the gated convolutional layer is connected to the output of the second 1×1 convolutional layer, and its output is connected to the second input of the sixth feature splicing module.

[0044] The output of the Alpha module is connected to the second input of the second pixel dot product module;

[0045] The output of the second pixel dot product module is connected to the third input of the sixth feature concatenation module;

[0046] The output of the sixth feature splicing module is connected to the second DAGF module;

[0047] The second and third DAGF modules have the same structure as the first DAGF module.

[0048] Furthermore, the gated convolutional layer includes a third pixel stacking module, a first batch of normalization layers, a seventh 1×1 convolutional layer, a first ReLU activation function, an eighth 1×1 convolutional layer, a second batch of normalization layers, a seventh Sigmoid function, a third pixel dot product module, and a ninth 1×1 convolutional layer, which are connected sequentially from the input end to the output end.

[0049] The output of the seventh Sigmoid function is connected to the first input of the third pixel multiplication module;

[0050] The input end of the third pixel overlay module is connected to the output end of the second depth separable convolutional layer and the output end of the second 1×1 convolutional layer, respectively, and its output end is also connected to the second input end of the third pixel dot multiplication module.

[0051] The output of the ninth 1×1 convolutional layer is connected to the second input of the sixth feature splicing module.

[0052] Furthermore, the MSEE module includes a Sobel operator module, a seventh feature concatenation module, a 10th 1×1 convolutional layer, a third batch normalization layer, a second ReLU activation function, a Softmax function, and a fourth pixel dot product module, which are connected sequentially from the input end to the output end.

[0053] The output of the Softmax function is connected to the first input of the fourth pixel dot product module;

[0054] The Sobel operator module includes Sobel operators in four directions: 0°, 45°, 90°, and 135°, which are used to extract edge gradient features in four directions from the original infrared image.

[0055] The output of the Sobel operator module is also connected to the second input of the fourth pixel dot product module;

[0056] The output of the fourth pixel dot product module is connected to the input of the first 1×1 convolutional layer.

[0057] Further, the prediction head unit includes a second 3×3 convolutional layer whose input is connected to the output of the fourth decoder, a third 3×3 convolutional layer whose input is connected to the output of the third decoder, a fourth 3×3 convolutional layer whose input is connected to the output of the second decoder, a fifth 3×3 convolutional layer whose input is connected to the output of the first decoder, a second Sigmoid function whose input is connected to the output of the second 3×3 convolutional layer, a third Sigmoid function whose input is connected to the output of the third 3×3 convolutional layer, a fourth Sigmoid function whose input is connected to the output of the fourth 3×3 convolutional layer, and an input... The system includes a fifth Sigmoid function connected to the output of the fifth 3×3 convolutional layer, a fifth 1×1 convolutional layer connected to the output of the third DAGF module, a first Sigmoid function connected to the output of the fifth 1×1 convolutional layer, a concatenation multiplication module connected to the outputs of the second to fourth Sigmoid functions respectively, a sixth 3×3 convolutional layer connected to the output of the concatenation multiplication module, and a sixth Sigmoid function connected to the output of the sixth 3×3 convolutional layer; the outputs of the first to sixth Sigmoid functions are all connected to the loss function calculation unit.

[0058] The output of the first Sigmoid function is also connected to the concatenation dot product module.

[0059] Furthermore, the splicing dot multiplication module includes a first to a fifth splicing module and a first pixel dot multiplication module;

[0060] The input terminal of the first splicing module is connected to the output terminal of the second Sigmoid function, and its output terminal is connected to the second input terminal of the second splicing module;

[0061] The first input terminal of the second splicing module is connected to the output terminal of the third Sigmoid function, and its output terminal is connected to the second input terminal of the third splicing module.

[0062] The first input terminal of the third splicing module is connected to the output terminal of the fourth Sigmoid function, and its output terminal is connected to the second input terminal of the fourth splicing module.

[0063] The first input terminal of the fourth splicing module is connected to the output terminal of the fifth Sigmoid function, and its output terminal is connected to the first input terminal of the first pixel dot multiplication module and the second input terminal of the fifth splicing module, respectively.

[0064] The second input of the first pixel multiplication module is connected to the output of the first Sigmoid function, and its output is connected to the first input of the fifth splicing module.

[0065] The output of the fifth splicing module is connected to the input of the sixth 3×3 convolutional layer.

[0066] Furthermore, the initial feature extraction unit includes a dilation and erosion module for inputting the original infrared image, a first pixel overlay module for inputting the original infrared image, and a first 3×3 convolutional layer whose input is connected to the output of the first pixel overlay module;

[0067] The output end of the expansion corrosion module is connected to the second input end of the first pixel overlay module;

[0068] The output of the first 3×3 convolutional layer is connected to the input of the first encoder.

[0069] This invention also provides a detection method for detecting small infrared targets, employing the aforementioned multi-directional edge-aware feature fusion network for detecting small infrared targets, characterized by the following steps:

[0070] Step 1: Construct the multi-directional edge sensing feature fusion network for detecting small infrared targets as described above;

[0071] Step 2: Obtain multiple known raw infrared images and their corresponding raw mask labels, and preprocess the raw infrared images; the number of raw infrared images is greater than the preset number of iterations;

[0072] Step 3: Perform steps 4 to 9 on one of the original infrared images and its original mask label through the multi-directional edge-aware feature fusion network, while retaining the corresponding network parameters;

[0073] Step 4: The initial feature extraction unit performs initial feature extraction processing on the original infrared image to obtain the initial feature map;

[0074] Step 5: The backbone feature network unit performs feature encoding and decoding on the initial feature map to obtain feature maps f1, f2, f3, and f4;

[0075] Step 6: The edge feature network unit extracts edge features from the original infrared image based on feature maps f1, f2, and f3 to obtain feature map F1;

[0076] Step 7: The prediction head unit performs convolution and sigmoid processing on feature maps f1, f2, f3, f4 and feature map F1 to obtain edge prediction result map pe and feature prediction result maps p1, p2, p3, p4. Then, the edge prediction result map pe and the feature prediction result maps p1, p2, p3, p4 are concatenated and multiplied, and then convolution and sigmoid processing are performed again to obtain feature map P.

[0077] Step 8: The loss function calculation unit calculates the edge loss based on the original infrared image and the edge prediction result map pe, and calculates the main feature loss based on the original mask label, feature prediction result maps p4, p3, p2, p1 and feature map P, to obtain the edge loss and the main loss;

[0078] Step 9: Backpropagate the edge loss and main loss in the multi-directional edge-aware feature fusion network to optimize the multi-directional edge-aware feature fusion network;

[0079] Step 10: Replace with another original infrared image and repeat steps 3-9 until the preset number of iterations is reached. Retain the optimal network parameters to obtain the optimal multi-directional edge perception feature fusion network.

[0080] Step 11: Detect the small target image under test using the optimal multi-directional edge-aware feature fusion network to obtain the target detection map and complete the detection.

[0081] Further, step 2 specifically involves acquiring multiple known raw infrared images I. input The original infrared image I was processed using Gaussian blur and data augmentation operations, respectively. input Preprocessing is performed to make all raw infrared images I input Consistent in size, and acquisition of each original infrared image I input The corresponding original mask label.

[0082] Furthermore, step 4 specifically includes:

[0083] 4.1 The expansion corrosion module uses the expansion corrosion method to process the original infrared image I. input Morphological enhancement is performed to obtain morphologically enhanced image I. morph ;

[0084] 4.2 The first pixel overlay module overlays the original infrared image I input and morphologically enhanced image I morph A fused image I is formed by stitching along the channel dimension. fusion ;

[0085] 4.3, The first 3×3 convolutional layer on the fused image I fusion Perform convolution processing to obtain the initial feature map x initial .

[0086] Furthermore, step 5 specifically includes:

[0087] 5.1 The first encoder extracts the initial feature map x initial The target features are used to obtain feature map e1;

[0088] 5.2 The second encoder extracts the target features from feature map e1 to obtain feature map e2;

[0089] 5.3 The third encoder extracts the target features from feature map e2 to obtain feature map e3;

[0090] 5.4 The fourth encoder extracts the target features from feature map e3 to obtain feature map e4;

[0091] 5.5. Extract target features from feature map e4 in the intermediate layer to obtain feature map m;

[0092] 5.6 The fourth decoder decodes feature map m and feature map e4 to obtain feature map f4;

[0093] 5.7 The third decoder decodes feature map f4 and feature map e3 to obtain feature map f3;

[0094] 5.8 The second decoder decodes feature map f3 and feature map e2 to obtain feature map f2;

[0095] 5.9 The first decoder decodes feature map f2 and feature map e1 to obtain feature map f1.

[0096] Furthermore, step 6 specifically includes:

[0097] 6.1 The MSEE module processes the original infrared image I input Feature extraction is performed to obtain the feature gradient map F;

[0098] 6.2 The first 1×1 convolutional layer adjusts the feature gradient map F from 1 channel to 64 channels, resulting in the feature map Fe;

[0099] 6.3 The second to fourth 1×1 convolutional layers adjust the feature maps f3, f2, and f1 from 1 channel to 64 channels, respectively, to obtain feature maps ff3, ff2, and ff1;

[0100] 6.4 The first DAGF module performs image edge feature enhancement on feature map Fe and feature map ff3 to obtain feature map F3;

[0101] 6.5 The second DAGF module enhances the image edge features of feature map F3 and feature map ff2 to obtain feature map F2;

[0102] 6.6 The third DAGF module enhances the image edge features of feature map F2 and feature map ff1 to obtain feature map F1.

[0103] Furthermore, step 6.1 specifically includes:

[0104] 6.1.1 The Sobel operator module extracts multi-directional edge gradient features from the original infrared image and performs channel-dimensional stitching through the seventh feature stitching module to obtain the gradient response map G;

[0105] 6.1.2 The tenth 1×1 convolutional layer performs convolution on the gradient response map G to learn the corresponding weight features of each direction of the gradient response map G. Then, it passes through the third batch normalization layer for batch normalization. Then, the second ReLU activation function is used to enhance the nonlinear expression. Finally, the gradient magnitude of each direction is normalized and weighted by the Softmax function to obtain the weight coefficient α of each direction.

[0106] 6.1.3 The fourth pixel dot product module performs a dot product between the directional weight coefficient α and the corresponding pixel in the gradient response map G, and sums them along the directional dimension to obtain the fused feature gradient map F.

[0107] Furthermore, step 6.4 specifically includes:

[0108] Step 6.4.1: The first depthwise separable convolutional layer processes the feature map Fe to obtain the feature map w1;

[0109] Step 6.4.2: The second depthwise separable convolutional layer processes the feature map w1 to obtain the feature map w2;

[0110] Step 6.4.3: The second pixel overlay module overlays feature map Fe and feature map w1 pixel by pixel to obtain feature map △p. Feature map △p is then convolved by the sixth 1×1 convolutional layer and the learnable parameters are adjusted by the Alpha module.

[0111] Step 6.4.4: The second pixel dot product module multiplies the feature map processed by the sixth 1×1 convolutional layer with the weight map processed by the Alpha module by pixels to obtain the feature map △p1;

[0112] Step 6.4.5: The gated convolutional layer processes feature map w2 and feature map ff3 to obtain feature map Pg;

[0113] Step 6.4.6: The sixth pixel overlay module overlays feature map △p1, feature map Pg, and feature map w2 pixel by pixel to obtain feature map F3.

[0114] Furthermore, step 7 specifically includes:

[0115] 7.1 The fifth 1×1 convolutional layer and the first Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map F1 to obtain the edge prediction result map pe of the target;

[0116] 7.2 The second 3×3 convolutional layer and the second Sigmoid function sequentially perform convolution and Sigmoid processing on feature map f4 to obtain feature prediction result map p4;

[0117] 7.3 The third 3×3 convolutional layer and the third Sigmoid function sequentially perform convolution and Sigmoid processing on feature map f3 to obtain feature prediction result map p3;

[0118] 7.4 The fourth 3×3 convolutional layer and the fourth Sigmoid function sequentially perform convolution and Sigmoid processing on feature map f2 to obtain feature prediction result map p2;

[0119] 7.5 The fifth 3×3 convolutional layer and the fifth Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map f1 to obtain the feature prediction result map p1;

[0120] 7.6 The concatenation dot product module performs concatenation dot product processing on the edge prediction result map pe and the feature prediction result maps p1, p2, p3, and p4 to obtain the feature map pu;

[0121] 7.7 The sixth 3×3 convolutional layer and the sixth Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map pu to obtain the feature map P.

[0122] Furthermore, in step 7.6, the concatenation dot product processing specifically includes:

[0123] 7.6.1 The first to fourth stitching modules adjust the size of the feature prediction result images p1, p2, p3, and p4 to match the original infrared image I. input The size will then be compared with the original infrared image I. input The feature prediction result images p4, p3, p2, and p1 of the same size are concatenated along the channel dimension to obtain the feature image pf;

[0124] 7.6.2 The first pixel dot multiplication module multiplies the feature map pf with the edge prediction result map pe by pixels to enhance edge details, resulting in the feature map pz;

[0125] 7.6.3 The fifth stitching module stitches feature map pz and feature map pf along the channel dimension to obtain feature map pu.

[0126] Furthermore, step 8 specifically includes:

[0127] 8.1 The loss function calculation unit acquires the original infrared image I. input Based on the corresponding edge mask labels, the edge loss of the edge prediction result image pe is calculated, and the edge loss L is obtained. EPI ;

[0128] 8.2 The loss function calculation unit obtains mask labels of the same size as those in steps 7.2-7.5 for the feature prediction result maps p4, p3, p2, and p1. Based on these mask labels and the original mask labels, it calculates the principal feature loss of the feature prediction result maps p4, p3, p2, and p1, as well as the feature map P, to obtain the principal loss L. SLS ;

[0129] Step 9 specifically involves setting the edge loss L... EPI and main loss L SLS Backpropagation is performed in the multi-directional edge-aware feature fusion network to optimize it.

[0130] The beneficial effects of this invention are:

[0131] 1. The present invention provides a multi-directional edge perception feature fusion network and detection method for detecting small infrared targets. By combining an initial feature extraction unit, a backbone feature network unit, an edge feature network unit, a prediction head unit and a loss function calculation unit, the infrared small target image can be detected. This can avoid the loss of the target or target details, improve the accuracy of the detection results, maintain the completeness of the target edge detail information, and greatly reduce the false alarm rate.

[0132] 2. The present invention provides a multi-directional edge perception feature fusion network and detection method for detecting small infrared targets. The MSEE module is used for feature extraction. By adaptively adjusting the weights of multiple edge extraction operators, a better feature extraction effect is achieved.

[0133] 3. The present invention provides a multi-directional edge-aware feature fusion network and detection method for detecting small infrared targets. The DAGF module used employs a gated cross-modal fusion strategy, combined with depthwise separable convolution and adaptive edge enhancement, to effectively model feature differences, thereby enhancing the network's ability to accurately represent and fuse edge differences. Attached Figure Description

[0134] Figure 1 This is a schematic diagram of the structure of an embodiment of a multi-directional edge perception feature fusion network for detecting small infrared targets according to the present invention;

[0135] Figure 2 This is a schematic diagram of the MSEE module in an embodiment of the present invention;

[0136] Figure 3 This is a schematic diagram of the structure of the first DAGF module in an embodiment of the present invention;

[0137] Figure 4 This is a schematic diagram of the structure of the gated convolutional layer in an embodiment of the present invention;

[0138] Figure 5 This is a schematic diagram of the visualization results of testing the method of this invention and various traditional methods on the NUAA-SIRST dataset;

[0139] Figure 6 This is a schematic diagram of the visualization results of testing the IRSTD-1K dataset using the method of this invention and a variety of traditional methods. Detailed Implementation

[0140] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0141] A multi-directional edge-aware feature fusion network for detecting small infrared targets, such as Figure 1 As shown, the multi-directional edge-aware feature fusion network includes an initial feature extraction unit, a backbone feature network unit, an edge feature network unit, a prediction head unit, and a loss function calculation unit.

[0142] The input end of the initial feature extraction unit is used to input the original infrared image, and its output end is connected to the input end of the backbone feature network unit to perform initial feature extraction on the original infrared image to obtain an initial feature map.

[0143] Specifically, the initial feature extraction unit includes a dilation and erosion module for inputting the original infrared image, a first pixel overlay module for inputting the original infrared image, and a first 3×3 convolutional layer whose input is connected to the output of the first pixel overlay module; the output of the dilation and erosion module is connected to the second input of the first pixel overlay module; and the output of the first 3×3 convolutional layer is connected to the input of the first encoder. As the name suggests, the first 3×3 convolutional layer is a convolution with a kernel size of 3×3.

[0144] The first input terminal of the edge feature network unit is used to input the original infrared image, its second input terminal is connected to the first output terminal of the backbone feature network unit, and its output terminal is connected to the first input terminal of the prediction head unit, which is used to perform edge feature extraction processing on the original infrared image.

[0145] Specifically, the edge feature network unit includes an MSEE module, a first 1×1 convolutional layer, a first DAGF module, a second DAGF module, a third DAGF module, and second to fourth 1×1 convolutional layers connected sequentially from input to output. The input of the MSEE module is used to input the original infrared image. The output of the first 1×1 convolutional layer is connected to the first input of the first DAGF module. The output of the first DAGF module is connected to the first input of the second DAGF module. The output of the second DAGF module is connected to the first input of the third DAGF module. The input of the second 1×1 convolutional layer is connected to the output of the third decoder, and its output is connected to the second input of the first DAGF module. The input of the third 1×1 convolutional layer is connected to the output of the second decoder, and its output is connected to the second input of the second DAGF module. The input of the fourth 1×1 convolutional layer is connected to the output of the first decoder, and its output is connected to the second input of the fourth DAGF module. The output of the third DAGF module is connected to the first input of the prediction head unit. The DAGF module is the gated differential sensing module.

[0146] like Figure 2 As shown, the MSEE module includes, from input to output, a Sobel operator module, a seventh feature concatenation module, a tenth 1×1 convolutional layer, a third batch normalization layer, a second ReLU activation function, a Softmax function, and a fourth pixel multiplication module. The output of the Softmax function is connected to the first input of the fourth pixel multiplication module. The Sobel operator module includes four Sobel operators in four directions: 0°, 45°, 90°, and 135°, used to extract edge gradient features in four directions of the original infrared image. After concatenation along the channel dimension, a gradient response map G∈R is obtained. 4*H*W The output of the Sobel operator module is also connected to the second input of the fourth pixel dot product module; the output of the fourth pixel dot product module is connected to the input of the first 1×1 convolutional layer. For the gradient response map G∈R... 4*H*W The response weight features of each directional channel are learned using 1×1 convolutional layers, followed by batch normalization. Then, the nonlinear expression is enhanced by the ReLU activation function. Finally, the gradient magnitude of each direction is normalized and weighted using a softmax operation to obtain the weight coefficients α∈R for each direction. 4*H*W The directional weighting coefficient α∈R 4 *H*W With gradient response map G∈R 4*H*W The corresponding pixel values ​​are multiplied, and then summed along the direction dimension to obtain the fused feature gradient feature map F∈R. 1*H*W .

[0147] The first DAGF module, the second DAGF module, and the third DAGF module have the same structure. This embodiment will use the first DAGF module as an example for description:

[0148] like Figure 3 As shown, the first DAGF module includes a first depthwise separable convolutional layer, a second pixel stacking module, a sixth 1×1 convolutional layer, a gated convolutional layer, an alpha module, a second pixel multiplication module, and a sixth feature concatenation module. The input of the first depthwise separable convolutional layer is connected to the output of the first 1×1 convolutional layer, and its output is connected to the first input of the second pixel stacking module and the input of the second depthwise separable convolutional layer, respectively. The second input of the second pixel stacking module is connected to the output of the first 1×1 convolutional layer, and its output is connected to the input of the sixth 1×1 convolutional layer. The sixth 1×1 convolutional layer... The output terminals are connected to the first input terminal of the second pixel multiplication module and the input terminal of the Alpha module, respectively; the output terminal of the second depthwise separable convolutional layer is connected to the first input terminal of the gated convolutional layer and the first input terminal of the sixth feature concatenation module, respectively; the second input terminal of the gated convolutional layer is connected to the output terminal of the second 1×1 convolutional layer, and its output terminal is connected to the second input terminal of the sixth feature concatenation module; the output terminal of the Alpha module is connected to the second input terminal of the second pixel multiplication module; the output terminal of the second pixel multiplication module is connected to the third input terminal of the sixth feature concatenation module; the output terminal of the sixth feature concatenation module is connected to the second DAGF module.

[0149] The structure of the gated convolutional layer is as follows: Figure 4 As shown, the gated convolutional layer includes, in sequence from input to output, a third pixel stacking module, a first batch of normalization layers, a seventh 1×1 convolutional layer, a first ReLU activation function, an eighth 1×1 convolutional layer, a second batch of normalization layers, a seventh Sigmoid function, a third pixel multiplication module, and a ninth 1×1 convolutional layer; the output of the seventh Sigmoid function is connected to the first input of the third pixel multiplication module; the input of the third pixel stacking module is connected to the output of the second depth-separable convolutional layer and the output of the second 1×1 convolutional layer, and its output is also connected to the second input of the third pixel multiplication module; the output of the ninth 1×1 convolutional layer is connected to the second input of the sixth feature concatenation module.

[0150] The second output of the backbone feature network unit is connected to the second input of the prediction head unit, and is used to perform feature encoding and decoding processing on the initial feature map.

[0151] Specifically: The backbone feature network unit includes first to fourth encoders, intermediate layers, and first to fourth decoders; the first to fourth encoders are all constructed using residual convolution modules; the first to fourth decoders upsample the features obtained by the first to fourth encoders using bilinear interpolation; their connection relationships are as follows: the input of the first encoder is connected to the output of the initial feature extraction unit, and its output is connected to the input of the second encoder and the first input of the first decoder, respectively; the output of the second encoder is connected to the input of the third encoder and the first input of the second decoder, respectively; the output of the third encoder is connected to the input of the fourth encoder and the first input of the third decoder, respectively; the output of the fourth encoder is connected to the input of the intermediate layer and the first input of the fourth decoder, respectively; the output of the intermediate layer is connected to the second input of the fourth decoder; the output of the fourth decoder is connected to the second input of the third decoder; the output of the third decoder is connected to the second input of the second decoder; the output of the second decoder is connected to the second input of the first decoder; simultaneously, the outputs of the first to fourth decoders are also connected to the second input of the prediction head unit, and the outputs of the first to third decoders are also connected to the second input of the edge feature network unit, respectively.

[0152] The output of the prediction head unit is connected to the loss function calculation unit, and is used to perform feature prediction processing on the feature map after feature encoding and decoding and the feature map after edge feature extraction.

[0153] Specifically, the prediction head unit includes a second 3×3 convolutional layer whose input is connected to the output of the fourth decoder, a third 3×3 convolutional layer whose input is connected to the output of the third decoder, a fourth 3×3 convolutional layer whose input is connected to the output of the second decoder, a fifth 3×3 convolutional layer whose input is connected to the output of the first decoder, a second Sigmoid function whose input is connected to the output of the second 3×3 convolutional layer, a third Sigmoid function whose input is connected to the output of the third 3×3 convolutional layer, a fourth Sigmoid function whose input is connected to the output of the fourth 3×3 convolutional layer, and a second Sigmoid function whose input is connected to the output of the fifth 3×3 convolutional layer. The fifth Sigmoid function, the fifth 1×1 convolutional layer whose input is connected to the output of the third DAGF module, the first Sigmoid function whose input is connected to the output of the fifth 1×1 convolutional layer, the concatenation dot product module whose input is connected to the output of the second to fourth Sigmoid functions respectively, the sixth 3×3 convolutional layer whose input is connected to the output of the concatenation dot product module, and the sixth Sigmoid function whose input is connected to the output of the sixth 3×3 convolutional layer; the outputs of the first to sixth Sigmoid functions are all connected to the loss function calculation unit; the output of the first Sigmoid function is also connected to the concatenation dot product module.

[0154] like Figure 1 As shown, the splicing dot product module includes first to fifth splicing modules and a first pixel dot product module; the input of the first splicing module is connected to the output of the second sigmoid function, and its output is connected to the second input of the second splicing module; the first input of the second splicing module is connected to the output of the third sigmoid function, and its output is connected to the second input of the third splicing module; the first input of the third splicing module is connected to the output of the fourth sigmoid function, and its output is connected to the second input of the fourth splicing module; the first input of the fourth splicing module is connected to the output of the fifth sigmoid function, and its output is connected to the first input of the first pixel dot product module and the second input of the fifth splicing module, respectively; the second input of the first pixel dot product module is connected to the output of the first sigmoid function, and its output is connected to the first input of the fifth splicing module; the output of the fifth splicing module is connected to the input of the sixth 3×3 convolutional layer.

[0155] The loss function calculation unit is used to calculate the edge loss and main loss based on the original mask label of the original infrared image and the feature map after feature prediction processing, and then backpropagation is used to optimize the multi-directional edge perception feature fusion network.

[0156] The above-mentioned multi-directional edge-aware feature fusion network is used to detect small target images under test, specifically including the following steps:

[0157] Step 1: Construct the multi-directional edge-aware feature fusion network described above.

[0158] Step 2: Obtain multiple known raw infrared images and their corresponding raw mask labels, and preprocess the raw infrared images; the number of raw infrared images is greater than the preset number of iterations; specifically, obtain multiple known raw infrared images I input The original infrared image I was processed using Gaussian blur and data augmentation operations, respectively. input Preprocessing is performed to make all raw infrared images I input Consistent in size, and acquisition of each original infrared image I input The corresponding original mask label.

[0159] Step 3: Perform steps 4 to 9 on one of the original infrared images and its original mask label through a multi-directional edge-aware feature fusion network, while retaining the corresponding network parameters.

[0160] Step 4: The initial feature extraction unit performs initial feature extraction processing on the original infrared image to obtain an initial feature map; specifically including:

[0161] Step 4.1: The expansion corrosion module processes the original infrared image I using the expansion corrosion method.input Morphological enhancement is performed to obtain morphologically enhanced image I. morph .

[0162] Step 4.2: The first pixel overlay module overlays the original infrared image I... input and morphologically enhanced image I morph A fused image I is formed by stitching along the channel dimension. fusion .

[0163] Step 4.3: The first 3×3 convolutional layer fuses the image I. fusion Perform convolution processing to obtain the initial feature map x initial .

[0164] Step 5: The backbone feature network unit performs feature encoding and decoding on the initial feature map to obtain feature maps f1, f2, f3, and f4; specifically including:

[0165] Step 5.1: The first encoder extracts the initial feature map x. initial The target features are used to obtain feature map e1.

[0166] Step 5.2: The second encoder extracts the target features from feature map e1 to obtain feature map e2.

[0167] Step 5.3: The third encoder extracts the target features from feature map e2 to obtain feature map e3.

[0168] Step 5.4: The fourth encoder extracts the target features from feature map e3 to obtain feature map e4.

[0169] Step 5.5: Extract the target features from feature map e4 in the intermediate layer to obtain feature map m.

[0170] Step 5.6: The fourth decoder decodes feature map m and feature map e4 to obtain feature map f4.

[0171] Step 5.7: The third decoder decodes feature map f4 and feature map e3 to obtain feature map f3.

[0172] Step 5.8: The second decoder decodes feature map f3 and feature map e2 to obtain feature map f2.

[0173] Step 5.9: The first decoder decodes feature map f2 and feature map e1 to obtain feature map f1.

[0174] Step 6: The edge feature network unit extracts edge features from the original infrared image based on feature maps f1, f2, and f3 to obtain feature map F1; specifically including:

[0175] Step 6.1: The MSEE module processes the original infrared image I.input Feature extraction is performed to obtain the feature gradient map F∈R. 1 *H*W Where R is the real number field, meaning the mean of each pixel in the tensor is a real number, H is the height of the original infrared image, and W is the width of the original infrared image. Specifically, it includes:

[0176] Step 6.1.1: The Sobel operator module extracts multi-directional edge gradient features from the original infrared image and performs channel-dimensional stitching through the seventh feature stitching module to obtain the gradient response map G;

[0177] Step 6.1.2: The tenth 1×1 convolutional layer performs convolution on the gradient response map G to learn the corresponding weight features of each direction of the gradient response map G. Then, it passes through the third batch normalization layer for batch normalization. Then, the nonlinear expression is enhanced by the second ReLU activation function. Finally, the gradient magnitude of each direction is normalized and weighted by the Softmax function to obtain the weight coefficient α of each direction.

[0178] Step 6.1.3: The fourth pixel dot product module performs a dot product between the directional weight coefficient α and the corresponding pixel in the gradient response map G, and sums them along the directional dimension to obtain the fused feature gradient map F.

[0179] Step 6.2: The first 1×1 convolutional layer adjusts the feature gradient map F from 1 channel to 64 channels to obtain the feature map Fe.

[0180] Step 6.3: The second to fourth 1×1 convolutional layers adjust the feature maps f3, f2, and f1 from 1 channel to 64 channels respectively, resulting in feature maps ff3, ff2, and ff1.

[0181] Step 6.4: The first DAGF module performs image edge feature enhancement on feature maps Fe and ff3 to obtain feature map F3; specifically including:

[0182] Step 6.4.1: The first depthwise separable convolutional layer processes the feature map Fe to obtain the feature map w1;

[0183] Step 6.4.2: The second depthwise separable convolutional layer processes the feature map w1 to obtain the feature map w2;

[0184] Step 6.4.3: The second pixel overlay module overlays feature map Fe and feature map w1 pixel by pixel to obtain feature map △p. Feature map △p is then convolved by the sixth 1×1 convolutional layer and the learnable parameters are adjusted by the Alpha module.

[0185] Step 6.4.4: The second pixel dot product module multiplies the feature map processed by the sixth 1×1 convolutional layer with the weight map processed by the Alpha module by pixels to obtain the feature map △p1;

[0186] Step 6.4.5: The gated convolutional layer processes feature map w2 and feature map ff3 to obtain feature map Pg;

[0187] Step 6.4.6: The sixth pixel overlay module overlays feature map △p1, feature map Pg, and feature map w2 pixel by pixel to obtain feature map F3.

[0188] The methods in steps 6.4.1 to 6.4.6 can effectively enhance the difference information of the target, and feature map F3 is the enhanced edge difference feature map.

[0189] Step 6.5: The second DAGF module performs image edge feature enhancement on feature map F3 and feature map ff2 to obtain feature map F2.

[0190] Step 6.6: The third DAGF module performs image edge feature enhancement on feature map F2 and feature map ff1 to obtain feature map F1.

[0191] Step 7: The prediction head unit performs convolution and sigmoid processing on feature maps f1, f2, f3, f4 and feature map F1 to obtain edge prediction result map pe and feature prediction result maps p1, p2, p3, p4. Then, the edge prediction result map pe and the feature prediction result maps p1, p2, p3, p4 are concatenated and multiplied, and then convolution and sigmoid processing are performed again to obtain feature map P. Specifically, this includes:

[0192] Step 7.1: The fifth 1×1 convolutional layer and the first Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map F1 to obtain the edge prediction result map pe of the target.

[0193] Step 7.2: The second 3×3 convolutional layer and the second Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map f4 to obtain the feature prediction result map p4.

[0194] Step 7.3: The third 3×3 convolutional layer and the third Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map f3 to obtain the feature prediction result map p3.

[0195] Step 7.4: The fourth 3×3 convolutional layer and the fourth Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map f2 to obtain the feature prediction result map p2.

[0196] Step 7.5: The fifth 3×3 convolutional layer and the fifth Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map f1 to obtain the feature prediction result map p1.

[0197] Step 7.6: The concatenation dot product module performs concatenation dot product processing on the edge prediction result map pe and the feature prediction result maps p1, p2, p3, and p4 to obtain the feature map pu.

[0198] The concatenation dot product processing specifically includes the following steps:

[0199] Step 7.6.1: The first to fourth stitching modules adjust the size of the feature prediction result images p1, p2, p3, and p4 to match the original infrared image I. input The size will then be compared with the original infrared image I. input Feature prediction result maps p4, p3, p2, and p1 of the same size are sequentially concatenated along the channel dimension to obtain feature map pf.

[0200] Step 7.6.2: The first pixel dot multiplication module multiplies the feature map pf with the edge prediction result map pe by pixels to enhance edge details, resulting in the feature map pz.

[0201] Step 7.6.3: The fifth stitching module stitches feature map pz and feature map pf along the channel dimension to obtain feature map pu.

[0202] Step 7.7: The sixth 3×3 convolutional layer and the sixth Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map pu to obtain the feature map P.

[0203] Step 8: The loss function calculation unit calculates the edge loss based on the original infrared image and the edge prediction result map pe, and calculates the main feature loss based on the original mask label, feature prediction result maps p4, p3, p2, p1, and feature map P, thus obtaining the edge loss and the main loss; specifically including:

[0204] Step 8.1: The loss function calculation unit acquires the original infrared image I. input Based on the corresponding edge mask labels, the edge loss of the edge prediction result image pe is calculated, and the edge loss L is obtained. EPI .

[0205] Step 8.2: The loss function calculation unit obtains mask labels of the same size as those in steps 7.2-7.5 for the feature prediction result maps p4, p3, p2, and p1. Based on these mask labels and the original mask labels, the main feature loss of the feature prediction result maps p4, p3, p2, and p1, as well as the feature map P, is calculated to obtain the main loss L. SLS .

[0206] Main loss L SLS The result is obtained by summing and averaging the feature prediction results p4, p3, p2, p1 and feature map P:

[0207]

[0208] Among them, L sls =L S +L L L S and L L These are scale-sensitive loss and location-sensitive loss, respectively.

[0209] L S The calculation method is as follows:

[0210]

[0211] in, A P A represents the set of target pixels predicted by the model. GT This represents the actual set of predicted target pixels, min(*) represents the minimum value function, max(*) represents the maximum value function, and var(*) represents the variance calculation function.

[0212] L L The calculation method is as follows:

[0213]

[0214] in, x p To predict the x-axis coordinates and y-axis coordinates of the center pixel... p To predict the y-axis coordinate of the center pixel, y gt Let x be the y-coordinate of the center pixel of the mask. gt The x-coordinate of the center pixel of the mask.

[0215] Edge loss L EPI The calculation method is as follows:

[0216] L EPI =L b +L bce

[0217] Among them, L b =∫ Ω φ G (q)s θ (q)dq, s:Ω→{0,1} is a binary indicator function for the region S, that is, if q∈S is the target, Otherwise, it is 0; Ω represents the image domain; φ G (q) represents the signed distance graph relative to the true boundary G; Let $G$ be the set of boundary points of the real target region $G$; this function can be directly calculated from the real target region $G$, and φ is given when $q \in G$. G (q)=-D G (q), when At that time, φ G (q)=D G (q), where D G (q) represents the Euclidean distance from any pixel q∈G in the image domain to the nearest point on the boundary.

[0218] M represents the number of target categories, P ic y is the predicted probability that observed sample i belongs to category c. ic This indicates that y represents the true class c when sample i is the true class c. ic =1, otherwise y ic =0.

[0219] Step 9: Backpropagate the edge loss and main loss in the multi-directional edge-aware feature fusion network to optimize the network; specifically, make the edge loss L... EPI and main loss L SLS Backpropagation is performed in the multi-directional edge-aware feature fusion network to optimize it.

[0220] Step 10: Replace with another original infrared image and repeat steps 3-9 until the preset number of iterations is reached. Retain the optimal network parameters to obtain the optimal multi-directional edge perception feature fusion network.

[0221] Step 11: Detect the small target image under test using the optimal multi-directional edge-aware feature fusion network to obtain the target detection map and complete the detection.

[0222] The simulation conditions are as follows:

[0223] The network was implemented on a PyTorch platform equipped with an NVIDIA GeForce RTX 4090 GPU (24GB RAM). All legacy methods were run on Matlab2021a with a 13th-generation Intel® Core™ i7-13650HX 2.60GHz and 32GB RAM. We selected two benchmark datasets (NUAA-SIRST and IRSTD-1K) for validation. To ensure consistency, the input images were resized to a uniform 256×256 and normalized. The network was optimized using the Adagrad optimizer. The initial learning rate was set to 0.05. The batch size was 8, and the epochs were 500. The warm epoch was set to 5.

[0224] The simulation results are as follows:

[0225] To verify the effectiveness of the infrared small target detection method based on edge feature fusion in this invention, we selected IoU (Intersection over Union), mIoU (mean Intersection over Union), Pd (probability of detection), and Fa (false alarm) as evaluation metrics, and tested them on two public datasets, NUAA-SIRST and IRSTD-1K. The comparison results are shown in Table 1.

[0226] Table 1: Experimental Results

[0227]

[0228]

[0229] Generally, higher values ​​for IoU, mIoU, and Pd are better, while lower false alarm rate Fa is better. To demonstrate the effectiveness of this invention, the visualization results are as follows: Figure 5 and Figure 6 As shown, through comparative analysis and visualization results, it was found that compared with other methods, the method provided in this embodiment can maintain the edge detail information of the target better while achieving higher accuracy, and the false alarm rate is lower than that of other methods.

[0230] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-directional edge sensing feature fusion network for detecting small infrared targets, characterized in that: It includes an initial feature extraction unit, a backbone feature network unit, an edge feature network unit, a prediction head unit, and a loss function calculation unit; The input end of the initial feature extraction unit is used to input the original infrared image, and its output end is connected to the input end of the backbone feature network unit to perform initial feature extraction on the original infrared image to obtain an initial feature map. The first input terminal of the edge feature network unit is used to input the original infrared image, its second input terminal is connected to the first output terminal of the backbone feature network unit, and its output terminal is connected to the first input terminal of the prediction head unit, for edge feature extraction processing of the original infrared image. The second output of the backbone feature network unit is connected to the second input of the prediction head unit, and is used to perform feature encoding and decoding processing on the initial feature map. The output of the prediction head unit is connected to the loss function calculation unit, and is used to perform feature prediction processing on the feature map after feature encoding and decoding and the feature map after edge feature extraction. The loss function calculation unit is used to calculate edge loss and main loss based on the original mask label of the original infrared image and the feature map after feature prediction processing, and then backpropagates to optimize the multi-directional edge perception feature fusion network.

2. The multi-directional edge sensing feature fusion network for detecting small infrared targets according to claim 1, characterized in that: The backbone feature network unit includes first to fourth encoders, an intermediate layer, and first to fourth decoders; The input terminal of the first encoder is connected to the output terminal of the initial feature extraction unit, and its output terminal is connected to the input terminal of the second encoder and the first input terminal of the first decoder, respectively. The output of the second encoder is connected to the input of the third encoder and the first input of the second decoder, respectively. The output of the third encoder is connected to the input of the fourth encoder and the first input of the third decoder, respectively. The output of the fourth encoder is connected to the input of the intermediate layer and the first input of the fourth decoder, respectively. The output of the intermediate layer is connected to the second input of the fourth decoder; The output of the fourth decoder is connected to the second input of the third decoder; The output of the third decoder is connected to the second input of the second decoder; The output of the second decoder is connected to the second input of the first decoder; The outputs of the first to fourth decoders are also connected to the second input of the prediction head unit, respectively. The outputs of the first to third decoders are also connected to the second input of the edge feature network unit, respectively.

3. The multi-directional edge sensing feature fusion network for detecting small infrared targets according to claim 2, characterized in that: The edge feature network unit includes an MSEE module, a first 1×1 convolutional layer, a first DAGF module, a second DAGF module, a third DAGF module, and a second to fourth 1×1 convolutional layers connected sequentially from the input end to the output end. The input terminal of the MSEE module is used to input the original infrared image; The output of the first 1×1 convolutional layer is connected to the first input of the first DAGF module; The output of the first DAGF module is connected to the first input of the second DAGF module; The output of the second DAGF module is connected to the first input of the third DAGF module; The input of the second 1×1 convolutional layer is connected to the output of the third decoder, and its output is connected to the second input of the first DAGF module. The input of the third 1×1 convolutional layer is connected to the output of the second decoder, and its output is connected to the second input of the second DAGF module. The input of the fourth 1×1 convolutional layer is connected to the output of the first decoder, and its output is connected to the second input of the fourth DAGF module. The output of the third DAGF module is connected to the first input of the prediction head unit.

4. The multi-directional edge sensing feature fusion network for detecting small infrared targets according to claim 3, characterized in that: The first DAGF module includes a first depthwise separable convolutional layer, a second pixel stacking module, a sixth 1×1 convolutional layer, a second depthwise separable convolutional layer, a gated convolutional layer, an Alpha module, a second pixel dot product module, and a sixth feature concatenation module; The input end of the first depth-separable convolutional layer is connected to the output end of the first 1×1 convolutional layer, and its output end is connected to the first input end of the second pixel overlay module and the input end of the second depth-separable convolutional layer, respectively. The second input terminal of the second pixel overlay module is connected to the output terminal of the first 1×1 convolutional layer, and its output terminal is connected to the input terminal of the sixth 1×1 convolutional layer. The output of the sixth 1×1 convolutional layer is connected to the first input of the second pixel dot product module and the input of the Alpha module, respectively. The output of the second depth separable convolutional layer is connected to the first input of the gated convolutional layer and the first input of the sixth feature stitching module, respectively. The second input of the gated convolutional layer is connected to the output of the second 1×1 convolutional layer, and its output is connected to the second input of the sixth feature splicing module. The output of the Alpha module is connected to the second input of the second pixel dot product module; The output of the second pixel dot product module is connected to the third input of the sixth feature concatenation module; The output of the sixth feature splicing module is connected to the second DAGF module; The second and third DAGF modules have the same structure as the first DAGF module.

5. The multi-directional edge sensing feature fusion network for detecting small infrared targets according to claim 4, characterized in that: The gated convolutional layer includes, in sequence from the input end to the output end, a third pixel stacking module, a first batch of normalization layers, a seventh 1×1 convolutional layer, a first ReLU activation function, an eighth 1×1 convolutional layer, a second batch of normalization layers, a seventh Sigmoid function, a third pixel dot product module, and a ninth 1×1 convolutional layer. The output of the seventh Sigmoid function is connected to the first input of the third pixel multiplication module; The input end of the third pixel overlay module is connected to the output end of the second depth separable convolutional layer and the output end of the second 1×1 convolutional layer, respectively, and its output end is also connected to the second input end of the third pixel dot multiplication module. The output of the ninth 1×1 convolutional layer is connected to the second input of the sixth feature splicing module.

6. The multi-directional edge sensing feature fusion network for detecting small infrared targets according to claim 5, characterized in that: The MSEE module includes a Sobel operator module, a seventh feature concatenation module, a 10th 1×1 convolutional layer, a third batch normalization layer, a second ReLU activation function, a Softmax function, and a fourth pixel dot product module, which are connected sequentially from the input end to the output end. The output of the Softmax function is connected to the first input of the fourth pixel dot product module; The Sobel operator module includes Sobel operators in four directions: 0°, 45°, 90°, and 135°, which are used to extract edge gradient features in four directions from the original infrared image. The output of the Sobel operator module is also connected to the second input of the fourth pixel dot product module; The output of the fourth pixel dot product module is connected to the input of the first 1×1 convolutional layer.

7. The multi-directional edge-aware feature fusion network for detecting small infrared targets according to any one of claims 4-6, characterized in that: The prediction head unit includes a second 3×3 convolutional layer whose input is connected to the output of the fourth decoder, a third 3×3 convolutional layer whose input is connected to the output of the third decoder, a fourth 3×3 convolutional layer whose input is connected to the output of the second decoder, a fifth 3×3 convolutional layer whose input is connected to the output of the first decoder, a second 3×3 convolutional layer whose input is connected to the output of the second 3×3 convolutional layer, a third 3×3 convolutional layer whose input is connected to the output of the third 3×3 convolutional layer, a fourth 3×3 convolutional layer whose input is connected to the output of the fourth 3×3 convolutional layer, and a fifth 3×3 convolutional layer whose input is connected to the output of the first decoder, a second 3×3 convolutional layer whose input is connected to the output of the second 3×3 convolutional layer, a third 3×3 convolutional layer whose input is connected to the output of the third 3×3 convolutional layer, a fourth 3×3 convolutional layer whose input is connected to the output of the third 3×3 convolutional layer, and a fifth 3×3 convolutional layer whose input is connected to the output of the first decoder, a second 3×3 convolutional layer whose input is connected to the output of the second 3×3 convolutional layer, a third 3×3 convolutional layer whose input is connected to the output of the third 3×3 convolutional layer, and a fifth 3×3 convolutional layer whose input is connected to the output of the fourth 3×3 convolutional layer. The output of the 3×3 convolutional layer is connected to the fifth Sigmoid function; the input of the fifth 1×1 convolutional layer is connected to the output of the third DAGF module; the input of the fifth 1×1 convolutional layer is connected to the output of the fifth 1×1 convolutional layer; the input of the concatenated dot product module is connected to the output of the second to fourth Sigmoid functions respectively; the input of the sixth 3×3 convolutional layer is connected to the output of the concatenated dot product module; and the input of the sixth 3×3 convolutional layer is connected to the output of the sixth 3×3 convolutional layer. The outputs of the first to sixth Sigmoid functions are all connected to the loss function calculation unit. The output of the first Sigmoid function is also connected to the concatenation dot product module.

8. The multi-directional edge sensing feature fusion network for detecting small infrared targets according to claim 7, characterized in that: The splicing dot product module includes a first to a fifth splicing module and a first pixel dot product module; The input terminal of the first splicing module is connected to the output terminal of the second Sigmoid function, and its output terminal is connected to the second input terminal of the second splicing module; The first input terminal of the second splicing module is connected to the output terminal of the third Sigmoid function, and its output terminal is connected to the second input terminal of the third splicing module. The first input terminal of the third splicing module is connected to the output terminal of the fourth Sigmoid function, and its output terminal is connected to the second input terminal of the fourth splicing module. The first input terminal of the fourth splicing module is connected to the output terminal of the fifth Sigmoid function, and its output terminal is connected to the first input terminal of the first pixel dot multiplication module and the second input terminal of the fifth splicing module, respectively. The second input of the first pixel multiplication module is connected to the output of the first Sigmoid function, and its output is connected to the first input of the fifth splicing module. The output of the fifth splicing module is connected to the input of the sixth 3×3 convolutional layer.

9. The multi-directional edge sensing feature fusion network for detecting small infrared targets according to claim 8, characterized in that: The initial feature extraction unit includes a dilation and erosion module for inputting the original infrared image, a first pixel overlay module for inputting the original infrared image, and a first 3×3 convolutional layer whose input is connected to the output of the first pixel overlay module. The output end of the expansion corrosion module is connected to the second input end of the first pixel overlay module; The output of the first 3×3 convolutional layer is connected to the input of the first encoder.

10. A detection method for detecting small infrared targets, employing the multi-directional edge perception feature fusion network for detecting small infrared targets as described in any one of claims 1-9, characterized in that, Includes the following steps: Step 1: Construct a multi-directional edge sensing feature fusion network for detecting small infrared targets as described in any one of claims 1-9; Step 2: Obtain multiple known raw infrared images and their corresponding raw mask labels, and preprocess the raw infrared images; the number of raw infrared images is greater than the preset number of iterations; Step 3: Perform steps 4 to 9 on one of the original infrared images and its original mask label through the multi-directional edge-aware feature fusion network, while retaining the corresponding network parameters; Step 4: The initial feature extraction unit performs initial feature extraction processing on the original infrared image to obtain the initial feature map; Step 5: The backbone feature network unit performs feature encoding and decoding on the initial feature map to obtain feature maps f1, f2, f3, and f4; Step 6: The edge feature network unit extracts edge features from the original infrared image based on feature maps f1, f2, and f3 to obtain feature map F1; Step 7: The prediction head unit performs convolution and sigmoid processing on feature maps f1, f2, f3, f4 and feature map F1 to obtain edge prediction result map pe and feature prediction result maps p1, p2, p3, p4. Then, the edge prediction result map pe and the feature prediction result maps p1, p2, p3, p4 are concatenated and multiplied, and then convolution and sigmoid processing are performed again to obtain feature map P. Step 8: The loss function calculation unit calculates the edge loss based on the original infrared image and the edge prediction result map pe, and calculates the main feature loss based on the original mask label, feature prediction result maps p4, p3, p2, p1 and feature map P, to obtain the edge loss and the main loss; Step 9: Backpropagate the edge loss and main loss in the multi-directional edge-aware feature fusion network to optimize the multi-directional edge-aware feature fusion network; Step 10: Replace with another original infrared image and repeat steps 3-9 until the preset number of iterations is reached. Retain the optimal network parameters to obtain the optimal multi-directional edge perception feature fusion network. Step 11: Detect the small target image under test using the optimal multi-directional edge-aware feature fusion network to obtain the target detection map and complete the detection.

11. The detection method for detecting small infrared targets according to claim 10, characterized in that: Step 2 specifically involves acquiring multiple known raw infrared images I input The original infrared image I was processed using Gaussian blur and data augmentation operations, respectively. input Preprocessing is performed to make all raw infrared images I input Consistent in size, and acquisition of each original infrared image I input The corresponding original mask label.

12. The detection method for detecting small infrared targets according to claim 11, characterized in that, Step 4 specifically includes: 4.1 The expansion corrosion module uses the expansion corrosion method to process the original infrared image I. input Morphological enhancement is performed to obtain morphologically enhanced image I. morph ; 4.2 The first pixel overlay module overlays the original infrared image I input and morphologically enhanced image I morph A fused image I is formed by stitching along the channel dimension. fusion ; 4.3, The first 3×3 convolutional layer on the fused image I fusion Perform convolution processing to obtain the initial feature map x initial .

13. The detection method for detecting small infrared targets according to claim 12, characterized in that, Step 5 specifically includes: 5.1 The first encoder extracts the initial feature map x initial The target features are used to obtain feature map e1; 5.2 The second encoder extracts the target features from feature map e1 to obtain feature map e2; 5.3 The third encoder extracts the target features from feature map e2 to obtain feature map e3; 5.4 The fourth encoder extracts the target features from feature map e3 to obtain feature map e4; 5.

5. Extract target features from feature map e4 in the intermediate layer to obtain feature map m; 5.6 The fourth decoder decodes feature map m and feature map e4 to obtain feature map f4; 5.7 The third decoder decodes feature map f4 and feature map e3 to obtain feature map f3; 5.8 The second decoder decodes feature map f3 and feature map e2 to obtain feature map f2; 5.9 The first decoder decodes feature map f2 and feature map e1 to obtain feature map f1.

14. The detection method for detecting small infrared targets according to claim 13, characterized in that, Step 6 specifically includes: 6.1 The MSEE module processes the original infrared image I input Feature extraction is performed to obtain the feature gradient map F; 6.2 The first 1×1 convolutional layer adjusts the feature gradient map F from 1 channel to 64 channels, resulting in the feature map Fe; 6.3 The second to fourth 1×1 convolutional layers adjust the feature maps f3, f2, and f1 from 1 channel to 64 channels, respectively, to obtain feature maps ff3, ff2, and ff1; 6.4 The first DAGF module performs image edge feature enhancement on feature map Fe and feature map ff3 to obtain feature map F3; 6.5 The second DAGF module enhances the image edge features of feature map F3 and feature map ff2 to obtain feature map F2; 6.6 The third DAGF module enhances the image edge features of feature map F2 and feature map ff1 to obtain feature map F1.

15. The detection method for detecting small infrared targets according to claim 14, characterized in that, Step 6.1 Details include: 6.1.1 The Sobel operator module extracts multi-directional edge gradient features from the original infrared image and performs channel-dimensional stitching through the seventh feature stitching module to obtain the gradient response map G; 6.1.2 The tenth 1×1 convolutional layer performs convolution on the gradient response map G to learn the corresponding weight features of each direction of the gradient response map G. Then, it passes through the third batch normalization layer for batch normalization. Then, the second ReLU activation function is used to enhance the nonlinear expression. Finally, the gradient magnitude of each direction is normalized and weighted by the Softmax function to obtain the weight coefficient α of each direction. 6.1.3 The fourth pixel dot product module performs a dot product between the directional weight coefficient α and the corresponding pixel in the gradient response map G, and sums them along the directional dimension to obtain the fused feature gradient map F.

16. The detection method for detecting small infrared targets according to claim 15, characterized in that, Step 6.4 specifically includes: Step 6.4.1: The first depthwise separable convolutional layer processes the feature map Fe to obtain the feature map w1; Step 6.4.2: The second depthwise separable convolutional layer processes the feature map w1 to obtain the feature map w2; Step 6.4.3: The second pixel overlay module overlays feature map Fe and feature map w1 pixel by pixel to obtain feature map △p. Feature map △p is then convolved by the sixth 1×1 convolutional layer and the learnable parameters are adjusted by the Alpha module. Step 6.4.4: The second pixel dot product module multiplies the feature map processed by the sixth 1×1 convolutional layer with the weight map processed by the Alpha module by pixels to obtain the feature map △p1; Step 6.4.5: The gated convolutional layer processes feature map w2 and feature map ff3 to obtain feature map Pg; Step 6.4.6: The sixth pixel overlay module overlays feature map △p1, feature map Pg, and feature map w2 pixel by pixel to obtain feature map F3.

17. The detection method for detecting small infrared targets according to claim 16, characterized in that, Step 7 specifically includes: 7.1 The fifth 1×1 convolutional layer and the first Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map F1 to obtain the edge prediction result map pe of the target; 7.2 The second 3×3 convolutional layer and the second Sigmoid function sequentially perform convolution and Sigmoid processing on feature map f4 to obtain feature prediction result map p4; 7.3 The third 3×3 convolutional layer and the third Sigmoid function sequentially perform convolution and Sigmoid processing on feature map f3 to obtain feature prediction result map p3; 7.4 The fourth 3×3 convolutional layer and the fourth Sigmoid function sequentially perform convolution and Sigmoid processing on feature map f2 to obtain feature prediction result map p2; 7.5 The fifth 3×3 convolutional layer and the fifth Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map f1 to obtain the feature prediction result map p1; 7.6 The concatenation dot product module performs concatenation dot product processing on the edge prediction result map pe and the feature prediction result maps p1, p2, p3, and p4 to obtain the feature map pu; 7.7 The sixth 3×3 convolutional layer and the sixth Sigmoid function sequentially perform convolution and Sigmoid processing on the feature map pu to obtain the feature map P.

18. The detection method for detecting small infrared targets according to claim 17, characterized in that, In step 7.6, the concatenation dot product processing specifically includes: 7.6.1 The first to fourth stitching modules adjust the size of the feature prediction result images p1, p2, p3, and p4 to match the original infrared image I. input The size will then be compared with the original infrared image I. input The feature prediction result images p4, p3, p2, and p1 of the same size are concatenated along the channel dimension to obtain the feature image pf; 7.6.2 The first pixel dot multiplication module multiplies the feature map pf with the edge prediction result map pe by pixels to enhance edge details, resulting in the feature map pz; 7.6.3 The fifth stitching module stitches feature map pz and feature map pf along the channel dimension to obtain feature map pu.

19. The detection method for detecting small infrared targets according to claim 18, characterized in that, Step 8 specifically includes: 8.1 The loss function calculation unit acquires the original infrared image I. input Based on the corresponding edge mask labels, the edge loss of the edge prediction result image pe is calculated, and the edge loss L is obtained. EPI ; 8.2 The loss function calculation unit obtains mask labels of the same size as those in steps 7.2-7.5 for the feature prediction result maps p4, p3, p2, and p1. Based on these mask labels and the original mask labels, it calculates the principal feature loss of the feature prediction result maps p4, p3, p2, and p1, as well as the feature map P, to obtain the principal loss L. SLS ; Step 9 specifically involves setting the edge loss L... EPI and main loss L SLS Backpropagation is performed in the multi-directional edge-aware feature fusion network to optimize it.

Images