Infrared small target detection method based on dual-stream enhancement network

Abstract

The present invention discloses an infrared small target detection method based on a dual-stream enhancement network, which mainly solves the problem of low detection accuracy of existing methods. The implementation steps are: constructing a training sample set and a test sample set; constructing a target detection network model based on the dual-stream enhancement network: an infrared small target detection network O including a first branch and a second branch dual-stream enhancement network arranged in parallel, and a fusion module cascaded with the network, wherein the first branch network extracts feature information of the target from both row and column directions according to low-level features to obtain more refined and complete high-level features, which is helpful to accurately locate the target; the network in the second branch can refine the features of the small infrared target through multi-scale feature fusion, thereby suppressing background information and retaining small target information; by iteratively training the infrared small target detection network model O, the present invention can improve the accuracy of infrared small target detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital image processing and relates to a method for detecting small targets in infrared images, and in particular to an infrared small target detection method based on a dual-stream enhancement network, which can be used for maritime rescue. Background Art

[0002] Infrared small target detection is a technique that uses computer technology to detect targets from infrared images. Infrared small target detection has a wide range of applications, such as traffic management and maritime rescue. However, due to noise, small and blurred targets, infrared small target detection remains challenging. To detect infrared small targets, researchers have proposed several methods based on image processing and machine learning, including filtering-based methods, methods based on the human visual system, and methods based on low-rank representations. However, these traditional methods have some limitations: they cannot suppress complex background noise, resulting in high detection error rates and unstable performance.

[0003] With the success of deep learning in many fields, researchers have applied it to the field of infrared small target detection and achieved effective results. For example, Chongqing University of Posts and Telecommunications proposed a method for infrared small target detection based on a generative adversarial network in its patent application document "Infrared Small Target Detection Method Based on Generative Adversarial Network" (patent application number: 202010077283.5, application publication number: CN111275692A). This method reconstructs a false background image block for the target image block containing the small target, pairs the false background image block with the target image block to obtain first paired data, and uses this data to train the generative adversarial network. Then, two original background image blocks in the image sequence block that do not contain the small target are paired to obtain second paired data, and this data is used to continue training to obtain a target generative adversarial network model. The image block sequence to be detected is input into the target generative adversarial network model to obtain a detection sequence result. The detection sequence result is spliced ​​according to the size of the original infrared image to be detected, and the spliced ​​image is compared with the original infrared image to be detected. The residual of the residual image is filtered to obtain the detection result. This invention has the advantage of fast computing speed, but the features extracted by the network are not detailed and complete enough, and lack attention to the edges of the target, and the ability to suppress noise is limited, resulting in low detection accuracy. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides an infrared small target detection method based on a dual-stream enhancement network, which is used to solve the problem of low detection accuracy of the existing infrared small target detection method based on deep learning.

[0005] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0006] (1) Obtain training sample set and test sample set:

[0007] Obtain K infrared images and mark the targets in each infrared image. Then, combine M infrared images and their corresponding labels into a training sample set R1, and combine the remaining KM infrared images and their corresponding labels into a test sample set E1, where K≥1000.

[0008] (2) Construct an infrared small target detection model O based on a dual-stream enhancement network:

[0009] Construct an infrared small target detection network O including a parallel arrangement of a first branch and a second branch dual-stream enhancement network, and a cascade fusion module with the network, wherein:

[0010] The first branch includes a sequentially connected Stem block, multiple cascaded residual blocks, a first transposed convolutional layer, a first spatial frequency attention module, a second transposed convolutional layer, a second spatial frequency attention module, and a head network; wherein the Stem block includes multiple convolutional layers and cascaded pooling layers; the residual block includes a sequentially cascaded convolutional layer, a nonlinear activation layer, and a convolutional layer; the spatial frequency attention module includes a sequentially cascaded contextual space attention module, a decomposition frequency attention module, and a convolutional layer; and the head network includes multiple convolutional layers;

[0011] The second branch includes an eyeball enhancement module and a head network connected in sequence; wherein the eyeball enhancement module includes a plurality of cascaded convolutional layers, an upsampling block, and a Gaussian convolutional layer connected in sequence, and the input end of the eyeball enhancement module is further connected to the output ends of the first spatial frequency attention module and the second spatial frequency attention module of the first branch;

[0012] (3) Iteratively train the infrared small target detection model O:

[0013] (3a) The number of initial iterations is s, the maximum number of iterations is S, S ≥ 20000, and the infrared small target detection model of the sth iteration is O s , O s The learnable weights and bias parameters are w s 、b s , and let s = 0, O s =O;

[0014] (3b) l training samples randomly selected with replacement from the training sample set R1 are used as the input of the infrared small target detection model O. The first branch reconstructs the infrared target for each training sample to obtain l small targets; at the same time, the second branch enhances the infrared small target details for each training sample to obtain l small target details; the fusion module supplements and refines the corresponding small target reconstruction results based on the result of each small target detail enhancement to obtain the infrared small target detection results of l training samples, where l ≥ 1000;

[0015] (3c) Using the Dice loss function L Dice With the cross entropy loss function L CE , and calculate O by generating l target detection results and their corresponding l labels s The loss value L DC , and then calculate L by the chain rule DC The weight parameter ω in the network s and bias parameter b s Partial derivative of and Then according to Right s 、b s Update and get the network model O of this iteration s ;

[0016] (3d) Determine whether s≥S holds true. If so, obtain the trained infrared small target detection model O*. Otherwise, set s=s+1 and execute step (3b);

[0017] (4) Obtain infrared image target detection results:

[0018] The test sample set E1 is used as the input of the trained infrared small target detection model O* for forward propagation to obtain the infrared small target detection results corresponding to all test samples.

[0019] Compared with the prior art, the present invention has the following advantages:

[0020] The dual-stream enhancement network model constructed by the present invention includes a first branch and a second branch arranged in parallel. During the process of training the model and obtaining infrared small target detection results, the spatial frequency attention module in the first branch can select the most useful features by fusing low- and high-level features. Based on the low-level features, the feature information of the target is extracted from both the row and column directions to obtain more refined and complete high-level features, which helps to accurately locate the target. The eyeball-shaped enhancement module in the second branch can refine the features of small infrared targets through multi-scale feature fusion, thereby suppressing background information and retaining small target information. Experimental results show that the present invention can effectively improve the accuracy of infrared small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Flowchart for the implementation of the present invention;

[0022] Figure 2 This is a schematic diagram of the structure of the dual-stream enhanced network model constructed by the present invention;

[0023] Figure 3 For the present invention Figure 2 A schematic diagram of the structure of the specific embodiment;

[0024] Figure 4 Schematic diagram of the structure of the spatial frequency attention module according to an embodiment of the present invention;

[0025] Figure 5 Schematic diagram of the structure of the eyeball-shaped enhancement module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Reference Figure 1 , the present invention comprises the following steps:

[0028] Step 1) Obtain training sample set and test sample set:

[0029] Obtain K infrared images, then mark the targets in each infrared image, and then form the training sample set R1 with M infrared images and their corresponding labels, and form the test sample set E1 with the remaining KM infrared images and their corresponding labels, where K≥1000.

[0030] Step 2) Construct an infrared detection network model O based on a dual-stream enhancement network:

[0031] An infrared small target detection network O is constructed, which includes a first branch and a second branch dual-stream network arranged in parallel, and a cascade fusion module with the network, wherein the first branch includes a Stem block, a first residual block, a second residual block, a third residual block, a first transposed convolution layer, a first spatial frequency attention module, a second transposed convolution layer, a second spatial frequency attention module, and a head network connected in sequence; the second branch includes an eyeball-shaped enhancement module and a head network connected in sequence; wherein the Stem block includes a pooling layer and multiple convolution layers; the first residual block includes a nonlinear activation layer and multiple convolution layers; the second residual block includes a nonlinear activation layer and multiple convolution layers; the third residual block includes a nonlinear activation layer and multiple convolution layers; the spatial frequency attention module includes a context space attention module, a decomposition frequency attention module and a convolution layer; the head network includes multiple convolution layers; the eyeball-shaped enhancement module includes multiple convolution layers, an upsampling block and a Gaussian convolution layer; and the fusion module realizes the function of element-by-element addition.

[0032] The first branch, in which the number of convolutional layers is 34 and the number of residual blocks is 3: the Stem block includes the first convolutional layer, the second convolutional layer, the third convolutional layer, and the maximum pooling layer connected in sequence; the first residual block includes the fourth convolutional layer, the first non-linear activation layer, and the fifth convolutional layer connected in sequence; the second residual block includes the sixth convolutional layer, the second non-linear activation layer, and the seventh convolutional layer connected in sequence; the third residual block includes the eighth convolutional layer, the third non-linear activation layer, and the ninth convolutional layer connected in sequence; the spatial frequency attention module includes the contextual space attention module, the decomposition frequency attention module and the convolutional layer connected in sequence, wherein the contextual space attention module includes the first bottleneck structure and the second bottleneck structure arranged in parallel and the fourth non-linear activation layer, the third bottleneck structure, and the fifth non-linear activation layer connected thereto, wherein the bottleneck structure includes multiple convolutional layers, the head network includes the third branch, the fourth branch, and the fifth branch arranged in parallel, and the third branch includes the tenth convolutional layer, the eleventh convolutional layer connected in sequence. Convolution layer, the fourth branch includes the twelfth convolution layer and the thirteenth convolution layer connected in sequence, and the fifth branch includes the fourteenth convolution layer and the fifteenth convolution layer connected in sequence. The specific parameters are: the convolution kernel size of the first convolution layer, the fourth convolution layer, the fifth convolution layer, the sixth convolution layer, the seventh convolution layer, the eighth convolution layer, and the ninth convolution layer is 3*3 with a step size of 2, the convolution kernel size of the second convolution layer, the third convolution layer, and the twelfth convolution layer is 3*3 with a step size of 1, and the convolution kernel size of the tenth convolution layer is 1*1. The eleventh, thirteenth, and fifteenth convolutional layers are all atrous convolutions, with convolution kernel sizes of 3*3 and expansion rates of 1, 3, and 5, respectively. The convolution kernel size of the fourteenth convolutional layer is 5*5, and the convolution kernel size of the first and second transposed convolutional layers is 3*3 with a step size of 2. The first, second, and third nonlinear activation layers are implemented using the ReLU function, and the fourth and fifth nonlinear activation layers are implemented using the Sigmoid function.

[0033] Contextual Spatial Attention Module (CSA): Since background features are helpful in reconstructing the information of small targets, it is very necessary to aggregate pixel information into background features. The CSA module is responsible for aggregating pixel-level features and forming contextual features by fusing low-level and high-level features. Specifically, the feature fusion process is carried out under the guidance of low-level features, because low-level features are rich in the characteristics of small targets. This paper introduces an attention mechanism to describe the correlation between foreground and background pixels. By using low-level features and related descriptions, the characteristics of infrared targets can be better described for subsequent detection and segmentation. The process can be described as follows:

[0034] a c =Cat([a low ,a high ]) (1)

[0035] a ca1 =Conv(ReLU(Conv(GAP((a c ))))) (2)

[0036] a ca2 =Conv(ReLU(Conv(GMP((a c ))))) (3)

[0037] a ca =Sigmoid(a ca1 +a ca2 )·a c (4)

[0038] a att =Sigmoid(Conv(ReLU(Conv((a ca ))))) (5)

[0039] where a c represents the connection between high-level and low-level features, a ca1 represents the channel attention feature calculated after global average pooling, a ca2 represents the channel attention feature calculated after global maximum pooling, a ca Indicates the attention characteristics after the channel is followed, a att Represents a spatial attention map to highlight the correlation between low-level features and high-level features.

[0040] Decomposition Frequency Attention Module (DFA): The DFA module is responsible for generating frequency features by taking low-level features and fused features from CSA as input. Frequency features are crucial for the detection of infrared small targets because small targets always have clear outlines, which produce high-frequency components in the frequency features. Unfortunately, noise in the input may also produce high-frequency responses in the frequency domain. In order to distinguish small targets with noise, the present invention proposes DFA to highlight useful target features. Specifically, the input features are decomposed into low-frequency and high-frequency parts in the frequency domain through DCT transformation; then, frequency attention is generated by selecting the top k frequency components in the high-frequency and low-frequency parts and applying separate convolution blocks on them; finally, the frequency attention map is used to modulate the high-frequency and low-frequency parts to obtain the final attention features. The 2D DCT transform can be described as:

[0041]

[0042] Among them, h∈{0,1,···,H-1}, w∈{0,1,···,W-1}, x 2d ∈R H×W is the input feature, f 2d ∈RH×W is the spectrum after 2D DCT transformation, then the above process can be described as:

[0043] a in =Cat(a low ,a s ) (7)

[0044]

[0045]

[0046] Among them, the input a of DFA in It means that it is divided into n parts along the channel dimension direction, which is expressed as [u',v'] and [u",v"] represent the two-dimensional indices of the low-frequency and high-frequency components, respectively. and Respectively represent the DCT transformed dimensional vector, and then concatenate the 2D transformation results to obtain low-frequency and high-frequency features:

[0047]

[0048]

[0049] Among them, a lf ∈R C , a hf ∈R C , then, the decomposed frequency attention can be defined as:

[0050] DF lf =Sigmoid(Conv(ReLU(Conv(a lf )))) (12)

[0051] DF hf =Sigmoid(Conv(ReLU(Conv(a hf )))) (13)

[0052] a f =Conv(Cat([DF lf ,DF hf ])) (14)

[0053] Among them, DF lf and DF hf Refers to the attention maps of low-frequency and high-frequency features, respectively. Experimental results show that the DFA module can preserve more object details with the help of attention on different channels.

[0054] The second branch has 8 convolutional layers; the eyeball enhancement module includes the sixteenth convolutional layer, the seventeenth convolutional layer, the upsampling block and the Gaussian convolutional layer connected in sequence; the head network includes the sixth branch, the seventh branch and the eighth branch arranged in parallel, the sixth branch includes the eighteenth convolutional layer and the nineteenth convolutional layer connected in sequence, the seventh branch includes the twentieth convolutional layer and the twenty-first convolutional layer connected in sequence, and the eighth branch includes the twenty-second convolutional layer and the twenty-third convolutional layer connected in sequence. The specific parameters are: the convolution kernel size of the sixteenth convolutional layer, the seventeenth convolutional layer and the twentieth convolutional layer is 3*3, the convolution kernel size of the eighteenth convolutional layer is 1*1, the nineteenth convolutional layer, the twenty-first convolutional layer and the twenty-third convolutional layer are all hollow convolutions, the convolution kernel size is 3*3, the expansion rates are 1, 3 and 5 respectively, and the convolution kernel size of the twenty-second convolutional layer is 5*5;

[0055] Eyeball Enhancement Module (EEM): EEM aims to improve detection and segmentation performance by refining the features of the target from a new perspective. During the training process, small infrared targets are easily ignored due to downsampling. The present invention retains small target information by using EEM to suppress background information as much as possible; EEM attempts to refine the coarse representation of small infrared targets from CMD through multi-scale feature fusion, where low-resolution features are gradually encoded into high-resolution features. However, due to the small size of the target, the upsampling process can easily lead to aliasing and overlap of the background and the target. To solve this problem, convolution is used as a filter after each upsampling to eliminate the aliasing effect. The process can be described as:

[0056] x up =Upsample(Conv(x1))+x2 (15)

[0057] x s =Conv(Conv(Upsample((Conv(x up )))) (16)

[0058] Among them, x1 and x2 represent the features generated by CMD, x up represents the output of multi-scale feature fusion, x s Represents the high-resolution feature map after super-resolution reconstruction, and then, through the Gaussian convolution kernel and the residual x res x s Decomposed into edge representation x edge , the present invention uses edge representation x edge and the residual represents x res To fully capture the useful information in the original input image, and further add convolution to filter low-frequency noise. The process can be described as:

[0059] x edge=Upsample((Conv(GSC(x s ))) (17)

[0060] x res =Upsample(Conv(x s ))-x edge (18)

[0061] Where GSC represents the function of Gaussian convolution kernel. Then the enhanced output x is obtained e :

[0062] x e =x edge x+x res ·x (19)

[0063] Where x is the original input infrared image.

[0064] Step 3) Iteratively train the infrared small target detection network model O:

[0065] (3a) The number of initial iterations is s, the maximum number of iterations is S, S ≥ 20000, and the target detection network model of the sth iteration is O s , and let s = 0, O s =O;

[0066] (3b) l training samples are randomly selected with replacement from the training sample set R1 as the input of the infrared small target detection model O:

[0067] (3b1) Input l infrared image samples into the first branch, the Stem block performs layer-by-layer feature extraction on each infrared image to obtain a feature map x1, the first residual block extracts local features x2 from the feature map x1, the second residual block refines the local features x2, the third residual block further refines the refined features x3 to obtain a feature map x4, the first transposed convolutional layer upsamples the feature map x4 to obtain a global feature x5, the first spatial frequency attention module extracts l high-level feature maps x6 from the global features x5, the second transposed convolutional layer upsamples the feature map x6 to obtain a feature map x7, the second spatial frequency attention module extracts l high-level features x8 from the feature map x7, and the head network normalizes the feature map x8 to obtain a normalized feature map x9;

[0068] (3b2) At the same time, the first and second spatial frequency attention module feature maps x6 and x8 of the l infrared image samples are used as the input of the eyeball enhancement module in the second branch to obtain the refined feature map x 10 , the head network transforms the feature map x 10 After normalization, we get l normalized feature maps x 11 ;

[0069] (3b3) The feature map x8 obtained by the first branch is combined with the feature map x obtained by the second branch through the fusion module 11 Perform element-by-element addition to achieve the result of the second branch to supplement and refine the result of the first branch, and obtain the target detection results of l training samples.

[0070] (3c) Update the parameters of the infrared model based on the dual-stream enhancement network to obtain the network model of this iteration;

[0071] (3d) Determine whether s≥S holds true. If so, obtain the trained target detection network model O*. Otherwise, set s=s+1 and execute step (3b);

[0072] Step 4) Obtain infrared image target detection results:

[0073] The test sample set E1 is used as the input of the trained target detection network model O* for forward propagation to obtain the infrared small target detection results corresponding to all test samples.

[0074] The technical effect of the present invention can be further illustrated by the following simulation experiments:

[0075] Simulation conditions and contents:

[0076] The simulation experiments were conducted on an Intel(R) Core i9-9900K CPU with a 3.6GHz clock speed, 32GB of RAM, and an NVIDIA GeForce RTX 2080Ti graphics card. The simulation software was run on the Ubuntu 16.04 operating system, using Python 3.7 and PyTorch 1.7.1. Experiments were conducted on the IRSTD-1k and NUAA-SIRST datasets. NUAA-SIRST contains 427 infrared images, while IRSTD-1k contains 1000 infrared images. Each dataset was split into training and test sets in a 6:4 ratio.

[0077] The intersection over union (IoU) of the prior art on the IRSTD-1k dataset is 76.2%, and the intersection over union (IoU) of the present invention on the IRSTD-1k dataset is 77.7%. The results are listed in Table 1. The intersection over union (IoU) of the prior art on the NUAA-SIRST dataset is 75.6%, and the intersection over union (IoU) of the present invention on the NUAA-SIRST dataset is 77.2%. The results are listed in Table 2. Compared with the prior art, the detection accuracy is significantly improved.

[0078] Table 1

[0079] method Existing technology The present invention Intersection and Union 76.2％ 77.7％

[0080] Table 2

[0081] method Existing technology The present invention Intersection and Union 75.6％ 77.2％

Claims

1. A method for detecting small infrared targets based on a dual-stream enhancement network, characterized in that: The following steps are involved: (1) Obtain training sample set and test sample set: Obtain K infrared images and mark the targets in each infrared image. Then, combine M infrared images and their corresponding labels into a training sample set R1, and combine the remaining KM infrared images and their corresponding labels into a test sample set E1, where K≥1000. (2) Construct an infrared small target detection model O based on a dual-stream enhancement network: Construct an infrared small target detection network O including a parallel arrangement of a first branch and a second branch dual-stream enhancement network, and a cascade fusion module with the network, wherein: The first branch includes a sequentially connected Stem block, multiple cascaded residual blocks, a first transposed convolutional layer, a first spatial frequency attention module, a second transposed convolutional layer, a second spatial frequency attention module, and a head network; wherein the Stem block includes multiple convolutional layers and cascaded pooling layers; the residual block includes a sequentially cascaded convolutional layer, a nonlinear activation layer, and a convolutional layer; the spatial frequency attention module includes a sequentially cascaded contextual space attention module, a decomposition frequency attention module, and a convolutional layer; and the head network includes multiple convolutional layers; The second branch includes an eyeball enhancement module and a head network connected in sequence; wherein the eyeball enhancement module includes a plurality of cascaded convolutional layers, an upsampling block, and a Gaussian convolutional layer connected in sequence, and the input end of the eyeball enhancement module is further connected to the output ends of the first spatial frequency attention module and the second spatial frequency attention module of the first branch; (3) Iteratively train the infrared small target detection model O: (3a) The number of initial iterations is s, the maximum number of iterations is S, S ≥ 20000, and the infrared small target detection model of the sth iteration is O s , O s The learnable weights and bias parameters are w s 、b s , and let s = 0, O s =O; (3b) l training samples randomly selected with replacement from the training sample set R1 are used as the input of the infrared small target detection model O. The first branch reconstructs the infrared target for each training sample to obtain l small targets; at the same time, the second branch enhances the infrared small target details for each training sample to obtain l small target details; the fusion module supplements and refines the corresponding small target reconstruction results based on the result of each small target detail enhancement to obtain the infrared small target detection results of l training samples, where l ≥ 1000; (3c) Using the Dice loss function L Dice With the cross entropy loss function L CE , and calculate O by generating l target detection results and their corresponding l labels s The loss value L DC , and then calculate L by the chain rule DC The weight parameter ω in the network s and bias parameter b s Partial derivative of and Then according to Right s 、b s Update and get the network model O of this iteration s ; (3d) Determine whether s≥S holds true. If so, obtain the trained infrared small target detection model O*. Otherwise, set s=s+1 and execute step (3b); (4) Obtain infrared image target detection results: The test sample set E1 is used as the input of the trained infrared small target detection model O* for forward propagation to obtain the infrared small target detection results corresponding to all test samples.

2. The infrared small target detection method based on the dual-stream enhancement network according to claim 1 is characterized in that: The dual-stream enhanced network model O described in step (2), wherein: The first branch, in which the number of convolutional layers is 34 and the number of residual blocks is 3; the specific structure of the Stem block includes the first convolutional layer, the second convolutional layer, the third convolutional layer, and the maximum pooling layer connected in sequence; the specific structure of the first residual block includes the fourth convolutional layer, the first non-linear activation layer, and the fifth convolutional layer connected in sequence; the specific structure of the second residual block includes the sixth convolutional layer, the second non-linear activation layer, and the seventh convolutional layer connected in sequence; the specific structure of the third residual block includes the eighth convolutional layer, the third non-linear activation layer, and the ninth convolutional layer connected in sequence; the specific structure of the spatial frequency attention module includes the contextual spatial attention module, the decomposition frequency attention module and the convolution layer connected in sequence, wherein the contextual spatial attention module includes the first bottleneck structure and the second bottleneck structure arranged in parallel and the fourth non-linear activation layer, the third bottleneck structure, and the fifth non-linear activation layer connected thereto, wherein the bottleneck structure includes multiple convolutional layers, the head network includes the third branch, the fourth branch, and the fifth branch arranged in parallel, and the third branch includes the sequentially connected The fourth branch includes the twelfth and thirteenth convolutional layers connected in sequence, and the fifth branch includes the fourteenth and fifteenth convolutional layers connected in sequence. The specific parameters are: the convolution kernel size of the first, fourth, fifth, sixth, seventh, eighth and ninth convolutional layers is 3*3 with a step size of 2; the convolution kernel size of the second, third and twelfth convolutional layers is 3*3 with a step size of 1; the convolution kernel size of the tenth convolutional layer is 3*3 with a step size of 1. The smallest convolution layer is 1*1. The eleventh, thirteenth, and fifteenth convolution layers are all atrous convolutions. The convolution kernel size is 3*3, and the expansion rates are 1, 3, and 5 respectively. The convolution kernel size of the fourteenth convolution layer is 5*5. The convolution kernel size of the first and second transposed convolution layers is 3*3, with a step size of 2. The first, second, and third nonlinear activation layers are implemented using the ReLU function, and the fourth and fifth nonlinear activation layers are implemented using the Sigmoid function. The second branch has 8 convolutional layers. The specific structure of the eyeball enhancement module includes the sixteenth convolutional layer, the seventeenth convolutional layer, the upsampling block and the Gaussian convolutional layer connected in sequence. The specific structure of the head network includes the sixth branch, the seventh branch and the eighth branch arranged in parallel. The sixth branch includes the eighteenth convolutional layer and the nineteenth convolutional layer connected in sequence. The seventh branch includes the twentieth convolutional layer and the twenty-first convolutional layer connected in sequence. The eighth branch includes the twenty-second convolutional layer and the twenty-third convolutional layer connected in sequence. The specific parameters are: the convolution kernel size of the sixteenth convolutional layer, the seventeenth convolutional layer and the twentieth convolutional layer is 3*3, the convolution kernel size of the eighteenth convolutional layer is 1*1, the nineteenth convolutional layer, the twenty-first convolutional layer and the twenty-third convolutional layer are all hollow convolutions, the convolution kernel size is 3*3, the expansion rates are 1, 3 and 5 respectively, and the convolution kernel size of the twenty-second convolutional layer is 5*5.

3. The infrared small target detection method based on dual-stream enhancement network according to claim 1 is characterized in that: The specific implementation steps of the infrared small target detection results of the l training samples described in step (3b) are as follows: (3b1) Input l infrared image samples into the first branch. The Stem block in the first branch performs layer-by-layer feature extraction on each infrared image to obtain a feature map x1. The first residual block extracts local features x2 from the feature map x1. The second residual block refines the local features x2. The third residual block further refines the refined features x3 to obtain a feature map x4. The first transposed convolutional layer upsamples the feature map x4 to obtain a global feature x5. The first spatial frequency attention module extracts l high-level feature maps x6 from the global features x5. The second transposed convolutional layer upsamples the feature map x6 to obtain a feature map x7. The second spatial frequency attention module extracts l high-level features x8 from the feature map x7. The head network normalizes the feature map x8 to obtain a normalized feature map x9. (3b2) At the same time, the feature maps x6 and x8 of the first and second spatial frequency attention modules of l infrared image samples are used as the input of the eyeball enhancement module in the second branch, and each feature x6 and each feature x8 are used to refine each infrared image sample, respectively, to obtain the refined feature map x 10 , the head network transforms the feature map x 10 After normalization, we get l normalized feature maps x 11 ; (3b3) The feature map x8 obtained by the first branch is combined with the feature map x obtained by the second branch through the fusion module 11 Perform element-by-element addition to achieve the result of the second branch to supplement and refine the result of the first branch, and obtain the target detection results of l training samples.

4. The infrared small target detection method based on dual-stream enhancement network according to claim 1 is characterized in that: The calculated loss value L described in step (3c) DC The expression of Right s 、b s The update formulas for updating are: L DC =λ Dice L Dice +λ CE L CE Among them, p m is the predicted probability of the pixel value belonging to the label, r m represents the true value of each pixel in the input image, λ Dice and λ CE is the weight coefficient, w s 、b s Indicates O s The weights and bias parameters of all learnable parameters, w s '、b s ' represents the updated learnable parameters, l r Represents the learning rate.

Images