Infrared small target detection method of shape sensing network based on background and target enhancement

Abstract

The invention provides a small target detection method of a shape sensing network based on background and target enhancement, and relates to the technical field of target detection and computer vision. The method comprises the following steps: 1, obtaining an infrared small target detection data set; 2, constructing a background and target enhancement module (BTAB) in an encoder feature extraction process, respectively enhancing target and background features, and further fusing the features through a channel cross attention mechanism; 3, after an encoder, constructing a multi-scale sensing module (SSB) to further process the features, and extracting the multi-scale features to enhance the sensing capability of targets of different sizes; 4, constructing a shape perception and detail recovery module (SPDR) at the decoder part, and enhancing the edge and details of the target in combination with large kernel convolution and central difference convolution; and 5, fusing the features extracted from the encoder and the decoder by adopting a multi-scale depth supervision optimization strategy to generate a final detection result. The method improves the accuracy of infrared small target detection, especially shows high robustness in low contrast, low signal-to-noise ratio and complex background scenes, has lower false alarm rate and missing report rate, and is widely applied to the fields of traffic monitoring, sea rescue, target early warning and the like.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and relates to an improvement of an infrared small target detection algorithm based on UNet. Background Art

[0002] Infrared small target detection has been widely used in traffic monitoring, maritime rescue and target early warning. Since infrared small targets have low contrast, strong background interference, and are usually small and motion blurred, extracting small target information in complex environments (such as lighting changes, clouds, smoke, etc.) has become a difficult problem to be solved.

[0003] People at home and abroad have studied infrared small target detection. According to different methods, infrared small target detection methods can be divided into two categories: traditional prior methods and deep learning-based methods. The former includes filtering-based methods, human vision-based methods, and low-rank sparse matrix recovery-based methods. These methods usually require less computing resources, but rely heavily on complex manual feature design and have poor robustness under complex backgrounds. The latter mostly adopts symmetrical encoder and decoder structures, among which the most common network structure is U-Net. The encoder and decoder play the role of extracting multi-scale features of the image and reconstructing detail information respectively. However, based on this type of method, the features of the scale of infrared small targets are less processed, so it is easy to lead to deficiencies in the reconstruction of detail information. In order to solve these problems, researchers have tried to introduce attention mechanisms, multi-scale feature fusion and other technologies, but the existing methods still need to be improved in terms of the accuracy of feature fusion, the comprehensiveness of contextual information, and the retention of edge details. Summary of the invention

[0004] In view of this, the present invention provides an image feature extraction and enhancement method that can effectively distinguish background and target features, make full use of multi-scale contextual information and enhance edge details, so as to improve the performance of image processing tasks.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] Based on the above problems, the present invention provides a small target detection method based on a shape perception network with background and target enhancement, and the specific steps are as follows:

[0007] Step 1: Obtain infrared small target detection dataset;

[0008] Step 2: Construct a background and target enhancement module (BTAB) during the encoder feature extraction process to enhance the target and background features respectively, and further fuse the features through the channel cross attention mechanism;

[0009] Step 3: After the encoder, a multi-scale perception module (SSB) is constructed to further process the features and extract multi-scale features to enhance the perception of objects of different sizes;

[0010] Step 4: Construct a shape perception and detail recovery module (SPDR) in the decoder part, combining large kernel convolution and center difference convolution to enhance the edge and details of the target;

[0011] Step 5: Use a multi-scale deep supervision optimization strategy to fuse the features extracted from the encoder and decoder to generate the final detection result.

[0012] Preferably, in step 1, the infrared small target detection dataset is collected from publicly available infrared small target detection datasets NUAA-SIRST (127), NUDT-SIRST (1327) and IRSTD-1K (1001), specifically, 213, 663 and 800 images in the above datasets are selected respectively as training sets for the infrared small target detection network;

[0013] Preferably, the image of the infrared small target training set is read, the image size is adjusted, and the pixel value of the image is preprocessed to obtain the input image M 0 (1, 1, 256, 256).

[0014] Preferably, in step 2, the background and target enhancement module (BTAB) described in the present application first uses a residual block to extract the initial feature M 1 (1, 32, 256, 256). Among them, background feature enhancement obtains background features through global average pooling global_pool, and uses the background attention module based on the dilated convolution group (r = 2, 4, 2) to generate the background attention weight S 1 (1, 32, 1, 1), the background features are enhanced to obtain M 12 (1, 32, 256, 256); The target feature enhancement uses the local convolution module of the full-dimensional dynamic convolution kernel ODConv to extract the target features. Among them, each ODConv module uses 3 different convolution kernels to process the input features and generates the target attention weight S through the target attention module 2 (1, 32, 256, 256), enhance the target features and get M 22 After the background enhancement features and the target enhancement features are concatenated, they are fused through the fusion module to obtain the feature representation M that comprehensively considers the background and target information. 3 (1, 32, 256, 256).

[0015] Preferably, in step three, the multi-scale perception module (SSB) described in the present application further processes the enhanced features. Among them, the multi-scale convolution feature extraction uses dilated convolution branches with different dilation rates to extract multi-scale features and capture contextual information of different scales. Global feature fusion obtains global features through global average pooling and global convolution modules, upsamples them to the input size, and fuses them with multi-scale features. Finally, an adaptive weighting parameter is introduced to perform weighted addition of the fused features and the input features to achieve adaptive fusion of features.

[0016] Preferably, in step 4, the shape perception and detail recovery module (SPDR) described in the present application combines large kernel convolution and center difference convolution after the feature fusion extracted in the encoder and decoder, effectively enhancing the edge details of the image. Among them, large kernel convolution is used to extract large-scale features of the image to capture more extensive contextual information; edge detail features are extracted through center difference convolution and added to large-scale features to enhance the edge details of the image.

[0017] Preferably, in step 5, the loss function is trained using the binary cross entropy BCE loss function, optimized using the Adam optimizer, with an initial learning rate of 0.001, and the cosine annealing strategy is used to gradually reduce the learning rate to 1×10 -5 .

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The present invention provides an infrared small target detection method based on a shape perception network with background and target enhancement, which is intended to effectively distinguish background and target features, make full use of multi-scale context information and enhance edge details. BTAB can effectively distinguish background and target features, respectively perform enhancement processing, improve the distinguishability of features, and contribute to subsequent image segmentation, target detection and other tasks; SSB uses multi-scale convolution and global feature fusion to make full use of the context information of the image, enhance the expressiveness of features, and improve the accuracy of image processing tasks; after the feature fusion extracted by SPDR in the encoder and decoder, large kernel convolution and center difference convolution are combined to effectively enhance the edge details of the image, reduce the loss of edge information, and improve the clarity and detail expression of the image. Each module of the present invention can be flexibly combined and applied to different image processing tasks, with good versatility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0021] Figure 1 This is the overall flow chart of this application;

[0022] Figure 2 A schematic diagram of the overall model provided by the present invention;

[0023] Figure 3 A schematic diagram of the structure of the background and target enhancement module (BTAB) provided by the present invention;

[0024] Figure 4 A schematic diagram of the structure of a multi-scale perception module (SSB) provided by the present invention;

[0025] Figure 5 A schematic diagram of the structure of the shape perception and detail recovery module (SPDR) provided by the present invention; DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0029] The embodiments of the present invention are described in detail below with reference to the accompanying drawings and examples.

[0030] Step 1: Get the infrared small target detection dataset:

[0031] The public infrared small target detection datasets NUAA-SIRST (127), NUDT-SIRST (1327) and IRSTD-1K (1001) are selected. Specifically, 213, 663 and 800 images in the above datasets are selected as the training sets of the infrared small target detection network respectively;

[0032] Step 2: Construct a background and target enhancement module (BTAB) during the encoder feature extraction process to enhance the target and background features respectively, and further fuse the features through the channel cross attention mechanism;

[0033] Step 2.1: Read the image of the infrared small target training set, adjust the image size, and preprocess the pixel values ​​of the image to obtain the input image M 0 (1, 1, 256, 256).

[0034] Step 2.2: Input image M 0 It is sent to the first background and target enhancement module and is used to extract the initial feature M through a residual block. 1 (1, 32, 256, 256). As shown in formula (1):

[0035] M 1 =relu(BN2d(Conv2d(relu(BN2d(Conv2d(M 0 ))))))+M 0 ) (1)

[0036] Among them, relu is the activation function, that is, after two convolution Conv2d, two batch normalization BN2d and two relu activation function processing, and with the input image M 0 The initial feature M obtained after addition 1 .

[0037] Step 2.3: Extract the initial features M 1 Send it to the background feature enhancement branch and obtain the background feature M through global average pooling gloal_pool 11 .

[0038] Step 2.4: The background feature M 11 It is sent to the dilated convolution group (r=2, 4, 2) with a convolution kernel size of 3×3 to generate the background attention weight S 1 (1, 32, 1, 1).

[0039] Step 2.5: Perform M 1 ×S 1 Operation, the background features are enhanced to obtain the enhanced background features M 12 (1, 32, 256, 256). As shown in formula (2):

[0040] M 12 =Sigmoid(DCG(AvgPool(M 1 ))) (2)

[0041] Among them, Sigmoid is the activation function, AvgPool is the global average pooling, and DCG is the dilated convolution group.

[0042] Step 2.6: Extract the initial features M 1 It is sent to the target feature enhancement branch and the target feature M is extracted through the local convolution module of the full-dimensional dynamic convolution kernel ODConv 2 Among them, each ODConv module uses 3 different convolution kernels to process the input features.

[0043] Step 2.7: Feed the target features into the depth-wise separable convolution and point-wise convolution with a kernel size of 3×3 to generate the target attention weight S 2 (1, 32, 256, 256).

[0044] Step 2.8: Then proceed to M 1 ×S 2 Operation, enhance the target feature, and obtain the enhanced target feature M 21 (1, 32, 256, 256). As shown in formula (3):

[0045] M 21 =Sigmoid(PW(DSC(ODConv(M 1 ))))×M 1 (3)

[0046] Among them, PW is group point convolution, DSC is depth-wise separable convolution, and ODConv refers to full-dimensional dynamic convolution.

[0047] Step 2.9: The enhanced background feature M 12 and the enhanced target feature M 21 The splicing is performed and the splicing results are fused through point convolution to obtain the feature representation M that comprehensively considers the background and target information. 3 (1, 32, 256, 256).

[0048] Step 2.10: M 3 After passing through a pooling layer, it is sent to the second background and target enhancement module, and steps 2.2 to 2.9 are repeated to obtain the feature representation M of the background and target information. 4 (1, 64, 128, 128).

[0049] Step 2.11: Repeat steps 2.2 to 2.9, and send them to the third and fourth background and target enhancement modules in turn to obtain the feature representation M of the background and target information in turn. 5 (1, 128, 64, 64), M 6 (1, 256, 32, 32).

[0050] Step 3: After the encoder, a multi-scale perception module (SSB) is constructed to further process the features and extract multi-scale features to enhance the perception of objects of different sizes;

[0051] Step 3.1: The feature M obtained by the encoder 6 For further processing, it is sent to a residual block through a pooling layer to obtain the feature M 7 (1, 256, 16, 16).

[0052] Step 3.2: Feature M 7 Send it to the global average pooling gloal_pool and 1×1 convolution module to obtain the global feature M 71 (1, 256, 1, 1), and then perform bilinear interpolation upsampling to obtain feature M 72 (1, 256, 16, 16).

[0053] Step 3.3: Set feature M 7 Send it to the dilated convolution with a dilation rate of 1, padding of 1, and a convolution kernel size of 3 to obtain the scale feature M 73 (1, 256, 16, 16).

[0054] Step 3.4: Feature M 7 Send it to the dilated convolution with a dilation rate of 4, padding of 4, and a convolution kernel size of 3 to obtain the scale feature M 74 (1, 256, 16, 16).

[0055] Step 3.5: Feature M 7 It is sent to the dilated convolution with a dilation rate of 8, a padding of 8, and a convolution kernel size of 3 to obtain the scale feature M. 75 (1, 256, 16, 16).

[0056] Step 3.6: Feature M 72 、M 73 、M 74 、M 75 Splicing, and then perform feature fusion through a 1×1 convolution to obtain feature M 76 .

[0057] Step 3.7: Introduce adaptive weighting parameters, perform weighted addition on the fusion features and input features, realize adaptive fusion of features, and obtain feature M 8 (1, 256, 16, 16). As shown in formula (4):

[0058] M 8 =p×M 76 +(1-p)×M 7 (4)

[0059] Where p is the adaptive weighting parameter.

[0060] Step 4: The output M of the four background and target enhancement modules 3 、M 4 、M 5 、M 6 , are sent to the spatial channel cross attention (SCTB) to obtain the output features O in turn 1 , O 2 , O 3 , O 4 .

[0061] Step 5: Construct a shape perception and detail recovery module (SPDR) in the decoder part, combining large kernel convolution and center difference convolution to enhance the edge and details of the target;

[0062] Step 5.1: Set feature M 8 Upsampling operation to obtain feature M 9 .

[0063] Step 5.2: Output feature O 4 and M 9 It is sent to the shape perception and detail recovery module, and the channel cross attention is used to weight the features to obtain feature N 1 .

[0064] Step 5.3: Send the weighted features to the 9×9 large kernel convolution to obtain feature N 2 .

[0065] Step 5.4: Send the weighted features into the central difference convolution to obtain feature N 3 .

[0066] Step 5.5: N 2 、N 3 Add up to get the output feature N 4 (1, 128, 32, 32). As shown in equations (5) and (6):

[0067] N 1 =M 9 ×(Sigmoid(AvgPool(O4 ))+Sigmoid(AvgPool(M 9 ))) (5)

[0068] N 4 =LKC(N 1 )+CDC(N 1 ) (6)

[0069] Among them, LKC refers to the large convolution kernel of 9×9, and CDC refers to the central difference convolution with a kernel size of 3×3.

[0070] Step 5.6: N 4 Upsampling and O 3 Send it to the shape perception and detail recovery module. Then repeat the above steps 5.2 to 5.5 to obtain feature N in turn. 3 (1, 64, 64, 64), N 2 (1, 32, 128, 128), N 1 (1, 1, 256, 256).

[0071] Step 6: M 8 、N 4 、N 3 、N 2 、N 1 Multi-scale deep supervision to generate the final detection results.

[0072] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modification, equivalent replacement of the improvement, etc. made within the spirit and principle of the claims of the present invention are also included in the protection scope of the claims of the present invention.

Claims

1. A small target detection method based on a shape-aware network with background and target enhancement, comprising the following steps: Step 1: Obtain infrared small target detection dataset; Step 2: Construct a background and target enhancement module (BTAB) during the encoder feature extraction process to enhance the target and background features respectively, and further fuse the features through the channel cross attention mechanism; Step 3: After the encoder, a multi-scale perception module (SSB) is constructed to further process the features and extract multi-scale features to enhance the perception of objects of different sizes; Step 4: Construct a shape perception and detail recovery module (SPDR) in the decoder part to fuse the upsampled features with the features obtained by cross-channel cross attention, and combine large kernel convolution and center difference convolution to enhance the edge and details of the target; Step 5: Use a multi-scale deep supervision optimization strategy to fuse the features extracted from the encoder and decoder to generate the final detection result.

2. According to the method of small target detection based on shape perception network with background and target enhancement described in claim 1, it is characterized in that: The step 2 specifically includes: The encoder is composed of four background and target enhancement modules. Each background and target enhancement module first passes through a residual block to extract initial features. The features are then sent to the background feature enhancement branch and the target feature enhancement branch respectively. The background feature enhancement branch first performs a global average pooling on the feature, and then sends it to a dilated convolution group with dilation rates of 2, 4, and 2. Then, a Sigmoid function is used to generate background attention weights to weight the initial features and obtain enhanced background features. The target feature enhancement branch first extracts local features through a full-dimensional dynamic convolution, then uses a depth-wise separable convolution and point convolution, and then uses Sigmoid to obtain the target attention weight to weight the initial features to obtain the enhanced target features. The features obtained from the two branches are then concatenated, and then a point-by-point convolution is performed to obtain the enhanced target and background features.

3. According to the method for small target detection based on shape perception network with background and target enhancement as described in claim 1, it is characterized in that: The step three specifically includes: The multi-scale perception module (SSB) further processes the enhanced features, wherein the multi-scale convolution feature extraction uses dilated convolution branches with different dilation rates to extract multi-scale features and capture contextual information of different scales; Global feature fusion obtains global features through global average pooling and global convolution module 1×1 convolution, upsamples them to the input size, and fuses them with multi-scale features; finally, adaptive weighting parameters are introduced to perform weighted addition of fused features and input features to achieve adaptive feature fusion.

4. The infrared small target detection method according to claim 1, characterized in that: In step 4, the shape perception and detail recovery module (SPDR) combines 9×9 large kernel convolution and center difference convolution after fusing the features extracted in the encoder and decoder, effectively enhancing the edge details of the image. Among them, large kernel convolution is used to extract large-scale features of the image to capture a wider range of contextual information; edge detail features are extracted through center difference convolution and added to the large-scale features to enhance the edge details of the image.

Images