Remote sensing target detection method and system based on selective feature space fusion

Abstract

The invention discloses a remote sensing target detection method and system based on selective feature space fusion, and belongs to the field of remote sensing target detection of computer vision, and the method is characterized in that firstly, multi-scale features, extracted by a backbone network, of an input remote sensing target image are subjected to mapping processing through a convolution layer and a group normalization layer; the method comprises the following steps: obtaining feature maps of different stages, then carrying out spatial dimension alignment and channel splicing operation on the feature maps of different stages, refining spliced features by using a channel attention mechanism, and carrying out pooling, large kernel convolution and activation function processing on the refined features to obtain a selective feature space attention weight; according to the method, the target detection capability in remote sensing target detection is improved, so that the problems of diversity of remote sensing images, large target size and complexity are solved.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing target detection of computer vision, and more specifically, it relates to a remote sensing target detection method and system based on selective feature space fusion. Background Art

[0002] Remote sensing target detection technology is an important part of remote sensing data processing and interpretation. Its development is closely dependent on the advancement of remote sensing imaging technology and the promotion of multi-field application needs. In recent years, with the rapid development of high-resolution remote sensing technology, the spatial resolution, spectral resolution and temporal resolution of remote sensing images have been significantly improved, providing a rich and high-quality data foundation for target detection.

[0003] Compared with natural images, remote sensing images have unique imaging characteristics. Remote sensing images are usually taken from a high altitude, and the scenes contain complex and changing background information. The scales of targets vary significantly and the number is huge. Among them, small targets, densely distributed targets, and targets of various shapes are particularly common. Natural images are mostly collected from a horizontal perspective, with relatively simple backgrounds, small changes in target scales, and a limited number. This difference brings unique technical challenges to target detection in remote sensing images.

[0004] Existing remote sensing target detection methods are mainly divided into traditional image processing methods and deep learning-based methods. Traditional methods rely on manually designed features and usually require expert knowledge for feature extraction, but they have poor adaptability to complex and changeable scenes in remote sensing images and their performance is significantly limited. In contrast, deep learning methods automatically learn features through neural networks, get rid of the limitations of manual design, and significantly improve detection efficiency and accuracy. In particular, the application of convolutional neural networks makes the feature extraction process more efficient, providing solid technical support for remote sensing target detection.

[0005] In target detection technology, multi-scale feature fusion is an important means to improve detection performance. By integrating information from different feature layers, multi-scale feature fusion can effectively enhance the spatial and contextual feature extraction capabilities of the target. For example, the Feature Pyramid Network (FPN) significantly improves the detection performance of small targets by propagating high-level semantic information from top to bottom and combining the underlying detail features; the Path Aggregation Feature Pyramid Network (PAFPN) further introduces a multi-path structure to optimize cross-layer feature flow and sharing. However, these methods are mainly designed for natural images. When directly applied to remote sensing images, they are difficult to deal with problems such as background complexity and large-scale changes in targets. In addition, the targets in remote sensing images are complex in morphology, have significant irregularities, and are usually strongly directional. These characteristics increase the difficulty of feature fusion and affect the positioning and detection accuracy of targets.

[0006] Therefore, in order to solve the above technical problems, the present application proposes a remote sensing target detection method and system based on selective feature space fusion. Summary of the invention

[0007] The purpose of the present invention is to overcome the deficiencies of the prior art and to better and effectively solve the problem that the multi-scale feature fusion method in the prior art is difficult to process remote sensing images.

[0008] To achieve the above object, the present invention provides the following technical solution: a remote sensing target detection method based on selective feature space fusion, step A: the multi-scale features extracted from the input remote sensing target image by the backbone network are mapped through a convolution layer and a group normalization layer to obtain feature maps at different stages;

[0009] Step B: perform spatial dimension alignment and channel concatenation operations on feature maps at different stages, and use the channel attention mechanism to refine the concatenated features;

[0010] Step C: Perform pooling, large kernel convolution and activation function processing on the refined features to obtain the selective feature space attention weights;

[0011] Step D: The spatial information in the feature map is weighted and added by the selective feature spatial attention weights obtained in step C, and then the attention features of the feature map at this stage are obtained by convolution;

[0012] Step E: Combine the attention feature obtained in step D with the original feature map in step A to obtain the selective feature space fusion feature;

[0013] Step F: Downsample the feature map of the last stage obtained in step A and combine it with the feature map after selective feature fusion Figure 1 Starting from the beginning, fine-grained and contextual features are extracted through dense convolution and void convolution, and the activation function is used to enhance the network's context perception ability. Finally, the fused fine-grained features and contextual features are sent to the detection head through convolution to complete the detection of remote sensing target images.

[0014] The aforementioned remote sensing target detection method based on selective feature space fusion, step A: the multi-scale features extracted by the backbone network of the input remote sensing target image are mapped through the convolution layer and the group normalization layer to obtain feature maps at different stages. The specific steps are as follows:

[0015] Step A1: The multi-scale features extracted by the backbone network of the input remote sensing target image are mapped through the convolution layer and the group normalization layer to obtain feature maps at different stages and prepare the input image object.

[0016] Among them, H is the height of the image, W is the width of the image, 3 is the number of channels of the color image, that is, the three values ​​of RGB, and if it is a grayscale image, it is 1;

[0017] Step A2: Input image Input into the backbone network for feature extraction. The backbone network will output feature maps with different characteristics at each stage according to its own network structure and parameters, which are represented as as well as

[0018] Among them, different C values ​​represent different feature information learned by the network model at each stage, H is the height of the image, and W is the width of the image;

[0019] Step A3: The feature maps X1, X2, X3 and X4 of different features are mapped into That is, the feature map of the first stage Mapping This process can be expressed as:

[0020] F l =GroupNorm(Conv 1×1 (X l ))

[0021] Among them, Conv 1×1 It is a convolutional layer with a kernel size of 1×1 and an output channel number of 256. GroupNorm is a group normalization layer, and the grouping is set to 4. Different C values ​​represent different feature information learned by the network model at each stage.

[0022] The aforementioned remote sensing target detection method based on selective feature space fusion, step B: perform spatial dimension alignment and channel splicing operations on feature maps at different stages, and use the channel attention mechanism to refine the spliced ​​features. The specific steps are as follows:

[0023] Step B1: For each stage feature map The feature map of the next stage Use interpolation to upsample To ensure that they are consistent in spatial dimensions, so that their size is consistent with the features of the current stage Figure 1 The operation can be expressed as:

[0024] F′ l+1 =UpSample nearest (F l+1 )

[0025] Among them, UpSample nearest is the nearest neighbor interpolation method;

[0026] Step B2: Upsample the feature map F′ l+1 With the feature map F of the current stage l Concatenate in the channel dimension and get This process can be expressed as:

[0027] P l =Concat(F l ,F′ l+1 )

[0028] Among them, Concat represents the concatenation operation of the channel dimension;

[0029] Step B3: Concatenate the feature map P l Use SE channel attention mechanism to refine Y l , the process can be expressed as:

[0030]

[0031] Among them, Conv 1×1 represents a 1×1 convolution operation, ⊙ represents element-by-element multiplication, Represents a global average pooling operation.

[0032] The aforementioned remote sensing target detection method based on selective feature space fusion, step C: performing pooling, large kernel convolution and activation function processing on the refined features to obtain the selective feature space attention weight; the specific steps are:

[0033] Step C1: Refine the feature Y lTwo average pooling operations are used along the channel dimension to obtain two feature maps. and And concatenate them in the channel dimension to highlight the information area in the feature map, and get This process can be expressed as:

[0034]

[0035] in, and is the channel average pooling operation, Concat represents the splicing operation of the channel dimension; Step C2: Use large kernel convolution to Y′ l Processed and processed by Sigmoid activation function to obtain the selective feature space attention weight This process can be expressed as:

[0036] A l =Sigmoid(Conv large (Y′ l ))

[0037] Among them, the Sigmoid function maps the attention feature map to [0,1], Conv large Represents a large kernel convolution operation, whose convolution kernel size is 7×7.

[0038] The aforementioned remote sensing target detection method based on selective feature space fusion, step D: weighting and adding the spatial information in the feature map by the selective feature space attention weight obtained in step C, and then obtaining the attention features of the feature map at this stage by convolution, the specific steps are:

[0039] The selective feature space attention weight A l Decompose into and Feature maps F applied to the two stages respectively l and F′ l+1 In step D2, the spatial information is weighted and added together. Then, the focus feature of the feature map of this stage is obtained through convolution with a kernel size of 1×1. This process can be expressed as:

[0040] A l,1 ,A l,2 =Split(A l )

[0041] O l =Conv 1×1 (F l ⊙A l,1 +F′ l+1 ⊙A l,2 )

[0042] Among them, Conv 1×1 The kernel size is 1×1, ⊙ represents element-by-element multiplication, and Split represents decomposition.

[0043] The aforementioned remote sensing target detection method based on selective feature space fusion, step E: combining the focus feature obtained in step D with the original feature map in step A to obtain the selective feature space fusion feature, the specific steps are:

[0044] The original feature map F of this stage l With attention feature O l Multiply; at the same time, to avoid gradient disappearance, follow the original FPN settings and the original feature map F l and F′ of the next stage l+1 Add together to get the final selective feature space fusion feature This process can be expressed as:

[0045] Z l =O l ⊙F l +F l +F′ l+1

[0046] Among them, ⊙ represents element-by-element multiplication.

[0047] The aforementioned remote sensing target detection method based on selective feature space fusion, step F: down-sample the feature map of the last stage obtained in step A, and combine it with the feature map after selective feature fusion Figure 1 From the beginning, fine-grained and contextual features are extracted through dense convolution and dilated convolution, and the activation function is used to enhance the network's context perception ability. Finally, the fused fine-grained features and contextual features are sent to the detection head through convolution to complete the detection of remote sensing target images. The specific steps are:

[0048] Step F1: The Downsample to In order to capture the fine-grained features of a large number of small targets, the process can be expressed as:

[0049] Z5=DownSample 3×3 (F4)

[0050] Among them, DownSample 3×3 It is a maximum pooling layer with a stride of 2 and a kernel of 3×3;

[0051] Step F2: The selective spatial fusion feature Zl is further extracted through dense convolution with a kernel size of 3×3 and dilation factor 2. The output is This process can be expressed as:

[0052] R l =Conv 3×3 (Z l )+DilatedConv 3×3 (Z l )

[0053] Among them, Conv 3×3 It is a dense convolution with a kernel size of 3×3 and a dilation factor of 1 to capture fine-grained features; DilatedConv 3×3 A dilated convolution with a kernel size of 3×3 and a dilation factor of 2 is used to capture contextual features;

[0054] Step F3: Processed R l The extracted fine-grained features and context features are fused through 1×1 convolution and sent to the detection head to input various types of targets in the image, thereby completing the detection of remote sensing target images.

[0055] In the aforementioned remote sensing target detection method based on selective feature space fusion, in step F2, dense convolution Conv 3×3 And dilatedConv 3×3 Output feature map In the process, the CSiLU activation function is used, and the function is defined as:

[0056]

[0057] in, represents an average pooling layer with a kernel size of 5×5, which is used to generate local context information. This process can be expressed as:

[0058] R l =CSiLU(Conv 3×3 (Z l ))+CSiLU(DilatedConv 3×3 (Z l ))

[0059] Among them, Conv 3×3 It is a dense convolution with a kernel size of 3×3 and a dilation factor of 1. 3×3 It is a dilated convolution with a kernel size of 3×3 and a dilation factor of 2.

[0060] A remote sensing target detection system based on selective feature space fusion includes a mapping processing module, an alignment splicing module, a weight acquisition module, a focus feature acquisition module, a combination module and a fusion module; the mapping processing module is used to map the multi-scale features extracted from the input remote sensing target image through a backbone network through a convolution layer and a group normalization layer to obtain feature maps of different stages; the alignment splicing module is used to perform spatial dimension alignment and channel splicing operations on the feature maps of different stages, and use a channel attention mechanism to refine the spliced ​​features; the weight acquisition module is used to perform pooling, large kernel convolution and activation function processing on the refined features to obtain selective feature space attention weights; the focus feature acquisition module is used to weight and add and fuse the spatial information in the feature map with the obtained selective feature space attention weights, and then obtain the focus feature of the feature map of this stage through convolution; the combination module is used to combine the focus feature obtained in step D with the original feature map of step A to obtain the selective feature space fusion feature; the fusion module is used to downsample the feature map of the last stage obtained in step A, and combine it with the feature map after selective feature fusion Figure 1 Starting from the beginning, fine-grained and contextual features are extracted through dense convolution and void convolution, and the activation function is used to enhance the network's context perception ability. Finally, the fused fine-grained features and contextual features are sent to the detection head through convolution to complete the detection of remote sensing target images.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. The present invention designs a selective feature spatial attention mechanism, which effectively improves the performance of remote sensing target detection by dynamically selecting and fusing the spatial information of multi-scale features; and introduces context-aware normalized linear units to enhance the network's modeling ability for complex context information of remote sensing images, thereby improving the detection ability of targets in remote sensing target detection, in order to cope with the diversity of remote sensing images, large target size, complexity and other issues. Compared with traditional feature fusion methods, this method performs well in remote sensing target detection and has better detection effects, so as to solve the problem that multi-scale feature fusion methods in the prior art are difficult to process remote sensing images in the background technology;

[0063] 2. The present invention designs a new activation function CSiLU to enhance the context perception ability of the network. CSiLU combines global pooling and local context information capture modules to model the global and local information of the input feature map during the activation process. By integrating global and local information, CSiLU can more comprehensively capture multi-scale and spatially distributed features, thereby improving the performance of the network in complex scenes. Compared with existing activation functions, the design of CSiLU aims to make up for the lack of spatial modeling capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0065] Figure 1 It is the overall flow chart of the present invention;

[0066] Figure 2 The schematic diagram of the selective feature space fusion of the present invention (the upper part of the figure, the multi-layer graphic representation between F and R: selective feature space fusion, represents element-wise multiplication, represents element-wise addition). DETAILED DESCRIPTION

[0067] The present invention will be further described below in conjunction with the accompanying drawings.

[0068] like Figure 1-2 As shown, a remote sensing target detection method based on selective feature space fusion includes the following steps:

[0069] Step A: The multi-scale features extracted by the backbone network from the input remote sensing target image are mapped through the convolution layer and the group normalization layer to obtain feature maps at different stages. The specific steps are as follows:

[0070] Step A1: The multi-scale features extracted by the backbone network of the input remote sensing target image are mapped through the convolution layer and the group normalization layer to obtain feature maps at different stages and prepare the input image object.

[0071] Among them, H is the height of the image, W is the width of the image, 3 is the number of channels of the color image, that is, the three values ​​of RGB, and if it is a grayscale image, it is 1;

[0072] Step A2: Input image The input is sent to the backbone network for feature extraction. The backbone network will output feature maps with different characteristics at each stage according to its own network structure and parameters, which are represented as as well as

[0073] Among them, different C values ​​represent different feature information learned by the network model at each stage, H is the height of the image, and W is the width of the image;

[0074] Step A3: The feature maps X1, X2, X3 and X4 of different features are mapped into That is, the L-th stage feature map Mapping This process can be expressed as:

[0075] F l =GroupNorm(Conv 1×1 (X l ))

[0076] Among them, Conv 1×1 is a convolutional layer with a kernel size of 1×1 and an output channel number of 256. GroupNorm is a group normalization layer with a grouping setting of 4. Different C values ​​represent different feature information learned by the network model at each stage.

[0077] Step B: Perform spatial dimension alignment and channel concatenation operations on feature maps at different stages, and use the channel attention mechanism to refine the concatenated features. The specific steps are as follows:

[0078] Step B1: For each stage feature map The feature map of the next stage Use interpolation to upsample To ensure that they are consistent in spatial dimensions, so that their size is consistent with the features of the current stage Figure 1 The operation can be expressed as:

[0079] F′ l+1 =UpSample nearest (F l+1 )

[0080] Among them, UpSample nearest is the nearest neighbor interpolation method;

[0081] Step B2: Upsample the feature map F′ l+1 With the feature map F of the current stage l Concatenate in the channel dimension and get This process can be expressed as:

[0082] P l =Concat(F l ,F′ l+1 )

[0083] Among them, Concat represents the concatenation operation of the channel dimension. It should be noted that the feature map F4 at the bottom of the feature pyramid does not perform the operation because it does not have the feature map of the next stage;

[0084] Step B3: Concatenate the feature map P l Use SE (Squeeze-and-Excitation) channel attention mechanism to refine and obtain Y l , the process can be expressed as:

[0085]

[0086] Among them, Conv 1×1 represents a 1×1 convolution operation, ⊙ represents element-by-element multiplication (also known as pixel-by-pixel multiplication), represents the global average pooling operation;

[0087] Step C: Perform pooling, large kernel convolution and activation function processing on the refined features to obtain the selective feature space attention weights; the specific steps are:

[0088] Step C1: Refine the feature Y l Two average pooling operations are used along the channel dimension to obtain two feature maps. and And concatenate them in the channel dimension to highlight the information area in the feature map, and get This process can be expressed as:

[0089]

[0090] in, and It is the channel average pooling operation, and Concat represents the concatenation operation in the channel dimension;

[0091] Step C2: Use large kernel convolution to Y′ l Processed and processed by Sigmoid activation function to obtain the selective feature space attention weight This process can be expressed as:

[0092] A l =Sigmoid(Conv large (Y′ l ))

[0093] Among them, the Sigmoid function maps the attention feature map to [0,1], Conv large Represents a large kernel convolution operation, whose convolution kernel size is 7×7;

[0094] Step D: The spatial information in the feature map is weighted and added by the selective feature spatial attention weights obtained in step C, and then the attention features of the feature map at this stage are obtained by convolution. The specific steps are as follows:

[0095] The selective feature space attention weight A l Decompose into and Feature maps F applied to the two stages respectively l and F′ l+1In the process, the spatial information is weighted and added, and then the focus feature of the feature map of this stage is obtained through convolution with a kernel size of 1×1. This process can be expressed as:

[0096] A l,1 ,A l,2 =Split(A l )

[0097] O l =Conv 1×1 (F l ⊙A l,1 +F′ l+1 ⊙A l,2 )

[0098] Among them, Conv 1×1 The kernel size is 1×1, ⊙ means element-by-element multiplication, and Split means decomposition;

[0099] Step E: Combine the attention feature obtained in step D with the original feature map in step A to obtain the selective feature space fusion feature. The specific steps are:

[0100] The original feature map F of this stage l With attention feature O l Multiply; at the same time, to avoid gradient disappearance, follow the original FPN settings and the original feature map F l and F′ of the next stage l+1 Add together to get the final selective feature space fusion feature This process can be expressed as:

[0101] Z l =O l ⊙F l +F l +F′ l+1

[0102] Among them, ⊙ represents element-by-element multiplication;

[0103] Step F: Downsample the feature map of the last stage obtained in step A and combine it with the feature map after selective feature fusion Figure 1 From the beginning, fine-grained and contextual features are extracted through dense convolution and dilated convolution, and the activation function is used to enhance the network's context perception ability. Finally, the fused fine-grained features and contextual features are sent to the detection head through convolution to complete the detection of remote sensing target images. The specific steps are:

[0104] Step F1: The Downsample to In order to capture the fine-grained features of a large number of small targets, the process can be expressed as:

[0105] Z5=DownSample 3×3 (F4)

[0106] Among them, DownSample 3×3 It is a maximum pooling layer with a stride of 2 and a kernel of 3×3;

[0107] Step F2: The selective spatial fusion feature Zl is further extracted through dense convolution with a kernel size of 3×3 and dilation factor 2. The output is This process can be expressed as:

[0108] R l =Conv 3×3 (Z l )+DilatedConv 3×3 (Z l )

[0109] Among them, Conv 3×3 It is a dense convolution with a kernel size of 3×3 and a dilation factor of 1 to capture fine-grained features; DilatedConv 3×3 A dilated convolution with a kernel size of 3×3 and a dilation factor of 2 is used to capture contextual features;

[0110] Step F3: Processed R l The extracted fine-grained features and context features are fused through 1×1 convolution and sent to the detection head to input various types of targets in the image, thereby completing the detection of remote sensing target images;

[0111] In step F2, dense convolution Conv 3×3 And dilatedConv 3×3 Output feature map In the process, the CSiLU activation function is used, and the function is defined as:

[0112]

[0113] in, represents an average pooling layer with a kernel size of 5×5, which is used to generate local context information. This process can be expressed as:

[0114] R l =CSiLU(Conv 3×3 (Z l ))+CSiLU(DilatedConv 3×3 (Z l ))

[0115] Among them, Conv3×3 It is a dense convolution with a kernel size of 3×3 and a dilation factor of 1. 3×3 It is a dilated convolution with a kernel size of 3×3 and a dilation factor of 2;

[0116] It should be noted here that CSiLU is a new activation function designed by the present invention, namely the context-aware Sigmoid Linear Unit (CSiLU), to enhance the context perception ability of the network. CSiLU combines global pooling and local context information capture modules to model the global and local information of the input feature map during the activation process. By integrating global and local information, CSiLU can more comprehensively capture multi-scale and spatially distributed features, thereby improving the performance of the network in complex scenes. Compared with existing activation functions, the design of CSiLU aims to make up for the lack of spatial modeling capabilities.

[0117] A remote sensing target detection system based on selective feature space fusion includes a mapping processing module, an alignment splicing module, a weight acquisition module, a focus feature acquisition module, a combination module and a fusion module; the mapping processing module is used to map the multi-scale features extracted from the input remote sensing target image through a backbone network through a convolution layer and a group normalization layer to obtain feature maps of different stages; the alignment splicing module is used to perform spatial dimension alignment and channel splicing operations on the feature maps of different stages, and use a channel attention mechanism to refine the spliced ​​features; the weight acquisition module is used to perform pooling, large kernel convolution and activation function processing on the refined features to obtain selective feature space attention weights; the focus feature acquisition module is used to weight and add and fuse the spatial information in the feature map with the obtained selective feature space attention weights, and then obtain the focus feature of the feature map of this stage through convolution; the combination module is used to combine the focus feature obtained in step D with the original feature map of step A to obtain the selective feature space fusion feature; the fusion module is used to downsample the feature map of the last stage obtained in step A, and combine it with the feature map after selective feature fusion Figure 1 Starting from the beginning, fine-grained and contextual features are extracted through dense convolution and void convolution, and the activation function is used to enhance the network's context perception ability. Finally, the fused fine-grained features and contextual features are sent to the detection head through convolution to complete the detection of remote sensing target images.

[0118] To better illustrate the use effect of the present invention, a specific embodiment of the present invention is introduced below: Step 1: Feature Mapping

[0119] The operation process of this step is described in detail. Since the remote sensing image is very large, it is difficult to directly feed it into the deep learning network for training and reasoning. Therefore, the original image is cropped into tiles one by one. Assuming that the crop size is 1024×1024, the image input into the deep learning The size of is 3×1024×1024. As the network depth increases, the number of output channels increases and the feature map after downsampling is

[0120] The channels of each layer are different feature information learned by the network model. For example, after Conv 1×1 After and GroupNorm, the feature map is output The same is true for X2, X3, and X4. Conv 1×1 The function of is to align the number of channels of the feature map of each stage to the same value and interact with the content learned in each layer of channels; GroupNorm normalizes the feature maps of each stage by grouping them so that the mean and variance of the features remain stable during the training process, thereby enhancing the feature expression ability and generalization.

[0121] Step 2: Feature alignment and stitching

[0122] The first stage feature map after step 1 Take this as an example to explain the operation of this step. Assume that the size of F1 is 256×256×256, and the size of its next stage feature map F2 is 256×128×128. Use the nearest neighbor interpolation method UpSample for F2. nearest get The nearest neighbor interpolation operation is to take the value of the new pixel in the upsampled image as the value of the nearest pixel in the image before upsampling. Concatenate F1 and F2 in the channel dimension to get Then, the SE channel attention mechanism is used to refine the feature information of P1 to obtain The SE channel attention mechanism can automatically adjust the importance of each feature channel according to the specific situation of the input, thereby enhancing useful features, suppressing useless features, and improving the expression ability and performance of the model; the same is true for F2 and F3. Since the feature map F4 at the bottom does not have the feature map of the next stage, no operation is performed on F4.

[0123] Step 3: Selective Feature Space Fusion

[0124] The first stage feature map after step 2 As an example, this step is explained in detail. Assume that the size of Y1 is 512×256×256, and use two average pooling operations along the channel dimension to obtain and Then concatenate in the channel dimension After large kernel convolution and Sigmoid activation function normalization, the attention weight is obtained The large kernel convolution is used to extract local context information to facilitate the normalization of the Sigmoid activation function to obtain more accurate weights. The weight is the importance of each pixel in the feature map. A1 is decomposed along the channel dimension into and Decomposed attention weight A 1,1 , A 1,2 Respectively with feature maps Element-wise multiplication and Conv 1×1 Convolution processing obtains the focus feature map for F1 It represents the area that the feature map F1 needs to focus on. Then, O1 is multiplied by the original feature map F1 to obtain Finally, due to the large number of multiplication operations in deep learning networks, for decimals, as the network depth increases, its value will become smaller and smaller, causing the gradient to disappear during the back propagation of the model. To avoid the gradient disappearance, Q1 is added to the original feature map F1 and F′2 to obtain The same is true for Y2 and Y3. and Since the bottom feature map Y4 does not have the feature map of the next stage, no operation is performed on Y4.

[0125] Step 4: Fine-grained feature extraction and context feature fusion

[0126] Take the feature map of the first stage after step 3 Take this as an example to explain the operation of this step in detail. Assume that the size of Z1 is 256×256×256. First, since there are a large number of small targets in the remote sensing image, the feature layer Use a max pooling layer DownSample with a stride of 2 and a kernel of 3×3 3×3 Downsampling to feature map for The feature is obtained by extracting fine-grained features and context features through dense convolution with a kernel size of 3×3 and dilation factor of 2. At the same time, the context-aware Sigmoid weighted linear unit further integrates global and local context information to improve the extraction and fusion of target features. The same is true for Z2, Z3, Z4, and Z5, and the feature maps are obtained respectively. The fused features are finally sent to the remote sensing target detection head after 1×1 convolution mapping to detect various targets in the input image to complete the detection of the remote sensing target image.

[0127] Experimental proof:

[0128] The effectiveness of the proposed method was evaluated on the remote sensing target detection datasets DOTA-v1.0 and UCAS-AOD in combination with different detection heads, and compared with several feature fusion methods. The benchmark feature fusion methods compared include feature pyramid network FPN, path-enhanced feature pyramid network PAFPN, and rotation equivariant feature pyramid network ReFPN. The evaluation index is the mean average precision mAP, which is a commonly used index in remote sensing target detection. It is the most commonly used index to measure the comprehensive performance of the model and is used to evaluate the overall performance of the model in multiple categories.

[0129] It can be analyzed from Table 1 and Table 2 that the methods designed in the present invention can achieve leading performance, which shows that the present invention has good generalization and robustness.

[0130] Table 1 Experimental results of combining different detectors on the DOTA-v1.0 dataset

[0131]

[0132] Table 2 Experimental results of combining different detectors on the UCAS-AOD dataset

[0133]

[0134] In summary, the present invention first maps the multi-scale features extracted by the backbone network of the input remote sensing target image through the convolution layer and the group normalization layer to obtain feature maps at different stages, then performs spatial dimension alignment and channel splicing operations on the feature maps at different stages, and uses the channel attention mechanism to refine the spliced ​​features, and performs pooling, large kernel convolution and activation function processing on the refined features to obtain selective feature spatial attention weights, and then weights and adds and fuses the spatial information in the feature map obtained by the selective feature spatial attention weights, and then obtains the attention features of the feature map of this stage through convolution, and combines the obtained attention features with the original feature map to obtain the selective feature spatial fusion features, and finally, performs the selective feature spatial fusion feature fusion on the feature layer selected based on the selective feature spatial fusion features. Downsampling is performed, and then fine-grained and contextual features are extracted through dense convolution and void convolution. At the same time, the activation function is used to enhance the network's context perception ability. Finally, the fused fine-grained features and contextual features are sent to the detection head through convolution to complete the detection of remote sensing target images. The present invention designs a selective feature spatial attention mechanism, which effectively improves the performance of remote sensing target detection by dynamically selecting and fusing the spatial information of multi-scale features; and introduces context-aware normalized linear units to enhance the network's modeling ability for complex contextual information of remote sensing images, thereby improving the detection ability of targets in remote sensing target detection, in order to cope with the diversity of remote sensing images, large target size and complexity, and other problems. Compared with traditional feature fusion methods, this method performs well in remote sensing target detection and has better detection effects.

[0135] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any ordinary technician in the industry can smoothly implement the present invention as shown in the drawings and described above. However, any equivalent changes, modifications and evolutions made by technicians familiar with the profession without departing from the scope of the technical solution of the present invention using the technical content disclosed above are all equivalent embodiments of the present invention. At the same time, any equivalent changes, modifications and evolutions made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the technical solution of the present invention.

Claims

1. A remote sensing target detection method based on selective feature space fusion, characterized in that: The following steps are involved: Step A: The multi-scale features extracted by the backbone network from the input remote sensing target image are mapped through the convolution layer and the group normalization layer to obtain feature maps at different stages; Step B: perform spatial dimension alignment and channel concatenation operations on feature maps at different stages, and use the channel attention mechanism to refine the concatenated features; Step C: Perform pooling, large kernel convolution and activation function processing on the refined features to obtain the selective feature space attention weights; Step D: The spatial information in the feature map is weighted and added by the selective feature spatial attention weights obtained in step C, and then the attention features of the feature map at this stage are obtained by convolution; Step E: Combine the attention feature obtained in step D with the original feature map in step A to obtain the selective feature space fusion feature; Step F: Downsample the feature map of the last stage obtained in step A, and extract fine-grained and contextual features together with the feature map after selective feature fusion through dense convolution and hole convolution, and use activation function to enhance the network context perception ability. Finally, the fused fine-grained features and contextual features are sent to the detection head through convolution to complete the detection of remote sensing target images.

2. The remote sensing target detection method based on selective feature space fusion according to claim 1, characterized in that: Step A: The multi-scale features extracted by the backbone network from the input remote sensing target image are mapped through the convolution layer and the group normalization layer to obtain feature maps at different stages. The specific steps are as follows: Step A1: The multi-scale features extracted by the backbone network of the input remote sensing target image are mapped through the convolution layer and the group normalization layer to obtain feature maps at different stages and prepare the input image object. Among them, H is the height of the image, W is the width of the image, 3 is the number of channels of the color image, that is, the three values ​​of RGB, and if it is a grayscale image, it is 1; Step A2: Input image Input into the backbone network for feature extraction. The backbone network will output feature maps with different characteristics at each stage according to its own network structure and parameters, which are represented as as well as Among them, different C values ​​represent different feature information learned by the network model at each stage, H is the height of the image, and W is the width of the image; Step A3: The feature maps X1, X2, X3 and X4 of different features are mapped into That is, the feature map of the first stage Mapping This process can be expressed as: F l =GroupNorm(Conv 1×1 (X l )) Among them, Conv 1×1 It is a convolutional layer with a kernel size of 1×1 and an output channel number of 256. GroupNorm is a group normalization layer, and the grouping is set to 4. Different C values ​​represent different feature information learned by the network model at each stage.

3. The remote sensing target detection method based on selective feature space fusion according to claim 1, characterized in that: Step B: Perform spatial dimension alignment and channel concatenation operations on feature maps at different stages, and use the channel attention mechanism to refine the concatenated features. The specific steps are as follows: Step B1: For each stage feature map The feature map of the next stage Use interpolation to upsample To ensure that they are consistent in spatial dimensions, so that their size is consistent with the feature map of the current stage, the operation can be expressed as: F′ l+1 =UpSample nearest (F l+1 ) Among them, UpSample nearest is the nearest neighbor interpolation method; Step B2: Upsample the feature map F′ l+1 With the feature map F of the current stage l Concatenate in the channel dimension and get This process can be expressed as: P l =Concat(F l ,F′ l+1 ) Among them, Concat represents the concatenation operation of the channel dimension; Step B3: Concatenate the feature map P l Use SE channel attention mechanism to refine Y l , the process can be expressed as: Among them, Conv 1×1 represents a 1×1 convolution operation, ⊙ represents element-by-element multiplication, Represents a global average pooling operation.

4. The remote sensing target detection method based on selective feature space fusion according to claim 1, characterized in that: Step C: Perform pooling, large kernel convolution and activation function processing on the refined features to obtain the selective feature space attention weights; The specific steps are: Step C1: Refine the feature Y l Two average pooling operations are used along the channel dimension to obtain two feature maps. and And concatenate them in the channel dimension to highlight the information area in the feature map, and get This process can be expressed as: in, and It is the channel average pooling operation, and Concat represents the concatenation operation in the channel dimension; Step C2: Use large kernel convolution to convolve Y l ′ is processed and the selective feature space attention weight is obtained through Sigmoid activation function. This process can be expressed as: A l =Sigmoid(Conv large (Y′ l )) Among them, the Sigmoid function maps the attention feature map to [0,1], Conv large Represents a large kernel convolution operation, whose convolution kernel size is 7×7.

5. The remote sensing target detection method based on selective feature space fusion according to claim 1, characterized in that: Step D: The spatial information in the feature map is weighted and added by the selective feature spatial attention weights obtained in step C, and then the attention features of the feature map at this stage are obtained by convolution. The specific steps are as follows: The selective feature space attention weight A l Decompose into and Feature maps F applied to the two stages respectively l and F′ l+1 In the process, the spatial information is weighted and added, and then the focus feature of the feature map of this stage is obtained through convolution with a kernel size of 1×1. This process can be expressed as: A l,1 ,A l,2 =Split(A l ) O l =Conv 1×1 (F l ⊙A l,1 +F′ l+1 ⊙A l,2 ) Among them, Conv 1×1 The kernel size is 1×1, ⊙ represents element-by-element multiplication, and Split represents decomposition.

6. The remote sensing target detection method based on selective feature space fusion according to claim 1, characterized in that: Step E: Combine the attention feature obtained in step D with the original feature map in step A to obtain the selective feature space fusion feature. The specific steps are: The original feature map F of this stage l With attention feature O l Multiply; at the same time, to avoid gradient disappearance, follow the original FPN settings and the original feature map F l and F′ of the next stage l+1 Add together to get the final selective feature space fusion feature This process can be expressed as: Z l =O l ⊙F l +F l +F′ l+1 Among them, ⊙ represents element-by-element multiplication.

7. The remote sensing target detection method based on selective feature space fusion according to claim 1, characterized in that: Step F: Downsample the feature map of the last stage obtained in step A3, and extract fine-grained and contextual features together with the feature map after selective feature fusion through dense convolution and dilated convolution, and use activation function to enhance the network context perception ability. Finally, the fused fine-grained features and contextual features are sent to the detection head through convolution to complete the detection of remote sensing target images. The specific steps are as follows: Step F1: The Downsample to In order to capture the fine-grained features of a large number of small targets, the process can be expressed as: Z5=DownSample 3×3 (F4) Among them, DownSample 3×3 It is a maximum pooling layer with a stride of 2 and a kernel of 3×3; Step F2: Fusing the selective spatial features Z l Fine-grained features and contextual features are further extracted through dense convolution with a kernel size of 3×3 and dilation factor 2, and the output is This process can be expressed as: R l =Conv 3×3 (Z l )+DilatedConv 3×3 (Z l ) Among them, Conv 3×3 It is a dense convolution with a kernel size of 3×3 and a dilation factor of 1 to capture fine-grained features; DilatedConv 3×3 A dilated convolution with a kernel size of 3×3 and a dilation factor of 2 is used to capture contextual features; Step F3: Processed R l The extracted fine-grained features and context features are fused through 1×1 convolution and sent to the detection head to input various types of targets in the image, thereby completing the detection of remote sensing target images.

8. The remote sensing target detection method based on selective feature space fusion according to claim 7, characterized in that: In step F2, dense convolution Conv 3×3 And dilatedConv 3×3 Output feature map In the process, the CSiLU activation function is used, and the function is defined as: in, represents an average pooling layer with a kernel size of 5×5, which is used to generate local context information. This process can be expressed as: R l =CSiLU(Conv 3×3 (From l ))+CSiLU(DilatedConv 3×3 (From l )) Among them, Conv 3×3 It is a dense convolution with a kernel size of 3×3 and a dilation factor of 1. 3×3 It is a dilated convolution with a kernel size of 3×3 and a dilation factor of 2.

9. A remote sensing target detection system based on selective feature space fusion, wherein the detection process of the detection system is based on the recognition method according to any one of claims 1 to 8, characterized in that: It includes a mapping processing module, an alignment splicing module, a weight acquisition module, a focus feature acquisition module, a combination module and a fusion module; The mapping processing module is used to map the multi-scale features extracted by the backbone network from the input remote sensing target image through the convolution layer and the group normalization layer to obtain feature maps at different stages; The alignment and splicing module is used to perform spatial dimension alignment and channel splicing operations on feature maps at different stages, and refine the spliced ​​features using a channel attention mechanism; The weight acquisition module is used to perform pooling, large kernel convolution and activation function processing on the refined features to obtain the selective feature space attention weights; The attention feature acquisition module is used to weight and add the obtained selective feature spatial attention weights to the spatial information in the feature map, and then obtain the attention feature of the feature map at this stage through convolution; The combining module is used to combine the focus feature obtained in step D with the original feature map in step A to obtain a selective feature space fusion feature; The fusion module is used to downsample the feature map of the last stage obtained in step A, and extract fine-grained and contextual features through dense convolution and dilated convolution together with the feature map after selective feature fusion, and at the same time use activation functions to enhance the network context perception ability. Finally, the fused fine-grained features and contextual features are sent to the detection head through convolution to complete the detection of remote sensing target images.

Images