Remote sensing image target detection method based on rotationally equivariant convolution channel attention enhancement and multi-scale feature fusion

Abstract

This invention discloses a remote sensing image target detection method based on rotationally isotropic convolutional channel attention enhancement and multi-scale feature fusion. The method includes: selecting a remote sensing image as input to a rotationally isotropic feature extraction network to obtain a multi-layer rotationally isotropic feature map of the input image; inputting the multi-layer feature map into a rotationally isotropic channel attention enhancement module and performing feature fusion to obtain a feature-enhanced rotationally isotropic multi-scale feature map; performing target detection and classification on the multi-scale feature map to obtain the center point coordinates, length, width, and angle of the target, as well as the target category. This invention designs a rotationally isotropic channel attention enhancement feature fusion module to overcome the difficulties in remote sensing image feature extraction, obtaining a multi-scale feature map after rotationally isotropic channel attention enhancement and multi-scale semantic information fusion. Finally, target detection is performed on the feature map to obtain the final detection result, applicable to remote sensing image target detection scenarios.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a remote sensing image target detection method based on rotationally equivariant convolution channel attention enhancement and multi-scale feature fusion. Background Technology

[0002] Remote sensing target detection is a crucial task in remote sensing image processing, with wide applications in both military and civilian fields. The purpose of remote sensing target detection is to calculate the location of targets to be detected and to identify the category of those targets from remote sensing images.

[0003] Remote sensing target detection inherently presents numerous challenges, such as the diverse scales and orientations of targets. Due to the lack of rotational equivariance, traditional convolutional neural network-based target detection methods cannot accurately extract directional information and are not well-suited for target detection in remote sensing images. To more accurately predict target orientation, rotational equivariant convolution has been proposed and applied to remote sensing target detection. Compared to traditional convolutional kernels, this convolution can extract additional directional information, improving the localization accuracy of remote sensing targets. However, with the same number of channels, the semantic information extracted by the rotational equivariant convolution kernel is reduced, leading to a decrease in target classification ability. Existing detection methods based on rotational equivariant convolution cannot improve target classification accuracy without increasing the number of convolutional kernels. Summary of the Invention

[0004] The purpose of this invention is to provide a remote sensing image target detection method based on rotationally equivariant convolution channel attention enhancement and multi-scale feature fusion.

[0005] The technical solution for achieving the objective of this invention is as follows: Firstly, this invention provides a remote sensing image target detection method based on rotationally equivariant convolution channel attention enhancement and multi-scale feature fusion, comprising:

[0006] Step S1: Select remote sensing images from the data as input to the rotational isotropic network, and obtain the rotational isotropic feature map of the input network image;

[0007] Step S2: Input the feature map into the feature fusion module with rotational equivariant channel attention enhancement. Starting from the high-level feature map, it is fused with the enhanced feature map of the upper layer in sequence to obtain the current layer rotational equivariant channel attention. The correlation model is performed on the feature channels with the same directional information after fusion. The feature channel attention is rearranged according to the original feature channels and applied to the fused feature map to obtain the feature map after rotational equivariant channel attention enhancement.

[0008] Step S3: Perform feature target detection on the feature map obtained in step S2 to obtain the center point coordinates of the feature target, the length, width and angle of the feature target, and the category to which the feature target belongs.

[0009] In a second aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0010] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0011] This invention fully utilizes the directional information extracted by rotationally equivariant convolutional networks, the correlation between feature channels, and multi-scale information. It designs a feature fusion module with rotationally equivariant channel attention enhancement to overcome the difficulties of insufficient semantic information extraction and large scale variations in remote sensing targets. This results in a feature map enhanced by rotational equivariant attention and multi-scale feature fusion. Finally, detection is performed on this enhanced feature map to obtain the final detection result. This invention is suitable for target detection in remote sensing images. Compared with existing technologies, this invention has the following advantages:

[0012] (1) To address the issue that the semantic information extraction of rotational equivariant networks is insufficient and the existing semantic information cannot be effectively used for classification, attention is enhanced to the feature channels by rotational equivariant channel attention while keeping the directional information contained in the rotational equivariant features unchanged.

[0013] (2) In view of the fact that the target scale of remote sensing images is varied and a single feature map cannot adapt to multi-scale target detection, the enhanced high-level semantic information is fused into the low-level feature map by first performing channel attention enhancement and then feature fusion, so as to better perform multi-scale target detection. Attached Figure Description

[0014] Figure 1 This is a flowchart of the remote sensing image target detection method based on rotationally equivariant convolution channel attention enhancement and multi-scale feature fusion according to the present invention.

[0015] Figure 2 This is the rotational isotropic channel attention module of the present invention.

[0016] Figure 3 The remote sensing image target detection results of the present invention Figure 1 .

[0017] Figure 4 The remote sensing image target detection results of the present invention Figure 2 . Detailed Implementation

[0018] To enable those skilled in the art to understand the technical solution of the present invention more clearly, the present invention will be described in further detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0019] like Figures 1-4 As shown, the remote sensing image target detection method based on rotationally equivariant convolution channel attention enhancement and multi-scale feature fusion provided in this embodiment includes:

[0020] Step S1: Select remote sensing images from the data as input to the rotational isotropic network, and obtain the rotational isotropic feature map of the input network image;

[0021] Step S2: Input the feature map into the feature fusion module with rotational equivariant channel attention enhancement. Starting from the high-level feature map, it is fused with the enhanced feature map of the upper layer in sequence to obtain the current layer rotational equivariant channel attention. The correlation model is performed on the feature channels with the same directional information after fusion. The feature channel attention is rearranged according to the original feature channels and applied to the fused feature map to obtain the feature map after rotational equivariant channel attention enhancement.

[0022] Step S3: Perform feature target detection on the feature map obtained in step S2 to obtain the center point coordinates (x, y) of the feature target, the length, width, and angle (w, h, θ) of the feature target, and the category c to which the feature target belongs. i .

[0023] In this embodiment, as Figure 1 As shown, step S1 specifically includes the following:

[0024] Step s1.1. Select remote sensing images from the data for preprocessing, and use the preprocessed images as input images for the rotational isotropic network;

[0025] Step s1.2. Perform multiple rotational isovariant convolutions on the image input to the network to extract rotational isovariant features. Select the second, third, fourth, and fifth layer feature maps output by the network as subsequent inputs, and let their scales from large to small be F1, F2, F3, and F4, respectively.

[0026] In this embodiment, as Figure 1 As shown, step S2 specifically includes the following:

[0027] Step s2.1. Input F4 into the rotational equal channel attention module, use the module to generate the rotational equal channel attention corresponding to the feature map, denoted as a4, and perform Hadamard product with the current layer feature map F4 to obtain the feature map F′4 after feature enhancement.

[0028] Step s2.2. From feature map F3 to feature map F1, process the current layer feature map F n The upper-layer enhanced feature map F′ n+1 Perform an upsampling operation to make its scale the same as the current layer feature map. The upsampled upper layer feature map is denoted as F″. n+1 The current layer feature map F n The fused feature map A is obtained by adding the corresponding pixels of the upsampled upper-layer feature map. n ;

[0029] Step s2.3. Combine the fused feature map A n In the input rotational isotropic channel attention module, the rotational isotropic channel attention corresponding to the feature map generated by the module is denoted as a. n The obtained channel attention a n With the current layer feature map F n Perform the Hadamard product to obtain the enhanced feature map F′. n Repeat steps s2.2 and s2.3 multiple times until the enhanced feature maps F′1, F′2 and F′3 corresponding to all input feature maps are obtained.

[0030] In this embodiment, as Figure 1 As shown, the specific formulas for generating the rotational isotropic channel attention in steps s2.1 and s2.3 are as follows:

[0031]

[0032]

[0033]

[0034]

[0035] Among them, M C (F) represents the rotationally isotropic channel attention generated by the rotationally isotropic feature map F, and σ represents the Sigmoid function. O represents F i Channel attention generated from directional feature maps This represents a feature with information in direction i, where i∈(0,1,...,O-1), and O is the orientation hyperparameter set by the rotation-equalized network. MLP stands for Multilayer Perceptron, where the layers are fully connected. AvgPool represents a global average pooling layer, which takes one layer of the feature map as input and outputs the average value within the input region. MaxPool represents a global max pooling layer, which takes one layer of the feature map as input and outputs the maximum value within the input region. ChannelArr represents a channel rearrangement operation, which rearranges the channel attention in each direction according to the original feature channel structure, forming channel attention that corresponds one-to-one with the original feature channels. W0 represents the region involved in the convolution calculation, which is the first layer weight in the rotationally equivariant channel attention operation, and its function is to reduce the channel dimension. W1 represents the second layer weight in the rotationally equivariant channel attention operation, and its function is to restore the channel dimension. C represents the total channel dimension of the rotationally equivariant feature map, and r is the hyperparameter of channel dimension reduction. O represents F i The result of performing global average pooling on the directional feature map. O represents F i The result of performing global max pooling on the directional feature map.

[0036] The rotationally equivariant channel attention module takes the multi-layer feature map output by the rotationally equivariant feature extraction network as its input. First, it extracts features containing information in the same direction according to the orientation hyperparameter O of the rotationally equivariant convolutional network. Specifically, based on the channel dimension, starting from the 0th channel of the rotationally equivariant convolutional feature map, channel features are extracted at intervals of O to form the feature map in the 0th direction. This process is repeated O times to form O feature maps with information in different directions. Each feature map undergoes global average pooling and global maximum pooling, and then is subjected to two convolution operations and added together to obtain the channel attention for each direction feature map. The convolution kernel parameters used in this process are shared. The channel attention of each direction feature map is rearranged according to the channels of the original rotationally equivariant feature map and then passed through the Sigmoid function to generate the final rotationally equivariant channel attention.

[0037] In this embodiment, as Figure 1 As shown, the specific formula for the feature fusion process in step s2.2 is as follows:

[0038] A n =F n +F″ n+1

[0039] F′ n+1 =Interpolate(F′ n+1 ,2)

[0040] Interpolate is a linear interpolation upsampling operation that doubles the scale of the feature map by linearly interpolating adjacent pixels.

[0041] In this embodiment, as Figure 1 As shown, the specific formulas for the rotational isotropic channel attention and feature map Hadamard product process in steps s2.1 and s2.3 are as follows:

[0042] F′ ni =(a n1 , ..., a nC )⊙(f i1 , ..., f iC )

[0043] F′ n =(F′ n1 , ..., F′ nn )

[0044] Among them, (a n1 , ..., a nC ) for channel attention a n All elements, (f i1 , ..., f iC ) is the feature map F n The vector element at the i-th pixel position, F′ ni For feature map F n The result of the Hadamard product of the vector at the i-th pixel position and the channel attention.

[0045] In this embodiment, as Figure 1 As shown, the specific formula for generating the highest-level feature map in step s2.1 is as follows:

[0046]

[0047] In this embodiment, as Figure 1 As shown, the specific formula for the feature map generation process in step s2.3 is as follows:

[0048]

[0049] This invention fully utilizes the directional information, correlation between feature channels, and multi-scale information extracted by rotationally equivariant convolutional networks. It designs a feature fusion module with enhanced attention through rotationally equivariant channels to overcome the difficulties of insufficient semantic information extraction by rotationally equivariant networks and the variability of remote sensing target scales. This results in a multi-scale feature map after the fusion of enhanced attention through rotationally equivariant channels and multi-scale semantic information. Finally, target detection is performed on the feature map to obtain the final detection result, which has good application prospects.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A remote sensing image target detection method based on rotationally equivariant convolution channel attention enhancement and multi-scale feature fusion, characterized in that, include: Step S1: Select remote sensing images from the data as input to the rotational isotropic network, and obtain the rotational isotropic feature map of the input network image; Step S2: Input the feature map into the feature fusion module with rotational equivariant channel attention enhancement. Starting from the high-level feature map, it is fused with the enhanced feature map of the upper layer in sequence to obtain the current layer rotational equivariant channel attention. The correlation model is performed on the feature channels with the same directional information after fusion. The feature channel attention is rearranged according to the original feature channels and applied to the fused feature map to obtain the feature map after rotational equivariant channel attention enhancement. The specific formula for generating attention in a rotating isotropic channel is as follows: ； ； ； ； ； in, Represents rotational isomorphic feature maps The generated rotational isotropic channel attention, express function, express Channel attention generated from directional feature maps Indicates having Characteristics of directional information , These are the directional hyperparameters set in the rotational isotropic network configuration. This represents a multilayer perceptron, where the layers are fully connected. This represents a global average pooling layer, which takes one layer of the feature map as input and outputs the average value within the input region. This represents a global max pooling layer, which takes one layer of the feature map as input and outputs the maximum value of the input region. This indicates a channel rearrangement operation, which rearranges the channel attention in each direction according to the original feature channel structure, forming channel attention that corresponds one-to-one with the original feature channels. This refers to the region involved in the convolution calculation. The first layer of weights in the rotational isotropic channel attention operation. The second layer of weights in the rotational isotropic channel attention operation. The total channel dimension of the rotationally isomorphic feature map. Hyperparameters; right The result of performing global average pooling on the directional feature map. Indicates to The result of performing global max pooling on the directional feature map; Step S3: Perform feature target detection on the feature map obtained in step S2 to obtain the center point coordinates of the feature target, the length, width and angle of the feature target, and the category to which the feature target belongs.

2. The remote sensing target detection method based on rotational isovariant convolution channel attention enhancement and multi-scale feature fusion according to claim 1, characterized in that, Step S1 specifically includes the following steps: Step S11: Select remote sensing images from the data for preprocessing, and use the preprocessed images as input images for the rotational isotropic network. Step S12: Perform multiple rotational isovariant convolutions on the image input to the network to extract rotational isovariant features. Select the second, third, fourth, and fifth layer feature maps output by the network as subsequent inputs, and let their scales from largest to smallest be... , , , .

3. The remote sensing target detection method based on rotational isovariant convolution channel attention enhancement and multi-scale feature fusion according to claim 2, characterized in that, Step S2 specifically includes the following steps: Step S21, will In the input rotational isotropic channel attention module, the rotational isotropic channel attention corresponding to the feature map generated by the module is denoted as . The obtained channel attention With the current layer feature map conduct The product is used to obtain the feature map after feature enhancement. ; Step S22, from the feature map To feature map For the current layer feature map Upper-layer enhanced feature map Perform an upsampling operation to make its scale the same as the current layer feature map. The upsampled upper layer feature map is denoted as... , the current layer feature map The fused feature map is obtained by adding the corresponding pixels of the upsampled upper-layer feature map. ; Step S23, fuse the feature maps In the input rotational isotropic channel attention module, the rotational isotropic channel attention corresponding to the feature map generated by the module is denoted as . The obtained channel attention With the current layer feature map conduct The product is used to obtain the feature map after feature enhancement. Repeat steps S22 and S23 multiple times until the enhanced feature maps corresponding to all input feature maps are obtained. , and .

4. The remote sensing target detection method based on rotational isovariant convolution channel attention enhancement and multi-scale feature fusion according to claim 3, characterized in that, The specific formula for the feature fusion process in step S22 is as follows: ； ； in This is a linear interpolation upsampling operation that doubles the scale of the feature map by linearly interpolating neighboring pixels.

5. The remote sensing target detection method based on rotational isovariant convolution channel attention enhancement and multi-scale feature fusion according to claim 3, characterized in that, Rotational Isotropic Channel Attention and Feature Map in Steps S21 and S23 The specific formula for the product process is as follows: ； ； in, For channel attention All elements, For feature map The vector element at the i-th pixel position, For feature map The vector at the i-th pixel position and channel attention The result of accumulation.

6. The remote sensing target detection method based on rotational isovariant convolution channel attention enhancement and multi-scale feature fusion according to claim 3, characterized in that, The specific formula for generating the highest-level feature map in step S21 is as follows: 。 7. The remote sensing target detection method based on rotational isovariant convolution channel attention enhancement and multi-scale feature fusion according to claim 3, characterized in that: The specific formula for the feature map generation process in step S23 is as follows: 。 8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images