A rotation-equivariant spatial local attention method for remote sensing image object detection

Abstract

The present invention discloses a rotation-equivariant spatial local attention remote sensing image target detection method, comprising: adopting multi-directional rotation convolution to construct a rotation-equivariant convolution module; cascading rotation-equivariant residual submodules to construct a feature tensor enhancement module; adopting neighborhood weight adaptive upsampling to construct an attention upsampling module; adopting a pyramid attention module to extract multi-scale feature tensors; adopting a learnable non-maximum suppression loss reconstruction loss function to train the network to guide the region proposal network to generate candidate regions; extracting features from the candidate regions and achieving target classification and positioning. The method of the present invention fully considers the impact of the scale and rotation of the target on the detection task, and the upsampling attention mechanism and the non-maximum suppression mechanism improve the detection accuracy and detection speed of multi-scale targets, and can effectively handle the detection problems of targets with large aspect ratios, dense distribution and arbitrary directions, and the network is lightweight and has strong small sample learning ability.

Description

Technical Field

[0001] The invention relates to remote sensing image target detection technology, in particular to a rotation equivariant spatial local attention remote sensing image target detection method. Background Art

[0002] Object detection aims to obtain the location and category of predefined categories of objects in an image. It is one of the most basic and important tasks in the field of computer vision. The object detection task for remote sensing images is of great significance for various civil applications such as geographic information system mapping, agriculture, transportation planning and navigation. Due to the wide spatial coverage of remote sensing images, objects in the images usually have large aspect ratios, dense distribution and arbitrary orientation. The above factors make it a challenging task to accurately identify and locate target objects in optical remote sensing images.

[0003] With the rapid development of deep learning, many remote sensing image target detection methods based on deep learning have been proposed. Optical remote sensing images are different from natural images that are usually taken from a horizontal perspective. Remote sensing images are usually bird's-eye views, which means that objects in the image are always arbitrarily oriented. The highly complex background and changing appearance further increase the difficulty of detecting objects. Ma et al. [Ma J, Shao W, Ye H, et al. Arbitrary-oriented scene text detection via rotation proposals [J]. IEEE Transactions on Multimedia, 2018, 20 (11): 3111-3122.] pre-designed a large number of anchor boxes with different angles, scales and aspect ratios to finally generate detection boxes with arbitrary directions, and showed good results in detecting sparsely distributed targets. However, since the directions of targets in remote sensing images are highly diverse and densely distributed, it is usually difficult to pair detection boxes with all targets in remote sensing images using detection boxes with limited directions. This type of method will significantly increase the computational complexity. In order to solve this problem, Ding et al. [Ding J, Xue N, Long Y, et al. Learning roi transformer for oriented object detection in aerial images [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2019: 2849-2858.] proposed ROI Transformer, which converts the horizontal candidate box generated by the region proposal network into a rotation candidate box, thereby avoiding the design of a large number of rotated anchor boxes and achieving good results. However, the traditional convolutional neural network framework has good generalization ability for translational equivariant features, but performs poorly in terms of rotational equivariant features and scale changes. The above method does not take into account the defects of the feature tensor enhancement module, so that the feature map extracted by the feature tensor enhancement module does not have rotational equivariance, which affects the subsequent process of identifying and locating the target object. Wang et al. [Wang Daolei, Du Wenbin, et al. A remote sensing image target detection method based on dense connection and feature enhancement: China, 10079059.4[P]. 2021-05-18.] By introducing a feature enhancement module with dilated convolution and a feature pyramid module, multi-scale feature tensors are extracted, which mainly addresses the problems of low resolution of remote sensing images and small target scale.This method achieves good detection results, but while the atrous convolution expands the receptive field, it does not have an attention mechanism for densely distributed targets. At the same time, the feature pyramid module uses the nearest neighbor interpolation method to upsample the feature tensor, and the upsampling result is not adaptive. Summary of the invention

[0004] The present invention discloses a rotation-equivariant spatial local attention remote sensing image target detection method, which fully considers the influence of the scale and rotation of the target on the detection task. The upsampling attention mechanism and non-maximum suppression mechanism improve the detection accuracy and speed of multi-scale targets, and can effectively handle the detection problems of targets with large aspect ratios, dense distribution and arbitrary directions. The network is lightweight and has strong small sample learning ability.

[0005] The technical solution to achieve the purpose of the present invention is: a rotationally equivariant spatial local attention remote sensing image target detection method, comprising the following steps:

[0006] In the first step, multi-directional rotation convolution is adopted to construct a rotation equivariant convolution module, which contains multiple directional rotation convolution layers and cascade feature tensor splicing operations;

[0007] The second step is to cascade the rotation equivariant residual submodules to construct a feature tensor enhancement module. That is, the rotation equivariant residual submodules are constructed using the first step of rotation equivariant convolution. The submodules are cascaded to form a feature tensor enhancement module to extract the rotation equivariant feature tensor.

[0008] The third step is to adopt neighborhood weight adaptive upsampling to build an attention upsampling unit. During sampling, the sampling weight is predicted by convolution, and the predicted pixel value comes from the combination of local spatial pixels rather than the nearest pixel.

[0009] The fourth step is to use the pyramid attention module to extract multi-scale feature tensors, that is, to use the attention upsampling method in the third step to build a pyramid attention module to extract multi-scale feature tensors;

[0010] The fifth step is to adopt a learnable non-maximum suppression loss reconstruction loss function to train the network and guide the region proposal network to generate candidate regions. That is, in the process of non-maximum suppression, pull loss and push loss are added. The pull loss is used to suppress the generation of low-score candidate regions of the same type, and the push loss hinders the generation of high-score candidate regions of different types.

[0011] The sixth step is to extract features from the candidate regions and classify and locate the targets. That is, the candidate regions are pooled into feature tensors of fixed size through pooling operations, and the fully connected layers and Softmax classifiers are cascaded for positioning and classification.

[0012] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned rotationally equivariant spatial local attention remote sensing image target detection method are implemented.

[0013] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned rotationally equivariant spatial local attention remote sensing image target detection method.

[0014] Compared with the prior art, the present invention has the following significant features: (1) rotation equivariant convolution is adopted to construct rotation equivariant residual submodule, which are cascaded to form rotation equivariant feature tensor enhancement module to extract rotation equivariant feature tensor; (2) the attention upsampling unit can aggregate context information in a large receptive field, and the pyramid attention module is constructed by using this module, so that the extracted multi-scale feature tensor is more accurate; (3) a learnable non-maximum suppression loss reconstruction loss function is adopted to train the network, and the region proposal network is guided to generate candidate regions, which can avoid false detection and missed detection; (4) by constructing an end-to-end detection model, it can effectively handle the detection problems of targets with large aspect ratios, dense distribution and arbitrary directions, and the network is lightweight and has strong small sample learning ability.

[0015] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic flow chart of the method of the present invention.

[0017] Figure 2 It is the structural diagram of the rotational equivariant residual unit.

[0018] Figure 3 It is the structural diagram of the attention upsampling unit.

[0019] Figure 4 Graph showing the detection results of the method of the present invention on the HRSC 2016 dataset.

[0020] Figure 5 This is a graph showing the detection results of the method of the present invention on the DOTA-v1.0 data set. DETAILED DESCRIPTION

[0021] Compared with the existing methods, the present invention applies rotational equivariant convolution to construct rotational equivariant residual submodules, which are cascaded to form a feature tensor enhancement module, so that the feature tensor obtained by the feature tensor enhancement module has rotational equivariance; neighborhood weight adaptive attention upsampling is applied to construct an attention upsampling unit, and the predicted pixel value comes from the combination of local space pixel points rather than the nearest pixel point; the attention upsampling method is used to construct a pyramid attention module, which can obtain a more accurate multi-scale feature tensor; by designing a learnable non-maximum suppression loss function to train the network, the region proposal network is guided to generate candidate regions, which can avoid false detection and missed detection of target objects to a certain extent.

[0022] Combination Figure 1 , the implementation process of the present invention is described in detail below, the steps are as follows:

[0023] The first step is to use multi-directional rotation convolution to construct a rotation equivariant convolution module, which contains multiple directional rotation convolution layers and cascade feature tensor splicing operations. The specific process is as follows:

[0024] (1) The original convolution kernel is rotated 8 times in different directions to obtain 8 convolution kernels sensitive to different directions. That is, the original convolution kernel is rotated counterclockwise around the center point by α r After the degree, the pixel values ​​of the rotated convolution kernel are resampled using bilinear interpolation. During the interpolation process, the pixel values ​​of the sampling points that fall outside the original convolution kernel are 0. The process of rotating the convolution kernel can be expressed as:

[0025]

[0026] in, represents the original convolution kernel, represents the convolution kernel after rotation transformation, n represents the size of the rotation convolution kernel, represents the rotation operation, α r Represents the rotation angle, denoted as R represents the discrete rotation direction, and the number of directions is set to 8.

[0027] (2) The input feature tensor is rotated through the convolutional layer in 8 directions to obtain 8 feature tensors sensitive to different directions, denoted as f i , i=1,2…8.

[0028] (3) These 8 f i By splicing along the channel dimension, we obtain a feature tensor that is equivariant to rotations in eight directions, which serves as the output of the rotational equivariant convolution module.

[0029] The second step is to cascade the rotation equivariant residual submodules to construct a feature tensor enhancement module. That is, the rotation equivariant residual submodules are constructed using the first step of rotation equivariant convolution. The submodules are cascaded to form a feature tensor enhancement module to extract the rotation equivariant feature tensor.

[0030] Each residual submodule is composed of rotational equivariant residual units of different scales. The residual submodules S1-S4 contain 3, 4, 6, and 3 residual units respectively. After the residual submodules S1-S4, multi-scale feature tensors with downsampling steps of 4, 8, 16, and 32 are obtained respectively. Among them, the rotational equivariant residual unit is as follows Figure 2 As shown, its structural process can be expressed as:

[0031] (1) The input feature tensor f passes through a 1×1 convolutional layer, a 3×3 rotational equivariant convolutional layer, and a 1×1 convolutional layer in sequence to obtain the feature tensor f′.

[0032] (2) The corresponding pixel values ​​of f′ and the feature tensor f″ obtained by the 1×1 convolution layer are added, and the obtained feature tensor is the output of the rotational equivariant residual unit.

[0033] The rotation equivariant residual submodules S1-S4 are cascaded module by module to form a feature tensor enhancement module, which is used to extract the rotation equivariant feature tensor. Its properties can be expressed as:

[0034]

[0035] Among them, I is the input of the feature tensor enhancement module, T r Represents the rotation transformation operation, L i It represents the convolutional layer operation in the feature tensor enhancement module, and M represents the number of convolutional layers in the feature tensor enhancement module.

[0036] The third step is to adopt neighborhood weight adaptive upsampling to construct an attention upsampling unit, such as Figure 3 As shown in the figure, the sampling weight is predicted by convolution during sampling, and the predicted pixel value comes from the combination of local spatial pixels rather than the nearest pixel. The specific process is:

[0037] (1) The number of channels of the input feature tensor is converted into dimensional, where δ is the upsampling rate and its value is set to ≥2; k up is the learned local convolution kernel size, the value is set ≥5.

[0038] (2) The δ of each pixel is converted into 2 The channels are rearranged into a two-dimensional δ×δ region, so that the size is The feature tensor of is rearranged into A feature tensor of size H, W, where H and W are the width and height of the feature map respectively.

[0039] (3) For each pixel corresponding to k up ×k up The local convolution kernel of size is normalized by Softmax function so that the sum of the convolution kernel values ​​is 1. The learned local convolution kernel is convolved with the input feature tensor. At this time, each pixel of the input feature tensor corresponds to δ 2 different local convolution kernels.

[0040] (4) A 1×1 convolutional layer is used to calculate a feature tensor of size 256×δ×H×δ×W with an upsampling rate of δ.

[0041] The fourth step is to use the pyramid attention module to extract the multi-scale feature tensor. That is, the attention upsampling method in the third step is used to build a pyramid attention module to extract the multi-scale feature tensor. The process is as follows:

[0042] (1) The feature tensors F1-F4 output by the rotation equivariant residual submodules S1-S4 in the feature tensor enhancement module are used as the input of the pyramid attention module. The number of channels of the feature tensors F1-F4 is converted to 256 dimensions through a 1×1 convolution layer, and four feature tensors with the same number of channels and different spatial scales are obtained, denoted as F′. i , i=1,2,3,4.

[0043] (2) The three feature tensors F′2, F′3, and F′4 with smaller spatial scales are respectively subjected to the attention upsampling unit to obtain the upsampled feature tensors, denoted as F i ″, i = 2, 3, 4, respectively, and the corresponding pixel values ​​of the feature tensors F′1, F′2, and F′3 are added to obtain a total of three fused feature tensors.

[0044] (3) The fused feature tensor and F′4 are respectively extracted through a 3×3 convolutional layer, and the obtained four feature tensors are used as the output of the pyramid attention module.

[0045] The fifth step is to use a learnable non-maximum suppression loss reconstruction loss function to train the network and guide the region proposal network to generate candidate regions. That is, pull loss and push loss are added in the non-maximum suppression process. The pull loss is used to suppress the generation of similar low-scoring candidate frames to avoid false detection, and the push loss hinders the generation of different high-scoring candidate frames to avoid missed detection. The non-maximum suppression loss can be defined as:

[0046] L nms =λ pull L pull +λ push L pu

[0047] Among them, L pull Indicates the pull loss, L push represents the push loss, λ pull and λ push Represents the weight coefficients, which are 0.5 and 0.05 respectively. The specific implementation process of the region proposal network is:

[0048] (1) The anchor frame generation layer allocates candidate frames to the feature tensor output by the pyramid attention module, setting the pixel area to 32 for each pixel. 2 , 64 2 , 128 2 , 256 2 , 512 2 , 15 types of candidate boxes with aspect ratios of 1:2, 1:1, and 2:1. These candidate boxes are scored through a 1×1 convolutional layer, and the candidate boxes are sorted from large to small according to the scores. The top 2000 candidate boxes are retained as the input of step (2).

[0049] (2) Select the candidate box with the highest current score as the current candidate box, and determine whether the true label corresponding to the candidate box is indexed with the existing candidate box. If so, calculate the pull loss between the current candidate box and the existing candidate box. Otherwise, calculate the value of the true label corresponding to the current candidate box index. The pull loss is defined as:

[0050]

[0051] Among them, ln(.) represents the logarithmic operation, Indicates the current candidate box. Indicates the candidate box with the highest score corresponding to the same true label, Indicates the score of the current candidate box, N t represents a fixed threshold value, whose value is 0.5, and IOU represents the intersection-union ratio of the current candidate box and other candidate boxes of the same target, which is defined as:

[0052]

[0053] Among them, area(·) represents the area calculation operator of the set, b i and b j Represents two different candidate boxes.

[0054] (3) The current candidate box is stored in the final retention list, and the IOU of the remaining candidate boxes and the current candidate box is calculated. When the IOU of the two is greater than the fixed threshold M, it is determined whether the true label values ​​corresponding to the two are the same: if they are the same, the candidate box compared with the current candidate box is deleted; if they are different, the push loss of the two candidate boxes is calculated and the compared candidate box is retained. The push loss is defined as:

[0055]

[0056]

[0057] Among them, b j Representation and Candidate boxes corresponding to different true label values, s j Represents the candidate box b j score.

[0058] (4) Delete the current candidate box in the score list, repeat steps (2) and (3), and continue to add candidate boxes that meet the requirements to the final retained list until the score list is empty. The final retained list is the candidate region finally obtained by the region proposal network.

[0059] The sixth step is to extract features from the candidate regions and classify and locate the target. The candidate regions are pooled into feature tensors of fixed size through pooling operations, and the fully connected layer and Softmax classifier are cascaded for positioning and classification. The specific implementation process of the pooling operation is as follows:

[0060] (1) Divide the candidate regions of different scales into 7×7 units of equal size.

[0061] (2) Each unit is divided into four equal parts. The center positions of the four regions are the sampling points of the pixel values ​​of the unit. The pixel values ​​of these four sampling points are calculated using the bilinear interpolation method.

[0062] (3) Perform a maximum pooling operation on each unit, that is, select the maximum value of the pixel values ​​of the four sampling points as the pixel value of the unit, and finally obtain a candidate area of ​​7×7 size.

[0063] After the pooling operation, the candidate region feature tensor of fixed spatial size is obtained and classified through a fully connected layer and Softmax classifier to obtain the classification result of the target. At the same time, a fully connected layer is used to regress the offset values ​​including the center point coordinates, width, height and counterclockwise rotation angle of the candidate region to obtain the accurate positioning result of the target.

[0064] The effect of the present invention can be further illustrated by the following simulation experiment:

[0065] Simulation conditions

[0066] The simulation experiment uses two sets of optical remote sensing image data: HRSC 2016 dataset and DOTA-v1.0 dataset. All images of HRSC 2016 dataset are from Google Earth. The training set, validation set and test set contain 436, 181 and 444 images respectively, with a total of 2976 targets, and the image size ranges from 300×300 to 1500×900 pixels. All images of DOTA-v1.0 dataset are from Google Earth, GF-2 and JL-1 satellites, and aerial images provided by CycloMedia Technology BV. The training set, validation set and test set contain 1411, 458 and 937 images respectively, with a total of 403318 targets, and the image size ranges from 800×800 to 20000×20000 pixels. Both datasets use the annotation format of oriented bounding boxes. In the experiment, all images in the HRSC 2016 dataset were resized to 800×512; all images in the DOTA-v1.0 dataset were cropped into image blocks of 1024×1024, and the step size was set to 824. The original training set and the original validation set were used as training sets, and the original test set was used as the test set. The two groups of experiments used the average detection accuracy (AP) within the class and the average detection accuracy (mAP) between classes as evaluation indicators. In addition, the comparison methods include: Region of Interest Transformer (RoI Transformer, RoI Trans.) method, Dynamic Refinement Network (DRN) method, Center Probability Map (CenterMap) method, Refined Rotation RetinaNet (Refined Rotation RetinaNet, R 3 Det) method and the Single-shot Alignment Network (S 2 A-Net) method.

[0067] In the experiment, the network optimizer uses the SGD optimizer, the initial learning rate is 0.01, and the learning rate is divided by 10 each time it decays. The momentum and weight decay are 0.9 and 0.0001 respectively. In addition, the network trains the model in the first 12 epochs of DOTA-v1.0 and the first 36 epochs of HRSC2016. Other network hyperparameter configurations are summarized in Table 1. The simulation experiments are all completed in the Linux operating system using Python3.8+pytorch1.8+cuda11.2.

[0068] Table 1 Network hyperparameter configuration

[0069]

[0070]

[0071] Analysis of simulation experiment results

[0072] Table 2-3 shows the detection accuracy (%) of the simulation experiment of the method of the present invention on the HRSC 2016 and DOTA-v1.0 data sets.

[0073] Table 2 Detection results of different methods on HRSC 2016 dataset

[0074] index RoI Trans. DRN CenterMap <![CDATA[R 3 It]]> <![CDATA[S 2 A-Net]]> The present invention mAP 86.20 87.17 87.96 89.26 90.17 90.58

[0075] Table 3 Detection results of different methods on DOTA-v1.0 dataset

[0076] Class RoI Trans. DRN CenterMap <![CDATA[R 3 It]]> <![CDATA[S 2 A-Net]]> The present invention 1 88.64 88.91 88.88 89.49 89.11 89.18 2 78.52 80.22 81.24 81.17 82.84 82.90 3 43.44 43.52 53.15 50.53 48.37 50.62 4 75.92 63.35 60.65 66.10 71.11 71.47 5 68.81 73.48 78.62 70.92 78.11 79.26 6 73.68 70.69 66.55 78.66 78.39 78.95 7 83.59 84.94 78.10 78.21 87.25 87.36 8 90.74 90.14 88.83 90.81 90.83 90.86 9 77.27 83.85 77.80 85.26 84.90 85.62 10 81.46 84.11 83.61 84.23 85.64 85.53 11 58.39 50.12 49.36 61.81 60.36 61.34 12 53.54 58.41 66.19 63.77 62.60 63.70 13 62.83 67.62 72.10 68.16 65.26 66.35 14 58.93 68.60 72.36 69.83 69.13 70.67 15 47.67 52.50 58.70 67.17 57.94 54.66 mAP 69.56 70.70 71.74 73.74 74.12 74.56

[0077] From the experimental results, we can find that the detection accuracy of the two datasets can be significantly improved by using the method of the present invention. On the HRSC 2016 dataset, the average accuracy of the method of the present invention is 90.58%. Figure 4 Compared with other methods, the proposed method can obtain better detection results, mainly due to the proposed rotation equivariant feature tensor enhancement module can more accurately extract the direction information of the target. On the DOTA-v1.0 dataset, the average detection accuracy of the proposed method is 74.56%. The detection image obtained by the proposed method is shown in Figure 5 As shown. Compared with other methods, the method of the present invention has better detection effect on basketball courts and ground track and field fields, two classes with large scale changes, thanks to the attention pyramid module proposed in the present invention, which can more accurately extract multi-scale feature information; it has better detection effect on ships and vehicles, two classes with large aspect ratios, mainly thanks to the non-maximum suppression loss function proposed in the present invention, which can delete redundant detection frames and make the positioning of detection frames more accurate. The above results fully demonstrate that the method of the present invention can effectively learn the feature information of remote sensing images and has high detection performance.

[0078] The above is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A rotationally equivariant spatial local attention remote sensing image target detection method, characterized in that: The following steps are involved: The first step is to use multi-directional rotation convolution to construct a rotation equivariant convolution module, which contains multiple directional rotation convolution layers and cascade feature tensor splicing operations, specifically: (1) The original convolution kernel is rotated 8 times in different directions to obtain 8 convolution kernels sensitive to different directions. The rotation transformation process of the convolution kernel is expressed as: in, represents the original convolution kernel, represents the convolution kernel after rotation transformation, n represents the size of the rotation convolution kernel, represents the rotation operation, α r Represents the rotation angle, denoted as R represents the discrete rotation direction, and the number of directions is set to 8; (2) The input feature tensor is rotated through the convolutional layer in 8 directions to obtain 8 feature tensors sensitive to different directions, denoted as f i , i=1,2…8; (3) These 8 f i Splicing along the channel dimension, we get the feature tensor that is equivariant to rotation in 8 directions, which is used as the output of the rotation equivariant convolution module; The second step is to cascade the rotation equivariant residual submodules to construct a feature tensor enhancement module. That is, the rotation equivariant residual submodules are constructed using the first step of rotation equivariant convolution. The submodules are cascaded to form a feature tensor enhancement module to extract the rotation equivariant feature tensor. The third step is to adopt neighborhood weight adaptive upsampling to build an attention upsampling unit. During sampling, the sampling weight is predicted by convolution, and the predicted pixel value comes from the combination of local spatial pixels rather than the nearest pixel. The fourth step is to use the pyramid attention module to extract multi-scale feature tensors, that is, to use the attention upsampling method in the third step to build a pyramid attention module to extract multi-scale feature tensors; The fifth step is to adopt a learnable non-maximum suppression loss reconstruction loss function to train the network and guide the region proposal network to generate candidate regions. That is, in the process of non-maximum suppression, pull loss and push loss are added. The pull loss is used to suppress the generation of similar low-score candidate regions, and the push loss hinders the generation of different high-score candidate regions. The sixth step is to extract features from the candidate regions and classify and locate the targets. That is, the candidate regions are pooled into feature tensors of fixed size through pooling operations, and the fully connected layers and Softmax classifiers are cascaded for positioning and classification.

2. The rotation-equivariant spatial local attention remote sensing image target detection method according to claim 1, characterized in that: The second step is to cascade the rotation equivariant residual submodules to construct a feature tensor enhancement module. That is, the rotation equivariant residual submodules are constructed using the first step of rotation equivariant convolution. The submodules are cascaded to form a feature tensor enhancement module to extract the rotation equivariant feature tensor. The properties of the rotationally equivariant feature tensor augmentation module can be expressed as: Among them, I is the input of the feature tensor enhancement module, T r Represents the rotation transformation operation, L i It represents the convolutional layer operation in the feature tensor enhancement module, and M represents the number of convolutional layers in the feature tensor enhancement module.

3. The rotation equivariant spatial local attention remote sensing image target detection method according to claim 1, characterized in that: The third step is to adopt neighborhood weight adaptive upsampling to build an attention upsampling unit. During sampling, the sampling weight is predicted by convolution. The predicted pixel value comes from the combination of local spatial pixels rather than the nearest pixel. The specific process is as follows: (1) The number of channels of the input feature tensor is converted into dimensional, where δ is the upsampling rate and its value is set to ≥2; k up is the learned local convolution kernel size, the value is set to ≥5; (2) The δ of each pixel is converted into 2 The channels are rearranged into a two-dimensional δ×δ region, so that the size is The feature tensor of is rearranged into A feature tensor of size, where H and W are the width and height of the feature map respectively; (3) For each pixel corresponding to k up ×k up The local convolution kernel of size is normalized by Softmax function so that the sum of the convolution kernel values ​​is 1; the learned local convolution kernel is convolved with the input feature tensor. At this time, each pixel of the input feature tensor corresponds to δ 2 Different local convolution kernels; (4) A 1×1 convolutional layer is used to calculate a feature tensor of size 256×δ×H×δ×W with an upsampling rate of δ.

4. The rotation equivariant spatial local attention remote sensing image target detection method according to claim 1, characterized in that: The fourth step is to use the pyramid attention module to extract the multi-scale feature tensor. That is, the attention upsampling method in the third step is used to build a pyramid attention module to extract the multi-scale feature tensor. The process is as follows: (1) The feature tensors F1-F4 output by the rotation equivariant residual submodules S1 to S4 in the feature tensor enhancement module are used as the input of the pyramid attention module. The number of channels of the feature tensors F1-F4 is converted to 256 dimensions through a 1×1 convolution layer, and four feature tensors with the same number of channels and different spatial scales are obtained, denoted as F' i , i=1,2,3,4; (2) The three feature tensors F'2, F'3, and F'4 with small spatial scales are respectively subjected to the attention upsampling unit to obtain the upsampled feature tensors, denoted as F' i ', i = 2, 3, 4, respectively, add the corresponding pixel values ​​of the feature tensors F'1, F'2, and F'3 to obtain a total of 3 fused feature tensors; (3) The fused feature tensor and F'4 are extracted through a 3×3 convolutional layer, and the obtained four feature tensors are used as the output of the pyramid attention module.

5. The rotation-equivariant spatial local attention remote sensing image target detection method according to claim 1, characterized in that: The fifth step is to use a learnable non-maximum suppression loss reconstruction loss function to train the network and guide the region proposal network to generate candidate regions. That is, pull loss and push loss are added in the non-maximum suppression process. The pull loss is used to suppress the generation of similar low-scoring candidate frames to avoid false detection, and the push loss hinders the generation of different high-scoring candidate frames to avoid missed detection. The non-maximum suppression loss can be defined as: L nms =λ pull L pull +λ pull L push Among them, L pull represents the pull loss, λ push represents the push loss, λ pull and λ push Represents the weight coefficient; the specific implementation process of the region proposal network is: (1) The anchor frame generation layer allocates candidate frames to the feature tensor output by the pyramid attention module, setting the pixel area to 32 for each pixel. 2 , 64 2 , 128 2 , 256 2 , 512 2 , 15 types of candidate boxes with aspect ratios of 1:2, 1:1, and 2:1; these candidate boxes are scored through a 1×1 convolutional layer, and the candidate boxes are sorted from large to small according to the scores, and the top 2000 candidate boxes are retained as the input of step (2); (2) Select the candidate box with the highest current score as the current candidate box, and determine whether the true label corresponding to the candidate box is indexed with the existing candidate box. If so, calculate the pull loss between the current candidate box and the existing candidate box. Otherwise, calculate the value of the true label corresponding to the current candidate box index. The pull loss is defined as: Among them, ln(·) represents the logarithmic operation, Indicates the current candidate box. Indicates the candidate box with the highest score corresponding to the same true label, Indicates the score of the current candidate box, N t represents a fixed threshold, and IOU represents the intersection-over-union ratio of the current candidate box and other candidate boxes of the same target, which is defined as: Among them, area(·) represents the area calculation operator of the set, b i and b j Represents two different candidate boxes; (3) The current candidate box is stored in the final retention list, and the IOU of the remaining candidate boxes and the current candidate box is calculated. When the IOU of the two is greater than the fixed threshold N t When the real label values ​​corresponding to the two are the same, it is determined whether they are the same: if they are the same, the candidate box compared with the current candidate box is deleted; if they are different, the push loss of the two candidate boxes is calculated and the compared candidate box is retained; the push loss is defined as: Among them, b j Representation and Candidate boxes corresponding to different true label values, s j Represents the candidate box b j score; (4) Delete the current candidate box in the score list, repeat steps (2) and (3), and continue to add candidate boxes that meet the requirements to the final retained list until the score list is empty. The final retained list is the candidate region finally obtained by the region proposal network.

6. The rotation equivariant spatial local attention remote sensing image target detection method according to claim 5, characterized in that: λ pull and λ push are 0.5 and 0.05 respectively, N t The value is 0.

5.

7. The rotation-equivariant spatial local attention remote sensing image target detection method according to claim 1, characterized in that: The sixth step is to extract features from the candidate regions and classify and locate the target. The candidate regions are pooled into feature tensors of fixed size through pooling operations, and the fully connected layer and Softmax classifier are cascaded for positioning and classification. The specific implementation process of the pooling operation is as follows: (1) Divide candidate regions of different scales into 7×7 units of equal size; (2) Each unit is divided into four equal parts. The center positions of the four regions are the sampling points of the pixel values ​​of the unit. The pixel values ​​of these four sampling points are calculated using the bilinear interpolation method. (3) Perform a maximum pooling operation on each unit, that is, select the maximum value of the pixel values ​​of the four sampling points as the pixel value of the unit, and finally obtain a candidate region of size 7×7; After the pooling operation, the candidate region feature tensor of fixed spatial size is obtained and classified through a fully connected layer and a Softmax classifier to obtain the classification result of the target; at the same time, a fully connected layer is used to regress the offset values ​​including the center point coordinates, width, height and counterclockwise rotation angle of the candidate region to obtain the accurate positioning result of the target.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the rotation-equivariant spatial local attention remote sensing image target detection method described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the rotationally equivariant spatial local attention remote sensing image target detection method described in any one of claims 1 to 7 are implemented.

Images