A Remote Sensing Image Object Detection Method Based on Single Anchor Box Sampling

Abstract

The present invention discloses a remote sensing image target detection method based on single anchor box sampling, which relates to the fields of computer vision and artificial intelligence. The method includes: (1) First, the feature map is divided into an N×N grid, and the coordinates of each grid are calculated. Then, a sampling box is generated with the unit length of a grid as the radius. When the grid point of a certain grid falls into the sampling box, the grid point is a positive sample responsible for predicting the current label, and a certain proportion of anchor boxes are laid on each positive sample; (2) Further, calculate the distances from the instance label coordinates of each category to the four boundaries of the corresponding sampling box, and then obtain a penalty value by using an error conversion formula for the obtained distances; (3) Further, introduce a vision transformer in the backbone network to enhance the feature expression ability and improve the overall detection accuracy of the network. The present invention can effectively improve the detection accuracy in remote sensing images and has broad application prospects in fields such as target detection.

Description

Technical Field

[0001] The present invention relates to a remote sensing image target detection method based on single anchor box sampling, belonging to the field of computer vision. Background Art

[0002] Object detection is the cornerstone of computer vision tasks and has achieved good detection performance in natural scenes. Object detection mainly focuses on two tasks: classification and regression, aiming to know what the detected object is and its relative position in the picture. Object detection in natural scenes mainly focuses on research based on the MS COCO and PASVAL VOC datasets, which cover 80 common categories in life and belong to horizontal box-based detection algorithms. However, in recent years, with the development of object detection, object detection based on remote sensing images has become a research hotspot. Due to its imaging characteristics, remote sensing images have the characteristics of high resolution, wide coverage, and clear targets, making them suitable for data monitoring. It has important value in security fields such as military and national defense.

[0003] Currently, remote sensing object detection is mainly divided into one-stage detection methods and two-stage detection methods. Specifically, one-stage detection methods usually preset a large number of anchor points on the image, and then use the clustering method to preset prior anchor boxes with different ratios, and refine the coordinates and categories of each anchor box multiple times. Finally, the selected anchor boxes are used as detection results; the two-stage method has a higher degree of refinement for anchor boxes than the one-stage method. Therefore, the detection accuracy of the latter is generally more accurate than that of the former, but the computational efficiency of the latter is lower. In real remote sensing scenarios, the distribution of its targets is different from that of general natural scenes. The remote sensing images cover many categories and the number of instances between categories is inconsistent, and the arrangement of objects is sometimes sparse and sometimes dense, and the target directions are variable, which brings great challenges to the detection task. Usually, horizontal detectors need to preset multiple anchor boxes with different ratios for each positive sample, and different scenarios require recalculating the size of the anchor boxes, with poor generalization and high complexity. At the same time, if the positive sample sampling strategy is not properly designed, the detection accuracy of individual categories will be very low. Therefore, object detection algorithms in natural scenes are difficult to achieve good results in remote sensing images. Summary of the Invention

[0004] In order to solve the deficiencies of the prior art, a remote sensing image target detection method based on single anchor box sampling aims to propose a new sampling strategy and penalty mechanism to alleviate the problem of sample imbalance caused by excessive differences in instances between categories in remote sensing images, reduce the complexity of the model, and reduce the detection box overlap ambiguity in densely arranged scenes, that is, the same positive sample may match multiple different instances.

[0005] The present invention adopts the following technical solution: A remote sensing image target detection method based on single anchor box sampling, the method comprising the following steps:

[0006] (1) First, the feature map is divided into an N×N grid, and the coordinates (g x , g y ) of each grid are calculated. Then, a sampling box is generated with a radius of r = 1 (the unit length of a grid). When the grid point (g x , g y ) of a certain grid falls into the sampling box, then this grid point is a positive sample responsible for predicting the current label. Finally, a certain proportion of anchor boxes are laid on each positive sample;

[0007] (2) On the basis of the sampling strategy described in (1), further calculate the distances from the coordinates (c x , c y ) of the instance label of each category to the four boundaries of the corresponding sampling box, and then obtain a penalty value by using an error conversion formula for the obtained distances;

[0008] (3) On the basis of (1) and (2) above, further introduce a vision transformer into the backbone network to enhance the feature expression ability and improve the overall detection accuracy of the network;

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0010] 1. The present invention designs a sampling strategy using a single anchor box. This sampling method can make the number of each positive sample stable, and can ensure that each instance has a corresponding positive sample, increasing the robustness of the detector. At the same time, this sampling method only needs to lay a single anchor box, which is more flexible and has fewer parameters compared to the method of laying three proportional anchor boxes in traditional object detection algorithms;

[0011] 2. The present invention designs a penalty strategy based on single anchor box sampling. This strategy will punish different positive samples according to the distance between the positive sample coordinates and the true label coordinates during sampling. The purpose is to make the loss value of the positive samples close to the true label large and the loss of the positive samples far away small. Further strengthen the ability of positive samples to identify the position of the true label and reduce the phenomenon of ambiguity of the detection box during prediction. This method does not introduce additional hyperparameters, has a low training time cost, and the effect is obvious.

[0012] 3. The present invention introduces a vision transformer mechanism to enhance the feature extraction ability of the entire network. This transformer in the form of an attention mechanism can better enrich the features of the target without introducing additional complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is the schematic diagram of single anchor box sampling of the present invention;

[0014] Figure 2 This is the schematic diagram of the punishment mechanism of the present invention;

[0015] Figure 3 This is the overall network block diagram of the vision transformer of the present invention. Specific embodiments

[0016] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0017] As Figure 1 shown, a remote sensing image target detection method based on a single anchor box sampling strategy includes the following parts:

[0018] (1) First, the feature map is divided into an N×N grid, and the coordinates (g x , g y ) of each grid are calculated. Then, a sampling box is generated with a radius of r = 1 (the unit length of a grid). When the grid point (g x , g y ) of a certain grid falls into the sampling box, then this grid point is a positive sample responsible for predicting the current label. Finally, a certain proportion of anchor boxes are laid on each positive sample;

[0019] (2) On the basis of the sampling strategy described in (1), further calculate the distances from the instance label coordinates (c x , c y ) of each category to the four boundaries of the corresponding sampling box, and then obtain a punishment value by using an error conversion formula for the obtained distances;

[0020] (3) Introduce a vision transformer module into the backbone network, and accept the output of the pyramid network structure as the input of this module, and then perform feature maps and send them into the prediction head of the network.

[0021] The details are elaborated as follows:

[0022] 1. Regarding the entire single anchor box sampling principle and the definition of the regression function, the specific description is as follows:

[0023] First, the feature map is divided into an N×N grid, and the coordinates (g x , g y ) of each grid are calculated. Then, a sampling box is generated with a radius of r = 1 (the unit length of a grid). When the grid point (g x , g y) If it falls within the sampling frame, then this grid point is a positive sample responsible for predicting the current label. Finally, a certain proportion of anchor boxes are laid on each positive sample. During the sampling stage, the anchor boxes generated by the grid points around the current label can match the current label to the greatest extent and obtain high-quality positive samples, and the number of positive samples is stable, without the need to set additional hyperparameters related to sampling.

[0024] When the position (g x , g y ) is within the grid range, this position is regarded as a positive sample, and the class label of this position obj = 1 (i.e., the foreground class). Otherwise, it is a negative sample and obj = 0 (i.e., the background class). In addition to the foreground-background distinction, this positive sample also corresponds to a five-dimensional vector t = (t x , t y , t w , t h , t θ ) as the regression parameter for training. To sum up, when the position (g x , g y ) is the center of the grid point of the positive sample, the coordinate function during regression is defined as Formula 1:

[0025]

[0026] Among them, σ represents the sigmoid() function, and the value range is [0, 1]; (t x , t y , t w , t h , t θ ) are the five coordinate offset amounts predicted by the network; (g x , g y ) are the coordinates of the current grid point; (p w , p h ) are the width and height of the label; finally, the coordinates and width and height (b x , b y , b w , b h ) of the actual center point are calculated through the predicted offset amounts.

[0027] 2. Regarding the principle of the penalty mechanism, the specific description is as follows:

[0028] As Figure 2 shown, based on the single-anchor-box sampling method described above, its sampling radius radius is set to 1. First, a sampling rectangle is formed with the coordinates (x, y) of the true label as the center and radius as the radius, and the upper-left coordinates of the current rectangle (target l , target t) and the bottom - right coordinates (target r , target b ). Then calculate the four distances from the grid coordinates (pos_grid_x, pos_grid_y) corresponding to the positive samples to the upper, lower, left, and right boundaries of the sampling rectangle, namely (top * , bottom * , left * , right * ). Finally, calculate the final penalty value through the above parameters.

[0029] The coordinates of the rectangle box generated by calculating the true label gt are described by Formula 2: [[ID=1^6]]

[0030]

[0031] The four distances from the positive - sample grid coordinates to the gt rectangle box boundary are described by Formula 3:

[0032]

[0033] Finally, the calculation form of the penalty value between the positive sample and the true - label value is described by Formula 4:

[0034]

[0035] Where Relu represents the activation function, which ensures that the whole result is positive. If the result is negative and does not pass through the activation function, gradient explosion will occur during backpropagation. The value range of the penalty value is between [0, 1]. Finally, taking the square root of the whole result is to attenuate the descending speed.

[0036] Since in the sampling stage, the same positive sample is very likely to match multiple label instances, this phenomenon is particularly obvious in densely arranged objects. Therefore, when the positive sample of each true - label instance is closer to the corresponding label, its loss contribution degree is greater, and vice versa. This method for object - detection tasks, without introducing additional complexity and hyperparameters and being able to adapt to different scenarios, improves the robustness of the whole network when alleviating the overlapping ambiguity phenomenon.

[0037] 3. Regarding the vision transformer module, the specific description is as follows:

[0038] The vision transformer reconstructs the two - dimensional image object into a series of two - dimensional image patches where (H, W) is the resolution of the original image, C represents the number of channels of the image, P represents the resolution of each image patch, and finally N = HW / P 2Represents the effective sequence length of the entire image, and this sequence length can be used as the input for other transformers for stacking processing. For example, for an image with a resolution of 300×300, which is divided into a sequence of 100×100 image patches, then the final effective sequence length N = 300×300 / 100 2 = 9, and the calculation method of the vision transformer is described by Formula 5:

[0039]

[0040] Where MSA represents the multi-head attention mechanism. MLP represents the multi-layer perceptron, which consists of Layer Normalization, fully connected layers, and activation layers. LN represents Layer Normalization for hierarchical regularization, which performs normalization processing on channels and spaces respectively.

[0041] Furthermore, the calculation of the multi-head attention mechanism is described by Formula 6:

[0042]

[0043] Where the projection parameters are respectively represented as and h represents the number of attention heads that need to be parallelized. d k = d v = d model / h.

[0044] The calculation expression of Attention is described by Formula 7:

[0045]

[0046] Where (Q, K, V) respectively represent the vectors of the query value (Query), key (Key), and value (Value). softmax is the normalization function. d k is the hyperparameter factor. Matrix operations are usually divided into dot product operations and addition operations. The additive attention mechanism uses a feed-forward network with a single hidden layer to calculate the compatibility function. The transformer shows that the theoretical complexity of dot product attention is similar to that of additive attention, but in practice, dot product attention has higher spatial efficiency and lower complexity.

[0047] To verify the effectiveness of the method of the present invention, the present invention uses the challenging remote sensing image dataset DOTA-v1.5 version for training and testing. Select the mainstream comparison methods of remote sensing image object detection algorithms in deep learning. The specific results are as follows:

[0048] Method 1: The method proposed by CHEN et al., reference "CHEN K, PANG J, WANG J, et al. Hybrid task cascade for instance segmentation; proceedings of the Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, F, 2019[C]."

[0049] Method 2: The method proposed by DING et al., reference "DING J, XUE N, LONG Y, et al. Learning roi transformer for oriented object detection in aerial images; proceedings of the Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, F, 2019[C]."

[0050] Method 3: The method proposed by YANG et al., reference "YANG X, YAN J, MING Q, et al. Rethinking rotated object detection with gaussian wasserstein distance loss; proceedings of the International Conference on Machine Learning, F, 2021[C]. PMLR."

[0051] Method 4: The method proposed by YANG et al., reference "YANG X, YANG X, YANG J, et al. Learning high-precision bounding box for rotated object detection via kullback-leibler divergence[J]. Advances in Neural Information Processing Systems, 2021, 34."

[0052] Method 5: The method proposed by He et al., reference "He K, Gkioxari G, Dollár P, et al. Maskr-cnn[C] / / Proceedings of the IEEE international conference on computervision. 2017:2961-2969."

[0053] As shown in Table 1, the method proposed in the present invention uses mAP 50 As an evaluation index in the DOTA-v1.5 dataset, compared with the mainstream SOTA algorithms, the detection accuracy of this method has a significant advantage.

[0054] Table 1 Test results with mainstream algorithms in DOTA-v1.5 (%)

[0055]

[0056] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A remote sensing image target detection method based on single anchor box sampling, characterized in that, It includes the following steps: (1) First, the feature map is divided into an N×N grid, and the coordinates (g x , g y ) of each grid are calculated. Then, a sampling frame is generated with a radius of r = 1, i.e., the unit length of one grid. When the grid point (g x , g y ) of a certain grid falls into the sampling frame, then this grid point is a positive sample responsible for predicting the current label. Finally, a certain proportion of anchor boxes are laid on each positive sample; (2) Based on the sampling strategy described in (1), further calculate the distances between the instance label coordinates (c x , c y ) of each category and the four boundaries of the corresponding sampling frame, and then use the error conversion formula for the obtained distances to get a penalty value; (3) On the basis of (1) and (2), further introduce a vision transformer into the backbone network to enhance the feature expression ability and improve the overall detection accuracy of the network; In step (2), a penalty mechanism module is further added on the basis of step (1). The structure of this module is as follows: First, a sampling rectangle is formed with the coordinates (x, y) of the true label as the center and radius as the radius, and the upper-left coordinates (target l , target t ) and the lower-right coordinates (target r , target b ) of the current rectangle are calculated; then the distances from the grid coordinates (pos_grid_x, pos_grid_y) corresponding to the positive samples to the upper, lower, left, and right boundaries of the sampling rectangle are calculated, namely (top * , bottom * , left * , right * ); finally, the final penalty value is calculated through the above parameters. Calculate the rectangular box coordinates generated by the true label gt as shown in Formula 2: The calculation method of the four distances from the positive sample grid coordinates to the boundaries of the gt rectangular box is as shown in Formula 3: Finally, the calculation method of the penalty value between the positive sample and the true label value is as shown in Formula 4: Where Relu represents the activation function, so as to ensure that the whole result is positive. If the result is negative and does not pass through the activation function, gradient explosion will occur during backpropagation.

2. The remote sensing image target detection method based on single anchor box sampling according to claim 1, wherein The specific principle in step (1) is as follows: First, an input image passes through a convolutional layer to obtain a feature map. The feature map is divided into an N×N grid, and the coordinates (g x , g y ) of each grid are calculated. Then, a sampling box is generated with a radius of r = 1, i.e., the unit length of one grid. When the grid point (g x , g y ) of a certain grid falls into the sampling box, then this grid point is a positive sample responsible for predicting the current label. Finally, a certain proportion of anchor boxes are laid on each positive sample. During the sampling stage, the anchor boxes generated by the grid points of the surrounding grids of the current label can match the current label to the greatest extent and obtain high-quality positive samples, and the number of positive samples is stable, without the need to additionally set hyperparameters related to sampling; If the position of a positive sample (g x , g y ) is within the grid range, this position is regarded as a positive sample, and the class label of this position obj = 1, that is, the foreground class. Otherwise, it is a negative sample and obj = 0, that is, the background class. In addition to the distinction between foreground and background, this positive sample also corresponds to a five-dimensional vector t = (t x , t y , t w , t h , t θ ) as the regression parameter for training. To sum up, the coordinate function during regression is shown in Equation 1: where σ represents the sigmoid function; (t x , t y , t w , t h , t θ ) respectively represent the five offsets predicted by the network; (g x , g y ) represents the grid coordinates; (p w , p h ) respectively represent the width and height of the true label; (b x , b y , b w , b h ) represents the actual coordinates finally predicted by the network.

3. A remote sensing image target detection method based on single anchor box sampling according to claim 1, characterized in that, In step (3), a vision transformer based on the attention mechanism is additionally introduced into the original network. The last layer of downsampling in the network is used as the input of the vision transformer, and the features after passing through the vision transformer are used as the output and sent to the upsampling of the network. The vision transformer module in step (3) is as follows: The vision transformer reconstructs two-dimensional image objects into a series of two-dimensional image patches where (H, W) is the resolution of the original image, C represents the number of channels of the image, P represents the resolution of each image patch, and finally N = HW / P 2 represents the effective sequence length of the entire image, and this sequence length is processed by stacking as the input of other transformers; The calculation method of the vision transformer is as shown in Formula 5: Where MSA represents the multi-head attention mechanism; MLP represents the multi-layer perceptron, which consists of Layer Normalization, fully connected layer and activation layer; LN represents Layer Normalization for hierarchical regularization, which normalizes the channels and space respectively; Further, the calculation description of the multi-head attention mechanism is as shown in Formula 6: Among them, the projection parameters respectively represent and h represents the number of attention heads that need to be parallel; d k = d v = d model / h; The calculation of Attention is as shown in Formula 7: Among them, (Q, K, V) respectively represent the vectors of query value (Query), key (Key), and value (Value); softmax is a normalization function; d k is a hyperparameter factor; matrix operations use dot product operations and additive operations. The additive attention mechanism uses a feed-forward network with a single hidden layer to calculate the compatibility function. In practice, dot product attention is more space-efficient and has lower complexity. Therefore, vision transformers use dot product operations to reduce complexity.

Images