Method and system for detecting rotating targets in aerial remote sensing images

Abstract

The application discloses a method and system for detecting a rotating target in an aerial remote sensing image. The method comprises: acquiring an aerial remote sensing image to be detected; inputting the aerial remote sensing image to be detected into a trained target detector to output a center point, a horizontal bounding box and a rotating bounding box of a target; the trained target detector is used for carrying out block processing on the aerial remote sensing image to be detected, carrying out feature extraction on each block image, and carrying out refinement processing on the extracted features to obtain the center point, the horizontal bounding box and the rotating bounding box of the target. A novel rotating box representation method is fused into an Anchor-Free target detection method, which not only discards the expensive cost of generating a candidate box, but also can generate a rotating bounding box surrounding a directional object in a simple and direct manner, and the detector can ensure the accuracy and speed.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image target detection, and in particular to a method and system for detecting rotating targets in aerial remote sensing images. Background Art

[0002] The statements in this section merely mention background art related to the present invention and do not necessarily constitute prior art.

[0003] Remote sensing imagery is widely used in fields such as disaster monitoring, resource surveys, and agricultural output estimation. Therefore, object detection in remote sensing images holds significant research value and significance, and has garnered increasing attention in recent years. Unlike general object detection, objects in remote sensing images possess unique characteristics, including non-axisymmetric and arbitrarily aligned objects, small and densely distributed objects, and complex and diverse background factors. Since object detection in remote sensing images still falls within the realm of target detection, researchers initially focused on modifying existing object detection networks to achieve rotational object detection.

[0004] Most advanced methods for detecting oriented objects rely on a framework of region proposals, such as the two-stage target detection method Fast / Faster-RCNN. In the first stage, a series of regions where targets may exist, namely candidate regions, are extracted from the input image, and then the candidate regions are classified in the second stage. However, the computational cost of generating oriented proposals is very expensive. Due to the multi-scale and multi-directional characteristics of targets in remote sensing images, the setting of the initial anchor has a great impact on the accuracy. In addition, small target detection has always been a difficult problem to overcome, especially in remote sensing images. As the network deepens, the resolution of the feature map decreases, and the step size of the anchor increases, which usually leads to the loss of sampling of small targets.

[0005] Mainstream methods typically rely on direct angle-based orientation regression, treating object detection in remote sensing images as a rotated object localization problem. These methods typically add an additional orientation parameter to a standard detector to perform direct orientation prediction. While these methods offer promising performance, they still face challenges, such as loss discontinuity and regression inconsistency caused by the boundedness of angle orientation and the definition of the orientation of a rotated bounding box. Consequently, these detectors may not accurately predict the orientation of oriented objects.

[0006] In the process of realizing the present invention, the inventors discovered that the prior art has the following technical problems:

[0007] Targets in remote sensing images are often small and dense, with various cluttered background environments. Most importantly, they are non-axisymmetric and oriented in arbitrary directions. Therefore, the mainstream bounding box direction regression method applied to general objects is not suitable for representing remote sensing image targets. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this paper provides a method and system for detecting rotated objects in aerial remote sensing images. It also proposes a simple oriented anchor-free object detection framework, namely Oriented CenterNet. This paper integrates a novel rotated box representation into anchor-free object detection methods. This method eliminates the expensive process of generating candidate boxes while generating rotated bounding boxes around oriented objects in a simple and direct manner. This detector achieves both accuracy and speed.

[0009] In a first aspect, the present invention provides a method for detecting rotating targets in aerial remote sensing images;

[0010] Rotating target detection method in aerial remote sensing images, including:

[0011] Acquire the aerial remote sensing image to be detected;

[0012] Input the aerial remote sensing image to be detected into the trained object detector, and output the center point, horizontal bounding box and rotation bounding box of the object;

[0013] The trained target detector is used to perform block processing on the aerial remote sensing image to be detected, perform feature extraction on each block image, and perform refinement on the extracted features to obtain the center point, horizontal bounding box and rotation bounding box of the target.

[0014] In a second aspect, the present invention provides a system for detecting rotating targets in aerial remote sensing images;

[0015] Rotating target detection system in aerial remote sensing images, including:

[0016] An acquisition module is configured to: acquire an aerial remote sensing image to be detected;

[0017] The target detection module is configured to: input the aerial remote sensing image to be detected into the trained target detector and output the center point, horizontal bounding box and rotation bounding box of the target;

[0018] The trained target detector is used to perform block processing on the aerial remote sensing image to be detected, perform feature extraction on each block image, perform feature fusion on the extracted feature map, and predict the fused features to obtain the center point, horizontal bounding box and rotation bounding box of the target.

[0019] In a third aspect, the present invention further provides an electronic device, comprising:

[0020] a memory for non-transitory storage of computer-readable instructions; and

[0021] a processor for executing said computer-readable instructions,

[0022] When the computer-readable instructions are executed by the processor, the method described in the first aspect is executed.

[0023] In a fourth aspect, the present invention further provides a storage medium that non-temporarily stores computer-readable instructions, wherein when the non-temporary computer-readable instructions are executed by a computer, the instructions of the method described in the first aspect are executed.

[0024] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, wherein the computer program is used to implement the method described in the first aspect when running on one or more processors.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] The present invention proposes a detection framework that can represent the geometric information of instances facing any direction, called Oriented CenterNet, which is an Anchor-Free oriented target detector with high accuracy and speed. Specifically, the present invention uses a more lightweight backbone network Swin-Tiny, which can obtain a global receptive field compared to the backbone network based on CNN, and can establish connections with other pixels. In addition, the present invention proposes a new and simpler six-parameter representation method, called the corner offset representation method, which is used to represent a rotating aerial target, directly converting the horizontal bounding box into a rotating bounding box in an extremely low-cost manner, and facilitating the calculation of the loss function. For small targets in remote sensing images, the network adopts an effective feature fusion and effective sampling method, and at the same time integrates center pooling into the detection head, which enhances the features of small targets, weakens the background noise, and makes the response value of the center point high. In addition, the present invention redefines the loss function, divides the regression loss into two parts: the horizontal bounding box and the rotating bounding box, so that the network can be trained under stronger supervision. Oriented CenterNet (Swin-Tiny-FPN) demonstrates the effectiveness of the proposed method on the commonly used and challenging remote sensing image dataset DOTA.

[0027] This paper proposes an effective remote sensing image target detector, Oriented CenterNet, which can be used to detect arbitrarily oriented targets in remote sensing images. The Transformer-based backbone network significantly improves network performance, center pooling makes the center key point features more prominent, and the corner offset representation achieves the horizontal to oriented transformation at a very low cost. The redefined loss function enables the network to be trained under stronger supervision. The present invention conducted a large number of experiments on DOTA, and the experimental results demonstrated the effectiveness of the target detection framework proposed in this paper. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0029] Figure 1 (a) and Figure 1 (b) show the detection results of the horizontal target detection method CenterNet in Example 1 on the remote sensing image;

[0030] Figure 1(c) and Figure 1(d) show the detection results of the Oriented CenterNet method for rotating target detection in Example 1 on remote sensing images;

[0031] Figure 2 (a) to Figure 2 (d) are visualizations of the Oriented CenterNet algorithm flow in Example 1;

[0032] Figure 3 This is the overall network architecture of Example 1;

[0033] FIG4 (a) and FIG4 (b) are the window shifting method based on self-attention in Example 1;

[0034] Figure 5 This is the central pooling principle and visualization process of Example 1;

[0035] Figure 6 This is a schematic diagram of corner point offset representation in Example 1;

[0036] Figures 7(a) to 7(c) are parameter descriptions of bounding box regression using the corner offset representation method in Example 1;

[0037] Figures 8(a) to 8(o) are examples of detection results of the Oriented CenterNet algorithm of Example 1;

[0038] FIG9(a) to FIG9(d) are comparisons of the heat map experiments with and without center pooling in Example 1. DETAILED DESCRIPTION

[0039] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0040] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0042] All data in this embodiment is obtained in compliance with laws and regulations and based on the consent of the user, and is used legally.

[0043] Example 1

[0044] This embodiment provides a method for detecting rotating targets in aerial remote sensing images;

[0045] Rotating target detection method in aerial remote sensing images, including:

[0046] S101: Acquire an aerial remote sensing image to be detected;

[0047] S102: Input the aerial remote sensing image to be detected into the trained object detector, and output the center point, horizontal bounding box and rotation bounding box of the object;

[0048] The trained target detector is used to perform block processing on the aerial remote sensing image to be detected, perform feature extraction on each block image, perform feature fusion on the extracted feature map, and predict the fused features to obtain the center point, horizontal bounding box and rotation bounding box of the target.

[0049] Furthermore, the S101: obtaining the aerial remote sensing image to be detected, which is collected by a remote sensing satellite.

[0050] Furthermore, the trained target detector has a network structure comprising:

[0051] The Swin Transformer backbone network, feature pyramid network FPN and prediction module are connected in sequence.

[0052] Furthermore, the Swin Transformer backbone network includes: a Patch Partition module and a Swin-Tiny module connected in sequence; the Patch Partition module is used to implement block processing of the aerial remote sensing image to be detected; and the Swin-Tiny module is used to implement feature extraction for each block image.

[0053] Furthermore, the feature pyramid network FPN is used to realize feature fusion of the extracted feature maps, and the prediction module is used to predict the fused features to obtain the center point, horizontal bounding box and rotation bounding box of the target.

[0054] Furthermore, the Swin-Tiny module includes: a first stage module Stage1, a second stage module Stage2, a third stage module Stage3 and a fourth stage module Stage4 connected in sequence;

[0055] The first stage module Stage1 includes: a fully connected layer, a first swin transformer unit and a second swin transformer unit connected in sequence;

[0056] The second stage module Stage2 includes: a first Patch Merging unit, a third Swin transformer unit, and a fourth Swin transformer unit connected in sequence; wherein the first Patch Merging unit is connected to the second Swin transformer unit;

[0057] The third stage module Stage3 includes: a second Patch Merging unit, a fifth Swin transformer unit, a sixth Swin transformer unit, a seventh Swin transformer unit, an eighth Swin transformer unit, a ninth Swin transformer unit, and a tenth Swin transformer unit connected in sequence; wherein the second Patch Merging unit is connected to the fourth Swin transformer unit;

[0058] The fourth stage module Stage4 includes: a third Patch Merging unit, an eleventh Swin transformer unit and a twelfth Swin transformer unit connected in sequence; wherein the third Patch Merging unit is connected to the tenth Swin transformer unit.

[0059] Furthermore, the first swin transformer unit and the second swin transformer unit are two consecutive swin transformer units;

[0060] The first swin transformer unit includes: a first linear layer LN, a multi-head attention mechanism W-MSA, a first adder, a second linear layer LN, a multi-layer perceptron MLP, and a second adder connected in sequence;

[0061] Among them, the second swin transformer unit includes: a third linear layer LN, a multi-head attention mechanism SW-MSA, a third adder, a fourth linear layer LN, a multi-layer perceptron MLP and a fourth adder connected in sequence; the third linear layer is connected to the second adder.

[0062] It should be understood that the Swin Transformer backbone network adopts a hierarchical design and consists of 4 stages ( Figure 3 Figure 4 shows the Swin-Tiny architecture. Each stage reduces the resolution of the input feature map, gradually expanding the receptive field, generating feature maps of varying sizes. The Swin Transformer restricts attention to each window (Figure 4(a)). Its shifted window partitioning method introduces connections between non-overlapping adjacent windows in the previous layer, making it effective for image classification and object detection. Figures 4(a) and 4(b) illustrate the self-attention-based window shifting method. By generating feature maps at different levels, the Swin Transformer model facilitates the use of advanced techniques for dense prediction, such as feature fusion within a Feature Pyramid Network (FPN).

[0063] Furthermore, the prediction module includes three parallel branches: a first branch, a second branch, and a third branch;

[0064] The first branch includes: a 3*3 convolutional layer, a 1*1 convolutional layer, and a center pooling layer connected in sequence; the first branch is used to output the target center point coordinate prediction value and heat map;

[0065] The second branch includes: a 3*3 convolutional layer and a 1*1 convolutional layer connected in sequence; the second branch is used to refine the output of the heat map to improve the positioning accuracy and achieve the goal of bringing the bounding box close to the target;

[0066] The third branch includes: a 7*7 convolutional layer and a 7*7 convolutional layer connected in sequence, and the third branch is used to output a horizontal bounding box and a rotation bounding box of the prediction result.

[0067] Furthermore, the prediction module adopts a corner offset representation algorithm to realize the prediction of the center point coordinates of the target, the prediction of the width and height of the horizontal bounding box, and the prediction of the offset of the two corner points of the rotated bounding box.

[0068] Furthermore, the corner point offset representation algorithm refers to:

[0069] Using a method with six parameters oriented bounding box representation method, where is the center point of the horizontal bounding box, Represents the horizontal coordinate of the center point of the horizontal bounding box, Represents the vertical coordinate of the center point of the horizontal bounding box, are the width and height of the horizontal bounding box, yes Relative to the upper left corner of the horizontal bounding box The horizontal offset of

[0070] From the symmetry, we know that yes Relative to the lower right corner The horizontal offset of yes Relative to the lower right corner of the horizontal bounding box The vertical offset of ; from the symmetry, we know that yes The vertical offset relative to the upper left corner;

[0071] The four vertices of the oriented bounding box 、 、 and It is expressed as follows:

[0072]

[0073] The heat map obtained by convolution is used to generate and filter the center point coordinates P. The parameters of the bounding box and the offset generated by two 7×7 convolutions are used to obtain four parameters. The prediction of the horizontal bounding box width and height and the offset prediction of the two corner points is used to represent an oriented bounding box in this way, by predicting the parameters of the horizontal bounding rectangle and inferring the corner offset parameters , to achieve the regression of the directional scheme.

[0074] like Figure 6 As shown, the coordinates of the center points of the rotated bounding box O and its circumscribed horizontal rectangle are , the upper left corner and the lower right corner are the two corner points of the horizontal bounding box, point - are the four corner points of the rotated bounding box respectively.

[0075] It should be understood that the prediction module contains three branches. The first branch first performs center pooling on the feature map to predict the center key point, outputs the center point heat map, and obtains the prediction of the center point; the second branch outputs the offset to solve the quantization problem in downsampling and generate a tighter bounding box; the third branch outputs the parameters of the rotated bounding box, including the width and height of the circumscribed rectangle, the offset With the various information of the horizontal bounding box and the prediction of the offset, the transformation from the horizontal bounding box to the rotated bounding box is finally achieved simply.

[0076] It should be understood that the trained target detector relies on the center key point to locate and classify objects, but the geometric center of the object does not necessarily represent a local area with important semantic information. The center pooling layer helps the center key point obtain more recognizable visual information in the target, and the additive effect on the subsequent center point heat map is more significant. The present invention predicts the center key point by finding the maximum sum of the horizontal and vertical responses of the center key point on the feature map. Figure 5 It shows the principle of the center pooling layer.

[0077] Center Pooling: The input is a feature map. To determine whether a pixel in the feature map is a center key point, it is necessary to find the maximum value in the horizontal and vertical directions of the current pixel, fill the maximum value in both directions, and add the two filled feature maps together. The pixel with the maximum value obtained by the addition operation is used as the center key point. The center pooling layer makes the target center point have a higher response value in the heatmap, which helps to better determine the center key point.

[0078] Furthermore, the training process of the trained target detector includes:

[0079] Constructing a training set, wherein the training set is an aerial remote sensing image with a horizontal bounding box and a rotation bounding box of a known target;

[0080] The training set is input into the target detector to train the target detector. When the total loss function value of the target detector no longer decreases, the training is stopped to obtain the trained target detector.

[0081] Furthermore, the total loss function is expressed as:

[0082] ;

[0083] in, is the classification loss, is the regression loss.

[0084] Expressed as:

[0085] ;

[0086] in, is the focal loss Focal Loss, represents the confidence of the predicted category, represents the truth value class, Represents the total number of samples in a batch.

[0087] The focal loss function is expressed as:

[0088] ;

[0089] in, Reflects the closeness to the true value category, The larger it is, the closer it is to the true category, that is, the more accurate the classification is. is an adjustable factor.

[0090] For the regression of the prediction box (Figure 7(a)-Figure 7(c)), Expressed as:

[0091] ;

[0092] in, represents the regression loss of the horizontal bounding box, represents the regression loss of the rotated bounding box, and Represent the weights of the two parts of loss, . The IoU (Intersection over Union) loss is used.

[0093] for The calculation of , using affine transformation to replace the IoU (Intersection over Union) calculation between polygons, is expressed as:

[0094]

[0095] in, , .

[0096] So Expressed as:

[0097]

[0098]

[0099] in, is the smooth loss function SmoothL1, is the numerical difference between the predicted box and the true box, ,in, is the coordinate difference between the center point of the true value frame and the predicted frame, is the difference between the width and height of the true value box and the predicted box, is the difference in horizontal and vertical offset between the true value box and the predicted box, is the center coordinate of the rectangle surrounding the true value box, is the width of the rectangle surrounding the true value box, is the height of the rectangle surrounding the true value box, and is a relative offset.

[0100] The network structure of the present invention is based on CenterNet, which is a rotating target detection algorithm for remote sensing images. Specifically, the network detection head output of the present invention consists of three parts:

[0101] The first part is the center point heatmap, which is used to determine the center point's location. Prior to this, center pooling, feature fusion, and effective sampling methods are the key to enhancing the detector's ability to detect small objects.

[0102] The second part is offset, which is used to solve the quantization problem during downsampling;

[0103] The third part is the parameters of the predicted box, which includes the width and height of the horizontal bounding box, and the offsets of the two rotated boxes. Here, a new six-parameter representation is used to represent a rotated bounding box, called the corner offset representation. The horizontal bounding box is converted into a rotated bounding box by predicting two offsets. In addition, the loss function is redefined so that the regression loss consists of two parts: horizontal and rotation. The experimental comparison is shown in Figures 1 (a), 1 (b), 1 (c), and 1 (d). The method of the present invention can greatly eliminate invalid information in the background content or foreground area, directly perform filtering operations through the heat map, and eliminate the problems of error detection and incomplete detection.

[0104] This paper evaluates Oriented CenterNet on a popular oriented object detection benchmark, the DOTA dataset. Oriented CenterNet surpasses the accuracy of CenterNet detectors using both ResNet50-FPN and Swin-Tiny-FPN backbones, achieving a maximum mean average performance (mAP) of 70.21% on the DOTA dataset.

[0105] Figures 1(a) and 1(b) show the detection results of the horizontal target detection method CenterNet on remote sensing images, and Figures 1(c) and 1(d) show the Oriented CenterNet rotation target detection method of the present invention, which can not only eliminate the expensive cost of region proposals, but also eliminate the problems of error filtering and incomplete detection.

[0106] like Figure 3 As shown in the figure, Swin-Tiny is selected as the backbone network. The first and second branches of the prediction module obtain the heat map and offset through 3×3 and 1×1 convolution layers (followed by Batchnorm and ReLU activation function), and the third branch obtains the partial parameter representation of the rotation box through two consecutive 7×7 convolutions. Figure 6 Schematic diagram of the corner point offset representation method of Example 1.

[0107] As a branch of target detection, oriented object detection has received widespread attention due to its wide application and important social research value. Most recent oriented target detection methods based on remote sensing images introduce a direction regression into the classic target detector. These methods mainly use angles to describe the direction of the rotated target, and achieve good performance in oriented detection of remote sensing images by predicting the rotation angle of the bounding box. Most methods are mainly based on the typical proposal-based target detection framework, placing anchors with different angles, scales and aspect ratios at each position. These densely rotated anchors are very expensive. In addition, some studies on "One-Stage" or Anchor-Free oriented object detection methods directly output the object category and rotation bounding box without the need for region proposals and RoI (region of interest) alignment operations.

[0108] General target detection methods (such as Faster RCNN) rely on horizontal bounding boxes to regress targets and cannot closely represent directional targets in remote sensing images because a horizontal bounding box may contain many redundant background factors or multiple objects, which affects the detector's final classification and positioning. To solve this problem, researchers have proposed a variety of representations for directional target bounding boxes. In terms of angle definition, the main ones are the OpenCV definition method and the long side definition method. Since angles and lengths and widths are periodic, the accuracy of the algorithm will also decrease. The present invention introduces a more effective rotation bounding box representation method and integrates it into the Anchor-Free conventional target detection method CenterNet.

[0109] The target detection method proposed in this paper is called Oriented CenterNet ( Figure 3 This invention is based on the CenterNet detector. CenterNet is an anchor-free general object detection algorithm based on keypoint detection. It eliminates the large computational overhead associated with anchors, thereby improving detection speed. This invention uses the Transformer-based backbone network, Swin-Tiny. Compared to other convolutional neural network-based backbone networks, this significantly reduces computational resources during training and is more efficient and scalable. In the prediction module, this invention adds center pooling before outputting the heatmap. This allows the center keypoints to form a more recognizable visual pattern within the object, making it easier to understand the center of the proposed region. To transition from horizontal bounding boxes to rotated bounding boxes, this invention uses a corner offset representation. Based on the horizontal bounding box, it only regresses the horizontal and vertical offsets of two corner points, generating oriented candidate boxes at a very low cost. The network process is visualized in Figures 2(a)-2(d). Figure 7(a) shows the ground-truth box, and Figure 7(b) shows the predicted box. The overlapping areas and intersections of polygons may be non-rectangular, and are shown in Figure 7(c). First, put Figure 2(a) into the backbone network and put the output into the prediction module. First, after center pooling and convolution, the center point heat map 2(b) is output. The target center point is predicted, and the parameters of the horizontal bounding box are predicted to obtain Figure 2(c). The final target category and positioning regression prediction Figure 2(d) are calculated and output by the corner point offset obtained by inference.

[0110] To evaluate the proposed method, we conducted extensive experiments on the DOTA dataset, a popular benchmark for oriented object detection.

[0111] DOTA is a large-scale dataset for evaluating the performance of oriented object detection in aerial imagery. It contains 2,806 images and 188,282 instances with oriented bounding box annotations, covering the following 15 object categories: bridge (BR), harbor (HA), ship (SH), plane (PL), helicopter (HC), small vehicle (SV), large vehicle (LV), baseball diamond (BD), ground runway (GTF), tennis court (TC), basketball court (BC), soccer field (SBF), roundabout (RA), swimming pool (SP), and storage tank (ST). The image sizes of the DOTA dataset range from 800×800 to 4000×4000. We use the training and validation sets for training, and the remaining sets for testing. We randomly resize and flip images to avoid overfitting. Detection accuracy is obtained by submitting test results to the DOTA evaluation server.

[0112] In order to verify the effectiveness of the detection framework of the present invention, a series of ablation experiments were conducted on the DOTA dataset.

[0113] Evaluation of activation functions. The convolution model mainly uses the simple and efficient ReLU. Although GELU did not improve the accuracy of the model in the experiment, GELU is smoother than ReLU and is a feasible solution. In terms of the number of activation functions, only the MLP (Multi-layer Perceptrons) module in each Transformer block has an activation function. The common practice of convolutional neural networks (CNN) is to attach an activation function after each convolution layer. The Transformer approach only retains the GELU activation function between two 1x1 layers. The present invention selects ResNet50 and Swin-Tiny for comparison. The experimental results are shown in Table 1. The results show that this approach also improves the accuracy of the model.

[0114] Table 1 Comparison of detection performance of different backbone networks using different activation functions

[0115]

[0116] Evaluation of Center Pooling. Center pooling can help enhance the response value of the center keypoint in the heatmap. Heatmap comparison tests, shown in Figures 9(a)-9(d), show that center pooling can increase the response value of the center point and provide better center point information features. When targets are small and dense, center pooling is less affected by background factors, resulting in higher center point response values ​​and a smaller response range, allowing for more accurate determination of the center point of an target. Table 2 evaluates the performance of center pooling. The results show that adding center pooling can improve the detector's detection of small targets.

[0117] Table 2 Performance evaluation of Center Pooling

[0118]

[0119] Among them, w / Center Pooling means there is a center pooling operation, and wo / Center Pooling means there is no center pooling operation; BR, SP, GTF, BD, and SV represent the categories of Bridge, Swimming pool, Ground trackfield, Baseball diamond, and Small vehicle, respectively.

[0120] Figures 9(a) and 9(c) are without Center Pooling, while Figures 9(b) and 9(d) are with Center Pooling. The comparison shows that Center Pooling can enhance the response value of the target center point and weaken the background effect.

[0121] Evaluation of loss function. The loss function of general detectors is composed of regression loss and classification loss, and the regression loss is only calculated from the final prediction box. The final rotation bounding box prediction is converted from the horizontal bounding box prediction, so the loss function consists of two parts. The performance evaluation of the loss function of the present invention is shown in Table 3. The experimental results show that the loss function of the present invention has a promoting effect on improving network performance. and The proportion of regression loss , Different, the present invention has carried out a lot of comparative experiments, as shown in Table 4, the experimental results show that , The network performance is best when

[0122] Table 3 Comparison of experimental results of different loss functions, w means participating in loss calculation, wo means not participating in loss calculation

[0123]

[0124] Table 4 Performance comparison experiments with different weights

[0125]

[0126] Evaluation of Different Backbone Networks. In computer vision, most common object detectors use ResNet50 / 101 as their backbone networks, primarily dominated by convolutional neural networks. We compared different backbone networks in our experiments, with the results shown in Table 5. The results show that in complex scenarios with varying scales, the detector with the Swin-Tiny backbone network achieves the best detection performance. Compared to ResNet50 and ResNet101, the mAP improves by 2.19% and 0.71%, respectively. This demonstrates that the choice of backbone network is crucial for improving network performance.

[0127] Table 5 Performance comparison of different backbone networks

[0128]

[0129] The detection results of the Oriented CenterNet algorithm using the Swin-Tiny-FPN backbone network on the DOTA dataset are visualized in Figures 8(a)-8(o). Table 6 also lists some experimental results using ResNet50-FPN and Swin-Tiny-FPN as the backbone networks. Our method achieved 68.02% and 70.21% mAP, respectively, representing improvements of +8.92% and +11.11% mAP, respectively, compared to the original CenterNet algorithm.

[0130] Table 6 compares the performance of the Oriented CenterNet proposed in this paper with that of the original method CenterNet on the DOTA dataset. The detection performance of some categories is listed.

[0131]

[0132] Example 2

[0133] This embodiment provides a system for detecting rotating targets in aerial remote sensing images;

[0134] Rotating target detection system in aerial remote sensing images, including:

[0135] An acquisition module is configured to: acquire an aerial remote sensing image to be detected;

[0136] The target detection module is configured to: input the aerial remote sensing image to be detected into the trained target detector and output the center point, horizontal bounding box and rotation bounding box of the target;

[0137] The trained target detector is used to perform block processing on the aerial remote sensing image to be detected, perform feature extraction on each block image, perform feature fusion on the extracted feature map, and predict the fused features to obtain the center point, horizontal bounding box and rotation bounding box of the target.

[0138] It should be noted that the acquisition module and target detection module described above correspond to steps S101 to S102 in Example 1. The examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the modules described above, as part of a system, can be executed in a computer system, such as a set of computer-executable instructions.

[0139] The descriptions of the various embodiments in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0140] Example 3

[0141] This embodiment also provides an electronic device, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes the method described in the above embodiment one.

[0142] Example 4

[0143] This embodiment further provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method described in the first embodiment is performed.

[0144] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for detecting rotating targets in aerial remote sensing images, characterized by: include: Acquire the aerial remote sensing image to be detected; Input the aerial remote sensing image to be detected into the trained object detector, and output the center point, horizontal bounding box and rotation bounding box of the object; The trained target detector is used to perform block processing on the aerial remote sensing image to be detected, perform feature extraction on each block image, and perform refinement on the extracted features to obtain the center point, horizontal bounding box and rotation bounding box of the target; The trained target detector has a network structure comprising: The Swin Transformer backbone network, feature pyramid network FPN and prediction module are connected in sequence; The Swin Transformer backbone network includes: a Patch Partition module and a Swin-Tiny module connected in sequence; the Patch Partition module is used to implement block processing of the aerial remote sensing image to be detected; the Swin-Tiny module is used to implement feature extraction for each block image; The feature pyramid network FPN is used to achieve feature fusion of the extracted feature maps, and the prediction module is used to predict the fused features to obtain the center point, horizontal bounding box and rotation bounding box of the target; The prediction module includes three parallel branches: a first branch, a second branch, and a third branch; The first branch includes: a 3*3 convolutional layer, a 1*1 convolutional layer, and a center pooling layer connected in sequence; the first branch is used to output the target center point coordinate prediction value and heat map; The second branch includes: a 3*3 convolutional layer and a 1*1 convolutional layer connected in sequence; the second branch is used to refine the output of the heat map to improve the positioning accuracy and achieve the goal of bringing the bounding box close to the target; The third branch includes: a 7*7 convolutional layer and a 7*7 convolutional layer connected in sequence, and the third branch is used to output a horizontal bounding box and a rotation bounding box of the prediction result.

2. The method for detecting rotating targets in aerial remote sensing images according to claim 1, wherein: The prediction module adopts a corner offset representation algorithm to realize the prediction of the center point coordinates of the target, the width and height of the horizontal bounding box, and the offset prediction of the two corner points of the rotation bounding box.

3. The method for detecting rotating targets in aerial remote sensing images according to claim 2, wherein: The corner offset representation algorithm is: Using a method with six parameters oriented bounding box representation method, where is the center point of the horizontal bounding box, Represents the horizontal coordinate of the center point of the horizontal bounding box, Represents the vertical coordinate of the center point of the horizontal bounding box, are the width and height of the horizontal bounding box, yes Relative to the upper left corner of the horizontal bounding box The horizontal offset of From the symmetry, we know that yes Relative to the lower right corner The horizontal offset of yes Relative to the lower right corner of the horizontal bounding box The vertical offset of ; from the symmetry, we know that yes The vertical offset relative to the upper left corner; The four vertices of the oriented bounding box 、 、 and It is expressed as follows: The heat map obtained by convolution is used to generate and filter the center point coordinates P. The parameters of the bounding box and the offset generated by two 7×7 convolutions are used to obtain four parameters. The prediction of the horizontal bounding box width and height and the offset prediction of the two corner points is used to represent an oriented bounding box in this way, by predicting the parameters of the horizontal bounding rectangle and inferring the corner offset parameters , to achieve the regression of the directional scheme.

4. The method for detecting rotating targets in aerial remote sensing images according to claim 1, wherein: The training process of the trained target detector includes: Constructing a training set, wherein the training set is an aerial remote sensing image with a horizontal bounding box and a rotation bounding box of a known target; The training set is input into the target detector to train the target detector. When the total loss function value of the target detector no longer decreases, the training is stopped to obtain the trained target detector.

5. The method for detecting rotating targets in aerial remote sensing images according to claim 4, wherein: The total loss function is expressed as: ； in, is the classification loss, is the regression loss; Expressed as: ； in, is the focal loss Focal Loss, represents the confidence of the predicted category, represents the truth value class, Indicates the total number of samples in a batch; The focal loss function is expressed as: ； in, Reflects the closeness to the true value category, The larger it is, the closer it is to the true category, that is, the more accurate the classification is. is an adjustable factor; For the regression of the prediction box Expressed as: ； in, represents the regression loss of the horizontal bounding box, represents the regression loss of the rotated bounding box, and Represent the weights of the two parts of loss, ; The intersection-over-union loss is used; for The calculation of , using affine transformation to replace the intersection and union ratio calculation between polygons, is expressed as: in, , ; So Expressed as: in, is the smooth loss function SmoothL1, is the numerical difference between the predicted box and the true box, , is the coordinate difference between the center point of the true value frame and the predicted frame, is the difference between the width and height of the true value box and the predicted box, is the difference in horizontal and vertical offset between the true value box and the predicted box, is the center coordinate of the rectangle surrounding the true value box, is the width of the rectangle surrounding the true value box, is the height of the rectangle surrounding the true value box, and is a relative offset.

6. The system for detecting rotating targets in aerial remote sensing images according to any one of claims 1 to 5, characterized in that: include: An acquisition module is configured to: acquire an aerial remote sensing image to be detected; The target detection module is configured to: input the aerial remote sensing image to be detected into the trained target detector and output the center point, horizontal bounding box and rotation bounding box of the target; The trained target detector is used to perform block processing on the aerial remote sensing image to be detected, perform feature extraction on each block image, perform feature fusion on the extracted feature map, and predict the fused features to obtain the center point, horizontal bounding box and rotation bounding box of the target.

7. An electronic device, comprising: a memory for non-transitory storage of computer-readable instructions; as well as a processor for executing said computer-readable instructions, When the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 5 is executed.

8. A storage medium, characterized in that: Computer-readable instructions are non-transitory stored, wherein when the non-transitory computer-readable instructions are executed by a computer, the instructions of the method according to any one of claims 1 to 5 are executed.

Images