A rotating target detection method for the field of remote sensing
By using (x,y,w,h,sin2θ,cos2θ) to describe the rotating bounding box and optimizing the loss function, the remote sensing rotating target detection network was improved, solving the problems of difficult angle boundary regression and poor detection effect, and improving the accuracy of remote sensing rotating target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2023-03-28
- Publication Date
- 2026-05-29
AI Technical Summary
Existing methods for detecting rotating targets in remote sensing images suffer from difficulties in angle boundary regression and poor detection performance during training, especially the five-parameter and eight-parameter methods, which have insufficient prediction accuracy during network training.
The rotation box is described using (x,y,w,h,sin2θ,cos2θ), and the loss function for network training is improved. The network training is optimized by using intersection-union loss and angle regression loss, and the network structure is improved to enhance detection accuracy.
This study improves the network training efficiency and final detection performance of remote sensing rotating target detection, solves the problems of difficult angle boundary regression and poor detection results, and improves the accuracy of rotating target detection.
Smart Images
Figure CN116310833B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, and relates to remote sensing target detection, specifically to a method for detecting rotating targets in the field of remote sensing. Background Technology
[0002] Target detection is a fundamental and important research area in remote sensing image processing. Its main research content is to identify the category and location of targets in an image, and it has wide applications in military operations, intelligent transportation, and smart cities. Remote sensing images are taken from aerial or satellite perspectives, often capturing ground targets from a top-down angle. Therefore, targets are usually not neatly arranged horizontally or vertically, but rather in arbitrary directions. This phenomenon is particularly common in targets with large aspect ratios (ships, bridges, etc.).
[0003] In recent years, with the rise of deep learning technology, object detection technology has made a qualitative leap. Representative methods include anchor-based single-stage object detection algorithms SSD and the YOLO series, as well as the two-stage object detection algorithm Faster R-CNN; there are also anchor-free methods, such as CornerNet and CenterNet. Most of these methods are based on horizontal bounding box (HBB) detection. However, for targets arranged at different angles in remote sensing images, the inferred bounding boxes cannot effectively represent the directionality of the targets, resulting in a lot of redundant information.
[0004] With the continuous integration of target detection technology and remote sensing, rotating target detection networks more suitable for remote sensing scenarios have emerged. Common representations include the five-parameter method and the eight-parameter method. The five-parameter method uses (x, y, w, h, θ) to represent a target box, where x and y represent the coordinates of the center point of the rotating box, w and h represent the width and height of the rotating box, and θ represents the angle value of the rotating box. The eight-parameter method uses (x1, y1, x2, y2, x3, y3, x4, y4), that is, the coordinates of the four corner points of the rotating box, to represent a rotating box. The angle definition of the five-parameter method includes the OpenCV definition method (0°~90°) and the long side definition method (-90°~90°). Since the target angle is mathematically periodic, while the angle defined in the network is fragmentary, the target boxes at the angle boundary are very difficult to regress during network training, resulting in poor detection performance. The eight-parameter method, due to the large number of prediction parameters, increases the difficulty of network training, leading to insufficient network prediction accuracy. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention provides a rotating target detection method for remote sensing. This method uses (x,y,w,h,sin2θ,cos2θ) to describe a rotating bounding box and improves the loss function for network training, thereby enhancing the training efficiency and final performance of the network.
[0006] This invention proposes a method for detecting rotating targets in the field of remote sensing, which specifically includes the following steps:
[0007] Step 1: Obtain the remote sensing image dataset, perform sliding segmentation on each high-resolution remote sensing image, and obtain the location label box information of each target in each segmented image;
[0008] The location label box for each target is represented in the form (x, y, w, h, sin2θ, cos2θ);
[0009] Where x and y represent the coordinates of the center point of the label box, w and h represent the width and height of the label box, and θ represents the angle value of the label box;
[0010] Step 2: Construct a rotating target detection network and input the segmented remote sensing image to predict each target box, obtaining a predicted box in the same form as the label box.
[0011] The network is specifically as follows:
[0012] First, the segmented remote sensing images are sequentially input into a convolutional residual neural network to obtain feature maps at different scales;
[0013] Then, the feature pyramid structure is used to fuse feature maps of different scales;
[0014] Finally, the fused feature map is input into the detection module, which outputs the predicted bounding box of each target in the current remote sensing image.
[0015] The predicted bounding box for each target is represented by (x,y,w,h,sin2θ,cos2θ);
[0016] Step 3: Optimize the network loss function from two aspects: cross-union loss and angle regression loss, and use it to train the rotating target detection network;
[0017] Specifically, the regression loss function takes the following form:
[0018]
[0019] in, The penalty term represents the distance between the center points of the label box and the prediction box. The farther the center points are, the larger the penalty term is. This indicates the difference in width and height parameters between the label box and the predicted box; the more significant the difference, the larger the value of this penalty term. This represents the intersection-union ratio (IU) of the label boxes and the predicted boxes. The smaller the IU, the larger the penalty term. This represents the angular difference between the label box and the predicted box; the greater the angular difference, the larger the penalty.
[0020] The calculation formula is as follows:
[0021]
[0022] In the formula, c gt c represents the center point of the label box. pred ρ(c) represents the center point of the prediction box. gt -c pred ρ represents the distance between the center points of the label box and the prediction box. diagonal ρ represents the diagonal distance of the smallest bounding rectangle that simultaneously contains the label box and the prediction box; where ρ diagonal The calculation of is not involved in the gradient update.
[0023] The calculation formula is as follows:
[0024]
[0025]
[0026]
[0027] In the formula This indicates the shape and proportion of the prediction box to focus on; ε represents the shape parameter of the prediction box; ε represents the penalty factor for the similarity in shape ratio between the prediction box and the label box; ω represents the penalty factor for the similarity in shape parameters between the prediction box and the label box; w gt ,h gt Indicates the width and height of the label box; w pred ,h pred represents the width and height of the predicted bounding box; RIoU represents the rotation, intersection, and union ratio of the label box and the predicted bounding box; where a and b are coefficients that do not participate in gradient update.
[0028] The calculation formula is as follows:
[0029]
[0030] The calculation formula is as follows:
[0031]
[0032] Where, θ gt The θ value represents the angle label of the target bounding box.pred This represents the angle value of the target bounding box predicted by the network.
[0033] Step 4: Using the trained rotating target detection network, inference is performed on the segmented remote sensing image to obtain the parameters sin2θ and cos2θ, and the angle θ is solved inversely.
[0034] Since the range of arcsinθ is The range of arccosθ is [0, π].
[0035] Therefore, by inversely solving sin2θ, we obtain:
[0036]
[0037]
[0038] Where, α sin This can be represented as the arcsine transform function in Python with an input of 2θ. sin The output value of the inverse solution.
[0039] Inverse solution of cos2θ yields:
[0040] 2θ cos =α cos or-α cos
[0041]
[0042] Where, α cos This refers to the inverse cosine transform function in Python when the input is 2θ. cos The output value of the inverse solution.
[0043] Choose one value each from the inverse solutions of sin2θ (β1, β2) and cos2θ (γ1, γ2) and combine them. Select the combination with the smallest difference from the four combinations {(β1, γ1), (β1, γ2), (β2, γ1), (β2, γ2)}, and let it be (β, γ). Then the angle θ obtained by the inverse solution is calculated as follows:
[0044]
[0045] Step 5: Integrate the angle θ, center point x, y, width, height w, h obtained from the inverse solution; and draw the final prediction box in the image through maximum suppression processing.
[0046] The advantages of this invention are:
[0047] (1) A rotating target detection method for remote sensing solves the problem that the target box at the angular boundary is difficult to regress during network training and the detection effect is poor. The detection network structure is optimized and the prediction of the θ parameter is changed from the previous prediction of the θ parameter alone to the prediction of two parameters, sin2θ and cos2θ.
[0048] (2) A rotating target detection method for remote sensing, which optimizes the network loss function from two aspects: cross-union loss and angle regression loss, thereby improving the accuracy of the rotating target detection network. Attached Figure Description
[0049] Figure 1 This is a flowchart of a rotating target detection method for the field of remote sensing according to the present invention;
[0050] Figure 2 This is a schematic diagram of the parameters of the regression loss function of this invention;
[0051] Figure 3 This is a schematic diagram of the rotation detection network structure of the present invention. Detailed Implementation
[0052] The specific implementation method of the present invention will be described in detail below with reference to the accompanying drawings.
[0053] This invention discloses a rotating target detection method for remote sensing. By improving the network structure for predicting target angles, the method transforms the prediction of the θ parameter from a single parameter to two parameters, sin2θ and cos2θ. At the same time, the network regression loss function is optimized from both cross-union loss and angle regression loss, thereby improving the prediction accuracy of the network.
[0054] The rotating target detection method used in the field of remote sensing, such as Figure 1 As shown, the specific steps include the following:
[0055] Step 1: Obtain the remote sensing image dataset, perform sliding segmentation on each high-resolution remote sensing image in the dataset, and obtain the location label box information of each target in each segmented image;
[0056] Each target rotation box is represented by (x, y, w, h, sin2θ, cos2θ);
[0057] Where x and y represent the coordinates of the center point of the rotating frame, w and h represent the width and height of the rotating frame, and θ represents the rotation angle of the rotating frame;
[0058] Step 2: Construct a rotating target detection network and input the segmented remote sensing image to predict each target box, obtaining a predicted box in the same form as the label box.
[0059] like Figure 3As shown, the network is specifically as follows:
[0060] First, the segmented remote sensing images are sequentially input into a convolutional residual neural network (ResNet_50) to obtain feature maps at different scales;
[0061] Then, the feature pyramid structure is used to fuse feature maps of different scales;
[0062] Finally, the fused feature map is input into the detection module, and each target is predicted in the form of (x,y,w,h,sin2θ,cos2θ).
[0063] The prediction box for each target is represented in the form of (x,y,w,h,sin2θ,cos2θ);
[0064] Step 3: Optimize the network loss function from two aspects: cross-union loss and angle regression loss, and use it to train the rotating target detection network;
[0065] Specifically, the cross-union ratio loss function is used for the regression of the rotated box parameters, which effectively aligns the evaluation indicators while suppressing boundary problems. The factors affecting the similarity between the predicted box and the label box mainly include the deviation of the center point (x,y), the deviation of the shape parameters w,h, and the deviation of the rotation angle θ.
[0066] like Figure 2 As shown, the regression loss function takes the following form:
[0067]
[0068] in, The penalty term represents the distance between the center points of the label box and the prediction box. The farther the center points are, the larger the penalty term is. This indicates the difference in width and height parameters between the label box and the predicted box; the more significant the difference, the larger the value of this penalty term. This represents the intersection-union ratio (IU) of the label boxes and the predicted boxes. The smaller the IU, the larger the penalty term. This represents the angular difference between the label box and the predicted box; the greater the angular difference, the larger the penalty.
[0069] The calculation formula is as follows:
[0070]
[0071] In the formula, c gt c represents the center point of the label box. pred ρ(c) represents the center point of the prediction box. gt -c pred ρ represents the distance between the center points of the label box and the prediction box. diagonalρ represents the diagonal distance of the smallest bounding rectangle that simultaneously contains the label box and the prediction box; where ρ diagonal The calculation of is not involved in the gradient update.
[0072] The calculation formula is as follows:
[0073]
[0074]
[0075]
[0076] In the formula This indicates the shape and proportion of the prediction box to focus on; ε represents the shape parameter of the prediction box; ε represents the penalty factor for the similarity in shape ratio between the prediction box and the label box; ω represents the penalty factor for the similarity in shape parameters between the prediction box and the label box; w gt ,h gt Indicates the width and height of the label box; w pred ,h pred represents the width and height of the predicted bounding box; RIoU represents the rotation, intersection, and union ratio of the label box and the predicted bounding box; where a and b are coefficients that do not participate in gradient update.
[0077] The RIoU calculation process uses the shoelace formula, which involves sorting the vertex set of the overlapping regions clockwise {(x1,y1),(x2,y2),…(x...}}. n ,y n The overlapping area Area is calculated using the following formula. overlap calculate:
[0078]
[0079] RIoU can then be calculated using the following formula:
[0080]
[0081] In the formula Area gt Area represents the area of the label box. pred This represents the area of the prediction box.
[0082] The calculation formula is as follows:
[0083]
[0084] The calculation formula is as follows:
[0085]
[0086] Where, θ gt The θ value represents the angle label of the target bounding box. pred This represents the angle value of the target bounding box predicted by the network.
[0087] The classification loss of the network uses Fcoal loss, which is expressed as follows:
[0088] Loss cls =-α t (1-p t ) γ log(p t )
[0089] In the formula, α t This invention uses α to represent the positive and negative sample balance factor. t =0.25, γ represents the balance factor between easy and difficult samples, p t The predicted probability represents the true category.
[0090] Step 4: Using the trained rotating target detection network, the parameters sin2θ and cos2θ can be obtained by reasoning on the segmented remote sensing image, and the angle θ can be obtained by inverse solution.
[0091] The parameters for the angle obtained through network inference are sin²θ and cos²θ. In this invention, the angle is defined using the longer side, since the range of arcsinθ in Python is... The range of arccosθ is [0, π].
[0092] Therefore, by inversely solving sin2θ, we obtain:
[0093]
[0094]
[0095] Where, α sin This can be represented as the arcsine transform function in Python with an input of 2θ. sin The output value of the inverse solution.
[0096] Inverse solution of cos2θ yields:
[0097] 2θ cos =α cos or-α cos
[0098]
[0099] Where, α cos This refers to the inverse cosine transform function in Python when the input is 2θ. cos The output value of the inverse solution.
[0100] Choose one value each from the inverse solutions of sin2θ (β1, β2) and cos2θ (γ1, γ2) and combine them. Select the combination with the smallest difference from the four combinations {(β1, γ1), (β1, γ2), (β2, γ1), (β2, γ2)}, and let it be (β, γ). Then the angle θ obtained by the inverse solution is calculated as follows:
[0101]
[0102] Step 5: Integrate the angle θ, center point x, y, width, height w, h obtained from the inverse solution; and draw the final prediction box in the image through maximum suppression processing.
[0103] Example:
[0104] First, acquire remote sensing images and perform preprocessing;
[0105] We acquire remote sensing datasets labeled with rotated bounding boxes, such as the DOTA remote sensing dataset, which uses the coordinates of four vertices to determine a rotated target. In this embodiment, we obtain the minimum bounding rectangle of the target based on the coordinates of the four vertices as the target bounding box, and express it in the form of (x, y, w, h, sin2θ, cos2θ). Considering the high resolution of remote sensing images, we perform sliding segmentation, dividing each original image into multiple 1024×1024 images for subsequent network training and testing.
[0106] Then, a rotating target detection network is constructed; feature maps of different scales are obtained based on a convolutional residual neural network and fused; the single prediction θ branch is transformed into two branches, sin2θ and cos2θ, to construct the detection module, so that each target box is predicted in the form of (x,y,w,h,sin2θ,cos2θ).
[0107] Next, the network loss function is optimized from two aspects: cross-union loss and angle regression loss. The improved loss function is used to train the rotating target detection network. The trained rotating target detection network is then used to perform remote sensing target detection, output inference results, and perform inverse solving.
[0108] Finally, the network prediction results are further processed: Since the input high-resolution image is subjected to sliding segmentation, the inference results from the same image are summarized in this step, and the redundant target boxes are processed by nonmaximum suppression to obtain the final rotated target boxes.
Claims
1. A method for detecting rotating targets in the field of remote sensing, characterized in that, Specifically, the steps include the following: First, acquire the remote sensing image dataset, perform sliding segmentation on each high-resolution remote sensing image, and obtain the location label box information of each target in each segmented image; The location label box for each target is... The formal representation; in Indicates the coordinates of the center point of the label box. This indicates the width and height of the label box. This indicates the angle value of the label box; Then, a rotating target detection network is constructed, and the segmented remote sensing image is input to predict each target box, resulting in a predicted box in the same form as the label box. Next, the network loss function is optimized from two aspects: crossover union loss and angle regression loss, and then used for training of the rotating target detection network; Finally, the trained rotating target detection network is used to infer parameters from the segmented remote sensing images. And solve the angle inversely. ; The angle obtained by integrating the inverse solution center point and width, height The final prediction box is drawn in the image through maximum suppression processing. The inverse solution angle The process is as follows: because The range of values is , The range of values is ; Therefore, for Inverse solution yields: in, This is represented by the arcsine transform function in Python, with the input being... The output value of the inverse solution; right Inverse solution yields: in, This indicates that the inverse cosine transform function in Python takes the input as... The output value of the inverse solution; Will The value obtained by inverse solution and The value obtained by inverse solution Choose one from each of the four combinations. Select the group with the smallest difference and let it be. The angle obtained by inverse solution The calculation is as follows: .
2. The method for detecting rotating targets in the field of remote sensing according to claim 1, characterized in that, The aforementioned rotating target detection network is specifically as follows: First, the segmented remote sensing images are sequentially input into a convolutional residual neural network to obtain feature maps at different scales; Then, the feature pyramid structure is used to fuse feature maps of different scales; Finally, the fused feature maps The input detection module is divided into two branches: one to complete the network's regression task and the other to complete the network's classification task. Modify the network regression branch, by Convolution changes the number of output channels, ultimately affecting the network output. The calculation formula is as follows: in Indicates the width of the feature map. Indicates the height of the feature map. This indicates the number of anchor boxes generated for each point on the feature map. Finally, the predicted bounding box for each target in the current remote sensing image is output. .
3. The method for detecting rotating targets in the field of remote sensing according to claim 1, characterized in that, The regression loss function takes the following form: in, The penalty term represents the distance between the center points of the label box and the prediction box. The farther the center points are, the larger the penalty term is. This indicates the difference in width and height parameters between the label box and the predicted box; the more significant the difference, the larger the value of this penalty term. This represents the intersection-union ratio (IU) of the label boxes and the predicted boxes; the smaller the IU, the larger the penalty term. This represents the angular difference between the label box and the predicted box; the greater the angular difference, the larger the penalty. The calculation formula is as follows: In the formula, Indicates the center point of the label box. Indicates the center point of the prediction box. This represents the distance between the center points of the label box and the prediction box. This represents the diagonal distance of the smallest bounding rectangle that simultaneously includes the label box and the prediction box; where The calculation of does not participate in the gradient update; The calculation formula is as follows: In the formula This indicates the shape and proportion of the prediction box to focus on; This indicates that the shape parameters of the prediction box are of interest. A penalty factor representing the degree of similarity in shape ratio between the predicted bounding box and the label box; A penalty factor representing the degree of similarity in shape parameters between the predicted bounding box and the label box; Indicates the width and height of the label box; Indicates the width and height of the prediction box; This represents the rotation, intersection, and union ratio (IoU) of the label box and the prediction box; where These are coefficients that do not participate in gradient updates; The calculation formula is as follows: The calculation formula is as follows: in, Indicates the angle label value of the target bounding box. This represents the angle value of the target bounding box predicted by the network.