An aerial image target oblique frame detection method and device

Abstract

This application discloses a method and apparatus for detecting oblique bounding boxes of aerial images, belonging to the field of target detection technology. Specifically, it includes: extracting a first feature map of the aerial image using a backbone network, and obtaining a second feature map of the first feature map using a feature pyramid network; generating horizontal candidate boxes of the second feature map using a region generation network, and extracting horizontal feature maps of the horizontal candidate boxes using a feature extractor; obtaining a first rotated rectangle from the horizontal feature map using a rotated rectangle learner, and setting the first rotated rectangle as the initial rotated rectangle; iteratively executing a rotated rectangle update step for a preset number of iterations; determining whether the current iteration number has reached the preset number of iterations; if the current iteration number has not reached the preset number of iterations, setting a higher threshold, setting the second rotated rectangle as the initial rotated rectangle, and executing the rotated rectangle update step; otherwise, using the current second rotated rectangle as the final rotated rectangle.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a method and apparatus for detecting oblique frames of targets in aerial images. Background Technology

[0002] Object detection, also known as object extraction, is a research area in computer vision, with wide applications in facial recognition, intelligent transportation, autonomous driving, and healthcare. The core problem that computer vision aims to solve is how to extract information from images that can be understood by computers. Especially in complex scenes where multiple targets need to be processed in real time, automatic object extraction and recognition become particularly important.

[0003] In existing object detection tasks, deep learning network models are often used to automatically learn image features, and then manual correction is used to improve the object detection accuracy of the deep learning network models. However, in practical applications, object detection scenarios are diverse, and there are often object detection scenarios where the target is not horizontal or vertical. Manually correcting the detection boxes is time-consuming and labor-intensive. Currently, the mainstream algorithms used are mainly the one-stage YOLO series and the two-stage Faster-RCNN series. The YOLO series algorithms make predictions based on the information of the entire image, directly classifying and locating all targets in the entire image without using complex frames. The detection speed is fast, but the object detection accuracy is relatively low. The Faster-RCNN series algorithms first select a large number of target candidate regions in the image to be tested, and then detect the candidate regions. The detection accuracy is high, but the speed is slow, making it difficult to achieve real-time object detection. Summary of the Invention

[0004] This application provides a method for detecting target oblique frames in aerial images, which solves the bottleneck problem in improving the accuracy of target oblique frame detection in the prior art. It realizes the application of the cascade concept to target detection, thereby improving the accuracy of target oblique frame detection.

[0005] In a first aspect, embodiments of this application provide a method for detecting the oblique bounding box of a target in aerial imagery, the method comprising:

[0006] The system uses a backbone network to extract a first feature map from the acquired aerial imagery, and a feature pyramid network to obtain a second feature map from the first feature map. A region generation network is used to generate horizontal candidate boxes for the second feature map, and a feature extractor is used to extract the horizontal feature maps of the horizontal candidate boxes. A rotation rectangle learner is used to obtain a first rotation rectangle from the horizontal feature map, and the first rotation rectangle is set as the initial rotation rectangle. A rotation rectangle update step is iteratively executed for a preset number of iterations. The rotation rectangle update step includes: using a rotation feature extractor to obtain the first rotation feature map of the initial rotation rectangle, and using a rotation target detection head to extract a second rotation rectangle from the first rotation feature map. If the preset number of iterations has not been reached, a higher threshold is set, and the second rotation rectangle is set as the initial rotation rectangle; otherwise, the current second rotation rectangle is used as the final rotation rectangle.

[0007] In conjunction with the first aspect, in one possible implementation, the dimensions of the acquired aerial image are H×W×C, and the dimensions of the first feature map obtained after feature extraction are... The second feature map includes four dimensions, namely: and

[0008] In conjunction with the first aspect, in one possible implementation, generating horizontal candidate boxes for the second feature map using a region generation network includes:

[0009] The region generation network generates nine preset anchor boxes with different areas and aspect ratios at each anchor point in the second feature map. These preset anchor boxes are then classified, with samples having a threshold greater than 0.7 designated as positive samples and samples with a threshold less than 0.3 designated as negative samples. The threshold is the intersection-union ratio (IUU) between the ground truth positive boxes and the predicted regression parameter values. The region generation network, trained with a first number of positive and negative samples, predicts the anchor box coordinate offsets. The corrected anchor box coordinates are calculated based on the coordinate offsets between the ground truth positive boxes and the predicted anchor boxes, thus obtaining the horizontal candidate boxes in the second feature map.

[0010] In conjunction with the first aspect, in one possible implementation, the backbone network is a swin-transformer backbone network.

[0011] In conjunction with the first aspect, in one possible implementation, obtaining the first rotated rectangle from the horizontal feature map using a rotated rectangle learner includes:

[0012] The horizontal candidate boxes are classified, with samples having a threshold greater than 0.5 designated as positive samples and samples having a threshold less than 0.5 designated as negative samples; wherein, the threshold is the intersection-union ratio (IUU) of the bounding rectangles of the horizontal candidate boxes and the ground truth bounding boxes; the rotated rectangle learner, trained with a second number of the positive and negative samples, is used to regress and predict the rotated rectangle; the regression process outputs five regression parameters, including center point regression parameters, length and width regression parameters, and angle; the coordinates corresponding to the aerial image are obtained by inverse transformation using the horizontal candidate boxes and the regression parameters, thus obtaining the first rotated rectangle.

[0013] In conjunction with the first aspect, in one possible implementation, extracting the second rotated bounding box from the first rotated feature map using the rotated target detection head includes:

[0014] The first rotating bounding box is classified, with samples having a threshold greater than 0.6 designated as positive samples and samples having a threshold less than 0.6 designated as negative samples; wherein, the threshold is the intersection-union ratio (IU) of the first rotating bounding box and the ground truth bounding box; the rotating target detection head trained with a third number of the positive and negative samples is used to regress and predict the rotating bounding box; the regression process outputs five regression parameters, including center point regression parameters, length and width regression parameters, and angle regression parameters; the coordinates corresponding to the aerial image are obtained by inverse transformation of the first rotating bounding box and the regression parameters, thus obtaining the second rotating bounding box.

[0015] In conjunction with the first aspect, in one possible implementation, the preset number of iterations does not exceed 3.

[0016] Secondly, embodiments of this application provide a target slant frame detection device for aerial images, the device comprising:

[0017] The system includes a feature acquisition module for extracting a first feature map from the acquired aerial image using a backbone network and a second feature map from the first feature map using a feature pyramid network; a feature extraction module for generating horizontal candidate boxes from the second feature map using a region generation network and extracting horizontal feature maps from the horizontal candidate boxes using a feature extractor; an initialization module for obtaining a first rotated rectangle from the horizontal feature map using a rotated rectangle learner and setting the first rotated rectangle as the initial rotated rectangle; an iteration module for iteratively executing a rotated rectangle update step for a preset number of iterations, wherein the rotated rectangle update step includes: obtaining a first rotated feature map of the initial rotated rectangle using a rotated feature extractor and extracting a second rotated rectangle from the first rotated feature map using a rotated target detection head; and a regression module for setting a higher threshold and setting the second rotated rectangle as the initial rotated rectangle if the preset number of iterations has not been reached, otherwise setting the current second rotated rectangle as the final rotated rectangle.

[0018] In conjunction with the second aspect, in one possible implementation, the dimension of the acquired aerial image in the feature acquisition module is H×W×C, and the dimension of the first feature map obtained after feature extraction is... The second feature map includes four dimensions, namely: and

[0019] In conjunction with the second aspect, in one possible implementation, the generation of horizontal candidate boxes for the second feature map using a region generation network in the feature extraction module includes: generating nine preset anchor boxes with different areas and aspect ratios at each anchor point in the second feature map using the region generation network; classifying the preset anchor boxes, taking samples with a threshold greater than 0.7 as positive samples and samples with a threshold less than 0.3 as negative samples; wherein, the threshold is the intersection-union ratio (IUU) between the ground truth positive boxes and the predicted values ​​of the regression parameters; predicting the anchor box coordinate offsets using the region generation network trained with a first number of positive samples and negative samples; and calculating the corrected anchor box coordinates based on the coordinate offsets between the ground truth positive boxes and the predicted anchor boxes to obtain the horizontal candidate boxes of the second feature map.

[0020] In conjunction with the second aspect, in one possible implementation, the initialization module's method of obtaining a first rotated rectangle from the horizontal feature map using a rotated rectangle learner includes: classifying the horizontal candidate boxes, treating samples with a threshold greater than 0.5 as positive samples, and samples with a threshold less than 0.5 as negative samples; wherein the threshold is the intersection-union ratio (IUU) of the bounding rectangles of the horizontal candidate boxes and the ground truth boxes; using the rotated rectangle learner trained with a second number of positive and negative samples to regress and predict the rotated rectangle; the regression process outputs five regression parameters, including center point regression parameters, length and width regression parameters, and angle; and obtaining the coordinates corresponding to the aerial image through a reverse transformation using the horizontal candidate boxes and the regression parameters to obtain the first rotated rectangle.

[0021] In conjunction with the second aspect, in one possible implementation, the iterative module extracts the second rotating bounding box from the first rotating feature map using the rotating target detection head, comprising: classifying the first rotating bounding box, classifying samples with a threshold greater than 0.6 as positive samples, and samples with a threshold less than 0.6 as negative samples; wherein the threshold is the intersection-union ratio (IUGR) of the first rotating bounding box and the ground truth bounding box; using the rotating target detection head trained with a third number of positive and negative samples to regress and predict the rotating bounding box; the regression process outputs five regression parameters, including a center point regression parameter, length and width regression parameters, and angle regression parameters; and obtaining the coordinates corresponding to the aerial image by inverse transformation of the first rotating bounding box and the regression parameters to obtain the second rotating bounding box. The preset number of iterations does not exceed three.

[0022] Thirdly, embodiments of this application provide an apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it implements the method as described in the first aspect or any possible implementation thereof.

[0023] Fourthly, embodiments of this application provide a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium including storage for storing a computer program or instructions that, when executed, cause the method described in the first aspect or any possible implementation of the first aspect to be implemented.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects:

[0025] This application embodiment employs an aerial image target oblique frame detection method. By adding a rotating feature extractor layer after the rotating target detection head and setting a higher threshold, the rotating feature extractor iteratively extracts the rotation feature map of the previous rotating feature map, and the rotating target detection head extracts the rotating rectangular box of the rotation feature map. This effectively solves the problem of the accuracy bottleneck encountered by the existing target oblique frame detection method, and thus achieves further improvement in the accuracy of oblique frame target detection at a certain detection speed. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1A A schematic flowchart illustrating the aerial image target bounding box detection method provided in the embodiments of this application;

[0028] Figure 1B A flowchart illustrating the steps for updating the rotating rectangle provided in this application embodiment;

[0029] Figure 2 A schematic diagram of the process for generating horizontal candidate boxes of the second feature map provided in an embodiment of this application;

[0030] Figure 3 A schematic diagram of the process for obtaining the first rotating rectangle provided in an embodiment of this application;

[0031] Figure 4 A schematic diagram of the process for extracting the second rotating rectangle provided in an embodiment of this application;

[0032] Figure 5 A schematic diagram of the aerial image target slant frame detection device provided in the embodiments of this application;

[0033] Figure 6 This is a network structure diagram of the aerial image target bounding box detection method provided in the embodiments of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0035] This application provides a method and apparatus for detecting oblique bounding boxes in aerial imagery, which improves the accuracy of oblique bounding box target detection. Experiments were conducted on the publicly available DOTA oblique bounding box detection dataset, and the results show that the embodiments of this application improve the single-stage rating metric (mAP) by 2.31% compared to RoITransformer, fully verifying the effectiveness of the proposed method.

[0036] The following description illustrates some of the technologies involved in the embodiments of this application, including custom terminology used in the embodiments of this application, to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and conciseness, some descriptions of well-known functions and structures are omitted in the following description.

[0037] Backbone Network: In computer vision tasks, the network that extracts features from images is called the backbone network. Feature extraction is fundamental to computer vision tasks, and a good feature extraction network can significantly improve the performance of the algorithm. In object detection tasks, the backbone network is the basic feature extractor, and its main task is to output feature maps of the corresponding input images.

[0038] Feature Pyramid Network (FPN) consists of two paths: bottom-up and top-down. The bottom-up path typically uses convolutional networks to extract features, decreasing in spatial resolution as it detects more high-level structures, with a corresponding increase in the semantic values ​​of the network layers. The top-down path builds higher-resolution layers based on semantically richer layers. Because the reconstructed layers undergo downsampling and upsampling, the target's location is no longer accurate. FPN adds lateral connections between the reconstructed layers and the corresponding feature maps to help the detector better predict the location.

[0039] Region Proposal Network (RPN) is used to find regions in an image where objects may pose a problem, and then extract candidate bounding boxes. The RPN consists of two parts: an anchor generator and fully connected layers. The anchor generator generates nine anchor boxes with different areas and aspect ratios for each anchor point, given a feature map size of W*H, resulting in a total of W*H*9 anchor boxes. The fully connected layers perform classification and bounding box regression by applying a 3*3 convolution followed by two 1*1 convolutions to the feature map.

[0040] Figure 1AThis is a flowchart illustrating the aerial image target bounding box detection method provided in the embodiments of this application, including steps 101 to 107. Figure 6 The diagram shown is a network structure diagram according to an embodiment of this application. Wherein, Figure 1A This is merely one execution order shown in the embodiments of this application and does not represent the only execution order for the target bounding box detection method in aerial imagery. The execution order can be adjusted to achieve the desired final result. Figure 1A The steps shown can be performed in parallel.

[0041] Step 101: Extract the first feature map of the acquired aerial image using a backbone network, and obtain the second feature map of the first feature map using a feature pyramid network. In this embodiment, the backbone network used is the Swin-transformer backbone network. Provided that the goal of extracting feature maps based on aerial images can be achieved without affecting subsequent operation steps, any backbone network or other algorithm can be used to achieve the feature extraction effect described in this step. In this embodiment, the dimensions of the acquired aerial image are H×W×C, and the dimensions of the first feature map obtained after feature extraction are... The second feature map includes four dimensions, namely: and

[0042] Step 102: Generate horizontal candidate boxes for the second feature map using a region generation network, and extract the horizontal feature maps of the horizontal candidate boxes using a feature extractor. For the specific implementation and technical effects of Step 102, please refer to [link to relevant documentation]. Figure 2 The illustrated embodiment includes steps 201 to 204, which are described in detail below.

[0043] Step 201: Use the region generation network to generate 9 preset anchor boxes with different areas and aspect ratios at each anchor point in the second feature map. Specifically, Step 201 involves the Anchor generator in the region generation network generating 9 preset anchor boxes with different areas and aspect ratios at each anchor point when the size of the second feature map is W*H, resulting in a total number of preset anchor boxes of W*H*9.

[0044] Step 202: Classify the preset anchor boxes, treating samples with a threshold greater than 0.7 as positive samples and samples with a threshold less than 0.3 as negative samples; where the threshold is the intersection-union ratio (IUR) of the ground truth positive boxes and the predicted regression parameter values. Specifically, under the embodiment described in Step 201, the region generation network's predicted category value is W*H*9, and the predicted ground truth value is W*H*9; the predicted regression parameter value is W*H*9,4, and the true regression parameter value is W*H*9,4; the bounding rectangle of the ground truth diagonal box is W*H*9,4. Here, the true regression parameter value is the deviation between the ground truth positive box and the bounding rectangle of the ground truth diagonal box. Based on the threshold, i.e., the IUR of the ground truth positive box and the bounding rectangle of the ground truth diagonal box, classify the preset anchor boxes, treating samples with a threshold greater than 0.7 as positive samples and samples with a threshold less than 0.3 as negative samples.

[0045] Step 203: Use the regions trained with the first number of positive and negative samples to generate a network that predicts anchor box coordinate offsets. Specifically, Step 203 involves selecting 128 positive and 128 negative samples for backpropagation training, with 256 positive and negative samples used to calculate the classification loss and 128 positive samples used to calculate the regression loss. The number 128 is an illustrative example of the first number in Step 203 and is not intended to limit the specific steps. Those skilled in the art can adjust this number according to actual conditions, as long as the number of positive and negative samples used for training is equal, and the number of samples should not be too small to ensure effective training.

[0046] Step 204: Calculate the corrected anchor box coordinates based on the coordinate offset between the ground truth bounding box and the predicted anchor box to obtain the horizontal candidate boxes of the second feature map. Specifically, Step 204 involves: obtaining the first 2000 preset anchor boxes based on the foreground score of the category prediction value from Step 201; adjusting out-of-bounds preset anchor boxes to the boundary; removing preset anchor boxes with width and height less than the threshold and those with low foreground scores to obtain processed preset anchor boxes; performing non-maximum suppression on the processed preset anchor boxes; selecting the first k preset anchor boxes; and calculating their angle deviation. The calculation method is as follows:

[0047] d x =1 / w p (x g -x p )

[0048] d y =1 / h p (y g -y p )

[0049] d w =log(w g / w p )

[0050] d h =log(h g / h p )

[0051] In the formula, x g y g w g h g The center point coordinates and length and width of the truth box assigned to each preset anchor frame, x p y p w p h p The center point coordinates and length and width of each preset anchor frame.

[0052] The coordinates corresponding to the aerial image are calculated by reverse calculation using the coordinate offset deviation between the true frame and the predicted anchor frame.

[0053] Step 103: Use a rotating rectangle learner to obtain a first rotating rectangle from the horizontal feature map, and set the first rotating rectangle as the initial rotating rectangle. For the specific implementation of Step 103 and its resulting technical effects, please refer to [link to relevant documentation]. Figure 3 The illustrated embodiment includes steps 301 to 304. A detailed description follows.

[0054] Step 301: Classify the horizontal candidate boxes, and take samples with a threshold greater than 0.5 as positive samples and samples with a threshold less than 0.5 as negative samples; where the threshold is the intersection-union ratio of the bounding rectangle of the horizontal candidate box and the ground truth box.

[0055] Step 302: Use the rotated bounding box learner trained with the second number of positive and negative samples to regress and predict the rotated bounding box. Specifically, Step 302 involves selecting 256 positive samples and 256 negative samples for backpropagation training, with 512 positive and negative samples used to calculate the classification loss and 256 positive samples used to calculate the regression loss. The number 256 is an illustrative example of the second number in Step 302 and is not intended to limit the specific steps. Those skilled in the art can adjust it according to actual conditions in specific applications, as long as the number of positive and negative samples used for training is equal, and the number of samples should not be too small to ensure training effectiveness.

[0056] Step 303: The regression process outputs five regression parameters, including the center point regression parameters, the length and width regression parameters, and the angle.

[0057] Step 304: Obtain the coordinates corresponding to the aerial image through inverse transformation using the horizontal candidate box and regression parameters, thus obtaining the first rotated rectangle. Specifically, Step 304 involves obtaining the coordinates corresponding to the aerial image through inverse transformation using the horizontal candidate box and regression parameters. The calculation formula for obtaining the first rotated rectangle is as follows:

[0058] d x =1 / w p (x g -x p )

[0059] d y =1 / h p (y g -y p )

[0060] d w =log(w g / w p )

[0061] d h =log(h g / h p )

[0062] d θ =θ g / 2π

[0063] In the formula, x g y g w g h g θ g The center coordinates, length, width, and rotation angle of the truth box assigned to each horizontal candidate box; x p y p w p h p The center point coordinates and length and width of each horizontal candidate box.

[0064] Step 104: Iteratively execute the rotating rectangle update step for a preset number of iterations. The rotating rectangle update step is as follows: Figure 1B As shown, steps 108 to 109 are included.

[0065] Step 108: Use a rotation feature extractor to obtain the first rotation feature map of the initial rotated rectangle. Specifically, the rotation feature extractor and regression parameters are transformed back to the coordinates in the original image through angle deviation calculation to obtain a more accurate candidate box, i.e., the first rotation feature map.

[0066] Step 109: Extract the second rotated bounding box from the first rotated feature map using the rotating target detection head. For details on the specific implementation and technical effects of the rotating target detection head extracting the second rotated bounding box from the first rotated feature map, please refer to [link to relevant documentation]. Figure 4 The illustrated embodiment includes steps 401 to 404, which are described in detail below.

[0067] Step 401: Classify the first rotated rectangle, taking samples with a threshold greater than 0.6 as positive samples and samples with a threshold less than 0.6 as negative samples; where the threshold is the intersection-union ratio of the first rotated rectangle and the ground truth rectangle.

[0068] Step 402: Use the rotated object detection head trained with a third number of positive and negative samples to regress and predict the rotated bounding box. Specifically, based on the first rotation feature map and Step 401, select 256 positive samples and 256 negative samples to perform backpropagation training on the rotated object detection head, where 512 positive and negative samples are used to calculate the classification loss and 256 positive samples are used to calculate the regression loss.

[0069] Step 403: During the regression process, five regression parameters are output, including the center point regression parameter, the length and width regression parameters, and the angle regression parameter.

[0070] Step 404: Obtain the coordinates corresponding to the aerial image through the inverse transformation of the first rotated rectangle and regression parameters, and obtain the second rotated rectangle. The angle deviation calculation method is as follows:

[0071] d x =1 / w p ((x g -x p cosθ p +(y g -y p sinθ p )

[0072] d y =1 / h p ((y g -y p cosθ p -(x g -x p sinθ p )

[0073] d w =log(w g / w p )

[0074] d h =log(h g / h p )

[0075] d θ =((θ) g -θ p (mod2π) / 2π

[0076] In the formula, x g y g w g h g θ g The coordinates of the center point of the truth box assigned to the first rotated rectangle for each input, along with its length, width, and rotation angle; x p y p w p h p θ p For each input, input the coordinates of the center point of the first rotating rectangle, its length, width, and rotation angle.

[0077] Step 105: Determine whether the preset number of iterations has been reached. The preset number of iterations is the cumulative preset number of iterations, which does not exceed 3 times, and the initial determination includes step 104 above, that is, the initial preset number of iterations is 1.

[0078] Step 106: Set a higher threshold and set the second rotating rectangle as the initial rotating rectangle. Specifically, this step is executed if the preset number of iterations has not been reached after the judgment in step 105. At this time, the second rotating rectangle obtained in step 104 is set as the initial rotating rectangle and step 104 is executed. The threshold increases in increments of 0.5, 0.6, and 0.7.

[0079] Step 107: Use the current second rotated rectangle as the final rotated rectangle. When the preset number of iterations is met, the current second rotated rectangle becomes the final rotated rectangle.

[0080] like Figure 5 As shown in the figure, this application embodiment also provides an aerial image target slant frame detection device 500. The device includes a feature acquisition module 501, a feature extraction module 502, an initialization module 503, an iteration module 504, and a regression module 505.

[0081] Feature acquisition module 501: used to extract the first feature map of the acquired aerial image using a backbone network, and to obtain the second feature map of the first feature map using a feature pyramid network.

[0082] Feature extraction module 502: This module generates horizontal candidate boxes for the second feature map using a region generation network, and extracts the horizontal feature maps of the candidate boxes using a feature extractor. Specifically, feature extraction module 502 generates nine preset anchor boxes with different areas and aspect ratios at each anchor point in the second feature map using the region generation network; classifies the preset anchor boxes, classifying samples with a threshold greater than 0.7 as positive samples and samples with a threshold less than 0.3 as negative samples; wherein the threshold is the intersection-union ratio (IUU) of the bounding rectangles of the ground truth positive box and the ground truth diagonal box; predicts the anchor box coordinate offset using the region generation network trained with a first number of positive and negative samples; and calculates the corrected anchor box coordinates based on the coordinate offsets of the bounding rectangles of the ground truth positive box and the ground truth diagonal box, thereby obtaining the horizontal candidate boxes of the second feature map.

[0083] Initialization module 503: Used to obtain a first rotated rectangle from the horizontal feature map using a rotated rectangle learner, and set the first rotated rectangle as the initial rotated rectangle. Specifically, initialization module 503 is used to classify the horizontal candidate boxes, taking samples with a threshold greater than 0.5 as positive samples and samples with a threshold less than 0.5 as negative samples; wherein the threshold is the intersection-union ratio (IUGR) of the bounding rectangles of the horizontal candidate boxes and the ground truth rectangles; using the rotated rectangle learner trained with a second number of positive and negative samples to regress and predict the rotated rectangle; the regression process outputs five regression parameters, including center point regression parameters, length and width regression parameters, and angle; and obtaining the coordinates corresponding to the aerial image through a reverse transformation using the horizontal candidate boxes and the regression parameters to obtain the first rotated rectangle.

[0084] Iteration module 504: Iteratively executes a rotation rectangle update step for a preset number of iterations. The rotation rectangle update step includes: using a rotation feature extractor to obtain a first rotation feature map of the initial rotation rectangle, and using a rotation target detection head to extract a second rotation rectangle from the first rotation feature map. Specifically, iteration module 504 is used to: classify the first rotation rectangle, taking samples with a threshold greater than 0.6 as positive samples and samples with a threshold less than 0.6 as negative samples; wherein the threshold is the intersection-union ratio (IU) of the first rotation rectangle and the ground truth bounding box; using the rotation target detection head trained with a third number of positive and negative samples to regress and predict the rotation rectangle; the regression process outputs five regression parameters, including center point regression parameters, length and width regression parameters, and angle regression parameters; and obtaining the coordinates corresponding to the aerial image through a back-calculation transformation of the first rotation rectangle and the regression parameters to obtain the second rotation rectangle. The preset number of iterations does not exceed three times.

[0085] Regression module 505: This module sets a higher threshold and sets the second rotated rectangle as the initial rotated rectangle if the preset number of iterations has not been reached; otherwise, it uses the current second rotated rectangle as the final rotated rectangle. When performing the preset step in this module, module 504 should be included; therefore, the initial iteration count here is 1.

[0086] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0087] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. In implementing this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0088] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, for example, as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. Memory controllers can also be implemented as part of the control logic of memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functions can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0089] This application also provides an apparatus, the apparatus comprising: a processor; a memory for storing processor-executable instructions; and, when the processor executes the executable instructions, implementing the method described in this application.

[0090] This application also provides a non-volatile computer-readable storage medium storing a computer program or instructions thereon, which, when executed, enables the method described in this application to be implemented.

[0091] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software and necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. ， This can also be demonstrated during the data migration process. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0092] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0093] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A method for detecting the oblique bounding box of a target in aerial imagery, characterized in that, include: The first feature map of the acquired aerial image is extracted using a backbone network, and the second feature map of the first feature map is obtained using a feature pyramid network. A region generation network is used to generate horizontal candidate boxes for the second feature map, and a feature extractor is used to extract the horizontal feature maps of the horizontal candidate boxes. A first rotated rectangle is obtained from the horizontal feature map using a rotated rectangle learner, and this first rotated rectangle is set as the initial rotated rectangle. The process of obtaining the first rotated rectangle from the horizontal feature map using the rotated rectangle learner includes: classifying the horizontal candidate boxes, treating samples with a threshold greater than 0.5 as positive samples and samples with a threshold less than 0.5 as negative samples; wherein the threshold is the intersection-union ratio (IUU) of the bounding rectangles of the horizontal candidate boxes and the ground truth rectangles; using the rotated rectangle learner trained with a second number of positive and negative samples to regress and predict the rotated rectangle; the regression process outputs five regression parameters, including center point regression parameters, length and width regression parameters, and angle parameters; and obtaining the coordinates corresponding to the aerial image by inverse transformation using the horizontal candidate boxes and the regression parameters to obtain the first rotated rectangle. The iterative execution of a rotation rectangle update step for a preset number of iterations includes: using a rotation feature extractor to obtain a first rotation feature map of the initial rotation rectangle, and using a rotation target detection head to extract a second rotation rectangle from the first rotation feature map; wherein, the step of using the rotation target detection head to extract the second rotation rectangle from the first rotation feature map includes: classifying the first rotation rectangle, taking samples with a threshold greater than 0.6 as positive samples and samples with a threshold less than 0.6 as negative samples; wherein, the threshold is the intersection-union ratio (IU) of the first rotation rectangle and the ground truth bounding box; using the rotation target detection head trained with a third number of positive samples and negative samples to regress and predict the rotation rectangle; the regression process outputs five regression parameters, including center point regression parameters, length and width regression parameters, and angle regression parameters; and obtaining the coordinates corresponding to the aerial image by inverse transformation of the first rotation rectangle and the regression parameters to obtain the second rotation rectangle. If the preset number of iterations has not been reached, a higher threshold is set, and the second rotating rectangle is set as the initial rotating rectangle; otherwise, the current second rotating rectangle is used as the final rotating rectangle.

2. The method according to claim 1, characterized in that, The dimensions of the acquired aerial images are The dimension of the first feature map obtained after feature extraction is ; The second feature map includes four dimensions, namely: , , and .

3. The method according to claim 1 or 2, characterized in that, The step of generating horizontal candidate boxes for the second feature map using a region generation network includes: The region generation network is used to generate nine preset anchor frames with different areas and aspect ratios at each anchor point in the second feature map. The preset anchor boxes are classified, with samples having a threshold greater than 0.7 as positive samples and samples having a threshold less than 0.3 as negative samples; wherein, the threshold is the intersection-union ratio of the true positive boxes and the predicted values ​​of the regression parameters; The region generation network, trained with a first number of positive samples and negative samples, is used to predict the anchor box coordinate offset. The corrected anchor frame coordinates are calculated based on the coordinate offset between the ground truth bounding box and the predicted anchor frame, and the horizontal candidate box of the second feature map can be obtained.

4. The method according to claim 1, characterized in that, The backbone network is the swin-transformer backbone network.

5. The method according to claim 1, characterized in that, The preset number of iterations shall not exceed 3.

6. A device for detecting the oblique frame of a target in aerial imagery, characterized in that, include: The feature acquisition module is used to extract the first feature map of the acquired aerial image using a backbone network, and to obtain the second feature map of the first feature map using a feature pyramid network. The feature extraction module is used to generate horizontal candidate boxes of the second feature map using a region generation network, and to extract the horizontal feature map of the horizontal candidate boxes using a feature extractor. An initialization module is used to obtain a first rotated rectangle from the horizontal feature map using a rotated rectangle learner, and set the first rotated rectangle as the initial rotated rectangle. The process of obtaining the first rotated rectangle from the horizontal feature map using the rotated rectangle learner includes: classifying the horizontal candidate boxes, treating samples with a threshold greater than 0.5 as positive samples, and samples with a threshold less than 0.5 as negative samples; wherein the threshold is the intersection-union ratio (IUGR) of the bounding rectangles of the horizontal candidate boxes and the ground truth rectangles; using the rotated rectangle learner trained with a second number of positive and negative samples to regress and predict the rotated rectangle; the regression process outputs five regression parameters, including center point regression parameters, length and width regression parameters, and angle parameters; and obtaining the coordinates corresponding to the aerial image by inverse transformation using the horizontal candidate boxes and the regression parameters to obtain the first rotated rectangle. An iterative module is used to iteratively execute a rotation rectangle update step for a preset number of iterations. The rotation rectangle update step includes: obtaining a first rotation feature map of the initial rotation rectangle using a rotation feature extractor, and extracting a second rotation rectangle from the first rotation feature map using a rotation target detection head; wherein, extracting the second rotation rectangle from the first rotation feature map using the rotation target detection head includes: classifying the first rotation rectangle, classifying samples with a threshold greater than 0.6 as positive samples, and samples with a threshold less than 0.6 as negative samples; wherein the threshold is the intersection-union ratio (IUGR) of the first rotation rectangle and the ground truth bounding box; using the rotation target detection head trained with a third number of positive and negative samples to regress and predict the rotation rectangle; the regression process outputs five regression parameters, including center point regression parameters, length and width regression parameters, and angle regression parameters; and obtaining the coordinates corresponding to the aerial image by inverse transformation of the first rotation rectangle and the regression parameters to obtain the second rotation rectangle. The regression module is used to perform a preset judgment step, which includes: judging whether the preset number of iterations has been reached; if the current number of iterations has not reached the preset number of iterations, setting a higher threshold and setting the second rotating rectangle as the initial rotating rectangle, and performing the rotating rectangle update step; otherwise, using the current second rotating rectangle as the final rotating rectangle.

7. A device, characterized in that, include: processor; Memory used to store processor-executable instructions; When the processor executes the executable instructions, it implements the method as described in any one of claims 1 to 5.

8. A non-volatile computer-readable storage medium, characterized in that, Includes storage of computer programs or instructions that, when executed, cause the method as described in any one of claims 1 to 5 to be implemented.

Images