A coupled dual-frequency phase-shift rotation angle encoding method for rotating target detectors

Abstract

The present invention discloses a coupled dual-frequency phase shift rotation angle encoding method for a rotating target detector, comprising the following steps: first, performing phase mapping and phase encoding based on the coupled dual-frequency phase shift method on the GT rotation angle; second, decoding the output result of the angle regression branch of the target detector based on deep learning to obtain a high-frequency phase and a low-frequency phase; then, performing phase unpacking on the high-frequency phase to obtain an absolute high-frequency phase; finally, using the absolute high-frequency phase as the main phase to decode the predicted rotation angle, and performing an additional square-like angle normalization method for square-like targets. The present invention not only simultaneously solves the boundary discontinuity problem and the square-like problem inherent in the classic deep learning rotating target detection model based on regression schemes, but also reduces the number of model parameters. In addition, the present invention can be easily embedded in various mainstream target detectors to improve their detection accuracy for rotating targets.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image intelligent interpretation and target detection, and in particular relates to a coupled dual-frequency phase-shift rotation angle encoding method for a rotating target detector. Background Art

[0002] Rotated object detection, as a subtask of object detection, has been widely used in remote sensing images, scene text, and industrial detection in recent years. Unlike traditional object detection methods, rotated object detection uses a rotated bounding box (RBB) instead of a conventional horizontal bounding box (HBB) to accurately represent the object. Currently, mainstream rotated object detection algorithms (also known as rotated object detectors), such as RoI-Transformer, R3Det, ​​and S2ANet, all use additional regression convolution output channels or angle regression branches to predict the rotation angle. This type of method, which modifies a general object detector into a rotated object detector, is called a regression-based scheme. However, this type of regression-based scheme suffers from boundary discontinuity and square-like problems.

[0003] In theory, the boundary discontinuity problem and the square-like problem can be unified into a rotational symmetry problem. The boundary discontinuity problem corresponds to the target having 180-degree rotational symmetry, while the square-like problem corresponds to 90-degree rotational symmetry. Therefore, these two types of problems are very similar to the periodic ambiguity problems encountered when performing absolute phase measurement in the field of optical measurement. To this end, Yu et al. proposed a phase shift coding method (PSC) based on optical measurement phase shift technology to re-represent the rotation angle, and proposed a corresponding dual-frequency variant version PSCD to simultaneously solve the boundary discontinuity problem and the square-like problem. However, the effect of PSCD on the DIOR-R dataset, which has a large number of square-like instances, is still poor, and the detection effect on the HRSC dataset, which is mainly composed of targets with large aspect ratios, is weaker than the single-frequency PSC method. Summary of the Invention

[0004] In order to completely solve the boundary discontinuity problem and the square-like problem and improve the detection accuracy of the rotating target detector based on deep learning, the present invention uses optical measurement technology to invent a coupled dual-frequency phase-shift rotation angle encoding method for rotating target detectors. This method can be easily embedded in various mainstream deep learning-based rotating target detectors to enhance the detection accuracy of rotating targets.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A coupled dual-frequency phase-shift rotation angle encoding method for a rotating target detector comprises the following steps:

[0007] Step 1: Phase mapping and phase encoding based on coupled dual-frequency phase shifting are performed on the ground truth (GT) rotation angles given by any dataset during deep learning model training.

[0008] Step 2: Phase decode the output prediction of the angle regression branch of the deep learning object detection model to obtain high-frequency phase and low-frequency phase;

[0009] Step 3: Use the low-frequency phase to unwrap the high-frequency phase, restore the absolute phase of the high-frequency phase, and set the main phase to the absolute high-frequency phase;

[0010] Step 4: Obtain the pre-decoding rotation angle predicted by the deep learning model based on the main phase decoding;

[0011] Step 5: Use the square-like angle normalization method to limit the pre-decoding rotation angle and GT rotation angle of the square-like predicted RBB and GT RBB to [-π / 4,π / 4], thereby obtaining the final predicted rotation angle of the deep learning model.

[0012] Furthermore, the phase mapping formula used in step 1 is:

[0013]

[0014] in: is the high frequency phase, θ g is the GT RBB rotation angle, is the low-frequency phase;

[0015] Phase encoding formula based on coupled dual-frequency phase shift method:

[0016]

[0017] Where: x n is the coded phase shift component, B h and B l They are all hyperparameters, and n is the index of the phase shift component, starting from 0.

[0018] Furthermore, the high and low frequency phase decoding formula used in step 2 is:

[0019]

[0020]

[0021] Furthermore, the high-frequency phase unpacking formula used in step 3 is:

[0022]

[0023]

[0024] Among them: Round() represents the rounding operation, k is the unpacking cycle order, High frequency phase The absolute phase of As the main phase.

[0025] Furthermore, the main phase decoding used in step 4 recovers the pre-decoding rotation angle θ p The decoding formula is:

[0026]

[0027] Furthermore, in step 5, a square-like angle normalization method is used, using the following formula:

[0028] α g =h g / w g

[0029] α p =h p / w p

[0030]

[0031]

[0032] Where: h g is the length of GT RBB, w g is the width of GT RBB, α g is the aspect ratio of GT RBB, h p The length of RBB predicted by the model regression branch, w p The width of RBB is predicted by the model regression branch, α p To predict the aspect ratio of RBB, thr g The threshold value for judging whether the current GTRBB is a square instance, thr p To determine whether the current predicted RBB is a square-like instance, there is usually a threshold of thr g =thr p ;

[0033] range_norm π / 4 To limit the range of the rotation angle to the function in [-π / 4,π / 4], range_norm π / 4 The specific calculation formula is as follows:

[0034]

[0035] Among them: mod() is the remainder operation, is any angle variable.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] (1) In the phase encoding process designed by the present invention, both high-frequency phase shift and low-frequency phase shift adopt a 5-step phase shift encoding method, which theoretically has stronger anti-interference ability. And the final phase encoding result is obtained by additive coupling of high-frequency phase shift and low-frequency phase shift. Compared with the PSCD method, the method designed by the present invention reduces the number of angle regression prediction branch channels required by the deep learning model from 6 to 5, reducing the number of model parameters and being more conducive to the lightweight model.

[0038] (2) This paper specifically designs a square-like angle normalization method to address the inherent angular ambiguity of square-shaped rotated objects when the long side is rotated 90 degrees. Furthermore, this square-like angle normalization method can also be applied to square-like predicted objects and ground truth instances to further improve detection accuracy.

[0039] (3) The present invention theoretically ensures a complete solution to the boundary discontinuity problem and the square-like problem. There is no need to resort to complex Gaussian distribution modeling and transformation, and there is no need to resort to complex probability distribution to approximate the rotation IoU deriving loss. Only a small number of angle prediction channels need to be added to achieve a significant improvement in detection accuracy while still using the L1 loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is the specific location where the present invention is embedded in any deep learning-based rotation target detector training process.

[0041] Figure 2 Part of the detection results of the RTMDet-R-CDFP embodiment of the present invention on the DIOR-R dataset are shown. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with the embodiments.

[0043] The present invention discloses a coupled dual-frequency phase shift rotation angle encoding method for a rotating target detector, comprising the following steps: first, performing phase mapping and phase encoding based on the coupled dual-frequency phase shift method on the GT rotation angle; second, decoding the output result of the angle regression branch of the target detector based on deep learning to obtain a high-frequency phase and a low-frequency phase; then, performing phase unpacking on the high-frequency phase to obtain an absolute high-frequency phase; finally, using the absolute high-frequency phase as the main phase to decode the predicted rotation angle, and performing an additional square-like angle normalization method for square-like targets. The present invention not only simultaneously solves the boundary discontinuity problem and the square-like problem inherent in the classic deep learning rotating target detection model based on regression schemes, but also reduces the number of model parameters. In addition, the present invention can be easily embedded in various mainstream target detectors to improve their detection accuracy for rotating targets.

[0044] Example 1

[0045] A coupled dual-frequency phase-shift rotation angle encoding method for a rotating target detector comprises the following steps:

[0046] Step 1: Phase mapping and phase encoding based on coupled dual-frequency phase shifting are performed on the ground truth (GT) rotation angles given by any dataset during deep learning model training.

[0047] The phase mapping formula used in step 1 is:

[0048]

[0049] in: is the high frequency phase, θ g is the GT RBB rotation angle, is the low-frequency phase;

[0050] Phase encoding formula based on coupled dual-frequency phase shift method:

[0051]

[0052] Where: x n is the coded phase shift component, B h and B l They are all hyperparameters, and n is the index of the phase shift component, starting from 0.

[0053] Step 2: Phase decode the output prediction of the angle regression branch of the deep learning object detection model to obtain high-frequency phase and low-frequency phase;

[0054] The high and low frequency phase decoding formula used in step 2 is:

[0055]

[0056]

[0057] Step 3: Use the low-frequency phase to unwrap the high-frequency phase, restore the absolute phase of the high-frequency phase, and set the main phase to the absolute high-frequency phase;

[0058] The high frequency phase unpacking formula used in step 3 is:

[0059]

[0060]

[0061] Among them: Round() represents the rounding operation, k is the unpacking cycle order, High frequency phase The absolute phase of As the main phase.

[0062] Step 4: Obtain the pre-decoding rotation angle predicted by the deep learning model based on the main phase decoding;

[0063] The main phase decoding used in step 4 recovers the pre-decoding rotation angle θ p The decoding formula is:

[0064]

[0065] Step 5: Use the square-like angle normalization method to limit the pre-decoding rotation angle and the GT rotation angle of the square-like predicted RBB and GT RBB to [-π / 4,π / 4], thereby obtaining the final predicted rotation angle of the deep learning model;

[0066] In step 5, a square-like angle normalization method is used, using the following formula:

[0067] α g =h g / w g

[0068] α p =h p / w p

[0069]

[0070]

[0071] Where: h g is the length of GT RBB, w g is the width of GT RBB, α g is the aspect ratio of GT RBB, h pThe length of RBB predicted by the model regression branch, w p The width of RBB is predicted by the model regression branch, α p To predict the aspect ratio of RBB, thr g The threshold value for judging whether the current GTRBB is a square instance, thr p To determine whether the current predicted RBB is a square-like instance, there is usually a threshold of thr g =thr p ;

[0072] range_norm π / 4 To limit the range of the rotation angle to the function in [-π / 4,π / 4], range_norm π / 4 The specific calculation formula is as follows:

[0073]

[0074] Among them: mod() is the remainder operation, is any angle variable.

[0075] The square-like angle normalization method of the first embodiment can limit the rotation angle of the square-like predicted RBB and the GT RBB to [-π / 4, π / 4], which theoretically ensures that the square-like problem is completely solved.

[0076] Example 2

[0077] like Figure 1 As shown, the present invention is a rotation angle encoding method that is easily embedded in various deep learning-based rotation target detectors. The following takes the lightweight and high-inference-efficiency RTMDet-R rotation target detector as an example to explain in detail how to embed the present invention into a deep learning-based rotation target detector and the specific implementation details of the present invention. The deep learning target rotation detection model obtained by combining RTMDet-R with the present invention is referred to as RTMDet-R-CDFP. RTMDet-R-CDFP includes the following specific implementation steps:

[0078] Step 1: Process the dataset following the conventional training process for deep learning-based object detection models. This method is not dataset-specific and is therefore applicable to publicly available aerial image rotation object detection datasets such as DOTA, DIOR-R, and HRSC, as well as any dataset annotated with RBB. In particular, when using a private, task-specific dataset with custom annotations, the dataset is typically partitioned into training, validation, and test sets in a ratio of 5:2:3.

[0079] Step 1-1: Using the open-source rotated object detection library MMRotate as an example, for the training and validation sets divided from the dataset, first read the images and GT annotations in the specified folder into memory through the Python language interface and process them into the common Numpy data type. In particular, the GT RBB annotations use the common long-side 90-degree rotation representation scheme;

[0080] Step 1-2: Then, perform data augmentation strategies to avoid overfitting of deep learning model training. Usually, random flip augmentation and random rotation augmentation are used by default. Stronger data augmentation strategies are also applicable.

[0081] Steps 1-3: Finally, convert the image and GT annotations from Numpy data types to PyTorch-specific tensor data types. Through the dataloader interface, these data are processed to output only a single training tensor of the specified batch size. These steps are implemented in MMRotate, and the interface is packaged for a more advanced and user-friendly interface. However, this invention is not limited to its specific implementation in MMRotate and can be easily implemented using other object detection libraries or other deep learning frameworks.

[0082] Step 2: Specific implementation of a deep learning-based rotated object detector. This paper is not specific to a specific rotated object detector. In experiments, our invented CDFP can be combined with the well-known anchor-box-based RetinaNet, the well-known anchor-free FCOS, and the well-known lightweight and efficient RTMDet-R object detection model. Using RTMDet-R as an example, we briefly introduce the specific implementation steps of a deep learning-based rotated object detector.

[0083] Step 2-1: Backbone Network Implementation. The backbone network used by the RTMDet-R model is CSPNetXt, a convolutional neural network with a main network structure similar to the CSPDarknet network of the YOLOX model. (Specialized researchers and engineers in the field of target detection should be familiar with the Darknet series of convolutional neural networks and their corresponding variants, so I will not elaborate on them in detail here.) The most important improvement is the use of a large kernel convolution standard block constructed by depthwise separable large kernel convolution to replace the standard convolution block in the original CSPDarknet. Specifically, the large kernel convolution standard block of CSPNetXt consists of a 3×3 Conv layer plus a BN layer plus a SiLU activation function to form the first layer of classic convolution block. The second layer of depthwise separable large kernel convolution layer is composed of a 5×5 Depthwise Conv layer plus a 1×1 Pointwise Conv layer plus a BN layer plus a SiLU activation function. Finally, the first layer of classic convolution block and the second layer of depthwise separable large kernel convolution layer are connected in series and added with a residual connection.

[0084] Step 2-2: Implementation of the neck network. The neck network used by the RTMDet-R model is the CSPNetXtPAN network. Specifically, the classic CSPPAN neck network is improved by combining it with a depthwise separable large kernel convolutional layer.

[0085] Step 2-3: Implementation of the detection head network. In order to predict the rotation angle, the present invention needs to add an additional angle prediction branch to the detection head network, such as Figure 1 As shown. It consists of a 3×3Conv with 5 channels, each channel is responsible for predicting a coded phase shift component x of our CDFP n , the results of all output channels constitute the model's angle branch prediction result X P The regression and classification branches of the detection head remain unchanged. Similarly, for the RetinaNet and FCOS models, the transition from a general object detection model to a rotational object detection model can be achieved by simply adding an angle prediction branch consisting of a 5-channel 3×3 Conv matrix, while keeping the regression and classification branches of the backbone network, neck network, and detection head unchanged.

[0086] Step 3: Specific implementation of the coupled dual-frequency phase-shift rotation angle encoding method in the training process of the rotating target detection model. Taking the RTMDet-R model as an example, by modifying the detection head in steps 2-3, the RTMDet-R-CDFP model can be obtained. Figure 1 As shown in Figure 2, the detailed implementation steps of the coupled dual-frequency phase shift rotation angle encoding method during the training process of RTMDet-R-CDFP are as follows:

[0087] Step 3-1: First, the rotation angle θ of each GT RBB of the input model is calculated according to the following formula: g Do phase mapping:

[0088]

[0089] in: is the high frequency phase, θ g is the GT RBB rotation angle, is the low-frequency phase;

[0090] Step 3-2: Perform phase encoding according to the coupled dual-frequency phase shift phase encoding formula of the present invention, as follows:

[0091]

[0092] Where: x n is the coded phase shift component, B h and B l All of them are hyperparameters. n is the index of the phase shift component, starting from 0. In order to limit the encoding value range to the interval [-1, 1], the hyperparameter B h ,B l The default setting is 0.5. Based on this, we can get the GTRBB rotation angle θ g The encoding result X gt ={x n |n=0,1,...,4}.

[0093] Step 3-3: Get the predicted output X of the angle regression branch of the RTMDet-R-CDFP model P , and then according to the default angle loss function Calculate X gt With X P Specifically, the present invention uses L1Loss as the angle loss function by default. Therefore, the calculation formula is as follows:

[0094]

[0095] However, the present invention and the use of It is irrelevant, so the commonly used Smooth L1Loss or L2Loss can also be used as Even more complex loss functions.

[0096] Step 3-4: If combined with a rotation target detection model such as RTMDet-R that uses the rotation IoU to derive the loss as the regression branch loss function, the following rotation angle decoding process of the present invention needs to be performed:

[0097] Step 3-4-1: Phase decoding of the coupled dual-frequency phase shift rotation angle encoding method. The model prediction output X is obtained according to the following formula P High frequency phase and low-frequency phase

[0098]

[0099]

[0100] In the specific implementation of the Pytorch deep learning framework, the arctan function is replaced by Pytorch's atan2 function because the value range of the atan2 function is [-π,π), which is more practical.

[0101] Step 3-4-2: Phase unpacking of the coupled dual-frequency phase shift rotation angle encoding method. The high-frequency phase is calculated according to the following formula: Perform phase unpacking and restore its absolute phase value.

[0102]

[0103]

[0104] Among them: Round() represents the rounding operation, k is the unpacking cycle order, High frequency phase The absolute phase of , the value range is [-2π, 2π), and As the main phase.

[0105] Step 3-4-3: Decoding the rotation angle of the coupled dual-frequency phase-shift rotation angle encoding method. After obtaining the main phase, the pre-decoding rotation angle θ can be calculated according to the following formula p :

[0106]

[0107] Step 3-4-4: Optionally, the square-like angle normalization method without the coupled dual-frequency phase-shift rotation angle encoding method can be used to further process the pre-decoding rotation angle. If not used, the pre-decoding rotation angle obtained in step 3-4-3 is directly used as the final decoding rotation angle. If used, the following steps are required:

[0108] Step 3-4-4-1: First, return to step 3-1 and calculate the length of GT RBB. g With width w g Calculate the aspect ratio α of the current RBB g =h g / w g, and then determine whether to rotate the GT RBB by the following formula g Normalize the square-like angles:

[0109]

[0110] Where: h g is the length of GT RBB, w g is the width of GT RBB, α g is the aspect ratio of GT RBB, thr g The threshold for determining whether the current GTRBB is a square-like instance is a hyperparameter, and its optimal value is determined to be 0.9 through experiments.

[0111] And range_norm π / 4 To limit the range of the rotation angle to the function in [-π / 4,π / 4], the specific calculation formula is as follows:

[0112]

[0113] Among them: mod() is the remainder operation, is any angle variable.

[0114] Step 3-4-4-2: Return to the result of step 3-4-3, and similarly perform regression on the length h according to the model regression branch. p and width p The prediction result is used to calculate the aspect ratio α of the current predicted RBB p =h p / w p , and then determine whether to rotate the decoder angle θ according to the following formula p Normalize the square-like angles:

[0115]

[0116] Where: h p The length of RBB predicted by the model regression branch, w p The width of RBB is predicted by the model regression branch, α p To predict the aspect ratio of RBB, thr p The threshold for judging whether the current predicted RBB is a square-like instance is a hyperparameter and is consistent with the threshold in GT RBB, that is, thr g =thr p .

[0117] Step 3-4-5: After obtaining the model-predicted decoding rotation angle, it can be combined with the predicted output of the regression branch to obtain the model-predicted decoding RBB.

[0118] Step 3-5: Calculate the total loss function for model training as follows:

[0119]

[0120] in: and is the weighted loss function of the classification branch and regression branch of the specific rotation target detector used. w3 is the weighted hyperparameter of the angle prediction branch, and the angle loss function Calculated in step 3-3. In particular, when using the RetinaNet or FCOS model, if you do not use the derived loss of rotation IoU as the loss function of the regression branch, you do not need to perform steps 3-4 and can jump directly to the current step to calculate the total loss function.

[0121] Steps 3-6: Optimize and update the model parameters according to the specified optimizer. When the specified training termination condition is reached, the training is terminated to obtain the final trained model.

[0122] Step 4: Model inference test of the rotating target detector combined with the coupled dual-frequency phase shift rotation angle encoding method. Taking the RTMDet-R-CDFP obtained in step 3 as an example, after training, it is only necessary to follow the inference process of the conventional deep learning model and execute the forward network once to obtain the classification results, regression results and angle branch prediction results predicted by the model. In particular, the prediction result of the angle branch is X P Therefore, it is necessary to execute the rotation angle decoding process in steps 3-4 to obtain the predicted decoding angle θ p In this way, the predicted RBB can be combined with the prediction results of the regression branch. Finally, the final inference result is obtained by post-processing with the rotational NMS. Figure 2 Part of the detection results of RTMDet-R-CDFP on DIOR-R in an embodiment of the present invention are shown.

[0123] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A coupled dual-frequency phase-shift rotation angle encoding method for a rotating target detector, characterized in that: The following steps are involved: Step 1: Phase mapping and phase encoding based on coupled dual-frequency phase shifting are performed on the ground truth rotation angles given by any dataset during deep learning model training. Step 2: Phase decode the output prediction of the angle regression branch of the deep learning target detection model to obtain high-frequency phase and low-frequency phase Step 3: Use the low-frequency phase to unwrap the high-frequency phase, restore the absolute phase of the high-frequency phase, and set the main phase to the absolute high-frequency phase; Step 4: Obtain the pre-decoding rotation angle predicted by the deep learning model based on the main phase decoding; Step 5: Use the square-like angle normalization method to limit the pre-decoding rotation angle and GT rotation angle of the square-like predicted RBB and GT RBB to [-π / 4,π / 4], thereby obtaining the final predicted rotation angle of the deep learning model.

2. The coupled dual-frequency phase-shift rotation angle encoding method for a rotating target detector according to claim 1, characterized in that: The phase mapping formula used in step 1 is: in: is the high frequency phase, θ g is the GT RBB rotation angle, is the low-frequency phase; Phase encoding formula based on coupled dual-frequency phase shift method: Where: x n is the coded phase shift component, B h and B l They are all hyperparameters, and n is the index of the phase shift component, starting from 0.

3. The coupled dual-frequency phase-shift rotation angle encoding method for a rotating target detector according to claim 2, characterized in that: The high and low frequency phase decoding formula used in step 2 is: 。 4. The coupled dual-frequency phase-shift rotation angle encoding method for a rotating target detector according to claim 3, characterized in that: The high frequency phase unpacking formula used in step 3 is: Among them: Round() represents the rounding operation, k is the unpacking cycle order, High frequency phase The absolute phase of As the main phase.

5. The coupled dual-frequency phase-shift rotation angle encoding method for a rotating target detector according to claim 4, characterized in that: The main phase decoding used in step 4 recovers the pre-decoding rotation angle θ p The decoding formula is:

6. The coupled dual-frequency phase-shift rotation angle encoding method for a rotating target detector according to claim 5, characterized in that: In step 5, a square-like angle normalization method is used, using the following formula: a g =h g / w g a p =h p / w p Where: h g is the length of GT RBB, w g is the width of GT RBB, α g is the aspect ratio of GT RBB, h p The length of RBB predicted by the model regression branch, w p The width of RBB is predicted by the model regression branch, α p To predict the aspect ratio of RBB, thr g The threshold value for judging whether the current GT RBB is a square instance, thr p To determine whether the current predicted RBB is a square-like instance, there is usually a threshold of thr g =thr p ; range_norm π / 4 To limit the range of the rotation angle to the function in [-π / 4,π / 4], range_norm π / 4 The specific calculation formula is as follows: Among them: mod() is the remainder operation, is any angle variable.

Images