Primary direction rotating target detection method for deformable quadrangle attention

Abstract

The invention discloses a main direction rotating target detection method for deformable quadrilateral attention, and belongs to the technical field of computer vision. The method comprises the following steps: 1) loading a rotating target data set, preprocessing an image, and converting the image into a model training format; 2) through a detection head of the deformable quadrilateral attention module, mapping the fixed window into a quadrilateral adaptive to the geometrical shape of the target through a dynamic prediction transformation matrix; 3) a principal component analysis module is integrated on the detection head, and a principal direction and a rotation bounding box are calculated based on vertex coordinates of the target quadrangle; and 4) inputting the test set into the trained model, and outputting an angle-containing bounding box, category and confidence of the rotating target. According to the method, the mAPs on DOTA-v1.0 and HRSC2016 data sets are 79.38% and 98.90% respectively, compared with an existing method, the precision of the method is improved by 3.2%-5.6%, particularly, the detection robustness of irregular and multi-attitude rotating targets (such as ships and airplanes) is remarkably enhanced, and the method is suitable for remote sensing image and aerial photographing target detection scenes.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically a rotating target detection method based on deformable quadrilateral attention and principal component analysis, applicable to rotating target detection in scenarios such as remote sensing images and aerial images. Background Technology

[0002] Targets in remote sensing imagery (such as aircraft and ships) often exhibit diverse orientations and shapes, making it difficult for traditional horizontal bounding boxes to meet precise localization requirements. Therefore, rotated bounding boxes are necessary. Existing methods for rotating target detection mostly add angular information to horizontal bounding boxes, but fail to fully utilize the geometric characteristics of the rotated boxes, resulting in insufficient accuracy for irregular targets. For example, R-CNN-based frameworks rely on complex region proposal networks, while Transformer-based methods use fixed-window attention mechanisms, limiting the model's adaptability to target shape and orientation. Furthermore, complex backbone networks may introduce noisy features, affecting detection performance. Therefore, a method that can dynamically adapt to target geometry and improve detection accuracy is urgently needed. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of existing target detection technologies and provide a rotating target detection method based on Deformable Quadrilateral Attention (DQA) and Principal Component Analysis (PCA). This method dynamically adjusts the attention region through a Deformable Quadrilateral Attention module and optimizes angle prediction by combining it with a Principal Component Analysis module, significantly improving the accuracy and robustness of rotating target detection.

[0004] The flowchart of the method of the present invention is as follows Figure 1 As shown, the specific implementation steps are as follows:

[0005] The first step is to load the data from the rotating target detection using the appropriate database;

[0006] The second step is to preprocess the image, converting the input image into the model training format. Preprocessing includes: (2.1) Image scaling: scaling the image to the model input size of 1024×1024; (2.2) Normalization: normalizing pixel values ​​to the range [0,1]; (2.3) Channel conversion: converting the image from BGR format to RGB format; (2.4) Box conversion: converting the original target labels to quadrilateral format to adapt to the rotated target detection task; dividing the training and testing sets, and inputting the obtained input feature images into a format such as... Figure 2 In the detection network shown;

[0007] The third step is to input the preprocessed image data into the feature extraction network: (3.1) The feature extraction network inputs an image of size 1024×1024×3 through the first convolutional layer. The convolutional layer has a kernel size of 3, a stride of 2, and a boundary expansion of 1. Then it outputs a feature map of size 512×512 with 64 channels, which is then transmitted to the second convolutional layer.

[0008] (3.2) After passing through the second convolutional layer, the output feature map is 256×256 in size and has 128 channels, which is then transmitted to the third convolutional layer;

[0009] (3.3) After the third convolutional layer, the output feature map is 128×128 in size and has 256 channels. It is then transmitted to the Region Proposal Network (RPN) to generate candidate regions.

[0010] (3.4) The region proposal network outputs multiple candidate regions. Fixed-size region features are extracted through the RoI alignment layer to obtain feature maps of size 7×7 with 256 channels. These feature maps are then directly input into the deformable quadrilateral attention module.

[0011] The fourth step is to input the feature map output by the feature extraction network into the deformable quadrilateral attention module (DQA), such as... Figure 2 As shown:

[0012] (4.1) Window partitioning and feature extraction: The input feature map is divided into windows. Divide into multiple basic windows of size w×w ( The width and height of the feature map, (Number of channels), extract queries from each window ,key Sum Feature markers:

[0013] ,

[0014] (4.2) Transformation parameter prediction: The feature map Predict the 9-dimensional transformation parameter vector using global average pooling, LeakyReLU activation function, and a 1×1 convolutional network. :

[0015] (4.3) Generation of composite projection transformation matrix: The nine learnable parameters generated by the above equation ~ The following five basic transformation matrices are obtained through equation (3):

[0016]

[0017]

[0018]

[0019] The basic transformation matrices are defined as: scaling matrices Shearing matrix Rotation matrix Translation matrix Perspective Matrix .in =W / w and =H / w is used to scale the translation to fit different input sizes. Finally, the composite projection transformation matrix T is obtained by multiplying the fundamental transformation matrices:

[0020]

[0021] (4.4) Quadrilateral window mapping: Mapping the coordinates of a fixed rectangular window through projection transformation. Mapping to obtain the coordinates of the target quadrilateral window :

[0022] (5)

[0023] (6)

[0024] in The median value;

[0025] (4.5) Feature Sampling and Attention Calculation: Using the bilinear interpolation grid sampling function to perform feature sampling and attention calculation. , Features are sampled, and then attention weights are calculated to achieve quadrilateral feature aggregation:

[0026] (7)

[0027] Where r is the relative position encoding matrix, used to embed spatial position information;

[0028] The fifth step involves inputting the quadrilateral features output by the deformable quadrilateral attention module into the principal orientation analysis module (PCA): the PCA module calculates the precise principal orientation angle of the target based on the quadrilateral vertex coordinates, such as... Figure 3 As shown, it specifically includes:

[0029] (5.1) Vertex coordinate centering: centering the quadrilateral vertices of the input feature map Centralized processing:

[0030] (8)

[0031] in , The vertex coordinates are respectively at and The mean of the dimension;

[0032] (5.2) Covariance Matrix Calculation: Calculate the covariance matrix of the centered data. , The centered matrix obtained in the previous step:

[0033] (9)

[0034] (5.3) Eigenvalue decomposition and principal direction extraction: for the covariance matrix Perform eigenvalue decomposition to find the two eigenvectors corresponding to the largest eigenvalue. and ;

[0035] (5.4) Calculation of principal direction angle: Calculate the principal direction angle of the target based on the direction of the feature vector. ,like Figure 4 As shown:

[0036] (11)

[0037] (5.5) Generation of the rotated bounding box: Calculate the length and width w of the rotated bounding box in the main direction and the height h of the rotated bounding box perpendicular to the main direction through projection, where... yes The corresponding coordinates of the four vertices, , The range of projected coordinates ( ), and They are respectively ( The maximum and minimum values ​​of ) and They are respectively ( The maximum and minimum values ​​of )

[0038] (12)

[0039] (13)

[0040] (14)

[0041] (15)

[0042] (5.5) Generation of rotated bounding boxes: based on the principal direction angle and ( Generate accurate rotated bounding box parameters:

[0043] (16)

[0044] in , Corresponding to the displacement of the bounding box corner from the center, for The coordinates of the center point;

[0045] Step 6: Input the rotated bounding box features into the detection head network: (6.1) The detection head network adopts a dual-branch structure. The first branch is used for target classification and outputs the class probability of each candidate region.

[0046] (6.2) The second branch is used for regression prediction, and its width can be obtained from the vertex coordinates. ,high The final output is the coordinates of the center of the rotated frame. ,width ,high and angle parameters ;

[0047] Step 7, Model Training and Optimization:

[0048] (7.1) Calculate the classification loss based on the prediction results and the true labels. Regression loss and angle constraint loss Total loss Defined as:

[0049] (13)

[0050] Where λ is the weighting coefficient, set to 0.1.

[0051] (7.2) The AdamW optimizer is used to perform backpropagation and parameter update. The learning rate is set to 1e-4 and the weight decay coefficient is 0.05. (7.3) The training is repeated until the number of iterations reaches 100, and the optimal rotating target detection model is obtained.

[0052] Step 8, Model Testing and Inference: Input the test set into the trained optimal rotating object detection model and output the object detection results containing the rotated bounding box, including the bounding box coordinates, category, and confidence score.

[0053] This invention proposes a rotating target detection method based on deformable quadrilateral attention and principal orientation analysis. While maintaining accuracy in detecting rotating targets, it improves the algorithm's adaptability to targets with irregular geometries, solving the problem of detecting multi-pose rotating targets in remote sensing images. In terms of feature extraction and geometric modeling, the algorithm introduces a deformable quadrilateral attention mechanism and a principal orientation analysis module. Experimental results on the DOTA-v1.0, HRSC2016, and DIOR-R datasets show that compared to the original O-RCNN, the improved model achieves a 3.2%-5.6% improvement in mAP. In orientation-sensitive categories such as helicopters (HC), the AP improvement is significant (+14.3%), greatly enhancing the detector's accuracy in detecting rotating targets. Attached Figure Description

[0054] Figure 1 This is a flowchart of the rotating target detection method based on deformable quadrilateral attention and principal direction analysis proposed in this invention.

[0055] Figure 2 This is a diagram of the overall network structure of the present invention.

[0056] Figure 3 A visualization of the Principal Direction Analysis (PCA) module.

[0057] Figure 4 This is a schematic diagram of quadrilateral generation during the projection transformation process. Detailed Implementation

[0058] A specific embodiment of the present invention will now be described in detail with reference to the technical solution and accompanying drawings.

[0059] The platform used in this paper is a Windows 11 64-bit operating system, an NVIDIA GeForce RTX 3080ti GPU, and the model was trained using a PyTorch 3.11 deep learning framework based on CUDA 12.1 and CUDNN version 8.9.4. This invention employs a rotating target detection method based on deformable quadrilateral attention and principal direction analysis, with a batch size of 16, a total of 100 iterations, an initial iteration count of 1, a learning rate of 0.0001, and the AdamW optimizer selected. Figure 1 As shown, a method for detecting targets rotating in the main direction using deformable quadrilateral attention includes the following steps:

[0060] The first step is to load the corresponding dataset into this rotating target detection framework;

[0061] The second step is to preprocess the image, converting the input image into the model training format. Preprocessing includes: (2.1) Image scaling: scaling the image to the model input size of 1024×1024×3, where 3 is the number of color image channels;

[0062] (2.2) Normalization: Normalize pixel values ​​to the range of [0, 1]; (2.3) Channel conversion: Convert the image from BGR format to RGB format; (2.4) Box conversion: Convert the original target labels to quadrilateral format to adapt to the rotating target detection task;

[0063] Divide the dataset into training and testing sets, and input the resulting feature images into a dataset such as... Figure 2 In the detection network shown;

[0064] The third step is to input the preprocessed image data into the feature extraction network: (3.1) The feature extraction network inputs an image of size 1024×1024×3 through the first convolutional layer. The convolutional layer has a kernel size of 3, a stride of 2, and a boundary expansion of 1. Then it outputs a feature map of size 512×512 with 64 channels, which is then transmitted to the second convolutional layer.

[0065] (3.2) After passing through the second convolutional layer, the output feature map is 256×256 in size and has 128 channels, which is then transmitted to the third convolutional layer;

[0066] (3.3) After the third convolutional layer, the output feature map is 128×128 in size and has 256 channels. It is then transmitted to the Region Proposal Network (RPN) to generate candidate regions.

[0067] (3.4) The region proposal network outputs multiple candidate regions. The region features of a fixed size are extracted through the RoI alignment layer to obtain a feature map of size 7×7 with 256 channels. These feature maps will be directly input into the deformable quadrilateral attention module.

[0068] The fourth step is to input the feature map output by the feature extraction network into the deformable quadrilateral attention module (DQA) in the detection head:

[0069] (4.1) For each basic window Where w is the window size, set to 7. With the number of channels set to 256, according to formula (2), the feature map... Predict the 9-dimensional transformation parameter vector using global average pooling, LeakyReLU activation function, and a 1×1 convolutional network. ;

[0070] (4.2) According to formula (3), according to the parameter vector Generate composite projection transformation matrix ,like Figure 4 As shown, this matrix is ​​formed by multiplying five basic matrices: scaling matrix Shearing matrix Rotation matrix Translation matrix Perspective Matrix Finally, according to formula (4), the transformation matrix is... It is obtained by multiplying the basic transformation matrices;

[0071] (4.3) The fixed rectangular window is mapped to the target quadrilateral window by projection transformation, as obtained from Equations 5 and 6;

[0072] (4.4) Using the grid sampling function to... , Features are sampled, and then attention weights are calculated;

[0073] Fifth step, input the quadrilateral features output by the deformable quadrilateral attention module into the principal direction analysis module (PCA): (5.1) Input the quadrilateral vertex coordinates Centralized processing is implemented.

[0074] (5.2) Calculate the covariance matrix using the obtained coordinates of the four vertices;

[0075] (5.3) For the covariance matrix Perform eigenvalue decomposition to find the two eigenvectors corresponding to the largest eigenvalue. ;

[0076] (5.4) Calculate the principal direction angle of the target based on the direction of the feature vector. ;

[0077] (5.5) From Equation 12, we obtain the following based on... The generated rotated frame parameters include the center point coordinates. ,width ,high and angle parameters ;

[0078] Step 6: Analyze the prediction results of the rotating bounding box detection network. The detection network adopts a dual-branch structure. The first branch is used for target classification, outputting the class probability of each candidate region. The second branch is used for regression prediction, outputting the center coordinates, width, height, and angle parameters of the rotating bounding box.

[0079] Step 7: Calculate the gradient based on the loss, backpropagate to update the optimizer parameters, and repeat steps 4 to 8 until the number of iterations reaches the maximum, thus obtaining the optimal rotation detection model.

[0080] Step 8: Input the test set into the optimal rotation detection model and output the rotation detection results.

Claims

1. A method for detecting targets rotating in the main direction using deformable quadrilateral attention, characterized in that... Includes the following steps: The first step is to acquire the remote sensing image dataset and preprocess the input data, including image normalization, rotation enhancement, random cropping, and bounding box conversion, converting the original target labels into quadrilateral format to adapt to the rotating target detection task. The second step is to input the preprocessed image into the ResNet feature extraction network to extract feature maps at different scales. In the preprocessing steps, data augmentation includes horizontal flipping, vertical flipping, and random rotation to improve the model's generalization ability; The third step is to input the feature map output by the backbone network into the detection head of a deformable quadrilateral attention module. The module generates a composite projection transformation matrix by predicting the transformation parameter vector, maps the fixed rectangular window into the target quadrilateral window, and uses a sampling function to perform grid sampling on the feature map features, calculates the attention weights, and realizes quadrilateral feature aggregation. The fourth step is to input the quadrilateral features output by the deformable quadrilateral attention module into a main direction analysis module branch, calculate the main direction angle of the target through principal component analysis, and then generate the rotation box parameters based on the main direction angle, including the center point coordinates, width, height and angle information. The sixth step involves inputting the rotated bounding box features into the detection head network, which outputs the target category and bounding box prediction results. The detection head network has a two-branch structure. The first branch is used for target classification, outputting the category probability of each candidate region. The second branch is used for regression prediction, outputting the center, width, height, and angle parameters of the rotated bounding box. Step 7: Calculate the classification loss and regression loss based on the prediction results and the true labels. The total loss includes the classification loss, regression loss and angle constraint loss. Use the AdamW optimizer to perform backpropagation and parameter update, and repeat the training until the number of iterations reaches the preset maximum value to obtain the optimal rotating target detection model. The eighth step is to input the test image into the trained model and output the object detection results containing the rotated bounding box.

2. The method for a detection head of a deformable quadrilateral attention module according to claim 1, characterized in that: In the deformable quadrilateral attention module, the input feature map ( The width and height of the feature map, (Number of channels) is used to predict the 9-dimensional transform parameter vector through global average pooling, LeakyReLU activation, and a 1×1 convolutional network. : Where Conv represents a 1×1 convolution operation, and LeakyReLU is the LeakyReLU activation function applied after average pooling, which divides the feature map X into multiple basic windows of size w×w; based on Calculate the scaling matrix Shearing matrix Rotation matrix Translation matrix and perspective matrix : in =W / w and =H / w is used to scale the translation to fit different input sizes; finally, the transformation matrix T is obtained by multiplying the basic transformation matrices: Given a transformation matrix T, calculate the final coordinates of the target quadrilateral using the standard projection procedure: in This is the median value.

3. The method for branching a main direction analysis module according to claim 1, characterized in that: In the main direction analysis module, the coordinates of the four vertices of the obtained quadrilateral are... First, we represent it as a matrix and then center the data: in , The vertex coordinates are respectively at and Mean of the dimension; calculate the covariance matrix of the centered data. And for Perform eigenvalue decomposition to obtain the two eigenvectors corresponding to the largest eigenvalue. and Its angle It can be calculated using the following formula: The length and width w of the rotated bounding box in the principal direction and the height h of the rotated bounding box perpendicular to the principal direction are calculated by projection. yes The corresponding coordinates of the four vertices, , The range of projected coordinates : in and They are respectively The maximum and minimum values, and They are respectively The maximum and minimum values; based on the principal direction angle and Generate accurate rotated bounding box parameters: ， in , Corresponding to the displacement of the bounding box corner from the center, for The coordinates of the center point.

Images