Meta-detr small sample rotation target detection method based on directional candidate box generation mechanism

Abstract

The application discloses a Meta-DETR small sample rotating target detection method based on a directional candidate frame generation mechanism, and adopts the following steps: the method comprises the following seven steps: step 1, feature extraction; step 2, position coding; step 3, a related aggregation module; step 4, the features are encoded by an encoder; step 5, directional candidate frame generation; step 6, a decoder layer; and step 7, a prediction head; after the directional candidate frame generation mechanism is introduced into the Meta-DETR, the model can more accurately locate the target, reduces the false detection and missed detection, thereby improving the overall detection precision, and the directional candidate frame generation mechanism can utilize the direction information of the target, so that the feature alignment is more effective. This is particularly important for targets with a specific direction.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target detection, more specifically, to a Meta-DETR small sample rotating target detection method based on a directed candidate box generation mechanism. BACKGROUND

[0002] Object Detection is one of the core tasks in the field of computer vision, aiming to locate and identify target objects from images. In recent years, deep learning-based object detection methods (such as Faster R-CNN, YOLO, DETR, etc.) have made significant progress with the support of large-scale labeled datasets (such as COCO, PASCAL VOC). However, these methods usually rely on large-scale labeled data: a large number of labeled samples are needed to train the model, and high-performance hardware is needed to support large-scale training.

[0003] While deep learning models excel in object detection tasks, their reliance on large-scale labeled data presents several challenges: labeling object detection data requires precise bounding box annotations for each object, which is time-consuming and expensive. In some domains (such as medical imaging and remote sensing images), annotation requires specialized knowledge, further increasing costs. In real-world scenarios, object categories often exhibit a long-tail distribution, with minority class samples being scarce, making it difficult for models to learn the features of these categories. In dynamic scenarios, it may be necessary to detect new object categories, but re-annotating and retraining the model is prohibitively costly. Few-shot object detection (FSOD) aims to quickly learn the features of new categories using a small number of labeled samples (typically 1 to 10). FSOD introduces the idea of ​​few-shot learning into object detection tasks, aiming to quickly learn and detect new object categories with only a few labeled samples. It maintains detection performance for known categories while possessing generalization ability for new categories. The emergence of Few-shot object detection (FSOD) technology provides a new solution to these challenges. Few-shot object detection methods can achieve accurate identification and localization of targets with limited labeled data. This is mainly due to the innovations in model design, optimization, and transfer learning in these methods. In terms of model design, few-shot object detection methods emphasize lightweight and efficient models. By employing concise model structures and effective feature extraction methods, these algorithms can learn the essential features of targets from limited data. Simultaneously, by introducing techniques such as attention mechanisms, the model can focus more on key information. The Transformer model, with its unique self-attention mechanism, has achieved significant breakthroughs in natural language processing and is gradually showing broad application prospects in computer vision. Especially in the field of few-shot object detection, Transformer-based methods are gradually becoming the focus of researchers. Unlike traditional convolutional neural networks, Transformer can dynamically focus on different parts of an image and capture the correlations between them, thus performing excellently when handling targets in occluded, overlapping, or complex scenes. However, despite these advantages of the Transformer model, existing methods still face many challenges in few-shot object detection tasks. To address issues such as high model complexity and insufficient detection accuracy, researchers have begun exploring combining Transformer with advanced techniques such as meta-learning to improve the performance of target detection in small samples.The Meta-DETR model, a key representative in this field, combines meta-learning and the DETR framework, achieving effective object detection with limited data. Through a carefully designed model structure and training strategy, the Meta-DETR model can effectively learn feature representations of targets under limited data conditions, thus achieving accurate detection of new target classes. This characteristic makes the Meta-DETR model a promising candidate for small-sample object detection, providing a practical method and means to solve real-world problems in related fields. However, the Meta-DETR model typically performs poorly when detecting angled objects (such as rotated or tilted targets). These objects may appear in scenarios such as remote sensing images, industrial inspection, and text detection (e.g., tilted text). Specifically, this manifests as: inaccurate localization: the detection box cannot accurately cover angled targets, often representing targets with a horizontal bounding box (HBB) instead of a rotated bounding box (RBB). Classification errors: due to feature differences caused by angle changes, the model may struggle to correctly classify targets. Missed detections: Angled targets may be overlooked, especially when the target angle is large or the shape is complex. Therefore, by improving the performance of Meta-DETR in angular target detection tasks, it can be made more widely applicable in fields such as remote sensing, industrial inspection, and text detection. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a Meta-DETR small sample rotation target detection method based on a directed candidate box generation mechanism to achieve the detection of rotation targets with a small number of samples.

[0005] To achieve the above objectives, the technical solution and experimental steps adopted in this invention are as follows:

[0006] The Meta-DETR few-shot rotation target detection method based on the directed candidate box generation mechanism includes the following 7 steps:

[0007] Step 1. Feature extraction;

[0008] The feature extraction stage is divided into two parts: support set feature extraction and query set feature extraction. The task of support set feature extraction is to extract features of the target region from the support set image, while the task of query set feature extraction is to extract full-image features from the query set image, providing input for the object detection task. The feature extractors for the support set and query set are shared to ensure that their feature representations are in the same semantic space.

[0009] Step 2. Location encoding;

[0010] Location encoding enables models to distinguish features at different locations in a feature map by providing spatial location information for the features.

[0011] Step 3. Related aggregation modules;

[0012] The relevant aggregation module effectively integrates the category information provided by the support set into the feature representation of the query set, enabling the model to accurately locate and identify targets in the query image that correspond to the categories of the support set. Specifically, CAM achieves this goal in the following ways:

[0013] Feature alignment: Ensure that the categorical features of the support set are aligned with the image features of the query set in the same semantic space.

[0014] Feature enhancement: Enhance the feature representation of specific categories in the query set by leveraging category information from the support set.

[0015] Contextual association: Captures the association between specific regions in the query set and the categories of the support set through an attention mechanism.

[0016] Step 4. The Encoder encodes the features;

[0017] The enhanced query feature map has incorporated category information from the support set, but it still needs further modeling via the Encoder to capture the global relationships between different locations in the query feature map.

[0018] Step 5. Generate directed candidate boxes;

[0019] When generating directed candidate boxes, first initialize a rotated box, denoted as (p ix p iy p iw p ih p iθ ), where i represents one of the pixels in the feature map, and (p ix p iy )∈[0,1], we use the prediction result Δr and the initial rotated box p obtained from the directed candidate box generation mechanism. i This is used to obtain the final coordinates of each pixel i.

[0020] Step 6. Decoder layer;

[0021] The obtained directed candidate boxes are passed as Object Queries to the Decoder layer. In the Decoder, they interact with the encoded features output by the Encoder to obtain an output embedding containing target feature information. Then, the output is embedded into the input prediction head branch.

[0022] Step 7. Predict the head

[0023] The prediction head branch includes a category prediction head branch and a bounding box prediction branch. A rotation angle prediction branch is added in parallel on top of these two branches. The prediction of the target rotation angle is achieved by using the feature information contained in the embedding vector and a three-layer fully connected network. The ReLU activation function is used to perform non-linear feature mapping before the two adjacent fully connected layers.

[0024] Advantages of this invention:

[0025] Introducing a directed candidate box generation mechanism into Meta-DETR enables the model to more accurately locate targets, reducing false positives and false negatives, thereby improving overall detection accuracy. Furthermore, the directed candidate box generation mechanism can utilize the directional information of the target, making feature alignment more effective. This is particularly important for targets with specific orientations. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the Meta-DETR small sample rotation target detection method based on the directed candidate box generation mechanism provided in an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of aligned convolution provided in an embodiment of the present invention;

[0029] Figure 3 A flowchart of the Meta-DETR small sample rotation target detection method based on the directed candidate box generation mechanism provided in an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of the directed candidate box generation mechanism provided in an embodiment of the present invention. Detailed Implementation

[0031] The present invention will now be further described with reference to the accompanying drawings.

[0032] See attached document Figure 1 The implementation steps of this invention are as follows:

[0033] Step 1. Feature Extraction

[0034] The feature extraction stage is divided into two parts: support set feature extraction and query set feature extraction. Support set feature extraction extracts features of the target region from the support set image, while query set feature extraction extracts full-image features from the query set image, providing input for the object detection task. The weighted shared feature extractor first encodes them into the same feature space. Subsequently, the correlation aggregation module (CAM), which will be detailed later, performs the matching between the query features and the support class set.

[0035] Step 2. Location Encoding

[0036] Location encoding is used as a feature F for the query set. query The feature formula, which provides spatial location information to enable the model to distinguish different locations in the feature map, is shown below.

[0037]

[0038] In the formula, pos represents the position of the embedding vector in the image feature vector, i represents the index in the embedding vector, and d model This represents the output dimension of the model. If the dimension index of the embedding vector is even, the result of the sine formula is used; if the index is odd, the result of the cosine formula is used.

[0039] Step 3. Related Aggregation Module

[0040] The relevance aggregation module differs from existing aggregation methods because it can aggregate multiple support classes simultaneously. This allows it to capture their inter-class correlations, reducing misclassification and enhancing model generalization. Given a query and support features, a weight-sharing multi-head attention module first encodes them into the same feature space, then obtains the prototype of each support class by applying Region of Interest Alignment (RoIAlign) followed by average pooling of the support features. The relevance aggregation module then performs feature matching and encoding matching. Feature matching is accomplished through a single-head attention mechanism, given the support set class features F... class ∈R C×d and query set F query ∈R H×W×d The matching coefficient is obtained by the following formula:

[0041]

[0042] Where W is a linear projection shared by Q and S, ensuring they are embedded in the same feature space. The output of feature matching is then calculated using the following equation:

[0043] Q F =Aσ(F class )⊙F query

[0044] Where σ() represents the sigmoid function and ⊙ represents the Hada code product, σ(F class As a feature filter for each supporting class, σ(S) is designed to extract class-specific features from the query features. By applying the matching coefficient A to σ(S), we filter out any features that do not match the supporting class, producing a feature map Q that highlights objects belonging to the given supporting class. F .

[0045] To achieve meta-learning that requires class-agnostic prediction, a set of predefined task codes is introduced, and a given support class is mapped to these task codes so that the final prediction can be made on the task code rather than the specific class. The task code T∈R is implemented using a sine function. C×d Similar to positional encoding, coded matching uses the same matching coefficients as feature matching, and the matched code Q... E Obtained through the following methods:

[0046] Q E =AT

[0047] Step 4. The Encoder encodes the features;

[0048] The feature map enhanced by CAM already incorporates the category information of the support set, but it still needs further modeling by the Encoder to capture the global relationships between different locations in the query feature map. The Encoder performs global modeling and integrates contextual information on the input enhanced query feature map. Simultaneously, considering the need for few-shot learning, it enhances the ability to model the relationship between the support set and the query set. Through a self-attention mechanism, the Encoder can capture the similarities and differences between the support set and the query set, thereby helping the model better identify targets in the query set that belong to the category of the support set.

[0049] Step 5. Generate directed candidate boxes;

[0050] Meta-DETR is a Transformer-based few-shot object detection model that combines meta-learning and the DETR framework, enabling object detection with a small number of labeled samples. However, the original Meta-DETR's ObjectQueries are randomly initialized, independent of image features, and primarily target horizontal bounding boxes (HBB), failing to directly handle rotated objects (OBB). Furthermore, horizontal bounding boxes often contain significant background and sometimes even multiple objects densely packed together. This leads to pooling operations introducing irrelevant features into the object's features, reducing its match with the ground truth bounding box. Additionally, the intersection-over-union (IoU) ratio between horizontal and ground truth bounding boxes is small, failing to clearly reflect the degree of matching. Therefore, generating directed candidate boxes is essential. We introduce a directed candidate box generation mechanism that learns the angle, center, and size of each candidate box to generate more accurate ones. These are then provided to the decoder as object queries, offering better positional priors. When generating directed candidate boxes, a rotated bounding box is first initialized, represented as (p...). ix p iy p iw p ih p iθ ), where i represents one of the pixels in the feature map, and (p ix p iy In the directed candidate box generation mechanism, which generates boxes ∈ [0, 1], there are two branches: a distance branch and an angle branch. The distance branch predicts the distances d = (l, t, r, b) from the initial rotated box to the four boundaries of the target box, where l is the distance to the left boundary, t is the distance to the top boundary, r is the distance to the right boundary, and b is the distance to the bottom boundary. The loss function uses a logarithmic regression loss. The angle branch predicts the rotation angle θ of the candidate box and uses mean squared error loss for training. Combining the predictions from the distance and angle branches, a coarse directed candidate box B = (x, y, w, h, θ) is generated. After the coarse candidate box is generated, there is an alignment problem between the features and the candidate box. To address this, AlignConv (see [reference]) is introduced. Figure 3 Alignment convolution performs feature alignment. In aligned convolution, the calculation of the offset field is the core step in achieving feature alignment. The goal of aligned convolution is to adjust the sampling points of the feature map using the offset field, aligning them with the geometry of the candidate bounding box (including center point, width, height, and rotation angle), thereby resolving the geometric mismatch between features and candidate boxes. The offset field calculation for aligned convolution is based on the geometric parameters B = (x, y, w, h, θ) of the candidate bounding box. For a standard 3×3 convolution kernel, its sampling point set R is regularly distributed. AlignConv constrains the sampling points within the candidate bounding box, and the position r of the sampling points... boxThe sampling point position r within the candidate box is determined by the geometric parameter B of the candidate box. box The calculation formula is:

[0051]

[0052] Where s i Let p be the stride of the feature map, p be the position index on the feature map, R be the set of sampling points of the standard convolution kernel, and O be the offset of the sampling point relative to the feature map grid. The calculation formula is as follows:

[0053]

[0054] Where r box Let p be the location of the sampling point within the candidate box, p be the position index on the feature map, and r be the sampling point of the standard convolutional kernel. Through alignment convolution, the sampling points of the feature map are aligned to the boundaries of the candidate boxes, thereby eliminating the geometric mismatch between the features and the candidate boxes. The aligned feature maps are then further optimized into coarse candidate boxes using a lightweight fully convolutional network, generating high-quality directed candidate boxes.

[0055] Step 6. Decoder layer

[0056] The obtained directed candidate boxes are passed as Object Queries to the Decoder layer. In the Decoder, they interact with the encoded features output by the Encoder to obtain an output embedding containing target feature information. Then, the output is embedded into the input prediction head branch.

[0057] Step 7. Predict the head

[0058] The prediction head branch includes a category prediction head branch and a bounding box prediction branch. A rotation angle prediction branch is added in parallel on top of these two branches. The prediction of the target rotation angle is achieved by using the feature information contained in the embedding vector and a three-layer fully connected network. The ReLU activation function is used to perform non-linear feature mapping before the two adjacent fully connected layers.

[0059] Experiments and Analysis

[0060] 1. Experimental conditions

[0061] The hardware environment for this chapter's experiments consisted of an Intel(R) Xeon(R) CPU E5-2678, an NVIDIA RTX 3090 GPU, and 32GB of RAM. The model in this chapter was developed using Python and implemented with the PyTorch deep learning framework on Ubuntu 18.04.

[0062] 2. Experimental Data

[0063] This invention uses the MAR20 dataset to train and test the model. The MAR20 dataset has the following three characteristics: (1) Large scale: MAR20 is currently the largest publicly available remote sensing image dataset for military aircraft target recognition, containing 3842 images, 20 military aircraft models, and 22341 instances. In addition, the dataset provides two annotation methods: horizontal bounding boxes and directed bounding boxes, corresponding to horizontal bounding box target recognition tasks and directed bounding box target recognition tasks, respectively, which facilitates researchers to evaluate the performance of different algorithms. (2) Small differences between target classes of different models: Since these 20 fine-grained categories all belong to the same aircraft category, different aircraft models often have similar features, which leads to high similarity between fine-grained categories. The differences between different aircraft models are mainly reflected in the differences of local components. For example, the difference between the SU-35 and SU-34 aircraft models is the canard wing at the nose. (3) Large intra-class differences for targets of the same model: Due to the influence of factors such as climate, season, lighting, occlusion, and even atmospheric scattering during the remote sensing image acquisition process, targets of the same model often have large visual differences and high intra-class differences. In this experiment, classes A1, A2, A3, A4, A5, A6, A8, A9, A12, A14, A15, A16, A17, A18, and A19 in the MAR20 dataset were used as base classes, for a total of 15 classes. Classes A7, A10, A11, A13, and A20 were used as new classes, for a total of 5 classes. During the small sample training process, the number of Object Queries was 300, and the number of Transformer encoder and decoder layers was 6. In the base class training stage, the batch size was 4, the initial learning rate of the model was 0.0002, the training iterations were 50 rounds, and the learning rate was reduced by 1 / 10 at the 45th round. During the few-shot fine-tuning training phase, the initial learning rate for 1-shot was 0.00015, and after 700 training iterations, the learning rate decayed to 1 / 10 at 350 and 600 iterations, respectively. The initial learning rate for 3-shot was 0.0001, and after 600 training iterations, the learning rate decayed to 1 / 10 at 300 and 550 iterations, respectively. The initial learning rate for 5-shot and 10-shot was 0.00005, and after 500 training iterations, the learning rate decayed to 1 / 10 at 250 and 450 iterations, respectively. With an IoU threshold of 0.5, the detection performance of each model on rotated targets in the base class is shown in Table 1. With an IoU threshold of 0.5, the detection performance of each model on rotated targets in the new class is shown in Table 2. Since there is no existing few-shot method for rotating target detection, the detection performance of the original Meta-DETR model on horizontally labeled data is compared with the detection performance of this model on rotated labeled data.

[0064] 3. Experimental Analysis

[0065] We use the same evaluation metric as PASCAL VOC 2007 (Everingham et al., 2010), namely the mean average precision (mAP), which is the average precision (AP) of different target models. Average precision is defined as the area under the precision-recall curve. The calculations for precision and recall are shown below:

[0066]

[0067] In the formula, TP represents true positives, FN represents false negatives, and FP represents false positives.

[0068] Table 1 shows the detection performance of each model on rotated targets in the base class under an IoU threshold of 0.5. Experimental results show that the proposed model improves the target detection performance of the base class by 4.6% compared to the original Meta-DETR model. To compare with other rotated target detection methods, it is compared with R-CenterNet, and the detection performance of this model is 16.6% higher. The few-shot detection capability of the model is validated on the new class of the MAR20 dataset. Since the R-CenterNet model has poor applicability to few-shot target detection tasks, and there are few rotating target detection models based on few shots, this experiment compares the detection performance of the original Meta-DETR model on horizontally labeled data with the detection performance of this model on rotated labeled data. Table 4-2 shows the target detection performance of each model under an IoU threshold of 0.5. Experimental results show that the proposed model improves the performance of the original model by 0.1%, 2.3%, 1.0%, and 9.9% respectively at 1-shot, 3-shot, 5-shot, and 10-shot. Therefore, the model proposed in this chapter can effectively detect rotating targets in small samples and has good small sample detection performance.

[0069] In summary, this invention proposes a Meta-DETR method for detecting rotated targets in small samples based on a directed candidate box generation mechanism. Building upon Meta-DETR, a novel directed candidate box generation mechanism is proposed. By using the generated directed candidate boxes as object queries in the decoder, it provides positional priors for subsequent features. Furthermore, an angle branch is designed to detect the rotation angle of targets in small samples, improving the detection accuracy of the original model and expanding its application scenarios.

[0070] Table 1 shows the experimental results of each model on the MAR20 dataset base class.

[0071]

[0072]

[0073] Table 2 shows the experimental results of each model on the new class of the MAR20 dataset.

[0074]

Claims

1. A Meta-DETR small-sample rotated target detection method based on a directed candidate box generation mechanism, characterized in that, It includes the following 7 steps: Step 1. Feature extraction; The feature extraction stage is divided into two parts: support set feature extraction and query set feature extraction. The task of support set feature extraction is to extract features of the target region from the support set image, and the task of query set feature extraction is to extract full-image features from the query set image to provide input for the target detection task. The feature extractors of the support set and the query set are shared to ensure that the feature representations of the two are in the same semantic space. Step 2. Location encoding; Location encoding enables models to distinguish features at different locations in a feature map by providing spatial location information for the features. Step 3. Related aggregation modules; The relevant aggregation module effectively integrates the category information provided by the support set into the feature representation of the query set, enabling the model to accurately locate and identify targets corresponding to the categories of the support set in the query image. Specifically, CAM achieves this goal in the following ways: Feature alignment: Ensure that the categorical features of the support set are aligned with the image features of the query set in the same semantic space; Feature enhancement: Enhance the feature representation of specific categories in the query set by leveraging category information from the support set; Contextual association: Captures the association between specific regions in the query set and the categories of the support set through an attention mechanism; Step 4. The Encoder encodes the features; The enhanced query feature map has already incorporated the category information of the support set, but it still needs to be modeled by the Encoder to capture the global relationships between different positions in the query feature map; Step 5. Generate directed candidate boxes; When generating directed candidate boxes, first initialize a rotated box, denoted as (p ix p iy p iw p ih p iθ ), where i represents one of the pixels in the feature map, and (p ix p iy )∈[0,1], using the directed candidate box generation mechanism to obtain the prediction result Δr and the initial rotated box p i To obtain the final coordinates of each pixel i; Step 6. Decoder layer; The obtained directed candidate boxes are passed into the Decoder layer as Object Queries. In the Decoder, they interact with the encoded features output by the Encoder to obtain the output embedding containing the target feature information. Then, the output is embedded into the input prediction head branch. Step 7. Predict the head; The prediction head branch includes a category prediction head branch and a bounding box prediction branch. A rotation angle prediction branch is added in parallel on top of these two branches. The prediction of the target rotation angle is achieved by using the feature information contained in the embedding vector and a three-layer fully connected network. The ReLU activation function is used to perform non-linear feature mapping before the two adjacent fully connected layers.

2. The Meta-DETR small-sample rotated target detection method based on a directed candidate box generation mechanism according to claim 1, characterized in that, In step 2, the location encoding is performed using the query set feature F. query The feature formulas that provide spatial location information, enabling the model to distinguish different locations in the feature map, are shown below; In the formula, pos represents the position of the embedding vector in the image feature vector, i represents the index in the embedding vector, and d model This represents the output dimension of the model; if the dimension index of the embedding vector is even, the result of the sine formula is used, and if the index is odd, the result of the cosine formula is used.

3. The Meta-DETR small-sample rotated target detection method based on a directed candidate box generation mechanism according to claim 1, characterized in that, In step 3, feature matching is performed using a single-head attention mechanism, given the class features F of the support set. class ∈R C×d and query set F query ∈R H×W×d The matching coefficient is obtained by the following formula: Where W is a linear projection shared by Q and S, which ensures that they are embedded into the same feature space; Subsequently, the output of feature matching is obtained using the following formula: Q F =Aσ(F class )⊙F query Where σ() represents the sigmoid function and ⊙ represents the Hada code product, σ(F class As a feature filter for each supporting class, σ(S) is designed to extract class-specific features from the query features; by applying the matching coefficient A to σ(S), we filter out any features that do not match the supporting class, producing a feature map Q that highlights objects belonging to a given supporting class. F ; To achieve meta-learning that requires class-agnostic prediction, a set of predefined task codes is introduced, and a given support class is mapped to these task codes to make the final prediction on the task code rather than the specific class. A sine function is used to implement the task code T∈R. C×d Similar to positional encoding, coded matching uses the same matching coefficients as feature matching, and the matched code Q... E Obtained through the following methods: Q E =AT Step 4. The Encoder encodes the features; The feature map enhanced by CAM already incorporates the category information of the support set, but it still needs to be further modeled by the Encoder to capture the global relationships between different positions in the query feature map. The Encoder performs global modeling and integrates contextual information on the input enhanced query feature map. At the same time, in combination with the need for few-shot learning, it enhances the ability to model the relationship between the support set and the query set. Through the self-attention mechanism, the Encoder can capture the similarities and differences between the support set and the query set, thereby identifying targets in the query set that belong to the category of the support set.

4. The Meta-DETR small-sample rotated target detection method based on a directed candidate box generation mechanism according to claim 1, characterized in that, In step 5, a directed candidate box generation mechanism is introduced. By learning the angle, center, and size of each directed candidate box, more accurate directed candidate boxes are generated. These are then provided to the decoder as object queries, providing the decoder with better positional priors. When generating directed candidate boxes, a rotated box representation is first initialized as (p... ix p iy p iw p ih p iθ ), where i represents one of the pixels in the feature map, and (p ix p iy In the directed candidate box generation mechanism, there are two branches: a distance branch and an angle branch. The distance branch predicts the distances d = (l, t, r, b) from the initial rotated box to the four boundaries of the target box, where l is the distance to the left boundary, t is the distance to the top boundary, r is the distance to the right boundary, and b is the distance to the bottom boundary. The loss function uses a logarithmic regression loss. The angle branch predicts the rotation angle θ of the candidate box. It is trained using mean squared error loss. Combining the predictions from the distance branch and the angle branch, a coarse directed candidate box B = (x, yw, h, ..., yw) is generated. After generating coarse candidate boxes, an alignment convolution is introduced for feature alignment. In the alignment convolution, the calculation of the offset field is the core step in achieving feature alignment. The goal of the alignment convolution is to adjust the sampling points of the feature map using the offset field, aligning them with the geometry of the candidate box, including its center point, width, height, and rotation angle. The offset field calculation of the alignment convolution is based on the geometric parameters B = (x, yw, h, θ) of the candidate box. For a standard 3×3 convolution kernel, its sampling point set R is regularly distributed. AlignConv constrains the sampling points within the candidate box, and the position r of the sampling points... box The position of the sampling point r within the candidate box is determined by the geometric parameter B of the candidate box. box The calculation formula is: Where s i Let p be the stride of the feature map, p be the position index on the feature map, R be the set of sampling points of the standard convolution kernel, and O be the offset of the sampling point relative to the feature map grid. The calculation formula is as follows: Where r box Let p be the sampling point position within the candidate box, p be the position index on the feature map, and r be the sampling point of the standard convolution kernel. Through alignment convolution, the sampling points of the feature map are aligned to the boundary of the candidate box, eliminating the geometric mismatch between the feature and the candidate box. The feature map after alignment convolution is then optimized into coarse candidate boxes through a lightweight fully convolutional network to generate high-quality directed candidate boxes.

Images