A single-stage rotating target detection method based on decoupled detection head

Abstract

The application discloses a single-stage rotating target detection method based on a decoupling detection head, and comprises the following steps: S1, performing data preprocessing on a remote sensing image; S2, training a rotating target detection network model on a rotating target detection data set of the remote sensing image through the decoupling detection head; and S3, performing inference through the trained rotating target detection network model to obtain a classification result, an angle classification result, a bounding box offset and a foreground score of the rotating target. The application introduces angle prediction into an existing single-stage horizontal detector for the more common rotating target in a real scene, and realizes rotating target detection.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and specifically to a single-stage rotating target detection method based on a decoupled detection head. Background Technology

[0002] Object detection is one of the fundamental tasks in computer vision. With the continuous development of deep learning, detectors are constantly being updated to further improve detection performance. Based on the prediction stage, current detectors can be broadly divided into two categories: two-stage detectors and one-stage detectors. One-stage detectors directly predict the target bounding box and category; classic algorithms include the YOLO series, SSD, and Retina-Net. Two-stage detectors include an additional region of interest feature extraction, achieving coarse-to-fine target feature extraction; related classic algorithms include R-CNN, SPPNet, Fast R-CNN, Faster R-CNN, and Pyramid Networks. Although significant progress has been made in horizontal object detection in natural environments, the complexity of backgrounds and the large variations in scale, orientation, and shape of object instances in remote sensing images make it difficult to directly apply horizontal detectors to the detection of rotated objects in remote sensing images.

[0003] To predict the position and orientation of rotating objects in remote sensing images, some researchers use five parameters—center point coordinates, width, height, and rotation angle—to describe the position and orientation of the rotating target. Others use four corner point coordinates, for a total of eight parameters, to directly describe the bounding box of the rotated object. Both the five-parameter and eight-parameter methods suffer from boundary discontinuity issues due to the periodicity of the angle and the order of the corner points, leading to instability in the training process and inaccurate model predictions. Therefore, it is necessary to introduce appropriate solutions to address the boundary discontinuity problem. Furthermore, rotating target detection is a comprehensive task comprising four sub-tasks: bounding box regression, foreground score prediction, target classification, and rotation angle prediction. During training, feature misalignment arises due to differences in the targets fitted by the sub-tasks. Implementing the four sub-tasks using a shared detection head leads to contradictions in feature extraction and fails to generate optimal specific features for each sub-task. Currently, for horizontal target detection, researchers have revealed a feature misalignment problem between target classification and boundary regression, which significantly impacts detection performance. Further exploration and resolution of the feature misalignment problem in rotating target detection are crucial for improving its performance. Summary of the Invention

[0004] To address the aforementioned shortcomings in the existing technology, this invention provides a single-stage rotating target detection method based on a decoupled detection head, which solves the problem of low performance in rotating target detection.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: a single-stage rotating target detection method based on a decoupled detection head, comprising the following steps:

[0006] S1. Perform data preprocessing on remote sensing images;

[0007] S2. Train the rotating target detection network model on the rotating target detection dataset of remote sensing images by decoupling the detection head;

[0008] S3. Inference is performed using the trained rotating target detection network model to obtain the classification results, angle classification results, bounding box offset, and foreground score of the rotating target.

[0009] Further: Step S1 specifically involves: cropping a series of 1024×1024 image blocks from the original image, with 200 pixels overlapping between the image blocks, and converting the data labels into a long-side representation, i.e., each label includes the center point coordinates of the target, the long side size, the short side size, and the angle between the long side and the x-axis.

[0010] Furthermore, the rotating target detection network model is constructed by introducing an angle classification module based on circular smooth labels into the YOLOv5 level detector.

[0011] Furthermore, the rotating target detection network model includes a convolutional layer for fusing channel information and three parallel branches, each consisting of a residual layer and a convolutional layer. The residual layer increases the network depth and improves network performance.

[0012] Furthermore: the loss function L for training the rotating object detection network model is:

[0013]

[0014] In the above formula, v′ n 、po′ n ,pc′ n and θ′ n The prediction vector, v, represents the bounding box offset, foreground score, class score, and rotation angle. n ,po n pc n and θ n The target vector, λ, represents the bounding box offset, foreground score, class score, and rotation angle. reg , λ obj , λ cls and λ angle It is to control the hyperparameters of the balance, L reg The loss for bounding box prediction is called CIoU loss, L obj L clsand L angle Let represent the losses for foreground score prediction, classification, and angle classification, respectively, and N be the number of positive samples in each training image, n = 1, 2, ..., N.

[0015] Furthermore: the bounding box offset is restored to the bounding box coordinates. After restoring the bounding box, based on the predicted classification results and foreground scores, overlapping redundant predicted bounding boxes are filtered out by rotation non-maximum suppression.

[0016] The specific formula for restoring the bounding box coordinates is as follows:

[0017] b x =2σ(t) x )-0.5+c x

[0018] b y =2σ(t) y )-0.5+c y

[0019] b w =p w (2σ(t w )) 2

[0020]

[0021] In the above formula, t x t y t w , b represents the predicted center point's horizontal and vertical offsets and its length and width offsets. x b y b w , This represents the x-coordinate, y-coordinate, length, and width of the center point of the restored bounding box. x and c y p represents the x and y coordinates of the top-left corner of the grid where the target center point is located. w and σ represents the length and width of the reference anchor frame, and σ represents the sigmoid equation, which restricts the range of predicted offsets to [0, 1].

[0022] Furthermore, the rotating target detection network model incorporates category labels based on distance penalty, and the formula for calculating the distance penalty is as follows:

[0023]

[0024] In the above formula, gt i ps represents the i-th truth value. i，j This represents the j-th positive sample with the i-th true value. and ps represents positive samples i，j The coordinates of the center point, ps represents positive samples i，j Distance penalty and Indicates gt i The coordinates of the center point of the truth value,

[0025] The formula for calculating category labels based on distance penalty is:

[0026]

[0027] In the above formula, This represents the class label of the j-th positive sample corresponding to the i-th true value after adjustment. This indicates the category labels that were shared before the adjustment.

[0028] Furthermore, the decoupled detection head has three paths, including a classification branch, an angle branch, and a regression-foreground branch, which are used for target classification, angle classification, bounding box regression, and foreground score prediction, respectively. Bounding box regression and foreground score prediction share a single detection head.

[0029] Furthermore: during network training, bounding box regression and foreground score prediction rely on the same fitting objective: the intersection-union ratio of the predicted and ground truth bounding boxes; the fitting objective for angle classification is based on the circularly smoothed angle category label, and the fitting objective for classification is based on the target category label based on the distance penalty term.

[0030] The beneficial effects of this invention are as follows:

[0031] (1) This invention addresses rotating targets, which are more common in real-world scenarios, by introducing angle prediction into the existing single-stage horizontal detector to achieve rotating target detection.

[0032] (2) This invention reveals the feature misalignment problem in single-stage rotating target detection. It fully considers the correlation and contradiction between tasks, constructs a corresponding decoupled detection head, alleviates the feature misalignment problem, and improves detection performance.

[0033] (3) The present invention adds a distance penalty term to the positive sample category label, which fully considers the classification potential of the positive sample, improves the goodness of fit of the classification branch, and further enhances the detection performance. Attached Figure Description

[0034] Figure 1 This is a framework diagram for single-stage rotating target detection based on a decoupled detection head.

[0035] Figure 2 This is a detection head network based on residual networks. Detailed Implementation

[0036] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0037] A single-stage rotating target detection method based on a decoupled detection head is proposed. The main features of this method are:

[0038] (1) An angle classification module based on circular smooth labels was introduced into the YOLOv5 level detector to construct a rotating target detection network model based on YOLOv5.

[0039] (2) A three-way decoupled detection head is proposed to realize classification, orientation prediction, bounding box regression and foreground score prediction respectively. Among them, bounding box regression and foreground score prediction share a detection head.

[0040] (3) A detection head network structure based on residual network was constructed;

[0041] (4) A class label based on distance penalty is proposed, that is, the class label is penalized by the offset of positive samples relative to the true value, and positive samples are constrained in the classification branch.

[0042] Among them, the rotating target detector based on YOLOv5 adds a rotation direction classification module based on a circular smooth label (CSL) to the one-stage horizontal detector YOLOv5, realizing rotating target detection based on YOLOv5 and effectively reducing the loss discontinuity problem in rotating target detection.

[0043] The three-way decoupling detection head proposed in this invention is as follows: Figure 1 As shown, the network includes a classification branch, an angle branch, and a regression-foreground branch, used for object classification, angle classification, bounding box regression, and foreground score prediction, respectively. Bounding box regression and foreground score prediction share a single detection head. During network training, only bounding box regression and foreground score prediction rely on the same fitting objective: the intersection-over-union (IoU) ratio between the predicted and ground truth bounding boxes. The fitting objectives for other subtasks differ: angle classification uses a smooth, circular angle category label, while object classification uses a target category label based on a distance penalty term, leading to feature misalignment. This three-way decoupled detection head continuously updates the network parameters of each branch during training to fit their respective target labels, thereby extracting task-specific features, effectively reducing subtask conflicts, and improving detector performance.

[0044] The detection head splitting network proposed in this invention is based on a residual network, such as... Figure 2 As shown, it includes a convolutional layer for fusing channel information and three parallel branches. Each parallel branch consists of a residual layer and a convolutional layer. The residual layer increases the network depth to improve network performance and prevents gradient vanishing caused by excessive depth in the YOLOv5 network. The convolutional layer reduces the channel dimension to generate the predicted feature vector. This branch network structure fully considers the balance between detection accuracy and network depth and improves detector performance.

[0045] This invention proposes a category labeling method based on distance penalty. Since features closer to the target center are more beneficial for classification tasks (i.e., positive samples closer to the ground truth have stronger classification capabilities), a distance penalty term is added. This results in a smaller penalty for positive samples closer to the ground truth and a larger penalty for samples further away. The penalty term is shown in equation (1), where gt... i ps represents the i-th truth value. i，j This represents the j-th positive sample with the i-th true value. ps i，j The penalty items, and This represents the coordinates of the center point of the positive sample. and This represents the coordinates of the center point of the true value. Relative to the original true value, there are three positive samples that share the same class label. A penalty factor is introduced, as shown in equation (2), where... This represents the class label of the j-th positive sample corresponding to the i-th true value after adjustment. This represents the shared category label before adjustment. This label, with the addition of a distance penalty term, makes the network more likely to fit positive samples close to the true value, thereby improving the accuracy of target classification and ultimately enhancing detector performance.

[0046]

[0047]

[0048] Figure 1 In this paper, single-stage rotating target detection based on a decoupled detection head specifically includes the following steps:

[0049] (1) Data Preprocessing: Due to the large size of a single remote sensing image, a series of 1024×1024 image patches are first cropped from the original image, with 200 pixels overlapping between the patches. Simultaneously, this invention is based on five-parameter rotating target detection, requiring the data labels to be converted to a long-side representation, where each label includes the target's center point coordinates, long-side dimensions, short-side dimensions, and the angle between the long side and the x-axis. The training set, validation set, and test set are then divided into 1 / 2, 1 / 6, and 1 / 3, respectively.

[0050] (2) Network training: The loss function for network training is shown in equation (3), where v′ n 、po′ n ,pc′ n and θ′ n The prediction vector, v, represents the bounding box offset, foreground score, class score, and rotation angle. n ,po n pc n and θ n λ represents the target vector, which includes bounding box offset, foreground score, class score, and rotation angle. reg , λ obj , λ cls and λ angle These are hyperparameters used to control the balance, set to 0.1, 1.0, 0.5, and 0.8 respectively. L reg The loss for bounding box prediction is called CIoU loss, L obj L cls and L angle These represent the losses for foreground score prediction, classification, and angle classification, respectively, all of which are Focal losses.

[0051]

[0052] The network was trained on the publicly available remote sensing image rotation target detection dataset DOTA1.5 for 120 epochs to obtain the optimal model. The initial learning rate was set to 0.01, and the learning rate was updated using a cosine annealing function. The optimizer was selected as the SGD momentum optimizer, with a weight decay of 0.01 and an SGD momentum of 0.937. A warm-up strategy was also used to find a suitable learning rate during the third epoch of training.

[0053] (3) Network reasoning: In the network reasoning stage, such as Figure 1 As shown, the network outputs three feature vectors: classification result, angle classification result, bounding box offset, and foreground score. First, the bounding box offset predicted by the network is restored to bounding box coordinates, as shown in equation (4), where t x t y t w and th This indicates the predicted center point offset and width / length offset, b x b y b w and This represents the center point, length, and width of the restored bounding box. x and c y p represents the coordinates of the top-left corner of the grid where the target center point is located. w and σ represents the length and width of the reference anchor box. σ represents the sigmoid equation, which restricts the predicted offset range to [0, 1]. After obtaining the recovered bounding box, based on the predicted classification results and foreground scores, overlapping redundant predicted bounding boxes are filtered out using rotation non-maximum suppression (R-NMS).

[0054]

[0055] Network inference and model performance were tested on the publicly available DOTA 1.5 test set, using mean average precision (mAP) as the metric. As shown in Table 1, the algorithm achieved the best performance compared to mainstream single-stage and two-stage rotating target detection methods. The abbreviations for the category names in Table 1 are defined as follows: PL - Airplane, BD - Baseball Field, BR - Bridge, GTF - Track and Field, SV - Car, LV - Vehicle, SH - Ship, TC - Tennis Court, BC - Basketball Court, ST - Oil Tank, SBF - Football Field, RA - Roundabout, HA - Port, SP - Swimming Pool, HC - Helicopter, CC - Container Crane.

[0056] Table 1 Comparison of detection results on the DOTA 1.5 test suite

[0057]

[0058] The YOLOv5 horizontal detector adds a rotation direction classification module based on circular smooth labels, realizing rotating target detection based on YOLOv5 and effectively reducing the loss discontinuity problem in rotating target detection.

[0059] The three branches of the decoupled detection head include a classification branch, an angle branch, and a regression-foreground branch, which are used for object classification, angle classification, bounding box regression, and foreground score prediction, respectively. Among them, bounding box regression and foreground score prediction share a single detection head. Since the correlation between sub-tasks and feature misalignment issues are taken into account, this three-way decoupled detection head can extract sub-task-specific features, effectively reduce sub-task conflicts, and improve the performance of the detector.

[0060] The network structure includes a convolutional layer for fusing channel information and three parallel branches. Each parallel branch consists of a residual layer and a convolutional layer. The residual layer increases the network depth to improve network performance and prevents gradient vanishing caused by excessive depth in the YOLOv5 network. The convolutional layer reduces the channel dimension to generate the predicted feature vector. This branch network structure fully considers the balance between detection accuracy and network depth and improves detector performance.

[0061] Instead of the original multiple positive samples corresponding to the same target having the same class label, a class label with a distance penalty term was designed. Specifically, the closer the positive sample is to the ground truth, the smaller the penalty factor, and vice versa. This label with a distance penalty term can make the network tend to fit positive samples that are close to the ground truth, thereby improving the accuracy of target classification and thus improving the performance of the detector.

Claims

1. A single stage rotating target detection method based on decoupled detection head, characterized in that, The method comprises the following steps: S1, data preprocessing is performed on a remote sensing image; S2, a rotating target detection network model is trained on a rotating target detection data set of the remote sensing image by a decoupled detection head; S3, inference is performed by using the trained rotating target detection network model to obtain a classification result, an angle classification result, a bounding box offset, and a foreground score of the rotating target; A loss function L for training the rotating target detection network model is: In the above formula, , , and represent the prediction vector of the bounding box offset, the foreground score, the category score and the rotation angle, , , and represent the target vector of the bounding box offset, the foreground score, the category score and the rotation angle, , , and are hyperparameters for control balance, represents that the loss of the bounding box prediction is CIoU loss, , and respectively represent the loss of the foreground score prediction, the classification, the angle classification, and N is the number of positive samples in each training image, n=1,2,…,N. The bounding box offset is restored to a bounding box coordinate, and after the bounding box is restored, based on the predicted classification result and the foreground score, a rotating non-maximum suppression is performed to filter out redundant predicted bounding boxes; A specific formula for restoring the bounding box coordinate is: In the above formula, , , , represents the predicted center point horizontal and vertical offsets and length and width offsets, , , , represents the horizontal and vertical coordinates of the center point of the recovered bounding box, and the length and width, and represent the horizontal and vertical coordinates of the upper left corner of the grid where the target center point is located, and represent the length and width of the reference anchor box, represents the sigmoid equation, which limits the range of the predicted offset to [0, 1]. A distance penalty-based class label is added to the rotating target detection network model, and a calculation formula of the distance penalty is: In the above formula, denotes the true value, denotes the true value of the positive sample, and denotes the center point coordinate of the positive sample , denotes the distance penalty of the positive sample , and denotes the center point coordinate of the true value; A calculation formula of the distance penalty-based class label is: In the above formula, represents the class label of the jth positive sample corresponding to the ith true value after adjustment, represents the common class label before adjustment.

2. The single stage rotary target inspection method based on decoupled inspection head of claim 1, wherein, The step S1 specifically comprises: a series of 1024*1024 image blocks are cropped from an original image, the image blocks overlap by 200 pixels, and data labels are converted into a long-side representation method, that is, each label includes a center point coordinate of a target, a long-side size, a short-side size, and an angle between the long side and an x axis.

3. The single stage rotary target detection method based on decoupled detection head of claim 1, wherein, The rotating target detection network model is constructed by introducing an angle classification module based on a circular smooth label into a YOLOv5 horizontal detector.

4. The single stage rotary target inspection method based on decoupled inspection head of claim 1, wherein, The rotating target detection network model comprises a convolution layer for fusing channel information, and three parallel branches in parallel, the parallel branches are composed of a residual layer and a convolution layer, and the residual layer increases the network depth and improves the network performance.

5. The single stage rotary target inspection method based on decoupled inspection head of claim 1, wherein, The decoupled detection head has three paths, including a classification branch, an angle branch, and a regression-foreground branch, which are respectively used for target classification, angle classification, bounding box regression, and foreground score prediction, and the bounding box regression and the foreground score prediction share one detection head.

6. The single stage rotary target detection method based on decoupled detection head of claim 5, wherein, In network training, the bounding box regression and the foreground score prediction depend on the same fitting target: an intersection over union of a predicted bounding box and a true value bounding box; the fitting target of the angle classification is a circular smooth angle class label, and the fitting target of the classification is a target class label based on a distance penalty term.

Images