A rotation-based CenterNet2 target detection method

Abstract

The application discloses a CenterNet2 target detection method based on rotation, and designs a rotation angle detection branch based on CenterNet2; the CenterNet2 network is a two-stage anchor-free target detection algorithm, a heatmap and a bounding box regression feature map are obtained through first stage detection, the approximate position of the center point of a target is obtained through the heatmap, the bounding box size of the target is obtained from the bounding box feature map through the index of the center point of the target on the heatmap, and then the center point coordinate of the target and the width and height of the bounding box thereof are obtained; the application constructs a rotation angle detection branch based on CenterNet2, adds an angle regression branch in the one-stage detection module of CenterNet2, and the angle regression branch is used for regressing the rotation angle of each target; the multi-scale fusion feature pyramid structure is utilized, an angle regression branch is added to the output feature maps of multiple scales of the feature pyramid module, the angle regression branch is aligned with the heatmap under the corresponding scale, a rotation angle regression branch based on multi-scale is formed, and the accuracy of angle regression is further improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a rotation-based CenterNet2 object detection method. Background Technology

[0002] Computer vision technology uses computers to perceive and judge the various attributes and spatial relative positions of objects, thus saving human resources. Object detection is a crucial task in computer vision technology, which involves extracting the location information of objects from images and identifying predefined object categories. Thanks to the rapid development of deep learning over the past decade, computers have fully utilized their storage and computational capabilities, allowing them to acquire knowledge from vast amounts of real-world experience without requiring formal descriptions of knowledge from humans. By designing and continuously increasing the number of layers in neural networks, computers can learn more complex experiences, enabling them to solve complex problems in real-world contexts. Due to these advantages, object detection technology has been widely applied in fields such as remote sensing imagery, facial recognition, autonomous driving, and medical imaging.

[0003] Object detection algorithms can be broadly categorized into traditional object detection algorithms and deep learning-based object detection algorithms. Traditional algorithms generate a large amount of redundant candidate region computation and require manual feature design based on different task scenarios, making them unsuitable for complex real-world scenarios. With the rapid development of deep learning, deep learning-based object detection algorithms have demonstrated superior capabilities compared to traditional algorithms. From the perspective of anchor boxes, deep learning-based object detection algorithms can be further divided into anchor-box-based and anchor-free object detection algorithms.

[0004] Anchor-bound object detection algorithms primarily start with the RPN network. After feature extraction from the image through the backbone network, a multi-channel feature map with rich information is obtained. Then, multiple bounding boxes with different scaling ratios and aspect ratios are generated, centered on each pixel in the feature map. These numerous bounding boxes are called anchor boxes. However, anchor-bound object detection algorithms require class discrimination and width / height offset regression for each anchor box. The entire network bears a significant computational burden during the training and prediction phases, resulting in substantial consumption of computer resources. Furthermore, since only a minority of anchor boxes (belonging to the foreground) are truly effective, a large number of anchor boxes belong to the background category, leading to severe sample imbalance and affecting the model's detection speed. Anchor-boundless object detection algorithms can better solve the above problems. They only predict the center position and width / height of the object, eliminating the need to manually design the scaling ratio and aspect ratio of the anchor boxes. This makes the network's head branches lighter, significantly reducing the computational load of the model and saving computational resources.

[0005] Two-stage object detection algorithms are a type of object detection algorithm that offers higher detection accuracy compared to one-stage algorithms. One-stage object detection directly performs multi-class classification and bounding box regression on the feature map after feature extraction via the backbone network. Two-stage object detection algorithms, after feature extraction via the backbone network, use a lightweight binary classification branch and regression branch to obtain coarse bounding boxes. These boxes are then cropped onto the feature map, pooled, and used for the second stage of classification and bounding box regression. Overall, two-stage object detection algorithms fine-tune the results of one-stage object detection, further improving the model's detection accuracy.

[0006] The CenterNet2 network model is a two-stage anchorless object detection algorithm. Current two-stage object detection algorithms, such as the RPN network, aim to maximize recall, generating numerous redundant proposals and increasing the detection burden of the second stage. The CenterNet2 two-stage anchorless object detection model adopts a probabilistic approach, using the anchorless single-stage detector CenterNet network as the first stage to perform binary classification prediction of foreground and background, thereby generating more accurate proposals and reducing the computational cost of the second-stage network. This probabilistic two-stage detector links the probability estimation of the first and second stages with loss and gradient calculations, significantly reducing the model's training speed. In practice, it achieves the same detection speed as the one-stage detector YOLOv4 while maintaining the accuracy of two-stage detection. Summary of the Invention

[0007] To better address the problem of rotating target detection, this invention provides a rotation-based CenterNet2 target detection method, comprising two parts:

[0008] The first part is the design of a rotation angle detection branch based on CenterNet2;

[0009] CenterNet2 is a two-stage anchorless object detection algorithm. The first stage generates a heatmap and a bounding box regression feature map. The heatmap provides the approximate location of the object's center point, and the bounding box size is obtained from the index of the center point on the heatmap, leading to the center point coordinates and the width and height of the bounding box. In the first stage, a rotation angle prediction module is added. This is an angle regression branch added to the end of the main feature extraction network of CenterNet2. A convolutional operation generates an angle regression feature map aligned with the heatmap, and the rotation angle is obtained from the center point index. Simultaneously with predicting proposals in the first stage, the bounding box rotation angle Angle∈[0,180) for each proposal is predicted, with 0 degrees in the positive y-axis direction and 180 degrees in the negative y-axis direction. The proposals are then further classified and the bounding box regression accuracy is improved in the second stage, and finally, the rotation angle is combined with the prediction to output the final result.

[0010] S2. Design a multi-scale rotation angle detection branch;

[0011] To accommodate targets at different scales, the CenterNet2 network employs a Feature Pyramid Network (FPN) structure to fuse target information at various downsampling scales. It integrates the rotation regression branch with the FPN results, designing a multi-scale rotation target detection branch with feature layers at different scales responsible for detecting targets and regressing angles at those scales. For each output branch in the first stage, angle prediction is performed, and the angle predictions from different scales are fused to generate angle-labeled proposals. During training, the angles predicted in the first stage are compared with the ground truth angle labels, and since different scales need to be considered, the angle labels are also bound to their corresponding bounding box labels for calculating the angle loss at the corresponding scale. During inference, the angle information generated based on the multi-scale model is bound to the corresponding prediction boxes. Finally, after the category, bounding box, and angle information are bound together, the second stage of regression and category classification is performed.

[0012] The advantages of this invention compared to existing technologies are as follows: A rotation angle detection branch based on CenterNet2 is constructed. An angle regression branch is added to the one-stage detection module of CenterNet2 to regress the rotation angle of each target. Compared to the CenterNet2 model, each detected bounding box has an additional rotation angle information, allowing the bounding box to more tightly surround the actual object and more accurately describe the object's position in the image. Furthermore, combining the characteristics of CenterNet2's one-stage detector, and fully utilizing its multi-scale fusion feature pyramid structure, an angle regression branch is added to the output feature maps of the feature pyramid module at multiple scales, and aligned with the corresponding thermal images to form a multi-scale rotation angle regression branch, further improving the accuracy of angle regression. Attached Figure Description

[0013] Figure 1 This is a structural diagram of the two-stage CenterNet2 object detection model based on multiple scales.

[0014] Figure 2 This is a structural diagram of the angle regression head branch network module of the present invention.

[0015] Figure 3 This is a structural diagram of the CenterNet2 object detection model based on rotation according to the present invention.

[0016] Figure 4 This is a schematic diagram comparing the experimental results of the present invention with three other methods; (a) CenterNet, (b) CenterNet2; (c) R-CenterNet; (d) R-CenterNet2.

[0017] Figure 5 This is a schematic diagram comparing the experimental results of the present invention with three other methods. (a) CenterNet, (b) CenterNet2; (c) R-CenterNet; (d) R-CenterNet2. Detailed Implementation

[0018] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] like Figure 1 The first stage of detection yields a heatmap and a bounding box regression feature map. The heatmap provides the approximate location of the target's center point, and the index of the target's center point on the heatmap allows us to obtain the size of the target's bounding box from the bounding box feature map. This enables us to obtain the coordinates of the target's center point and the width and height of its bounding box.

[0020] like Figure 3This invention employs a Feature Pyramid (FPN) structure to fuse target information at different downsampling scales. It integrates the rotation regression branch with the feature pyramid results to design a multi-scale rotation target detection branch. Feature layers at different scales are responsible for the detection and angle regression of targets at different scales.

[0021] The specific technical implementation analysis is described below:

[0022] The implementation of the rotation-based CenterNet2 object detection method can be divided into the following two steps: constructing a rotation-based CenterNet2 network model and training the rotation-based CenterNet2 network model. For ease of understanding, the specific steps of the embodiments of the present invention are described below with reference to the accompanying drawings.

[0023] Step 1: Construct a rotation-based CenterNet2 object detection network;

[0024] Step 1.1. Backbone Network;

[0025] The Res2Net101 model is used as the backbone feature extraction network of CenterNet2 to extract features from the input image. Compared with ResNet, each building unit of the Res2Net model adds small residual blocks to the original residual unit structure, which increases the size of the receptive field of each layer. By using grouped convolution, the output channel dimension is divided into multiple groups on an even scale, and convolution is performed on each group. Finally, the output results are stacked in the channel dimension. Without increasing the computational cost of the model, the model can capture more details and global features.

[0026] Conventional convolution operations involve sliding a square kernel (which appears as a square in a two-dimensional plane) across the feature map to capture features within the area covered by the kernel (receptive field). However, targets in real-world images can be of arbitrary shapes and often rotate at different angles. Therefore, this invention employs deformable convolution in the Res2Net101 model. Compared to ordinary convolution, deformable convolution adds learnable positional offset parameters to the kernel. For a 3×3 deformable convolution kernel, in addition to the weight parameters at nine positions, it also includes offsets at each position of the kernel. The offset parameter R for a regular receptive field can be represented by formula (1). When encountering targets with different shapes, the position of the convolution operation in the feature map can be adjusted by learning the offset at each position.

[0027] R = {(-1,-1),(-1,0),…,(0,1),(1,1)} Equation (1)

[0028] When a convolution kernel actually performs a convolution operation, the operation position can be discretely distributed across multiple locations on the feature map to adapt to feature extraction of targets with different shapes and enhance the model's feature extraction capability.

[0029] Step 1.2. Feature Pyramid Network;

[0030] The Feature Pyramid Network takes several downsampled feature layers from the backbone feature extraction network as input. This invention uses the third, fourth, and fifth downsampled feature maps from Res2Net as input to the Feature Pyramid module. This module fuses the three feature maps from bottom to top (feature maps from smallest to largest) to enhance the information contained in the feature maps. Furthermore, to enhance the model's ability to detect large-scale targets, this invention uses a method of downsampling the input features from the fifth downsampled layer twice more to generate the sixth and seventh downsampled feature maps. Finally, the third, fourth, fifth, sixth, and seventh downsampled feature maps are fed into the next stage of the model.

[0031] Step 1.3. CenterNet one-stage detection;

[0032] The CenterNet2 network employs the CenterNet network structure for coarse target detection in the first stage. The CenterNet network head branch includes a heatmap prediction branch and an object bounding box prediction branch. During this process, the model generates a large number of proposals based on the five feature maps output by the feature pyramid structure. This invention adds an angle prediction branch to the first-stage detection head branch of CenterNet, such as... Figure 2 .

[0033] For each feature map, firstly, it undergoes a first convolutional layer with a 3×3 kernel, stride = 1, padding = 1, bias = True, and 256 input and output channels. Secondly, it passes through a ReLU activation function for non-linear mapping. Thirdly, it undergoes a second convolutional layer with a 1×1 kernel, stride = 1, padding = 0, bias = True, 256 input channels, and 1 output channel, and then passes through a ReLU activation function again. Finally, a single-channel feature map with the same size as the input resolution is obtained, which is the angle prediction result, and this feature map corresponds one-to-one with the pixels in the heatmap.

[0034] After obtaining the center point of the target from the heatmap, the coordinate index of the target in the heatmap is obtained. The corresponding pixel value is then obtained from the angle prediction map using this coordinate index; this pixel value represents the rotation angle of the target. To fully utilize multi-scale feature information, this invention applies angle prediction branches to all five input feature maps. Each angle prediction branch corresponds to a heatmap prediction branch at the corresponding scale; that is, the same index corresponds to the same target. Through this stage, the model obtains potential targets based on heatmaps at five different scales and obtains the target's center point coordinates, bounding box width and height, and rotation angle value.

[0035] Step 1.4. Cascade two-stage detection head branches;

[0036] The cascaded two-stage detection head is a model that cascades multiple detection networks. By progressively increasing the Intersection over Union (IOU) threshold of the proposals, the resampled proposals from the previous detection network can adapt to the next detection network with a higher threshold. In the cascaded detection stage, each detector takes the proposals obtained from the previous detector and the third and fourth downsampled feature maps output from the feature pyramid as input. Since the target information obtained in the previous stage is obtained at five different scale feature maps, the proposal boxes first need to be mapped to the third and fourth scale feature maps. Second, feature information is extracted from the feature maps based on the target's coordinates, and the feature maps of different sizes are pooled to unify them into a 7×7 resolution feature map. After each feature map is expanded, it is fed into the classification branch and the bounding box regression branch to obtain the category information and the fine-tuned bounding box for each target. Finally, after non-maximum suppression (NMS) operation, redundant detection boxes are removed, resulting in the final target boxes with rotation.

[0037] Step 2: Train the rotation-based CenterNet2 object detection model;

[0038] Step 2.1. Forward propagation;

[0039] This invention is based on two-stage object detection using CenterNet2. Therefore, a batch of images needs to go through the first stage of detection to obtain proposal boxes. Then, the proposal boxes are mapped onto the feature map, and the regions of interest (ROIs) are cropped. After pooling, all ROIs are adjusted into small 7×7 feature maps. Finally, the width and height dimensions of the feature maps are flattened, and then a fully connected operation is performed to obtain the final classification information and more accurate bounding box regression information.

[0040] First, the input image is normalized, with the mean of the RGB three channels being [123.675, 116.280, 103.530] and the variance being [58.395, 57.12, 57.375]. Second, the input image is randomly cropped and scaled, ensuring that the longest side does not exceed 640 pixels to avoid overfitting the dataset during model training. If the input image size is 640×640, after feature extraction by the backbone network Res2Net101, the feature maps after the third, fourth, and fifth downsampling are input into the feature pyramid network. The sizes of the three feature maps are [B,C,80,80], [B,C,40,40], and [B,C,20,20], respectively. The feature pyramid network fuses deep semantic information with shallow texture information to obtain richer target information, where B represents the batch size and C represents the number of channels in the current feature map. Furthermore, to improve the detection capability for large targets, the feature map from the fifth downsampling is downsampled twice more to obtain feature maps of sizes [B,C,10,10] and [B,C,5,5], respectively. Finally, the feature pyramid module outputs five feature layers of different scales for the first-stage target detection.

[0041] The first-stage detector processes five feature maps through four independent detection branches: a heatmap detection branch, a coordinate offset branch, a width-height regression branch, and an angle regression branch. The angle regression branches output feature maps at five different scales with sizes of [B,1,80,80], [B,1,40,40], [B,1,20,20], [B,1,10,10], and [B,1,5,5]. Based on the size of the ground truth bounding box, the target is mapped to a feature layer that matches the current scale, which is responsible for object prediction. After the target is matched with the heatmap, globally unique index coordinates at multiple scales are obtained. These index coordinates are used to obtain the predicted target's position offset, width-height regression values, and angle regression values, thus generating a large number of proposals.

[0042] Based on the probability-based interpretation of CenterNet2, the number of proposal boxes is reduced from 2000 to 512, reducing the computational cost of the two-stage detector. A cascaded detector system is used in the second stage, employing three cascaded detectors. The thresholds for positive IOU (Intersection over Union) between the proposal box and the ground truth box are [0.6, 0.7, 0.8]. After three cascaded detection stages, the classification result and bounding box regression value for each proposal box are obtained. Finally, the predicted boxes are filtered according to the thresholds, and redundancy is removed using Non-Maximum Suppression (NMS) to obtain the final prediction result.

[0043] Step 2.2. Loss Function;

[0044] After building the model, the total loss during the entire network training process includes two stages of loss. The first stage is the one-stage detection loss, which consists of a binary classification loss L for foreground and background. cls Boundary regression loss L box , Angle regression loss L ang It consists of three parts, and the first-stage loss formula is expressed as equation (2).

[0045] L one =λ cls L cls +λ bbox L bbox +λ ang L ang Equation (2)

[0046] Where λ cls , λ bbox , λ ang These are the coefficients of classification loss, bounding box regression loss, and angle regression loss, respectively. The coefficients of the three hyperparameters used in this invention are 0.5, 1.0, and 0.05, respectively.

[0047] For binary classification loss L cls The present invention uses the Focal Loss function, as shown in equation (3).

[0048]

[0049] Where α represents the weight hyperparameter of positive and negative samples, and γ represents the learning weight of difficult examples. In this invention, the α hyperparameter is 0.25 and the γ hyperparameter is 2. Focal Loss can alleviate the class imbalance problem and increase the contribution of loss to difficult learning samples.

[0050] For the bounding box regression loss L bbox The present invention uses the GIOU loss function, as shown in equation (4).

[0051]

[0052] Where C represents the area of ​​the minimum bounding rectangle of the predicted bounding box and the ground truth bounding box, and A∪B represents the union of the areas of the predicted bounding box and the ground truth bounding box. The GIOU loss function can solve the problem of unclear optimization direction when the predicted bounding box and the ground truth bounding box do not overlap.

[0053] For the angle regression loss L ang The present invention uses the Smoooth L1 loss function to calculate the regression error of the angle, as shown in formula (5).

[0054]

[0055] In the formula, x represents the difference between the predicted angle value and the true angle value. The Smooth L1 loss function combines the advantages of the L1 loss function and the L2 loss function. When the difference between the regressed angle value and the true value is large, the gradient will not be too large, which can ensure the stability of the gradient. When the difference between the angle value and the true value is small, the gradient can smoothly decrease, so that the angle value can converge to the optimal value.

[0056] The detection loss in the second stage is the loss of the cascaded detectors, which includes the losses of three cascaded components. The loss for each cascaded component is further divided into two parts: the classification loss L. c And bounding box regression loss L reg The formula is expressed as in equation (6).

[0057]

[0058] Among them, the bounding box regression loss L reg The Smooth L1 loss function is used to regress the center point offset and width and height offset of the bounding box, respectively, and the cross-entropy loss function is used for classification loss.

[0059] Cross-entropy loss function L c The formula is expressed as in equation (7).

[0060]

[0061] Where p represents the predicted probability value of the prediction result under the current category after softmax.

[0062] The total loss in the second stage includes the losses of the three cascaded parts, as expressed in equation (8).

[0063]

[0064] Thus, the total loss function of CenterNet2 is expressed as in equation (9).

[0065] L total =L one +L two Equation (9)

[0066] Step 2.3. Optimizer;

[0067] Stochastic Gradient Descent (SGD) selects a mini-batch instead of the entire dataset each time and uses gradient descent to update the model parameters. The parameter update steps are as follows: calculate the gradient of the objective function with respect to the current parameters, calculate the descent gradient at the current time step, and update the parameters based on the descent gradient. Let the parameters to be optimized be ω, the objective function be f(x), the initial learning rate be α, and the number of iterations be epochs, where g... t This is the gradient of the current parameters. The parameter update formula is as shown in equation (10):

[0068] W t+1 =W t -α·g t Equation (10)

[0069] The CenterNet2 network model uses gradient descent for optimization, with a batch size of 8, a maximum iteration count of 47475, and a learning rate α initialized to 0.01. The learning rate is then decayed to 1 / 10 of its current value at iterations of 15000 and 25000. The optimizer guides the parameters of the loss function (objective function) to update appropriately in the correct direction, ensuring that the updated parameters continuously approach the global minimum of the loss function.

[0070] The experimental setup and analysis are briefly described below. Through comparative analysis of the experimental results, the actual detection effect has been improved.

[0071] 1. Experimental conditions

[0072] The hardware test platform for this invention consists of an Intel(R) Core(TM) i7-10700F CPU with a clock speed of 2.90 GHz, 16 GB of memory, and an Nvidia GeForce GTX 3060 Super graphics card. The software platform is an Ubuntu 18.04 operating system, and the development environment is Visual Studio Code. The programming language is Python, and the deep learning network architecture is PyTorch.

[0073] 2. Experimental Data

[0074] The performance evaluation of this invention primarily uses the HRSC2016 dataset: a high-resolution ship dataset collected by Google Earth for scientific research.

[0075] The HRSC2016 dataset contains images of two scenarios: ships at sea and ships in coastal waters. All images were collected from six well-known ports, with ships grouped into a single category across all images. Image resolutions range from 2-m to 0.4-m, and image sizes range from 300×300 to 1500×900, with most exceeding 1000×600. This patent selected 626 images as the training set and 444 images as the test set for experimental verification and comparison.

[0076] 3. Performance Comparison

[0077] The three prior art comparison and classification methods used in this invention are as follows:

[0078] (1) ZhouX et al. proposed CenterNet, a one-stage detection model based on no anchor box, in "Objects as points". CenterNet is used to locate the target by predicting the center point, offset, width and height of the target.

[0079] (2) Based on the CenterNet method, the angle prediction of the bounding box is added. An angle rotation regression branch is added to the head branch of the CenterNet model, which is referred to as the R-CenterNet method.

[0080] (3) Zhou et al. proposed a probability-based two-stage target detection algorithm without anchor boxes in “Probabilistic two-stage detection”, referred to as the CenterNet2 method.

[0081] In the experiment, since there is only one target category, the AP (Average Precision) metric was used to evaluate the performance of the present invention, and the method proposed in this invention is denoted as R-CenterNet2.

[0082] For AP calculation, within a specific category, different confidence thresholds are set. Samples exceeding the confidence threshold are considered positive samples. The Intersection over Union (IOU) ratio is calculated between positive samples and all actual ground truth bounding boxes. When the maximum IOU exceeds a preset IOU threshold (0.5, 0.75), the target is considered a true positive (TP); otherwise, the target box is considered a false positive (FP). Under the same IOU threshold, a higher AP value indicates better model detection performance.

[0083] Table 1 shows the AP values ​​of the present invention and three other methods under different crossover ratio (CUP) thresholds in the HRSC2016 dataset.

[0084] Table 1

[0085]

[0086] As can be seen from Table 1, under the ship category in the HRSC2016 dataset, the detection method R-CenterNet2 proposed in this invention has a higher detection AP than other detection methods under different intersection-union thresholds, and is significantly higher than horizontal bounding box target detection methods.

[0087] Experimental results comparison Figure 4 , Figure 5 As can be seen from the performance comparison chart, the CenterNet and CenterNet2 methods lack rotation angle values, resulting in a low intersection-union (IU) ratio between the detected bounding boxes and the ground truth objects, leading to fewer true positive (TP) samples and thus lower AP values. The R-CenterNet method, by introducing a rotation detection branch, shows a significant improvement in AP compared to the previous two methods. However, it only uses feature maps at a single scale for target detection, resulting in poor performance for detecting targets of different sizes within the same image. The R-CenterNet2 method proposed in this patent utilizes both multi-scale and rotation detection branches, enabling the model's target detection boxes to have rotation angle values ​​while also accommodating the detection of targets of different scales within the same image. Compared to the R-CenterNet method, the AP value is improved by approximately 30.7%.

[0088] In summary, to better address the problem of target detection with rotation angles in real-world scenarios, this invention proposes a rotation-based CenterNet2 target detection method. A rotation angle detection branch based on CenterNet2 is constructed, adding an angle regression branch to the original one-stage detection head branch. The output size is aligned with the heatmap index, and a rotation angle is added to the output suggestion box, thereby improving detection performance. A multi-scale rotation angle detection branch is employed, combined with the feature pyramid structure of the CenterNet2 network, further enhancing the accuracy of target rotation angle regression, making the model more accurate in detecting rotating targets. Experimental data analysis shows that the proposed method effectively improves the AP value of target detection and outperforms the R-CenterNet target detection network with rotation.

Claims

1. A rotation-based CenterNet2 target detection method, characterized in that: The network model based on the rotation of CenterNet2 includes the following steps: Step 1.

1. Main network; The Res2Net model is used as the backbone feature extraction network of CenterNet2 to perform feature extraction on the input image. In the Res2Net model, each construction unit adds a small residual block to the original residual unit structure, which increases the receptive field size of each layer. The output channel dimension is divided into multiple groups using group convolution, and the results are stacked in the channel dimension. This method captures more details and global characteristics without increasing the computational complexity of the model. Variable convolution is used in the Res2Net model. The offset parameter R for the regular receptive field can be represented by formula (1). When encountering targets with different shapes, the offset of each position is learned to adjust the operation position of the convolution operation in the feature map. （1）； When the convolution kernel actually performs convolution operation, the operation position can be discretely distributed in multiple positions on the feature map to adapt to the feature extraction of targets with different shapes and enhance the feature extraction capability of the model. Step 1.

2. Feature pyramid network; The feature pyramid network takes some down-sampled feature layers of the backbone feature extraction network as input and uses the third, fourth, and fifth down-sampled feature maps of Res2Net as input to the feature pyramid module. The three feature maps are fused from bottom to top to enhance the information contained in the feature maps. The input feature of the fifth down-sampling is further down-sampled twice to generate the sixth and seventh down-sampled feature maps. Finally, the third, fourth, fifth, sixth, and seventh down-sampled feature maps are sent to the next stage of the model. Step 1.

3. CenterNet one-stage detection; The CenterNet2 network uses the CenterNet network structure in the one-stage to complete the rough detection of the target. The CenterNet network head branch includes a heat map prediction branch and an object bounding box prediction branch. In this process, the model generates a large number of proposal boxes based on the five feature maps output by the feature pyramid structure. In the CenterNet one-stage detection head branch, an angle prediction branch is added. For each feature map, a single-channel feature map with the same size and input resolution is obtained through convolution, which is the angle prediction result. The feature map is one-to-one corresponding to the pixel points in the heat map. After obtaining the center point of the target from the heat map, the coordinate index of the target in the heat map is obtained. The pixel value at the corresponding position in the angle prediction map is obtained through the coordinate index, which is the rotation angle of the target. Step 1.

4. Cascaded two-stage detection head branch; The two-stage detection head in cascade is a model of cascading multiple detection networks, and by gradually increasing the intersection over union IOU threshold of Proposal, the Proposals resampled by the previous detection network can adapt to the next detection network with a higher threshold. In the cascade detection stage, each detector takes the Proposals obtained by the previous detector and the third and fourth down-sampling feature maps output by the feature pyramid as inputs. Since the target information obtained in the previous stage is obtained under five different scale feature maps, the proposal box needs to be mapped into the third and fourth scale feature maps first, and secondly, the feature information is intercepted from the feature map according to the coordinate information of the target, and the feature maps of different sizes are pooled into a feature map with a resolution of 7 7. After each feature map is unfolded, it is sent to the classification branch and the bounding box regression branch, and then the class information and the fine-tuned bounding box of each target are obtained. Finally, after the non-maximum suppression NMS operation, the redundant detection boxes are removed, and the final target box with rotation is obtained.

2. The rotation-based CenterNet2 target detection method according to claim 1, wherein: Training the CenterNet2 target detection model based on rotation includes the following steps: Step 2.

1. Forward propagation; Two-stage target detection based on CenterNet2, so a batch of images needs to go through the first stage detection to get the proposal box, then map the proposal box to the feature map, and cut the ROI of interest. After the pooling operation, all the regions of interest are adjusted to 7 7 small feature maps, and finally the width and height dimensions of the feature map are flattened, and the final classification information and more accurate bounding box regression information are obtained through full connection operation. The one-stage detector passes the five feature maps through the heat map prediction branch, coordinate offset branch, width-height regression branch, and angle prediction branch, respectively. Each input feature map corresponds to an independent four detection branches. According to the size of the real target frame, the target is mapped into the feature layer conforming to the current scale, and the feature layer is responsible for predicting the object; After the target is matched with the heat map, global unique index coordinates at multiple scales are obtained, and the position offset, width and height regression values and angle regression values of the predicted target are obtained according to the index coordinates, and then the proposal frame is obtained; after three cascade stages of detection, the classification result and the bounding box regression value of each proposal frame are obtained, and finally the threshold is filtered, and after the redundant prediction frame is removed by NMS, the final prediction result is obtained; Step 2.

2. Loss function; After the model is constructed, the total loss contains two-stage losses during the training process of the whole network; the first stage is one-stage detection loss, which is divided into three parts: two-classification loss of foreground and background , bounding box regression loss , and angle regression loss , and the one-stage loss formula is shown as formula (2). （2）； wherein , , are coefficients of classification loss, bounding box regression loss, and angle regression loss, respectively, and three hyperparameters coefficients are 0.5, 1.0, and 0.05, respectively. For binary classification loss , a loss function is adopted, as in equation (3). （3）； wherein, a weight hyperparameter representing positive and negative samples, a weight representing hard example learning; for the bounding box regression loss , a loss function is adopted, formula as formula (4); loss function, formula as formula (4); （4）； wherein C represents an area of a minimum circumscribed rectangle of the predicted bounding box and the real bounding box, represents a union of areas of the predicted bounding box and the real bounding box, The loss function can solve the problem that the optimization direction is not clear when the predicted bounding box and the real bounding box do not overlap. For angle regression loss ,use The loss function calculates the regression error of the angle, as shown in formula (5). （5）； In the formula represents the difference between the angle prediction value and the true value; The loss function combines The loss function and The loss function has the advantages that when the difference between the regression angle value and the true value is large, the gradient will not be too large, and the stability of the gradient can be ensured; when the difference between the angle value and the true value is small, the gradient can be smoothly reduced, so that the angle value can converge to the optimal value. The detection loss of the second stage is the loss of the cascade detector, including the loss of three cascade parts; and the loss of each cascade part is divided into two parts, i.e. the classification loss and the bounding box regression loss , which is expressed as formula (6); （6）； Wherein, the bounding box regression loss Adopt Loss function, respectively, the center point offset and width height offset of the bounding box are regressed, and the classification loss adopts cross entropy loss function; cross-entropy loss function The formula is represented as equation (7); （7）； wherein, represents the predicted probability value under the current class after the prediction result has been processed by the prediction model. The total loss of the second stage contains the loss of the three cascade parts, and the formula is represented as formula (8); （8）； Up to now, the total loss function of CenterNet2 is represented as formula (9); （9）； Step 2.

3. Optimizer; a gradient of the target function with respect to the current parameter is calculated, a descending gradient at the current time is calculated, and the parameter is updated according to the descending gradient; the parameter to be optimized is set as , the target function is , the initial learning rate is , the iteration period is epoch, and wherein is the gradient of the current parameter; and the parameter update formula is as formula (10): （10）； The CenterNet2 network model is optimized using gradient descent, a batch size BatchSize of 8, a total iteration number Max Iteration of 47475, a learning rate initialized to 0.01, and the learning rate is decayed to the current value at iterations 15000 and 25000, respectively; and an optimizer that updates the parameters of the loss function in the right direction and by the right amount, so that the updated parameters make the loss function value continuously approach the global minimum.

Images