Method for implementing a weakly supervised training of a rotated object detector with horizontal bounding boxes

Abstract

The application discloses a kind of weakly supervised training rotating target detector implementation methods with horizontal bounding box, by using the image training set with only target class and HBB label to weakly supervise training OBB detector model, and delete the redundant components only used in weakly supervised training in trained model, obtain the OBB target detector that can directly predict OBB frame, in online stage, the data to be detected of same task is inferred, and OBB target detection result is obtained.The application uses image data and corresponding artificial labeling horizontal bounding box HBB, directly weakly supervises training neural network by HBB label, obtains the rotating target that can be identified, detected, and gives target rotating bounding box OBB, and the detection performance of OBB detector obtained by the application is substantially the same as the contrast detector trained by using OBB label while reducing labeling cost.

Description

Technical Field

[0001] This invention relates to a technology in the field of neural network applications, specifically a method for implementing a rotating target detector trained with weak supervision using horizontal bounding box annotations. Background Technology

[0002] In many applications that require precise detection of target size and orientation, such as satellite and UAV remote sensing image target tracking and robot object picking, rotating target detectors (OBB target detectors) that can identify the oriented bounding box (OBB) of the target are more functionally advantageous than horizontal target detectors (HBB target detectors) that can only identify the horizontal bounding box (HBB) of the target, because OBB target detectors can predict the precise size and orientation of rotatable targets.

[0003] In general practice, HBB object detectors can be directly trained using datasets with HBB manual annotations, and OBB object detectors can be directly trained using datasets with OBB manual annotations. However, to date, no existing method can directly train an OBB object detector using only datasets with HBB manual annotations. Furthermore, manually annotating OBB is 10% to 20% more expensive than manually annotating HBB. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by proposing a method for weakly supervised training of a rotated object detector using horizontal bounding box annotations. This method allows for the direct weak-supervised training of an OBB object detector using a dataset containing only HBB (Hard Boolean) annotations. While reducing annotation costs, the OBB object detector obtained by this method achieves performance comparable to or even surpasses that of a control OBB object detector trained on a dataset with HBB annotations. Therefore, this method offers advantages in both performance and cost.

[0005] This invention is achieved through the following technical solution:

[0006] This invention relates to a method for implementing a rotated target detector trained with weak supervision using horizontal bounding box annotation, comprising:

[0007] Step S1: For a specific directed bounding box (OBB) target detection task, manually collect the image data of the task, and have the annotator annotate the target category and horizontal bounding box (HBB) in the task image data. Construct an image training set with only target category and HBB annotation based on the image and the corresponding annotation, and preprocess the image training set with HBB annotation.

[0008] The bounding box refers to an image, feature map, or high-dimensional feature tensor with the top-left pixel coordinates (0, 0), the positive x-axis direction being the right of the image, the positive y-axis direction being the bottom of the image, and the distance in pixels.

[0009] The horizontal bounding box (HBB) is represented by four real numbers (x, y, w, h), where (x, y) are the horizontal and vertical coordinates of the center of the HBB in the image coordinate system, (w, h) are the length and width dimensions of the HBB, and the side with length w is parallel to the x-axis.

[0010] The directed bounding box (OBB) is represented by five real numbers (x, y, w, h, θ), where (x, y) are the x and y coordinates of the OBB's center in the image coordinate system, (w, h) are the length and width dimensions of the OBB, and θ is the rotation angle of the OBB in radians. θ is defined as 0 when the side of length w is parallel to the x-axis, and the +θ direction is clockwise.

[0011] The preprocessing described varies depending on the OBB target detection task: For remote sensing image OBB target detection tasks, due to the large image size, the preprocessing includes: large image slicing, image RGB pixel value normalization, and online image enhancement; For natural scene text OBB target detection tasks, robot target object grasping, and other tasks, the image size is generally small, so the preprocessing includes: image RGB pixel value normalization and online image enhancement.

[0012] The aforementioned large image slices specifically include:

[0013] 1) Image Sliding Window Slicing: Let the length and width pixel dimensions of the input image I be W×H. Select the sliding window step size S and the slice window size M, then the image can be sliced ​​in the x-direction using a sliding window. The window position and y-direction can be slid to obtain... At each window position, the image is placed in a set of smaller images I after being sliced. split ={I 00 I 01 ,...I 10 I 11 ,...I P-1Q-1}, where small figure I ab (x, y) = I(x + aS, y + bS), with a size of M × M.

[0014] 2) HBB annotation sliding window segmentation: for small image set I split For each small image in the diagram, select HBBs whose midpoint coordinates lie within the small image size range as the cutout labels for that small image. That is, when the midpoint coordinates (x, y) of any HBB: (x, y, w, h) lie within the range of the small image size, select the cutout labels for that small image. abWithin the corresponding window range, the corresponding (x-aS, y-bS, w, h) will be filtered as I. ab The cut-out label.

[0015] The aforementioned image RGB pixel value normalization specifically refers to: when the input image The image after pixel value normalization Where: μ r μ g μ b For the normalized mean, σ r , σ g , σ b To normalize the standard deviation, fine-tuning can be performed on the task data.

[0016] The aforementioned online image enhancement specifically refers to: using processing methods including but not limited to random flipping, random color transformation enhancement, and random noise to enhance images online, thereby increasing the diversity of training data.

[0017] Step S2: Construct a weakly supervised training network for the OBB object detector. Use an image training set containing preprocessed training images, manually labeled object categories, and manually labeled object bounding boxes for weakly supervised training to optimize the model's weights.

[0018] The weakly supervised training network of the OBB object detector includes: a first transformation module, a second transformation module, a first feature extraction network, a second feature extraction network, a classification network, a first regression network, a second regression network, a first OBB decoder, a second OBB decoder, a first inverse transformation module, a second inverse transformation module, a first allocator, a classification loss calculation module, a centrality loss calculation module, a circumscribed regression loss calculation module, a second allocator, and a transformation consistency loss calculation module. The first transformation module, the second transformation module, the first inverse transformation module, and the second inverse transformation module work collaboratively to transform and inversely transform the image and the OBB detection result, respectively. The Siamese OBB regression neural network, composed of the first feature extraction network, the first regression network, the second feature extraction network, and the second regression network, performs OBB regression on the image after different transformations.

[0019] The first transformation module performs random similarity transformations on I, including scaling, rotation, and translation, i.e., the first transformation T1, and outputs the first transformation training image. Alternatively, a congruent transformation can be used, in which case... Record the transformation parameters of T1 for use by the first inverse transformation module.

[0020] The second transformation module performs random similarity transformations on I, including scaling, rotation, and translation, i.e., the second transformation T2, and outputs the second transformation training image. Generally, T2 ≠ T1. Record the transformation parameters of T2 for use by the second inverse transformation module.

[0021] The first feature extraction network extracts The texture and semantic features of the image are used to generate N scales at L scales. f First Transformation Feature Map of Dimension Structures such as ResNet50 connected to FPN and ResNext101 connected to BiFPN can be used.

[0022] The second feature extraction network is a Siamese network of the first feature extraction network. The texture and semantic features of the image are used to generate N scales at L scales. f The second transformation feature map of dimension The second feature extraction network uses the exact same network structure and the exact same network parameters as the first feature extraction network.

[0023] The classification network module transforms the second feature map. Classify the features to obtain Target category prediction results at various scales in The value of the c-th dimension is (s) at the l-th scale l x, s l The confidence level for the existence of a target of class c at position y) is given by s. l is the stride of the feature map at scale l. A basic unit can be formed by concatenating 3×3 convolutional layers with Group-Norm layers and then with ReLU layers. This basic unit can be concatenated multiple times, and then with Sigmoid layers to form a classification network.

[0024] The first regression network comprises: a first shared subnetwork and three independent branch subnetworks, wherein: during weakly supervised training, the first shared subnetwork is based on the second transformed feature map. The first transform regression feature map is obtained, and then passed through three independent branch sub-networks to output... First transformation OBB position size prediction at various scales First transformation OBB orientation prediction First transformation OBB centering prediction Its prediction is (s) at the l-th scale l x, s l If a target exists at position y), then (s) l x, s l y) The degree of relative deviation from the center of the target.

[0025] The first shared subnetwork includes: several cascaded basic units, each of which consists of a 3×3 convolutional layer connected to a GroupNorm layer and a cascaded ReLU layer.

[0026] The three independent branch sub-networks are each implemented using a 1×1 convolutional layer.

[0027] The second regression network comprises a second shared subnetwork and two independent branch subnetworks, wherein: during weakly supervised training, the second shared subnetwork is trained according to the second transform feature map. The second transform regression feature map is obtained, and then passed through two independent branch sub-networks to output... Second transformation OBB location size prediction at various scales Second transformation OBB orientation prediction

[0028] The second shared subnetwork has the same structure and network parameters as the first shared subnetwork.

[0029] The two independent branch sub-networks are each implemented using 1×1 convolutional layers, preferably with the same structure and network parameters as the branch sub-networks of the first regression network.

[0030] The first OBB decoder pair and Combined with step size s l Decode to obtain First transform OBB prediction at various scales in: For is (s) at the l-th scale l x, s l If a target exists at position y), the predicted bounding box OBB of that target is calculated.

[0031] The second OBB decoder pair and Combined with step size s l Decode to obtain Second transform OBB prediction at various scales in: For is (s) at the l-th scale l x, s l If a target exists at position y), the predicted bounding box OBB of that target is calculated.

[0032] The first inverse transformation module for All Inverse transform and output the first OBB prediction This makes the original location The OBB prediction results from the image viewpoint are all inversely transformed back to the image viewpoint of I.

[0033] The second inverse transformation module is for All Inverse transform and output the second OBB prediction This makes the original location The OBB prediction results from the image viewpoint are all inversely transformed back to the image viewpoint of I.

[0034] The first allocator, according to a predetermined allocating strategy, will... Matching HBB at various scales and locations gt The first allocation result Match1 is obtained: (l, x, y) → j, where: -1 ≤ j < N gt The meaning is that if j = Match1(l, x, y) ≥ 0, then Prediction Category Label gt (j), Return to HBB gt (j); if j = -1, then Predicting categories with no target or background, at this time No regression to any objective. The established allocation strategy can be, for example, based on anchor boxes and HBBs as used by RetinaNet. gt The IoU value allocation, such as that used by FCOS based on the anchor point in HBB. gt The allocation of areas near the center, etc.

[0035] The classification loss calculation module calculates based on Match1. Category prediction loss cls (l, x, y), where The truth value of `Label` is when `Match1(l, x, y) ≥ 0`. gt (Match1(l, x, y)), where Match1(l, x, y) = -1 is the background class. For all {Loss cls The loss can be obtained by taking the mean of (l, x, y). cls Focal-Loss or Binary-Cross-Entropy-Loss can be used. cls It expresses the degree of accuracy of the current model in classifying the target.

[0036] The centrality loss calculation module calculates based on Match1. Centrality loss centernessFocal-Loss or Binary-Cross-Entropy-Loss can be used.

[0037] The external regression loss calculation module calculates based on Match1. The outer regression loss from the outer HBB to the corresponding HBB truth value is... cirreg Where Match1(l, x, y) ≥ 0 The truth value of the extrinsic HBB is HBB. gt (Match1(l, x, y)), for and HBB gt The Loss is obtained after calculating (Match1(l, x, y)). cirreg (l, x, y); when Match1(l, x, y) = -1 No need to participate in Loss cirreg Calculate. For all {Loss} cirreg The loss can be obtained by averaging the values ​​of (l, x, y)|Match1(l, x, y)≥0. cirreg Loss cirreg This indicates the degree of accuracy of the current model in OBB regression.

[0038] The Loss cirreg The specific calculation process for (l, x, y) is as follows: Figure 2 As shown, If OBB is composed of vertices ABCD, then its circumcircle HBB is composed of vertices A'B'C'D'. If the truth value of its corresponding HBB is HBB... gt (Match1(l, x, y)) is an HBB consisting of vertices EFGH, then the Loss cirreg (l, x, y) can be expressed as the loss function value commonly used in HBB bounding box regression, such as IoULoss, GIoULoss, or CIoULoss between A'B'C'D' and EFGH.

[0039] The second allocator, based on a predetermined allocation strategy and the result of Match1, allocates... The predicted target location is matched to The predicted positions of the same targets are used to obtain the second allocation result, Match. 12 The allocation strategy adopted in this scheme is as follows: First, the same allocation strategy as the first allocator is adopted to match... To HBB gt We obtain Match2: (l, x, y) → j, Match 12 For set Match 12A specific term (l1, x1, y1, l2, x2, y2) describes and All predicted HBB gt (j).

[0040] The transformation consistency loss calculation module is based on Match 12 Calculate and predict the same HBB gt (j) and Loss between consistency (l1, x1, y1, l2, x2, y2). For all {Loss consistency (l1,x1,y1,l2,x2,y2)|(l1,x1,y1,l2,x2,y2)∈Match 12 Calculate the mean to obtain the loss. consistency Loss consistency The difference in regression results between an OBB regression neural network composed of a first feature extraction network and a first regression network, and a corresponding Siamese OBB regression neural network composed of a second feature extraction network and a second regression network, is measured. This is because both essentially perform OBB regression on the same image I after different transformations; the OBB regression results, after inverse transformation back to the image view of I, are completely identical.

[0041] The Loss consistency The calculation method is as follows: When The corresponding prediction is the same HBB. gt (j) Then Loss consistency (l1, x1, y1, l2, x2, y2)=min(Loss consistency1 (l1, x1, y1, l2, x2, y2), Loss consistency2 (l1, x1, y1, l2, x2, y2)), where: Loss consistency1 (l1, x1, y1, l2, x2, y2)=ShapeLoss((0,0,w1,h1),(0,0,w2,h2))+|sin(θ1-θ2)|, Loss consistency2 (l1, x1, y1, l2, x2, y2)=ShapeLoss((0,0,w1,h1),(0,0,h2,w2))+|cos(θ1-θ2)|. Loss consistency (l1, x1, y1, l2, x2, y2) is divided into two parts. The first part uses ShapeLoss to align the values ​​through the center and the rotation angle. and The second part addresses the consistency constraints on dimensions, and the second part focuses on... and Consistency constraints are applied based on the angle. Since θ has a period of π, the w, h, and θ of OBB are... Because it is a coupling period, therefore Loss consistency (l1, x1, y1, l2, x2, y2) is designed to take the loss. consistency1 (l1, x1, y1, l2, x2, y2), Loss consistency2 The minimum value among (l1, x1, y1, l2, x2, y2) is used to solve the periodicity problem. In practice, ShapeLoss can be implemented using IoULoss, GIoULoss, or CIoULoss, which are commonly used in HBB bounding box regression.

[0042] The image training set includes: preprocessed training image I and manually labeled target category labels. gt :j→Z, where:Label gt (j)∈{0,1,...,N} c -1} represents the manually labeled target category number, N c The total number of target categories and manually annotated bounding boxes for the OBB target detection task. in For image I, these represent the four real numbers corresponding to the j-th HBB bounding box manually annotated by the annotator.

[0043] The weakly supervised training described above utilizes optimizers such as AdamW and SGD to simultaneously minimize the loss. cls Loss cirreg Loss consistency The neural network parameters in the first feature extraction network, the second feature extraction network, the classification network module, the first regression network, and the second regression network of the model are optimized.

[0044] Step S3: Simplify the redundant components in the model trained in step S2 that are only used for weakly supervised training to obtain the inference model of the OBB object detector. In the inference stage, the detection data of the same task is preprocessed and then input into the inference model of the OBB object detector to obtain the OBB object detection result.

[0045] The aforementioned simplification of redundant components refers to: retaining the first feature extraction network, classification network, first regression network, and first OBB decoder, deleting the remaining modules, and adding a post-processing module to calculate the target confidence at each location on the feature maps of each scale. in: satisfy

[0046] The inference mentioned refers to: simplifying the calculation of target confidence at each location on feature maps of various scales by using redundant components. Then, the bounding boxes (OBBs) at each location on the feature maps of each scale are filtered according to the target confidence level and a threshold is set. The OBBs with high target confidence are retained, and the predicted bounding boxes (OBBs) are subjected to rotation nonmaximum suppression to remove highly overlapping OBB predictions.

[0047] Technical effect

[0048] This invention employs a weakly supervised paradigm for training an OBB target detector using HBB annotations. Specifically, the external HBB of the OBB prediction results approximates the ground truth HBB annotation. The model's OBB prediction results from different similarity transformation images of the same image are inversely transformed to approximate each other from the perspective of the original image. Furthermore, the external regression loss calculation module enables the supervision of the model's OBB prediction results by HBB-annotated data. Attached Figure Description

[0049] Figure 1 This is a flowchart of the present invention;

[0050] Figure 2 This is a schematic diagram illustrating the image coordinate system and bounding box description method of the present invention;

[0051] Figure 3 For Loss cirreg Calculation process diagram;

[0052] Figure 4 This is a schematic diagram of the weakly supervised training model of the OBB target detector in the embodiment;

[0053] Figure 5 This is a schematic diagram of the inference model of the OBB target detector in the embodiment;

[0054] Figure 6 This is a schematic diagram illustrating the effect of an example.

[0055] Specific implementation methods

[0056] like Figure 1 As shown, this embodiment relates to a method for implementing a rotating target detector trained with weak supervision using horizontal bounding box annotations, including:

[0057] Step S1: In this embodiment, the open-source remote sensing OBB detection datasets DOTA-v1 and DIOR-R datasets, which have both HBB and OBB annotations, are used. In this embodiment, only the training set and the HBB annotations of the training set are used to train the OBB target detector. Then, the validation set and the OBB annotations of the validation set are used to verify the performance of the weakly supervised OBB target detector and perform preprocessing.

[0058] For the DOTA-v1 dataset, the preprocessing described above requires large image slicing, normalization of image RGB pixel values, and online image enhancement.

[0059] For the DIOR-R dataset, the preprocessing described above requires image RGB pixel value normalization and online image enhancement.

[0060] In the preprocessing described above, the large image slicing step of the DOTA-v1 dataset uses parameters S=824 and M=8024, resulting in small image sizes of 1024×1024, with a 200-pixel overlap between the small images.

[0061] In the preprocessing described above, the image RGB pixel value normalization step uses parameter μ. r =123.675, μ g =116.28, μ b =103.53 is the normalized mean, σ r =58.395, σ g =57.12, σ b =57.375 is the normalized standard deviation.

[0062] In the preprocessing described above, the online image enhancement step only uses random left and right flipping as the online enhancement method.

[0063] Step S2, construct as follows Figure 4 The weakly supervised training network of the OBB object detector shown uses an image training set containing preprocessed training images, manually labeled object categories, and manually labeled object bounding boxes for weakly supervised training to optimize the model's weights.

[0064] In the weakly supervised training network of the OBB target detector:

[0065] The first transformation module performs an identity transformation T1 and outputs a first transformation training graph.

[0066] The second transformation module performs a random similarity transformation T2 on I and outputs a second transformation training graph. After transformation, discard excess Pixels within the boundary range, and for those in Pixels within the boundary range but outside the I boundary range are filled with 0, and the transformation parameters of T2 are output to the second inverse transformation module.

[0067] The first feature extraction network includes a ResNet50 backbone network and a cascaded FPN as a multi-scale fusion network, which together form and output a 256-dimensional first transform feature map across five scales.

[0068] The second feature extraction network is a Siamese network of the first feature extraction network. Its structure and parameters are exactly the same as the first feature extraction network, and it outputs a 256-dimensional second transform feature map at five scales.

[0069] The classification network comprises four cascaded basic units and a sigmoid layer. Each basic unit consists of a 3×3 convolutional layer, a Group-Norm layer, and a ReLU layer cascaded together, thereby outputting target class predictions at five scales.

[0070] The first regression network comprises: a shared subnetwork and three independent 1×1 convolutional layers, wherein the shared subnetwork consists of four cascaded basic units, wherein: a basic unit consists of a 3×3 convolutional layer, a Group-Norm layer, and a ReLU layer cascaded together, wherein: After passing through the shared subnetwork, the first transform regression feature map is obtained. This first transform regression feature map is then passed through the first 1×1 convolutional layer, which outputs the first transform OBB position size predictions at five scales. The first transformed regression feature map is passed through a second 1×1 convolutional layer to output the first transformed OBB orientation prediction. The first transformed regression feature map, after passing through a third 1×1 convolutional layer, outputs predictions of the first transformed OBB centerness at five scales.

[0071] The second regression network is based on the Siamese network of the first regression network, with the third 1×1 convolutional layer used for centering prediction removed. Otherwise, its structure and parameters are identical to the first regression network, thus outputting the second transform OBB position size prediction at five scales. Second transformation OBB orientation prediction

[0072] The first and second OBB decoders employ the OBB decoder from the open-source Rotated-FCOS and output the first transform OBB predictions at five scales, respectively. Second Transformation OBB Prediction

[0073] The first inverse transformation module will pass Inverse transform to the image view of I and output the first OBB prediction Because T1 uses the identity transformation, therefore and They are completely equal.

[0074] The second inverse transformation module will pass Inversely transform to the image view of I and output the second OBB prediction. Specifically: For any bounding box OBB represented as (x, y, w, h, θ), set the image center coordinates to (512, 512), and perform a transformation T2 including scaling α2, translation t2, and rotation α2. If the three transformation components are given, then the OBB will undergo an inverse transformation T. -1 Received satisfy:

[0075] The first allocator uses the central region allocator in FCOS to obtain the first allocation result Match1.

[0076] The classification loss calculation module uses the Focal-Loss function in FCOS to obtain the classification loss. cls .

[0077] The centrality loss calculation module uses the centrality loss function Binary-Cross-Entropy-Loss in FCOS to obtain the centrality loss. centerness .

[0078] The external regression loss calculation module calculates the loss. cirreg When (l, x, y), calculate The external HBB and its corresponding HBB truth value HBB gt IoULoss, GIoULoss, or CIoULoss are used for the intervals (Match1(l, x, y)), with IoULoss being preferred. The final result is the lateral regression loss. cirreg .

[0079] The transformation consistency loss calculation module, based on The corresponding prediction is the same HBB. gt (j) Calculate the transformation consistency loss. consistency (l1, x1, y1, l2, x2, y2)=min(Loss consistency1 (l1, x1, y1, l2, x2, y2), Loss consistency2 (l1, x1, y1, l2, x2, y2)), where Loss consistency1 (l1, x1, y1, l2, x2, y2)=IoULoss((0,0,w1,h1),(0,0,w2,h2))+|sin(θ1-θ2)|, Loss consistency2(l1, x1, y1, l2, x2, y2)=IoULoss((0,0,w1,h1),(0,0,h2,w2))+|cos(θ1-θ2)|.

[0080] The weakly supervised training described above uses the AdamW optimizer to optimize the classification loss. cls Centrality loss centerness extrinsic regression loss (LoSs) cirreg Transformation consistency loss consistency Minimization optimization was performed, during which gradient calculations and backpropagation were conducted on the neural network parameters of the first feature extraction network, the second feature extraction network, the classification network, the first regression network, and the second regression network. A learning rate of lr = 0.0001, momentum parameters β1 = 0.9, β2 = 0.999, and regularization parameter wd = 0.05 were used. The training process lasted for 36 epochs, with the learning rate reduced by a factor of 10 twice, at the beginning of epochs 24 and 33.

[0081] Step S3: Simplify the trained weakly supervised network model: retain the first feature extraction network, classification network, first regression network, and first OBB decoder; delete the remaining modules; add a post-processing module; and finally obtain the model as shown below. Figure 5 The inference model of the OBB object detector is shown. During the inference phase, the data to be detected for the same task is preprocessed and then input into the inference model of the OBB object detector for inference, obtaining the OBB object detection results.

[0082] The post-processing module calculates the target confidence at each location on the feature map at each scale. Then, the bounding boxes (OBBs) at each location on the feature maps of each scale are filtered according to the target confidence level and a threshold is set. The OBBs with high target confidence are retained, and the predicted bounding boxes (OBBs) are subjected to Rotated Non-Maximum Suppression (Rotated-NMS) to remove highly overlapping OBB predictions.

[0083] Using the OBB object detector of this embodiment, after training on the DOTA-v1 training set using only HBB annotations, its performance metrics on the DOTA-v1 test set are evaluated as shown in Table 1. The highest mAP50 reached 70.4%; while the control OBB object detector trained using OBB annotations achieved a maximum mAP50 of 72.2%, and the performance of the two is very similar. Figure 6 The image shows the effect of the OBB target detector in this embodiment on the DOTA-v1 test set.

[0084] Table 1 Comparison of mAP50 performance metrics of the OBB target detector in this embodiment on the DOTA-v1 test set.

[0085]

[0086] Using the OBB object detector of this embodiment, after training on the DIOR-R training set using only HBB annotations, its performance metrics on the DIOR-R test set are evaluated as shown in Table 2. The highest mAP50 reached 58.1%; while the highest mAP50 of the control OBB object detector trained using OBB annotations reached 55.7%, indicating that the performance of this embodiment is better.

[0087] Table 2 Comparison of mAP50 performance metrics of WS-Rotated-FCOS on the DIOR-R test set

[0088]

[0089]

[0090] In summary, compared with existing technologies, this method can directly train an OBB object detector using a dataset with only HBB manual annotations to obtain an OBB object detector through weak supervision. It takes advantage of the fact that HBB annotation is more common (more annotation tools to choose from), simpler, faster, and 20% cheaper than OBB annotation based on market prices. It eliminates the need for a relatively complex, time-consuming, and expensive OBB manual annotation process on the task dataset, and only requires a relatively simple, fast, and inexpensive HBB manual annotation process on the task dataset.

[0091] While reducing annotation costs, the OBB object detector obtained by our method achieves detection performance that is essentially equal to or surpasses that of a control OBB object detector trained on a dataset with manually annotated OBB data. Therefore, our method has advantages in both performance and cost.

[0092] The specific embodiments described herein may be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the specific embodiments described herein. All implementation schemes within the scope thereof are subject to the constraints of the present invention.

Claims

1. A method for implementing a rotating target detector using weakly supervised training with horizontal bounding box annotation, characterized in that, include: Step S1: For a specific directed bounding box (OBB) target detection task, manually collect the image data of the task, and have the annotator annotate the target category and horizontal bounding box (HBB) in the task image data. Construct an image training set with only target category and HBB annotation based on the image and corresponding annotation, and preprocess the image training set with HBB annotation. Step S2: Construct a weakly supervised training network model for the OBB object detector. Use an image training set containing preprocessed training images, manually labeled object categories, and manually labeled object bounding boxes for weakly supervised training, and optimize the model weights. Step S3: Simplify the redundant components in the weakly supervised training network model after step S2 that are only used for weakly supervised training, and obtain the inference model of the OBB object detector. In the inference stage, the detection data of the same task is preprocessed and then input into the inference model of the OBB object detector to obtain the OBB object detection result. The weakly supervised training network of the OBB target detector includes: a first transformation module, a second transformation module, a first feature extraction network, a second feature extraction network, a classification network, a first regression network, a second regression network, a first OBB decoder, a second OBB decoder, a first inverse transformation module, a second inverse transformation module, a first allocator, a classification loss calculation module, a centrality loss calculation module, a circumscribed regression loss calculation module, a second allocator, and a transformation consistency loss calculation module. The first transformation module, the second transformation module, the first inverse transformation module, and the second inverse transformation module work collaboratively to transform and inversely transform the image and the OBB detection result, respectively. The Siamese OBB regression neural network, composed of the first feature extraction network, the first regression network, the second feature extraction network, and the second regression network, performs OBB regression on the image after different transformations. The first regression network comprises: a first shared subnetwork and three independent branch subnetworks, wherein: during weakly supervised training, the first shared subnetwork is based on the second transformed feature map. The first transform regression feature map is obtained, and then passed through three independent branch sub-networks to output... First transformation OBB position size prediction at various scales First transformation OBB orientation prediction First transformation OBB centrality prediction Its prediction is In the Scale If a target exists at the location, then The degree of relative deviation from the center of the target; The second regression network comprises a second shared subnetwork and two independent branch subnetworks, wherein: during weakly supervised training, the second shared subnetwork is trained according to the second transform feature map. The second transform regression feature map is obtained, and then passed through two independent branch sub-networks to output... Second transformation OBB location size prediction at various scales Second transformation OBB orientation prediction .

2. The method for implementing a rotating target detector according to claim 1, characterized in that, The preprocessing described varies depending on the OBB target detection task: For remote sensing image OBB target detection tasks, due to the large image size, the preprocessing includes: large image slicing, image RGB pixel value normalization, and online image enhancement; For natural scene text OBB target detection tasks and robot target object grasping tasks, the image size is generally small, so the preprocessing includes: image RGB pixel value normalization and online image enhancement.

3. The method for implementing a rotating target detector according to claim 2, characterized in that, The aforementioned large image slices specifically include: 1) Image sliding window slicing: Let the input image be... The length and width pixel dimensions are Select the sliding window step size and the size of the cutting window ,but You can access the direction by sliding the window. window position, You can access the direction by sliding the window. The image is displayed in a set of smaller images after being sliced ​​at each window location. The small image The size is ; 2) HBB annotation sliding window segmentation: for small image sets For each sub-image, select HBBs whose midpoint coordinates lie within the sub-image size range as the cutout labels for that sub-image. That is, when any HBB: midpoint coordinates lie in Within the corresponding window range, the corresponding Filtered as The cut-out label.

4. The method for implementing a rotating target detector according to claim 2, characterized in that, The aforementioned image RGB pixel value normalization specifically refers to: when the input image The image after pixel value normalization ,in: For the normalized mean, To normalize the standard deviation, fine-tuning can be performed on the task data.

5. The method for implementing a rotating target detector according to claim 2, characterized in that, The aforementioned online image enhancement specifically refers to: using methods including random flipping, random color transformation enhancement, and random noise processing to enhance images online, thereby increasing the diversity of training data.

6. The method for implementing a rotating target detector according to claim 2, characterized in that, The aforementioned simplification of redundant components refers to: retaining the first feature extraction network, classification network, first regression network, and first OBB decoder, deleting the remaining modules, and adding a post-processing module to calculate the target confidence at each location on the feature maps of each scale. ,in: satisfy .

7. The method for implementing a rotating target detector according to claim 6, characterized in that, The aforementioned inference refers to: simplifying the calculation of target confidence at each location on feature maps of various scales by using redundant components. Then, the bounding boxes (OBBs) at each location on the feature maps of each scale are filtered according to the target confidence level and a threshold is set. The OBBs with high target confidence are retained, and the predicted bounding boxes (OBBs) are subjected to rotation nonmaximum suppression to remove highly overlapping OBB predictions.

Images