An anchor weight distribution optimization method, device and medium
By optimizing the anchor weight allocation method and adjusting the loss weight according to the number of positive samples assigned to each ground box, the problem of unbalanced anchor weight allocation is solved, and the detection effect of extreme or small objects is improved.
Patent Information
- Application Number
- CN202210651611.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-10
- Publication Date
- 2026-07-28
- Estimated Expiration
- 2042-06-10
AI Technical Summary
In existing object detection technologies, the anchor weight allocation method ignores the balance of the internal distribution of the object, resulting in poor detection performance for objects with more extreme proportions or smaller sizes.
By calculating the IoU between each anchor and the ground truth box, positive, ignored, and negative samples are assigned, and the weights of the classification and regression losses are adjusted according to the number of positive samples assigned to each ground truth box, thus achieving optimized allocation of anchor weights.
It significantly improves the detection performance of objects with poor position, size, and proportion, thereby enhancing the detection accuracy of the model.
Smart Images
Figure CN115170864B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of image processing and deep learning technology, and in particular to an anchor weight allocation optimization method, device and medium. Background Technology
[0002] In deep learning, object detection is a very important subfield. Object detection refers to detecting an indefinite number of objects of a specified category in an image to determine the location, size, and category of each object. After early evolution, the main principle of current object detection is to use the translation invariance of convolution to classify and regress regions in the image. Classification determines the presence or absence of objects and their categories, while regression determines the size and location of the objects. Typically, the size and location of an object are represented by its rectangular bounding box, called the box.
[0003] Current object detection can be divided into two main categories: 1-stage and 2-stage (or more multi-stage). 1-stage refers to classifying and regressing regions on an image in one step to obtain the final detection result. 2-stage refers to first obtaining bounding boxes of suspected objects in the image through some means (usually similar to 1-stage detection), then cropping the corresponding regions from the feature map of the image or deep convolutional neural network based on the bounding boxes, and then classifying and regressing these patches of regions again to obtain the final detection result.
[0004] In the first stage of both 1-stage and 2-stage object detection, the common approach is to divide the image into many locations, then use a deep convolutional neural network to extract features and obtain a feature map. Next, convolution is used to predict the information of each cell in the feature map to obtain the object's presence, category, and bounding box for each cell. Finally, non-maximum suppression (NMS) is used to obtain the final object's category and bounding box. In this process, based on the pre-defined image location division method, two main approaches have emerged: anchor-based detection and anchor-free detection.
[0005] Anchor-based detection maps the center point of each cell in the feature map to its position in the original image, and then sets K (e.g., 4, 9, 12, etc.) rectangular boxes of different sizes and aspect ratios, centered at the corresponding position in the image, called anchors (or prior boxes in SSDs), on this cell. Ultimately, the image is filled with anchors of different positions, sizes, and aspect ratios; the number of anchors depends on the input size of the image. Convolutional neural networks predict K*M numbers for each cell in the feature map, where M represents information such as existence, class, and relative box position. This yields the existence, class, and relative box position of the K anchors corresponding to that cell. Then, based on the anchorbox information of each anchor, the absolute position of the box is obtained.
[0006] Anchor-free detection also maps the center point of each cell in the feature map to the position in the original image based on its position in the feature map, but it no longer sets anchors with fixed size and aspect ratio. Instead, it directly predicts the existence, category, and box of the object based only on the points corresponding to these cells.
[0007] Whether anchor-based or anchor-free, the detection problem will ultimately be transformed into a classification problem plus a regression problem. Therefore, it will also face the problems that classification models and regression models may encounter. The loss functions and common methods used are also similar.
[0008] In anchor-based detection, the problem is transformed into: given K anchors and T ground truth boxes (GTBoxes) with their respective coordinates, and the class of each GTBox, classify and regress each anchor. Typically, during model training, each anchor calculates the Intersection over Union (IoU) with each ground truth box (IoU of the two boxes) and is then categorized into three classes based on IoU thresholds G1 and G2: anchors with the largest IoU_k among all ground truth boxes (greater than G1) are positive samples; those between G2 and G1 are ignored; and those less than G2 are negative samples. For positive anchors, they are assigned the class of the ground truth box with the largest IoU, and their regression target is calculated based on the position of that ground truth box relative to the anchor. For ignored anchors, all classification and regression losses are ignored. For negative anchors, they are assigned the background class, and regression losses are ignored. Ultimately, the problem becomes finding the effective classification loss and effective regression loss for all anchors.
[0009] In practice, classification and regression losses are typically normalized. A common approach is to let K1 be the number of positive anchors, K2 be the number of ignored anchors, and K3 be the number of negative anchors. Each anchor has a classification weight of cls_w_k (1 for positive or negative samples, 0 otherwise) and a regression weight (1 for positive samples) of loc_w_k. Then, sum(cls_w_k) = K1 + K3, and sum(loc_w_k) = K1. Therefore, the classification loss is set to sum(cls_loss_k * cls_w_k) / K1, and the regression loss to sum(loc_loss_k * loc_w_k) / K1. Some improved approaches may normalize the loss based on the balance of positive and negative samples. For example, focalloss is specifically optimized to address the problem of overall imbalance in detecting positive and negative samples.
[0010] However, these approaches essentially normalize the classification and regression of each anchor based on the overall positive and negative sample statistics, ignoring the true balance of internal object distribution. This is because the number of positive anchors assigned to each ground truth box varies greatly depending on the anchor settings (size, location, and overall dimensions) and the size, location, and overall dimensions of the ground truth box. Generally, ground truth boxes whose location and size closely match the general anchor distribution are assigned more anchors; more anchors mean a larger proportion of the training loss for that ground truth box, resulting in better training performance. A typical example is that, with a commonly used IoU threshold of 0.5, extremely small or overly large ground truth boxes may only receive one anchor because they are required to be assigned at least once, while high-quality ground truth boxes can be assigned 30 anchors. Furthermore, to save computation and post-processing time, the model may inherently assign fewer anchors per cell for small objects, further reducing the number of anchors assigned to small objects. Ultimately, the model doesn't perform as well when training objects with more extreme or smaller proportions. Summary of the Invention
[0011] To address the aforementioned issues, this invention proposes an anchor weight allocation optimization method, device, and medium, which can significantly improve the detection performance of objects with extreme position, size, and width-to-height ratios.
[0012] The technical solution adopted in this invention is as follows:
[0013] An anchor weight allocation optimization method includes the following steps:
[0014] S1: Collect the corresponding image based on the target to be detected;
[0015] S2: Detect and label the collected images;
[0016] S3: Design the first stage of an anchor-based one-stage detection model or a two-stage detection model, set the anchor (candidate box), and input the image and detection annotations into the detection model;
[0017] S4: For each batch of input image set and detection annotation set, calculate the IOU between each anchor and all ground truth boxes of each image, that is, the ratio of the intersection area of the candidate box anchor and the union area of the ground truth box ground truth box.
[0018] S5: For each ground truth box in each image, take the anchor with the largest IOU and set it as the positive sample anchor that matches the ground truth box;
[0019] S6: For each unmatched anchor in each image, extract the ground truth box (GTBox) with the largest IoU. If the largest IoU is greater than a set IoU threshold G1, the anchor is assigned to the GTBox as a positive sample. If the largest IoU is between the IoU thresholds G2 and G1, the anchor is set as an ignored anchor; otherwise, the anchor is set as a negative sample anchor. Where 0 <G2<G1<1;
[0020] S7: The number of positive sample anchors in this batch of images is K1, the number of ignored sample anchors is K2, the number of negative sample anchors is K3, and the total number of ground boxes is T1.
[0021] S8: Count the number of positive sample anchors matched for each gtbox in each image, denoted as p_t;
[0022] S9: For each anchor of each image, set the corresponding classification loss weight cls_w_k and regression loss weight loc_w_k according to the sample type;
[0023] S10: For each set cls_w_k and loc_w_k, normalize or adjust the weights again according to the traditional additional loss weight normalization method set by the current detection model.
[0024] S11: Based on the set target values and weights for classification and regression, calculate the total classification loss, i.e., cls_loss = sum(cls_loss_k * cls_w_k), and the total regression loss, i.e., loc_loss = sum(loc_loss_k * loc_w_k), where cls_loss_k and loc_loss_k are calculated from the classification loss and regression loss set by the corresponding detection model; then calculate the final total loss, i.e., loss = cls_loss + a * loc_loss, where a is a set coefficient;
[0025] S12: After the model training is completed, the final detection model is obtained. The final detection model is then used to detect the target.
[0026] Furthermore, in step S9, if the anchor is a negative sample, the category ID required for its classification loss is set to the background class, the weight of the classification loss is cls_w_k = 1 / max(1,K1), and the weight of its regression loss is set to loc_w_k = 0.
[0027] Furthermore, in step S9, if the anchor is an ignored sample, set its classification loss weight cls_w_k = 0 and its regression loss weight loc_w_k = 0.
[0028] Further, in step S9, if the anchor is a positive sample, then find its matched gtboxt, and the number of positive sample anchors assigned to the gtbox is p_t; set the category ID required for its classification loss to be the category ID corresponding to the gtbox, and the weight of the classification loss cls_w_k=(K1 / (T1*p_t))*(1 / K1)=1 / (T1*p_t); set the target value of its regression loss to be calculated from the anchor and the gtbox, and the weight of the regression loss loc_w_k=(K1 / (T1*p_t))*(1 / K1)=1 / (T1*p_t).
[0029] Further, in step S9, if the anchor is a positive sample, let T2 = sum(log(p_t+1)), which means that for all gtboxes in this batch of images, log(p_t+1) is calculated and summed for each corresponding p_t. The weight of its classification loss is set to cls_w_k = (T2 / (T1*log(p_t+1)))*(1 / K1), and the weight of its regression loss is set to loc_w_k = (T2 / (T1*log(p_t+1)))*(1 / K1).
[0030] Furthermore, in step S10, for each set cls_w_k and loc_w_k, the normalization method performed again according to the traditional additional loss weight normalization method set by the current detection model includes positive and negative sample ratio normalization.
[0031] Furthermore, in step S11, cls_loss_k and loc_loss_k are calculated from the classification loss and regression loss set by the corresponding detection model, including crossentropy loss, focal loss, smoothL1 loss and IOU loss.
[0032] Furthermore, in step S5, if an anchor selection conflict occurs between gtboxes, the selection is carried over to the next level based on the maximum IOU value.
[0033] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the anchor weight allocation optimization method of claim 1.
[0034] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the anchor weight allocation optimization method.
[0035] The beneficial effects of this invention are as follows:
[0036] The anchor weight allocation optimization method of this invention differs from the traditional weight allocation method that only focuses on the overall anchor weight balance in detection. Instead, it focuses on the anchor balance between objects. This method can be added to any traditional detection setup and can effectively improve the detection effect of objects with poor position, size, and proportion. Attached Figure Description
[0037] Figure 1 This is a flowchart of the anchor weight allocation optimization method according to an embodiment of the present invention. Detailed Implementation
[0038] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments are now described. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention; that is, the described embodiments are only a part of the embodiments of the invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0039] Example 1
[0040] like Figure 1 As shown, this embodiment provides an anchor weight allocation optimization method, including the following steps:
[0041] S1: Collect the corresponding image based on the target to be detected;
[0042] S2: Detect and label the collected images;
[0043] S3: Design the first stage of an anchor-based one-stage detection model or a two-stage detection model, set the anchor (candidate box), and input the image and detection annotations into the detection model;
[0044] S4: For each batch of input image set and detection annotation set, calculate the IOU between each anchor and all ground truth boxes of each image, that is, the ratio of the intersection area of the candidate box anchor and the union area of the ground truth box ground truth box.
[0045] S5: For each ground truth box in each image, take the anchor with the largest IOU and set it as the positive sample anchor to be matched for that ground truth box; preferably, if there is a conflict in anchor selection between ground truth boxes, then continue to the next ground truth box according to the largest IOU value.
[0046] S6: For each unmatched anchor in each image, extract the ground truth box (GTBox) with the largest IoU. If the largest IoU is greater than a set IoU threshold G1, the anchor is assigned to the GTBox as a positive sample. If the largest IoU is between the IoU thresholds G2 and G1, the anchor is set as an ignored anchor; otherwise, the anchor is set as a negative sample anchor. Where 0 <G2<G1<1;
[0047] S7: The number of positive sample anchors in this batch of images is K1, the number of ignored sample anchors is K2, the number of negative sample anchors is K3, and the total number of ground boxes is T1.
[0048] S8: Count the number of positive sample anchors matched for each gtbox in each image, denoted as p_t;
[0049] S9: For each anchor in each image, set the corresponding weights cls_w_k for classification loss and loc_w_k for regression loss based on the sample type. Specifically:
[0050] If the anchor is a negative sample, set the category ID required for its classification loss to the background class (usually 0 under softmax, and 0 for each category ID under sigmoid), set the weight of the classification loss cls_w_k = 1 / max(1,K1), and set the weight of its regression loss loc_w_k = 0.
[0051] If the anchor is an ignored sample, set its classification loss weight cls_w_k = 0 and its regression loss weight loc_w_k = 0;
[0052] If the anchor is a positive sample, find its matched ground truth box t, and the number of positive sample anchors assigned to the ground truth box is p_t; set the category ID required for its classification loss to be the category ID corresponding to the ground truth box, and the weight of the classification loss cls_w_k=(K1 / (T1*p_t))*(1 / K1)=1 / (T1*p_t); set the target value of its regression loss to be calculated from the anchor and the ground truth box, and the weight of the regression loss loc_w_k=(K1 / (T1*p_t))*(1 / K1)=1 / (T1*p_t);
[0053] S10: For each set cls_w_k and loc_w_k, normalize or adjust the weights again according to the traditional additional loss weight normalization method set by the current detection model.
[0054] S11: Based on the set target values and weights for classification and regression, calculate the total classification loss, i.e., cls_loss = sum(cls_loss_k * cls_w_k), and the total regression loss, i.e., loc_loss = sum(loc_loss_k * loc_w_k), where cls_loss_k and loc_loss_k are calculated from the classification loss and regression loss set by the corresponding detection model, such as crossentropy loss, focal loss, smoothL1 loss, and IOU loss; then calculate the final total loss, i.e., loss = cls_loss + a * loc_loss, where a is a set coefficient;
[0055] S12: After the model training is completed, the final detection model is obtained. The final detection model is then used to detect the target.
[0056] Preferably, in step S9, if the anchor is a positive sample, another less extreme formula can be used: Let T2 = sum(log(p_t+1)), which means that for all gtboxes in the batch of images, log(p_t+1) is calculated and summed for each corresponding p_t. The weight of its classification loss is set to cls_w_k = (T2 / (T1*log(p_t+1)))*(1 / K1), and the weight of its regression loss is set to loc_w_k = (T2 / (T1*log(p_t+1)))*(1 / K1).
[0057] The anchor weight allocation optimization method in this embodiment performs additional normalization on the classification loss and regression loss based on the number of anchors assigned to each object during model training, which can effectively improve the detection effect of objects with poor position, size, and proportion.
[0058] Example 2
[0059] This embodiment is based on embodiment 1:
[0060] This embodiment provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the anchor weight allocation optimization method of Embodiment 1. The computer program can be in the form of source code, object code, executable file, or some intermediate form.
[0061] Example 3
[0062] This embodiment is based on embodiment 1:
[0063] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the anchor weight allocation optimization method of Embodiment 1. The computer program can be in the form of source code, object code, executable file, or some intermediate form. The storage medium includes any entity or device capable of carrying computer program code, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content contained in the storage medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the storage medium does not include electrical carrier signals and telecommunication signals.
[0064] It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
Claims
1. An anchor weight allocation optimization method, characterized in that, Includes the following steps: S1: Collect the corresponding image based on the target to be detected; S2: Detect and label the collected images; S3: Design the first stage of an anchor-based one-stage detection model or a two-stage detection model, set the anchor (candidate box), and input the image and detection annotations into the detection model; S4: For each batch of input image set and detection annotation set, calculate the IOU between each anchor and all ground truth boxes of each image, that is, the ratio of the intersection area of the candidate box anchor and the union area of the ground truth box ground truth box. S5: For each ground truth box in each image, take the anchor with the largest IOU and set it as the positive sample anchor to be matched for that ground truth box; if there is a conflict in anchor selection between ground truth boxes, continue to the next ground truth box according to the largest IOU value; S6: For each unmatched anchor in each image, extract the ground truth box (GTBox) with the largest IoU. If the largest IoU is greater than a set IoU threshold G1, the anchor is assigned to the GTBox as a positive sample. If the largest IoU is between the IoU thresholds G2 and G1, the anchor is set as an ignored anchor; otherwise, the anchor is set as a negative sample anchor. Where 0 <G2<G1<1; S7: The number of positive sample anchors in this batch of images is K1, the number of ignored sample anchors is K2, the number of negative sample anchors is K3, and the total number of ground boxes is T1. S8: Count the number of positive sample anchors matched for each gtbox in each image, denoted as p_t; S9: For each anchor of each image, set the corresponding classification loss weight cls_w_k and regression loss weight loc_w_k according to the sample type; S10: For each set cls_w_k and loc_w_k, normalize or adjust the weights according to the traditional additional loss weight normalization method set by the current detection model. S11: Based on the set classification and regression target values and weights, calculate the total classification loss, i.e., cls_loss = sum(cls_loss_k). cls_w_k), and the total regression loss, i.e., loc_loss = sum(loc_loss_k). loc_w_k), where cls_loss_k and loc_loss_k are calculated from the classification loss and regression loss set by the corresponding detection model; then the final total loss is calculated, i.e., loss = cls_loss + a loc_loss, where a is a set coefficient; S12: After the model training is completed, the final detection model is obtained. The final detection model is then used to detect the target to be detected. In step S9, if the anchor is a positive sample, find its matched ground truth box t, and the number of positive sample anchors assigned to this ground truth box is p_t; set the category ID required for its classification loss to be the category ID corresponding to the ground truth box, and the weight of the classification loss cls_w_k=(K1 / (T1)) p_t)) (1 / K1)=1 / (T1 p_t); The target value of the regression loss is calculated from the anchor and the ground truth box, and the weight of the regression loss is loc_w_k=(K1 / (T1)). p_t)) (1 / K1) = 1 / (T1) p_t); or In step S9, if the anchor is a positive sample, let T2 = sum(log(p_t+1)), which means calculating log(p_t+1) for each p_t corresponding to all ground truth boxes in this batch of images and summing them. The weight of its classification loss is set as cls_w_k = (T2 / (T1)). log(p_t+1))) (1 / K1), set its regression loss weights loc_w_k=(T2 / (T1)). log(p_t+1))) (1 / K1).
2. The anchor weight allocation optimization method according to claim 1, characterized in that, In step S9, if the anchor is a negative sample, set the category ID required for its classification loss to the background class, set the weight of the classification loss to cls_w_k=1 / max(1,K1), and set the weight of its regression loss to loc_w_k=0.
3. The anchor weight allocation optimization method according to claim 1, characterized in that, In step S9, if the anchor is an ignored sample, set its classification loss weight cls_w_k=0 and its regression loss weight loc_w_k=0.
4. The anchor weight allocation optimization method according to claim 1, characterized in that, In step S10, for each set cls_w_k and loc_w_k, the normalization method performed again according to the traditional additional loss weight normalization method set by the current detection model includes positive and negative sample ratio normalization.
5. The anchor weight allocation optimization method according to claim 1, characterized in that, In step S11, cls_loss_k and loc_loss_k are calculated from the classification loss and regression loss set by the corresponding detection model, including crossentropy loss, focal loss, smoothL1 loss and IOU loss.
6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the anchor weight allocation optimization method according to any one of claims 1-5.
7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the anchor weight allocation optimization method according to any one of claims 1-5.