Object Detection Method Based on Cluster Box Fusion
Through the cluster box fusion method, a new prediction box is generated using the aggregation characteristics and classification scores of the prediction box, which solves the problems of missed detection and false detection in the existing algorithm, and improves the recall and accuracy of target detection.
Patent Information
- Application Number
- CN202210674635.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-15
- Publication Date
- 2025-07-29
- Estimated Expiration
- 2042-06-15
AI Technical Summary
Existing object detection algorithms rely on the classification score of the prediction box in the post-processing stage, resulting in missed detection and misdetection problems, especially in the case of fewer targets.
Through a cluster box fusion method, the clustering characteristics of the prediction box are used, combined with classification scores and quantity, cluster division, clustering and fusion are performed to generate a new prediction box.
Improve the recall and accuracy of the model, reduce missed and missed detection, and the generated prediction box boundaries are more accurate.
Smart Images
Figure CN115170913B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision, and particularly to an object detection method based on cluster box fusion. Background Art
[0002] Currently, there are already a large number of mature algorithms in the field of object detection such as images and videos. Most of the algorithms based on convolutional neural networks will generate a large number of prediction boxes at the end, especially some anchor box-based algorithms, such as Faster R-CNN, YOLO, SSD, etc. In order to filter redundant prediction boxes, most of these algorithms use the non-maximum suppression algorithm in the post-processing stage. However, the non-maximum suppression algorithm only depends on the classification scores of the prediction boxes. There may be many boxes with higher boundary accuracy but lower classification scores. Moreover, if the objects are very close to each other, it will cause missed detection. At the same time, in the case of fewer objects, too many prediction boxes will be retained, resulting in an increase in false detection. The final effect of prediction box filtering depends on the detection effect of the previous model. However, the non-maximum algorithm cannot well exert the detection effect of the model. Therefore, how to better exert the detection effect of the model through filtering and correction of prediction boxes in the post-processing stage is a major problem at present. Summary of the Invention
[0003] The purpose of the present invention is to overcome the deficiencies of the prior art and provide an object detection method based on cluster box fusion, which can effectively improve the recall rate and precision rate of the model.
[0004] To achieve the above purpose, the technical solution of the present invention is: an object detection method based on cluster box fusion, including the following steps:
[0005] Step S1, obtaining a data set: different data sets are established according to different task requirements. The data set consists of pictures with or without objects and object annotation information;
[0006] Step S2, preprocessing the data set;
[0007] Step S3, using a feature extraction network to extract features from the data set to obtain a feature map, and then generating a series of anchor boxes on the feature map. These anchor boxes obtain prediction boxes after passing through a classification and boundary regression network;
[0008] Step S4, classifying the prediction box categories, and then dividing the prediction boxes into clusters in each prediction box category;
[0009] Step S5, filtering out illegal clusters. The legality of the cluster is described by the aggregation degree and credibility of the prediction boxes in the cluster;
[0010] Step S6, using a clustering algorithm to cluster the prediction boxes in the cluster;
[0011] Step S7. Perform fusion correction on the clustering centers after clustering;
[0012] Step S8. For the clustering centers obtained after clustering, fuse those with relatively high similarity, and delete the clustering centers with relatively low credibility among those with relatively low similarity. The finally obtained clustering centers are used as the final prediction boxes.
[0013] Compared with the prior art, the present invention has the following beneficial effects:
[0014] 1) In practice, the prediction boxes of the model have an aggregation characteristic. These prediction boxes that are aggregated together have similar boundary information and classification scores, and there is a greater probability that the area where a large number of prediction boxes are aggregated is the area where the target is located. The present invention utilizes this characteristic of the prediction boxes for post-processing of the prediction boxes. Compared with the non-maximum suppression algorithm that only relies on the classification score, the present invention simultaneously considers the classification score and the number of aggregations, and can filter the prediction boxes more effectively.
[0015] 2) Since the aggregation characteristic of these prediction boxes is considered, it is possible to better retain some prediction boxes with more precise boundaries but lower classification scores, reduce the situation of missed detections, and at the same time filter more prediction boxes according to the number of aggregated prediction boxes and the classification score situation, reducing the situation of false detections.
[0016] 3) The final prediction boxes are not the original prediction boxes, but new prediction boxes generated by integrating the information of these clustered prediction boxes, with more precise boundaries, and can improve the accuracy under strict metrics. Description of the Drawings
[0017] Figure 1 It is a flowchart of the object detection method based on cluster box fusion in the present invention.
[0018] Figure 2 It is an example of the original parathyroid image in the embodiment of the present invention.
[0019] Figure 3 It is an example of the effect diagram of the parathyroid gland in the embodiment of the present invention. Detailed Embodiments
[0020] The technical solution of the present invention will be specifically described below with reference to the drawings.
[0021] As Figure 1 shown, the present invention is an object detection method based on cluster box fusion, which is implemented according to the following steps:
[0022] Step S1. Acquisition of the data set. Different data sets need to be established according to different tasks. The data set consists of pictures with (or without) targets and target annotation information, and the format needs to meet the input requirements of the model.
[0023] Step S2: Perform some preprocessing operations on the dataset, mainly including data augmentation, normalization, and standardization.
[0024] Step S3: After the data is input into the model, feature extraction is first performed to obtain a feature map, and then a series of anchor boxes are generated on the feature map. These anchor boxes are passed through the classification and bounding box regression networks to obtain prediction boxes.
[0025] Step S4: Divide the prediction boxes into several parts according to the different categories of the prediction boxes. The same category belongs to the same part, and different categories belong to different parts. Then, cluster the prediction boxes in each part. The purpose of this step is to divide the prediction boxes into several non-overlapping clusters according to the prediction boxes with larger classification scores.
[0026] Step S5: Filter out some illegal clusters. The legality of the clusters is mainly described by the aggregation degree and credibility of the prediction boxes in the clusters, mainly reflected in whether the largest classification score in the cluster is too low and whether the number of prediction boxes in the cluster is too small.
[0027] Step S6: Use clustering algorithms such as k-means to cluster the prediction boxes in the clusters.
[0028] Step S7: Perform fusion and correction on the clustering centers after clustering.
[0029] Step S8: For the clustering centers obtained after clustering, those with high similarity are fused, and those with low similarity delete the clustering centers with low confidence. The finally obtained clustering centers are used as the final prediction boxes.
[0030] In step S4 of the present invention, the following steps are further included:
[0031] Step S41: Put all the prediction boxes of the same category into set S. If the number of prediction boxes in set S is greater than zero, then execute step S42; otherwise, step S4 ends.
[0032] Step S42: Take the prediction box with the largest classification score in set S and denote it as box max . Calculate the intersection over union (IoU) between box max in S and other remaining prediction boxes.
[0033] Step S43: Group the prediction boxes with IoU greater than the threshold, including box max , into a cluster bunch i , and save the cluster bunch i . The threshold here is mainly used to define the distance between different clusters. The smaller this value is, the fewer the final number of clusters and the smaller the intersection between the clusters.
[0034] Step S44: Delete the bunch in set S i , if the number of prediction boxes in set S is greater than zero, execute step S42, otherwise step S4 ends.
[0035] In step S5 of the present invention, the following steps are further included:
[0036] Step S51. In step S4, a set of bunches [bunch0, bunch1,..., bunch n-1 will be obtained. Each bunch contains multiple prediction boxes box = [x1, y1, x2, y2, score], where box are the upper left coordinates, lower right coordinates, and classification score of the prediction box. The bunch with the largest number of prediction boxes in the set of bunches is denoted as bunch max , and the largest classification score in bunch max is denoted as score max , and the number of prediction boxes in bunch max is denoted as count max .
[0037] Step S52. Traverse all the bunches in the set of bunches and delete the bunches that meet the following conditions:
[0038] Denote the current bunch as bunch i , and the largest classification score in bunch i is denoted as score i , and the number of prediction boxes in bunch i is denoted as count i .
[0039] 1) Bunches with a too large difference in classification score from bunch max . Specifically, the value of score max - score i is greater than the threshold thr1, where the threshold thr1 is used to measure whether the difference in classification score between the current bunch bunch i and bunch max is too large. The higher the threshold thr1 is set, the fewer bunches will be filtered, and the more prediction boxes will be retained.
[0040] 2) Bunches with a too large difference in the number of prediction boxes from bunch max . Specifically, the value of (count max - count i ) / count max is greater than the threshold thr2, where the threshold thr2 is used to measure whether the difference in the number of prediction boxes between the current bunch bunch i and bunch maxWhether the difference in the number of predicted boxes is too large. The higher the threshold is set, the fewer clusters will be filtered out, and the more predicted boxes will be retained.
[0041] In step S6 of the present invention, the following steps are further included:
[0042] Before performing k-means clustering on the clusters, it is necessary to first predict the number of cluster centers. The prediction steps are as follows:
[0043] Step S61: Set the initial value of the number of cluster centers count to 0. Put all the predicted boxes in the cluster into the set S. If the number of predicted boxes in the set S is greater than zero, then execute step S62; otherwise, return the value of count as the number of cluster centers.
[0044] Step S62: Take the predicted box with the largest classification score in the set S and denote it as box max . Calculate the intersection over union (IoU) between box max in S and other remaining predicted boxes, and increment the value of count by one.
[0045] Step S63: Delete all the predicted boxes whose IoU is greater than the threshold, including box max . Here, the threshold is mainly used to represent the IoU between the boxes that may be clustered into the same cluster. The smaller this value is, the fewer the number of cluster centers and the fewer the number of final predicted boxes.
[0046] Step S64: If the value of count is greater than or equal to 3, then return 3 as the number of cluster centers; otherwise, execute step S62.
[0047] After obtaining the number of cluster centers, it is necessary to set the initial cluster centers. The steps are as follows:
[0048] Step S65: Set the initial cluster centers according to the number of cluster centers count. Specifically:
[0049] Denote the predicted box corresponding to the largest classification score in the cluster as box max , and the predicted box corresponding to the smallest classification score as box min .
[0050] 1) If the number of cluster centers is 1, then the initial cluster center is set to [box max .
[0051] 2) If the number of cluster centers is 2, then the initial cluster centers are set to [box max , box min .
[0052] 3) If the number of cluster centers is 3, calculate box mid =(box max+ box min ) / 2.0, then the initial cluster centers are set as [box max , box mid , box min .
[0053] Step S66: Define the distance function as the Euclidean distance, or use distance = 1 - IOU(box1, box2) as the distance function, where IOU is the intersection over union of two predicted boxes. Use the above number of cluster centers and initial cluster centers to perform k-means clustering to obtain multiple clusters of predicted boxes and the cluster centers corresponding to the clusters.
[0054] In step S7 of the present invention, the following steps are further included:
[0055] This step is an optional step and can be omitted. The fusion method can adopt the Weighted boxes fusion method proposed by scholars such as Solovyev Roman, or the following method.
[0056] Step S71: Select the cluster center as the base box baseBox. Select the boxes that are relatively similar to the base box and have relatively high classification scores in the cluster of predicted boxes corresponding to this cluster center and put them into the set S. The similarity is defined as
[0057] similarity = 1 - IOU(box1, box2), where IOU is the intersection over union of two predicted boxes.
[0058] Step S71: Calculate the new cluster center box according to the following formula, where x is the predicted box in the set S and score is its corresponding classification score.
[0059] .
[0060] In step S8 of the present invention, the following steps are further included:
[0061] Fuse the similar cluster centers as follows:
[0062] Step S81: Denote the two cluster centers as center1 and center2 respectively; calculate the similarity of the two cluster centers. The similarity is defined as similarity = IOU(center1, center2), where IOU is the intersection over union of two predicted boxes, that is, the ratio of the intersection area to the union area of the two predicted boxes;
[0063] Step S82: If the similarity between two cluster centers is less than the threshold, then merge the two cluster centers, and the merged center is center=(center1 + center2) / 2.0.
[0064] Among the dissimilar cluster centers, the one with lower credibility needs to be deleted.
[0065] Step S83: Delete the cluster center with lower credibility. The specific operations are as follows:
[0066] 1) If the difference in classification scores between the two is too large, delete the cluster center with the lower score. The specific manifestation is that the value of score max - score min is greater than the threshold thr1, where the threshold thr1 is used to measure whether the difference in classification scores between the two is too large. The higher the threshold thr1 is set, the fewer cluster centers will be filtered, and the more prediction boxes will be retained.
[0067] 2) If the difference in the number of corresponding prediction boxes between the two is too large, delete the cluster center with the smaller number. The specific manifestation is that the value of (count max - count min ) / count max is greater than the threshold thr2, where the threshold thr2 is used to measure whether the difference in the number of corresponding prediction boxes between the two is too large. The higher the threshold is set, the fewer clusters will be filtered, and the more prediction boxes will be retained.
[0068] The following is the specific implementation process of the present invention.
[0069] The specific steps of applying the object detection method based on cluster box fusion proposed by the present invention to parathyroid detection are as follows:
[0070] 1. Establish a parathyroid-related data set. This data set contains multiple pictures, and each picture contains parathyroid targets and information such as the boundary labels of the targets.
[0071] 2. Use a feature extraction network to extract features from the input image.
[0072] 3. The feature map obtained after feature extraction is used in the classification and boundary regression networks.
[0073] 4. Divide the obtained prediction boxes into clusters.
[0074] 5. Filter out some clusters with lower credibility.
[0075] 6. Perform k-means clustering within each cluster.
[0076] 7. Fuse similar cluster centers.
[0077] 8. Delete illegal clustering centers.
[0078] 9. Output the clustering centers as the final predicted bounding boxes.
[0079] Figure 2 is the original image of the above-mentioned parathyroid detection example, and the resolution of the experimental image is 2000 1600. Figure 3 is the Figure 2 effect diagram obtained after inputting into the present invention. The red rectangular box is where the real target is located, and the blue rectangular box is the target predicted by the present invention.
[0080] The above are the preferred embodiments of the present invention. All changes made according to the technical solution of the present invention, when the functions and effects generated do not exceed the scope of the technical solution of the present invention, shall fall within the protection scope of the present invention.
Claims
1. A target detection method based on cluster box fusion, characterized in that, It includes the following steps: Step S1, obtaining a data set: Different data sets need to be established for different tasks. The data set consists of pictures with or without targets and target annotation information; Step S2, preprocessing the data set; Step S3, using a feature extraction network to extract features from the data set to obtain a feature map, and then generating a series of anchor boxes on the feature map. These anchor boxes obtain prediction boxes after passing through a classification and boundary regression network; Step S4, classifying the prediction box categories, and then dividing the prediction boxes into clusters in each prediction box category; Step S5, filtering out illegal clusters. The legality of a cluster is described by the aggregation degree and credibility of the prediction boxes in the cluster; Step S6, using a clustering algorithm to cluster the prediction boxes in the cluster; The specific implementation method is as follows: Before performing k-means clustering on the cluster, it is necessary to first predict the number of clustering centers. The prediction steps are as follows: Step S61, set the initial value of the number of clustering centers count to 0, put all the prediction boxes in the cluster into the set S. If the number of prediction boxes in the set S is greater than zero, then execute step S62, otherwise return the value of count as the number of clustering centers; Step S62: Take the prediction box with the largest classification score in set S and denote it as box max ; Calculate the intersection over union (IoU) between box max in set S and the remaining prediction boxes in set S, and increment the value of count by one; Step S63: Delete the prediction boxes and the box whose intersection over union is greater than the threshold. max Both are deleted; the threshold here is used to represent the intersection over union between the boxes that may be clustered into the same cluster. The smaller the threshold, the fewer the number of clustering centers, and ultimately the fewer the number of prediction boxes. Step S64, if the value of count is greater than or equal to 3, then return 3 as the number of clustering centers, otherwise execute step S62; Step S65, set the initial clustering centers according to the number of clustering centers count. Specifically: Record the prediction box corresponding to the maximum classification score in the memory cluster as box max , and record the prediction box corresponding to the minimum classification score as box min ; 1) If the number of cluster centers is 1, then the initial cluster center is set to [box max ; 2) If the number of cluster centers is 2, then the initial cluster centers are set to [box max , box min ; 3) If the number of cluster centers is 3, calculate box mid =(box max +box min ) / 2.0, then the initial cluster centers are set to [box max , box mid , box min ; Step S66, use the number of clustering centers and the initial clustering centers to perform k-means clustering to obtain multiple prediction box clusters and the clustering centers corresponding to the prediction box clusters; Step S7, perform fusion correction on the clustering centers after clustering; Step S8, for the clustering centers obtained after clustering, those with high similarity are fused, and those with low similarity delete the clustering center with low credibility. The finally obtained clustering center is used as the final prediction box. The specific implementation method is as follows: Step S81, denote two clustering centers as center1 and center2 respectively; calculate the similarity between the two clustering centers. The similarity is defined as similarity = IOU(center1, center2), where IOU is the intersection over union of two prediction boxes, that is, the ratio of the intersection area to the union area of two prediction boxes; Step S82, if the similarity between the two clustering centers is less than the threshold, then merge the two clustering centers, and the merged center is center = (center1 + center2) / 2.0; Step S83, delete the clustering center with low credibility. The specific operation is as follows: 1) The classification score gap between the two cluster centers is too large. Delete the cluster center with the lower classification score, i.e., score max -score min value is greater than the threshold thr1, where the threshold thr1 is used to measure whether the classification score gap between the two cluster centers is too large. The higher the threshold thr1 is set, the fewer cluster centers will be filtered out, and the more prediction boxes will be retained; 2) The number of prediction boxes corresponding to the two clustering centers differs significantly. Delete the clustering center with the smaller number, i.e., (count max - count min ) / count max is greater than the threshold thr2. The threshold thr2 is used to measure whether the difference in the number of prediction boxes corresponding to the two is significant. The higher the threshold is set, the fewer clusters will be filtered out, and the more prediction boxes will be retained.
2. The object detection method based on cluster box fusion according to claim 1, wherein In step S4, the prediction boxes are divided into clusters by using the intersection over union between rectangular boxes. The specific method is as follows: Put all the prediction boxes of the same category into the set S. If the number of prediction boxes in the set S is greater than zero, then continue to execute, otherwise step S4 ends; Select the prediction box with the largest classification score in set S and denote it as box max ; Calculate the intersection over union (IoU) between box max in set S and the remaining prediction boxes in set S; Predictive bounding boxes and boxes with an intersection over union greater than a threshold max are all grouped into a cluster bunch i and the cluster bunch is saved i ; The threshold here is used to define the distance between different clusters. The smaller the threshold, the fewer the final number of clusters and the smaller the intersection between clusters; Delete the bunch in set S i , if the number of prediction boxes in set S is greater than zero, repeat the above steps, otherwise step S4 ends.
3. The object detection method based on cluster box fusion according to claim 1, characterized in that, The specific implementation method of step S5 is as follows: In step S4, a set of clusters [bunch0, bunch1,..., bunch n-1 will be obtained, where n is the number of clusters in the set of clusters. Each cluster contains multiple prediction boxes box = [x1, y1, x2, y2, score], where (x1, y1) are the coordinates of the upper left corner of the prediction box, (x2, y2) are the coordinates of the lower right corner of the prediction box, and score is the classification score of the prediction box. The cluster with the largest number of prediction boxes in the set of clusters is denoted as bunch max , and the largest classification score in bunch max is denoted as score max , and the number of prediction boxes in bunch max is denoted as count max ; Traverse all the clusters in the cluster set and delete the clusters that meet the following conditions: Denote the current cluster as bunch i , bunch i The maximum classification score in bunch is denoted as score i , bunch i The number of prediction boxes in bunch is denoted as count i , i ∈ [1, n]; 1) Clusters with a too large classification score gap from bunch max That is, score max -score i whose value is greater than the threshold thr1, where the threshold thr1 is used to measure whether the classification score gap of the current cluster bunch i from bunch max is too large. The higher the threshold thr1 is set, the fewer clusters will be filtered out and the more prediction boxes will be retained; 2) Clusters with a large difference in the number of predicted bounding boxes from bunch max , that is: (count max - count i ) / count max whose value is greater than the threshold thr2, where the threshold thr2 is used to measure whether the difference in the number of predicted bounding boxes between the current cluster bunch i and bunch max is too large. The higher the threshold is set, the fewer clusters will be filtered out and the more predicted bounding boxes will be retained.
4. The object detection method based on cluster box fusion according to claim 1, wherein Step S7 can be optionally not executed and directly jump to step S8, or use the Solovyev Roman method to perform fusion correction on the clustering centers after clustering.
5. The object detection method based on cluster box fusion according to claim 4, wherein The above-mentioned step S7 can also be implemented by the following method: Step S71: Select the clustering center as the base box baseBox; select the boxes that are similar to the base box and have a high classification score from the prediction box cluster corresponding to the clustering center and put them into the set S; where the similarity is defined as: similarity = 1 - IOU(box1, box2), where IOU is the intersection over union of two prediction boxes, and box1 and box2 are any two prediction boxes in the set S; Step S71. Calculate a new clustering center box according to the following formula, where x i is the i-th prediction box in the set S, and score i is the classification score corresponding to the i-th prediction box, and n is the number of prediction boxes in the set S: