Active learning methods for object detection based on classification consistency before and after object regression

By measuring the classification and localization uncertainty of object detection through the consistency method before and after object regression, the problem of high annotation cost in object detection is solved, and efficient object detection annotation is achieved.

CN117372822BActive Publication Date: 2026-11-17CHONGQING UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311382599.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-24
Publication Date
2026-11-17
Estimated Expiration
2043-10-24

AI Technical Summary

Technical Problem

Existing object detection methods suffer from high annotation costs and a lack of evaluation of localization performance in object detection, resulting in weak interpretability and poor scalability.

Method used

We adopt an active learning method for object detection based on the consistency of classification before and after object regression. By calculating the classification uncertainty and localization uncertainty of the object detection results, we select the most uncertain samples for labeling, thus avoiding dependence on additional modules and specific network structures.

Benefits of technology

It simultaneously considers the uncertainties of classification and localization in object detection, has good scalability and portability, and reduces annotation costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117372822B_ABST
    Figure CN117372822B_ABST
Patent Text Reader

Abstract

This invention discloses an active learning method for object detection based on the consistency of object classification before and after regression. It includes: selecting suitable bounding boxes based on a threshold; generating an image from the bounding box coordinates; comparing the differences between the two classification results; calculating the consistency score of the object based on the two classification results; calculating the object's uncertainty based on the consistency score; and calculating the image's uncertainty based on the object's uncertainty. This method can simultaneously consider both object classification uncertainty and localization uncertainty without relying on a specific network structure or additional module construction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of object detection and active learning, specifically to an active learning method for object detection based on the consistency of object classification before and after regression. Background Technology

[0002] With the development of deep learning, computer vision has made significant progress, and object detection methods based on deep neural networks have achieved good results. However, deep neural networks require a large number of labeled samples for training, which introduces the problem of labeling costs. Active learning is a technique used to solve the labeling cost problem. By allowing the model to autonomously select samples, active learning can greatly reduce the number of samples that need to be labeled. Therefore, it is necessary to apply active learning to object detection problems to reduce the labeling costs in object detection.

[0003] To apply active learning to object detection, one direct approach is to adopt active learning methods from classification problems. This involves estimating the uncertainty of the classification result to measure the value of a sample and selecting the most uncertain sample for labeling. However, due to the lack of evaluation of localization performance in object detection, this method suffers from weak interpretability and lacks performance guarantees. Another approach attempts to model the localization uncertainty of objects, comprehensively considering both classification and localization uncertainties. This method often requires designing additional modules or specific network structures, resulting in poor scalability. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] To address the shortcomings of existing technologies, this invention provides an active learning method for object detection based on the consistency of object classification before and after regression. This method can simultaneously consider the classification uncertainty and localization uncertainty of objects in the object detection problem, and is also convenient to deploy and expand.

[0006] (II) Technical Solution

[0007] To address the aforementioned problems, this invention designs a novel method for maintaining consistency before and after object regression to model classification and localization uncertainties in object detection. To effectively model these two types of uncertainties, the specific technical solution employed in this invention is as follows:

[0008] An active learning method for object detection based on the consistency of classification before and after object regression includes the following steps:

[0009] S1: Initial conditions: Given a small amount of labeled image dataset and a large number of unlabeled image datasets This represents an image sample with object annotations. Marked The object category and location information in the data. This represents unlabeled image samples. Specifically, in the labeled image dataset, the number of images in the dataset is less than or equal to 20% of the total sample size, while in the unlabeled image dataset, the number of images in the dataset is greater than or equal to 20% of the total sample size.

[0010] S2: In Train existing object detection models .

[0011] S3: In any Using the model Through reasoning, we arrive at this... Object detection results in, Indicates the category to which an object belongs; Represents the classification score of an object; The bounding box coordinates of the object are represented by a four-dimensional vector consisting of the x and y coordinates of the top-left and bottom-right corners of the bounding box, describing the object's position in the image. Different objects in the model are indexed by the subscript i; the superscript I indicates that this is a model. Initial test.

[0012] S4: Based on the obtained bounding box coordinates of each object Crop and regenerate a new image

[0013] S5: In Using the model Through reasoning, we obtain Object detection results II indicates that this is a model. The second test, Different objects in the array are indexed by the subscript j.

[0014] S6: Compare the classification results of the two detections and calculate the uncertainty of the object.

[0015] S7: Combine the uncertainties of all objects in the image to obtain the image uncertainty.

[0016] S8: Sort the corresponding images in the unlabeled image dataset according to the magnitude of their uncertainty values, select the objects and object categories from the top K images for labeling, and sort all images in the unlabeled image dataset according to their uncertainty values. The images corresponding to the top K uncertainty values ​​are selected for annotation. Manual annotation can be used. The annotated data is removed from the unannotated image dataset and added to the labeled image dataset. At the same time, the labeled image dataset and the unannotated image dataset are updated.

[0017] S9: Repeat S2-S8 process T times. In this way, the selection process is carried out in a total of T rounds, and T*K labeled images are obtained.

[0018] Preferably, the step S3 to obtain the object detection result is as follows:

[0019] S31: Regarding Input it into the object detection model In this process, the detection results of the model are obtained.

[0020] S32: The detection results of the model are converted into a set of triples using existing methods. The bounding box coordinates of the object are converted to absolute coordinates {x1, y1, x2, y2}, where x1 and y1 represent the x and y coordinates of the top left point, and x2 and y2 represent the x and y coordinates of the bottom right point.

[0021] S33: For the model's detection results, there are low-confidence interference items. Filter out objects with classification scores below the threshold θ1 to obtain the final object detection results.

[0022] Preferably, step S4 regenerates a new image. The process is as follows:

[0023] For a given bounding box coordinate The object is cropped from the original image based on its coordinates and then resized to the original image size to obtain a new image.

[0024] Preferably, step S5 obtains the object detection result of the image. The process is as follows:

[0025] S51: Existing methods will be used to improve the object detection model. The detection results obtained from the model are converted into triples.

[0026] S52: Filter out objects with classification scores below the threshold θ2 to obtain the final object detection results.

[0027] Preferably, step S6 calculates the uncertainty of the object. The process is as follows:

[0028] S61: For Object detection result of the i-th object Its regenerated image Afterwards, multiple test results may be obtained. Extract each test result.

[0029] S62: For a given two detection results and Comparing the classification results of the two tests, the classification results fall into three categories:

[0030] The object category obtained in the first detection is the same as the object category obtained in the second detection. The object classification score obtained from the first detection is less than or equal to the object classification score obtained from the second detection.

[0031] The object category obtained in the first detection is the same as the object category obtained in the second detection. The object classification score obtained from the first detection is greater than the object classification score obtained from the second detection.

[0032] The object categories obtained from the first detection are different from those obtained from the second detection.

[0033] S63: For the above three cases, the process of calculating the object recognition consistency after regression of the j-th object detected by the i-th object is as follows:

[0034] The object recognition consistency of the j-th object obtained from the detection of the i-th object can be calculated by the following formula:

[0035]

[0036] S64: For the regressed object recognition consistency of all objects detected for the i-th object, we need to unify them into the uncertainty of a single object, and use the average of all consistency values ​​to represent the uncertainty of the object. It is calculated using the following formula:

[0037]

[0038] Where n represents the number of objects.

[0039] Preferably, the uncertainty of the S7 image It is obtained through the following method:

[0040] The uncertainty of the image is represented by the maximum value of the uncertainty of all objects. It is calculated using the following formula:

[0041]

[0042] Where m represents the total number of objects.

[0043] Compared with the prior art, the present invention has at least the following advantages:

[0044] This paper proposes a novel active learning method for object detection based on the consistency of classification before and after object regression. By comparing the classification consistency before and after object regression, it can simultaneously consider the uncertainty of object classification and localization. In this way, the method does not require building additional model modules or relying on specific network structures, making our method more scalable and portable. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention and the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a simplified flowchart of the method of the present invention.

[0047] Figure 2 It concerns methods for measuring the uncertainty of object positioning.

[0048] Figure 3 It describes how the method performs on different types of images.

[0049] Figure 4 It is the main structure of the method.

[0050] Figure 5 It refers to the calculation process of the method.

[0051] Figure 6 It refers to the influence of parameters in the method on the result.

[0052] Figure 7 This is the result of the method on the VOC 07+12 dataset. Detailed Implementation

[0053] The present invention will now be described in further detail.

[0054] This invention designs an active learning method for object detection based on the consistency of object classification before and after regression. This method can measure the uncertainty of object localization to a certain extent without adding additional network modules or relying on a specific network structure. Specifically, by measuring the object recognition consistency (ORCR) before and after regression, the classification uncertainty and localization uncertainty of each object can be considered simultaneously.

[0055] Specifically, Figure 2 This paper demonstrates how the method of the present invention measures the localization uncertainty of objects. In object detection problems, the localization uncertainty of an object can be viewed as the accuracy of the predicted bounding box. The more accurate the bounding box prediction, the lower the localization uncertainty of the object. Conversely, the higher the localization uncertainty of the object, the greater the deviation of the predicted bounding box. In image classification problems, in most cases, for a given image, the more accurately it displays the object, the more accurate the prediction result. For example, images containing multiple objects, or images with occlusion or interaction relationships among the objects, often have lower reliability in classification results than images that clearly contain the entire object. Similarly, images that only contain a part of an object often have lower reliability in classification results than images that clearly contain the entire object. Based on this premise, we construct a new method for measuring the localization uncertainty of objects. Figure 2As shown, a given image may contain multiple objects. After the first round of detection, the bounding box coordinates, object categories, and category scores of the objects in the image are obtained. For each detected object, the bounding box localization can be divided into three cases: accurate bounding box localization, loose bounding box localization, or inaccurate bounding box localization. For objects with loose bounding box localization, if we cut out this part of the image and re-predict, the result may contain other objects, leading to a significant difference between the two classification results. For objects with inaccurate bounding box localization, if we cut out this part of the image and re-predict, since it only contains partial information about the object, it may lead to different classification results or low-confidence classification results. For objects with accurate bounding box localization, if we cut out this part of the image and re-predict, the classification result should not be lower than the result obtained in the first detection. There is an exception: when the object contains other objects inside it, that is, two or more objects have an overlapping relationship, and this overlap relationship can also help the model training. Therefore, if there is a discrepancy between the classification results of the two tests for a specific object in the first test, it indicates a problem with the object's localization, thus reflecting localization uncertainty. Simultaneously, the two classification results also reflect classification uncertainty; for example, more difficult objects may lead to incorrect classification results. Therefore, by comparing the classification results from the two tests, we can simultaneously consider both classification uncertainty and localization uncertainty.

[0056] Figure 3 This demonstrates the performance of our method on different image types. For the first column of images, the "bird" category represents a simple set of images that are easy to identify and locate; therefore, the classification results in both tests are unbiased. For the second column of images, the "motorcycle" category represents a set of images that are difficult to classify, as they are hard to distinguish from the "bicycle" category. Therefore, even with accurate localization, there may still be conflicts in the classification results during the second detection. For the third column of images, there are overlapping and interactive relationships between objects. After the initial localization was not tight, the second detection detected other objects within the bounding box, reflecting the image's localization uncertainty. Therefore, by comparing the classification consistency before and after regression, both classification uncertainty and localization uncertainty can be considered simultaneously.

[0057] The present invention will now be further described in conjunction with the accompanying drawings and specific implementation methods, such as... Figure 4 As shown, the active learning method for object detection based on the consistency of classification before and after object regression includes the following steps:

[0058] S1: The object detection problem and the active learning problem are defined as follows:

[0059] Given an image dataset, it is divided into two categories: a small number of labeled image datasets. Large amounts of unlabeled image datasets This indicates labeled samples, by This indicates the corresponding label. This indicates an unlabeled sample.

[0060] For a given image Model It will predict a set of triples to represent the detected object. in, Indicates the category to which an object belongs. This represents the classification score of the object. The bounding box coordinates of the object are represented by a four-dimensional vector that describes the object's position in the image.

[0061] For a given image According to the model Predicted object detection results Active learning methods calculate the uncertainty of a specific object, which reflects the model's... The level of familiarity with the object. Lower uncertainty indicates a more familiar model with the object, and a lower value for that object. Conversely, higher uncertainty indicates a more familiar model with the object. The less familiar the image is with the objects, the greater the uncertainty. In an image, multiple objects may be detected. Active learning methods aggregate the uncertainties of these multiple objects, ultimately yielding the overall uncertainty of the image. Similarly, the image's uncertainty reflects its learning value; images with higher uncertainty have greater learning value. Finally, images are ranked based on their uncertainty, and the most valuable samples are selected for annotation, thus achieving the active learning process.

[0062] S2: In labeled datasets Training the model The training process differs depending on the detection model; it can be completed by referring to the requirements of different detection models. (Object detection model) For existing models, object detection models The training methods also adopt existing training methods.

[0063] S3: In unlabeled image datasets Using the model Inference is performed to obtain the object detection results of the image. Indicates the category to which an object belongs; Represents the classification score of an object; The bounding box coordinates of the object are represented by a four-dimensional vector consisting of the x and y coordinates of the top-left and bottom-right corners of the bounding box, describing the object's position in the image. Different objects in the model are indexed by the subscript i; the superscript I indicates that this is a model. Initial test.

[0064] S31: Regarding Input it into the object detection model In this process, the detection results of the model are obtained. The process is consistent for different detection networks. After image preprocessing, feature extraction, classification and regression prediction, the prediction results of possible objects in the image are obtained.

[0065] S32: Different object detection networks may have some differences in their detection results, such as the presentation of classification results and bounding box coordinates. However, these results can all be converted into a set of triples using existing methods. The category of an object is represented by a corresponding scalar, with different scalars corresponding to different categories. This correspondence information is used in the detection model. The value in the middle is given; the classification score for that category is a score between 0 and 1, representing the model. For a certain degree of certainty in a prediction, the closer the predicted value is to 1, the better the model... The closer the predicted value is to 0, the more reliable the model's predictions. The more unreliable the prediction, the less accurate it becomes. The bounding box coordinates of the object reflect its position in the image. It's worth noting that different models... The predicted bounding box coordinates may be represented differently. For example, the object's position might be represented using the center point coordinates plus the width and height, or using the top-left and bottom-right vertices. The scale of the coordinates may also differ; some models... The predicted coordinates may have been scaled. Specifically, to facilitate subsequent cropping, the bounding box coordinates of the object are converted to absolute coordinates {x1, y1, x2, y2}, representing the x and y values ​​of the top-left and bottom-right coordinates, respectively. These coordinate values ​​are relative to the original image size, not the image size obtained after preprocessing and scaling.

[0066] S33: The model's detection results contain low-confidence interference items. For example, objects with a classification score of 0.01 are often worthless and may be interference in the image. Retaining these objects increases computation and interferes with the method's results. Therefore, a threshold θ1 is used to filter out objects with classification scores below the threshold, resulting in the final object detection results. The value of the threshold θ1 depends on the specific network performance and dataset. For example, for the SSD network, a θ1 of 0.3 will result in good performance on the PASCAL VOC dataset. This value can be determined through a small number of comparative experiments. Alternatively, if you don't want to increase training overhead, θ1 can simply be set to 0.3. Experiments show that different θ1 values ​​have little impact on the model. However, it's important to note that an excessively low θ1 may increase the algorithm's runtime.

[0067] S4: Based on the obtained bounding box coordinates Crop and regenerate a new image The specific process is as follows:

[0068] For bounding box coordinates The object is cropped from the original image based on its coordinates and then resized to the original image size to obtain a new image.

[0069] We compared different image generation methods. One method involves masking other regions of the image by setting them to zero. This allows the regenerated image to be manipulated directly without resizing. However, this approach doesn't produce the same ideal results as cropping. This is likely because cropping alters the image scale after resizing, requiring us to consider the object's appearance at different scales.

[0070] S5: In Using the model Inference is performed to obtain the object detection results of the image. II indicates that this is the second detection, and j indicates that this is the j-th object in the image.

[0071] S51: Similarly, the output prediction of the model yields results similar to those described in S32. Different networks may have different expressions. Through simple transformation, the detection results predicted by the network can be converted into triples.

[0072] S52: Similarly, as described in S33, a threshold θ2 is used to filter out objects with classification scores below the threshold, resulting in the final object detection result. For SSD networks, setting θ² to 0.3 on the PASCAL VOC dataset yields good performance. Similarly, setting θ² to 0.3 is a simple and convenient approach if you don't want to add extra overhead. Different θ² values ​​will not significantly degrade network performance. Figure 6 The effects of the values ​​of θ1 and θ2 on the model performance are shown.

[0073] S6: After obtaining two detection results, compare the classification results of the two detections and calculate the uncertainty of the object.

[0074] S61: For images Object detection result of the i-th object Its regenerated image Afterwards, multiple test results may be obtained. This could be due to loose positioning, such as... Figure 3 As shown in the example, loose localization can lead to the generated image containing other objects, resulting in multiple detection results. After thresholding θ2 filtering, each remaining detection result is extracted.

[0075] S62: For a given two detection results and Comparing the classification results of the two tests, the classification results can be divided into three types:

[0076] The object category obtained in the first detection is the same as the object category obtained in the second detection. The object classification score obtained from the first detection is less than or equal to the object classification score obtained from the second detection. In this scenario, the second detection result is no worse than the first, reflecting to some extent that the model's classification and localization of the object are relatively accurate, with a low probability of loose or inaccurate localization. Furthermore, the consistent classification results across different scales indicate a low probability of misclassification, reflecting a low level of uncertainty regarding the object.

[0077] The object category obtained in the first detection is the same as the object category obtained in the second detection. The object classification score obtained from the first detection is greater than the object classification score obtained from the second detection. In this scenario, although the two classification results are consistent, the confidence level of the second classification is lower than that of the first. This could be due to inaccurate localization, where the generated image only contains partial information about the object, leading to inaccurate classification. Alternatively, it could be due to differences in object classification at different scales; for example, a complete image becomes difficult to classify after scaling. Therefore, this situation reflects a degree of uncertainty regarding the object's classification.

[0078] The object categories obtained from the first detection are different from those obtained from the second detection. In this scenario, the inconsistent classification results indicate a significant difference between the two detections. This could be due to loose localization, resulting in the generated image containing other categories and thus producing different classification outcomes. Alternatively, it could be due to the inherent difficulty in classifying the object itself, leading to different results at different scales. Therefore, this situation reflects a high degree of uncertainty regarding the object.

[0079] S63: For the three cases, the object recognition consistency in regression (ORCR) of the j-th object detected by the i-th object is calculated. The calculation process is as follows:

[0080] In the first case Then the object recognition consistency of the j-th object detected by the i-th object is 0.

[0081] For the second case Then the object recognition consistency of the j-th object detected by the i-th object is:

[0082] For the third case The object recognition consistency of the j-th object obtained from the detection of the i-th object is:

[0083] Therefore, the ORCR of the j-th object detected by the i-th object can be calculated by the following formula:

[0084]

[0085] S64: For the regressed object recognition consistency of all objects detected for the i-th object, we need to unify them into the uncertainty of a single object, and use the average of all consistency values ​​to represent the uncertainty of the object. It can be calculated using the following formula:

[0086]

[0087] It's worth noting that for the i-th object detected in the first detection, the second detection may yield multiple results, indicating a one-to-many relationship. Therefore, considering the consistency of all objects in the second detection is more reasonable. Considering only the consistency of a single object can lead to extreme cases, negatively impacting model performance. For example, selecting the object with the lowest consistency score to represent object uncertainty will cause the algorithm to completely ignore other potentially existing objects (objects that are not closely located but are included). Conversely, selecting the object with the highest consistency score to represent object uncertainty will only focus on erroneous results, making the algorithm too absolute and allowing noise in the image to significantly influence its performance. Therefore, considering the consistency of each object on an average basis will yield a more balanced result.

[0088] S7: Combine the uncertainties of all objects in the image to obtain the image uncertainty.

[0089] The uncertainty of the image is represented by the maximum value of the uncertainty of all objects. It can be calculated using the following formula:

[0090]

[0091] For all objects in an image, most existing object detection and active learning methods represent the maximum uncertainty of an object as the uncertainty of the image. Experiments show that this approach achieves the best results; therefore, we use the maximum uncertainty of all objects to represent the uncertainty of the image.

[0092] S8: Sort all images according to their uncertainty, select the K samples with the highest uncertainty for labeling, and remove the labeled data from the unlabeled image dataset and add it to the labeled image dataset.

[0093] S81: Based on image uncertainty, sort the images in the unlabeled image dataset and select the K images with the highest uncertainty. The value of K can be arbitrarily given depending on the problem and the dataset. It is worth noting that in large-scale datasets, the value of K should not be too small, as this will lead to efficiency problems. In the PASCAL VOC 2007 dataset, we set K to 400. In the PASCALVOC 07+12 dataset, we set K to 1000.

[0094] S82: For the selected K images, human annotation is performed to label the objects and their categories within the images. This annotation is done manually. It's worth noting that, to avoid the cost of manual annotation during algorithm testing, a portion of the labeled dataset can be manually partitioned into an unlabeled image dataset for algorithm testing.

[0095] S83: Remove the K labeled images from the unlabeled image dataset and add them to the labeled image dataset.

[0096] S9: Repeat S2-S8 process T times. In this way, the selection process is carried out in a total of T rounds, and T*K samples are labeled.

[0097] First, retrain the model on the labeled dataset to obtain the model for the current labeled dataset.

[0098] The uncertainty of each image is calculated using the classification results predicted by the current model.

[0099] Based on the uncertainty of each image, select the K most uncertain images for annotation and update the dataset.

[0100] Repeat the above process until the number of iterations T is reached, at which point T*K samples have been labeled.

[0101] The choice of T is arbitrary, depending on the model and the dataset. T*K represents the final number of labeled images, which indicates the cost of labeling. Appropriately increasing the value of K and decreasing the value of T can speed up the model training process and reduce the time cost of model training.

[0102] The core idea of ​​this invention is to indirectly infer the classification uncertainty and localization uncertainty of an object based on the difference in classification results before and after regression. Figure 5The diagram illustrates the computational process of the method on a specific model and dataset. The left half of the diagram shows the predicted classification result obtained from the first detection, the middle half shows the result obtained from the second classification, and the right half shows the specific consistency values. In the first example, the first object was incorrectly classified as a "cat" in the first detection. This error was detected in the second detection and reflected in the calculated consistency score. The second object had errors in both classification and localization, but due to its low prediction confidence, its consistency score was not high. The third object, due to its low confidence, had all detection results removed in the second detection, resulting in a consistency score of 0. The fourth object had a severe localization error, including too many other objects, thus yielding a high consistency score (notably, even with accurate localization, other objects would still be detected in the second detection due to overlapping and occluded objects in the image; our method also selects and labels such images). The second example demonstrates the same effect; the classification result of the second object was biased, resulting in a high consistency score.

[0103] This invention uses the PASCAL VOC dataset as experimental data. The PASCAL VOC dataset contains 20 object categories. We use VOC 2007trainval and VOC 07+12trainval as training sets and test on VOC 2007test. For fair comparison, we use existing network frameworks for model training. We compare a series of active learning methods, including random selection, entropy-based selection, core set-based selection, and probabilistic model-based selection. Random selection randomly selects samples for labeling. Entropy-based selection selects the sample with the highest entropy based on the predicted object category probability vector. Core set-based selection selects samples furthest from the labeled dataset. The probabilistic model-based method models the classification uncertainty and localization uncertainty of objects and then aggregates them for selection. Table 1 shows the results of training the method on VOC 2007trainval. Figure 7 The results of this invention, based on VOC 07+12trainval training, are presented. Under different training conditions, this invention achieved good results.

[0104] Table 1 Comparison of the present invention with existing methods

[0105]

[0106]

[0107] As shown in Table 1, whether compared with traditional methods or current state-of-the-art models, the method of this invention achieved optimal performance in every selection, proving its effectiveness. The performance improvement brought by the method of this invention was more significant in the earlier selections, demonstrating that the samples selected by the method of this invention are more valuable and more conducive to model training.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An active learning method for object detection based on the consistency of object classification before and after regression, characterized in that, Includes the following steps: S1: Initial conditions: Given a labeled image dataset and unlabeled image datasets ; This represents an image sample with object annotations. Marked The object category and location information in the data. This represents unlabeled image samples, where labeled images comprise less than 20% of the dataset, and unlabeled images comprise 20% or more of the dataset. S2: In Train existing object detection models ; S3: In any Using the model Through reasoning, we arrive at this... Object detection results ;in, Indicates the category to which an object belongs; Represents the classification score of an object; The bounding box coordinates of the object are represented by a four-dimensional vector consisting of the x and y coordinates of the top-left and bottom-right corners of the bounding box, describing the object's position in the image. Different objects in the model are indexed by the subscript i; the superscript I indicates that this is a model. Initial test; The steps for obtaining the object detection result in S3 are as follows: S31: Regarding Input it into the object detection model In this process, the detection results of the model are obtained; S32: The detection results of the model are converted into a set of triples using existing methods. Convert the bounding box coordinates of the object to absolute coordinates , These represent the x and y coordinates of the top-left point, respectively. These represent the x and y coordinates of the bottom right coordinate point, respectively. S33: For the model's detection results, filter out those with classification scores below the threshold. The objects are identified, and the final object detection result is obtained. ; S4: Based on the obtained bounding box coordinates of each object Crop and regenerate a new image ; S5: In Using the model Through reasoning, we obtain Object detection results II indicates that this is a model. Second test, Different objects in the text are indexed by the subscript j; The object detection results of the image are obtained in step S5. The process is as follows: S51: Existing methods will be used to improve the object detection model. The detection results obtained from the model are converted into triples. ; S52: Filter out categories with classification scores below the threshold The objects are identified, and the final object detection result is obtained. ; S6: Compare the classification results of the two detections and calculate the uncertainty of the object. ; S7: Combine the uncertainties of all objects in the image to obtain the image uncertainty. ; S8: Sort the corresponding images in the unlabeled image dataset according to the size of the uncertainty value of the images, select the objects and object categories in the first K images for labeling, remove the labeled data from the unlabeled image dataset and add it to the labeled image dataset, and update the labeled image dataset and the unlabeled image dataset at the same time. S9: Repeat steps S2-S8 T times. Thus, the selection process consists of T rounds, resulting in T labeled images. K.

2. The active learning method for object detection based on the consistency of classification before and after object regression as described in claim 1, characterized in that: S4 regenerates a new image. The process is as follows: For a given bounding box coordinate The object is cropped from the original image based on its coordinates and then resized to the original image size to obtain a new image. .

3. The active learning method for object detection based on the consistency of classification before and after object regression as described in claim 2, characterized in that: The uncertainty of the object is calculated in S6. The process is as follows: S61: For The object detection result of the i-th object Its regenerated image Afterwards, multiple test results were obtained. Extract each test result; S62: For a given two detection results and Comparing the classification results of the two tests, the classification results are divided into three types: The object category obtained in the first detection is the same as the object category obtained in the second detection. The object classification score obtained in the first detection is less than or equal to the object classification score obtained in the second detection. ; The object category obtained in the first detection is the same as the object category obtained in the second detection. The object classification score obtained from the first detection is greater than the object classification score obtained from the second detection. ; The object categories obtained in the first detection are different from those obtained in the second detection. ; S63: For the above three classification results, the regression object recognition consistency calculation process for the j-th object detected by the i-th object is as follows: The object recognition consistency of the j-th object obtained from the detection of the i-th object is calculated by the following formula: S64: For the regressed object recognition consistency of all objects detected for the i-th object, we need to unify them into the uncertainty of a single object, and use the average of all consistency values ​​to represent the uncertainty of the object. It is calculated using the following formula: Where n represents the number of objects.

4. The active learning method for object detection based on the consistency of classification before and after object regression as described in claim 3, characterized in that: The uncertainty of the S7 image It is obtained through the following method: The uncertainty of the image is represented by the maximum value of the uncertainty of all objects. It is calculated using the following formula: Where m represents the total number of objects.