A method and system for image data filtering based on object detection learning algorithm

Abstract

This invention discloses an image data filtering method and system based on an object detection learning algorithm, relating to the field of intelligent parking management. The method includes: using a modified object detection model training method and an initially labeled image dataset to obtain a basic object detection model; and on this basis, using a partitioned test set to define image-level uncertainty based on image content recognition accuracy information to obtain preliminary image filtering indicators; further, when the amount of image data filtered in one step is much greater than the expected amount of labeled data, a second image filtering is performed through image feature clustering to ensure that the filtered image content has a diverse distribution under the same data volume conditions; through several iterative model updates, the energy efficiency ratio of newly added labeled data to model accuracy can be ultimately improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent parking management, and in particular to a method and system for image data filtering based on an object detection learning algorithm. Background Technology

[0002] Object detection, a classic task in computer vision, has numerous practical applications and serves as a foundational method for other object detection tasks. In recent years, while unsupervised object detection methods have seen significant improvements in accuracy, speed, and the complexity of applicable scenarios, maintaining high accuracy in real-world applications generally still requires supervised model training using large amounts of scene data. Furthermore, as recognition accuracy continues to improve, the difficulty of obtaining the required labeled data increases dramatically. Therefore, identifying the most effective data for improving model performance from massive amounts of real-world data has become a crucial problem that enterprises need to solve.

[0003] Currently, the active learning-based object detection algorithms developed in recent years offer a valuable approach for filtering high-information-value data and efficiently improving model accuracy. The core of this approach lies in how to leverage the predictive characteristics of object detection models to define an indicator of the amount of information unlabeled images can provide for improving the current model. Existing methods often estimate image uncertainty based on the predicted probability values ​​of object detection, but rarely estimate the uncertainty of the bounding box, or the methods are quite cumbersome. For example, they might involve learning the Gaussian mixture probability density of the predicted bounding box or cropping the image and repeatedly inputting it into the model for comparison of the predicted bounding box results. However, these methods have serious limitations for densely packed and highly occluded target images. In such complex cases, the predicted probability from commonly used object detection methods cannot characterize the prediction uncertainty of the bounding box; instead, it characterizes the classification uncertainty after cropping. Since the two are not stably correlated, it is impossible to conveniently define appropriate filtering indicators using this predicted value to achieve greater model gain after acquiring new labeled data. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an image data filtering method and system based on an object detection learning algorithm. This method can solve the current problem that the instability of object detection models and the representativeness of image features cannot be used as filtering criteria for image data.

[0005] To achieve the above objectives, on the one hand, the present invention provides an image data filtering method based on an object detection learning algorithm, the method comprising:

[0006] The initial dataset of images to be labeled is divided into training set, validation set, and test set;

[0007] The initial target detection model is trained using the training set and the validation set to obtain the target detection model;

[0008] Based on the test set, the comprehensive scoring index of the object detection model, and the prediction results of the object detection model, a comprehensive image uncertainty index corresponding to the initial unlabeled image dataset is determined, and the initial unlabeled image dataset is filtered according to the comprehensive image uncertainty index.

[0009] The target detection model is used to perform a secondary screening of the filtered image data to be labeled.

[0010] The image dataset to be labeled after secondary screening is used to predict the target detection model to obtain pre-labeled labels;

[0011] The target detection model is trained by correcting the image dataset to be labeled based on the pre-labeled images and updating the training set with the corrected image dataset.

[0012] Further, the step of determining the comprehensive image uncertainty index corresponding to the initial unlabeled image dataset based on the test set, the comprehensive scoring index of the object detection model, and the prediction results of the object detection model includes:

[0013] Two sets of prediction results are obtained by using the upper and lower limits of the comprehensive scoring index, and an image-level target box localization instability index is obtained based on the two sets of prediction results.

[0014] The image-level classification prediction instability index is obtained by using the comprehensive scoring index and the prediction results obtained by the target detection model under the expected final output conditions.

[0015] Based on the annotation information of the test set, obtain the image-level average accuracy of the prediction results relative to the annotation information;

[0016] A comprehensive image uncertainty index is determined by using the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy.

[0017] Further, the step of determining a comprehensive image uncertainty index by means of the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy, and then filtering the initial unlabeled image dataset based on the comprehensive image uncertainty index, includes:

[0018] Based on the image-level bounding box localization instability index, image-level classification prediction instability index, and image-level average accuracy of all test set images, obtain the comprehensive image uncertainty index corresponding to each image.

[0019] Images with a comprehensive image uncertainty index greater than a preset threshold are selected as the initial dataset of images to be labeled.

[0020] Furthermore, the step of performing a secondary screening of the filtered data to be labeled using the target detection model includes:

[0021] The image data to be labeled is input into the target detection model, the multi-scale feature map output by the model's FPN layer is extracted, and the multi-scale feature map is pooled and concatenated to obtain the feature vector corresponding to each image to be labeled.

[0022] The feature vector matrix of the feature vector combination of all the images is compressed to obtain the feature representation of the expected dimension, and the cluster category of each image is obtained by performing cluster calculation on the feature representation set of all images.

[0023] The dataset of images to be labeled is further filtered based on the cluster categories of each image.

[0024] Furthermore, the step of performing a secondary filtering of the image dataset to be labeled based on the cluster categories of each image includes:

[0025] Based on the cluster category labels of each image, calculate the ratio of the amount of data in each cluster to the total amount of data in the dataset after the first screening, as well as the amount of data in each cluster after the second screening.

[0026] Based on the comprehensive image uncertainty index corresponding to the data within each cluster, a predetermined number of data points are selected from largest to smallest and integrated as the final secondary screening dataset.

[0027] On the other hand, the present invention provides an image data filtering system based on an object detection learning algorithm. The system includes: a partitioning unit, used to partition an initial dataset of images to be labeled into a training set, a validation set, and a test set;

[0028] The acquisition unit is used to train the initial target detection model using the training set and the validation set to acquire the target detection model;

[0029] The filtering unit is used to determine the comprehensive image uncertainty index corresponding to the initial image dataset to be labeled based on the test set, the comprehensive scoring index of the target detection model, and the prediction result of the target detection model, and to filter the initial image dataset to be labeled based on the comprehensive image uncertainty index.

[0030] The filtering unit is also used to perform secondary filtering on the filtered image data to be labeled using the target detection model;

[0031] The acquisition unit is further configured to predict the target detection model by passing the target detection model through the secondary-filtered image dataset to obtain pre-labeled labels; correct the target detection model by the pre-labeled image dataset to be labeled, and update the training set with the corrected image dataset to be labeled.

[0032] Furthermore, the filtering unit is specifically used to obtain two sets of prediction results through the upper and lower limits of the comprehensive scoring index, and to obtain the image-level target box localization instability index based on the two sets of prediction results.

[0033] The image-level classification prediction instability index is obtained by using the comprehensive scoring index and the prediction results obtained by the object detection model under the expected final output conditions; the image-level average accuracy of the prediction results relative to the annotation information is obtained based on the annotation information of the test set; and the comprehensive image uncertainty index is determined by using the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy.

[0034] Furthermore, the filtering unit is specifically used to obtain a comprehensive image uncertainty index for each image based on the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy corresponding to all test set images; and to obtain images with a comprehensive image uncertainty index greater than a preset threshold as the initial dataset of images to be labeled after filtering.

[0035] Furthermore, the filtering unit is specifically used to input the image data to be labeled into the target detection model, extract the multi-scale feature map output by the model's FPN layer, and concatenate the multi-scale feature maps by pooling to obtain the feature vector corresponding to each image to be labeled; compress the feature vector matrix of the feature vector combination of all images to obtain the feature representation of the expected dimension, and obtain the cluster category of each image by performing cluster calculation on the feature representation set of all images; and perform a secondary filtering on the image dataset to be labeled according to the cluster category of each image.

[0036] Furthermore, the filtering unit is specifically used to calculate the ratio of the amount of data in each cluster to the total amount of data in the dataset after the first filtering, as well as the amount of data in each cluster after secondary filtering, based on the cluster category label of each image; and to select a preset number of data from largest to smallest based on the comprehensive image uncertainty index corresponding to the data in each cluster, and integrate them as the final secondary filtering dataset.

[0037] This invention provides an image data filtering method and system based on an object detection learning algorithm. It utilizes a modified object detection model training method and an initially labeled image dataset to obtain a basic object detection model. Based on this, it defines image-level uncertainty using a partitioned test set and image content recognition accuracy information, thereby obtaining preliminary image filtering indicators. Furthermore, when the amount of image data filtered in one step is significantly greater than the expected amount of labeled data, a second image filtering is performed through image feature clustering to ensure diverse distribution of filtered image content under the same data volume conditions. Through several iterative model updates, the energy efficiency ratio of newly added labeled data to model accuracy can be ultimately improved. Attached Figure Description

[0038] Figure 1 This is a flowchart of an image data filtering method based on an object detection learning algorithm provided by the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of an image data filtering system based on an object detection learning algorithm provided by the present invention. Detailed Implementation

[0040] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0041] like Figure 1 As shown in the figure, an image data filtering method based on an object detection learning algorithm provided by an embodiment of the present invention includes the following steps:

[0042] 101. Divide the initial dataset of images to be labeled into training set, validation set, and test set.

[0043] 102. Train the initial target detection model using the training set and the validation set to obtain the target detection model.

[0044] In this embodiment of the invention, steps 101 and 102 may specifically include: obtaining an initial labeled dataset file in a real-world scenario, used for training the basic object detection model based on the GFLv2 method and defining data filtering metrics; dividing the initial labeled dataset into a training set, a validation set, and a test set according to a certain ratio; and using the training set and the validation set to train the target model to obtain an initial version of the object detection model M1; wherein, the main feature of the GFLv2 method is that the regression of the single-stage object detection box is transformed from a deterministic method to a prediction method with arbitrary probability distribution, and the localization quality score LQE obtained by adding a side neural network is combined with the original object classification prediction score to obtain a comprehensive score index score1 that combines the localization and classification probabilities of the predicted box, which is used for subsequent post-processing operations such as NMS for filtering object detection prediction results; this method can maintain the consistency of the training and testing process, improve the reliability of the final calculated recognition results, and the obtained comprehensive score index score1 can simultaneously characterize the uncertainty of localization and classification, providing controllable data for the model-based image filtering metrics defined in subsequent steps.

[0045] 103. Based on the test set, the comprehensive scoring index of the target detection model, and the prediction results of the target detection model, determine the comprehensive image uncertainty index corresponding to the initial unlabeled image dataset, and filter the initial unlabeled image dataset according to the comprehensive image uncertainty index.

[0046] In this embodiment of the invention, step 103 may specifically include: obtaining two sets of prediction results through the upper and lower limits of the comprehensive scoring index, and obtaining an image-level bounding box localization instability index based on the two sets of prediction results; obtaining an image-level classification prediction instability index based on the prediction results obtained through the comprehensive scoring index and the scoring threshold of the target detection model under the expected final output conditions; obtaining the image-level average accuracy of the prediction results relative to the annotation information based on the annotation information of the test set; and determining a comprehensive image uncertainty index through the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy.

[0047] Specifically, for example, 103.1. Two different sets of prediction results are obtained by using the upper and lower threshold values of the prediction comprehensive scoring index score1. Define the instability index B_uc_score of the picture-level target box localization: Here, make full use of the characteristics that relatively low score1 results will produce more false positives (FP) and false negatives (FN) in the final prediction results, and indicate the prediction uncertainty of the current picture. The specific calculation method is as follows: According to the prediction scoring distribution characteristics of the object detection model, define a set of threshold values. Among them, thr_upper is the scoring threshold value of the model under the expected final output conditions, and thr_lower is determined according to the missed detection situation of the test results. The value is such that most of the obvious missed detection targets generated under the thr_upper condition can be ensured to be output under the thr_lower condition, and thr_lower < thr_upper. For example, thr_lower = 0.2 and thr_upper = 0.5. Input the test data set into the initial detection model M1 to obtain the initial prediction result R after only NMS post-processing, which can include the target box position box, the comprehensive scoring score1, and the corresponding category information label. Then, screen the result R corresponding to score1 through the threshold values thr_lower and thr_upper to obtain two sets of prediction results: R_lower, the results that satisfy score1 > thr_lower; R_upper, the results that satisfy score1 > thr_upper. Taking the result R_upper as the benchmark, calculate the picture-level mean average precision mmAP_est of R_lower relative to R_upper. Using the calculated picture mmAP_est value, define the picture localization uncertainty index B_uc_score = -log(mmAP_est)

[0048] Step 103.2. Using the prediction result R_upper obtained by the prediction comprehensive scoring index score1 and the thr_upper threshold value, define the instability index of the picture-level classification prediction, that is, calculate the total weighted information amount of all screened targets in the picture, C_uc_score = -sum((1 - score1_i)^r * log(score1_i)), i = 1,..n, where n represents the number of prediction targets screened by the thr_upper threshold value; r is the weighted exponential coefficient, which can be defined in advance, such as r = 1;

[0049] Step 103.3. Taking the annotation information of the test set as the benchmark, calculate the picture-level mean average precision mmAP of R_upper relative to the annotation information

[0050] Step 103.4: Using the image-level localization uncertainty index B_uc_score, classification uncertainty index C_uc_score, and image-level average accuracy mmAP calculated in the above steps, the weighting parameter alpha of the comprehensive uncertainty index mix_score is further determined by obtaining the optimal index with correlation to mmAP. The specific calculation method is as follows: the B_uc_score and C_uc_score calculated for all test set images are normalized respectively, i.e.

[0051] B_uc_score2=B_uc_score / max(B_uc_score),

[0052] C_uc_score2=C_uc_score / max(C_uc_score);

[0053] The parameter alpha to be calculated is divided into multiple values ​​alpha_i in the interval [0,1], and cyclically substituted into the formula g_i=alpha_i*B_uc_score2+(1-alpha_i)*C_uc_score2, and g_i and mmAp are calculated. The correlation coefficients corr_score_i (e.g., Pearson, Spearman, Kendall coefficients, etc.) are used to calculate the overall uncertainty index of a dataset. The value that maximizes the negative correlation index is retained as the final weighting coefficient alpha:=alpha_i, i=argmax(-corr_score_i). Thus, the comprehensive uncertainty index of a dataset is calculated as mix_score=alpha*B_uc_score2+(1-alpha)*C_uc_score2. The larger this value is, the higher the relative recognition instability of model M1 for the image within a dataset, and the higher the model performance gain that the image can provide. Based on the final mix_score value, by setting a threshold mix_score_thr, a first-stage filtered image dataset of N that satisfies the condition mix_score>mix_score_thr can be obtained as the dataset to be labeled.

[0054] 104. The target detection model and the preset image feature clustering algorithm are used to perform a second screening of the filtered image data to be labeled.

[0055] In this embodiment of the invention, step 104 may specifically include: inputting the image data to be labeled into the target detection model, extracting the multi-scale feature map output by the model's FPN layer, and concatenating the multi-scale feature maps by pooling to obtain the feature vector corresponding to each image to be labeled; compressing the feature vector matrix of the feature vector combination of all images to obtain the feature representation of the expected dimension, and obtaining the cluster category of each image by performing clustering calculation on the feature representation set of all images; and performing a secondary screening of the image dataset to be labeled based on the cluster category of each image.

[0056] Specifically, for example, the data to be labeled is input into the detection model M1, the multi-scale feature maps output by the model's FPN layer are extracted, and these maps are pooled and concatenated to obtain the feature vector corresponding to each image; the feature vectors of all images obtained above are combined into a feature vector matrix, and PCA compression is performed to obtain the feature representation of the expected dimension. By performing clustering calculations on the feature representation sets of all images, the cluster categories of each image are obtained, and the total number of clusters is m; assuming that the expected maximum amount of data to be labeled is M, and the amount of data N in a single screening dataset is much larger than M, then... The dataset was further compressed by performing a secondary screening on the primary screening dataset as follows: First, based on the cluster labels of the data, the ratio of the amount of data N1_j in each cluster to the total amount of data N in the primary screening dataset was calculated as s2_j = N1_j / N (j = 1,..,m). Then, the amount of secondary screening data in each cluster was calculated as N2_j = M * s2_j. Next, the data in each cluster were sorted from largest to smallest by mix_score, and the top N2_j data were selected and integrated as the final secondary screening dataset.

[0057] 105. The image dataset to be labeled after secondary screening is predicted by the target detection model to obtain pre-labeled labels.

[0058] 106. Correct the unlabeled image dataset according to the pre-labeled labels, update the training set with the corrected unlabeled image dataset, and then train the object detection model.

[0059] In this embodiment of the invention, the images to be labeled selected in the above steps are predicted by the target detection model M1 to obtain pre-labeled labels, which are then returned to human labelers for verification and modification. The corrected new labeled data is then added back to the training set, and the process is repeated in step 101 for several rounds of model iteration and data filtering. This allows for rapid improvement in model performance and a significant reduction in the amount of manually labeled data.

[0060] This invention provides an image data filtering method and system based on an object detection learning algorithm. It utilizes a modified object detection model training method and an initially labeled image dataset to obtain a basic object detection model. Based on this, it defines image-level uncertainty using a partitioned test set and image content recognition accuracy information, thereby obtaining preliminary image filtering indicators. Furthermore, when the amount of image data filtered in one step is significantly greater than the expected amount of labeled data, a second image filtering is performed through image feature clustering to ensure diverse distribution of filtered image content under the same data volume conditions. Through several iterative model updates, the energy efficiency ratio of newly added labeled data to model accuracy can be ultimately improved.

[0061] To implement the method provided in the embodiments of the present invention, the embodiments of the present invention provide an image data filtering system based on an object detection learning algorithm, such as... Figure 2 As shown, the system includes: a division unit 21, an acquisition unit 22, and a filtering unit 23.

[0062] The partitioning unit 21 is used to divide the initial unlabeled image dataset into a training set, a validation set, and a test set.

[0063] The acquisition unit 22 is used to train the initial target detection model using the training set and the validation set to acquire the target detection model.

[0064] The filtering unit 23 is used to determine the comprehensive image uncertainty index corresponding to the initial unlabeled image dataset based on the test set, the comprehensive scoring index of the object detection model, and the prediction result of the object detection model, and to filter the initial unlabeled image dataset based on the comprehensive image uncertainty index.

[0065] The filtering unit 23 is also used to perform secondary filtering on the filtered image data to be labeled using the target detection model.

[0066] The acquisition unit 22 is further configured to predict the target detection model by the target detection model after secondary screening of the image dataset to be labeled, and obtain pre-labeled labels; correct the target detection model by the pre-labeled image dataset to be labeled, and update the training set by the corrected target detection model.

[0067] Further, the filtering unit 23 is specifically used to obtain two sets of prediction results through the upper and lower limits of the comprehensive scoring index, and obtain an image-level bounding box localization instability index based on the two sets of prediction results; obtain an image-level classification prediction instability index based on the prediction results obtained through the comprehensive scoring index and the scoring threshold of the target detection model under the expected final output conditions; obtain the image-level average accuracy of the prediction results relative to the annotation information based on the annotation information of the test set; and determine a comprehensive image uncertainty index through the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy.

[0068] Furthermore, the filtering unit 23 is specifically used to obtain a comprehensive image uncertainty index corresponding to each image based on the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy corresponding to all test set images; and to obtain images with a comprehensive image uncertainty index greater than a preset threshold as the initial image dataset to be labeled after filtering.

[0069] Furthermore, the filtering unit 23 is specifically used to input the image data to be labeled into the target detection model, extract the multi-scale feature map output by the model's FPN layer, and concatenate the multi-scale feature maps by pooling to obtain the feature vector corresponding to each image to be labeled; compress the feature vector matrix of the feature vector combination of all images to obtain the feature representation of the expected dimension, and obtain the cluster category of each image by performing cluster calculation on the feature representation set of all images; and perform a secondary filtering on the image dataset to be labeled according to the cluster category of each image.

[0070] Furthermore, the filtering unit 23 is specifically used to calculate the ratio of the amount of data in each cluster to the total amount of data in the dataset after the first filtering, as well as the amount of data in each cluster after secondary filtering, based on the cluster category label of each image; and to select a preset number of data from largest to smallest based on the comprehensive image uncertainty index corresponding to the data in each cluster, and integrate them as the final secondary filtering dataset.

[0071] This invention provides an image data filtering system based on an object detection learning algorithm. It utilizes a modified object detection model training method and an initially labeled image dataset to obtain a basic object detection model. Based on this, it defines image-level uncertainty using a partitioned test set and image content recognition accuracy information, thereby obtaining preliminary image filtering indicators. Furthermore, when the amount of image data filtered in one step is significantly greater than the expected amount of labeled data, a second image filtering is performed through image feature clustering to ensure diverse distribution of filtered image content under the same data volume conditions. Through several iterative model updates, the system ultimately achieves an improved efficiency ratio between the amount of newly labeled data and model accuracy.

[0072] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0073] In the above detailed description, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features of the single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, wherein each claim stands alone as a preferred embodiment of the invention.

[0074] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0075] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

[0076] Those skilled in the art will also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, units, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present invention.

[0077] The various illustrative logic blocks or units described in the embodiments of this invention can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic system, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing systems, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0078] The steps of the methods or algorithms described in the embodiments of this invention can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0079] In one or more exemplary designs, the functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of these three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted on a computer-readable medium in the form of one or more instructions or code. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. Storage media can be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage systems, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection can be suitably defined as a computer-readable medium, for example, if the software is transmitted from a website, server, or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wirelessly, such as infrared, wireless, and microwave, it is also included in the defined computer-readable medium. The disks and discs mentioned include compressed disks, laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while disks typically copy data optically using lasers. Combinations of the above can also be contained in computer-readable media.

[0080] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for filtering image data based on an object detection learning algorithm, characterized in that, The method includes: The initial dataset of images to be labeled is divided into training set, validation set, and test set; The initial target detection model is trained using the training set and the validation set to obtain the target detection model; Based on the test set, the comprehensive scoring index of the object detection model, and the prediction results of the object detection model, a comprehensive image uncertainty index corresponding to the initial unlabeled image dataset is determined, and the initial unlabeled image dataset is filtered according to the comprehensive image uncertainty index. The target detection model and the preset image feature clustering algorithm are used to perform a second screening of the filtered image data to be labeled. The image dataset to be labeled after secondary screening is used to predict the target detection model to obtain pre-labeled labels; The target detection model is trained by correcting the image dataset to be labeled based on the pre-labeled labels and updating the training set with the corrected image dataset to be labeled. The step of determining the comprehensive image uncertainty index corresponding to the initial unlabeled image dataset based on the test set, the comprehensive scoring index of the object detection model, and the prediction results of the object detection model includes: Two sets of prediction results are obtained by using the upper and lower limits of the comprehensive scoring index, and an image-level target box localization instability index is obtained based on the two sets of prediction results. The image-level classification prediction instability index is obtained by using the comprehensive scoring index and the prediction results obtained by the target detection model under the expected final output conditions. Based on the annotation information of the test set, obtain the image-level average accuracy of the prediction results relative to the annotation information; A comprehensive image uncertainty index is determined by using the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy.

2. The image data filtering method based on an object detection learning algorithm according to claim 1, characterized in that, The step of determining a comprehensive image uncertainty index by using the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy, and then filtering the initial unlabeled image dataset based on the comprehensive image uncertainty index, includes: Based on the image-level bounding box localization instability index, image-level classification prediction instability index, and image-level average accuracy of all test set images, obtain the comprehensive image uncertainty index corresponding to each image. Images with a comprehensive image uncertainty index greater than a preset threshold are selected as the initial dataset of images to be labeled.

3. The image data filtering method based on the object detection learning algorithm according to claim 1, characterized in that, The step of performing a secondary screening of the filtered data to be labeled using the target detection model and the preset image feature clustering algorithm includes: The image data to be labeled is input into the target detection model, the multi-scale feature map output by the model's FPN layer is extracted, and the multi-scale feature map is pooled and concatenated to obtain the feature vector corresponding to each image to be labeled. The feature vector matrix of the feature vector combination of all the images is compressed to obtain the feature representation of the expected dimension, and the cluster category of each image is obtained by performing cluster calculation on the feature representation set of all images. The dataset of images to be labeled is further filtered based on the cluster categories of each image.

4. The image data filtering method based on the object detection learning algorithm according to claim 3, characterized in that, The step of performing secondary filtering on the dataset of images to be labeled based on the cluster categories of each image includes: Based on the cluster category labels of each image, calculate the ratio of the amount of data in each cluster to the total amount of data in the dataset after the first screening, as well as the amount of data in each cluster after the second screening. Based on the comprehensive image uncertainty index corresponding to the data within each cluster, a predetermined number of data points are selected from largest to smallest and integrated as the final secondary screening dataset.

5. An image data filtering system based on an object detection learning algorithm, characterized in that, The system includes: The partitioning unit is used to divide the initial dataset of images to be labeled into training, validation, and test sets; The acquisition unit is used to train the initial target detection model using the training set and the validation set to acquire the target detection model; The filtering unit is used to determine the comprehensive image uncertainty index corresponding to the initial image dataset to be labeled based on the test set, the comprehensive scoring index of the target detection model, and the prediction result of the target detection model, and to filter the initial image dataset to be labeled based on the comprehensive image uncertainty index. The filtering unit is also used to perform secondary filtering on the filtered image data to be labeled using the target detection model; The acquisition unit is further configured to predict the target detection model by the target detection model after secondary screening of the image dataset to be labeled, and obtain pre-labeled labels; correct the target detection model by the pre-labeled labels, and update the training set by the corrected target detection model; The filtering unit is specifically used to obtain two sets of prediction results through the upper and lower limits of the comprehensive scoring index, and to obtain an image-level bounding box localization instability index based on the two sets of prediction results; to obtain an image-level classification prediction instability index based on the prediction results obtained through the comprehensive scoring index and the scoring threshold of the object detection model under the expected final output conditions; to obtain the image-level average accuracy of the prediction results relative to the annotation information based on the annotation information of the test set; and to determine a comprehensive image uncertainty index through the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy.

6. The image data filtering system based on the object detection learning algorithm according to claim 5, characterized in that, The filtering unit is further configured to obtain a comprehensive image uncertainty index for each image based on the image-level bounding box localization instability index, the image-level classification prediction instability index, and the image-level average accuracy for all test set images; and to select images with a comprehensive image uncertainty index greater than a preset threshold as the initial dataset of images to be labeled after filtering.

7. The image data filtering system based on an object detection learning algorithm according to claim 5, characterized in that, The filtering unit is further configured to input the image data to be labeled into the target detection model, extract the multi-scale feature map output by the model's FPN layer, and concatenate the multi-scale feature maps by pooling to obtain the feature vector corresponding to each image to be labeled; compress the feature vector matrix of the feature vector combination of all images to obtain the feature representation of the expected dimension, and obtain the cluster category of each image by performing clustering calculation on the feature representation set of all images; and perform a secondary filtering of the image dataset to be labeled based on the cluster category of each image.

8. The image data filtering system based on the object detection learning algorithm according to claim 7, characterized in that, The filtering unit is further configured to calculate the ratio of the amount of data in each cluster to the total amount of data in the dataset after the first filtering, as well as the amount of data in each cluster after secondary filtering, based on the cluster category label of each image; and to select a preset number of data from largest to smallest based on the comprehensive image uncertainty index corresponding to the data in each cluster, and integrate them as the final secondary filtering dataset.

Images