Method for obtaining data samples of high labeling value and control device

Abstract

The application relates to a method for obtaining data samples with high labeling value and a control device, and the method comprises the following steps: putting labeled data into a trained reference model to obtain a labeled data feature vector; bringing the labeled data feature vector into a built multivariate Gaussian distribution model to obtain the probability density of the labeled data in a corresponding category, wherein the multivariate Gaussian distribution model is set based on the category; judging whether the labeled data is a difficult sample according to the probability density of the labeled data in the corresponding category; if the labeled data is a difficult sample, finding similar samples of the difficult sample in an unlabeled sample pool according to the labeled data. The labeled data which is a difficult sample is found through the reference model and the multivariate Gaussian distribution model, so that the similar samples of the difficult sample in the unlabeled sample pool are found. According to the scheme, the similar samples of the difficult sample in the unlabeled sample pool can be screened out more quickly, and the data samples with high labeling value can be screened out more quickly.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically providing a method and control device for acquiring data samples with high annotation value. Background Technology

[0002] With the continuous development of artificial intelligence, the demand for samples is increasing day by day, and the richness and accuracy of samples are becoming more and more important in model training.

[0003] In existing technologies, training samples are obtained by manually labeling samples in a database, and then a prediction model is built using these training samples. However, different samples have different roles in model training. Some samples have higher value in model training, and these samples are called high-value samples. However, the existing purely manual labeling method is not targeted at samples in the unlabeled sample library, and finding high-value samples is time-consuming and costly.

[0004] Accordingly, there is a need in this field for a new scheme to obtain high-value labeled samples to address the above problems. Summary of the Invention

[0005] To overcome the above-mentioned shortcomings, this invention is proposed to provide a solution, or at least a partial solution, to the problem that the cost of manually screening high-value samples in the prior art is too high.

[0006] In a first aspect, the present invention provides a method for obtaining data samples with high annotation value, the method comprising: feeding labeled data into a pre-trained benchmark model to obtain a labeled data feature vector; feeding the labeled data feature vector into a pre-built multivariate Gaussian distribution model to obtain the probability density of the labeled data in the corresponding category, wherein the multivariate Gaussian distribution model is set based on the category; determining whether the labeled data is a difficult sample based on the probability density of the labeled data in the corresponding category; if the labeled data is a difficult sample, then searching for similar samples of the difficult sample in an unlabeled sample pool based on the labeled data.

[0007] As an alternative or supplement to the above solutions, in a method according to an embodiment of the present invention, the step of determining whether the labeled data is a difficult sample based on the probability density of the corresponding category of the labeled data includes: determining whether the labeled data belongs to the outlier space of its category based on the probability density of the labeled data in the category, wherein the outlier space is the interval with the lowest probability density in a Gaussian distribution and an interval probability lower than a first threshold; if the labeled data belongs to the outlier space of its category, determining whether the labeled data belongs to the intra-class space of any category, wherein the intra-class space is the interval with an interval probability density in a Gaussian distribution greater than a second threshold and less than a third threshold, the second threshold being greater than the first threshold; if the labeled data belongs to both the outlier space of its category and the intra-class space of any category, then the labeled data is determined to be a difficult sample.

[0008] As an alternative or supplement to the above solutions, in a method according to an embodiment of the present invention, finding similar samples of the difficult sample in the unlabeled sample pool based on the labeled data includes: feeding the labeled data into a trained variational autoencoder to obtain a difficult sample feature vector; calculating the cosine similarity between the difficult sample feature vector and the unlabeled feature vectors of multiple unlabeled data in the unlabeled sample pool and the difficult sample feature vector, wherein the unlabeled feature vector is a feature vector obtained by passing the unlabeled data in the unlabeled sample pool through the trained variational autoencoder, and the unlabeled data with the higher cosine similarity ranking is taken as the similar sample of the difficult sample.

[0009] As an alternative or supplement to the above solutions, in a method according to an embodiment of the present invention, the benchmark model converts the data into a feature vector, which includes classification head features and detection features.

[0010] As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, unlabeled data is put into a pre-trained benchmark model to obtain an unlabeled data feature vector; the unlabeled data feature vector is substituted into a pre-built multivariate Gaussian distribution model to obtain the probability density of the unlabeled data in the corresponding category; based on the probability density of the unlabeled data in the corresponding category, it is determined that the unlabeled data is in the outlier space in the Gaussian distribution of all categories, wherein the outlier space is the interval with the lowest probability density in the Gaussian distribution and the interval probability is lower than the fourth threshold; if the unlabeled data is in the outlier space in the Gaussian distribution of all categories, then the unlabeled data is an outlier sample.

[0011] As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, the confidence score of unlabeled data in the corresponding category in the unlabeled sample pool is obtained; based on the confidence score of the unlabeled data in the corresponding category, the uncertainty of the unlabeled data is obtained, and the unlabeled data with the highest uncertainty ranking in the unlabeled sample pool is taken as the uncertain sample.

[0012] As an alternative or supplement to the above solutions, in a method according to an embodiment of the present invention, the method for calculating the uncertainty of unlabeled data includes:

[0013] H(I i ,k)=-p ik logp i,k -(1-p i,k log(1-p) i,k );

[0014] H(I i |D S )=∑ k∈[t] H(I i ,k),.

[0015] Where p i，k H(I) is the confidence score of the k-th instance category in the unlabeled data. i H(I, k) represents the uncertainty of the unlabeled data with respect to the k-th instance category. i |D S ) represents the overall uncertainty of the unlabeled data.

[0016] As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, the labeled misidentified samples are put into a variational autoencoder to obtain the feature vector of the misidentified samples; the feature vector of the misidentified samples is calculated with the unlabeled feature vectors of multiple unlabeled data in the unlabeled sample pool to obtain the cosine similarity between the unlabeled feature vectors of multiple unlabeled data in the unlabeled sample pool and the feature vector of the misidentified samples; and the unlabeled data with the higher cosine similarity ranking is taken as the similar sample of the misidentified sample.

[0017] In a second aspect, a control device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and executed by the processor to perform the method for acquiring data samples with high annotation value as described in any of the above-described technical solutions.

[0018] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the method for acquiring data samples with high annotation value as described in any of the above-described technical solutions.

[0019] The above-described technical solutions of the present invention have at least one or more of the following beneficial effects:

[0020] In implementing the technical solution of this invention, labeled data of difficult samples are found through a benchmark model and a multivariate Gaussian distribution model, thereby finding similar samples of the difficult samples in the unlabeled sample pool. This solution can more quickly filter out similar samples of difficult samples from the unlabeled sample pool, and more quickly filter out data samples with high labeling value. Compared with manual labeling, this solution is more efficient and saves more time and effort. Attached Figure Description

[0021] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:

[0022] Figure 1 This is a schematic flowchart of the main steps of a method for obtaining data samples with high annotation value according to an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of the outlier space of a method for obtaining data samples with high annotation value according to an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the intraclass space of a method for obtaining data samples with high annotation value according to an embodiment of the present invention;

[0025] Figure 4 This is a flowchart illustrating the secondary steps of a method for obtaining data samples with high annotation value according to an embodiment of the present invention. Detailed Implementation

[0026] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0027] In the description of this invention, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and may also include software components, such as program code, or a combination of software and hardware. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "a" or "this" can also include plural forms.

[0028] Here we will first explain some of the terms involved in this invention.

[0029] Gaussian model: A Gaussian model is a model that uses the Gaussian probability density function (normal distribution curve) to accurately quantify things, decomposing a thing into several models based on the Gaussian probability density function (normal distribution curve).

[0030] Current traditional deep learning methods, primarily supervised learning, often rely on a large number of labeled samples for training, allowing the model to learn more valuable knowledge. However, labeled samples are severely scarce in real-world applications. Furthermore, labeling a large number of samples incurs expensive labeling costs. Therefore, finding high-value samples from massive amounts of unlabeled data is extremely important.

[0031] Example 1:

[0032] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a method for obtaining data samples with high annotation value according to an embodiment of the present invention. Figure 1 As shown, the method for obtaining data samples with high labeling value in this embodiment of the invention mainly includes the following steps S10-S40.

[0033] Step S10: Input the labeled data into the trained benchmark model to obtain the labeled data feature vector.

[0034] In this embodiment, the labeled data refers to data that has been manually labeled, meaning data samples that have already been marked with their respective categories. The baseline model is a neural network model used to transform the data into feature vectors. The feature vectors are represented in matrix form.

[0035] In one implementation, the baseline model is built on the YOLOX framework. Preferably, in this implementation, the baseline model has a fixed number of anchors of 8500. In this implementation, the baseline model is a feature extractor. Unlike typical feature extractors, the feature vector obtained after converting the data into a feature vector includes both classification head features and detection features. The classification head features are the feature portions obtained from the classification head in YOLOX, and the detection features are the feature portions obtained from the detection head in YOLOX. In this implementation, the labeled data feature vector obtained through the baseline model includes both the classification head features and detection features of the labeled data.

[0036] Step S20: Substitute the labeled data feature vector into the established multivariate Gaussian distribution model to obtain the probability density of the labeled data in the corresponding category.

[0037] In this embodiment, a feature vector is a mathematical representation used to describe the characteristics of the data for subsequent classification and prediction. A multivariate Gaussian distribution model is a commonly used probabilistic model used to describe the relationship between multiple random variables. In this model, each random variable is assumed to follow a normal distribution, and the relationship between multiple random variables is described by a covariance matrix. In this embodiment, the random variables represent different categories. In this embodiment, the established multivariate Gaussian distribution model is used to calculate the probability density of the labeled data in the corresponding category. These probability densities can be used for subsequent classification and prediction. The probability density represents the distribution of the labeled data feature vectors in the samples.

[0038] In one implementation, it is generally assumed that the data in the sample database conforms to a basic multivariate Gaussian distribution model, and that even considering only one category, the single-category distribution closely approximates a Gaussian model. Depending on the types of data in the sample database, the multivariate Gaussian distribution in this implementation can be either a correlated multivariate Gaussian distribution or an independent multivariate Gaussian distribution. When the types of data in the sample database are highly correlated, a correlated multivariate Gaussian distribution is used; when the types of data in the sample database are low or uncorrelated, an independent multivariate Gaussian distribution is used. In this implementation, the data in the sample database are uncorrelated, and an independent multivariate Gaussian distribution is used, with the specific formula as follows:

[0039]

[0040] Where μ represents x n×m The mean in each dimension, where ∑ represents xn×m The covariance matrix. The multivariate Gaussian distribution model is determined by the parameters μ and ∑, and the parameters μ and ∑ in the equation are determined by the matrix x. n×m The calculations were obtained. The calculation methods are as follows:

[0041] μ i =E(x) i );

[0042] ∑ ij =cov(x i ,x j )=E[(x i -μ i (x) j -μ j )];

[0043] A multivariate Gaussian distribution model is constructed using the mean μ and covariance matrix ∑. Then, the labeled data feature vectors are fed into the multivariate Gaussian distribution model to obtain the probability density of the labeled data in the corresponding category. Assuming the system samples are divided into three categories, "wolf," "cat," and "wolf," the multivariate Gaussian distribution model will fit these three categories respectively. When a vectorized image is fed into the multivariate Gaussian distribution model, the model will provide its probability distribution in each category. For example, if a labeled data feature vector is determined to be "dog," feeding it into the constructed multivariate Gaussian distribution model will yield the following probability distribution for the corresponding category:

[0044] dog 3.25％ cat 0.10％ Wolf 37.29％

[0045] As can be seen, the probability density of the labeled data feature vector is 3.25% in the "dog" category, 0.10% in the "cat" category, and 37.29% in the "wolf" category. It's important to note that this probability density is not the probability that the labeled data belongs to any of these categories. Unlike probability, analyzing the probability density of a single point is meaningless; it needs to be analyzed in conjunction with the data from the entire interval. The significance of probability density lies in the comparison and reference between the data and the interval. In a multivariate Gaussian distribution, the integral of the interval probability density represents the probability of the interval occurring; the integral over the entire interval yields the overall probability of 1.

[0046] Step S30: Determine whether the labeled data is a difficult sample based on the probability density of the labeled data in the corresponding category.

[0047] In this embodiment, hard samples refer to data that are difficult to classify correctly, usually due to uneven data distribution or unclear data features.

[0048] In one implementation, difficult samples are identified through steps S301-S303:

[0049] Step S301: Based on the probability density of the labeled data in the corresponding category, determine whether the labeled data belongs to the outlier space of its category.

[0050] In this embodiment, the category of the labeled data is determined, and the category to which the labeled data belongs is its determined category. The outlier space is the interval with the lowest probability density in a Gaussian distribution and whose interval probability is lower than a first threshold. Labeled data that is in the outlier space of a certain category indicates that this labeled data is a relatively marginal sample within its category.

[0051] In one embodiment, the outlier space satisfies two conditions simultaneously: firstly, it is the interval with the lowest probability density in the Gaussian distribution, and secondly, the probability corresponding to the interval is lower than a first threshold.

[0052] In this embodiment, the outlier space can be two discontinuous segments of space. In this embodiment, the outlier space is primarily determined based on a first threshold. For example, in a standard Gaussian model, when the first threshold is set to 0.1, the outlier space is as follows: Figure 2 As shown. It should be noted that when the same first threshold is used in different Gaussian distributions, the probability density corresponding to the first threshold is not necessarily equal. In this embodiment, the threshold can be set manually or determined using techniques such as cross-validation.

[0053] Step S302: If the labeled data belongs to the outlier space of its category, determine whether the labeled data belongs to the intra-class space of any category.

[0054] In this embodiment, the intra-class space is the interval in a Gaussian distribution where the interval probability density is greater than a second threshold but less than a third threshold, and the second threshold is greater than the first threshold. Labeled data located in the intra-class space that is not of its own category indicates that this labeled data is not easily categorized.

[0055] In one implementation, the intra-class space can also be a discontinuous space. In this implementation, the intra-class space is mainly determined based on the second and third thresholds. For example, in a standard Gaussian model, when the second threshold is selected as 0.2 and the third threshold as 0.4, the intra-class space is as follows: Figure 3 As shown.

[0056] Step S303: If the labeled data belongs to the outlier space of its own category and also belongs to the intra-class space of any category, then the labeled data is judged as a difficult sample.

[0057] In this embodiment, a data point cannot simultaneously reside in both the outlier space of one category and the intra-category space of the same category. A labeled data point is considered a hard sample if it simultaneously belongs to both the outlier space of its own category and the intra-category space of any other category.

[0058] In one implementation, the labeled data is characterized by having been manually labeled, and manually labeled data has a clear category. Taking the labeled data described in step S20 as an example, this labeled data is a dog that looks very much like a wolf. Its probability density in the "dog" category is 3.25%, in the "cat" category is 0.10%, and in the "wolf" category is 37.29%.

[0059] Assuming the first threshold of the Gaussian distribution for "dog" is 0.1, the probability density interval for the outlier space is (0, 7.5%). For the Gaussian distribution for "wolf," the second threshold is 0.2, and the third threshold is 0.4, resulting in a probability density interval for the within-class space of (20%, 40%). Calculations show that since 0 < 3.25% < 7.5%, this labeled data point is in the outlier space within its own category, "dog," while simultaneously belonging to the within-class space of the "wolf" category. Since this labeled data point belongs to both the outlier space of its own category and the within-class space of another category, it is considered a hard sample.

[0060] Step S40: If the labeled data is a difficult sample, then find similar samples of the difficult sample in the unlabeled sample pool based on the labeled data.

[0061] In this embodiment, the unlabeled sample pool is the collection of samples that are not labeled.

[0062] In one implementation, the model is trained better by finding similar samples to the difficult sample and adding them to the training set. This achieves rapid identification of high-value labeled data samples and saves the cost of manual labeling. In this implementation, steps S401-S403 are used.

[0063] Step S401: Input the labeled data into the trained variational autoencoder to obtain the feature vector of hard samples.

[0064] In this embodiment, the labeled data are hard samples. A variational autoencoder is an unsupervised learning model that learns feature representations of the data during training.

[0065] In one implementation, the labeled data is converted into hard sample feature vectors through a variational autoencoder. The hard sample feature vectors can describe the data characteristics of hard samples and provide a basis for identifying hard samples in subsequent unlabeled samples.

[0066] Step S402: Calculate the cosine similarity between the feature vector of the difficult sample and the unlabeled feature vectors of multiple unlabeled data in the unlabeled sample pool, and the feature vector of the difficult sample.

[0067] In this embodiment, the unlabeled feature vector is the feature vector obtained by processing unlabeled data from the unlabeled sample pool through a trained variational autoencoder. Cosine similarity is a standard used to calculate the similarity between two vectors and to measure the similarity between vectors.

[0068] In one implementation, the cosine similarity between unlabeled data in the unlabeled sample pool and hard samples is calculated. The level of cosine similarity indicates the probability that the unlabeled data being compared with a hard sample is itself a hard sample. In this implementation, a higher cosine similarity indicates a higher probability that the compared unlabeled data is a hard sample; a lower cosine similarity indicates a lower probability. In this implementation, the calculation is performed between the hard sample and multiple unlabeled data sets, where multiple unlabeled data sets refer to at least one unlabeled data set. In this implementation, relevant parameters can be set, such as searching 100,000 unlabeled data sets in the unlabeled sample pool, or searching all unlabeled data sets in the unlabeled sample pool. Specific parameters can be determined by those skilled in the art based on their needs.

[0069] Step S403: Select the unlabeled data with the highest cosine similarity ranking as similar samples to the difficult samples.

[0070] In this embodiment, similar samples to difficult samples refer to unlabeled data selected from the unlabeled sample pool that have a relatively high probability of becoming difficult samples.

[0071] In one implementation, preferably, the TOP-K method is used to identify unlabeled data with high cosine similarity rankings as similar samples to the difficult samples. In this implementation, labeled data is converted into feature vectors, and then the similarity between the labeled data and the unlabeled data is calculated to find unlabeled data similar to the difficult samples as similar samples.

[0072] In this embodiment, similar samples to difficult samples are data samples with high annotation value.

[0073] Example 2:

[0074] Step S501: Input the unlabeled data into the trained benchmark model to obtain the unlabeled data feature vector.

[0075] In this embodiment, the baseline model is the same as that shown in Embodiment 1, and will not be repeated here.

[0076] In one implementation, unlabeled data can be fed into the baseline model in a batch input manner, that is, multiple unlabeled data are input into the model at once to obtain multiple unlabeled data feature vectors, or it can be input one by one, that is, one unlabeled data is input into the model to obtain a feature vector before the next unlabeled data is input for prediction.

[0077] Step S502: Substitute the feature vector of the unlabeled data into the established multivariate Gaussian distribution model to obtain the probability density of the unlabeled data in the corresponding category.

[0078] In this embodiment, the established multivariate Gaussian distribution model is the same as that shown in Embodiment 1, and will not be described again here. The probability density of unlabeled data in the corresponding category refers to the probability density value of the unlabeled data in the corresponding category obtained by substituting the feature vector of the unlabeled data into the established multivariate Gaussian distribution model.

[0079] Step S503: Based on the probability density of the corresponding category of the unlabeled data, determine that the unlabeled data is in the outlier space in the Gaussian distribution of all categories.

[0080] In this embodiment, the outlier space is the interval with the lowest probability density in the Gaussian distribution and whose interval probability is lower than the fourth threshold.

[0081] Step S504: If the unlabeled data is in the outlier space in the Gaussian distribution of all categories, then the unlabeled data is an outlier sample.

[0082] In this embodiment, the method for determining outlier spaces is the same as in Embodiment 1, and will not be repeated here.

[0083] In one implementation, preferably, the fourth threshold is adjusted according to the actual situation during outlier detection to achieve the best detection effect. In summary, this embodiment provides an outlier detection method based on a multivariate Gaussian distribution. By incorporating unlabeled data into a baseline model, outliers in the unlabeled sample library can be identified more quickly.

[0084] In this embodiment, the labeled outlier samples are data samples with high labeling value.

[0085] Example 3:

[0086] Step S601: Obtain the confidence scores of unlabeled data in the unlabeled sample pool across all categories.

[0087] In this embodiment, the confidence score of unlabeled data refers to the reliability of classifying each unlabeled data point into a certain category. These confidence scores can be used to evaluate the reliability of unlabeled data with respect to its corresponding category and serve as the basis for selecting which category to label.

[0088] In one implementation, the confidence score can be used to assess the reliability of unlabeled data relative to its corresponding category and as a basis for selecting which category to label.

[0089] Step S602: Based on the confidence scores of the unlabeled data in all categories, obtain the uncertainty of the unlabeled data.

[0090] In this embodiment, the uncertainty of unlabeled data refers to the degree of uncertainty regarding the overall category of the unlabeled data. Taking images as an example of unlabeled data:

[0091] In one implementation, the uncertainty is calculated as follows:

[0092] H(I i ,k)=-p i,k logp i,k -(1-p i，k log(1-p) i，k )

[0093]

[0094] Where p i,k H(I) is the confidence score of the k-th instance category in the i-th image. i H(I, k) is the uncertainty of the i-th image for the k-th instance category, where H(I, k) is the uncertainty of the i-th image for the k-th instance category. i |D S ) represents the overall uncertainty of the i-th image.

[0095] For ease of understanding, this embodiment assumes two categories, such as "table" and "chair". In this embodiment, unlabeled data may simultaneously exist in both categories; for example, an image containing both a table and a chair would present the situation described in this embodiment. The uncertainty of each category is then obtained based on its confidence score, and the uncertainty of the unlabeled sample is obtained by analyzing the uncertainties of different categories.

[0096] Step S603: Select the unlabeled data with the highest uncertainty ranking from the unlabeled sample pool as uncertain samples.

[0097] In this embodiment, we can select unlabeled data with high uncertainty ranking as uncertain samples based on the uncertainty of the unlabeled data, thereby enhancing the training dataset of the model. These uncertain samples can be used to retrain the model to improve its accuracy and generalization ability.

[0098] In one implementation, a top-k approach is used to select uncertain samples. Specifically, we can select the k unlabeled data points with the highest uncertainty as uncertain samples, where k can be adjusted as needed. For each unlabeled data point, its confidence score and uncertainty across all categories can be calculated, and the data can be sorted according to uncertainty. Then, the top k unlabeled data points are selected as uncertain samples. Besides using the top-k approach, uncertain samples can also be selected based on an uncertainty threshold, for example, selecting all unlabeled data points with uncertainties higher than a certain threshold.

[0099] In this embodiment, uncertain samples are data samples with high annotation value.

[0100] Example 4:

[0101] Step S701: Put the labeled misidentified samples into the variational autoencoder to obtain the feature vector of the misidentified samples.

[0102] In this embodiment, the labeled misidentified samples are input into a variational autoencoder, which converts them into feature vector representations. The misidentified samples in this embodiment include: misidentified samples from the training set and / or misidentified samples from the validation set.

[0103] Step S702: Calculate the cosine similarity between the feature vector of the misidentified sample and the unlabeled feature vectors of multiple unlabeled data in the unlabeled sample pool and the feature vector of the misidentified sample.

[0104] In this embodiment, the cosine similarity is close to that in step S402.

[0105] In one implementation, the cosine similarity is calculated as follows:

[0106]

[0107] Where x w To select the feature vector of the erroneous sample, x u These are the feature vectors of unlabeled samples.

[0108] Step S703: Select the unlabeled data with the highest cosine similarity ranking as similar samples to the misidentified samples.

[0109] In this embodiment, similar to step S603, the unlabeled data with the highest cosine similarity ranking is considered as a similar sample to the misidentified sample.

[0110] In this embodiment, similar samples to misidentified samples are data samples with high annotation value.

[0111] In this embodiment, all data samples with high annotation value found are labeled and reviewed, and then added to the training data.

[0112] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.

[0113] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0114] Furthermore, the present invention also provides a control device. In one embodiment of the control device according to the present invention, the control device includes a processor and a storage device. The storage device can be configured to store a program for executing the method for acquiring data samples with high annotation value described in the above-described method embodiments. The processor can be configured to execute the program in the storage device, which includes, but is not limited to, the program for executing the method for acquiring data samples with high annotation value described in the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. This control device can be a control device device comprising various electronic devices.

[0115] Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for performing the method for acquiring data samples with high annotation value described in the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described method for acquiring data samples with high annotation value. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium can be a storage device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0116] Furthermore, it should be understood that since the various modules are only provided to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the processor's software, a part of its hardware, or a combination of software and hardware. Therefore, the number of modules shown in the figures is merely illustrative.

[0117] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

[0118] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for obtaining a data sample with high label value, characterized in that, The method comprises the following steps: putting the labeled data into a trained benchmark model to obtain a labeled data feature vector; bringing the labeled data feature vector into a built multivariate Gaussian distribution model to obtain the probability density of the labeled data in the corresponding category, wherein the multivariate Gaussian distribution model is set based on the category; judging whether the labeled data is a difficult sample according to the probability density of the labeled data in the corresponding category; if the labeled data is a difficult sample, finding similar samples of the difficult sample in the unlabeled sample pool according to the labeled data.

2. The method of claim 1, wherein, The judgment of whether the labeled data is a difficult sample according to the probability density of the labeled data in the corresponding category comprises: judging whether the labeled data belongs to the out-of-class space of the category according to the probability density of the labeled data in the category, wherein the out-of-class space is the interval with the lowest probability density in the Gaussian distribution and the interval probability is lower than the first threshold value; if the labeled data belongs to the out-of-class space of the category, judging whether the labeled data belongs to the in-class space of any category, wherein the in-class space is the interval with the probability density greater than the second threshold value and less than the third threshold value in the Gaussian distribution, and the second threshold value is greater than the first threshold value; if the labeled data belongs to the out-of-class space of the category and also belongs to the in-class space of any category, it is judged that the labeled data is a difficult sample.

3. The method of claim 1, wherein, Finding similar samples of the difficult sample in the unlabeled sample pool according to the labeled data comprises: putting the labeled data into a trained variational autoencoder to obtain a difficult sample feature vector; calculating the difficult sample feature vector and the unlabeled feature vectors of a plurality of unlabeled data in the unlabeled sample pool to obtain the cosine similarity of the unlabeled feature vectors of a plurality of unlabeled data in the unlabeled sample pool and the difficult sample feature vector, wherein the unlabeled feature vector is the feature vector of the unlabeled data in the unlabeled sample pool obtained by the trained variational autoencoder, taking the unlabeled data with high ranking of cosine similarity as the similar sample of the difficult sample.

4. The method of claim 1-3, wherein, The benchmark model converts data into a feature vector, and the feature vector includes classification head features and detection features.

5. The method of claim 1-3, wherein, The method further comprises the following steps: putting the unlabeled data into a trained benchmark model to obtain an unlabeled data feature vector; bringing the unlabeled data feature vector into a built multivariate Gaussian distribution model to obtain the probability density of the unlabeled data in the corresponding category; judging whether the unlabeled data is in the out-of-class space in the Gaussian distribution of all categories according to the probability density of the unlabeled data in the corresponding category, wherein the out-of-class space is the interval with the lowest probability density in the Gaussian distribution and the interval probability is lower than the fourth threshold value; if the unlabeled data is in the out-of-class space in the Gaussian distribution of all categories, the unlabeled data is an out-of-class sample.

6. The method of claim 1-3, wherein, The method further comprises the following steps: obtaining the confidence score of the unlabeled data in the corresponding category in the unlabeled sample pool; based on the confidence score of the unlabeled data in the corresponding category, obtaining the uncertainty of the unlabeled data, Take the unannotated data with high uncertainty ranking in the unannotated sample pool as uncertain samples.

7. The method of claim 6, wherein, The method for calculating the uncertainty of unannotated data comprises: H(I i ,k) = -p i,k logp i,k -(1-p i,k )log(1-p i,k ) where p i,k is the confidence score of the kth instance class in the unlabeled data, H(I i ,k) is the uncertainty of the unlabeled data for the kth instance class, H(I i |D S ) is the uncertainty of the unlabeled data as a whole.

8. The method of claim 1-3, wherein, Further comprising: Put the misidentified sample into the variational autoencoder to obtain a misidentified sample feature vector; Calculate the misidentified sample feature vector and the unannotated feature vectors of the plurality of unannotated data in the unannotated sample pool to obtain the cosine similarity between the unannotated feature vectors of the plurality of unannotated data in the unannotated sample pool and the misidentified sample feature vector, Take the unannotated data with high cosine similarity ranking as the similar samples of the misidentified sample.

9. A control device comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to execute the method for obtaining data samples with high annotation value according to any one of claims 1 to 8.

10. A computer readable storage medium having stored therein a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to execute the method for obtaining data samples with high annotation value according to any one of claims 1 to 8.

Images