A method for constructing a classifier of an image labeling system based on active learning

Abstract

The application provides a kind of construction method of image annotation system classifier based on active learning, it is related to image annotation classification technical field.The method includes the training classifier stage based on ensemble learning and the training classifier stage based on improved committee voting method.The training classifier stage based on ensemble learning updates the weight of classifier according to the accuracy of each classifier classification each time, so that the classifier with good classification performance has higher weight when sample voting in next stage, and the classifier with poor classification effect has lower weight when voting.The training classifier stage based on improved committee voting method combines voting entropy and relative entropy to measure the difference of voting result, and uses the mean of cosine similarity to measure the similarity between unlabeled samples and all labeled samples, avoids data redundancy, ensures the diversity of samples, and promotes the improvement of classifier efficiency and classification accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image annotation classification, and particularly relates to a method for constructing an image annotation system classifier based on active learning. BACKGROUND

[0002] An annotation system is mainly divided into a non-interactive annotation system and an interactive annotation system. The non-interactive annotation system does not involve human participation in the annotation process, and the annotation result is generated by the system throughout the process. The interactive annotation system involves human participation in the annotation process of the system, and the system undertakes most of the annotation work, and human beings undertake a small part of the annotation work. Among them, the interactive annotation system needs to train a model by using a small number of labeled samples to obtain a classifier with good classification performance.

[0003] A traditional SVM (Support Vector Machine) classifier training method needs to train the classifier by using a large number of labeled samples, so that the performance of the classifier is good. However, at present, a large number of unlabeled samples can be easily obtained in many fields, but it is difficult to obtain a large number of labeled samples. Because this needs experts in the field to manually annotate, a large amount of time cost and economic cost will be spent. Training the classifier by using a large number of labeled samples means that a large amount of time will be spent to complete the training process, and a very high computational complexity will be generated in the training process, and a great consumption of storage space will also be generated. Therefore, a great cost will be paid to obtain a classifier with good performance by using the traditional SVM training method.

[0004] The prior art trains the classifier by using an active learning method. The algorithm of query-by-committee (QBC) is a common active learning algorithm. It establishes two or more classifiers according to the existing class label data, and forms a committee by using the classifiers. The committee is used to vote for the predicted sample, and the sample with the most inconsistent voting result is selected as the candidate sample according to the voting result. Compared with the traditional active learning algorithm, the method based on committee voting can make the sample with a relatively rich amount of information join the training set, does not need to detect the entire sample space, has the advantages of low computational complexity and fast learning speed. However, it also has certain problems, which are as follows.

[0005] (1) Weight of committee members

[0006] The most important step in the whole process of active learning is to select the most valuable samples for the whole model training to be manually labeled, and the quality of sample selection is directly related to the performance of the classifier. The committee voting based algorithm adopts the way of voting by each committee member to select samples, which defaults that the weights between multiple classifiers in the committee are equal, that is, each classifier has the same right when voting for the sample to be labeled. However, when multiple classifiers are trained at the same time, there will always be differences between the classifiers. If the results of these classifiers are measured by the same weight, some valuable samples for the classifier training may be missed, which will make the training efficiency low and will also affect the classification accuracy of the final classifier.

[0007] (2) Relative entropy and voting entropy

[0008] The commonly used method to measure the difference of voting results based on committee voting is mainly divided into two kinds, the first one is the relative entropy method proposed by McCallum and Nigam, which measures the difference between the voting results by calculating the relative entropy. The larger the value of relative entropy, the greater the difference between the voting results of the committee members on the sample label. This method has certain disadvantages, and it may miss some samples with inconsistent voting results among committee members, which will affect the training results. The second method is to use voting entropy to measure the difference of voting results proposed by Dagan et al. The larger the value of voting entropy, the greater the difference between the voting results of the committee members. This method also has certain problems, it does not consider the class conditional probability value of each committee member on the sample, which will lead to the loss of some samples with information quantity, and affect the training of the classifier.

[0009] (3) Similarity detection

[0010] The traditional committee member voting based method does not perform similarity detection on the samples to be labeled and the samples in the training set, resulting in high similarity between the selected samples to be labeled and the samples in the training set, which makes the selected sample set unable to fully exhibit the diversity characteristics of the data. High redundancy of training samples will lead to low training efficiency, slow training effect improvement and other problems, and for the training model that stops active learning when the number of training samples reaches the threshold, it will also affect the classification result accuracy of the model.

[0011] (4) Active learning stopping condition

[0012] The most important factor determining the performance of the active learning training classifier is whether the most valuable samples for training the classifier can be selected, but the stopping condition of active learning also has a certain influence on the performance of the classifier. Generally, when training is carried out by using active learning, a threshold value of the number of training set samples is set, and the active learning process is stopped when the number of samples in the training set reaches the threshold value. Because the training sample threshold value is set artificially, the effect of the classifier can be significantly improved if the training process of active learning is not stopped when the number of training set samples reaches the threshold value. Therefore, the conventional stopping condition of active learning may stop active learning at an inappropriate node, thereby affecting the final performance of the classifier. SUMMARY

[0013] The technical problem to be solved by the present application is to solve the above-mentioned problems of the prior art, and to provide a construction method of a classifier of an image labeling system based on active learning. The active learning method based on committee voting is used to train the classifier, so as to train the classifier with a small number of labeled samples and obtain a classifier with good performance.

[0014] To solve the above technical problems, the technical solution adopted by the present application is as follows: a construction method of a classifier of an image labeling system based on active learning, comprising the following contents:

[0015] Three classifiers are established according to existing class label data, and the three classifiers form a committee, which is used as a classifier model;

[0016] The committee members are iteratively trained based on an ensemble learning algorithm;

[0017] The committee classifier model after iterative training is used for classification, and the weights of the committee members and the weights of the training set samples are updated by using the classification results;

[0018] The classifier model is trained by using the active learning method based on committee voting, and a new training set is obtained;

[0019] The classifier model is trained by using the new training set.

[0020] The specific method for iteratively training the committee members based on the ensemble learning algorithm is as follows:

[0021] Step A1, in the initial state, the weights of the samples in the training set are equal, and the initial weights of the plurality of weak classifiers are the same;

[0022] Step A2, a certain amount of samples are randomly selected from the training set with sample weight distribution, and the first weak classifier is trained;

[0023] Step A3, label all samples in the training set using the trained first weak classifier, and calculate the classification error rate of the first weak classifier on all training set samples;

[0024] The classification error rate of the classifier on all training set samples is calculated by the following formula:

[0025]

[0026] wherein e m is the sum of the weights of all samples classified incorrectly by the classifier G m (x), that is, the classification error rate; n represents the number of samples, ω mi is the weight value corresponding to the sample x i , y i is the label corresponding to the sample x i , I(G m (x i )≠y i ): when G m (x i ) is not equal to y i , I takes the value of 1, and when G m (x i ) is equal to y i , I takes the value of 0.

[0027] Step A4, calculate the weight of the first weak classifier in the classifier model;

[0028] The weight of the classifier in the classifier model is shown in the following formula:

[0029]

[0030] wherein α m represents the maximum weight corresponding to the classifier G m (x).

[0031] Step A5, update the weight distribution of the samples in the training set during the current iteration training, for the next round of iteration training;

[0032] Suppose the mth weak classifier is being trained, then the weight vector of all samples in the training set of the m+1th classifier is:

[0033] D m+1 = (ω m+1,1 , ω m+1,2 , … ω m+1,n )

[0034] The weight of the ith sample in the training set of the m+1th classifier is shown in the following formula:

[0035]

[0036] wherein Z m is a normalization factor such that D m+1 becomes a probability distribution as shown in the following equation:

[0037]

[0038] Step A6, based on the samples misclassified by the first weak classifier, a certain number of samples are selected from the training set, and these samples are used to train the second weak classifier, so that the number of samples used to train the weak classifier is consistent each time;

[0039] Step A7, use the trained second weak classifier to label all samples in the training set, and execute steps A3-A6 to update the weight of the second weak classifier in the classifier model and the weight distribution of the samples in the training set;

[0040] Step A8, execute the content of step A6 for other weak classifiers in the committee members except the first weak classifier and the second weak classifier, until the classification error rates of the weak classifiers converge, and the final strong classifier model is obtained.

[0041] Preferably, the specific method of training the classifier based on the committee voting active learning method is:

[0042] Step B1, use each trained classifier in the committee to pre-label all samples in the training set, and iteratively train each classifier;

[0043] According to the comparison between the pre-labeling results of the classifier and the true results of the samples in the training set, the accuracy of the classifier is calculated. If the accuracy of all classifiers in the committee members has not changed or has changed by no more than a set threshold for consecutive times, the stopping condition of this active learning is reached, and the trained classifier is output. If not, continue training.

[0044] Step B2, the committee members pre-label the samples in the unlabeled sample set, and each unlabeled sample will obtain a classification label;

[0045] Step B3, each committee member selects samples whose confidence is lower than a threshold in the classification result; and the union of all samples whose confidence is lower than the threshold is obtained to obtain a sample set whose confidence is lower than the threshold;

[0046] Step B4, each classifier in the classifier model, i.e., each member in the committee, votes for these samples whose confidence is lower than the threshold; based on the voting result, samples whose voting entropy is higher than a threshold are selected;

[0047] Step B5, for the samples with voting entropy lower than the threshold, calculate the relative entropy thereof, select the samples according to the relative entropy threshold, and select the samples with relative entropy higher than the threshold;

[0048] Step B6, perform similarity test on the selected samples with relative entropy higher than the threshold: perform similarity test on the selected samples and the samples in the training set by using cosine similarity, and eliminate the samples with average similarity result higher than the threshold from the unlabeled sample set U;

[0049] Step B7, select u samples from the unlabeled sample set U, and manually label the u samples;

[0050] Step B8, update the labeled sample set L and the unlabeled sample set U, first inject the manually labeled samples into the training set to obtain a new training set, and then delete the u samples from the unlabeled sample set U.

[0051] The beneficial effects produced by the above technical solutions are that the method for constructing a classifier of an image labeling system based on active learning provided by the application has the following advantages:

[0052] (1) Assign weights to committee members by using an ensemble learning algorithm

[0053] Multiple classifiers are trained simultaneously, and differences in classification effects will inevitably occur. In order to give a higher weight to a classifier with good classification performance when voting samples, and at the same time, give a lower weight to a classifier with poor classification effect when voting, the Adaboost algorithm in ensemble learning is applied to the active learning process in the method, and the weight of the classifier is updated according to the accuracy of the classification of each classifier each time. The model based on committee voting, which is a multiple classifier simultaneous training model, can play a greater value, and the selected samples after each voting result are more valuable for the training of the entire model and are more conducive to the performance improvement of the model.

[0054] (2) Combine voting entropy and relative entropy

[0055] In order to improve the problem that some inconsistent voting samples are missed due to the use of relative entropy to measure the difference of voting results, and the problem that some samples with a large amount of information are missed due to the use of voting entropy to measure the difference of voting results, the method combines relative entropy and voting entropy to measure the difference of voting results. First, the voting entropy is judged, and the samples with voting entropy higher than the threshold are selected. Then, for the samples with voting entropy lower than the threshold, the relative entropy is calculated, and the samples with relative entropy higher than the threshold are further selected. This avoids missing inconsistent voting samples and samples with information value when selecting labeled samples.

[0056] (3) Cosine similarity is used to detect the similarity between samples

[0057] In order to improve the problem of high redundancy of selected training samples, the method of the present application detects the cosine similarity between the sample with large difference in voting result and all samples in the training set after the committee generates the voting result. If the cosine similarity between the sample to be labeled and the sample in the training set is higher than a certain threshold, the sample is excluded. The similarity between the unlabeled sample and all labeled samples is measured by the average of the cosine similarity between the unlabeled sample and all labeled samples, which avoids data redundancy and ensures the diversity of samples, and promotes the improvement of the efficiency and accuracy of the classifier.

[0058] (4) Set reasonable stopping conditions for active learning

[0059] In order to solve the problem of the influence of the threshold value of the training set sample on the accuracy of the classifier caused by active learning, the method of the present application proposes a more reasonable stopping condition for active learning. After each manual labeling is completed and the new labeled sample is updated to the training set, all classifiers in the committee are used to classify all samples in the training set. The classification result is compared with the true result of the sample, and the accuracy of classification is calculated. If the accuracy of all classifiers in the committee has not changed or improved slightly for many times, it is considered that the stopping condition of the present active learning is reached, and the training of the classifier can be stopped. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 The flowchart of the classifier construction method of the image labeling system based on active learning provided by the embodiment of the present application is shown in the figure.

[0061] Figure 2 The flowchart of the iterative training of the committee members based on the ensemble learning algorithm provided by the embodiment of the present application is shown in the figure.

[0062] Figure 3 The flowchart of the method of training the classifier to obtain a new training set based on the committee voting provided by the embodiment of the present application is shown in the figure.

[0063] Figure 4 The comparison chart of the three training methods provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0064] The specific embodiments of the present application will be further described in detail below in combination with the drawings and examples. The following examples are used to illustrate the present application, but not to limit the scope of the present application.

[0065] In this embodiment, a classifier construction method of an image annotation system based on active learning is implemented in two stages, i.e., a classifier training stage based on ensemble learning and a classifier training stage based on an improved committee voting method. In the classifier training stage based on ensemble learning, the weights of the classifiers are updated according to the accuracy of each classifier in each classification, so that the classifiers with good classification performance have higher weights in the next sample voting stage, and the classifiers with poor classification performance have lower weights in the voting stage, so that the voting result can play the maximum value. In the classifier training stage based on the improved committee voting method, the voting entropy and relative entropy are combined to measure the difference of the voting result, so as to avoid missing samples with inconsistent voting results and valuable samples when selecting labeled samples; and the mean cosine similarity is used to measure the similarity between the unlabeled sample and all labeled samples, so as to avoid data redundancy, ensure the diversity of the samples, and promote the improvement of the classifier efficiency and classification accuracy.

[0066] In this embodiment, the classifier training stage based on ensemble learning is used to complete the training of the committee member classifiers and the distribution of the weights of the classifiers, and the following functions are implemented:

[0067] (1) The training set is set to the weights distributed to each classifier as a training set;

[0068] (2) The error rate and weight coefficient obtained after the training of the previous classifier are used to update the weight of the training set of the next classifier, so as to solve the weight distribution problem of each member in the committee;

[0069] (3) The training set is updated according to whether the classification accuracy meets the condition, so as to prevent important samples from being lost.

[0070] (4) The multiple trained classifiers are integrated together according to the weight system;

[0071] The classifier training stage based on the improved committee voting method is used to detect the sample similarity and entropy to update the sample set, and the following functions are implemented:

[0072] When the accuracy of the classifiers in the committee does not meet the condition, the training sample set is updated to prevent important samples from being lost;

[0073] The voting entropy and relative entropy of the committee members are combined to avoid missing samples with inconsistent voting results and valuable samples when selecting labeled samples;

[0074] The similarity between the unlabeled sample and all the labeled samples is measured by the mean value of the cosine similarity between the unlabeled sample points and all the labeled sample points, the training set samples are updated, data redundancy is avoided, the diversity of the samples is ensured, and the efficiency and classification accuracy of the classifier are improved.

[0075] In the embodiment, a classifier construction method of an active learning-based labeling system is used for classifier construction when labeling medical images, as shown in the following formula (1), and specifically includes the following steps: Figure 1

[0076] Step 1. Three classifiers are established according to existing class label data, and the three classifiers form a committee as a classifier model;

[0077] Step 2. The committee members are iteratively trained based on an ensemble learning algorithm;

[0078] Step 3. The committee classifier model after iterative training is used for classification, and the weights of the committee members and the weights of the training set samples are updated using the classification results;

[0079] Step 4. The committee classifier model is trained based on an active learning method of committee voting to obtain a new training set;

[0080] Step 5. The new training set is used to train the classifier model.

[0081] The iterative training of the committee members based on the ensemble learning algorithm is as shown in the following formula (2), and the specific process is as follows: Figure 2

[0082] Step A1. Image samples are collected, and part of the images are labeled to obtain a labeled sample set L, and other unlabeled images form an unlabeled sample set U. The labeled sample set is used as a training set. In the initial state, the weights of the samples in the training set are equal, and the initial weights of the three weak classifiers G1, G2 and G3 are the same;

[0083] Step A2. A certain amount of samples are randomly selected from the training set with sample weight distribution, and the first weak classifier G1 is trained;

[0084] Step A3. The trained first weak classifier is used to label all the samples in the training set, and the classification error rate e1 of the first classifier on all the training set samples is calculated according to formula (1);

[0085]

[0086] wherein e m is the classification error rate of the classifier G m ​​(x) the sum of the weights of all samples classified wrongly, i.e. the classification error rate; n represents the number of samples, ω mi is the sample x i corresponding weight value, y i is the sample x i corresponding label; I(G m (x i ) ≠ y i ): when G m (x i ) is not equal to y i , I takes the value of 1, and when G m (x i ) is equal to y i , I takes the value of 0.

[0087] Step A4, calculate the weight α1 of the first weak classifier G1 in the classifier model according to formula (2);

[0088]

[0089] wherein, α m represents the maximum weight of the mth classifier G m (x) corresponding, when , α m ≥ 0, and α m increases with the decrease of e m , so as to ensure that the basic classifier with higher classification accuracy plays a greater role in the final classifier.

[0090] Step A5, update the weight distribution of the samples in the training set according to formula (3) for the next round of iteration;

[0091] Suppose the mth weak classifier is being trained, then the weight vector of all samples in the training set of the m+1th weak classifier is:

[0092] D m+1 = (ω m+1,1 , ω m+1,2 , … ω m+1,n )

[0093] The weight of a single sample in the training set of the m+1th classifier is calculated as follows

[0094]

[0095] wherein, Z m is a normalization factor, so that D m+1 becomes a probability distribution, as shown in the following formula:

[0096]

[0097] Step A6, based on the samples misclassified by the first weak classifier, a certain number of samples are selected from the training set, and these samples are trained on the second classifier G2, so that the number of samples trained on the weak classifier is consistent each time;

[0098] Step A7, use the trained classifier G2 to label all samples in the training set, and calculate the classification error rate e2 of the classifier G2 on all training samples according to formula (1);

[0099] Step A8, calculate the weight α2 of the second classifier G2(x) in the classifier model according to formula (2);

[0100] Step A9, update the weight distribution of the samples in the training set according to formula (3) for the next iteration;

[0101] Step A10, based on the samples misclassified by the first two weak classifiers, a certain number of samples are selected from the training set, and these samples are trained on the third weak classifier G3;

[0102] Step A11, use the trained classifier G3 to label all samples in the training set, and calculate the classification error rate e3 of G3 on all training samples according to formula (1);

[0103] Step A12, calculate the weight α3 of the third classifier G3(x) in the final classifier according to formula (2);

[0104] Step A13, update the weight distribution of the samples in the training set according to formula (3) for the next iteration;

[0105] Step A14, after executing step B to update the training, repeat steps A2-A13 until the classification error rates e m of the three classifiers converge, and the final strong classifier model f(x) is obtained, as shown in the following formula:

[0106]

[0107] In this embodiment, the active learning method based on committee voting (Query-by-committee, i.e. QBC) is used to train the classifier, and a new training set is obtained, as shown in Figure 3 The specific process is as follows:

[0108] Step B1, use each trained classifier in the committee to pre-label all samples in the training set, and train each classifier in the committee;

[0109] According to the pre-labeled results of all samples in the training set and the true results of the samples, the accuracy of the classifier is calculated, and if the accuracy of all classifiers in the committee has not changed or changed by less than a set threshold (slight improvement) for a plurality of times (five times or more), the stopping condition of this active learning is reached, and the trained classifier is output; if not, the training continues;

[0110] Step B2, committee members pre-label the samples in the unlabeled sample set, and each unlabeled sample obtains a classification label;

[0111] Step B3, each committee member selects samples whose confidence is lower than the threshold in the classification results; the union of all samples whose confidence is lower than the threshold is obtained to obtain the sample set whose confidence is lower than the threshold;

[0112] Step B4, each classifier in the classifier model, i.e., each member in the committee, votes for the samples whose confidence is lower than the threshold; based on the voting results, the voting entropy is calculated according to formula (5), and samples whose voting entropy is higher than the threshold are selected;

[0113]

[0114] Wherein, K is the number of committee members (i.e., the number of classifiers constituting the committee), C is the number of classes, and V(c, x i ) is the voting number of the sample x i belonging to the class c by each member in the committee. i

[0115] Step B5, for samples whose voting entropy is lower than the threshold, the relative entropy thereof is calculated according to formula (6), and the samples are selected according to the relative entropy threshold, and samples whose relative entropy is higher than the threshold are selected;

[0116]

[0117]

[0118]

[0119] Wherein, K is the number of committee members (i.e., the number of classifiers constituting the committee), P m (c|x i ) is the probability that the sample x i is classified as the class c by the mth member in the committee, and P avg (C|x i ) is the average value of the class conditional probability of the sample x i by all members in the committee. ​

[0120] Step B6, similarity test is performed on the samples with relative entropy higher than the threshold value: the cosine similarity is used to calculate the similarity between the screened samples and the samples in the training set, and the average similarity of the samples is calculated, and the samples with average similarity higher than the threshold value are deleted from the unlabeled sample set U;

[0121] In this embodiment, the similarity of two samples and the average similarity of samples are calculated by the following formula:

[0122]

[0123]

[0124] wherein a and b represent two samples (i.e. the screened sample and the sample in the training set) for similarity test, S(a, b) represents the cosine similarity of the two samples, S avg represents the average value of the cosine similarity of the samples with a total number of n;

[0125] Step B7, select u samples from the unlabeled sample set U, and manually label the u samples;

[0126] Step B8, update the labeled sample set L and the unlabeled sample set U, first inject the manually labeled samples into the training set to obtain a new training set, and then delete the u samples from the unlabeled sample set U.

[0127] In this embodiment, the classifier of the image labeling system is constructed based on laboratory equipment, and the classifier is used to label and classify diabetic retinopathy image data, and a PC machine with windows 10 system is used to realize it. The CPU used in the experiment is Intel Core i77700K processor, which has 4 cores and 8 threads, a basic frequency of 4.5 GHz and a dynamic acceleration frequency of 4.5 GHz. The GPU is NVIDIA GTX 1082Ti, which has 3584 CUDA cores, 11GB of video memory and 352bit of video memory bit width. The PC machine is also equipped with 16GB of memory and 1T of hard disk.

[0128] The data set used in this embodiment comes from the labeled fundus image in Kaggle competition data set. Kaggle mainly faces developers and data scientists, and provides them with a platform for holding machine learning competitions, and Kaggle contains rich competition data, which provides a good research basis for researchers.

[0129] The Kaggle Diabetic Retinopathy Detection database is a public database with many color fundus images. These images were collected from several primary care centers in California and elsewhere, which used a variety of digital cameras. The definition ranges from 433 x 289 pixels to 5184 x 3456 pixels, all files are in jpeg format, and these images are also uploaded to the free DR Scan platform.

[0130] At present, the traditional SVM training method needs to pay a great cost to obtain a better performance classifier; and the committee voting method for training SVM has low computational complexity, but also has the problems of committee member weight distribution, missing samples, and learning condition when to stop appropriately. The method improves the traditional committee voting method based on SVM training method, retains its advantage in training cost compared with the traditional SVM training method, and solves the problems faced by the traditional committee method. Therefore, the embodiment carries out a set of comparative experiments, respectively using the traditional SVM training method, the traditional committee voting method for training SVM, and the improved committee voting method for training SVM, using the same set of data to experiment on the three methods, and the experimental results are as shown in Figure 4 Figure 4 It can be seen from the experimental results that the improved committee voting method has good performance in accuracy, sensitivity, and specificity compared with the other two methods, among which the traditional SVM training method is superior to the traditional committee voting method in sensitivity, and the traditional committee voting method is superior to the traditional SVM training method in accuracy and specificity. The improved committee voting method not only has improved accuracy and specificity compared with the traditional committee voting method, but also approaches the traditional SVM training method in sensitivity, but its cost is much lower than that of the traditional SVM training method.

[0131] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the claims of the present application.​

Claims

1. A method for constructing a classifier of an active learning based labeling system, characterized in that: image samples are collected, part of the images are labeled to obtain a labeled sample set, and other unlabeled images constitute an unlabeled sample set; the labeled sample set is used as a training set; a plurality of weak classifiers are established according to existing class label data, a committee is formed by the plurality of weak classifiers, and the committee is used as a classifier model; committee members are iteratively trained based on an ensemble learning algorithm; the committee members after iterative training are used for classification, and the weights of the committee members and the weights of the samples in the training set are updated using the classification results; the classifier model is trained based on an active learning method of committee voting to obtain a new training set; the classifier model is trained using the new training set; and the specific method for training the classifier model based on the active learning method of committee voting to obtain the new training set is: step B1, each classifier in the committee is used to pre-label all samples in the training set, and each classifier is iteratively trained; step B2, the committee members pre-label samples in the unlabeled sample set, and each unlabeled sample obtains a classification label; step B3, each committee member selects samples with a confidence lower than a threshold value in the classification results; a set of samples with a confidence lower than the threshold value is obtained by taking the union of all samples with a confidence lower than the threshold value; step B4, each classifier in the classifier model, i.e., each member in the committee, votes for the samples with a confidence lower than the threshold value; based on the voting results, samples with a voting entropy higher than a threshold value are selected; step B5, for samples with a voting entropy lower than the threshold value, a relative entropy is calculated, samples with a relative entropy higher than a threshold value are selected according to the relative entropy threshold value; step B6, similarity inspection is performed on the selected samples with a relative entropy higher than the threshold value: cosine similarity is used to perform similarity inspection on the selected samples and samples in the training set, and samples with an average similarity result higher than a threshold value are deleted from the unlabeled sample set U; step B7, u samples are selected from the unlabeled sample set U, and the u samples are manually labeled; step B8, the labeled sample set L and the unlabeled sample set U are updated: the manually labeled samples are injected into the training set to obtain a new training set, and the u samples are deleted from the unlabeled sample set U. The specific method for iteratively training the committee members based on the ensemble learning algorithm is: step A1, in an initial state, the weights of the samples in the training set are equal, and the initial weights of the plurality of weak classifiers are the same; step A2, a certain amount of samples are randomly selected from the training set with sample weight distribution, and a first weak classifier is trained; step A3, the first weak classifier after training is used to label all samples in the training set, and the classification error rate of the first weak classifier on all training set samples is calculated; step A4, the weight of the first weak classifier in the classifier model is calculated; step A5, the sample weight distribution in the training set is updated for the next round of iterative training. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 2.The method of claim 1, wherein: ​ ​ ​ ​ ​ ​ Step A6, based on the samples misclassified by the first weak classifier, a certain number of samples are selected from the training set, and these samples are trained for the second weak classifier, so that the number of samples trained for the weak classifier is consistent each time; Step A7, use the trained second weak classifier to label all samples in the training set, and execute steps A3-A6 to update the weight of the second weak classifier in the classifier model and the weight distribution of the samples in the training set; Step A8, execute the content of step A6 for other weak classifiers in the committee member except the first weak classifier and the second weak classifier, until the classification error rates of the weak classifiers converge, and the final strong classifier model is obtained.

3. The method of claim 2, wherein the method further comprises: determining a label of the new data based on the classifier; and updating the classifier based on the label of the new data. The classification error rate of the classifier on all training set samples is calculated by the following formula: ; wherein, is the classifier the sum of the weights of all samples classified incorrectly, i.e. the classification error rate; n denotes the number of samples, is a sample the corresponding weight value, is a sample the corresponding label; : when is not equal to , I takes the value 1 when is equal to , I takes the value 0.

4. The method of claim 2, wherein the method further comprises: The weight of the classifier in the classifier model is shown in the following formula: ； wherein, representing the classifier corresponding maximum weight.

5. The method of claim 2, wherein the method further comprises: determining a label of the new data based on the classifier; and updating the classifier based on the new data and the label. The specific method of step A5 is: Suppose the current weak classifier being trained is the mth weak classifier, then the weight vector of all samples in the training set of the m+1th weak classifier is: ； The training set of the (m+1)th classifier is the... i The weights of each sample are shown in the following formula: ； wherein, is a normalization factor such that becomes a probability distribution as shown in the following equation: 。 6. The method of claim 5, wherein the method further comprises: The specific method of step B1 is: According to the comparison between the pre-labeled results of the classifier and the true results of the samples in the training set, the accuracy of the classifier is calculated, if the accuracy of all classifiers in the committee member has not changed or the change has not exceeded the set threshold for several times in succession, the stopping condition of this active learning is reached, and the trained classifier is output; if not, the training continues.

Images