An image labeling method based on semi-supervised learning

Abstract

The application provides an image labeling method based on semi-supervised learning, and relates to the technical fields of machine learning and image processing. The method can be considered as being realized based on two stages, i.e. a semi-supervised learning stage based on collaborative operation and a semi-supervised learning stage based on TSVM. The semi-supervised learning stage based on collaborative operation is used for completing the labeling task of the easy-to-label samples which account for the vast majority in the unlabeled sample set, controlling the number of samples labeled each time, and also well controlling the number of samples in the training set; the burden of the SVM classifier is reduced, and the classification accuracy is ensured. The semi-supervised learning stage based on TSVM is used for completing the labeling task of the remaining unlabeled samples, and the labeled samples which have been labeled by the semi-supervised learning stage based on collaborative operation and the remaining unlabeled samples are all used in the training, and the weight of the classifier and the weight of the training set samples are trained by integrating the method of ensemble learning in the training process.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and image processing technology, and in particular to an image annotation method based on semi-supervised learning. Background Technology

[0002] In recent years, diabetes has shown a significant trend of affecting younger people. Lack of exercise, irregular lifestyle, and being overweight can all be contributing factors. Diabetes is a major cause of blindness, specifically diabetic retinopathy.

[0003] With the increasing number of diabetic patients each year, the number of fundus images generated is also increasing, placing a heavier burden on doctors. After diabetic patients seek medical attention, medical equipment collects a wealth of information about their condition, enriching the retinal image database. However, doctors have limited time and energy. Manually annotating every medical image would be extremely resource-intensive. Therefore, it is crucial to utilize these valuable resources effectively and efficiently to extract valuable information. The rapid development of machine learning and the close connection between computer-aided medical imaging and modern medicine have greatly promoted its advancement. Machine learning methods can be used to process and annotate medical images, freeing doctors from arduous annotation tasks to focus on more important work. Furthermore, it can provide researchers and medical professionals in retinal imaging with referential annotation samples.

[0004] Current research on annotation systems mainly divides them into two types: interactive annotation systems and non-interactive annotation systems. Non-interactive annotation systems refer to systems where no human intervention is required throughout the annotation process; the system completes the annotation entirely on its own. Interactive annotation systems involve human involvement, but the majority of the annotation work is still handled by the system, with manual annotation accounting for only a small portion.

[0005] Most interactive annotation systems are based on active learning algorithms for model training. Active learning can achieve a high-performance classifier by training the model with only a small number of labeled samples. The specific steps are: first, train the model using labeled data; then, use the trained learner to identify the unlabeled data that offers the greatest performance improvement and consult "experts" for manual annotation. The annotated samples are then added to the labeled dataset, and the classifier is trained again using this new set of labeled samples. This iterative training continues until the active learning stops, and the trained classifier is output. However, active learning relies heavily on the labeled dataset, neglecting the unlabeled dataset and failing to grasp the overall distribution of the samples. In other words, while utilizing labeled samples, active learning only leverages a small number of unlabeled samples—the vast majority of which are difficult to annotate—that offer the greatest performance improvement. The remaining large number of unlabeled samples are not fully utilized. Therefore, semi-supervised learning methods that can further utilize a large number of unlabeled samples have gained more attention.

[0006] Semi-supervised learning is a learning method that combines supervised and unsupervised learning. It primarily considers how to train and classify using a small number of labeled samples and a large number of unlabeled samples. Semi-supervised learning can be further subdivided into pure semi-supervised learning and transductive learning. The former assumes that the unlabeled data in the training data is not the data to be predicted, while the latter assumes that the labeled samples considered during the learning process are precisely the data to be predicted. To utilize unlabeled samples, semi-supervised learning must make assumptions that link the data distribution information revealed by the unlabeled samples with the class labels; its essence is that "similar samples have similar outputs." Support Vector Machine (SVM) classifiers, due to their excellent generalization ability, achieve very accurate classification results with small samples, making them one of the best classifiers. However, they also have some drawbacks. When the amount of data is large, they become time-consuming and inefficient. Furthermore, excessively large amounts of data can also negatively impact the classification accuracy of SVM classifiers. TSVM is the most well-known representative of semi-supervised support vector machines. Like the standard SVM, it is an algorithm for binary classification. Its core idea is to try to find suitable label assignments for unlabeled samples to maximize the margin after the hyperplane partitions. TSVM uses a local search strategy for iterative solution. First, an initial SVM is trained using a labeled sample set. Then, this learner is used to label the unlabeled samples, so that all samples are labeled. The SVM is then retrained based on these labeled samples, and then error-prone samples are searched for for continuous adjustment. Finally, a classification hyperplane that maximizes the margin is found between the labeled samples and the pre-labeled unlabeled samples. Once this classification hyperplane is found, the samples at both ends of this hyperplane receive the final labeling results.

[0007] Traditional methods for training SVMs based on semi-supervised learning involve adding unlabeled samples to the training process all at once. Before generating the final labeled samples, an iterative process of finding the SVM hyperplane is performed, pre-labeling all samples in the unlabeled samples each time. Because there are many unlabeled samples, finding the classifier hyperplane each time becomes computationally complex, making each pre-labeling of unlabeled samples very time-consuming. Simultaneously, solving the SVM problem on these samples also requires significant space resources. However, if this cost yields satisfactory results, it is worthwhile. But due to the inherent characteristics of SVM classifiers, their performance is often unsatisfactory when faced with a large number of samples. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing an image annotation method based on semi-supervised learning to achieve automatic annotation of medical images.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: an image annotation method based on semi-supervised learning, comprising the following steps:

[0010] Step 1: Collect medical images and label some of them to obtain the labeled sample set L. The other unlabeled images form the unlabeled sample set U. At the same time, construct three SVM classifiers G1, G2 and G3 and a buffer pool U′.

[0011] Step 2: Divide the labeled sample set L into three equally labeled sample subsets h1, h2, and h3. The labeled sample subsets h1, h2, and h3 will be used to train the three classifiers G1, G2, and G3, respectively. Then, randomly select u samples from the unlabeled sample set U and put them into the buffer pool U′.

[0012] Step 3: Train classifier G1 using the labeled sample subset h1 to obtain the updated classifier G1′, and update the sample subset h1 by removing duplicates; use the updated classifier G1′ to pre-label the unlabeled samples in the buffer pool U′ and calculate the confidence score. Add samples with a confidence score greater than a set threshold to the other two labeled sample subsets h2 and h3. At the same time, select samples from the unlabeled sample set U to replenish the buffer pool with unlabeled samples; then train and update classifiers G2 and G3 and sample subsets h2 and h3 in the same way as classifier G1 and sample subset h1 to obtain the updated classifiers G2′ and G3′, and summarize the updated three sample subsets.

[0013] Step 3.1: Train classifier G1 using the labeled sample subset h1 to obtain the updated classifier G1′;

[0014] Step 3.2: Determine whether the number of samples in the labeled sample subset h1 has reached the threshold of the labeled training set samples. If the threshold is reached, the samples in h1 are labeled by classifier G1′. Samples with confidence greater than the set threshold are removed from h1 according to the ratio of positive samples to negative samples. The samples from the unlabeled sample set U among the removed samples are added to set L′.

[0015] Step 3.3: Use classifier G1′ to pre-label the samples in buffer pool U′. After labeling, select p positive samples with the highest confidence and q negative samples with the highest confidence and add these samples to h2 and h3. At the same time, draw samples from the unlabeled sample set U to supplement buffer pool U′. Then, perform deduplication on the samples in sample subsets h2 and h3.

[0016] Step 3.4: Use sample subsets h2 and h3 to perform the corresponding operations on classifiers G2 and G3 according to steps 3.1 to 3.3 respectively, and obtain classifiers G2′ and G3′ and deduplicated updated sample subsets h2 and h3; finally, summarize the deduplicated sample subsets h1, h2 and h3.

[0017] Step 4: Iterate through Step 3 until there are no remaining samples in the unlabeled sample set U, and obtain the classifiers G1′_m, G2′_m and G3′_m after m rounds of training;

[0018] Step 5: Combine the samples from all deduplicated labeled sample subsets h1, h2, and h3 into a single labeled sample set L. t In this section, the parameters of the TSVM classifier are initialized, and the unlabeled samples in the buffer pool U′ and L are initialized. t Each labeled sample in the dataset is assigned an influence factor, and slack variables are set for both labeled and unlabeled samples. The classifier G1′_m, obtained after m rounds of training in step 4, is used to evaluate L. t All samples are pre-labeled, the weights of classifier G1′_m are updated based on the pre-labeling results, and then samples that are mislabeled by classifier G1′_m are added to the training sample set of the next classifier G2′_m.

[0019] Step 5.1: Combine the labeled sample sets h1, h2, and h3 after multiple rounds of training into a new labeled sample set L. t Then, the parameters in the TSVM classifier are initialized, including the unlabeled samples in the buffer pool U′ and the aggregated labeled sample set L. t The labeled samples in the buffer pool U′ are configured with influence factors and slack variables to adjust the influence factor C of the unlabeled sample set. u Less than L t The influence factor C of the labeled sample set l One-thousandth;

[0020] Step 5.2: Determine C u Is it less than C? l If C is satisfied u Less than C l Then use the labeled sample set L t The classifier G1′_m is continuously trained; otherwise, the classifier G1′_m is used to label the unlabeled samples in the buffer pool U′.

[0021] Step 5.2.1: From the labeled sample set L t Select a certain number of samples to form a new labeled sample set D l Train classifier G1 to obtain the updated classifier G1″;

[0022] Step 5.2.2: Use the updated classifier G1″ to pre-label the unlabeled samples in the buffer pool U′, obtaining the label corresponding to each unlabeled sample, thus obtaining the labeled sample set D in the buffer pool U′. u ;

[0023] Step 5.2.3: Influence factor C based on the unlabeled sample set u The influence factor C of the labeled sample set l The set of labeled samples D in the buffer pool u The labeled sample set D l The predicted labels of unlabeled samples in the buffer pool are used to solve the optimization objective of the TSVM objective function, and the parameters (w,b) and relaxation factor ξ of the classification hyperplane are obtained.

[0024] The objective function of the TSVM algorithm is:

[0025]

[0026] The constraints are:

[0027]

[0028] Where l represents the number of labeled samples, u represents the number of unlabeled samples, and i and j represent the sample indices; z i For the labeled sample x i Tag, y * j For unlabeled sample x j Predicted label, ζ i For the labeled sample x i The relaxation factor, ξ j For unlabeled sample x j The relaxation factor;

[0029] Step 5.2.4: Predict the labeled label y using the relaxation factor ξ obtained in 5.2.3 and the samples in the unlabeled sample set. * Determine the relationship between two unlabeled samples x. i and x j Does it meet the conditions? For x j If the predicted labeling results meet the conditions, then swap samples x. i and x j Predict the labeled tags, then re-execute 5.2.3. If the conditions are not met, update C. u =min(2C) u C l );

[0030] Step 5.3: Iteratively execute step 5.2 until C. u <C l The condition is checked until the updated classifier G1″ is obtained, and the labeling results of the unlabeled samples in the buffer pool are obtained. Then, G1″ is used to label the already labeled sample set L. t All samples in the dataset are pre-labeled. Based on the labeling results, the weights of classifier G1″ are recalculated. Then, classifier G1″ is applied to the labeled sample set L. t Samples that are incorrectly labeled are passed to the next classifier;

[0031] Step 5.3.1: Iterate through step 5.2 until C. u <C l Determine whether the condition is met;

[0032] Step 5.3.2: Obtain the labeling results of classifier G1″ for the unlabeled samples, and use classifier G1″ to label the labeled sample set L. t All samples are pre-labeled, and the weights of classifier G1″ are recalculated based on the labeling results.

[0033] Step 5.3.3: Apply classifier G1″ to the labeled sample set L t Samples that are incorrectly labeled are passed to the next classifier and used as part of the training sample set for the next classifier.

[0034] Step 6: Repeat the training process of Step 5 for classifiers G2′_m and G3′_m obtained after training in Step 4, to obtain trained classifiers G2″ and G3″. After the training of the three classifiers is completed, obtain the labeling results of all unlabeled samples in the buffer pool U′ and the weights of classifiers G1″, G2″ and G3″. Then, add the labeling results of the three classifiers for all unlabeled samples in the buffer pool U′ according to the weight ratio of the classifiers to obtain the final labeling results of all unlabeled samples in the buffer pool U′.

[0035] This invention employs an active learning algorithm based on committee voting and ensemble learning to train three classifiers, thus achieving an improved labeling method based on semi-supervised learning. The main steps in training the three classifiers are as follows: First, the committee members are iteratively trained using the ensemble learning algorithm. The weights of the committee members and the weights of the training set samples are updated based on the classification results of the training samples after training. Simultaneously, the classification result of the previous committee member on the training set samples affects the training of the next committee member. After the training of the three committee members is completed, it is determined whether the active learning stopping condition has been met. If it has, the ensemble classifier with the latest weights is output; otherwise, the active learning training process continues, selecting samples to be labeled for manual annotation, and then performing active learning again with a new training set. In summary, this process can be divided into two parts: training an SVM classifier based on ensemble learning and training an SVM classifier based on active learning.

[0036] Since the annotation system requires a large amount of labeled data, directly annotating all unlabeled samples during training would put a lot of pressure on the SVM classifier and severely affect the classification results. Therefore, it is necessary to refer to the idea of ​​reducing the sample set to avoid putting all samples into training at once.

[0037] The beneficial effects of adopting the above technical solution are as follows: The image annotation method based on semi-supervised learning provided by this invention, in the result annotation stage, first trains the SVM classifier using a semi-supervised learning method based on collaborative training to complete the annotation of most of the easily annotated samples; for the remaining small portion of more difficult-to-annotate samples, a TSVM training method is used to continuously optimize the SVM solution, finally obtaining the final annotation result. The final experimental results show that the method of this invention, combining the collaborative training method and the TSVM method, performs better in terms of accuracy, sensitivity, and specificity than using either of these two algorithms alone. Attached Figure Description

[0038] Figure 1 A flowchart of an image annotation method based on semi-supervised learning provided in an embodiment of the present invention;

[0039] Figure 2 A flowchart of the algorithm for the semi-supervised learning stage based on cooperative computation provided in an embodiment of the present invention;

[0040] Figure 3 This is an algorithm flowchart for the semi-supervised learning stage based on TSVM provided in an embodiment of the present invention;

[0041] Figure 4 This is a comparison chart of annotation results based on three different semi-supervised learning algorithms provided in the embodiments of the present invention. Detailed Implementation

[0042] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0043] In this embodiment, an image annotation method based on semi-supervised learning is described, such as... Figure 1 As shown, this can be viewed as a two-stage process: a semi-supervised learning stage based on collaborative computation and a semi-supervised learning stage based on TSVM. The semi-supervised learning stage based on collaborative computation is used to label the majority of easily labeled samples in the unlabeled sample set, effectively controlling the number of labeled samples in each training iteration and also carefully managing the number of samples in the training set. This effectively reduces the burden on the SVM classifier and ensures classification accuracy. The semi-supervised learning stage based on TSVM is used to label the remaining unlabeled samples. It utilizes both the labeled samples already labeled in the semi-supervised learning stage based on collaborative computation and the remaining unlabeled samples in the training, incorporating ensemble learning methods to train the classifier weights and the weights of the training set samples.

[0044] In this implementation, the semi-supervised learning stage based on collaborative computation is used to complete the labeling task for the majority of easily labeled samples in the unlabeled sample set. The algorithm flow is as follows: Figure 2 As shown, the following functions are implemented:

[0045] (1) Use the samples with high confidence in each classifier as the training set for the other classifiers;

[0046] (2) After adding samples with high confidence to the training set, remove duplicate samples from the training set to complete sample deduplication and ensure the diversity of training samples.

[0047] (3) Set a threshold for training set samples. If the number of samples in the training set exceeds the threshold, the classifier will pre-label the samples and select a certain number of samples with the highest confidence to complete the sample reduction.

[0048] (4) Complete the labeling of the majority of easy-to-label samples in the unlabeled sample set.

[0049] The semi-supervised learning stage based on TSVM is used to complete the labeling task of the remaining unlabeled samples. Its algorithm flow is as follows: Figure 3 As shown, the following functions are implemented:

[0050] The weights of each classifier are recalculated based on their labeled performance in the TSVM-based semi-supervised learning phase.

[0051] Set separate influence factors for labeled and unlabeled samples, and continuously improve the influence factor of unlabeled samples through model training until it is equal to that of labeled samples;

[0052] By using ensemble learning, the final labeling results of the samples in buffer pool U' are calculated based on the updated weights of each classifier and the labeling results of the unlabeled samples in buffer pool U'.

[0053] The improvement of the cooperative training and learning algorithm in the semi-supervised learning stage based on cooperative computation of this invention lies in:

[0054] (1) Number of classifiers

[0055] Existing semi-supervised learning methods based on co-training for SVM classifiers employ a two-classifier approach, training them with different samples. The annotation method of this invention uses three classifiers for co-training. Since there are three classifiers, during co-training, unlabeled samples with high confidence from one classifier are labeled and then added to the training sets of the other two classifiers. This requires that each classifier be as different as possible. To ensure the diversity of the training set samples for each classifier, this embodiment divides the labeled sample set into three equal parts, training each classifier separately, resulting in three classifiers with certain differences.

[0056] (2) Methods of collaborative training among classifiers

[0057] If there are too many unlabeled samples, the computational load will be extremely high. Therefore, this embodiment uses an unlabeled sample buffer pool method. A certain number of samples are randomly selected from the unlabeled sample set U and added to the buffer pool U'. When the classifier performs pre-labeling, it only needs to pre-label the samples in the buffer pool U', without needing to consider the other samples in the unlabeled sample set U, thus greatly reducing the computational complexity. Furthermore, the size of the buffer pool U' is fixed. When a sample with high confidence in the buffer pool U' is selected as a training set sample for the classifier, a certain number of samples are randomly selected from U and added to the buffer pool U'. In this way, high-confidence samples are continuously selected, and samples from U are continuously added to the buffer pool U', resulting in fewer and fewer samples in U. When there are no samples in U, all unlabeled samples are in the buffer pool U'. This indicates that the labeling task for most easily labeled samples has been completed, and the samples in the current buffer pool U' are samples that are difficult for the classifier to label. These samples are then handed over to the semi-supervised learning stage based on TSVM for more thorough and detailed training before generating the corresponding labeling results.

[0058] Existing methods for collaborative training of two classifiers involve providing high-confidence samples from each classifier to the other's training set. For three classifiers, high-confidence samples from one classifier are added to the training sets of the other two. However, this process can lead to duplicate samples in the training sets of different classifiers. A single sample might have high confidence in one or two classifiers, or even all classifiers. These duplicate samples cause classifiers to repeatedly add them to each other's training sets, increasing redundancy and slowing down system performance improvement while consuming significant time. To avoid this problem, this invention incorporates a sample deduplication operation at this stage. After adding samples to the training set, duplicate samples are removed, ensuring the diversity of the training samples.

[0059] (3) Processing of samples in the training set

[0060] Existing methods simply add training samples to the training sets of each classifier. Only an appropriate number of training samples can positively impact model training performance. When the number of training samples is very large, it affects classifier efficiency and labeling results. Considering the characteristics of SVM classifiers, this invention proposes a method for timely cleaning of samples in the training set: A threshold for the number of training set samples is set. After training the classifier, it is checked whether the number of samples in the training set has reached the threshold. If the number of samples has reached the threshold, the classifier pre-labels the samples, selecting a certain number of samples with the highest confidence and removing them from the training set. After removing these samples from the training set, their origin is determined. If the sample originally belonged to the unlabeled sample set U, the sample and its labeling result are placed in L'; L' is used to store the labeling results of samples in U.

[0061] The improvement of the semi-supervised learning stage based on TSVM over existing TSVM-based annotation algorithms lies in:

[0062] (1) Training sample set of TSVM

[0063] TSVM utilizes both labeled and unlabeled samples in its training, thus assigning slack variables to both and using an influence factor as a coefficient to measure their importance in the model. Initially, unlabeled samples typically have little reference value. By continuously identifying outliers between two pre-labeled samples and swapping their labels, the value of unlabeled samples gradually increases as the pre-labeling results are adjusted. This invention supplements the sample set for each TSVM classifier by directly providing samples misclassified by the previous classifier as training samples for the next classifier. This ensures that the samples in the training set maximize their value.

[0064] (2) Processing of classifier labeling results

[0065] After the semi-supervised learning phase based on collaborative computation, the classification accuracy of the three classifiers has changed, and the weights of the classifiers should be updated accordingly. Therefore, it is necessary to utilize the idea of ​​ensemble learning to recalculate their weights based on the performance of each classifier in this phase. After each classifier generates its corresponding labeling results, it pre-labels all samples in the training set. The weights of the classifiers are then calculated based on the labeling results, resulting in the weights of the three classifiers. Finally, the labeling results are calculated based on the labeling results of the three classifiers for each unlabeled sample.

[0066] In this embodiment, the image annotation method based on semi-supervised learning specifically includes the following steps:

[0067] Step 1: Collect medical images and label some of them to obtain the labeled sample set L. The other unlabeled images form the unlabeled sample set U. At the same time, construct three SVM classifiers G1, G2 and G3 and a buffer pool U′.

[0068] Step 2: Divide the labeled sample set L into three equally labeled sample subsets h1, h2, and h3. The labeled sample subsets h1, h2, and h3 will be used to train the three classifiers G1, G2, and G3, respectively. Then, randomly select u samples from the unlabeled sample set U and put them into the buffer pool U′ to lay the foundation for the subsequent training process.

[0069] Step 3: Train classifier G1 using the labeled sample subset h1 to obtain the updated classifier G1′, and update the sample subset h1 by removing duplicates; use the updated classifier G1′ to pre-label the unlabeled samples in the buffer pool U′ and calculate the confidence score. Add samples with a confidence score greater than a set threshold to the other two labeled sample subsets h2 and h3 (if classifier G1 is used, add h2 and h3; if G2 is used, add h1 and h3; the same applies to G3). At the same time, select samples from the unlabeled sample set U to replenish the buffer pool with unlabeled samples; then train and update classifiers G2 and G3 and sample subsets h2 and h3 using the same process as classifier G1 and sample subset h1 to obtain the updated classifiers G2′ and G3′, and summarize the updated three sample subsets.

[0070] Step 3.1: Train classifier G1 using the labeled sample subset h1 to obtain the updated classifier G1′;

[0071] Step 3.2: Determine whether the number of samples in the labeled sample subset h1 has reached the threshold of the labeled training set samples. If the threshold is reached, the samples in h1 are labeled by classifier G1′. Samples with confidence greater than the set threshold are removed from h1 according to the ratio of positive samples to negative samples. The samples from the unlabeled sample set U among the removed samples are added to set L′.

[0072] The distribution of positive and negative samples in the data is not uniform; it follows a certain ratio. Most retinal images are healthy, with only a small percentage showing retinal abnormalities. Therefore, when selecting samples with high confidence levels as training set samples for other classifiers, the characteristics of the sample distribution need to be considered, and samples should be selected according to a certain ratio.

[0073] Step 3.3: Pre-label the samples in the buffer pool U′ using classifier G1′. After labeling, select p positive samples with the highest confidence and q negative samples with the highest confidence, and add these samples to h2 and h3 (both sample subsets are added). At the same time, draw samples from the unlabeled sample set U to replenish the buffer pool U′. Then, perform a deduplication operation on the samples in the sample subsets h2 and h3.

[0074] Step 3.4: Similar to the steps of operating classifier G1 using h1, use sample subsets h2 and h3 to perform the corresponding operations on classifiers G2 and G3 according to steps 3.1 to 3.3 respectively, and obtain classifiers G2′ and G3′ and deduplicated updated sample subsets h2 and h3; finally, summarize the deduplicated sample subsets h1, h2 and h3.

[0075] Step 4: Iterate through Step 3 until there are no remaining samples in the unlabeled sample set U, and obtain the classifiers G1′_m, G2′_m and G3′_m after m rounds of training; at this point, only the samples in the buffer pool U′ are unlabeled, indicating that the work of the semi-supervised learning stage based on collaborative operation has been completed, and the unlabeled samples in the buffer pool U′ are processed by the semi-supervised learning module based on TSVM.

[0076] Step 5: Combine the samples from all deduplicated labeled sample subsets h1, h2, and h3 into a single labeled sample set L. t In this section, the parameters of the TSVM classifier are initialized, and the unlabeled samples in the buffer pool U′ and L are initialized. t Each labeled sample in the dataset is assigned an influence factor, and slack variables are set for both labeled and unlabeled samples. The classifier G1′_m, obtained after m rounds of training in step 4, is used to evaluate L. tAll samples are pre-labeled, the weights of classifier G1′_m are updated based on the pre-labeling results, and then samples that are mislabeled by classifier G1′_m are added to the training sample set of the next classifier G2′_m.

[0077] Step 5.1: Combine the labeled sample sets h1, h2, and h3 after multiple rounds of training into a new labeled sample set L. t Then, the parameters in the TSVM classifier are initialized, including the unlabeled samples in the buffer pool U′ and the aggregated labeled sample set L. t The labeled samples in the buffer pool U′ are configured with influence factors and slack variables to adjust the influence factor C of the unlabeled sample set. u Less than L t The influence factor C of the labeled sample set l One-thousandth;

[0078] Step 5.2: Determine C u Is it less than C? l If C is satisfied u Less than C l Then use the labeled sample set L t The classifier G1′_m is continuously trained (i.e., trained based on the classifier G1′_m obtained after m training rounds after step 4), otherwise the classifier G1′_m is used to label the unlabeled samples in the buffer pool U′.

[0079] Both labeled and unlabeled samples have their own slack variables, and the influence factor is used as a coefficient to measure the importance of labeled and unlabeled samples in the classifier G1 (referring to the trained model). Initially, the model training relies primarily on labeled samples, and the reference value of unlabeled samples is minimal. Therefore, at the beginning of training, the influence factor of labeled samples is set much higher than that of unlabeled samples. As the model trains, it continuously identifies outliers between two pre-labeled samples and swaps their labels. Through this continuous adjustment of the pre-labeling results, the value of unlabeled samples gradually increases. Therefore, the influence of unlabeled samples continuously increases with model training. When the influence factors of unlabeled and labeled samples become equal, it indicates that this phase of semi-supervised learning has ended.

[0080] Step 5.2.1: From the labeled sample set L t Select a certain number of samples to form a new labeled sample set D l The classifier G1 is trained (the first training is the classifier G1′_m after m training rounds obtained in step 4; if it is not the first training, it is trained based on the classifier obtained in the previous training) to obtain the updated classifier G1″.

[0081] Step 5.2.2: Use the updated classifier G1″ to pre-label the unlabeled samples in the buffer pool U′, obtaining the label corresponding to each unlabeled sample, thus obtaining the labeled sample set C in the buffer pool U′. u ;

[0082] Step 5.2.3: Influence factor C based on the unlabeled sample set u The influence factor C of the labeled sample set l The set of labeled samples D in the buffer pool u The labeled sample set D l The predicted labels of unlabeled samples in the buffer pool are used to solve the optimization objective of the TSVM objective function, and the parameters (w,b) and relaxation factor ξ of the classification hyperplane are obtained.

[0083] The objective function of the TSVM algorithm is:

[0084]

[0085] The constraints are:

[0086]

[0087] Where l represents the number of labeled samples, u represents the number of unlabeled samples, and i and j represent the sample indices; z i For the labeled sample x i Tag, y * j For unlabeled sample x j Predicted label, ζ i For the labeled sample x i The relaxation factor, ξ j For unlabeled sample x j The relaxation factor;

[0088] Step 5.2.4: Predict the labeled label y using the relaxation factor ξ obtained in 5.2.3 and the samples in the unlabeled sample set. * Determine the relationship between two unlabeled samples x. i and x j Does it meet the conditions? For x j If the predicted labeling results meet the conditions, then swap samples x. i and x j Predict the labeled tags, then re-execute 5.2.3. If the conditions are not met, update C. u =min(2C) u C l );

[0089] Note: If there are two unlabeled samples x in the unlabeled sample group. i and x j Their pre-labeled results and It is the opposite, that is And their slack variables ξ i and ξ j The sum satisfies: ξ i >0∧ξ j >0∧ξ i +ξ j If the value is greater than 2, then these two samples are very likely to have been incorrectly pre-labeled.

[0090] The labeling results of these two samples and After the swap, repeat step 5.2.3 to obtain the parameters (w', b') and ξ' of the new classification hyperplane. Then iterate this process until no two samples that meet the conditions can be found.

[0091] Step 5.3: Iteratively execute step 5.2 until C. u <C l The condition is checked until the updated classifier G1″ is obtained, and the labeling results of the unlabeled samples in the buffer pool are obtained. Then, G1″ is used to label the already labeled sample set L. t All samples in the dataset are pre-labeled. Based on the labeling results, the weights of classifier G1″ are recalculated. Then, classifier G1″ is applied to the labeled sample set L. t Samples that are incorrectly labeled are passed to the next classifier;

[0092] Step 5.3.1: Iterate through step 5.2 until C. u <C l Determine whether the condition is met;

[0093] Step 5.3.2: Obtain the labeling results of classifier G1″ for the unlabeled samples, and use classifier G1″ to label the labeled sample set L. t All samples are pre-labeled, and the weights of classifier G1″ are recalculated based on the labeling results.

[0094] Step 5.3.3: Apply classifier G1″ to the labeled sample set L t Samples that are incorrectly labeled are passed to the next classifier and used as part of the training sample set for the next classifier.

[0095] Step 6: Repeat the training process of Step 5 for G2′_m and G3′_m obtained after training in Step 4 to obtain the trained classifiers G2″ and G3″. After the training of the three classifiers is completed, obtain the labeling results of all unlabeled samples in the buffer pool U′ and the weights of classifiers G1″, G2″ and G3″. Then, add the labeling results of the three classifiers for all unlabeled samples in the buffer pool U′ according to the weight ratio of the classifiers to obtain the final labeling results of all unlabeled samples in the buffer pool U′.

[0096] Steps 5 and 6 above describe the functions implemented in the semi-supervised learning stage based on TSVM.

[0097] This embodiment was conducted using laboratory equipment and a Windows 10 PC. The CPU used in the experiment was an Intel Core i7 7700K processor, which has 4 cores and 8 threads, a base frequency of 4.5 GHz, and a boost frequency of 4.5 GHz. The GPU was an NVIDIA GTX 1082 Ti, which has 3584 CUDA cores, 11 GB of video memory, and a 352-bit memory bus. The PC was also equipped with 16 GB of RAM and a 1 TB hard drive.

[0098] The dataset used in this embodiment comes from labeled fundus images in the Kaggle competition dataset. Kaggle primarily targets developers and data scientists, providing them with a platform to host machine learning competitions. Kaggle contains a wealth of competition data, providing researchers with a solid research foundation.

[0099] The Kaggle Diabetic Retinopathy Detection Database is a public database containing numerous color fundus images. These images were collected from several primary care centers in California and elsewhere, using a variety of digital cameras. The resolution ranges from 433×289 pixels to 5184×3456 pixels, and all files are in JPEG format. These images are also uploaded to a free DR scanning platform.

[0100] In the result labeling stage, the method of this invention first trains the SVM classifier using a semi-supervised learning method based on co-training to complete the labeling of most easily labeled samples. For the remaining small portion of more difficult-to-label samples, a TSVM training method is used to continuously optimize the SVM solution, finally obtaining the final labeling result. Existing semi-supervised labeling methods typically use only one of these two methods to train the SVM classifier, without combining both. Therefore, this embodiment conducted a set of comparative experiments, training the SVM using a semi-supervised method based on co-training, training the SVM using TSVM, and training the SVM using a combination of semi-supervised co-training and TSVM. The same set of data was used to conduct experiments on these three methods, and the experimental results are as follows: Figure 4 As shown. From the experimental results... Figure 4 It can be seen that combining the co-training semi-supervised algorithm and the TSVM algorithm performs better than using either of these algorithms alone in terms of accuracy, sensitivity, and specificity. Specifically, TSVM outperforms co-training SVM in both accuracy and sensitivity, while the co-training semi-supervised algorithm performs better than TSVM in terms of specificity.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. An image annotation method based on semi-supervised learning, characterized in that: Three classifiers are used for training and labeling, implemented through a semi-supervised learning stage based on cooperation and a semi-supervised learning stage based on TSVM. The semi-supervised learning stage based on cooperation is used to complete the labeling task of easily labeled samples in the unlabeled sample set, controlling the number of labeled samples in each training session, while also controlling the number of samples in the training set. The semi-supervised learning stage based on TSVM is used to complete the labeling task of the remaining unlabeled samples. Both the labeled samples already labeled in the semi-supervised learning stage based on cooperation and the remaining unlabeled samples are used in the training, and ensemble learning methods are incorporated into the training to train the weights of the classifiers and the weights of the training set samples. The semi-supervised learning stage based on cooperative computation includes the following steps: Step 1: Collect medical images and label a portion of them to obtain a labeled sample set L. The remaining unlabeled images form an unlabeled sample set U. Simultaneously, construct three SVM classifiers G1, G2, and G3, along with a buffer pool. ; Step 2: Divide the labeled sample set L into three equal subsets h1, h2, and h3. These subsets h1, h2, and h3 will be used to train the three classifiers G1, G2, and G3, respectively. Then, randomly select u samples from the unlabeled sample set U and place them into the buffer pool. middle; Step 3: Train classifier G1 using the labeled sample subset h1 to obtain the updated classifier. The sample subset h1 is then deduplicated and updated; the updated classifier is then used. For the buffer pool Unlabeled samples in the dataset are pre-labeled and their confidence scores are calculated. Samples with confidence scores greater than a set threshold are simultaneously added to two other labeled sample subsets, h2 and h3. Meanwhile, samples from the unlabeled sample set U are added to the buffer pool to replenish unlabeled samples. Then, classifiers G2 and G3, as well as sample subsets h2 and h3, are trained and deduplicated using the same process as classifier G1 and sample subset h1, resulting in updated classifiers. and And summarize the three updated sample subsets; Step 4: Iterate through Step 3 until there are no remaining samples in the unlabeled sample set U, obtaining the classifier after m rounds of training. , and ; The semi-supervised learning phase based on TSVM includes the following steps: Step S1: Combine the samples from all deduplicated labeled sample subsets h1, h2, and h3 in the semi-supervised learning stage based on collaborative computation into a labeled sample set L. t In this section, the parameters of the TSVM classifier are initialized, and a buffer pool is prepared. Each unlabeled sample and L t Each labeled sample is assigned an influence factor, and slack variables are set for both labeled and unlabeled samples; the classifier obtained from step 4 after m rounds of training is used. For L t All samples are pre-labeled, and the classifier is updated based on the pre-labeling results. The weights are then used to classify the classifier. Incorrectly labeled samples are added to the next classifier. The training sample set; Step S2: Train the classifiers obtained after the semi-supervised learning stage based on cooperative computation respectively. and Repeat step S1 to obtain the trained classifier. and After the three classifiers are trained, the buffer pool is obtained. Labeling results and classifiers for all unlabeled samples. , and The weights are then assigned to the three classifiers and the buffer is used. The annotation results of all unlabeled samples are summed according to the weight ratio of the classifier to obtain the buffer pool. The final annotation results for all unlabeled samples.

2. The image annotation method based on semi-supervised learning according to claim 1, characterized in that: The specific method for step 3 is as follows: Step 3.1: Train classifier G1 using the labeled sample subset h1 to obtain the updated classifier. ; Step 3.2: Determine whether the number of samples in the labeled sample subset h1 reaches the threshold of the labeled training set. If it reaches the threshold, then pass the classifier. The samples in h1 are labeled. Samples with a confidence level greater than a set threshold are removed from h1 according to the ratio of positive to negative samples. Samples from the unlabeled sample set U among the removed samples are added to the set. middle; Step 3.3: Using a classifier For the buffer pool The samples in the set are pre-labeled. After labeling, p positive samples with the highest confidence and q negative samples with the highest confidence are selected and added to h2 and h3. At the same time, samples are drawn from the unlabeled sample set U to replenish the buffer pool. Then, duplicate samples are removed from the sample subsets h2 and h3. Step 3.4: Using sample subsets h2 and h3, perform the corresponding operations on classifiers G2 and G3 according to steps 3.1 to 3.3 respectively, and obtain the classifiers. and The process involves deduplicating and updating sample subsets h2 and h3; finally, the deduplicated sample subsets h1, h2, and h3 are summarized.

3. The image annotation method based on semi-supervised learning according to claim 2, characterized in that: The specific method for step S1 is as follows: Step S1.1: Combine the labeled sample sets h1, h2, and h3 after multiple rounds of training into a new labeled sample set L. t Then, the parameters in the TSVM classifier are initialized, and the buffer pool is configured. The unlabeled samples and the summarized labeled sample set L t The labeled samples in the buffer pool are configured with influence factors and slack variables to make the buffer pool... The influence factors of the unlabeled sample set Less than L t The influence factors of the sample set have been labeled. One-thousandth; Step S1.2: Determine Is it less than If satisfied Less than Then use the labeled sample set L t Continuously improve the classifier Train the classifier; otherwise, use the classifier. For the buffer pool Label the unlabeled samples; Step S1.3: Iteratively execute step S1.2 until... < The updated classifier is obtained by determining whether the condition is met. For the labeling results of unlabeled samples in the buffer pool, use For the labeled sample set L t All samples are pre-labeled, and the classifier is recalculated based on the labeling results. Weights of the classifier; For the labeled sample set L t Samples that are incorrectly labeled are passed to the next classifier.

4. The image annotation method based on semi-supervised learning according to claim 3, characterized in that: The specific method for step S1.2 is as follows: Step S1.2.1: From the labeled sample set L t Select a certain number of samples to form a new labeled sample set. For classifiers Train the classifier to obtain an updated classifier. ; Step S1.2.2: Use the updated classifier For the buffer pool Unlabeled samples are pre-labeled to obtain the label corresponding to each unlabeled sample, thus obtaining the buffer pool. The sample set after labeling ; Step S1.2.3: Influence factors based on unlabeled sample sets The influence factors of the sample set have been labeled. The set of labeled samples in the buffer pool Labeled sample set The parameters of the classification hyperplane are obtained by solving the optimization objective of the TSVM objective function, which includes the predicted labels of unlabeled samples in the buffer pool and the predicted labels of unlabeled samples in the buffer pool. and relaxation factor ; Step S1.2.4: Utilize the relaxation factor obtained in S1.2.3 Predicted labeling of samples in unlabeled sample sets Determine the difference between two unlabeled samples. and Does it meet the conditions? , for If the predicted labeling results meet the conditions, the samples are swapped. and Predict the labeled tags, then re-execute S1.2.

3. If the conditions are not met, update... .

5. The image annotation method based on semi-supervised learning according to claim 4, characterized in that: The objective function of the TSVM algorithm described in step S1.2.3 is: （1）； The constraints are: ； ； （2）； ； Where l represents the number of labeled samples, and u represents the number of unlabeled samples. Indicates the sample subscript; For labeled samples Label, Unlabeled samples Predicted labeling For labeled samples relaxation factor, Unlabeled samples The relaxation factor.

6. The image annotation method based on semi-supervised learning according to claim 5, characterized in that: The specific method for step S1.3 is as follows: Step S1.3.1: Iterate through step S1.2 until... < The condition is checked to determine if the condition is met; Step S1.3.2: Obtain the classifier Labeling results for unlabeled samples, and using a classifier For the labeled sample set L t All samples are pre-labeled, and the classifier is recalculated based on the labeling results. The weights; Step S1.3.3: Classify the classifier For the labeled sample set L t Samples that are incorrectly labeled are passed to the next classifier as part of the training sample set for the next classifier.

Images