Disturbance-based active learning medical image classification method and system

Abstract

The invention discloses an active learning medical image classification method and system based on disturbance. The sample is evaluated according to the influence of the characteristic disturbance on the model decision and the disturbance degree. Firstly, a mandatory label is allocated to each category to guide disturbance and obtain a new prediction of a sample. And if the model prediction of the sample changes, the sample is regarded as uncertain. Then, the disturbance distance obtained by the characteristic disturbance device is regarded as a diversity vector, and the diversity vector can be used for selecting valuable samples through clustering; and finally, selecting data points closest to each clustering center in the clustering space, wherein the number of the data points is the same as that of the clustering centers. According to the method, an efficient data utilization scheme is provided for medical image classification research, output of scientific achievements is accelerated, technical innovation and clinical application in the field of medical image analysis are promoted, and the workload of doctors is relieved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image classification, and in particular to a disturbance-based active learning medical image classification method and system. Background Art

[0002] Existing medical image classification methods are usually based on supervised learning algorithms, which require a large amount of labeled data to train the model so that the model can learn the features to distinguish different categories of images.

[0003] However, in the medical field, obtaining high-quality labeled data is a challenging task, and the annotation of medical images requires professional medical knowledge, which is not only time-consuming but also costly.

[0004] In order to solve the above problems, active learning has received widespread attention and application in the field of medical image classification in recent years. Its core idea is to strategically select the most "valuable" samples from unlabeled datasets for manual annotation, and then add these newly annotated data to the training set to iteratively improve model performance.

[0005] Active learning strategies are mainly divided into uncertainty-based strategies and diversity-based strategies.

[0006] Uncertainty-based active learning methods rely on metrics such as entropy or confidence to assess uncertainty, such as evaluating the prediction consistency between unlabeled samples and their data-augmented samples to select samples with lower consistency scores; measuring the differences in model outputs under different optimization steps to estimate the uncertainty of samples.

[0007] Diversity-based active learning methods use representative information such as feature distribution or gradient direction to select diverse samples, such as selecting samples that are very different from labeled samples but are representative among unlabeled samples; training the GCN model to distinguish between labeled and unlabeled samples, and selecting unlabeled samples that are different from labeled samples.

[0008] The above methods mainly focus on measurement uncertainty or representative information, ignoring the distribution characteristics and boundary information of samples.

[0009] Compared with natural images, medical images have high heterogeneity. How to design more effective sample selection strategies remains to be explored. Therefore, we propose to use the information of feature perturbations to evaluate the uncertainty and diversity of samples. Summary of the invention

[0010] The purpose of the present invention is to overcome the above-mentioned shortcomings and deficiencies of the prior art and to provide a perturbation-based active learning medical image classification method and system to further optimize the efficiency and accuracy of medical image classification.

[0011] The present invention evaluates samples based on the impact of feature perturbations on model decisions and the degree of perturbation. First, a mandatory label is assigned to each category to guide the perturbation and obtain a new prediction for the sample. If the model prediction of a sample changes, it is considered uncertain. The perturbation distance obtained by the feature perturbator is then considered a diversity vector, which can be used to select valuable samples through clustering. Finally, the data points closest to each cluster center with the same number and cluster center are selected in the cluster space.

[0012] The present invention is achieved through the following technical solutions:

[0013] A perturbation-based active learning medical image classification method comprises the following steps:

[0014] S1, feature perturbation step:

[0015] S1-1, input the unlabeled samples into the frozen encoder to extract the initial features;

[0016] S1-2, input the initial features into the frozen classifier to obtain the initial prediction value;

[0017] S1-3, the initial features are input into multiple perturbators, each perturbator performs multiple rounds of perturbations to obtain multiple perturbed features; multiple rounds generally refers to 20 rounds.

[0018] S1-4, input the perturbed features into the frozen classifier to obtain the perturbed predicted value;

[0019] S1-5, all samples whose predicted values ​​after perturbation are not equal to the initial predicted values ​​but are equal to the forced labels will be added to the candidate set;

[0020] The above perturbator is an MLP layer, each perturbator is assigned a mandatory label, and the loss function is the cross entropy loss function and the L2 loss function.

[0021] The number of the above perturbations is equal to the number of categories in the classification task, and the mandatory labels are the labels of all categories.

[0022] S2, diversity mapping and clustering step:

[0023] S2-1, for the samples in the candidate set, calculate the diversity vector of each sample. The result obtained by each perturbator is one of the elements of the vector. For each element, if the initial prediction value is the same as the forced label, the element value is set to 0. Otherwise, if the predicted value after perturbation is different from the forced label, the element value is set to -1. If the predicted value after perturbation is the same as the forced label, the element value is set to the Euclidean distance between the initial feature and the perturbed feature.

[0024] S2-2, perform K-means clustering on the diversity vector and select the data points closest to each cluster center with the same number as the cluster center.

[0025] A perturbation-based active learning medical image classification system, comprising:

[0026] Uncertainty module for feature perturbation to use forced labels to guide limited perturbations; samples that are successfully perturbed to the forced label feature space will be selected as candidate samples for further sampling; samples that fail to perturb to the corresponding feature space will be ignored;

[0027] Diversity mapping and clustering module, which uses the direction and degree of perturbation to construct diversity vectors and clusters them to select diverse samples from candidate samples;

[0028] Repeat the sampling process of the above modules, iteratively use the screened samples to continuously perform deep learning, and eventually a system for medical image classification tasks can be obtained with limited annotation costs.

[0029] Compared with the prior art, the present invention has the following advantages and effects:

[0030] The present invention is based on the perturbation active learning method for medical image classification. The proposed active learning selects uncertain and diverse samples based on feature perturbation information. It selects the most valuable samples with a limited labeling budget and obtains a model with good performance.

[0031] The present invention can be effectively applied in the field of medical image classification. Compared with traditional active learning methods, the present invention can more accurately identify the key samples that have the greatest impact on model training by introducing perturbation analysis, thereby significantly improving the efficiency and effect of model training under a limited annotation budget.

[0032] Through intelligent sample screening strategies, the present invention can significantly reduce the reliance on professional medical knowledge and human annotation, effectively reducing the time and economic costs of the data preparation stage.

[0033] The present invention provides an efficient data utilization solution for medical image classification research, which helps to accelerate the output of scientific research results, promote technological innovation and clinical application in the field of medical image analysis, and reduce the workload of doctors. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of sample selection for the perturbation-based active learning medical image classification method of the present invention

[0035] Figure 2 This is a workflow diagram of the perturbation-based active learning method of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below in conjunction with specific embodiments.

[0037] like Figure 1-2 The present invention discloses a perturbation-based active learning medical image classification method, which selects samples with strong uncertainty from unlabeled samples as candidate samples, and then selects diverse samples from the candidate samples for labeling and uses them for model training, thereby training a model with good performance under limited labeling cost. It is mainly divided into two steps:

[0038] (1) Selecting uncertain samples

[0039] In the feature space, samples that are susceptible to small perturbations and lead to prediction changes can be regarded as uncertain samples. Therefore, an uncertainty module called feature perturbation is proposed to guide limited perturbations using forced labels. Samples that are successfully perturbed to the forced label feature space will be selected as candidate samples for further sampling, such as Figure 1 In contrast, samples that fail to perturb the corresponding feature space will be ignored, e.g. Figure 1 Sample b in (a). Our uncertainty module enhances the model’s ability to distinguish different categories.

[0040] (2) Selecting a diverse sample

[0041] The perturbation direction and distance of the mandatory labels reflect the differences between samples. Therefore, a diversity module, called diversity mapping and clustering, is proposed, which uses the direction and degree of perturbation to construct a diversity vector and cluster it to select diverse samples from candidate samples. Figure 1 In (b), candidate samples c and d with different diversity vectors are considered as diversity samples. The diversity module avoids information redundancy.

[0042] By repeating the above sampling process and iteratively using the screened samples for continuous deep learning, we can eventually obtain a model with better performance for medical image classification tasks with limited annotation costs.

[0043] Figure 2 A workflow diagram of the present invention is shown.

[0044] Figure 2 A is the main process of active learning. Initially, a small batch of samples are randomly selected to train the model. Subsequently, the labeled sample pool is used to fine-tune the model iteratively. After fine-tuning, an active learning strategy is used to select another batch of samples for manual labeling and put them into the labeled sample pool for model fine-tuning until a sufficient number of samples are accumulated. The model is mainly composed of an encoder and a classifier.

[0045] Figure 2B shows the perturbation-based active learning method proposed in this invention, which is mainly divided into two modules: feature perturbation, diversity mapping and clustering.

[0046] Feature perturbation module:

[0047] S1. Input the unlabeled samples into the frozen encoder to extract the initial features;

[0048] S2. Input the initial features into the frozen classifier to obtain the initial prediction value;

[0049] S3. Input the initial features into multiple perturbators, and perform 20 rounds of perturbations on each perturbator to obtain multiple perturbed features. The perturbator is an MLP layer, and each perturbator is assigned a mandatory label. The loss function is the cross entropy loss function and the L2 loss function. The number of perturbators is equal to the number of categories in the classification task, and the mandatory labels are the labels of all categories.

[0050] S4. Input the perturbed features into the frozen classifier to obtain the perturbed predicted value;

[0051] S5. Any sample whose predicted value after perturbation is not equal to the initial predicted value but equal to the forced label will be added to the candidate set;

[0052] Diversity Mapping and Clustering Modules:

[0053] S11. For the samples in the candidate set, calculate the diversity vector of each sample. The result obtained by each perturbator is one of the elements of the vector. For each element: if the initial prediction value is the same as the forced label, the element value is set to 0; otherwise, if the predicted value after perturbation is different from the forced label, the element value is set to -1; if the predicted value after perturbation is the same as the forced label, the element value is set to the Euclidean distance between the initial feature and the perturbed feature;

[0054] S22. Perform K-means clustering on the diversity vector and select data points closest to each cluster center whose number is the same as the cluster center.

[0055] As described above, the present invention can be implemented well.

[0056] The implementation methods of the present invention are not limited to the above-mentioned embodiments, and any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principles of the present invention shall be equivalent replacement methods and shall be included in the protection scope of the present invention.

Claims

1. A perturbation-based active learning method for medical image classification, characterized in that The steps include: S1, feature perturbation step: S1-1, input the unlabeled samples into the frozen encoder to extract the initial features; S1-2, input the initial features into the frozen classifier to obtain the initial prediction value; S1-3, input the initial features into multiple perturbators, each perturbator performs multiple rounds of perturbations, and obtains multiple post-perturbation features; S1-4, input the perturbed features into the frozen classifier to obtain the perturbed predicted value; S1-5, all samples whose predicted values ​​after perturbation are not equal to the initial predicted values ​​but are equal to the forced labels will be added to the candidate set; S2, diversity mapping and clustering step: S2-1, for the samples in the candidate set, calculate the diversity vector of each sample. The result obtained by each perturbator is one of the elements of the vector. For each element, if the initial prediction value is the same as the forced label, the element value is set to 0. Otherwise, if the predicted value after perturbation is different from the forced label, the element value is set to -1. If the predicted value after perturbation is the same as the forced label, the element value is set to the Euclidean distance between the initial feature and the perturbed feature. S2-2, perform K-means clustering on the diversity vector and select the data points closest to each cluster center with the same number as the cluster center.

2. The perturbation-based active learning medical image classification method according to claim 1, characterized in that: The perturbator described in S1-3 is an MLP layer, each perturbator is assigned a mandatory label, and the loss function is the cross entropy loss function and the L2 loss function.

3. The perturbation-based active learning medical image classification method according to claim 2, characterized in that: The number of perturbators is equal to the number of categories in the classification task, and the mandatory labels are the labels of all categories.

4. A perturbation-based active learning medical image classification system, characterized in that include: Uncertainty module for feature perturbations to guide limited perturbations using mandatory labels; The samples that are successfully perturbed to the forced label feature space will be selected as candidate samples for further sampling; the samples that fail to be perturbed to the corresponding feature space will be ignored; Diversity mapping and clustering module, which uses the direction and degree of perturbation to construct diversity vectors and clusters them to select diverse samples from candidate samples; Repeat the sampling process of the above modules, iteratively use the screened samples to continuously perform deep learning, and eventually a system for medical image classification tasks can be obtained with limited annotation costs.

Images