A medical image classification method based on small sample meta-learning

Abstract

This invention discloses a medical image classification method based on few-sample meta-learning. The invention employs a two-stage training approach to obtain a model with inter-domain generalization ability, addressing the problems of limited training samples and poor inter-domain generalization in medical images. Secondly, this invention designs a bi-branch adaptive meta-transfer network and a corresponding joint cross-entropy loss function. Through adversarial training of the network, the generalization ability of the feature extractor across different modalities and devices is enhanced, solving the overfitting problem of few-sample learning and obtaining effective recognition results for medical image analysis and processing applications. Finally, this invention designs a global similarity compatibility module to reduce noise caused by image background, imaging conditions, and human operation in few-sample meta-learning predictions, improving the model's prediction robustness. Simultaneously, the global similarity compatibility module can effectively establish a sample compatibility matrix, addressing the weakness of few-sample predictions being easily affected by noise, further improving prediction accuracy.

Description

Technical Field

[0001] This invention belongs to the field of medical image analysis and processing technology, specifically relating to a medical image classification method based on few-sample meta-learning. Background Technology

[0002] In recent years, the analysis and processing of medical images has received significant attention both domestically and internationally in the medical field. Its purpose is to perform preliminary identification and grading of medical images before clinical diagnosis, or to isolate target organs or lesions from medical images, making anatomical or pathological structural changes in the images clearer, providing reliable data for pathological diagnosis and research, and assisting doctors in diagnosing, quantifying, and monitoring diseases. Compared to traditional manual methods, deep learning-based image processing methods are more efficient, alleviating the pressure on doctors and reducing the probability of misdiagnosis and missed diagnosis. Furthermore, manual methods require substantial manpower and extensive experience from doctors, thus necessitating automated medical image processing methods. Deep learning-based image processing methods have achieved breakthroughs in many fields, particularly in various practical applications using natural images. However, compared to natural images, medical images often exhibit inconsistent quality due to factors such as equipment, imaging conditions, and patient characteristics. Moreover, obtaining medical images is difficult due to the complexity of annotation and restrictions related to patient privacy. Therefore, how to obtain accurate prediction results under constraints such as small sample sizes and low quality is a major research topic in medical image analysis and processing. In clinical applications, medical images are mainly categorized into images obtained through X-rays, computed tomography (CT) scans, and magnetic resonance imaging (MRI). However, due to factors such as imaging equipment, imaging conditions, and patient condition, even images of the same type can vary significantly between different hospitals and patients. Compared to natural images, medical images have a smaller sample size, making many algorithms prone to overfitting and unable to demonstrate good generalization performance across different batches of images. Summary of the Invention

[0003] The purpose of this invention is to provide a medical image classification method based on few-sample meta-learning to overcome the shortcomings of existing technologies.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A medical image classification method based on few-sample meta-learning includes the following steps:

[0006] Step 1, pre-training stage: Use a classification network to extract features and predict results from the training dataset;

[0007] Step 2: Reorganize the training dataset into a set of tasks. To construct a task, randomly select C different classes from the training dataset. For each class, select K samples as the labeled support set and N samples as the unlabeled query set. The prediction of the N samples in the query set is based on the K labeled samples in the support set. To construct the training task set, repeat this operation n times to generate a training task set containing n tasks, which will serve as the input data for subsequent training.

[0008] Step 3: Meta-training of the dual-branch adaptive meta-transfer network model. Based on the feature extractor obtained in Step 1 and the training task set obtained in Step 2 as input data, the dual-branch network model performs meta-training on the training task set. In the meta-training stage, the feature extractor learns meta-knowledge and calculates the similarity between the support set and the query set in each task.

[0009] Step 4: During the forward propagation process, the training task samples pass through the dual-branch network model to obtain two prediction results. The difference between the prediction results and the true labels is calculated using the loss function to obtain the prediction loss value.

[0010] Step 5: During backpropagation, a gradient inversion module is used before the linear classifier, and the two branches jointly optimize the feature extractor through adversarial training.

[0011] Step 6: Optimize the parameters of the two-branch network model using gradient descent strategy, update the network model according to the value of the loss function, and continuously reduce the loss function until convergence; calculate the distance between the true label and the model prediction, and when the training loss of the network model converges, save the model parameters and use the model as a medical image analysis and processing model.

[0012] A further improvement of this invention is that, in step one, the classification network is a combination of a ResNet-12 residual network with 12 convolutional layers and a linear classifier; ResNet-12 is the main body of the feature extractor, with a total of 4 basic modules, each basic module containing 3 3×3 convolutional operations, 1 batch regularization operation, 1 residual operation, and 1 ReLU operation; each basic module finally contains a max pooling operation with a stride of 2.

[0013] A further improvement of the present invention is that, in step one, the training data can accept multiple modal data inputs, including natural images, X-rays, computed tomography scans, and magnetic resonance images.

[0014] A further improvement of this invention is that the optimized representation of the classification network in step one is as follows:

[0015]

[0016] Among them, fθ This represents the feature extractor, with parameter θ. Represents a linear classifier, whose parameters are: The optimization objective of the classification network is to find the parameters θ and φ that minimize the energy function E.

[0017] A further improvement of the present invention is that step two reorganizes the training data, retains the original size and label information of the images, and organizes the arrangement of the images.

[0018] A further improvement of the present invention is that, in step three, a global similarity compatibility module is used to calculate the similarity between the support set and the query set samples. The global similarity compatibility module first calculates the similarity between each sample in the support set and the query set by matrix multiplication, and then calculates the compatibility between samples by cosine similarity.

[0019] A further improvement of this invention is that the loss function used to calculate the difference between the predicted result and the true label in step four is the cross-entropy loss function.

[0020] A further improvement of this invention is that the backpropagation process of the gradient reversal module in step five is defined as follows:

[0021]

[0022] Where τ is a hyperparameter of the gradient reversal module, defined as a constant, and Ι is the identity matrix.

[0023] A further improvement of the present invention is that, when inputting training data in steps one and three, random cropping, mirror flipping, and contrast enhancement are used to expand the training data during the training process.

[0024] Compared with the prior art, the present invention has the following beneficial technical effects:

[0025] This invention provides a medical image classification method based on few-sample meta-learning. It obtains a model with inter-domain generalization ability through a two-stage training approach, addressing the problems of limited training samples and poor inter-domain generalization in medical images. In the pre-training stage, the public dataset mini-ImageNet is used as the training set to train the feature extractor and linear classifier. The feature extractor acquires generalization ability across tasks within the same image domain. In the meta-training stage, the mini-ImageNet dataset is reorganized to obtain a training task set containing n tasks. The feature extractor is then retrained to acquire generalization ability across tasks in different image domains.

[0026] Secondly, this invention designs a bi-branch adaptive meta-transfer network and a corresponding joint cross-entropy loss function. Through adversarial training of this network, the generalization performance of the feature extractor across different modalities and devices is enhanced, while simultaneously addressing the overfitting problem inherent in small-sample learning, thus obtaining effective recognition results for practical applications in medical image analysis and processing. Furthermore, the network model's training data can accept inputs from various modalities, including natural images, X-rays, computed tomography scans, and magnetic resonance images, thus lowering the requirements for training data and broadening its application range.

[0027] Finally, this invention designs a global similarity compatibility module to reduce noise caused by factors such as image background, imaging conditions, and human operation in small-sample meta-learning predictions, thereby improving the robustness of model predictions. Simultaneously, for medical image prediction tasks with a small sample size, the global similarity compatibility module can effectively establish a compatibility matrix of related samples and calculate the compatibility between samples through cosine similarity calculations, addressing the weakness of small-sample predictions being easily affected by noise and further improving prediction accuracy. Attached Figure Description

[0028] Figure 1 This is a flowchart of the present invention;

[0029] Figure 2 This is a model structure diagram of the dual-branch adaptive meta-transfer network in this invention;

[0030] Figure 3 This is a model structure diagram of the global similarity compatibility module in this invention;

[0031] Figure 4 This is a model structure diagram of the feature extractor in this invention. Detailed Implementation

[0032] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] See Figure 1 This invention provides a medical image classification method based on few-sample meta-learning, comprising the following steps:

[0034] Step 1, the pre-training stage, uses a classification network to extract features and predict results on the training dataset; this stage allows the feature extractor of the classification network to learn transferable image features, enabling the classification network to generalize between tasks within the same image domain.

[0035] Step two involves reorganizing the training dataset into a set of tasks. To construct a task, C different classes are randomly selected from the training dataset. For each class, K samples are chosen as the labeled support set, and N samples are chosen as the unlabeled query set. Predictions for the N samples in the query set are based on the K labeled samples in the support set. To construct the training task set, this process is repeated n times to generate a training task set containing n tasks, which serves as the input data for subsequent training.

[0036] Step 3: Meta-training of the dual-branch adaptive meta-transfer network. Based on the feature extractor obtained in Step 1 and the training task set obtained in Step 2 as input data, the dual-branch network performs meta-training on the training task set. In the meta-training stage, the feature extractor learns meta-knowledge and calculates the similarity between the support set and the query set in each task. This stage enables the network to have the ability to generalize between different image domain tasks.

[0037] Step 4: During the forward propagation process, the training task samples pass through a dual-branch network to obtain two prediction results and a loss value.

[0038] Step 5: During backpropagation, a gradient inversion module is used before the linear classifier, and the two branches jointly optimize the feature extractor through adversarial training.

[0039] Step 6: Optimize the parameters of the two-branch network model using gradient descent strategy, update the network model according to the value of the loss function, and continuously reduce the loss function until convergence; calculate the distance between the true label and the model prediction, and when the training loss of the network model converges, save the model parameters and use the model as a medical image analysis and processing model.

[0040] Example

[0041] See Figure 1 The present invention includes the following steps:

[0042] Step 101: Perform data augmentation on the public dataset mini-ImageNet and its corresponding labels, including random cropping, mirroring, and contrast enhancement.

[0043] Step 102: Input the augmented data into the classification network for feature extraction, and output the prediction result through a linear classifier.

[0044] Step 103: Use cross-entropy loss to evaluate the distance between the current prediction result and the true label, and calculate the prediction loss of the classification network. The optimization process of the classification network can be represented as:

[0045]

[0046] Among them, f θ This represents the feature extractor, with parameter θ. Represents a linear classifier, whose parameters are: The optimization objective of the classification network is to find the parameters θ and φ that minimize the energy function E.

[0047] Step 104: Determine whether the current model loss value has converged. If it has converged, proceed to the second stage of training and proceed to step 105. Otherwise, continue the current training process.

[0048] Step 105: Reorganize the training dataset into a training task set. To construct a task, randomly select C different classes from the training dataset. For each class, select K samples as the labeled support set and N samples as the unlabeled query set. The prediction of the N samples in the query set is based on the K labeled samples in the support set. To construct the training task set, repeat this operation n times to generate a training task set containing n tasks, which will serve as the input data for subsequent tasks.

[0049] Step 106: Input the training task set organized in Step 105 into the two-branch adaptive meta-transfer network for training. Iteratively update the network using backpropagation and gradient descent algorithms. The optimization process of the two-branch adaptive meta-transfer network can be represented as:

[0050]

[0051]

[0052]

[0053] Among them, f θ This represents the feature extractor, with parameter θ. Represents a linear classifier, whose parameters are: L1 and L2 represent the cross-entropy loss functions for branch one and branch two, respectively. α represents the learning rate. The optimization objective of the two-branch network is to find the function that minimizes the energy. The parameters θ and

[0054] Step 107: The cross-entropy joint loss is used to evaluate the distance between the current network prediction result and the true label, and the prediction loss of the two-branch network is calculated. The cross-entropy joint loss function is a weighted sum of the two branch loss functions, expressed as:

[0055] L = L1 + L2.

[0056] Step 108: Determine whether the cross-entropy joint loss value has converged. If it has converged, proceed to step 109; otherwise, continue the current training process until convergence.

[0057] Step 109: Output the parameters of the converged bi-branch adaptive meta-transfer network model for subsequent testing tasks.

[0058] See Figure 2 The model structure diagram of the two-branch adaptive meta-transfer network includes the following parts:

[0059] 1) Feature extractor: The support set and query set are input into a shared feature extractor. After obtaining high-level features, the first and second branches are input into the feature extractor for further processing.

[0060] 2) Global Similarity Compatibility Measurement Module: This module calculates the similarity between samples in the support set and the query set to obtain a similarity compatibility matrix. Then, by calculating the similarity of the compatibility values ​​between any two samples in the compatibility matrix, it obtains the prediction result 1 for the query set samples. The prediction result 1 and the true label are compared using the cross-entropy loss function to obtain the loss value.

[0061] 3) Linear Classifier: The query set is fed separately into a linear classifier, which is a fully connected layer. This fully connected layer provides a prediction result for each sample in the query set. The prediction result and the true label are compared using the cross-entropy loss function to calculate the loss value.

[0062] See Figure 3 The model structure diagram of the global similarity compatibility module includes the following parts:

[0063] The support set and query set feature vectors form an N×M feature matrix. This feature matrix is ​​transposed to obtain an M×N transpose matrix. Multiplying these two matrices yields an N×N similarity compatibility matrix. Values ​​of 1 on the diagonal of the similarity compatibility matrix indicate compatibility between a sample and itself. Furthermore, the similarity compatibility matrix is ​​symmetric, meaning that the value at position (i,j) is the same as the value at position (j,i), indicating that the compatibility between sample i and sample j is equal to the compatibility between sample j and sample i. The global similarity compatibility module requires no additional trainable parameters and can be implemented through matrix multiplication.

[0064] See Figure 4The feature extractor model structure diagram includes the following parts:

[0065] The feature extractor comprises four residual modules, each with an identical structure. Specifically, each residual module contains three 3×3 convolutional operations and one 1×1 convolutional operation. Each convolutional operation is followed by a batch regularization operation to normalize the model's feature distribution and accelerate model convergence. ReLU activation is applied after the batch regularization of the first and second 3×3 convolutional operations. Residual connections are performed across layers after the input and the third 3×3 convolutional operation to fuse low-level and high-level features. Each residual module concludes with max pooling to reduce the model's feature dimensionality and save computation.

[0066] In summary, this invention provides a medical image classification method based on few-sample meta-learning, which designs a bi-branch adaptive meta-transfer network, including two-stage training, a global similarity compatibility module, and a joint optimization loss function.

[0067] The two-stage training process first involves a pre-training phase where the feature extractor learns transferable image features, enabling the network to generalize across tasks within the same image domain. Next, a meta-training phase allows the feature extractor to learn meta-knowledge, enabling the network to generalize across tasks in different image domains. Both phases use the same training dataset, but the data organization differs. The pre-training phase uses a traditional data organization, with the data itself serving as training data; the meta-training phase organizes the data into different tasks, which then serve as training data. Through learning from these different tasks, the meta-training phase enhances the feature extractor's generalization ability. Both phases use the same feature extractor, while the linear classifier serves only as an auxiliary training tool in the meta-training phase, used to remove representations that lack inter-domain generalization ability.

[0068] The global similarity compatibility module calculates the compatibility between samples in the support set and the query set. By jointly representing all samples, it reduces noise caused by factors such as image background and improves the robustness of network prediction. Furthermore, this module does not require additional training parameters; the compatibility matrix is ​​obtained through matrix multiplication. By calculating the cosine similarity between the i-th and j-th rows of the compatibility matrix, the compatibility between the i-th and j-th samples can be obtained. The closer the compatibility is to 1, the more likely the two samples belong to the same category.

[0069] The joint optimization loss function achieves adversarial training of the bi-branch adaptive meta-transfer network through the gradient reversal layer. While learning generalizable representations, it adaptively removes representations that do not have inter-domain generalization ability learned in the pre-training process, thereby improving the generalization ability of the model.

[0070] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A medical image classification method based on few-sample meta-learning, characterized in that, Includes the following steps: Step 1, pre-training stage: Use a classification network to extract features and predict results from the training dataset; Step 2: Reorganize the training dataset into a set of tasks. To construct a task, randomly select C different classes from the training dataset. For each class, select K samples as the labeled support set and N samples as the unlabeled query set. The prediction of the N samples in the query set is based on the K labeled samples in the support set. To construct the training task set, repeat this operation n times to generate a training task set containing n tasks, which will serve as the input data for subsequent training. Step 3 involves meta-training a two-branch adaptive meta-transfer network model, including a feature extractor. This feature extractor is shared by both the support set and the query set, and after obtaining high-level features, these features are simultaneously input into Branch 1 and Branch 2 for further processing. Branch 1 uses a global similarity compatibility metric module to calculate the similarity between samples in the support set and the query set, obtaining a similarity compatibility matrix. Then, by calculating the similarity of the compatibility values ​​between any two samples in the compatibility matrix, a first prediction result for the query set samples is obtained. The first prediction result and the true label are compared using the cross-entropy loss function to obtain the loss value. Branch 2 uses a linear classifier, where the query set is input separately. The linear classifier is a fully connected layer. Through this fully connected layer, a second prediction result for each sample in the query set is obtained. The second prediction result and the true label are compared using the cross-entropy loss function to obtain the loss value. Based on the feature extractor obtained in step one and the training task set obtained in step two as input data, the dual-branch network model performs meta-training on the training task set; in the meta-training stage, the feature extractor learns meta-knowledge and calculates the similarity between the support set and the query set in each task. Step 4: During the forward propagation process, the training task samples pass through the dual-branch network model to obtain two prediction results. The difference between the prediction results and the true labels is calculated using the loss function to obtain the prediction loss value. Step 5: During backpropagation, a gradient inversion module is used before the linear classifier, and the two branches jointly optimize the feature extractor through adversarial training. Step 6: Optimize the parameters of the two-branch network model using gradient descent strategy, update the network model according to the value of the loss function, and continuously reduce the loss function until convergence; calculate the distance between the true label and the model prediction, save the model parameters when the training loss of the network model converges, and use the model as a medical image analysis and processing model.

2. The medical image classification method based on few-sample meta-learning according to claim 1, characterized in that, In step one, the classification network is a combination of a ResNet-12 residual network with 12 convolutional layers and a linear classifier; ResNet-12 is the main body of the feature extractor, consisting of 4 basic modules, each containing 3 features. Each basic module includes a convolution operation, a batch regularization operation, a residual operation, and a ReLU operation; each basic module ends with a max pooling operation with a stride of 2.

3. The medical image classification method based on few-sample meta-learning according to claim 1, characterized in that, In step one, the training data can accept inputs from multiple modalities, including natural images, X-rays, computed tomography scans, and magnetic resonance images.

4. The medical image classification method based on few-sample meta-learning according to claim 1, characterized in that, The optimized representation of the classification network in step one is as follows: in, This represents the feature extractor, whose parameters are: ; Represents a linear classifier, whose parameters are: The optimization objective of the classification network is to find the function that minimizes the energy. parameters and .

5. The medical image classification method based on few-sample meta-learning according to claim 1, characterized in that, Step two involves reorganizing the training data, preserving the original image size and label information, and organizing the image arrangement.

6. The medical image classification method based on few-sample meta-learning according to claim 1, characterized in that, In step three, the global similarity compatibility module is used to calculate the similarity between the support set and the query set samples. The global similarity compatibility module first calculates the similarity between each sample in the support set and the query set by matrix multiplication, and then calculates the compatibility between samples by cosine similarity.

7. The medical image classification method based on few-sample meta-learning according to claim 1, characterized in that, In step four, the loss function used to calculate the difference between the predicted result and the true label is the cross-entropy loss function.

8. The medical image classification method based on few-sample meta-learning according to claim 1, characterized in that, The backpropagation process of the gradient reversal module in step five is defined as follows: in, These are the hyperparameters of the gradient reversal module, defined as constants. It is an identity matrix.

9. A medical image classification method based on few-sample meta-learning according to claim 1, characterized in that, In steps one and three, random cropping, mirroring, and contrast enhancement are used to augment the training data during the training process.

Images