Parameter stable transfer learning method for medical image classification

Abstract

The invention discloses a parameter stabilization transfer learning method for medical image classification, and belongs to the field of medical image classification transfer learning. The method comprises the steps of constructing a gradient mapping and gradient adjustment optimization module, for two different data sets, synchronously training a source domain and a target domain to a same convolutional neural network model, and in a pre-training stage, adding a target domain gradient as a momentum item into a source domain gradient updating process for model training; meanwhile, training a multi-layer perceptron model through a mean square error loss function by taking the multi-layer perceptron model as an input pair; and finally, in a fine tuning stage, carrying out weighted fusion on the target gradient and the source domain gradient after the conversion of the multi-layer perception model, and generating a final gradient for updating the model.

Description

Technical Field

[0001] This invention relates to the field of deep learning-based medical image analysis, particularly to the field of medical image classification transfer learning. The product is mainly used for fine-tuning of medical image classification, specifically a parameter-stabilized transfer learning method for medical image classification. Background Technology

[0002] Traditional deep learning-based medical image classification methods often suffer from unstable performance due to insufficient available labeled data and limited medical datasets. While few-shot learning and transfer learning can achieve some success in medical image classification, models trained using few-shot learning often lack practical application value. Although transfer learning is effective in real-world scenarios, it suffers from two inherent drawbacks: first, when the data distribution and feature spaces of the source and target domains differ significantly, direct fine-tuning can lead to "negative transfer," making training results highly unstable; second, during fine-tuning, the model's weight parameters are strongly constrained by the initial pre-trained values ​​in the source domain, making it difficult to fully adapt to the target domain task, severely hindering the model from achieving optimal performance in the target domain. To improve transfer performance, existing techniques mainly focus on fine-tuning strategies such as regularization, selective transfer, model pruning, and distillation learning, but none can fundamentally solve the inconsistency problem between the source and target domains in the gradient update direction. Therefore, we have invented a parameter-stabilized transfer learning method based on gradient adjustment.

[0003] Existing deep learning-based medical image classification methods face two major bottlenecks: insufficient available labeled data and a limited number of medical datasets. While existing techniques such as few-shot learning and transfer learning can initially address these issues, the scarcity of large datasets in the medical imaging field, coupled with significant feature differences between medical and natural data, leads to a lack of universality in models trained using few-shot learning and low reliability of transfer learning results, resulting in unstable performance in medical image classification. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to propose a parameter-stabilized transfer learning method for medical image classification, which can alleviate the impact of insufficient sample quantity on classification results to a certain extent, realize the alignment of the source domain and the target domain at the gradient level, and improve the stability of the transfer effect and classification performance.

[0005] Technical solution: A parameter-stabilized transfer learning method for medical image classification, which includes gradient mapping construction and gradient adjustment optimization modules, specifically: For two different datasets, first, the source domain... and target domain Simultaneously train the same convolutional neural network model, and in the pre-training phase, apply the gradient of the target domain. As a momentum term, it is added to the gradient of the source domain. During the update process, it is used for model training; at the same time, with As input pairs, a multilayer perceptron model is trained using the mean squared error loss function; finally, in the fine-tuning stage, the target gradient after the multilayer perceptron model is transformed. gradient of the source domain Weighted fusion is performed to generate the final gradient, which is then used to update the model.

[0006] To achieve the gradient of the target domain gradient to source domain The initial guidance of the gradient direction is constrained in the following two ways: first, the gradient of the target domain is added as a momentum term to the gradient update process of the source domain during the source domain gradient update process; second, the source-to-gradient mapping paradigm is learned through a multilayer perceptron model.

[0007] To achieve cross-domain transfer learning, this method sets the following constraints to improve the effectiveness of transfer learning in medical image classification: (1) Apply mini-batch stochastic gradient descent to update the gradient of the source domain. Weights, source domain weight Calculate as follows: ; ; in This is the current batch round number. It's the learning rate. It is the objective function of the model. Represents the gradient; (2) Incorporate the gradient of the target domain as a momentum term into the gradient update process of the source domain: ; ; in It is the first Batch target gradient Momentum; (3) Gradient mapping model, used to ensure the gradient of the source domain Able to be converted into target gradient And achieve the target gradient To further ensure the gradient of the source domain The guiding role; For convolutional neural network models, it is assumed that the gradient of any convolutional layer is... G ,but GEach row corresponds to a gradient update direction. If the gradients of two rows are equal, they will be updated along the same direction. For source domain data and target domain data ,like and If the results are similar and meet the set requirements, then the following optimization is performed using the mean squared error loss function: ; in, The gradient is represented in vector form. Since gradient mapping is a non-linear mapping, a multilayer perceptron model is used for gradient mapping, given the model weights. = { ,..., }and ={ ,..., The ultimate optimization goal is to minimize the mean squared error loss function argmin_MSEloss of the multilayer perceptron model parameters.

[0008] The function argmin_MSEloss, which minimizes the mean squared error loss, is:

[0009] in N It represents the total number of convolutional layers.

[0010] Furthermore, the method described in this invention uses a weighted average gradient G to update the model, wherein... This stage uses a weighted average gradient G and source domain data. Optimize the model, as follows: ,in and These are the weighting coefficients.

[0011] Beneficial Effects: The gradient-adjustment-based transfer learning method for medical image classification provided by this invention can alleviate the impact of insufficient sample size on classification results to a certain extent. Simultaneously, it can effectively align the gradients of the source and target domains, effectively mitigating negative transfer and improving the performance and robustness of medical image classification. In clinical diagnostic tasks, it can enhance the auxiliary diagnostic effect of deep models in rare diseases or diseases with limited sample sizes.

[0012] Existing medical image classification transfer learning methods mostly utilize large natural datasets as source domain data for pre-training the classification model, and then fine-tuning it using a smaller medical target dataset. However, existing methods cannot address the issue of model weights biased towards the pre-trained weights in the source domain during fine-tuning, leading to suboptimal fine-tuning results in the target domain and impacting the final classification performance and clinical diagnostic applications. This invention addresses this issue from the perspective of model gradient updates. Both the source and target domains are trained on the same CNN model, with the target domain consistently providing directional guidance for the source domain. A neural network-based mapping model guides the gradient update direction of the source domain to continuously approach the gradient update direction of the target domain during the update process, ensuring that the model weights are biased towards the target domain. This invention, even with limited medical datasets, allows the use of large-scale datasets from the medical or other fields as the source domain for pre-training and fine-tuning on the target dataset, improving the diagnostic performance of deep models in clinical practice. It holds broad application prospects in the diagnosis of rare diseases or diseases with limited clinical data samples. Attached Figure Description

[0013] Figure 1 This is an implementation framework diagram of the method described in this invention. Detailed Implementation

[0014] To illustrate the technical solutions disclosed in this invention in detail, further descriptions are provided below in conjunction with specific embodiments.

[0015] Combination Figure 1 As shown, this invention targets medical images. In medical image classification tasks, it uses large-scale open-source chest X-ray images as the source domain and the blood cell classification dataset from the Affiliated Hospital of Xuzhou Medical University as the target domain. Then, using the scheme provided by this invention, firstly, in the pre-training phase, a CNN model is trained simultaneously using both the source and target domains, calculating the gradients Gs and Gt of each layer. Gs is based on the source domain data Ds, and Gt is based on the target domain data Dt. Gradient updates are performed using their respective loss values ​​and backpropagation. During this process, the target gradient is added as a momentum term to the source domain gradient update process, and a gradient mapping paradigm from the source gradient to the target gradient is learned using an MLP-based gradient mapping model. Then, in the fine-tuning phase, the weighted average gradient formula G = α is applied. Gs + β The gradient Gt' (with α and β set to 0.7 and 0.3 respectively to balance inter-domain differences) is used to fine-tune the pre-trained model, resulting in the final classification result. This enables cross-domain transfer of medical image data, allowing the use of multi-domain datasets to improve the classification performance of the target dataset. As a derivative system, an integrated module can be built, including data preprocessing, a gradient calculation engine, and a classification output interface. This system, implemented in the PyTorch framework, supports real-time processing of medical image input, improving classification accuracy and robustness. This invention can play a significant role in clinical diagnostic tasks, particularly in the diagnosis of rare diseases or diseases with limited data samples.

[0016] Table 1. Comparison results with mainstream methods in ResNet50

Claims

1. A parameter-stabilized transfer learning method for medical image classification, characterized in that, This method includes a gradient mapping and gradient adjustment optimization module. For two different datasets, the source domain is first... and target domain Simultaneously train the same convolutional neural network model, and in the pre-training phase, apply the gradient of the target domain. As a momentum term, it is added to the gradient of the source domain. During the update process, it is used for model training; at the same time, with As input pairs, a multilayer perceptron model is trained using the mean squared error loss function; finally, in the fine-tuning stage, the target gradient after the multilayer perceptron model is transformed. gradient of the source domain Weighted fusion is performed to generate the final gradient, which is then used to update the model.

2. The parameter-stabilized transfer learning method for medical image classification according to claim 1, characterized in that, To achieve the gradient of the target domain gradient to source domain The initial guidance of the gradient direction is achieved by constraining the gradient direction in the following two ways: One approach is to incorporate the gradient of the target domain as a momentum term into the gradient update process of the source domain during the source domain gradient update process. Secondly, it learns the source-to-gradient mapping paradigm through a multilayer perceptron model.

3. The parameter-stabilized transfer learning method for medical image classification according to claim 2, characterized in that, The constraints are specifically: (1) Apply mini-batch stochastic gradient descent to update the gradient of the source domain. Weights, source domain weight Calculate as follows: ； ； in This is the current batch round number. It's the learning rate. It is the objective function of the model. Represents the gradient; (2) Incorporate the gradient of the target domain as a momentum term into the gradient update process of the source domain: ； ； in It is the first Batch target gradient Momentum; (3) Gradient mapping model, used to ensure the gradient of the source domain Able to be converted into target gradient And achieve the target gradient To further ensure the gradient of the source domain The guiding role; For convolutional neural network models, it is assumed that the gradient of any convolutional layer is... G ,but G Each row corresponds to a gradient update direction. If the gradients of two rows are equal, they will be updated along the same direction. For source domain data and target domain data ,like and If the results are similar and meet the set requirements, then the following optimization is performed using the mean squared error loss function: ； in, The gradient is represented in vector form. Since gradient mapping is a non-linear mapping, a multilayer perceptron model is used for gradient mapping, given the model weights. = { ,..., }and ={ ,..., The ultimate optimization goal is to minimize the mean squared error loss function argmin_MSEloss of the multilayer perceptron model parameters.

4. The parameter-stabilized transfer learning method for medical image classification according to claim 2, characterized in that, The function argmin_MSEloss, which minimizes the mean squared error loss, is: in N It represents the total number of convolutional layers.

5. The parameter-stabilized transfer learning method for medical image classification according to claim 3, characterized in that, The model is updated using a weighted average gradient G, where This stage uses a weighted average gradient G and source domain data. Optimize the model.

Images