Target task adaptive parameter optimization transfer learning method oriented to sparse model parameters

Abstract

The invention discloses a sparse model parameter-oriented target task adaptive parameter optimization transfer learning method, which is suitable for medical image classification and model optimization training. The classification contribution degree of each convolution kernel in a pre-trained CNN model to each target category of a target domain is calculated based on a feature attribution method, the convolution kernels are sorted according to the classification contribution degree, the convolution kernels with the correlation degree larger than a threshold gamma are screened out for optimization and fine tuning, and other parameters are frozen; and self-adaptive low-rank fine tuning: performing fine tuning on the frozen convolution kernel parameters by adopting low-rank adapters, including calculating function values to determine the rank of each adapter, and performing self-adaptive low-rank fine tuning on the low-correlation convolution kernels by allocating different ranks.

Description

Technical Field

[0001] This invention designs a parameter-guided transfer learning method for adaptive optimization of parameters in the field of deep learning image analysis, particularly for tasks with sparse parameters. The product is mainly used for fine-tuning medical image classification. Background Technology

[0002] Traditional deep learning-based medical image classification methods often suffer from unstable performance due to limitations such as insufficient target domain data, scarcity of labeled data, and the large number of parameters in deep models. Three main fine-tuning strategies exist in the medical image domain: selective layer fine-tuning, regularization, and model adaptation. Selective transfer improves transfer performance by choosing more relevant layers, but may lead to the loss of feature information from specific layers. Regularization improves fine-tuning by limiting the parameter similarity between the pre-trained model and the fine-tuned model, but the degree of regularization is difficult to control. Model adaptation requires structural adjustments to the transfer model, which improves its adaptability to the target domain task but increases complexity and may reduce fine-tuning efficiency. Therefore, we propose a task-adaptive optimization parameter transfer learning method oriented towards parameter sparsity.

[0003] Furthermore, it is pointed out that existing adaptive parameter optimization transfer learning methods for target tasks based on parameter sparsity face two bottleneck problems that urgently need to be solved: the limited number of samples in the target domain and the redundancy of model parameters. Existing methods, such as regularization-based methods, selective fine-tuning methods, and model distillation methods, can initially solve this problem. However, due to limitations such as the difficulty in controlling the degree of regularization, the potential loss of unique information in some layers or parameters due to selective fine-tuning, and insufficient data, the classification performance is unstable. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to propose a target task adaptive parameter optimization transfer learning method for sparse model parameters, which effectively alleviates the problems of limited target domain samples and redundant model parameters, improves the efficiency of fine-tuning, and achieves optimal optimization of model parameters.

[0005] Technical Solution: A target task adaptive parameter optimization transfer learning method for sparse model parameters, applicable to medical image classification and model optimization training, the method comprising: Target optimization-guided fine-tuning: Based on feature attribution methods, the classification contribution of each convolutional kernel in the pre-trained CNN model to each target category in the target domain is calculated. The convolutional kernels are then ranked according to their classification contributions, and those with a correlation greater than a threshold are selected. γ The convolution kernels are optimized and fine-tuned, while the remaining parameters are frozen; Adaptive low-rank fine-tuning: The frozen convolution kernel parameters are fine-tuned using low-rank adapters, including calculating the function value to determine the rank of each adapter, and adaptive low-rank fine-tuning is performed by assigning different ranks to low-correlation convolution kernels.

[0006] Furthermore, the target optimization-guided fine-tuning includes the following operations: For the The feature mapping of the convolutional kernel is processed through operations including convolution, pooling, activation functions, and fully connected layers to calculate the predicted class pair. k The first derivative of each feature map, the magnitude of which characterizes the degree of influence of the feature value on the predicted classification; The correlation value between the convolution kernel and the class to be predicted is calculated based on the first derivative. The magnitude of the correlation value represents the contribution of the convolution kernel to the predicted class. Based on the contribution of each convolutional kernel to the predicted classification, the correlation between the convolutional kernel and all classes in the target domain is calculated, including the correlation between the number of convolutional kernel layers n and the number of convolutional kernels in each convolutional layer. k Obtain the activation value matrix of the convolutional kernels of the entire CNN model, calculate the proportion of the activation value of each convolutional kernel in the activation value matrix, and obtain the contribution of each convolutional kernel to the classification of the target domain based on the product of the activation value and the proportion. The contribution is the correlation with the classification of the target domain.

[0007] Furthermore, for a convolutional neural network model pre-trained on the source dataset, feature attribution is used to find the match with the target dataset. Convolutional kernels relevant to each category are selected to be compatible with the target domain dataset. Each category-related convolutional kernel, where Represents the source domain. Indicates the target domain.

[0008] For the Layer convolution kernel is represented as ,total K There are , and the feature map corresponding to each convolutional kernel is represented as . Predicted categories , This represents operations including convolution, pooling, activation functions, and fully connected layers; No. k Feature mapping The eigenvalues ​​of the middle feature for predicting the category The degree of influence is calculated using the first derivative, and the calculation expression is: ; Calculate the convolution kernel For prediction categories The overall contribution, as defined by relevant values. The higher the correlation value, the greater the influence of the convolutional kernel on classification. Its expression is: ; Finally, based on the correlation values ​​of each convolutional kernel Convolutional kernels with a correlation greater than the threshold γ are selected as parameters that are more relevant to the target and classification.

[0009] The method expresses the contribution of the convolution kernel to the target domain as... : Represents the convolution kernel. This represents the activation value (weight) corresponding to the convolution kernel. It is the number of convolutional layers. It is the number of convolutional kernels in each convolutional layer. Indicates the number of relevant values; Furthermore, the adaptive low-rank fine-tuning specifically includes a measurement of the functionality of different convolutional layers, specifically including the following operations: The importance of each convolutional kernel is defined as the absolute value of the product of the gradient and the activation value, and then the importance of each convolutional layer is calculated. Calculate the variability of each convolutional layer, whereby variability is defined as the amount of change in the parameters of the current batch fine-tuned model relative to the source model; Calculate the functionality of each convolutional layer, defined as the absolute value of the product of the importance and variability of each convolutional layer.

[0010] Furthermore, the adaptive low-rank fine-tuning is based on the functionality of different convolutional layers, determining the rank of each convolutional layer, including the parameter dimensions of the pre-trained model. The low-rank decomposition is constructed, and the corresponding computational expression is:

[0011] In the formula, and They are two low-rank matrices. Used for fine-tuning bypass.

[0012] The number of parameters generated by each convolutional layer during fine-tuning depends only on the size of the convolutional kernel and the number of input channels. and number of output channels Assuming the functionality of different convolutional layers is as follows: Then the rank of the relevant convolutional layer Calculated using the following formula:

[0013] During fine-tuning, only the low-rank matrices A and B are updated, according to... and The parameters of the final fine-tuned model are calculated to obtain the fine-tuning results.

[0014] A medical image classification parameter optimization and fine-tuning system based on the above-mentioned target task adaptive parameter optimization transfer learning method for sparse model parameters includes: Data input module: used to input the source domain pre-trained dataset, the target domain medical image dataset, and the pre-trained CNN model; The target optimization guidance module is used to execute the target optimization guidance fine-tuning process in the method, including screening important convolutional kernels and fine-tuning them, and freezing the remaining parameters; Adaptive low-rank fine-tuning module: used to execute the adaptive low-rank fine-tuning process in the method, including calculating the function value of the convolutional layer, performing low-rank decomposition and parameter updating; Results output module: Used to output the fine-tuned model and medical image classification results.

[0015] Beneficial effects: The target task adaptive parameter optimization transfer learning method for sparse model parameters proposed in this invention can effectively alleviate the problems of limited target domain samples and redundant model parameters, improve fine-tuning efficiency, achieve optimal optimization of model parameters, and thus improve the accuracy and stability of medical image classification. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the logical framework structure of the method described in this invention. Detailed Implementation

[0017] To illustrate the technical solution provided by this invention in detail, further description is provided below with reference to the accompanying drawings.

[0018] In this invention, we propose a parameter optimization and fine-tuning method for medical image classification, applicable to the medical image field. This method consists of a target-oriented optimization-guided fine-tuning module and an adaptive low-rank fine-tuning module. First, the method calculates the classification contribution of each convolutional kernel to each target category based on a feature attribution method. Based on its overall contribution to the target domain classification, the convolutional kernels are sorted and selected to optimize and fine-tune the parameters, while the remaining parameters are frozen to improve fine-tuning efficiency. Then, based on this, a low-rank adapter is used to fine-tune the remaining (frozen) parameters. Importance and variability are then calculated, and their multiplication yields the function value. Based on the important convolutional kernel parameters selected in the first stage of fine-tuning, the remaining bias parameters are supplemented and subjected to low-rank fine-tuning to improve the final fine-tuning effect.

[0019] The above technical solution comprises the following two stages: (1) Target optimization guides fine-tuning Feature-based attribution methods extract important convolutional kernels from pre-trained models, and model attribution methods are used to select convolutional kernels that are more relevant to the target domain classification. For the source dataset... The pre-trained CNN is used to find convolutional kernels that are relevant to each category of the target dataset through feature attribution. The specific steps are as follows: 1) Calculate the classification contribution of each convolutional kernel to each target class. For the feature mapping of the r-th convolutional kernel, it is calculated using F (where F represents operations such as convolution, pooling, activation function, fully connected layer, etc.), and the predicted class is obtained for the r-th layer. The first derivative of each eigenmap The calculation is as follows:

[0020] The magnitude of the derivative indicates the degree of influence of the feature value on the classification. 2) Calculate the convolution kernel For prediction categories Relevant values :

[0021] in, This represents all feature values, i.e., the number of feature values ​​obtained. The magnitude of the correlation value indicates the contribution of the convolution kernel to the category; the larger the correlation value, the greater the influence on the classification. 3) Selecting important convolutional kernels Based on the contribution of each convolutional kernel to the classification, the correlation between the convolutional kernel and all classes in the target domain is calculated, and convolutional kernels with a correlation greater than the threshold γ are selected as parameters with a higher correlation to the classification of the target domain.

[0022] Assume the activation value (weight) of the convolution kernel is The contribution of the convolution kernel to the target domain can then be calculated as follows:

[0023] Where n is the number of convolutional kernel layers used to calculate the DataMap. It represents the number of convolutional kernels in each convolutional layer.

[0024] (2) Adaptive low-rank fine-tuning

[0025] Based on the target optimization-guided fine-tuning, a low-rank adapter is used to fine-tune the remaining (frozen) parameters. The rank of each adapter is determined by the function of its corresponding layer and is measured by importance and variability.

[0026] In fine-tuning CNNs, after individually fine-tuning the parameters most relevant to the task, bias parameters less relevant to the target domain can be efficiently fine-tuned using a low-rank approach. The functionality of different convolutional layers for classification can be measured by defining the importance and variability of the parameters, and adaptive low-rank fine-tuning can be performed on low-relevance convolutional kernels by assigning different ranks.

[0027] The specific steps are as follows: 1) Measurement of the functionality of different convolutional layers: In CNN models, this invention uses a single parameter The importance of a convolution kernel is defined as the absolute value of the product of the gradient and the weights, expressed as:

[0028] in These are gradients that can be used for training. These are the gradient weights; Assuming each convolutional layer has K convolutional kernels, the importance of each convolutional layer... Defined as:

[0029] Calculate the variability of each layer, that is, the amount of change in the parameters of the current batch fine-tuning model relative to the source model. The expression is Where T and S represent the target domain and the source domain, respectively; The functionality of each convolutional layer is calculated using importance and variability:

[0030] 2) Adaptive low-rank fine-tuning for different convolutional layers: The number of parameters generated by each convolutional layer during fine-tuning depends only on the kernel size (Width and Height) and the number of input channels. and number of output channels .

[0031] First, the size is 3 The 3 convolution kernel is divided into 3 1 and 1 The three parts initially reduce the size of the training parameters; Assume the parameter dimension of the pre-trained model is . Then a low-rank decomposition can be constructed:

[0032] in, and They are two low-rank matrices. Used for fine-tuning bypass, ,

[0033] Assume the functionality of different convolutional layers is as follows: Then the rank of the relevant convolutional layer Calculated using the following formula:

[0034] During fine-tuning, only the low-rank matrices A and B are updated, according to... and The parameters of the final fine-tuned model are calculated to obtain the fine-tuning results.

[0035] Table 1. Comparison results with mainstream methods in ResNet50

Claims

1. A target task adaptive parameter optimization transfer learning method for sparse model parameters, applicable to medical image classification and model optimization training, characterized in that, The method includes: Target optimization-guided fine-tuning: Based on feature attribution methods, the contribution of each convolutional kernel in the pre-trained CNN model to the classification of the target domain is calculated. The convolutional kernels are then ranked according to their classification contributions, and those with a correlation greater than a threshold are selected. γ The convolution kernels are optimized and fine-tuned, while the remaining parameters are frozen; Adaptive low-rank fine-tuning: The frozen parameters are fine-tuned using low-rank adapters, including calculating the function value to determine the rank of each adapter, and adaptive low-rank fine-tuning is performed by assigning different ranks to low-correlation convolution kernels.

2. The target task adaptive parameter optimization transfer learning method for sparse model parameters according to claim 1, characterized in that, The target optimization-guided fine-tuning includes the following operations: For the The feature mapping of the convolutional kernel is processed through operations including convolution, pooling, activation functions, and fully connected layers to calculate the predicted class pair. k The first derivative of each feature map, the magnitude of which characterizes the degree of influence of the feature value on the predicted classification; The correlation value between the convolution kernel and the class to be predicted is calculated based on the first derivative. The magnitude of the correlation value represents the contribution of the convolution kernel to the predicted class. Based on the contribution of each convolutional kernel to the predicted classification, the correlation between the convolutional kernel and all classes in the target domain is calculated, including the correlation between the number of convolutional kernel layers n and the number of convolutional kernels in each convolutional layer. k Obtain the activation value matrix of the convolutional kernels of the entire CNN model, calculate the proportion of the activation value of each convolutional kernel in the activation value matrix, and obtain the contribution of each convolutional kernel to the classification of the target domain based on the product of the activation value and the proportion. The contribution is the correlation with the classification of the target domain.

3. The target task adaptive parameter optimization transfer learning method for sparse model parameters according to claim 2, characterized in that, For a convolutional neural network model pre-trained on the source dataset, feature attribution is used to find the match with the target dataset. Convolutional kernels relevant to each category are selected to be compatible with the target domain dataset. Each category-related convolutional kernel, T represents the source domain, and T represents the target domain; For the Layer convolution kernel is represented as ,total K There are , and the feature map corresponding to each convolutional kernel is represented as . Predicted categories , This represents operations including convolution, pooling, activation functions, and fully connected layers; No. k Feature mapping The eigenvalues ​​of the middle feature for predicting the category The degree of influence is calculated using the first derivative, and the calculation expression is: ； Calculate the convolution kernel based on the degree of influence. For prediction categories The overall contribution, as defined by relevant values. The higher the correlation value, the greater the influence of the convolutional kernel on classification. Its expression is: ； Finally, based on the correlation values ​​of each convolutional kernel Convolutional kernels with a correlation greater than the threshold γ are selected as parameters that are more relevant to the target and classification.

4. The target task adaptive parameter optimization transfer learning method for sparse model parameters according to claim 2 or 3, characterized in that, The contribution of the convolution kernel to the target domain is expressed as: : Represents the convolution kernel. This represents the activation value corresponding to the convolution kernel. It is the number of convolutional layers. It is the number of convolutional kernels in each convolutional layer. This indicates the number of related values.

5. The target task adaptive parameter optimization transfer learning method for sparse model parameters according to claim 1, characterized in that, The adaptive low-rank fine-tuning specifically includes a measurement of the functionality of different convolutional layers, specifically including the following operations: The importance of each convolutional kernel is defined as the absolute value of the product of the gradient and the activation value, and then the importance of each convolutional layer is calculated. Calculate the variability of each convolutional layer, whereby variability is defined as the amount of change in the parameters of the current batch fine-tuned model relative to the source model; Calculate the functionality of each convolutional layer, defined as the absolute value of the product of the importance and variability of each convolutional layer.

6. The target task adaptive parameter optimization transfer learning method for sparse model parameters according to claim 5, characterized in that, The adaptive low-rank fine-tuning is based on the functionality of different convolutional layers, determining the rank of each layer, including the parameter dimensions of the pre-trained model. The low-rank decomposition is constructed, and the corresponding computational expression is: In the formula, and They are two low-rank matrices. Used for fine-tuning bypass.

7. The target task adaptive parameter optimization transfer learning method for sparse model parameters according to claim 6, characterized in that, The number of parameters generated by each convolutional layer during fine-tuning depends only on the size of the convolutional kernel and the number of input channels. and number of output channels Assuming the functionality of different convolutional layers is as follows: Then the rank of the relevant convolutional layer Calculated using the following formula: During fine-tuning, only the low-rank matrices A and B are updated, according to... and The parameters of the final fine-tuned model are calculated to obtain the fine-tuning results.

8. A medical image classification parameter optimization and fine-tuning system implemented using the target task adaptive parameter optimization transfer learning method for sparse model parameters as described in any one of claims 1-7, characterized in that, include: Data input module: used to input the source domain pre-trained dataset, the target domain medical image dataset, and the pre-trained CNN model; The target optimization guidance module is used to execute the target optimization guidance fine-tuning process in the method, including screening important convolutional kernels and fine-tuning them, and freezing the remaining parameters; Adaptive low-rank fine-tuning module: used to execute the adaptive low-rank fine-tuning process in the method, including calculating the function value of the convolutional layer, performing low-rank decomposition and parameter updating; Results output module: Used to output the fine-tuned model and medical image classification results.

Images