Large model fine-tuning optimization method based on multi-strategy fusion

Abstract

The invention discloses a large model fine tuning optimization method based on multi-strategy fusion, which comprises the following steps: designing a dynamic parameter selection mechanism, adaptively determining a parameter subset needing fine tuning according to a task demand and a model structure, and reducing unnecessary parameter updating calculation; constructing a dynamic low-rank decomposition framework, dynamically adjusting the rank of a low-rank matrix according to a model training state and data characteristics, and keeping key information while compressing a parameter scale; a self-adaptive task sensing mechanism is introduced, a fine adjustment strategy is automatically adjusted according to different task characteristics, and the adaptability of the model to various tasks is improved; and a mixed precision training method is adopted, so that the calculation complexity and the memory occupation are reduced on the premise of ensuring the model precision. According to the method, a parameter efficient fine tuning technology and a dynamic low-rank decomposition strategy are innovatively combined, and an adaptive task perception mechanism and a mixed precision training technology are introduced, so that the operand and resource requirements of model training are effectively reduced, and the fine tuning efficiency and the model performance are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a method for fine-tuning optimization of large models, which reduces the computational load and resource consumption of model training through parameter efficient fine-tuning technology and low-rank decomposition strategy. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, large models have shown strong performance in natural language processing, computer vision and many other fields. However, the increasing size and complexity of large models have brought high training costs and resource demands. In practical applications, in order to make large models better adapt to specific tasks, fine-tuning has become a common means. However, the traditional fine-tuning method needs to update a large number of parameters, resulting in huge computational load, and in the case of limited hardware resources, the memory bottleneck problem is prominent.

[0003] Although the currently introduced parameter efficient fine-tuning technology has alleviated the parameter update pressure to some extent, there is still room for optimization. At the same time, the method of reducing the size of matrix parameters by using low-rank decomposition does not fully consider the diversity of model structure and task characteristics in practical applications, resulting in less than ideal optimization effect. The existing technology lacks a systematic large model fine-tuning optimization scheme that can comprehensively consider multiple factors, making it difficult to significantly improve the fine-tuning efficiency and reduce resource consumption while maintaining model performance. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings of the prior art and provide a large model fine-tuning optimization method based on multi-strategy fusion, which effectively reduces the computational load and resource demand of model training, improves the fine-tuning efficiency and model performance by innovatively combining parameter efficient fine-tuning technology and dynamic low-rank decomposition strategy, and introducing adaptive task perception mechanism and mixed precision training technology.

[0005] The technical solution adopted by the present application is: a large model fine-tuning optimization method based on multi-strategy fusion, comprising the following steps: Step 1: Design a dynamic parameter selection mechanism to adaptively determine the parameter subset to be fine-tuned according to task requirements and model structure, reducing unnecessary parameter update calculations; Step 2: Construct a dynamic low-rank decomposition framework to dynamically adjust the rank of the low-rank matrix according to the model training state and data characteristics, compressing the parameter size while preserving key information; Step 3: Introduce an adaptive task perception mechanism to automatically adjust the fine-tuning strategy according to different task characteristics, improving the adaptability of the model to various tasks; Step 4: Use a mixed precision training method to reduce computational complexity and memory usage while ensuring model accuracy.

[0006] Furthermore, the dynamic parameter selection mechanism in step 1 is specifically as follows: For a given large model M, its parameter set is: ;θ n Let M be the nth trainable parameter in the parameter set of the large model; define a task-related parameter importance metric function. , where θ i Let M be the i-th trainable parameter in the parameter set of the large model, and T represent a specific task. This function evaluates the importance of the parameter by analyzing the correlation between the parameter and the task objective, for example, by calculating the sensitivity of the parameter to the task loss function. Set a dynamic threshold based on the importance measurement results. Select a subset of parameters whose importance is higher than a threshold. , For parameter importance measurement function, The parameter selection criteria based on task T are determined by the parameter importance measurement results, and are applied only to... The parameters in the calculation are fine-tuned to reduce the amount of computation.

[0007] Furthermore, the dynamic low-rank decomposition framework in step 2 is specifically as follows: The weight matrix W that needs fine-tuning in the model is dynamically decomposed into a low-rank value. , where U and V are low-rank matrices, and T represents a specific task; during training, the rank r of the low-rank matrix is ​​dynamically adjusted according to the training progress of the model, changes in data features, and task requirements; Define the rank adjustment function ,in This represents the loss value for the current training step. Let r be the variance of the current data features, and T be the task type. For example, when the loss value decreases slowly and the variance of the data features is large, the rank r can be increased appropriately to retain more information; when the loss value decreases rapidly and the data features are relatively stable, the rank r can be decreased to further reduce the computational load.

[0008] Furthermore, the adaptive task awareness mechanism in step 3 specifically includes: Construct a task feature extractor to extract task feature vectors from aspects such as task description and training data distribution. By training a task classifier, the task feature vector is mapped to the corresponding fine-tuning policy space, and the appropriate parameter selection threshold, initial value of low-rank decomposition rank, and precision configuration in mixed precision training are automatically selected for the task. For example, for text classification tasks, since the data features are relatively concentrated, a lower parameter selection threshold and a smaller initial low-rank decomposition rank can be used; while for image generation tasks, due to the complexity of the data features, a higher parameter selection threshold and a larger initial low-rank decomposition rank may be required.

[0009] Furthermore, the hybrid precision training method in step 4 is specifically as follows: During model training, model parameters and intermediate calculation results are divided into two parts: single-precision (FP32) and half-precision (FP16). For parts with high precision requirements, such as key weight layers of the model and important steps in gradient calculation, FP32 is used for calculation; for parts with relatively low precision requirements, such as activation function calculation and some intermediate matrix operations, FP16 is used for calculation. By setting precision switching conditions, such as the numerical range of the calculation results and their impact on the model output, the calculation precision can be dynamically adjusted, thereby reducing computational complexity and memory usage while ensuring model accuracy.

[0010] The beneficial effects of this invention are: 1. Significantly reduced computational load: Through dynamic parameter selection mechanism and dynamic low-rank decomposition framework, the number of parameters that need to be updated and the scale of matrix operations are reduced. Compared with traditional fine-tuning methods, the computational load can be reduced by 50% - 80%.

[0011] 2. Improved fine-tuning efficiency: The adaptive task awareness mechanism enables the model to quickly adapt to different tasks, reducing the workload and time cost of manual parameter tuning. At the same time, the mixed precision training technology further accelerates the training process, improving the overall fine-tuning efficiency by 2 to 4 times.

[0012] 3. Effectively saves resources: Reduces memory usage and computing resource requirements, enabling efficient fine-tuning of large models even under limited hardware resources. For example, large-scale model fine-tuning can be achieved even in a single GPU environment.

[0013] 4. Maintain model performance: While reducing computation and resource consumption, the model's performance on various tasks can be maintained through reasonable strategy design. Compared with full parameter fine-tuning, the accuracy loss can be controlled within 1%. Attached Figure Description

[0014] Figure 1 The overall flowchart of this invention. Detailed Implementation

[0015] The present invention will be further described below with reference to the accompanying drawings and specific embodiments: like Figure 1As shown, a large-scale model fine-tuning optimization method based on multi-strategy fusion includes the following steps: I. Model Preparation Stage Choose a large model M to be fine-tuned, assuming it has L layers, each containing multiple weight matrices. , , , The number of weight matrices in the l-th layer; Define a parameter importance measurement function related to the task. ,For example: in Let T be the loss function corresponding to task T; Initialize the task feature extractor and task classifier. The task feature extractor can use structures such as convolutional neural networks (CNN) or recurrent neural networks (RNN) to extract features from the task description and training data distribution. The task classifier can use multilayer perceptron (MLP) to map the extracted feature vectors to the fine-tuning policy space. II. Dynamic Parameter Selection Stage For each task T, calculate each parameter. Importance ; Based on the calculated importance, the dynamic threshold is calculated using the following formula. : in This is an adjustable parameter with a value range of [0.5, 1], which can be adjusted according to the complexity of the task and the sensitivity to parameter updates; Filter out a subset of parameters whose importance is higher than a threshold. ,Right now: If and only if ; III. Dynamic Low-Rank Decomposition Stage For the weight matrix that needs fine-tuning The initial low-rank decomposition is performed using singular value decomposition (SVD), i.e. Then take the front Approximating the parts corresponding to the singular values, we obtain ,in The initial rank can be set based on the output of the task classifier; During training, according to the rank adjustment function Dynamically adjust the rank r; for example, a simple linear adjustment function can be used as follows: in and The coefficients can be adjusted and optimized based on experiments. This represents the loss value for the current training step. The characteristic variance of the current data; When the rank r changes, the weight matrix is ​​re-decomposed into a low-rank matrix to update the U and V matrices; IV. Mixed Precision Training Phase Define precision switching conditions, for example, for matrix multiplication operations. ,when and When FP16 is used for calculation; when When the result is converted to FP32, it will be used for subsequent calculations; where and The threshold is adjustable and can be set according to the specific circumstances of the model; During training, the precision of model parameters and intermediate calculation results is managed according to the precision switching condition. For example, when calculating gradients, for important parameter layers, FP32 is used to calculate gradients first, and then it is decided whether to switch to FP16 for subsequent parameter update calculations based on the precision switching condition. The selected subset of parameters is updated using an optimizer (such as AdamW), and the update formula is as follows: in For parameters The value at step t, For learning rate, For parameters For loss function gradient, and Estimate the first and second moments in the AdamW optimizer; V. Training Process Load the training data and input it into the model for forward propagation calculation. Select an appropriate precision for calculation based on the mixed precision training strategy. Calculate the loss function corresponding to task T. And perform precision management on the loss value according to the precision switching conditions; Backpropagation is performed, and the subset of parameters that need to be updated is determined according to the dynamic parameter selection mechanism. The appropriate precision is selected for gradient calculation according to the mixed precision training strategy. The weight matrix is ​​updated according to the dynamic low-rank decomposition strategy, and the parameter subset is updated according to the optimizer.

[0016] Repeat the above steps until the training termination condition is met (such as the loss value converges, the maximum number of training steps is reached, etc.).

[0017] The proposed large-model fine-tuning optimization method based on multi-strategy fusion effectively solves the problems of high computational cost and resource consumption during large-model fine-tuning by leveraging the synergistic effects of a dynamic parameter selection mechanism, a dynamic low-rank decomposition framework, an adaptive task-aware mechanism, and mixed-precision training techniques. Experiments show that this method can significantly improve fine-tuning efficiency and reduce resource requirements under various large models and different task scenarios, while maintaining the high performance of the model. It provides strong support for the efficient deployment of large models in practical applications and has broad application prospects and promotional value.

[0018] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-strategy fusion-based large model fine-tuning optimization method, characterized in that: The method comprises the following steps: Step 1: design a dynamic parameter selection mechanism to adaptively determine the parameter subset to be fine-tuned according to task requirements and model structure, and reduce unnecessary parameter update calculation; Step 2: build a dynamic low-rank decomposition framework to dynamically adjust the rank of the low-rank matrix according to the model training state and data characteristics, compress the parameter size while maintaining key information; Step 3: introduce an adaptive task-aware mechanism to automatically adjust the fine-tuning strategy according to different task characteristics, and improve the adaptability of the model to various tasks; Step 4: use a mixed precision training method to reduce computational complexity and memory usage while ensuring model accuracy.

2. The method of claim 1, wherein the method is characterized by: The dynamic parameter selection mechanism in step 1 is specifically: For a given large model M, its parameter set is , θ n is the n-th trainable parameter of the parameter set of the large model M; define a task-dependent parameter importance measure function , where θ i is the i-th trainable parameter of the parameter set of the large model, and T represents a specific task; The function evaluates the importance of parameters by analyzing the correlation between parameters and task objectives; According to the importance measurement result, set a dynamic threshold , filter out a parameter subset with importance higher than the threshold , For the parameter importance measurement function, Based on the parameter filtering criteria of task T, only the parameters in are fine-tuned, thereby reducing the computational load.

3. The method of claim 2, wherein the method is characterized by: In step 1, the sensitivity of parameters to the task loss function is calculated.

4. The method of claim 3, wherein the method is characterized by: The dynamic low-rank decomposition framework in step 2 is specifically: The weight matrix W to be fine-tuned in the model is dynamically low-rank decomposed, that is wherein U and V are low-rank matrices, and T represents a specific task; during the training process, the rank r of the low-rank matrix is dynamically adjusted according to the training progress of the model, the data feature change, and the task requirement; Defining a rank adjustment function wherein is a loss value of a current training step, is a feature variance of the current data, and T is a task type.

5. The method of claim 4, wherein the method is characterized by: When the loss value decreases slowly and the data feature variance is large, increase the rank r appropriately to retain more information; when the loss value decreases rapidly and the data features are stable, reduce the rank r to further reduce the computational load.

6. The method of claim 5, wherein the method is characterized by: The adaptive task-aware mechanism in step 3 is specifically: Construct a task feature extractor to extract a task feature vector from the task description and the training data distribution ; by training a task classifier, map the task feature vector to the corresponding fine-tuning strategy space, automatically select the parameter selection threshold, the initial value of the low-rank decomposition rank, and the precision configuration in mixed precision training suitable for the task.

7. The method of claim 6, wherein the method is characterized by: For text classification tasks, due to the relatively concentrated data features, a lower parameter selection threshold and a smaller initial low-rank decomposition rank are used; for image generation tasks, due to the complexity of data features, A higher parameter selection threshold and a larger initial low-rank decomposition rank are required.

8. The method of claim 7, wherein the method is characterized by: The mixed precision training method in step 4 is specifically: During model training, model parameters and intermediate calculation results are divided into FP32 and FP16 two parts; for parts with higher accuracy requirements, such as key weight layers of the model and important steps in gradient calculation, use FP32 for calculation; for parts with relatively low accuracy requirements, such as activation function calculation and part of intermediate matrix operation, use FP16 for calculation; By setting precision switching conditions, such as according to the numerical range of the calculation result and the degree of influence on the model output, the calculation precision is dynamically adjusted to reduce the computational complexity and memory usage while ensuring the model accuracy.

Images