Efficient Parameter Fine-Tuning Method and Apparatus Based on Dynamic Low-Rank Matrix Adaptation

Abstract

This invention provides a method and apparatus for efficient parameter fine-tuning based on dynamic low-rank matrix adaptation, relating to the field of neural network technology. The method includes: freezing the original weights of the neural network for the current business scenario; inserting low-rank matrix pairs into the fine-tunable layer of the neural network to determine the current weights of the fine-tunable layer; processing the input sample data based on the current weights through the neural network to obtain the current validation loss; updating the low-rank matrix pairs using a slot machine-driven controller based on a compressible candidate rank set, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable layer, repeating this process until a condition is met; and determining the target weights of the fine-tunable layer based on the updated low-rank matrix pairs and the frozen original weights. This invention can alleviate the overfitting or underfitting problems caused by fixed-rank methods in small-sample fine-tuning scenarios.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to a method and apparatus for efficient parameter fine-tuning based on dynamic low-rank matrix adaptation. Background Technology

[0002] With the widespread application of deep learning in natural language processing, fine-tuning methods based on pre-trained language models (PLMs) have become the mainstream technical approach for classification tasks such as text, audio, and video. However, in specialized fields such as medicine and finance, due to high data acquisition costs and high annotation thresholds, there are often few-shot learning scenarios, where the amount of training data is extremely limited.

[0003] In existing technologies, Low-Rank Adaptation (LoRA) methods adjust the parameters of pre-trained models by introducing trainable low-rank matrices, achieving efficient fine-tuning while keeping most parameters frozen. A typical implementation involves injecting trainable rank decomposition matrices into each layer of the Transformer architecture, optimizing these low-rank matrices to adapt to downstream tasks. Traditional LoRA methods typically use a fixed rank size (e.g., rank 8 or 16), a design that effectively balances model capacity and generalization ability when data is plentiful.

[0004] However, setting a fixed rank presents a dual challenge in small-sample scenarios: when the rank is set too high, the model is prone to overfitting due to capturing noise in the training data; when the rank is set too low, the model's expressive power is insufficient, leading to underfitting. This contradiction is particularly prominent in professional fields such as medicine and finance, because semantic understanding of specialized terms often requires more refined parameter adjustments, but the available training samples are extremely limited. Current technologies do not provide an adaptive rank adjustment mechanism for small-sample scenarios, making model performance highly dependent on manually set fixed rank values. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a parameter-efficient fine-tuning method and apparatus based on dynamic low-rank matrix adaptation, which can alleviate the overfitting or underfitting problems caused by the fixed-rank method in small sample fine-tuning scenarios.

[0006] In a first aspect, the present invention provides an efficient parameter fine-tuning method based on dynamic low-rank matrix adaptation, comprising:

[0007] Step 1: Freeze the original weights of the neural network to be fine-tuned in the current business scenario. The current business scenario is one where the number of sample data is lower than a preset threshold.

[0008] Step 2: Insert low-rank matrix pairs into the fine-tunable layers of the neural network, and determine the current weights of the fine-tunable layers based on the low-rank matrix pairs and the frozen original weights.

[0009] Step 3: Using a neural network, process the input sample data based on the current weights to obtain the current validation loss;

[0010] Step 4: Using the slot machine driver controller, update the low-rank matrix pairs based on the compressible candidate rank set, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable level, and return to step 2 until the condition is met. Then, determine the target weights of the fine-tunable level based on the updated low-rank matrix pairs and the frozen original weights, so as to achieve efficient parameter fine-tuning of the neural network in the current business scenario.

[0011] In one implementation, there are multiple slot machine drive controllers, each corresponding one-to-one with a fine-tunable level. Through these slot machine drive controllers, the low-rank matrix pairs are updated based on a compressible candidate rank set, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable level, including:

[0012] For any fine-tunable level, perform the following operation:

[0013] Determine whether the current iteration step meets the first preset condition;

[0014] If so, prune and compress the candidate rank set corresponding to the fine-tunable level;

[0015] The slot machine-driven controller determines the optimal rank and its corresponding rank quality index from the pruned and compressed candidate rank set. Based on the rank quality index, the current verification loss, and the historical rank selection sequence corresponding to the fine-tunable level, it determines whether to update the low-rank matrix pair. If the determination result is yes, the low-rank matrix pair inserted at the fine-tunable level is updated using the optimal rank.

[0016] In one implementation, determining whether the current iteration step number meets a first preset condition includes:

[0017] Determine the current iteration step number and the step interval between the previous iteration step number for pruning and compressing the candidate rank set. If the step interval meets the preset interval, determine that the first preset condition is met.

[0018] The candidate rank set corresponding to the fine-tunable level is pruned and compressed, including:

[0019] For any candidate rank contained in the candidate rank set corresponding to the fine-tunable level, determine the rank quality index of the candidate rank at the current iteration step. If the average of multiple rank quality indices of the candidate rank between the current iteration step and the historical iteration step is less than the preset index threshold, determine to remove the candidate rank from the candidate rank set.

[0020] In one implementation, determining the optimal rank and its corresponding rank quality index from the pruned and compressed candidate rank set includes:

[0021] The optimal rank is determined from the pruned and compressed candidate rank set, and the corresponding multi-objective reward is determined.

[0022] Determine whether the current iteration step meets the second preset condition;

[0023] If so, update the reward statistics of the slot machine driver controller using multi-objective rewards, and determine the rank quality index corresponding to the optimal rank based on the updated reward statistics and the current gradient norm.

[0024] In one implementation, based on the rank quality index, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable level, it is determined whether to update the low-rank matrix pair, including:

[0025] The current validation loss and the historical rank selection sequence corresponding to the fine-tunable level are encoded to obtain the state vector;

[0026] The optimal rank and historical rank are evaluated based on the rank quality index corresponding to the optimal rank and the rank quality index corresponding to the historical rank of the low-rank matrix, combined with the state vector.

[0027] If the evaluation results indicate that the optimal rank is superior to the historical rank, then the optimal rank is used to update the low-rank matrix pair.

[0028] In one implementation, updating the currently inserted low-rank matrix pair at the fine-tunable level using the optimal rank includes:

[0029] Singular value decomposition is performed on the current incremental weights based on the optimal rank to obtain new low-rank matrix pairs; where the current incremental weights are determined based on the low-rank matrix pairs currently inserted at this fine-tunable level.

[0030] Based on the buffering mechanism, the currently inserted low-rank matrix pair at the fine-tunable level is replaced with a new low-rank matrix pair.

[0031] In one implementation, based on a buffering mechanism, the currently inserted low-rank matrix pair for the fine-tunable level is replaced with a new low-rank matrix pair, including:

[0032] Buffer weights are determined based on a preset number of steps;

[0033] Using buffer weights, the target low-rank matrix in the new low-rank matrix pair and the target low-rank matrix in the currently inserted low-rank matrix pair at the fine-tunable level are weighted and fused.

[0034] The target low-rank matrix obtained by weighted fusion, and another low-rank matrix in the new low-rank matrix pair, are used to replace the currently inserted low-rank matrix pair at the fine-tunable level.

[0035] Secondly, the present invention also provides a parameter-efficient fine-tuning device based on dynamic low-rank matrix adaptation, comprising:

[0036] The weight freezing module is used to freeze the original weights of the neural network that needs to be fine-tuned in the current business scenario. The current business scenario is a business scenario where the number of sample data is lower than a preset threshold.

[0037] The matrix insertion module is used to insert low-rank matrix pairs into the fine-tunable layers of the neural network, and determines the current weights of the fine-tunable layers based on the low-rank matrix pairs and the frozen original weights.

[0038] The loss determination module is used to process the input sample data based on the current weights through a neural network to obtain the current validation loss.

[0039] The weight fine-tuning module is used to update the low-rank matrix pairs based on the compressible candidate rank set, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable level, through the slot machine-driven controller, and then call the matrix insertion module until the condition is met. Based on the updated low-rank matrix pairs and the frozen original weights, the target weights of the fine-tunable level are determined, thereby realizing efficient parameter fine-tuning of the neural network in the current business scenario.

[0040] Thirdly, the present invention also provides an electronic device including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement any of the methods provided in the first aspect.

[0041] Fourthly, the present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement any of the methods provided in the first aspect.

[0042] This invention provides a method and apparatus for efficient parameter fine-tuning based on dynamic low-rank matrix adaptation. First, the original weights of the neural network to be fine-tuned in the current business scenario are frozen. Then, low-rank matrix pairs are inserted into the fine-tunable layers of the neural network, and the current weights of the fine-tunable layers are determined based on the low-rank matrix pairs and the frozen original weights. Next, the neural network processes the input sample data based on the current weights to obtain the current validation loss. Then, a controller updates the low-rank matrix pairs based on a compressible candidate rank set, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable layers. This process is repeated until a condition is met. The target weights of the fine-tunable layers are determined based on the updated low-rank matrix pairs and the frozen original weights, thus achieving efficient parameter fine-tuning of the neural network in the current business scenario. This method proposes using a multi-armed controller to dynamically configure the low-rank matrix, thereby fine-tuning the original weights of the neural network. This invention can alleviate the overfitting or underfitting problems caused by fixed-rank methods in small-sample fine-tuning scenarios, while improving the parameter fine-tuning efficiency and task performance of the neural network.

[0043] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0044] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0045] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating an efficient parameter fine-tuning method based on dynamic low-rank matrix adaptation provided in an embodiment of the present invention;

[0047] Figure 2 This invention provides an integrated system architecture for slot machine-driven adaptive rank selection.

[0048] Figure 3 This is a detailed structural diagram of a rank selector in a slot machine drive controller provided in an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of a parameter-efficient fine-tuning device based on dynamic low-rank matrix adaptive adjustment provided in an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Currently, in small-sample fine-tuning scenarios, using fixed-rank pairs to fine-tune neural network parameters leads to overfitting or underfitting issues. Taking the Transformer model as an example, while it exhibits excellent performance in natural language processing and computer vision, its large number of parameters results in high computational and memory costs for fine-tuning. To address this issue, Parameter-Efficient Fine-Tuning (PEFT) techniques have emerged. The core idea is to freeze most of the parameters of the pre-trained model and optimize only a small number of newly added or specific module parameters to achieve efficient adaptation. Among these, the LoRA method approximates weight updates through low-rank matrix factorization, significantly reducing the number of trainable parameters. However, existing LoRA methods typically require pre-fixing the rank value, and the choice of rank has a crucial impact on model performance and efficiency: excessively high rank leads to redundant parameters, while excessively low rank may impair the model's expressive power.

[0053] For dynamic rank optimization problems, the Multi-Armed Bandit (MAB) algorithm has shown potential in hyperparameter tuning. However, the core shortcomings of existing work are: decoupling of rank selection from the training process: traditional methods treat rank as a static hyperparameter, ignoring the task adaptability requirements during training; simplistic feedback mechanism: existing MAB applications rely solely on validation accuracy as the reward signal, without incorporating parameter efficiency metrics; and inefficiency in discrete space optimization: gradient hyperparameter optimization methods are difficult to directly apply to discrete rank spaces.

[0054] Based on this, the present invention provides a parameter fine-tuning method and apparatus based on dynamic low-rank matrix adaptation, which can automatically select the optimal low-rank matrix rank configuration during the fine-tuning process in small sample fine-tuning scenarios, alleviate the overfitting or underfitting problems caused by fixed-rank methods, and improve the parameter fine-tuning efficiency and task performance of the model.

[0055] To facilitate understanding of this embodiment, a detailed description of an efficient parameter fine-tuning method based on dynamic low-rank matrix adaptation disclosed in this embodiment of the invention will be provided first. (See [link to relevant documentation]). Figure 1 The diagram shows a flowchart of an efficient parameter fine-tuning method based on dynamic low-rank matrix adaptation. This method mainly includes the following steps 1 to 4:

[0056] Step 1: Freeze the original weights of the neural network to be fine-tuned in the current business scenario. The current business scenario is one where the number of sample data is lower than a preset threshold.

[0057] For example, the current business scenario could be medical text classification, financial sentiment analysis, or image-text multimodal classification, and the neural network to be fine-tuned could be a neural network based on the Transformer architecture.

[0058] Step 2: Insert low-rank matrix pairs into the fine-tunable layers of the neural network, and determine the current weights of the fine-tunable layers based on the low-rank matrix pairs and the frozen original weights.

[0059] In this context, fine-tunable layers refer to those layers in a neural network that require parameter fine-tuning, such as self-attention layers and feedforward layers in a Transformer-based neural network. The low-rank matrix pair inserted in the first iteration can be a pre-defined initial low-rank matrix pair. Subsequent iterations insert low-rank matrix pairs obtained by performing singular value decomposition on the optimal rank selected in the previous iteration. Each low-rank matrix pair consists of two low-rank matrices. The incremental weights are determined based on these two low-rank matrices, and the current weights of the fine-tunable layer are obtained by superimposing these incremental weights with the frozen original weights.

[0060] Step 3: Using a neural network, process the input sample data based on the current weights to obtain the current validation loss.

[0061] In one example, the sample data includes input data and training labels. The input data is the text data or image data to be classified. The neural network will generate output results for the input data, which are the classification results corresponding to the text data or image data. The current validation loss is determined based on the output results and training performance.

[0062] Taking medical text classification as an example, the task objective is to classify medical texts (such as medical records, clinical notes, and research papers) into predefined categories (such as disease type, diagnosis results, and treatment stage). The input data is the text sequence to be classified (e.g., "The patient complains of headache and fever, with a body temperature of 38.5℃"), and the output is the probability distribution of the text sequence belonging to each category. Taking financial sentiment analysis as an example, the task objective is to determine the sentiment tendency (positive, negative, neutral) or the impact of a specific event (such as stock price increase or decrease) of financial texts (such as news, financial reports, and social media comments). The input data is the original financial text (e.g., "The company's Q3 profit exceeded expectations, and the stock price is expected to rise"), and the output is the probability of the original financial text belonging to each sentiment tendency. Taking image-text multimodal classification as an example, the task objective is to jointly analyze and classify images and text (such as social media posts and product descriptions) (e.g., fake information detection and product category identification). In practical applications, the sample data in the above business scenarios may be smaller than the preset threshold, which may lead to underfitting or overfitting of the corresponding neural network models. This results in poor reliability of the neural network's prediction results. For example, in medical text classification scenarios, the model may not be able to accurately predict the category of the input text sequence, and in financial sentiment analysis scenarios, the model may not be able to accurately analyze the sentiment of the input financial text, thus affecting the judgment conclusions of the corresponding users.

[0063] Step 4: Using the slot machine driver controller, update the low-rank matrix pairs based on the compressible candidate rank set, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable level, and return to step 2 until the condition is met. Then, determine the target weights of the fine-tunable level based on the updated low-rank matrix pairs and the frozen original weights, so as to achieve efficient parameter fine-tuning of the neural network in the current business scenario.

[0064] The system comprises multiple slot machine-driven controllers, each corresponding to a fine-tunable layer. The candidate rank set includes multiple candidate ranks, and the historical rank selection sequence is the set of ranks selected by the slot machine-driven controller for its corresponding fine-tunable layer during historical iterations. In one implementation, the slot machine-driven controller selects the optimal rank from the compressible candidate rank set according to a confidence upper bound strategy and determines the multi-objective reward corresponding to this optimal rank to update the reward statistics of the slot machine-driven controller. Then, it combines the current gradient norm to determine the corresponding rank quality index. Based on the rank quality index, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable layer, the low-rank matrix pairs are updated. This process is repeated until a condition is met. Based on the updated low-rank matrix pairs and the frozen original weights, the target weights of the fine-tunable layer are determined, enabling efficient parameter fine-tuning of the neural network for the current business scenario.

[0065] The present invention provides an efficient parameter fine-tuning method based on dynamic low-rank matrix adaptation. It proposes to use a multi-armed slot machine to realize the dynamic configuration of the low-rank matrix, and then fine-tun the original weights of the neural network. The present invention can alleviate the overfitting or underfitting problem caused by the fixed-rank method in the case of small sample fine-tuning, and at the same time improve the parameter fine-tuning efficiency and task performance of the neural network.

[0066] For ease of understanding, the embodiments of the present invention provide, as follows: Figure 2 This paper presents an integrated system architecture for slot machine-driven adaptive rank selection. The pre-trained Transformer model contains core components of the standard Transformer architecture for feature extraction and task prediction, specifically including: an Embedding Layer: mapping input tokens to vector representations; Self-Attention Layers: capturing contextual dependencies through a self-attention mechanism; FeedForward Layers: performing nonlinear feature transformations; and an Output Layer: generating the final prediction result. The slot machine-driven controller is the core subsystem for implementing dynamic rank selection, specifically including: a BanditDriven Rank Selector: selecting the optimal rank based on multi-objective reward signals; and LowRank Matrices: storing the parameters of the low-rank matrices for dynamically adjusting the rank. In this embodiment, the slot machine-driven controller module replaces the fixed-rank design of traditional LoRA, achieving dynamic rank optimization through closed-loop control of gradient feedback and validation loss.

[0067] Further, see Figure 3 The diagram shows a detailed structural schematic of a rank selector in a slot machine driver controller. The rank selector is the core algorithm module for dynamic rank decision-making. The rank selector includes a context encoder, a UCB (Upper Confidence Bound) decision-maker, and a reward calculator (i.e., a reward function). The context encoder encodes the current validation loss and historical rank selection sequences into a state vector. The UCB decision-maker selects candidate rank sets based on the confidence upper bound strategy. Select the optimal rank The reward calculator is used to determine the optimal rank. The corresponding multi-objective reward, gradient-based rank refinement calculator, is used to evaluate the optimal rank based on the gradient information of the low-rank matrix. The rank quality index. In this embodiment of the invention, the gradient feedback mechanism incorporates matrix gradient information into the reward calculation, solving the problem of low efficiency in the traditional Bandit algorithm for discrete rank space optimization.

[0068] Based on the above architecture, this invention provides a specific implementation of an efficient parameter fine-tuning method based on dynamic low-rank matrix adaptation:

[0069] Step 1: Freeze the original weights of the neural network to be fine-tuned in the current business scenario. Taking a neural network based on the Transformer architecture as an example, the original weights of the neural network can be preserved. Unchanged, among which For input dimensions, This is the output dimension. Optionally, you can freeze the high-contribution singular vectors corresponding to old tasks and only adjust the rank of the low-rank matrix directions relevant to the new task.

[0070] Step 2 involves inserting low-rank matrix pairs into the fine-tunable layers of the neural network, determining the current weights of each fine-tunable layer based on the low-rank matrix pairs and the frozen original weights. In one example, low-rank matrix pairs are inserted into each fine-tunable layer. and Its current rank With time step The dynamic changes mean that the current rank can be understood as the optimal rank (historical rank) selected by the slot machine-driven controller for this fine-tunable level in the previous iteration. During the forward propagation process, the incremental weights are determined based on the low-rank matrix pairs. : Incremental weights The current weights of the adjustable level are obtained by superimposing the original weights that are frozen.

[0071] Step 3: Using a neural network, process the input sample data based on the current weights to obtain the current validation loss.

[0072] Step 4: Using the slot machine driver controller, update the low-rank matrix pairs based on the compressible candidate rank set, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable level, and return to step 2 until the condition is met. Then, determine the target weights of the fine-tunable level based on the updated low-rank matrix pairs and the frozen original weights, so as to achieve efficient parameter fine-tuning of the neural network in the current business scenario.

[0073] In one implementation, the present invention extends the standard implementation to a hierarchical rank selection architecture, that is, assigning independent slot machine driver controllers to different layers of the neural network and maintaining independent reward statistics, but sharing an initial low-rank matrix, thereby allowing each layer to adapt to different rank values ​​according to its sensitivity, and realizing a differentiated rank adaptation strategy.

[0074] In one implementation, the present invention proposes a cross-layer sharing mechanism, that is, the higher-level slot machine driver controller can be passed as prior knowledge to the lower-level slot machine driver controller. For example, the optimal rank distribution of the higher-level slot machine driver controller is used to initialize the candidate rank set of the lower-level slot machine driver controller. Here, higher level and lower level are relative concepts. Taking the aforementioned neural network based on the Transform architecture as an example, the self-attention layer is a higher level relative to the feedforward neural network layer, and the feedforward neural network layer is a lower level, thereby accelerating convergence.

[0075] In one example, the process of updating the low-rank matrix pair of any fine-tunable level is as follows:

[0076] Step 4.1: Determine whether the current iteration step meets the first preset condition. In one implementation, determine the step interval between the current iteration step and the previous iteration step for pruning and compressing the candidate rank set. If the step interval meets a preset interval, it is determined that the first preset condition is met. The first preset condition is used to determine whether to perform rank pruning.

[0077] Specifically, this embodiment of the invention employs a two-stage training protocol, namely: an exploration phase (first 20% of training steps): forcibly uniformly sampling all candidate ranks to construct an initial reward distribution; and a development phase (remaining steps): selecting removable candidate ranks based on the rank quality index (i.e., UCB score), for example, performing rank pruning every K steps (e.g., K=50). In this embodiment of the invention, the two-stage training protocol includes an exploration phase with forced uniform sampling and a development phase based on dynamic candidate set pruning, gradually eliminating inefficient rank configurations to improve optimization efficiency.

[0078] Step 4.2: If yes, prune and compress the candidate rank set corresponding to the fine-tunable level. In one implementation, for any candidate rank included in the candidate rank set corresponding to the fine-tunable level, determine the rank quality index of the candidate rank at the current iteration step. If the average of multiple rank quality indices of the candidate rank between the current iteration step and the historical iteration step is less than a preset index threshold, determine to remove the candidate rank from the candidate rank set.

[0079] The rank construction process is as follows: If a candidate rank The mean of the rank quality index Below the preset threshold ( ), that is Then from the candidate rank set Remove candidate rank This enables dynamic candidate set shrinking, for example, reducing the candidate rank set {4,8,16,32} to {2,4,8}, thereby reducing decision complexity.

[0080] Furthermore, for any slot machine drive controller, its maximum candidate rank can be automatically adjusted according to the task difficulty. The formula is ;

[0081] in, This represents the maximum allowed rank of task p. It is a proportional value used to limit the maximum rank of the task in a low-rank matrix (LoRA-like), ensuring that computational resources are dynamically allocated according to task difficulty and avoiding redundant parameters caused by overuse of high-rank matrices; This represents the historical average reward of task p, which is the average of the historical multi-objective rewards. This is the accumulated reward signal in the Multi-Armed Slots (MAB) algorithm, reflecting the overall performance of the task in past training steps, such as the combination of accuracy and parameter efficiency. This is the minimum reward value among all tasks. It is a global minimum value used for normalization calculations to ensure that the formula output is between 0 and 1. This is the maximum reward value across all tasks; it is a global maximum value. Together they are used for normalization, so that Proportional adjustment. This formula essentially allocates the maximum rank proportionally based on the relative reward intensity of the task (relative to the global minimum / maximum reward). Tasks with higher rewards are allowed higher ranks to accommodate more complex task requirements. The formula is proportional (∝), and in practice, it may be multiplied by a fixed upper limit (such as the maximum value of the initial rank set) to obtain the specific value. For example, taking the ViT neural network model as an example, a fixed rank of 8 is used for the patch embedding layer, dynamic rank (range 4-32) is enabled for the multi-head attention layer, and the maximum candidate rank is limited to 16 for the feedforward network layer, achieving differentiated allocation of computational resources.

[0082] Furthermore, when a new category of sample is detected, the slot machine driver controller can automatically temporarily increase the maximum candidate rank by 50%, and restore the original range after the model stabilizes.

[0083] Step 4.3: Using the slot machine drive controller, determine the optimal rank and its corresponding rank quality index from the pruned and compressed candidate rank set. Based on the rank quality index, the current verification loss, and the historical rank selection sequence corresponding to the fine-tunable level, determine whether to update the low-rank matrix pair. If the determination result is yes, update the low-rank matrix pair inserted at the fine-tunable level using the optimal rank.

[0084] In one example, the process of determining the optimal rank and its corresponding rank quality index from the pruned and compressed candidate rank set is as follows:

[0085] (1) Determine the optimal rank from the pruned and compressed candidate rank set, and determine the multi-objective reward corresponding to the optimal rank.

[0086] In one implementation, the optimal rank can be determined from the pruned and compressed candidate rank set using a confidence upper bound strategy.

[0087] In one implementation, any candidate rank can be determined according to the following reward function. Multi-objective rewards: ;

[0088] in, For any candidate rank; For candidate rank The multi-objective reward is the reward value in the MAB algorithm, used to update the rank selection strategy, comprehensively considering task performance and parameter efficiency; 0.7 and 0.3 represent weight coefficients, used to balance the trade-off between accuracy and sparsity (parameter efficiency). They are typically between 0 and 1 (e.g., 0.5), with larger values ​​favoring performance and smaller values ​​favoring efficiency. For candidate rank The validation accuracy is a task performance metric based on the validation set, reflecting the model's performance at the current rank. This rank can be selected based on the historical data of the slot machine driver controller. The difference between the output of the time neural network and the training labels is determined; 32 represents the maximum rank value, which is usually the maximum value in the candidate rank set (e.g., 32), used to normalize the sparsity term so that (1-r_i / 32) is between 0 and 1 (the smaller the rank, the higher the sparsity reward).

[0089] Preferably, an inference delay term can also be introduced into the reward function: =0.6 ;

[0090] in, For any candidate rank; For candidate rank The multi-objective reward is the reward value in the MAB algorithm, used to update the rank selection strategy, taking into account both task performance and parameter efficiency; 0.6, 0.2, and 0.2 represent fixed weight coefficients, which are example values ​​and sum to 1, used to balance accuracy, sparsity, and latency (which can be adjusted according to the actual scenario). For candidate rank The verification accuracy; 8 represents the reference maximum rank, which is an example fixed value (possibly based on a compressed rank set, such as {2,4,8}), used to normalize the sparsity term, as mentioned in the documentation (compressed rank set). To represent the current inference delay, this is the actual inference time cost of the model at the current rank; To represent the maximum allowed delay, this is a preset threshold used to normalize the delay term (the smaller the delay, the higher the reward). This is the inference latency term. This formula extends multi-objective rewards by incorporating latency awareness, enabling the system to prioritize low-latency, low-rank configurations on edge devices while maintaining performance.

[0091] Based on the above formula, the optimal rank for selecting the slot machine drive controller can be determined. Multi-objective reward in real time. In this embodiment of the invention, the multi-objective reward function simultaneously considers model accuracy and parameter sparsity, and generates a composite reward signal driving rank selection through weighted summation.

[0092] In one implementation, the slot machine drive controller collects historical selection candidate ranks. average gradient norm at time , representing the loss function In low-rank matrices The gradient. Based on the above formula, the optimal rank for the slot machine drive controller can be determined. The gradient norm at time.

[0093] (2) Determine whether the current iteration step meets the second preset condition. The second preset condition is used to determine whether to update the reward statistics of the slot machine driver. In one example, the reward statistics of the slot machine driver controller can be updated every N steps (e.g., N=100) in the validation set.

[0094] (3) If so, update the reward statistics of the slot machine driver controller using multi-objective rewards, and determine the rank quality index corresponding to the optimal rank based on the updated reward statistics and the current gradient norm. Among them, the rank quality index is the core score in the UCB policy, used for MAB decision-making, and integrates performance and gradient information.

[0095] Under the condition that the second preset condition is met, an exponential decay factor γ=0.95 can be introduced to weight the reward statistics of the old task of the slot machine drive controller to ensure that the new task data dominates the current decision.

[0096] With any candidate rank For example, the candidate rank can be determined using the following formula. Rank quality index :

[0097] ;

[0098] in, This is a reward statistic based on validation performance, which is part of traditional UCB and relies solely on performance metrics such as accuracy. Indicates the slot machine driver for candidate rank The total number of reward observations conducted; Represents the slot machine driver number Next pair of candidate ranks Conduct reward observations; Indicates the slot machine driver number Next pair of candidate ranks Multi-objective rewards obtained during reward observations; These are hyperparameters used to control the exploration items. Rank quality index The degree of impact; Represents candidate rank The number of times it was selected; Indicates the current time step; Representative historical selection candidate rank The average gradient norm at that time is the gradient information collected during the fine-tuning process, reflecting the parameter update activity at that rank. A higher norm indicates that the rank is more efficient. Representative candidate rank set The largest average gradient norm is used to normalize the gradient term, making it between 0 and 1, which facilitates weighting. That is, the candidate rank with the largest average gradient norm; Represents the gradient-sensitive term; The adjustment coefficient, a hyperparameter (e.g., 0.1~0.5), controls the weight of the gradient-sensitive term; a larger value emphasizes gradient information. This formula introduces gradient feedback, improving the stability of rank selection: rank with active gradients receives additional preference, accelerating the selection of effective ranks. This metric is part of the UCB score and is used to balance exploration and exploitation. Based on the above formula, the optimal rank can be determined. The rank quality index. Compared to the traditional UCB's average reward. Relying solely on performance verification, this embodiment of the invention introduces a gradient-sensitive term. This design imparts a preference to the rank corresponding to active gradient directions, accelerating the selection of effective rank. In this embodiment, the rank selector of the slot machine driver controller includes a gradient feedback mechanism, incorporating the gradient norm of the low-rank matrix as an additional quality metric into the confidence upper bound calculation, thus optimizing the decision-making efficiency in the discrete rank space.

[0099] In one example, based on the rank quality metric, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable level, a decision is made as to whether to update the low-rank matrix pair, including:

[0100] (1) Encode the current verification loss and the historical rank selection sequence corresponding to the fine-tunable level to obtain the state vector.

[0101] (2) Based on the rank quality index corresponding to the optimal rank and the historical rank (i.e., the current rank) of the low-rank matrix. The rank quality index corresponding to each rank is used to evaluate the optimal rank and historical ranks in conjunction with the state vector. In one implementation, each rank can be encoded into a state vector, which describes information such as the current validation loss, rank quality index, and number of selections corresponding to the rank. The current rank is calculated. The similarity (such as cosine similarity) between the state vector and the state vector of the historical rank is used to filter out the state vectors that are similar to the current rank from the historical rank. Based on the historical ranks with high similarity, the rank quality index of that historical rank is used to evaluate the current rank. Adjust the corresponding rank quality index to change the current rank. The adjusted rank quality index is compared with the rank quality index of the optimal rank to determine the better rank.

[0102] (3) If the evaluation results indicate that the optimal rank is better than the historical rank, determine to use the optimal rank to update the low-rank matrix pair.

[0103] In one example, updating the currently inserted low-rank matrix pair at the fine-tunable level using the optimal rank includes:

[0104] (1) Based on the optimal rank, perform singular value decomposition on the current incremental weight to obtain a new low-rank matrix pair. The current incremental weight is determined based on the low-rank matrix pair currently inserted at this fine-tunable level. The singular value decomposition process is as follows:

[0105] ;

[0106] in, The current incremental weights are weight updates in the low-rank adaptation module (LoRA-like), which require reparameterization to adapt to the new rank. for orthogonal matrix, for The row number represents the left singular vector matrix, used to capture the main direction of the input dimension; for A diagonal matrix whose diagonal elements are singular values, arranged in descending order, reflecting the importance of features; for orthogonal matrix, for The column number represents the transpose of the right singular vector matrix, capturing the main direction of the output dimension; the SVD process decomposes the weight matrix, truncates it, and reconstructs a low-rank approximation to achieve seamless rank switching. , This forms a new low-rank matrix, and the two together constitute a new low-rank matrix pair. For optimal rank, " indicates a slice operation, Indicates to retain the previous text A singular vector is used to ensure that the main feature orientation is not violated. To express [the opinion / towards] Take the square root of each diagonal element of the diagonal submatrix. Representation submatrix transpose, submatrix From the matrix Extracting the first Column construction The submatrix.

[0107] (2) Based on the buffering mechanism, replace the currently inserted low-rank matrix pair at the fine-tunable level with a new low-rank matrix pair. When the slot machine selects the optimal rank... During this process, the system performs the following operations to ensure training continuity: Determine buffer weights based on a preset number of steps; using the buffer weights, perform a weighted fusion of the target low-rank matrix in the new low-rank matrix pair and the target low-rank matrix in the currently inserted low-rank matrix pair at the fine-tunable level; use the weighted fused target low-rank matrix and another low-rank matrix in the new low-rank matrix pair to replace the currently inserted low-rank matrix pair at the fine-tunable level. Specifically, to mitigate gradient abrupt changes during rank switching, momentum buffer updates are used: , The target low-rank matrix obtained by weighted fusion, For buffer weights, For the total number of training steps, using and The currently inserted low-rank matrix pair at the fine-tunable level is replaced to achieve a smooth transition from the old rank to the new rank. This embodiment of the invention supports seamless rank switching during training by reparameterizing the existing low-rank matrix through truncated singular value decomposition, preserving the main feature directions while adapting to the new rank configuration.

[0108] Optionally, for step 4 above, the slot machine drive controller can perform a rank update every 500 steps instead of 100 steps to reduce runtime overhead.

[0109] In summary, compared with existing technologies, the innovations of this invention include: (i) the first introduction of MAB into low-rank adaptation: traditional LoRA relies on manually setting a fixed rank; this invention achieves dynamic adjustment of the rank through sequence decision-making, proposing a gradient-enhanced UCB strategy. (ii) a differentiable rank switching mechanism: existing hyperparameter optimization methods require restarting training; the truncated SVD reparameterization of this invention supports seamless rank switching during training. (iii) a multi-objective reward function: traditional MAB only optimizes accuracy; this invention designs... Achieving Pareto optimality. (iv) Dynamic candidate set pruning: Conventional Bandit maintains a fixed action space; this invention gradually eliminates inefficient ranks through a two-stage protocol, improving the efficiency of later optimization. The above innovations enable this system to surpass existing solutions in both parameter efficiency (saving 15-40% of parameters compared to LoRA) and task performance (improving accuracy by 1-3%), especially in small data scenarios where its advantages are significant.

[0110] This invention proposes an innovative dynamic low-rank matrix adaptive system, the core of which lies in achieving online optimization of the rank configuration of the low-rank matrix through a multi-armed slot machine algorithm. Compared with the traditional fixed-rank LoRA method, the breakthrough of this invention is reflected in: dynamic adaptability: transforming rank selection from a static hyperparameter to a real-time decision-making process based on training dynamics, automatically adjusting the rank size through a feedback mechanism; multi-objective optimization: jointly optimizing task performance (such as classification accuracy) and parameter efficiency (such as rank sparsity), avoiding the suboptimal nature of manual parameter tuning; gradient-aware design: for the first time, incorporating the gradient information of the low-rank matrix into the reward calculation of MAB, enhancing the stability and convergence speed of the decision. This design overcomes the rigidity of static rank configuration and avoids the high computational cost of traditional hyperparameter search. At the application level, this invention is particularly suitable for small-sample fine-tuning scenarios (such as medical text classification and financial sentiment analysis), where the limited amount of data makes fixed-rank methods prone to overfitting or underfitting. Through the dynamic rank reduction strategy, complex patterns can be captured in the early stage of training (high-rank stage), and redundant parameters can be automatically compressed in the later stage (low-rank stage), realizing an adaptive process of learning first and then simplifying.

[0111] Based on the foregoing embodiments, this embodiment of the invention takes IMDb movie review sentiment analysis as an example:

[0112] Input: Pre-trained RoBERTa model + 25k comments (20k training / 5k validation);

[0113] Configuration: Initial rank set Learning rate 2e-5, batch size 32;

[0114] Dynamic process: Steps 0-1k: 32-rank is selected in the exploration phase, and the accuracy is verified to be 82%; Steps 1k-3k: UCB automatically switches to 16-rank, and the accuracy increases to 85%; Steps 3k-5k: Gradient feedback triggers a rank reduction to 8, and the final accuracy is 86.5%, saving 37% of parameters compared to the fixed rank of 32.

[0115] Key observation: The system detected a 16-rank gradient norm decrease at step 2k. The effectiveness of gradient feedback was verified by combining reward descent-triggered rank pruning.

[0116] This invention proposes a dynamic low-rank matrix adaptive system that achieves a breakthrough innovation in the field of efficient parameter fine-tuning through a multi-armed slot machine algorithm. This system transforms the traditional static rank selection into a data-driven dynamic optimization process, resolving the inherent contradiction between task adaptability and parameter efficiency in fixed-rank configuration. The core technological contributions are reflected in three aspects:

[0117] Closed-loop decision-making mechanism: By verifying the real-time fusion of performance feedback and gradient information, an adaptive rank selection closed-loop system was constructed, enabling the model to dynamically adjust its expressive power according to the task complexity.

[0118] Resource-aware optimization: The innovative multi-objective reward function design explicitly incorporates computational resource constraints into the optimization objective, significantly reducing memory and computational overhead while maintaining model performance;

[0119] Architecture independence: The system design does not depend on any specific Transformer variant and can be seamlessly integrated into various architectures such as encoder-decoder and pure decoder, making it widely applicable.

[0120] Experimental results show that the embodiments of the present invention save an average of 28% of trainable parameters in the GLUE benchmark, while achieving a 1.8% accuracy improvement in low-resource scenarios (such as the RTE dataset). Ablation studies confirm the key role of the gradient feedback mechanism—compared to the purely performance-driven Bandit strategy, the gradient-enhanced version improves the rank convergence speed by 2.3 times.

[0121] The industrial application value of the embodiments of this invention is mainly reflected in:

[0122] In cloud computing scenarios: By dynamically reducing the rank, the communication volume during distributed training can be reduced, and in actual tests, the bandwidth consumption of AllReduce operations can be reduced by up to 35%.

[0123] Edge computing scenarios: In the deployment of BERT models on mobile devices, this embodiment of the invention automatically selects the rank 4-8 ​​configuration, achieving a 42% reduction in inference latency while maintaining 98% of the original model accuracy;

[0124] Cross-modal transfer: In the image-text multimodal task, the embodiments of the present invention assign a higher rank (16-32) to the visual module and a lower rank (4-8) to the text module, verifying the effectiveness of hierarchical rank selection.

[0125] Future research directions include extending dynamic rank optimization to the field of quantization-aware training, and exploring methods for automatically generating candidate rank sets based on neural architecture search. The open-source implementation of this invention will provide dual backend support for PyTorch and TensorFlow, promoting the ecological development of efficient parameter fine-tuning techniques.

[0126] Based on the foregoing embodiments, this invention provides a parameter fine-tuning device based on dynamic low-rank matrix adaptation, see [link to relevant documentation]. Figure 4 The diagram shows a structural schematic of a parameter-efficient fine-tuning device based on dynamic low-rank matrix adaptation. This device mainly includes the following components:

[0127] The weight freezing module 402 is used to freeze the original weights of the neural network to be fine-tuned in the current business scenario, where the number of sample data is lower than a preset threshold.

[0128] Matrix insertion module 404 is used to insert low-rank matrix pairs into the fine-tunable layers of the neural network and determine the current weights of the fine-tunable layers based on the low-rank matrix pairs and the frozen original weights.

[0129] The loss determination module 406 is used to process the input sample data based on the current weights through a neural network to obtain the current validation loss.

[0130] The weight fine-tuning module 408 is used to update the low-rank matrix pairs based on the compressible candidate rank set, the current validation loss, and the historical rank selection sequence corresponding to the fine-tunable level through the slot machine-driven controller, and return to call the matrix insertion module until the condition is met. Based on the updated low-rank matrix pairs and the frozen original weights, the target weights of the fine-tunable level are determined, so as to realize efficient parameter fine-tuning of the neural network in the current business scenario.

[0131] The present invention provides a parameter-efficient fine-tuning device based on dynamic low-rank matrix adaptation, which proposes to use a multi-armed slot machine to realize the dynamic configuration of the low-rank matrix, and then fine-tunes the original weights of the neural network. The present invention can alleviate the overfitting or underfitting problem caused by the fixed-rank method in the case of small sample fine-tuning, and at the same time improve the parameter fine-tuning efficiency and task performance of the neural network.

[0132] In one implementation, there are multiple slot machine drive controllers, each corresponding to a fine-tunable level; the weight fine-tuning module 408 is specifically used for:

[0133] For any fine-tunable level, perform the following operation:

[0134] Determine whether the current iteration step meets the first preset condition;

[0135] If so, prune and compress the candidate rank set corresponding to the fine-tunable level;

[0136] The slot machine-driven controller determines the optimal rank and its corresponding rank quality index from the pruned and compressed candidate rank set. Based on the rank quality index, the current verification loss, and the historical rank selection sequence corresponding to the fine-tunable level, it determines whether to update the low-rank matrix pair. If the determination result is yes, the low-rank matrix pair inserted at the fine-tunable level is updated using the optimal rank.

[0137] In one implementation, the weight fine-tuning module 408 is specifically used for:

[0138] Determine the current iteration step number and the step interval between the previous iteration step number for pruning and compressing the candidate rank set. If the step interval meets the preset interval, determine that the first preset condition is met.

[0139] In one implementation, the weight fine-tuning module 408 is specifically used for:

[0140] For any candidate rank contained in the candidate rank set corresponding to the fine-tunable level, determine the rank quality index of the candidate rank at the current iteration step. If the average of multiple rank quality indices of the candidate rank between the current iteration step and the historical iteration step is less than the preset index threshold, determine to remove the candidate rank from the candidate rank set.

[0141] In one implementation, the weight fine-tuning module 408 is specifically used for:

[0142] The optimal rank is determined from the pruned and compressed candidate rank set, and the corresponding multi-objective reward is determined.

[0143] Determine whether the current iteration step meets the second preset condition;

[0144] If so, update the reward statistics of the slot machine driver controller using multi-objective rewards, and determine the rank quality index corresponding to the optimal rank based on the updated reward statistics and the current gradient norm.

[0145] In one implementation, the weight fine-tuning module 408 is specifically used for:

[0146] The current validation loss and the historical rank selection sequence corresponding to the fine-tunable level are encoded to obtain the state vector;

[0147] The optimal rank and historical rank are evaluated based on the rank quality index corresponding to the optimal rank and the rank quality index corresponding to the historical rank of the low-rank matrix, combined with the state vector.

[0148] If the evaluation results indicate that the optimal rank is superior to the historical rank, then the optimal rank is used to update the low-rank matrix pair.

[0149] In one implementation, the weight fine-tuning module 408 is specifically used for:

[0150] Singular value decomposition is performed on the current incremental weights based on the optimal rank to obtain new low-rank matrix pairs; where the current incremental weights are determined based on the low-rank matrix pairs currently inserted at this fine-tunable level.

[0151] Based on the buffering mechanism, the currently inserted low-rank matrix pair at the fine-tunable level is replaced with a new low-rank matrix pair.

[0152] In one implementation, the weight fine-tuning module 408 is specifically used for:

[0153] Buffer weights are determined based on a preset number of steps;

[0154] Using buffer weights, the target low-rank matrix in the new low-rank matrix pair and the target low-rank matrix in the currently inserted low-rank matrix pair at the fine-tunable level are weighted and fused.

[0155] The target low-rank matrix obtained by weighted fusion, and another low-rank matrix in the new low-rank matrix pair, are used to replace the currently inserted low-rank matrix pair at the fine-tunable level.

[0156] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0157] This invention provides an electronic device, specifically, the electronic device includes a processor and a storage device; the storage device stores a computer program, and the computer program, when run by the processor, executes the method described in any of the above embodiments.

[0158] Figure 5 The present invention provides a schematic diagram of the structure of an electronic device 100, which includes a processor 50, a memory 51, a bus 52 and a communication interface 53. The processor 50, the communication interface 53 and the memory 51 are connected through the bus 52. The processor 50 is used to execute executable modules, such as computer programs, stored in the memory 51.

[0159] The memory 51 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 53 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0160] Bus 52 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0161] The memory 51 is used to store programs. After receiving an execution instruction, the processor 50 executes the programs. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 50 or implemented by the processor 50.

[0162] Processor 50 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 50 or by instructions in software form. Processor 50 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 51. The processor 50 reads the information in memory 51 and, in conjunction with its hardware, completes the steps of the above method.

[0163] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0164] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A parameter efficient fine-tuning method based on dynamic low-rank matrix adaptation, characterized in that, The method comprises the following steps: Step 1, freezing the original weights of the neural network to be fine-tuned in the current business scenario, wherein the current business scenario is a business scenario in which the number of sample data is less than a preset threshold, the sample data comprises input data and training labels, the input data is text data or image-text data to be classified, and an output result is generated for the input data by the neural network, wherein the output result is a classification result corresponding to the text data or the image-text data; Step 2, inserting a low-rank matrix pair into a tunable level of the neural network, and determining the current weights of the tunable level based on the low-rank matrix pair and the frozen original weights; Step 3, processing the input sample data based on the current weights by the neural network to obtain a current validation loss; Step 4, updating the low-rank matrix pair based on the compressible candidate rank set, the current validation loss and the historical rank selection sequence corresponding to the tunable level by a slot machine driving controller, and returning to step 2 until a condition is met to stop, determining the target weights of the tunable level based on the updated low-rank matrix pair and the frozen original weights, and realizing efficient parameter fine-tuning of the neural network in the current business scenario; wherein the number of slot machine driving controllers is multiple, the slot machine driving controller corresponds to the tunable level one by one, the slot machine driving controller is the core subsystem for realizing dynamic low-rank matrix selection, and specifically comprises: a rank selector for selecting an optimal rank according to a multi-objective reward; and a low-rank matrix storage for storing a dynamically adjusted low-rank matrix.

2. The parameter efficient fine-tuning method based on dynamic low-rank matrix adaptation of claim 1, wherein, The slot machine driving controller updates the low-rank matrix pair based on the compressible candidate rank set, the current validation loss and the historical rank selection sequence corresponding to the tunable level, which comprises: For any tunable level, the following operations are performed: determining whether the current iteration step number meets a first preset condition; if yes, pruning and compressing the candidate rank set corresponding to the tunable level; determining an optimal rank and a rank quality index corresponding to the optimal rank from the pruned and compressed candidate rank set by the slot machine driving controller, and determining whether to update the low-rank matrix pair based on the rank quality index, the current validation loss and the historical rank selection sequence corresponding to the tunable level, and when the determination result is yes, updating the low-rank matrix pair inserted into the tunable level by using the optimal rank.

3. The parameter efficient fine-tuning method based on dynamic low-rank matrix adaptation of claim 2, wherein, determining whether the current iteration step number meets a first preset condition, which comprises: determining the step interval between the current iteration step number and the historical iteration step number at which the candidate rank set is pruned and compressed last time, and determining that the first preset condition is met when the step interval meets a preset interval; pruning and compressing the candidate rank set corresponding to the tunable level, which comprises: For any candidate rank included in the candidate rank set corresponding to the fine-tunable level, a rank quality indicator of the candidate rank at the current iteration step is determined, and in a case that a mean value of a plurality of the rank quality indicators of the candidate rank between the current iteration step and the historical iteration step is less than a preset indicator threshold, it is determined that the candidate rank is removed from the candidate rank set.

4. The parameter efficient fine-tuning method based on dynamic low-rank matrix adaptation of claim 2, wherein, Determining an optimal rank and a rank quality indicator corresponding to the optimal rank from the pruned and compressed candidate rank set, comprising: Determining an optimal rank from the pruned and compressed candidate rank set, and determining a multi-objective reward corresponding to the optimal rank; Determining whether the current iteration step satisfies a second preset condition; If yes, updating a reward statistical quantity of the slot machine driving controller by using the multi-objective reward, and determining the rank quality indicator corresponding to the optimal rank based on the updated reward statistical quantity and a current gradient norm.

5. The parameter efficient fine-tuning method based on dynamic low-rank matrix adaptation of claim 2, wherein, Based on the rank quality indicator, the current validation loss, and a historical rank selection sequence corresponding to the fine-tunable level, determining whether to update the low-rank matrix pair, comprising: Encoding the current validation loss and the historical rank selection sequence corresponding to the fine-tunable level to obtain a state vector; Based on the rank quality indicator corresponding to the optimal rank, the rank quality indicator corresponding to the historical rank to which the low-rank matrix pair belongs, and the state vector, evaluating the optimal rank and the historical rank; In a case that the evaluation result represents that the optimal rank is superior to the historical rank, it is determined that the low-rank matrix pair is updated by using the optimal rank.

6. The parameter efficient fine-tuning method based on dynamic low-rank matrix adaptation of claim 2, wherein, Updating the low-rank matrix pair currently inserted by the fine-tunable level by using the optimal rank, comprising: Based on the optimal rank, performing singular value decomposition on a current incremental weight to obtain a new low-rank matrix pair; wherein the current incremental weight is determined based on the low-rank matrix pair currently inserted by the fine-tunable level; Based on a buffer mechanism, replacing the low-rank matrix pair currently inserted by the fine-tunable level by using the new low-rank matrix pair.

7. The parameter efficient fine-tuning method based on dynamic low-rank matrix adaptation of claim 6, wherein, Based on a buffer mechanism, replacing the low-rank matrix pair currently inserted by the fine-tunable level by using the new low-rank matrix pair, comprising: Determining a buffer weight based on a preset step number; Using the buffer weight to perform weighted fusion on a target low-rank matrix in the new low-rank matrix pair and the target low-rank matrix in the low-rank matrix pair currently inserted by the fine-tunable level; Using the target low-rank matrix obtained by the weighted fusion and another low-rank matrix in the new low-rank matrix pair to replace the low-rank matrix pair currently inserted by the fine-tunable level.

8. A device for efficient fine-tuning of parameters based on dynamic low-rank matrix adaptation, characterized in that, Comprising: A weight freezing module is configured to freeze original weights of a neural network to be fine-tuned in a current business scenario, the current business scenario being a business scenario in which a quantity of sample data is less than a preset threshold, the sample data including input data and training labels, the input data being text data or image-text data to be classified, and an output result being generated by the neural network for the input data, the output result being a classification result corresponding to the text data or the image-text data; a matrix insertion module, configured to insert a pair of low-rank matrices into a tunable level of the neural network, and determine current weights of the tunable level based on the pair of low-rank matrices and the original frozen weights; a loss determination module, configured to process input sample data through the neural network based on the current weights to obtain a current validation loss; a weight tuning module, configured to update the pair of low-rank matrices based on a compressible candidate rank set, the current validation loss, and a historical rank selection sequence corresponding to the tunable level by a slot machine driving controller, and return to call the matrix insertion module until a condition is met, determine target weights of the tunable level based on the updated pair of low-rank matrices and the original frozen weights, and realize efficient parameter tuning of the neural network for the current business scenario; wherein the slot machine driving controllers are in a plurality, and each of the slot machine driving controllers corresponds to one of the tunable levels, the slot machine driving controller is a core subsystem for dynamic low-rank matrix selection, and specifically includes: a rank selector configured to select an optimal rank according to multi-objective rewards; and a low-rank matrix storage configured to store dynamically adjusted low-rank matrices.

9. An electronic device, comprising: A processor and a memory are included, the memory stores computer executable instructions capable of being executed by the processor, and the processor executes the computer executable instructions to implement the method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions, and when the computer executable instructions are called and executed by the processor, the computer executable instructions cause the processor to implement the method in any one of claims 1 to 7.

Images