Low-rank adaptation system based on differential polarization rank search

Abstract

The invention relates to the field of natural language processing, in particular to a low-rank adaptation system capable of differential polarization rank search, which comprises an initialization module, a warm-up training module, a joint optimization module, a sparse pruning module and a final fine tuning module. According to the method, dynamic rank allocation and continuous search are realized by introducing trainable rank architecture parameters, and model parameter waste and calculation overhead are remarkably reduced by combining polarization regularization and L2 regularization strategies. According to the method, the performance of the large model in the downstream task can be effectively improved, meanwhile, the method is suitable for scenes with multi-task concurrence, resource-constrained device deployment and high real-time requirements, and a technical solution is provided for efficient adaptation of the large-scale pre-training model.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically a low-rank adaptive system based on differentiable polarization rank search. Background Technology

[0002] In the field of natural language processing, pre-trained language models such as BERT, T5, and GPT-3 are widely used due to their superior performance. However, these models typically contain a massive number of parameters; for example, BERT contains 300 million parameters, T5 reaches 11 billion parameters, and GPT-3 boasts a staggering 1.75 trillion parameters. This enormous parameter scale presents a severe memory challenge for full fine-tuning. In multi-task concurrent scenarios, traditional full fine-tuning requires maintaining an independent copy of the model for each task, resulting in extremely high memory resource consumption.

[0003] To address this issue, existing parameter-efficient fine-tuning techniques (PEFT) achieve adaptation by freezing the backbone parameters and optimizing a small number of newly added modules. Typical methods include adapter module insertion, trainable prefix injection, and cue-based adjustment. While these methods alleviate resource pressure to some extent, they still have significant drawbacks. Taking low-rank adaptation as an example, it achieves efficient parameter updates through low-rank decomposition, but assumes that all increment matrices have the same rank, ignoring the differences in the importance of weight matrices in different modules and layers of the pre-trained model. Experiments show that, under the same parameter scale, fine-tuning the feedforward network (FFN) outperforms the self-attention module, the top-level weight matrix is ​​more important than the bottom-level weight matrix, and uniform parameter allocation strategies often lead to suboptimal results. In addition, existing adaptive parameter allocation schemes also have technical bottlenecks. Ada low-rank adaptation's heuristic scoring based on singular value decomposition and frequent weight modifications limit its performance ceiling, while Dy low-rank adaptation's discretized rank selection mechanism and weight truncation operation restrict the model's expressive power. Existing technologies generally lack dynamic perception of module importance and continuous rank allocation capabilities, making it difficult to achieve efficient parameter optimization.

[0004] Therefore, there is an urgent need for an optimization scheme that can dynamically adjust rank allocation, support continuous rank search, and improve model performance to overcome the shortcomings of the aforementioned technologies and meet the practical needs of multi-task concurrency, resource-constrained device deployment, and efficient fine-tuning of large-scale models. This invention proposes an innovative solution to address these problems, aiming to achieve efficient fine-tuning of large model parameters through dynamic rank allocation and neural architecture search techniques, while reducing computational resource consumption and memory usage. Summary of the Invention

[0005] This invention addresses the problem of parameter waste caused by fixed-rank allocation in existing low-rank adaptation methods by providing a low-rank adaptation system based on differentiable polarizable rank search. The system achieves dynamic rank allocation and adaptive optimization through a modular architecture design, making it suitable for scenarios involving efficient fine-tuning of large model parameters.

[0006] This invention provides a low-rank adaptive system based on differentiable polarization rank search, including an initialization module, a warm-up training module, a joint optimization module, a sparse pruning module, and a final fine-tuning module.

[0007] The initialization module is responsible for preprocessing the parameters of the low-rank adaptation module, setting the rank of the low-rank adaptation module to 1.5 times the target rank, and setting the rank architecture parameter λ{( m) The algorithm provides a sufficient search space and initializes all rank architecture parameters to zero. Simultaneously, it initializes vector matrix A with a random normal distribution and vector matrix B with a zero matrix to ensure that the increment matrix Δ is zero at the beginning of training, thereby avoiding the disruption of the original mapping relationship of the pre-trained weights. Then, it performs an initialization warm-up training for a period of time to allow the low-rank adaptation parameter λ{(m)} to converge initially, and the model to initially adapt to the characteristics of the downstream task, laying the foundation for subsequent rank optimization.

[0008] After the warm-up training module completes its initialization, it maintains the current rank and continues to adapt the low-rank parameter λ{( m) Warm-up training is conducted to further stabilize the parameters and enhance the model's adaptability to task features.

[0009] The joint optimization module is the core of the system, introducing L2 regularization and polarization regularization terms into the loss function. The L2 regularization term controls the overall size of the rank architecture parameters to avoid overfitting; the polarization regularization term forces the rank architecture parameters away from the mean, promoting a "polarized" distribution, where some parameters converge to non-zero valid values ​​(corresponding to retained ranks), while others approach zero (corresponding to redundant ranks). During training, the rank architecture parameters and vector matrix are updated synchronously through backpropagation, dynamically adjusting the rank distribution of each module.

[0010] Specifically, by combining polarization regularization with L2 regularization, the following optimization objective is constructed:

[0011]

[0012] in: This represents the task loss (such as cross-entropy loss); L2 regularization for rank architecture parameters, used to control the overall parameter size; The polarization regularization term is defined as follows:

[0013]

[0014] in, Let be the mean of the rank architecture parameters, and t be a hyperparameter that adjusts the polarization intensity. This regularization term forces the rank architecture parameters away from the mean, causing some rank architecture parameters to converge to non-zero valid values ​​(corresponding to the retained ranks), while others approach zero (corresponding to the pruned redundant ranks).

[0015] Specifically, the sparse pruning module is activated after polarization training. By setting a threshold, it sets the rank architecture parameters that approach zero to zero and deletes the corresponding vector matrix rows and columns, completing dynamic rank pruning at extremely low cost, reducing the number of model parameters and computational overhead. During the pruning process, the redundant rank architecture parameters that have been set to zero remain unchanged, ensuring the stability of the model structure. Furthermore, the final fine-tuning module is activated after sparse pruning. It removes the L2 regularization and polarization regularization loss terms and performs final fine-tuning training on the model under the condition of a fixed target rank. This allows the model to fully learn the task features under the optimized rank configuration, improving its performance on downstream tasks.

[0016] The technical innovations of this invention lie in the design of a dynamic rank adjustment module, rank space continuity technology, automated rank search metrics, learnable rank architecture parameter design, polarization regularization rank selection technology, and a one-step rank pruning strategy. The dynamic rank adjustment module designs an adaptive rank allocation algorithm to address the differences in rank requirements across different layers of the model. It achieves dynamic scheduling of rank resources through a neural architecture search module, significantly reducing training complexity. Furthermore, the rank space continuity technology overcomes the structural limitations of traditional micro-adaptors by introducing rank architecture parameters, enabling the model to autonomously search for optimal rank parameters during training. This supports continuous rank search and overcomes the limitations of rank upper and lower bounds within the module.

[0017] Specifically, the automated rank search metric directly represents the weight importance of different ranks using rank architecture parameters. It controls the rank reduction of the fine-tuning adapter by retaining some rank architecture parameters and setting others to zero, avoiding the difficulty of rank selection caused by subjective importance scoring. Furthermore, the learnable rank architecture parameters are designed to introduce independent rank architecture parameters into each layer of low-rank matrices. Redundant rank is dynamically identified through sparse training, supporting continuous rank space search and overcoming the limitations of rank upper and lower bounds within a module.

[0018] The polarization-regularized rank selection technique introduces a learnable scaling factor into the rank architecture parameters. Through a combination of L1 sparse regularization and polarization regularization, it enhances the distinguishability between important and redundant ranks, minimizing the loss of model accuracy during rank reduction. Furthermore, the one-step rank pruning strategy improves the Ada low-rank adaptation multi-mask training mode, designing a one-step rank reduction strategy to address the severe model accuracy loss problem of traditional schemes, reduce training time and computational overhead, and improve model training efficiency and final performance.

[0019] Specifically, the end-to-end rank optimization training framework constructs a closed-loop optimization system that includes rank architecture search, importance evaluation, and dynamic allocation. It filters effective ranks through forward propagation and optimizes the rank allocation strategy through backpropagation, avoiding performance loss caused by weight truncation. Furthermore, the system does not require modification of the pre-trained model architecture; efficient parameter fine-tuning can be achieved simply by adding trainable modules, making it suitable for various large-scale pre-trained models such as BERT, T5, and GPT.

[0020] The beneficial effects of this invention lie in achieving model performance optimization through dynamic rank allocation and neural architecture search techniques. Specifically, the system overcomes the limitations of traditional discrete rank selection by automatically searching the continuous rank space using differentiable architecture parameters, avoiding the subjectivity and suboptimal nature of manual parameter tuning. The end-to-end training framework can complete rank allocation, importance assessment, and structure optimization in a single training session, significantly simplifying the training process. The polarization-regularized rank selection technique in this system promotes the polarization of rank architecture parameters, enhancing the model's robustness to noise and redundant structures. The final fine-tuning stage is performed on the optimized rank structure, further improving the model's generalization ability in downstream tasks and reducing the risk of overfitting. In addition, the system can significantly reduce memory consumption and computational overhead while improving the model's performance in downstream tasks. Furthermore, the system is suitable for various scenarios such as multi-task concurrent processing, downstream task fine-tuning, resource-constrained device deployment, efficient fine-tuning of large-scale language models, and applications with high real-time requirements. By dynamically adjusting the rank, it reduces the number of model parameters and computational load, achieving lightweight deployment.

[0021] Through the aforementioned modular design and technological innovation, this invention achieves intelligent allocation and optimization of rank resources during the fine-tuning of large models. Experimental verification shows that it can effectively improve model fine-tuning performance while reducing computational resource consumption, providing a technical solution for the efficient adaptation of large-scale pre-trained models. Attached Figure Description

[0022] Figure 1 This is an overall schematic diagram of the present invention;

[0023] Figure 2 This is a schematic diagram of the initialization module;

[0024] Figure 3 This is a schematic diagram of the warm-up training module;

[0025] Figure 4 This is a schematic diagram of the joint optimization module;

[0026] Figure 5 This is a schematic diagram of the sparse pruning module;

[0027] Figure 6 This is a schematic diagram of the final fine-tuning module. Detailed Implementation

[0028] This invention provides a low-rank adaptive system based on differentiable polarizable rank search, achieving dynamic rank allocation and adaptive optimization through a modular architecture design. (See attached diagram.) Figure 1 To be continued Figure 6 This implementation method details the specific operating principle, operation process, and actual application scenarios of the system.

[0029] like Figure 1 As shown, the overall system framework consists of five core modules: initialization, warm-up training, joint optimization, sparse pruning, and final fine-tuning. Each module performs a specific function within the system, working together to efficiently fine-tune the parameters of a large model. The implementation methods and technical details of each module will be explained below with reference to the accompanying diagrams and specific steps.

[0030] First, the core function of the initialization module is to preprocess the parameters of the low-rank adaptation module and set the rank of the low-rank adaptation module to 1.5 times the target rank, thereby expanding the search space for rank architecture parameters. For example... Figure 2 As shown, the system initializes vector matrix A with a random normal distribution and vector matrix B with a zero matrix, ensuring that the increment matrix Δ is zero at the beginning of training, thus avoiding disruption of the original mapping relationship of the pre-trained weights. Simultaneously, all rank architecture parameters are initialized to zero, providing a foundation for subsequent continuous rank space search. Based on this, the initialization module performs a period of initialization warm-up training, allowing the low-rank adaptation parameters to converge initially and the model to initially adapt to the characteristics of the downstream task, laying the foundation for subsequent rank optimization.

[0031] Next, we will move on to the warm-up training module, such as... Figure 3 As shown in the diagram, after initialization, this module maintains the current rank and continues warm-up training on the low-rank adaptation parameters to further stabilize the parameters and enhance the model's adaptability to task features. This process significantly reduces training complexity caused by unreasonable initial rank allocation. For example, in multi-task concurrent scenarios, different tasks have different requirements for each layer of the model; the warm-up training module can effectively capture these changes in requirements, providing a basis for subsequent dynamic rank adjustment.

[0032] The joint optimization module is the core part of the system, and its operating principle is as follows: Figure 4As shown in the diagram, in this module, the system optimizes the loss function by introducing L2 regularization and polarization regularization terms. The main function of the L2 regularization term is to control the overall size of the rank architecture parameters and prevent overfitting; while the polarization regularization term forces the rank architecture parameters away from the mean, promoting a "polarized" distribution. During training, some rank architecture parameters converge to non-zero valid values, corresponding to the retained ranks; another part approaches zero, corresponding to redundant ranks. Through the backpropagation mechanism, the system synchronously updates the rank architecture parameters and vector matrices A7 and B8, dynamically adjusting the rank allocation of each module. This design ensures that the system can autonomously search for the optimal rank configuration in the continuous rank space, breaking through the limitations of traditional discrete rank selection mechanisms.

[0033] In particular, the joint optimization module incorporates an automated rank search metric. The system directly represents the weight importance of different ranks using rank architecture parameters. By retaining some rank architecture parameters and setting the rest to zero, it controls the rank reduction of the fine-tuning adapter. This design avoids the difficulty of rank selection caused by subjective importance scoring, achieving more accurate rank allocation. For example, in efficient fine-tuning scenarios for large-scale language models, the system can automatically identify key modules and allocate more rank resources based on the characteristics of downstream tasks, thereby reducing computational overhead while ensuring performance.

[0034] Then the sparse pruning module will be used for operation, such as... Figure 5 As shown, this module starts after polarization training. It prunes rank architecture parameters that approach zero by setting a threshold and deletes the corresponding vector matrix rows and columns. This operation performs dynamic rank pruning at extremely low cost, significantly reducing the number of model parameters and computational overhead. It is important to note that the redundant rank architecture parameters that have been set to zero are kept unchanged during the pruning process to ensure the stability of the model structure. For example, in resource-constrained device deployment scenarios, the sparse pruning module can significantly reduce the number of model parameters and computational load, enabling it to run efficiently on mobile or embedded devices and achieve lightweight deployment.

[0035] The final fine-tuning module operation is as follows: Figure 6 As shown. After sparse pruning, the system removes the L2 regularization and polarization regularization terms, and performs final fine-tuning training on the model under a fixed target rank. The goal of this stage is to enable the model to fully learn task features under the optimized rank configuration, thereby improving its performance on downstream tasks. For example, in applications with high real-time requirements, such as real-time dialogue systems or real-time translation, the final fine-tuning module can further optimize the model's inference time and computational efficiency to meet the performance requirements of real-time applications.

[0036] Furthermore, this invention introduces several technological innovations, further enhancing the system's performance and applicability. The dynamic rank adjustment module designs an adaptive rank allocation algorithm to address the varying rank requirements of different layers in the model. Dynamic scheduling of rank resources is achieved through the neural architecture search module, significantly reducing training complexity. For example, in multi-task concurrent scenarios, the system can dynamically adjust the allocation of rank resources according to the different needs of each task for each layer of the model, avoiding the need to maintain independent large model copies for each task, thereby reducing memory consumption and computational overhead.

[0037] The rank space continuity technique overcomes the structural limitations of traditional micro-adaptors by introducing rank architecture parameters, enabling the model to autonomously search for optimal rank parameters during training. This design supports continuous rank search, overcoming the limitations of intra-module rank upper and lower bounds. For example, in efficient fine-tuning scenarios for large-scale language models, the system can explore the optimal rank configuration within a continuous rank space, avoiding the limitations of traditional discretized rank selection mechanisms.

[0038] The polarization-regularized rank selection technique introduces a learnable scaling factor into the rank architecture parameters and enhances the distinguishability between important and redundant ranks through a combination of L1 sparse regularization and polarization regularization. This design minimizes the loss of model accuracy during rank reduction. For example, in resource-constrained deployment scenarios, the polarization-regularized rank selection technique allows the system to reduce the number of model parameters while maximizing the preservation of key model performance.

[0039] A one-step rank reduction strategy improves upon the Ada low-rank adaptation multi-mask training mode by introducing a one-step rank reduction strategy. This design addresses the severe accuracy loss in traditional approaches, reduces training time and computational overhead, and improves training efficiency and final performance. For example, in real-time applications of natural language processing, the one-step rank reduction strategy enables the system to quickly reduce the model's rank, meeting the performance requirements of real-time applications.

[0040] The end-to-end rank optimization training framework constructs a closed-loop optimization system that includes rank architecture search, importance evaluation, and dynamic allocation. It filters effective ranks through forward propagation and optimizes the rank allocation strategy through backpropagation, avoiding performance loss caused by weight truncation. For example, in multi-task concurrent scenarios, the end-to-end rank optimization training framework enables intelligent allocation and optimization of rank resources, improving the model's efficiency and performance in multi-task processing.

[0041] In summary, this invention, through the aforementioned modular design and technological innovations, achieves intelligent allocation and optimization of rank resources during large-scale model fine-tuning. Experimental verification shows that this system can significantly reduce memory consumption and computational overhead, while simultaneously improving model performance in downstream tasks. The system is suitable for various scenarios, including multi-task concurrent processing, downstream task fine-tuning, resource-constrained device deployment, efficient fine-tuning of large-scale language models, and applications with high real-time requirements. By dynamically adjusting the rank, it reduces the number of model parameters and computational load, achieving lightweight deployment.

Claims

1. A low-rank adaptive system based on differentiable polarimetric rank search, characterized in that: The system includes an initialization module, a warm-up training module, a joint optimization module, a sparse pruning module, and a final fine-tuning module. The initialization module preprocesses the parameters of the low-rank adaptation module, sets the rank of the low-rank adaptation module to 1.5 times the target rank, and performs warm-up training on the low-rank adaptation parameters to achieve initial convergence, while initializing all rank architecture parameters to zero. After the initialization module is completed, the warm-up training module keeps the current rank unchanged and continues to perform warm-up training on the low-rank adaptation parameters to further stabilize the parameters. The joint optimization module introduces L2 regularization and polarization regularization terms into the loss function to dynamically adjust the rank distribution of each module. The sparse pruning module prunes rank architecture parameters that approach zero by setting a threshold and deletes the corresponding vector matrix rows and columns. The final fine-tuning module removes the regularization terms under the condition of a fixed target rank and performs final fine-tuning training on the model.

2. The low-rank adaptive system based on differentiable polarizable rank search as described in claim 1, characterized in that: The initialization module initializes vector matrices A and B using a random normal distribution to ensure that the increment matrix Δ is zero at the beginning of training.

3. A low-rank adaptive system based on differentiable polarizable rank search as described in claim 2, characterized in that: The initialization module further provides a foundation for continuous rank space search through the design of rank architecture parameters.

4. A low-rank adaptive system based on differentiable polarizable rank search as described in claim 1, characterized in that: The warm-up training module provides ample search space for the target rank architecture parameters by expanding the rank of the low-rank adaptation module to 1.5 times the target rank.

5. A low-rank adaptive system based on differentiable polarizable rank search as described in claim 4, characterized in that: The warm-up training module further enables the model to initially adapt to the characteristics of downstream tasks, thereby reducing the training complexity caused by unreasonable initial rank allocation.

6. A low-rank adaptive system based on differentiable polarizable rank search as described in claim 1, characterized in that: The joint optimization module dynamically adjusts the rank allocation of each module by synchronously updating the rank architecture parameters and vector matrices A and B through backpropagation.

7. A low-rank adaptive system based on differentiable polarizable rank search as described in claim 6, characterized in that: The joint optimization module further combines the automated rank search index with rank architecture parameters to represent the weight importance of different ranks.

8. A low-rank adaptive system based on differentiable polarizable rank search as described in claim 1, characterized in that: The sparse pruning module performs dynamic rank pruning in a one-step process and maintains the redundant rank architecture parameters that have been set to zero during the pruning process to ensure the stability of the model structure.

9. A low-rank adaptive system based on differentiable polarizable rank search as described in claim 1, characterized in that: The final fine-tuning module controls the model to learn task features under the optimized rank configuration under the condition of a fixed target rank.

10. A low-rank adaptive system based on differentiable polarizable rank search as described in claim 9, characterized in that: The final fine-tuning module further filters effective ranks through forward propagation and optimizes the rank allocation strategy through backpropagation.

Images