Large model optimization method for multi-matrix aggregation multi-dimensional features

Abstract

The invention belongs to the technical field of electric digital data processing, and particularly relates to a large model optimization method for multi-matrix aggregation multi-dimensional features. The method comprises the following steps: firstly, designing a basic optimization operator FoRA: Fusion of Rank; secondly, the basic optimization operator designed in the first step is adapted to a whole TransformerLayer layer of a Transformer large model based on federated learning; and finally, quantifying the Transform large model which completes optimization operator adaptation in the previous step so as to reduce the demand of training the large model for resources, then locally finely adjusting the quantized large model based on federated learning and a client, and converging trainable low-rank parameters after fine adjustment to a global model. The problems that an existing large language model needs a large amount of high-quality data and computing resource training, and downstream tasks cannot be well processed are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic digital data processing, and more specifically, relates to a large model optimization method of multi-matrix aggregation and multi-dimensional features. Background Art

[0002] Large language models are currently in a stage of rapid development, and there are many aspects that deserve attention. For example, the latest large language models, such as GPT-4.0, have billions or even hundreds of billions of parameters. These huge models help improve the ability of language understanding and generation, bringing it closer to the human level. Among them, some advanced language models have begun to integrate multimodal information, not only understanding and generating text, but also processing multiple forms of data such as images and audio. This allows the model to perform well on a wider range of tasks. In addition, the application of models in different fields has become more common, and they can better adapt to tasks in specific fields through methods such as pre-training and fine-tuning, improving performance in specific scenarios. However, as the scale of language models increases, some challenges and controversies related to them have emerged, including the energy consumption of the model, data privacy, and potential risks of abuse, and social attention to these issues is also increasing. During this period, large language models have demonstrated strong language understanding and generation capabilities on various tasks, including many applications such as question-and-answer chats, copywriting, and code generation. However, there are still two key problems with large language models in practical applications. First, training large language models requires a large amount of high-quality data. Villalobos2022will and others believe that the existing high-quality open source data will be exhausted in 2026. In the future, it will be necessary to combine industry data for training. However, due to the protection of the privacy of private domain data, it cannot be used directly like public domain data such as CommonCrawl and Wikipedia. At the same time, in order to further apply the emergence capabilities of large models to specific downstream industries such as medical care, law, and finance, it is also necessary to use industry private domain data for transfer learning based on general pre-training models to train large industry models. The scale of individual hospital and bank data is not enough to train a powerful model. Second, the training of large language models requires a lot of computing resources as support, which may mean that a single enterprise cannot obtain enough resources to fine-tune the large model locally, thereby limiting the specific application of large language models in downstream tasks.

[0003] In order to make the pre-trained model better handle downstream tasks and reduce the demand for computing resources, the most common method is to efficiently fine-tune the pre-trained model. The efficient fine-tuning algorithm can obtain performance similar to full fine-tuning by training only a small part of the parameters, and can also better overcome the catastrophic forgetting problem of full fine-tuning. There are many existing fine-tuning algorithms, and they focus on different aspects. The core idea of ​​the Adapter Tuning method proposed by the Google Research team is to introduce adapters into the model and use a relatively small set of parameters to perform specific tasks. Adapters can be added to different layers of the pre-trained model to make the model more easily adaptable to new tasks during fine-tuning without modifying the parameters of the entire model. The PrefixTuning proposed by the Facebook AI Research (FAIR) research team focuses on adding a specified prefix before the input sequence to guide the model to better adapt to specific tasks. By including task-related information in the prefix, Prefix Tuning aims to improve the adaptability and performance of the model for fine-tuning tasks. The LoRA method proposed by Edward J et al. based on the thinking of parameter sharing and model compression freezes the original training parameters and uses two low-rank matrices as trainable parameters. With its excellent performance, it has been applied to a variety of downstream tasks.

[0004] Chinese patent document CN117033641A discloses a network structure optimization and fine-tuning method for a large pre-trained language model. The method first collects the conversation data of voice marketing and text marketing of telephone sales, performs data cleaning and preprocessing; annotates the collected conversation data, designates some conversation data as marketing failures, and classifies and annotates the reasons for marketing failures; optimizes the large pre-trained language model, including using dictionary reduction and network layer reduction methods; uses the LoRA low-rank adaptation method to fine-tune the optimized large pre-trained language model; verifies and iteratively trains the fine-tuned large pre-trained language model. Although the above methods have achieved good results in fine-tuning the pre-trained model, there are still problems such as making the overall number of layers of the model deeper, increasing the training cost and reasoning time, reducing the effective information length of the input layer, and failing to extract sufficient knowledge with limited training parameters, resulting in insufficient ability to handle downstream tasks. At the same time, even if the efficient fine-tuning algorithm still needs to load all weight files in the fine-tuning stage, as the number of model parameters continues to increase, this will still pose certain challenges to client resource requirements. In order to further reduce the demand for resources during fine-tuning of large models, researchers have proposed methods such as quantization, distillation, and pruning to compress the original weights, but model compression has led to a loss of accuracy.

[0005] The limitations of the above existing methods for model fine-tuning have hindered the training and application of large industry models, calling for the birth of a new and efficient model fine-tuning method. In view of this, the present invention designs a large model optimization method that aggregates multi-matrix and multi-dimensional features. Summary of the invention

[0006] The present invention aims to overcome at least one defect of the above-mentioned prior art and provide a large model optimization method of multi-matrix aggregation and multi-dimensional features to solve the problem that the existing large language model requires a large amount of high-quality data and computing resources for training and cannot handle downstream tasks well.

[0007] The detailed technical scheme of the present invention is as follows: A large model optimization method for multi-matrix aggregation and multi-dimensional features, the method comprising: S1. Design a comprehensive and effective basic optimization operator FoRA: Fusion of Rank; S2, adapting the basic optimization operator designed in step S1 to the entire TransformerLayer layer of the Transformer large model based on federated learning; S3. Quantize the Transformer large model that has completed the optimization operator adaptation in step S2 to reduce the resource requirements for training the large model. Then, based on federated learning, the client fine-tunes the quantized large model locally and aggregates the fine-tuned trainable low-rank parameters to the global model.

[0008] According to the preferred embodiment of the present invention, the basic optimization operator FoRA proposed in step S1 is introduced in detail as follows. Based on the idea of ​​the federated learning framework, the basic optimization operator FoRA divides the fine-tuning process into two stages: the knowledge extraction stage and the knowledge fusion stage; Knowledge extraction stage: Assuming that the pre-trained weight is W, the fine-tuned training parameters are expressed as W; First is initialization. FoRA will create five low-rank matrices A, B, C, D, E, and two hyperparameters Rank and Scale_Rank for the pre-trained weight W, where Rank represents the output dimensions of matrices A, C, and E, and Scale_Rank represents the output dimensions of B and the input dimensions of C; and use Gaussian to initialize matrices A, D, and C, and use zero to initialize B and E; Compared with the original low-rank decomposition matrices A and E of LoRA, Scale_Rank limits the low-rank matrices B, C, and D to smaller matrices, which can obtain more knowledge information under the condition of only increasing limited training parameters, which ensures the efficiency of training and further improves the model's ability to handle downstream tasks; Then knowledge extraction is performed, and the original data extracts initial features, low-rank features, and global complex features. Features of three different dimensions; It is worth noting that the optimized structure does not introduce additional delay because it is a parallel structure; Knowledge fusion stage: The three different dimensions of knowledge, namely the extracted initial features, low-rank features and global complex features, are fused and calculated, and the result of the fusion calculation is further added to the frozen weight knowledge as the final output result h, which is finally expressed as Based on the product of the input x, plus the features of each dimension obtained by fine-tuning The product of the input x is as follows: (1) In formula (1), ,in, represents the initial features extracted, represents the extracted low-rank features, Represents the extracted global complex features; represents the initial weight; x represents the input.

[0009] Preferably, according to the present invention, the extracting of initial features is based on optimizing the original adapter tuning; The original adapter tuning formula is as follows: (2) In formula (2), f represents the linear activation function, Represents a dimensional matrix, which projects the input state into a higher-dimensional space and increases the expressive power of the adapter. Refers to the dimension reduction matrix, which is used to reduce the dimension of the state after processing. It indicates the weight obtained after the dimension is raised or lowered; The optimization process formula for the original Adapter Tuning is as follows: (3) In formula (3), represents the initial features extracted, , , They are the dimension-upgrading matrix, the dimension-reducing matrix, and the matrix added by the optimization process, which are further expressed as low-rank matrices ， ， ; The extracted low-rank features are extracted from the original LoRA, and the formula is as follows: (4) In formula (4), represents the extracted low-rank features, and They are the dimension-raising and dimension-lowering matrices used for fine-tuning, which can be expressed as low-rank matrices ， ; The global complex features are extracted by introducing the Scale-Rank scaling operator to obtain the global information content, and further extracted. The formula is as follows: (5) In formula (5), represents the extracted global complex features, , Represents a common dimension-raising and lowering matrix, which can be expressed as a low-rank matrix ， , , Represents the scaling matrix used to extract complex deep features, which can be expressed as a low-rank matrix , .

[0010] Preferably, according to the present invention, step S2 is specifically as follows: First, search the TransformerLayer layer of the Transformer large model to find all linear layers that meet the insertion conditions; then insert the optimization operator FoRA structure into the q, k, v matrix of the Attention layer of the Transformer large model, and freeze the initial weights of the q, k, v matrix. At the same time, insert the optimization operator FoRA structure into all linear layers that meet the insertion conditions in the feedforward network layer FFN of the Transformer large model; it should be noted that the Transformer large model may generate a new head layer according to different tasks, and FoRA is also inserted into the newly generated head layer to extract information that is closer to the downstream tasks; through this step, the coverage of the linear layer of the entire Transformer large model is completed, further improving the model's performance in processing downstream tasks.

[0011] Preferably, according to the present invention, the Attention layer formula is as follows: (6) In formula (6), Q represents the query vector of the input data, K represents the key value vector, V represents the value vector, and T represents the transposition operation of the K vector. is the dimension of the key-value vector, which is used to prevent the dot product value from being too large, leading to the gradient vanishing problem. The softmax function converts it into a probability score, and finally obtains the attention score of V to Q. ; The formula of the feedforward network layer FFN is as follows, which represents the result after the input x is transformed by the feedforward network layer FFN: (7) In formula (7), and represents the weights and biases of the first layer, and Represents the weights and biases of the second layer.

[0012] Preferably, according to the present invention, step S3 is specifically as follows: Based on the Transformer large model that has completed the optimization operator adaptation, the orthogonal combination of quantization and federated learning is applied to specific fields: the global model on the server loads the original complete weight network and initializes the FoRA optimization operator, which is then sent to the client for local fine-tuning after quantization, as follows: First, the server obtains the complete structure and pre-trained weights of the Transformer large model, and then sends the quantized and compressed Transformer large model and the structure of the basic optimization operator FoRA to each client respectively; then, several clients use the basic optimization operator FoRA to perform local efficient fine-tuning training on the Transformer large model that has completed the optimization operator adaptation based on their own private domain data; during the local efficient fine-tuning training process, the weight file will be frozen locally, and the pre-trained weights will be quantized using INT4 to reduce the resource requirements of local training.

[0013] In order to ensure consistency with other clients when converging and updating the global model, the basic optimization operator FoRA uses BF16 for quantization for efficient local fine-tuning training on the client; after the client has performed several iterations of efficient fine-tuning training on the Transformer large model, each client participating in fine-tuning sends the updated low-rank matrix to the server for global model convergence; it is worth noting that the data set used in the training process is always stored in each company's own security infrastructure, which is the feature of the federated learning architecture to ensure the security of industry data. Finally, the server receives the low-rank matrices uploaded by different clients for aggregation and updates the weight file on the server. The weights and data involved in the entire efficient fine-tuning training process on the server are all quantized in BF16 format.

[0014] Preferably, according to the present invention, the INT4 quantization pre-training weights are used to reduce the resource requirements of local training, as shown in formula (8): (8) In formula (8), represents the downlink weight, , c1, c2 represent specific quantization constants, DQ represents the quantization process, k-bit represents the specifiable quantization form, including FP32, INT4 and NF4, represents the initial weight after quantization using NF4, , , , , They represent the low-rank matrices after using BF16 quantization, and finally the output tensor in BF16 form is obtained .

[0015] Preferably, according to the present invention, the weights and data involved in the entire efficient fine-tuning training process on the server side are all quantized in BF16 format, and the formula is as follows: (9) In formula (9), represents the input after quantization using BF16, and Represents the initial weight after using BF16 quantization form, , , , , They represent the low-rank matrix after using BF16 quantization, and finally obtain the quantized output uplink weight .

[0016] Compared with the prior art, the present invention has the following beneficial effects: (1) The FoRA optimization operator proposed in this paper uses multiple low-rank matrices to perform network layer decomposition and aggregation, which realizes the extraction of multi-dimensional deep-level complex nonlinear features and further improves the model's processing capabilities for downstream tasks.

[0017] (2) The present invention combines the idea of ​​federated learning to perform fine-tuning training, which greatly reduces the demand for computing resources in the training process and deeply protects data privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic diagram of fine-tuning of a large industry model for downstream fields based on a federated learning architecture.

[0019] Figure 2 This is a structural diagram of FoRA from a microscopic perspective according to the present invention.

[0020] Figure 3This is the adaptation process of the FoRA of the present invention in the linear layer of TransfomerLayer.

[0021] Figure 4 This is the F1 score comparison between the present invention and other methods in terms of question answering ability.

[0022] Figure 5 It is a comparison of EM scores between the present invention and other methods in terms of question-answering ability.

[0023] Figure 6 It is the comparison result of the micro-macro ablation experiment of the present invention. DETAILED DESCRIPTION

[0024] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0025] Embodiment 1, Ginseng Figure 1 This embodiment provides a large model optimization method for multi-matrix aggregation and multi-dimensional features, the method comprising: S1. Design a comprehensive and effective basic optimization operator FoRA: Fusion of Rank; The reason for adopting this method is that, under the same training parameter scale or the same rank, using FoRA as the core basic optimization operator, some dense layers in the neural network are indirectly trained by performing multiple low-rank decomposition matrices on the dense layers, and then aggregating knowledge through a multi-path method to extract solutions for complex nonlinear features in multiple dimensions and depths, which can further improve model performance in specific downstream task tests. The FoRA structure diagram from a micro perspective is shown in the figure below: Figure 2 As shown, Pretrained Weights is the pre-trained weight, expressed as a d×d matrix W, A, B, C, D, E are low-rank matrices for extracting features, γ2, r, scale-r are the parameters for matrix initialization, and x and h are input and output.

[0026] S2, adapting the basic optimization operator designed in step S1 to the entire TransformerLayer layer of the Transformer large model based on federated learning; S3. Quantize the Transformer large model that has completed the optimization operator adaptation in step S2 to reduce the resource requirements for training the large model. Then, based on federated learning, the client fine-tunes the quantized large model locally, and aggregates the trainable low-rank parameters that have been fine-tuned by the private domain hospitals or other industry data into the global model.

[0027] The basic optimization operator FoRA proposed in step S1 is introduced in detail as follows. Based on the idea of ​​the federated learning framework, the basic optimization operator FoRA divides the fine-tuning process into two stages: knowledge extraction stage and knowledge fusion stage; Knowledge extraction stage: Assuming that the pre-trained weight is W, the fine-tuned training parameters are expressed as W; First is initialization. FoRA will create five low-rank matrices A, B, C, D, E and two hyperparameters Rank and Scale_Rank for the pre-trained weight W, where Rank represents the output dimensions of matrices A, C and E, and Scale_Rank represents the output dimensions of B and the input dimensions of C; and use Gaussian to initialize matrices A, D, C, and initialize B and E with zero; Compared with the original low-rank decomposition matrices A and E of LoRA, Scale_Rank limits the low-rank matrices B, C, and D to smaller matrices, which can obtain more knowledge information while only increasing limited training parameters, which ensures the efficiency of training and further improves the model's ability to handle downstream tasks; Then knowledge extraction is performed, and the original data extracts features of three different dimensions: initial features, low-rank features, and global complex features. The extracting of initial features is based on optimizing the original adapter tuning; The original adapter tuning formula is as follows: (1) In formula (1), f represents the linear activation function, Represents a dimensional matrix, which projects the input state into a higher-dimensional space and increases the expressive power of the adapter. Refers to the dimension reduction matrix, which is used to reduce the dimension of the state after processing. It indicates the weight obtained after the dimension is raised or lowered; The optimization process formula for the original Adapter Tuning is as follows: (2) In formula (2), represents the initial features extracted, , , They are the dimension-upgrading matrix, the dimension-reducing matrix, and the matrix added by the optimization process, which are further expressed as low-rank matrices ， ， ; The extracted low-rank features are extracted from the original LoRA, and the formula is as follows: (3) In formula (3), represents the extracted low-rank features, and They are the dimension-raising and dimension-lowering matrices used for fine-tuning, which can be expressed as low-rank matrices ， ; The global complex features are extracted by introducing the Scale-Rank scaling operator to obtain the global information content, and further extracted. The formula is as follows: (4) In formula (4), represents the extracted global complex features, , Represents a common dimension-raising and lowering matrix, which can be expressed as a low-rank matrix ， , , Represents the scaling matrix used to extract complex deep features, which can be expressed as a low-rank matrix , ; It is worth noting that the optimized structure does not introduce additional delay because it is a parallel structure; Knowledge fusion stage: The three different dimensions of knowledge, namely the extracted initial features, low-rank features and global complex features, are fused and calculated, and the result of the fusion calculation and the frozen weight knowledge are further added as the final output result, which is finally expressed as the product of the initial weight W0 and the input x, plus the fine-tuned features of each dimension. The product of the input x is as follows: (5) In formula (5), , represents the initial weight and x represents the input.

[0028] like Figure 3 As shown, in addition to fine-tuning the Attention layer, step S2 also fine-tunes all linear layers such as the feedforward network FFN. Since the information of the Attention layer is more related to the semantic position, and the FFN layer is more related to the downstream tasks, based on this, the present invention combines the micro-optimization algorithm with multiple positions of the Transformer and applies it to each component of the Transformer, thereby further improving the accuracy of the model in processing downstream tasks. The details are as follows: First, search the TransformerLayer layer of the Transformer large model to find all linear layers that meet the insertion conditions; then insert the optimization operator FoRA structure into the q, k, v matrix of the Attention layer of the Transformer large model, and freeze the initial weights of the q, k, v matrix. At the same time, insert the optimization operator FoRA structure into all linear layers that meet the insertion conditions in the feedforward network layer FFN of the Transformer large model; it should be noted that the Transformer large model may generate a new head layer according to different tasks, and FoRA is also inserted into the newly generated head layer to extract information that is closer to the downstream tasks; through this step, the coverage of the linear layer of the entire Transformer large model is completed, further improving the model's performance in processing downstream tasks.

[0029] The Attention layer formula is as follows: (6) In formula (6), Q represents the query vector of the input data, K represents the key value vector, V represents the value vector, and T represents the transposition operation of the K vector. is the dimension of the key-value vector, which is used to prevent the dot product value from being too large, leading to the gradient vanishing problem. The softmax function converts it into a probability score, and finally obtains the attention score of V to Q. ; The formula of the feedforward network layer FFN is as follows, which represents the result after the input x is transformed by the feedforward network layer FFN: (7) In formula (7), and represents the weights and biases of the first layer, and Represents the weights and biases of the second layer.

[0030] The step S3 is specifically as follows: Based on the Transformer large model that has completed the optimization operator adaptation, the orthogonal combination of quantization and federated learning is applied to specific fields: the global model on the server loads the original complete weight network and initializes the FoRA optimization operator, which is then sent to the client for local fine-tuning after quantization, as follows: First, the server obtains the complete structure and pre-trained weights of the Transformer model, and then sends the quantized and compressed Transformer model and the structure of the basic optimization operator FoRA to each client. Then, several clients use the basic optimization operator FoRA to perform local efficient fine-tuning training on the Transformer model that has completed the optimization operator adaptation based on their own private domain data. During the local efficient fine-tuning training process, the weight file will be frozen locally, and the pre-trained weights will be quantized using INT4 to reduce the resource requirements of local training, that is, the model weights or activation values ​​are compressed to a 4-bit integer format, as shown in formula (8): (8) In formula (8), represents the downlink weight, , , Represents a specific quantization constant, DQ represents the quantization process, and k-bit represents the specifiable quantization form, including FP32, INT4, and NF4. represents the initial weight after quantization using NF4, , , , , They represent the low-rank matrices after using BF16 quantization, and finally the output tensor in BF16 form is obtained .

[0031] In order to ensure consistency with other clients when converging and updating the global model, the basic optimization operator FoRA uses BF16 for quantization for efficient fine-tuning training on the client side. After the client has performed several iterations of efficient fine-tuning training on the Transformer large model, each client participating in fine-tuning sends the updated low-rank matrix to the server side for global model convergence. It is worth noting that the data set used in the training process is always stored in the security infrastructure of each enterprise. This is the feature of the federated learning architecture to ensure the security of industry data. Finally, the server side receives the low-rank matrices uploaded by different clients for aggregation and updates the weight file on the server side. The weights and data involved in the entire efficient fine-tuning training process on the server side are all quantized in BF16 format. It should be pointed out that although quantization will increase training time, it has prompted the local model to further reduce the demand for resources. The quantization process is shown in formula (9): (9) In formula (9), represents the input after quantization using BF16, and Represents the initial weight after using BF16 quantization form, , , , , They represent the low-rank matrix after using BF16 quantization, and finally obtain the quantized output uplink weight .

[0032] The gain effect obtained by fine-tuning the Transformer large model after optimizing the operator adaptation using the overall optimization method of the present invention can be expressed by the model performance of formula (10), formula (11), formula (12) and the resource requirement of formula (13): (10) )(11) (12) (13) In the above formulas (10) and (11), and Represent the global model and the client-side fine-tuning model respectively, is the initial weight, represents the weight obtained after fine-tuning. Formula (12) indicates that the performance Performance of each client model after fine-tuning is weaker than the performance of the global model obtained by its aggregation. The computing resources Require source required for fine-tuning using the optimization method of the present invention are much smaller than the resources r required by the existing method, as shown in formula (14).

[0033] Experimental example On the hardware device NVIDIA A100 GPU, based on the publicly available Huggingface Transformers and deepspeed code libraries, PyTorch and Federated-LLM frameworks are used to verify the effectiveness of the basic optimization operator proposed in the present invention. The FoRA proposed in the microscopic perspective is applied to the entire mesoscopic perspective, called meso-FoRA, namely GlobalTransformer FoRA, which is used to fine-tune the DeBERTaV 3-base model. The experiment evaluates the effectiveness of the proposed meso-FoRA algorithm on the natural language understanding GLUE and the question-answering task SQuAD. In addition, in order to test the comprehensive and comprehensive performance of the algorithm, the experiment evaluates and analyzes the automobile fault extraction task for the mechanical field on the natural language generation task, and also combines the micro-macro algorithm with federated learning to fine-tune the Baichuan 13B model of tens of billions and the Baichuan 7B model of one billion. More importantly, in the medical field, which pays special attention to data privacy, the experiment applies the tens of billions of large models to actual downstream scenarios, and finds that the method of the present invention can complete tens of billions of scale pre-training while ensuring fine-tuning accuracy using only limited resources. Among them, the experiment compares FoRA with full fine-tuning, BitFit, LoRA, HAdapter, AdaLoRA and other methods.

[0034] 1. In the verification of natural language understanding ability, FoRA was compared with experiments using other baseline methods. The experimental results are shown in Table 1: Table 1 Experimental comparison of the understanding ability of the present invention and other fine-tuning methods on the GLUE development set method Parameter quantity MNLI Sst2 CoLA QQP QNLI RTE MRPC STSB ALL Full-fine-tuning 184 89.90 95.64 69.22 92.40 94.38 83.75 89.71 91.60 88.25 BitFit 0.1 89.37 94.84 66.96 88.41 92.24 78.70 87.75 91.35 86.20 HAdapter 1.22 90.13 95.53 68.64 91.91 94.11 84.48 89.95 91.48 88.28 PAdapter 1.21 90.33 95.61 68.77 92.04 94.29 85.20 89.46 91.54 88.41 LoRAr=8 1.33 90.45 95.87 69.77 91.99 93.92 85.92 89.70 91.72 88.66 AdaLoRA 1.27 90.35 96.10 71.02 92.23 94.31 88.09 90.44 91.63 89.27 LoRAr=16 2.65 90.55 95.75 69.92 92.39 93.70 87.73 89.95 91.71 88.96 meso-FoRA 1.66 90.67 96.10 72.21 93.56 94.45 88.81 91.42 91.99 89.90 It can be seen that FoRA has achieved better performance than existing methods on any dataset at the same rank level or the same parameter scale. Under the premise that the trainable parameters are only increased by 0.2% of the original weight, the overall accuracy is 0.63% higher than AdaLoRA and 1.23% higher than LoRAr=8. Considering that the method of the present invention adds additional trainable parameters, in order to ensure fairness, it is chosen to set the Rank of lora to 16, double the trainable parameters and then compare with the proposed FoRA. As shown in Table 1, the method of the present invention is still higher than when Lorar=16. Furthermore, compared with Lora with Rank set to 8, it can be observed that setting it to 16 does not improve the performance comprehensively, and even some natural language understanding subtasks are worse than Rank=8. Therefore, it can be considered that increasing efficient trainable parameters is the key to solving downstream tasks. Specifically, the experiment found that FoRA achieved an accuracy of 93.56% on QQP, which is 1.33% higher than the best baseline AdaLora. Cola improves by 1.19% over the best baseline, RTE improves by 0.72% over the best baseline, and MRPC (similarity and interpretation task dataset) improves by 0.98%, and the performance in these subtasks has been significantly improved.

[0035] 2. In the question-answering ability test, the QA task was evaluated on the SQuADv1.1 dataset. The proposed method was tested on LoRA, AdaLora and different ranks. The rank r was set to {1, 2, 4, 8}, the batch size was set to 16, and AdamW was used as the optimizer. The experimental results are shown in Figure 4 , Figure 5 As shown, Figure 4 The horizontal axis represents the rank of the low-rank matrix used, and the vertical axis represents the F1 precision score, which is an indicator used to measure the accuracy of the model. Figure 5 The horizontal axis represents the rank of the low-rank matrix, and the vertical axis represents the EM (Exact Match) score, which measures the proportion of generated answers that are completely consistent with the correct answer. Obviously, the method proposed in this paper is better than the best baseline AdaLora in terms of F1 score and EM score. 3. In order to comprehensively evaluate the algorithm generation capability based on the combination of micro-macro perspective and federated learning, the experiment used 13B billion-scale and 7B billion-scale models for evaluation of triple extraction tasks in the mechanical field. The method of the present invention and LoRA were evaluated on different ranks, and the client model and global model performance were evaluated for 13B and 7B large models based on the federated learning architecture. The results show that the performance of the quantized client model is weaker than that of the global model. This fact verifies the orthogonal combination of the large model and the federated learning framework. On the one hand, it plays the advantage of using quantization technology to reduce client resource requirements. On the other hand, it avoids the insufficient performance of the model after local quantization fine-tuning and limits the performance of the large model in downstream tasks, turning the disadvantages of quantization into advantages, proving the enthusiasm of the method proposed in the present invention.

[0036] It can be seen more clearly from Table 2: Table 2 Comparison results of the generation capability test between the present invention and other methods in the extraction of automobile fault tasks

[0037] The method of the present invention comprehensively surpasses the LoRA method in the task of automobile fault extraction. Further observation of the performance of the 13B and 7B models in the task of automobile fault extraction shows that the performance of the 13B model far exceeds that of the 7B model, which proves that the larger the model, the stronger the performance, thus verifying the advantage of introducing the 10 billion model into the federated learning architecture.

[0038] 4. In the question-answering test for the medical field that pays more attention to privacy, Chatdoctor data is used based on the federated architecture, and two groups of experiments are conducted with LoRA as the baseline and the method FoRA proposed in the present invention, respectively, to compare the performance of the client model and the global model. Use GPT-4 to rate the performance of different combinations on the benchmark test. Given a query, a reference answer, model_lora_fate, model_lora_client, model_FoRA_Fate, and model_FoRA_Client list are constructed. GPT-4 will be prompted to assign a full score of 10 to the four assistants based on the reference answer and provide an explanation. The experiment exchanges the order of the model assistant's answers and uses ChatGPT-4 to evaluate twice and take the average value to make the evaluation more accurate. The prediction results are evaluated using ChatGPT4. In order to make the evaluation fair, the experiment uses multiple weighted averages as the final result. It is found that the medical local model fine-tuned using FoRA as the optimization operator has more advantages in medical question generation.

[0039] 5. In terms of studying the contribution of each part to the accuracy, ablation experiments were conducted on micro, meso and macro methods. Lora was used as a reference for the ablation experiment to explore the impact of FoRA, TransformerLayer and quantization on the entire method. At the same time, the impact of different Scale_Rank on the overall accuracy was verified, such as Figure 6 As shown, the horizontal and vertical axes represent the percentage of trainable parameters and F1 accuracy compared to all parameters. In addition to Lora as the benchmark, the experiment also conducted 5 sets of experiments with different training parameters. Each set of experiments includes three parts: 1) fine-tuning the Attention layer only from a micro perspective; 2) fine-tuning all Linear layers from a meso perspective; 3) fine-tuning the large model from a unified micro-macro perspective through quantization technology and a federated learning framework. Figure 6 It shows that the method of the present invention is superior to LoRA in all aspects. Specifically, when the Scale_Rank used is set to 0.1 of the input dimension, it can be seen that the method of the present invention has a 6% higher performance than Lora, but the trainable parameters of the present invention are only increased by 0.016%. At the same time, it can be observed that the quantized algorithm affects the performance, and this shortcoming can be used as an advantage of the present invention combined with the federated learning framework. The central server performs non-quantitative learning, while the client uses quantitative learning to effectively protect the rights and interests of the model owner, and this method achieves dual protection of data and model. Figure 6 It can also be seen that as the number of trainable parameters gradually increases, the loss caused by quantization gradually decreases and tends to be stable.

[0040] 6. In order to comprehensively evaluate the processing performance of the algorithm proposed in this invention, the experiment compared the comprehensive performance of full-scale fine-tuning, meso-FoRA, and macro-FoRA combined with quantization technology in terms of video memory usage, training time usage, and throughput. The results are shown in Table 3: Table 3 Test results of comprehensive performance of the present invention

[0041] Obviously, compared with full parameter fine-tuning, the local fine-tuning method proposed by the present invention can greatly reduce the consumption of video memory by about 10 times, which also confirms the ability of the algorithm of the present invention to locally fine-tune models of tens of billions of scale.

Claims

1. A large model optimization method for multi-matrix aggregation and multi-dimensional features, characterized in that: The method comprises: S1. Design a basic optimization operator FoRA: Fusion of Rank; S2, adapting the basic optimization operator designed in step S1 to the entire TransformerLayer layer of the Transformer large model based on federated learning; S3. Quantize the Transformer large model that has completed the optimization operator adaptation in step S2 to reduce the resource requirements for training the large model. Then, based on federated learning, the client fine-tunes the quantized large model locally and aggregates the fine-tuned trainable low-rank parameters to the global model.

2. According to claim 1, a large model optimization method of multi-matrix aggregation and multi-dimensional features is characterized in that: The basic optimization operator FoRA proposed in step S1 is introduced in detail as follows. The basic optimization operator FoRA divides the fine-tuning process into two stages: knowledge extraction stage and knowledge fusion stage: Knowledge extraction stage: Assuming that the pre-trained weight is W, the fine-tuned training parameters are expressed as W; First, initialization. FoRA will create five low-rank matrices A, B, C, D, E, and two hyperparameters Rank and Scale_Rank for the pre-trained weight W, where Rank represents the output dimension of matrices A, C, and E, and Scale_Rank represents the output dimension of B and the input dimension of C; and use Gaussian to initialize matrices A, D, and C, and use zero to initialize B and E; compared with the original low-rank decomposition matrices A and E of LoRA, Scale_Rank limits the low-rank matrices B, C, and D to smaller matrices, which can obtain more knowledge information under the condition of only increasing limited training parameters; then knowledge extraction is performed, and the original data extracts initial features, low-rank features, and global complex features. Features of three different dimensions; Knowledge fusion stage: The three different dimensions of knowledge, namely the extracted initial features, low-rank features and global complex features, are fused and calculated, and the result of the fusion calculation is further added to the frozen weight knowledge as the final output result h, which is finally expressed as Based on the product of the input x, plus the features of each dimension obtained by fine-tuning The product of the input x is as follows: （1） In formula (1), ,in, represents the initial features extracted, represents the extracted low-rank features, Represents the extracted global complex features; represents the initial weight; x represents the input.

3. The large model optimization method of multi-matrix aggregation and multi-dimensional features according to claim 2 is characterized in that: The extracting of initial features is based on optimizing the original adapter tuning; The original adapter tuning formula is as follows: （2） In formula (2), f represents the linear activation function, Represents a dimensional matrix, which projects the input state into a higher-dimensional space and increases the expressive power of the adapter. Refers to the dimension reduction matrix, which is used to reduce the dimension of the state after processing. It indicates the weight obtained after the dimension is raised or lowered; The optimization process formula for the original Adapter Tuning is as follows: （3） In formula (3), represents the initial features extracted, , , They are the dimension-upgrading matrix, the dimension-reducing matrix, and the matrix added by the optimization process, which are further expressed as low-rank matrices ， ， ; The extracted low-rank features are extracted from the original LoRA, and the formula is as follows: （4） In formula (4), represents the extracted low-rank features, and They are the dimension-raising and lowering matrices used for fine-tuning, which can be expressed as low-rank matrices ， ; The global complex features are extracted by introducing the Scale-Rank scaling operator to obtain the global information content, and further extracted. The formula is as follows: （5） In formula (5), represents the extracted global complex features, , Represents a common dimension-raising and lowering matrix, which can be expressed as a low-rank matrix ， , , Represents the scaling matrix used to extract complex deep features, which can be expressed as a low-rank matrix , .

4. The large model optimization method of multi-matrix aggregation and multi-dimensional features according to claim 1 is characterized in that: The step S2 is specifically as follows: First, search the TransformerLayer layer of the Transformer large model to find all linear layers that meet the insertion conditions; then insert the optimization operator FoRA structure into the q, k, v matrix of the Attention layer of the Transformer large model, and freeze the initial weights of the q, k, v matrix. At the same time, insert the optimization operator FoRA structure into all linear layers that meet the insertion conditions in the feedforward network layer FFN of the Transformer large model; also insert FoRA into the newly generated head layer of the Transformer large model to extract information that is closer to the downstream tasks.

5. The large model optimization method of multi-matrix aggregation and multi-dimensional features according to claim 4 is characterized in that: The Attention layer formula is as follows: （6） In formula (6), Q represents the query vector of the input data, K represents the key value vector, V represents the value vector, and T represents the transposition operation of the K vector. is the dimension of the key-value vector, which is used to prevent the dot product value from being too large, leading to the gradient vanishing problem. The softmax function converts it into a probability score, and finally obtains the attention score of V to Q. ; The formula of the feedforward network layer FFN is as follows, which represents the result after the input x is transformed by the feedforward network layer FFN: （7） In formula (7), and represents the weights and biases of the first layer, and Represents the weights and biases of the second layer.

6. The large model optimization method of multi-matrix aggregation and multi-dimensional features according to claim 1 is characterized in that: The step S3 is specifically as follows: Based on the Transformer large model that has completed the optimization operator adaptation, the orthogonal combination of quantization and federated learning is applied to specific fields: First, the server obtains the complete structure and pre-trained weights of the Transformer large model, and then sends the quantized and compressed Transformer large model and the structure of the basic optimization operator FoRA to each client respectively; then, several clients use the basic optimization operator FoRA to perform local efficient fine-tuning training on the Transformer large model that has completed the optimization operator adaptation based on their own private domain data. During the local efficient fine-tuning training process, the weight file will be frozen locally, and the pre-trained weights will be quantized using INT4 to reduce the resource requirements of local training; The basic optimization operator FoRA uses BF16 for quantization and is used for efficient local fine-tuning training on the client side. After the client has performed several iterations of efficient fine-tuning training on the Transformer large model, each client participating in fine-tuning sends the updated low-rank matrix to the server for global model convergence; finally, the server receives the low-rank matrices uploaded by different clients for aggregation and updates the server's weight file. The weights and data involved in the entire efficient fine-tuning training process on the server are all quantized in BF16 format.

7. The large model optimization method of multi-matrix aggregation and multi-dimensional features according to claim 6 is characterized in that: The use of INT4 quantized pre-trained weights reduces the resource requirements for local training, as shown in formula (8): （8） In formula (8), represents the downlink weight, , c1, c2 represent specific quantization constants, DQ represents the quantization process, k-bit represents the specifiable quantization form, including FP32, INT4 and NF4, represents the initial weight after quantization using NF4, , , , , They represent the low-rank matrices after using BF16 quantization, and finally the output tensor in BF16 form is obtained .

8. The large model optimization method of multi-matrix aggregation and multi-dimensional features according to claim 6 is characterized in that: The weights and data involved in the entire efficient fine-tuning training process on the server side are all quantized in BF16 format, and the formula is as follows: （9） In formula (9), represents the input after BF16 quantization, and Represents the initial weight after using BF16 quantization form, , , , , They represent the low-rank matrix after using BF16 quantization, and finally obtain the quantized output uplink weight .

Images