Low-rank projection matrix learning based large language model zeroth-order fine-tuning method and system

Abstract

The application discloses a large language model zero-order fine-tuning method and system based on low-rank projection matrix learning, and belongs to the technical field of model fine-tuning, which comprises the following steps: in the process of learning a low-rank projection matrix, the parameters of a large language model to be used for text processing are frozen, small-batch training sample data is used, and the low-rank projection matrix is updated through zero-order optimization; the low-rank projection matrix is multiplied by a low-dimensional random matrix during the updating of model parameters, a disturbance of the same dimension as the model parameters is constructed, the low-rank disturbance space composed of the low-rank projection matrix is adjusted, and the disturbance direction in the space can be close to the descending direction of a loss function; after the low-rank disturbance matrix is constructed by using the learned low-rank projection matrix during the updating of model parameters, the zero-order gradient estimation of the large language model model parameters is performed in combination with all training sample data and is updated. In this way, the optimization efficiency in a high-dimensional parameter space is improved, the gradient variance is reduced, and the text processing capability of the large language model is improved.

Description

Technical Field

[0001] This invention belongs to the field of large language model fine-tuning technology, specifically involving a zero-order fine-tuning method for large language models based on low-rank projection matrix learning. Background Technology

[0002] With the rapid development of deep learning technology, Large Language Models (LLMs) have demonstrated astonishing capabilities in tasks such as natural language processing, code generation, and logical reasoning. However, as the number of model parameters grows exponentially (from billions to trillions of parameters), the computational resources and memory overhead required for full parameter fine-tuning of these models become extremely enormous, which greatly limits the application of LLMs in specific domains or on edge devices.

[0003] To address these issues, researchers have proposed various fine-tuning methods, such as the LoRA method, which freezes the weights of the pre-trained model and introduces two low-rank matrices with fewer parameters in the sidepath of a specific layer for training, significantly reducing computational cost and GPU memory usage. Although parameter-efficient fine-tuning methods reduce the number of trainable parameters, a large number of intermediate activation values ​​still need to be stored during backpropagation, making GPU memory usage a bottleneck.

[0004] To further reduce memory requirements, zero-order optimization techniques have been introduced into large model fine-tuning. These techniques only require forward propagation of the model, calculating the loss and estimating the gradient by perturbing the parameters, thus avoiding massive memory overhead. The classic MeZO method directly applies random perturbations in the high-dimensional parameter space, resulting in extremely large variance in gradient estimation, slow convergence, and instability. The subsequently proposed SubZero method uses a random Gaussian projection matrix to map a low-dimensional random matrix back to the high-dimensional parameter space, thereby constructing a low-rank perturbation space with the same dimension as the model parameters. However, the SubZero method uses a random Gaussian matrix as the projection matrix; this randomness leads to low optimization efficiency and a tendency to get trapped in local optima.

[0005] In summary, designing a zero-order optimization method that can learn from low-rank projection matrices and adjusting the low-rank perturbation space composed of low-rank projection matrices so that the perturbation direction in this space can be close to the descent direction of the loss function has become a key technological breakthrough for achieving efficient fine-tuning of large language models. Summary of the Invention

[0006] The technical problem this invention aims to solve is how to effectively reduce the variance of zero-order gradient estimation in high-dimensional parameter space during fine-tuning of large language models used for text processing, and overcome the randomness defects of existing SubZero methods. This is achieved by adjusting the low-rank perturbation space composed of low-rank projection matrices, so that the perturbation direction within this space is closer to the descent direction of the loss function, thereby improving the text processing performance of large language models. Therefore, the purpose of this invention is to propose a zero-order fine-tuning method for large language models based on low-rank projection matrix learning.

[0007] To achieve the above-mentioned objectives, an embodiment provides a zero-order fine-tuning method for a large language model based on low-rank projection matrix learning, comprising the following steps: Data processing: Processing text data and its corresponding labels to construct training sample data; Low-rank projection matrix learning: Freeze the parameters of the large language model for text processing to be fine-tuned, use mini-batch training sample data, and update the low-rank projection matrix through zero-order optimization. When the model parameters are updated, the low-rank projection matrix will be multiplied with the low-dimensional random matrix to construct a perturbation of the same dimension as the model parameters. By adjusting the low-rank perturbation space composed of the low-rank projection matrix, the perturbation direction in this space can be made closer to the descent direction of the loss function. Model parameter update: Construct a low-rank perturbation matrix using the learned low-rank projection matrix, and combine all training sample data and the low-rank perturbation matrix to estimate and update the zero-order gradient of the large language model parameters.

[0008] Preferably, the low-rank projection matrix is ​​multiplied with a low-dimensional random matrix during model parameter updates to construct a perturbation of the same dimension as the model parameters, including: Each iteration t At that time, the parameters of each layer of the large language model are... Independently sampled Gaussian random matrix ,Will , perturbation amplitude multiplied by the low-rank projection matrix Then add and subtract to the low-rank projection matrix of each level respectively. and Finally, the global positive parameter perturbation is obtained. and inverse parameter perturbation The superscript T denotes matrix transpose. Indicates the total number of model layers. For model layer index, This represents the set of all parameter perturbations.

[0009] Preferably, by adjusting the low-rank perturbation space composed of low-rank projection matrices, the perturbation direction within this space can be made closer to the descent direction of the loss function, including: The sampled small batch of training sample data Calculate the difference between the forward and backward parameter perturbation losses using a large language model as input, and update the low-rank projection matrix of each layer based on the difference results. and : ; ; ; ; ; ; in, The parameters of the large language model after t iterations. Indicates based on training sample data The calculated loss function and These are the forward and reverse parameter perturbation losses, respectively. and These are low-rank projection matrices. and gradient, The learning rate is represented by the symbol. For update operation.

[0010] Preferably, constructing a low-rank perturbation matrix using the learned low-rank projection matrix includes: For low-rank projection matrix and Perform QR decomposition separately to obtain orthogonal matrices. and As an orthogonal base and ,in, For model layer index; For each layer, an independently sampled Gaussian random matrix and combined with orthogonal bases and Construct the low-rank perturbation matrix for each layer The superscript T stands for matrix transpose.

[0011] Preferably, the zero-order gradient of the large language model parameters is estimated and updated by combining all training sample data and the low-rank perturbation matrix, including: Through the low-rank perturbation matrix Perturbation magnitude of model parameter updates The product of these factors constructs the positive and negative parameter perturbations for each layer, resulting in the global positive parameter perturbation. Inverse parameter perturbation ,in, Indicates the total number of model layers. This represents the set of all parameter perturbations; All training sample data Calculate the positive parameter perturbation loss from the input large language model and inverse parameter perturbation loss : ; ; Estimation of loss difference using forward and reverse parameter perturbation gradient Then update the model parameters : ; ; in, This is the learning rate.

[0012] Preferably, processing the text data includes word segmentation, truncation, and padding of the text, and the tags corresponding to the text data are the target token sequence.

[0013] To achieve the above-mentioned objectives, the embodiments also provide a zero-order fine-tuning system for large language models based on low-rank projection matrix learning, comprising: The data processing module is used to process text data and its corresponding labels to construct training sample data. The low-rank projection matrix learning module is used to freeze the parameters of a large language model for text processing that is to be fine-tuned. It uses mini-batch training sample data to update the low-rank projection matrix through zero-order optimization. When the model parameters are updated, the low-rank projection matrix is ​​multiplied with a low-dimensional random matrix to construct a perturbation of the same dimension as the model parameters. By adjusting the low-rank perturbation space composed of the low-rank projection matrix, the perturbation direction in this space can be made closer to the descent direction of the loss function. The model parameter update module is used to construct a low-rank perturbation matrix using the learned low-rank projection matrix, and to perform zero-order gradient estimation and update the model parameters of the large language model by combining all training sample data and the low-rank perturbation matrix.

[0014] To achieve the above-mentioned objectives, the embodiments also provide a computing device, including a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the above-mentioned zero-order fine-tuning method for large language models based on low-rank projection matrix learning.

[0015] To achieve the above-mentioned objectives, the embodiments also provide a computer-readable storage medium storing a program that, when executed by a processor, implements the above-mentioned zero-order fine-tuning method for large language models based on low-rank projection matrix learning.

[0016] To achieve the above-mentioned objectives, the embodiments also provide a computer product comprising a computer program, which, when executed by a processor, implements the above-mentioned zero-order fine-tuning method for large language models based on low-rank projection matrix learning.

[0017] Compared with the prior art, the beneficial effects of the present invention include at least the following: The parameter fine-tuning of the large language model is divided into two stages based on the text generation task. In the first stage, a projection matrix learning mechanism is proposed to solve the randomness defect of the SubZero method. By adjusting the low-rank perturbation space composed of the projection matrix, the perturbation direction in this space can be close to the descent direction of the loss function. In the second stage, by constructing a low-rank perturbation matrix, the parameter perturbation is strictly restricted to the low-rank space, thereby overcoming the difficulty of huge variance in zero-order gradient estimation in high-dimensional parameter space, significantly reducing the gradient estimation variance, and thus improving the model's fine-tuning performance, resulting in a significant improvement in the model's text processing capability. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the zero-order fine-tuning method for a large language model based on low-rank projection matrix learning provided in the embodiment; Figure 2 This is a detailed flowchart of the zero-order fine-tuning method for a large language model based on low-rank projection matrix learning provided in the embodiment; Figure 3 This is a schematic diagram of the structure of the zero-order fine-tuning system for a large language model based on low-rank projection matrix learning, provided in the embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0021] For large language models used for text processing, this embodiment provides a zero-order fine-tuning scheme for large language models based on low-rank projection matrix learning. This improves the optimization efficiency and reduces gradient variance in high-dimensional parameter space, while enhancing the text processing capabilities of large language models.

[0022] like Figure 1 As shown in the embodiment, a zero-order fine-tuning method for a large language model based on low-rank projection matrix learning is provided, including the following steps: S1, Data Processing: Processing text data and its corresponding labels to construct training sample data; S2, Low-rank projection matrix learning: Freeze the parameters of the large language model for text processing to be fine-tuned, use mini-batch training sample data, and update the low-rank projection matrix through zero-order optimization. When the model parameters are updated, the low-rank projection matrix will be multiplied with the low-dimensional random matrix to construct a perturbation of the same dimension as the model parameters. By adjusting the low-rank perturbation space composed of the low-rank projection matrix, the perturbation direction in this space can be close to the descent direction of the loss function. S3, Model Parameter Update: Construct a low-rank perturbation matrix using the learned low-rank projection matrix, and combine all training sample data and the low-rank perturbation matrix to estimate and update the zero-order gradient of the large language model parameters.

[0023] The alternating iteration of steps S2 and S3 above can achieve zero-order fine-tuning of the parameters of a large language model. To illustrate the zero-order fine-tuning process in detail, in a specific implementation, as follows... Figure 2 As shown, the detailed steps are as follows: Step 1: Construct the training sample dataset. Perform preprocessing on the original text data, such as word segmentation, truncation, and padding, to construct a training sample dataset containing the input sequence and label sequence, and divide it into several batches.

[0024] The original text dataset is preprocessed by first using a tokenizer corresponding to a pre-trained large language model for text processing to convert the text sequence into a token sequence. A maximum sequence length is then set. For lengths exceeding The sequence is truncated, and insufficient sequences are padded with padding characters to generate a uniform-dimensional input tensor. The processed data is then divided into batches according to a set batch size to generate a training sample dataset. .

[0025] Step 2: Initialize system parameters.

[0026] Loading parameters of a pre-trained large language model , Representing the The model parameters for each step. The number of layers that need fine-tuning for a large language model is... The model layer index is the learnable low-rank projection matrix initialized for each layer. and (Used to construct the low-rank perturbation matrix for each layer), where r is the dimension of the low-dimensional random space, usually set to a small value, satisfying... m and n are the first two digits of the large language model. The number of rows and columns in the layer model parameter matrix. The learning period for the low-rank projection matrix is ​​set. Learning Steps Disturbance amplitude Learning rate and normalized scaling factor and the magnitude of the perturbation in the model parameter update. and learning rate Hyperparameters, etc.

[0027] Step 3: Start iterative training and determine the current iteration step. Does the triggering condition (i.e., is t mod K equal to 0) meet? If the process fails, proceed to step 4 to begin learning the low-rank projection matrix; otherwise, proceed to step 7 to begin updating the model parameters.

[0028] Step 4: Freeze the parameters of the large language model to be fine-tuned. From the training set A small batch of training sample data was sampled. .

[0029] Step 5: Perform the projection on the low-rank matrix. Next iteration update.

[0030] This invention uses zero-order gradient estimation (ZO-SGD) to update the low-rank projection matrix. First, the model parameters for each layer are... Independently sampled Gaussian random matrix .Will perturbation amplitude multiplied by the low-rank projection matrix Then add and subtract to the low-rank projection matrix of each level respectively. and Finally, the global positive parameter perturbation is obtained. and inverse parameter perturbation .

[0031] Secondly, the sampled data Input model to calculate positive parameter perturbation loss and inverse parameter perturbation loss The formula is as follows: ; ; Then, the difference in perturbation loss between the positive and negative parameters is used to estimate... , gradient , Last updated and The formula is as follows: ; ; ; ; in, Indicates based on training sample data The calculated loss function, symbol For update operation.

[0032] In the low-rank projection matrix update, the large language model performs a zero-order optimization task of the low-rank projection matrix based on the input text sequence and the corresponding label sequence, and the calculated loss function adopts the cross-entropy loss function.

[0033] Step 6: Update the , Perform orthogonalization to extract orthogonal basis. , To maintain numerical stability, return to step 3.

[0034] To prevent , After multiple updates, column vectors may exhibit linear dependence or numerical explosion. This invention introduces QR decomposition. After the optimization step is completed, for each layer and The decomposition is as follows: ; ; in That is, the matrix of each layer Perform QR decomposition to obtain an orthogonal matrix. And upper triangular matrix, , Similarly, extract the orthogonal matrix. and This is a low-rank projection matrix used in subsequent stages.

[0035] Step 7: Construct the low-rank perturbation matrix for each layer based on the orthogonal basis. .

[0036] Sample a Gaussian random matrix for each layer. The low-rank perturbation matrix of each layer is constructed using the orthogonal basis from step 6. ,in .at this time, Although it is A matrix of dimension 1, but whose rank does not exceed 1. .

[0037] Step 8: Using the low-rank perturbation matrix Update the model parameters for each layer.

[0038] Through the low-rank perturbation matrix Perturbation magnitude of model parameter updates The product of these factors constructs the positive and negative parameter perturbations for each layer, ultimately yielding the global positive parameter perturbation. Inverse parameter perturbation Training data Input model to calculate positive parameter perturbation loss and inverse parameter perturbation loss The formula is as follows: ; ; Estimation of loss difference using forward and reverse parameter perturbation gradient: Last updated The formula is as follows: ; ; Step 9: Repeat steps 3-8 until the preset number of iterations is reached or the model converges, at which point training ends.

[0039] Repeat steps 3 through 8. Each time... In each iteration, the algorithm pauses model parameter updates and restarts the low-rank projection matrix learning. By adjusting the low-rank perturbation space composed of the low-rank projection matrices, the perturbation direction within this space is brought closer to the descent direction of the loss function. When the preset number of iterations is reached or the model converges, training stops, and the fine-tuned model parameters are output. .

[0040] This invention combines low-rank projection matrix learning with the construction of low-rank perturbation matrices, which significantly reduces the gradient estimation variance in high-dimensional parameter spaces. It has broad application prospects in scenarios such as large language model customization in vertical domains and local fine-tuning of edge devices.

[0041] like Figure 3As shown in the embodiment, a zero-order fine-tuning system for a large language model based on low-rank projection matrix learning is provided. This system includes a data processing module, a low-rank projection matrix learning module, and a model parameter update module. The data processing module processes text data and its corresponding labels to construct training sample data. The low-rank projection matrix learning module freezes the parameters of the large language model to be fine-tuned for text processing. Using a small batch of training sample data, it updates the low-rank projection matrix through zero-order optimization. During model parameter updates, the low-rank projection matrix is ​​multiplied with a low-dimensional random matrix to construct a perturbation of the same dimension as the model parameters. By adjusting the low-rank perturbation space composed of the low-rank projection matrix, the perturbation direction within this space is made closer to the descent direction of the loss function. The model parameter update module constructs a low-rank perturbation matrix using the learned low-rank projection matrix and, combined with all training sample data and the low-rank perturbation matrix, performs zero-order gradient estimation and updates the large language model parameters.

[0042] It should be noted that the zero-order fine-tuning system for large language models based on low-rank projection matrix learning provided in the above embodiments should be illustrated using the above-described functional module division as an example when performing zero-order fine-tuning of large language models based on low-rank projection matrix learning. The functions described above can be assigned to different functional modules as needed, i.e., the internal structure of the terminal or server can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the zero-order fine-tuning system for large language models based on low-rank projection matrix learning provided in the above embodiments and the zero-order fine-tuning method embodiments for large language models based on low-rank projection matrix learning belong to the same concept. For details of its implementation process, please refer to the zero-order fine-tuning method embodiments for large language models based on low-rank projection matrix learning, which will not be repeated here.

[0043] Based on the same inventive concept, the embodiment also provides a computing device, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the above-described zero-order fine-tuning method for a large language model based on low-rank projection matrix learning, specifically including the following steps: S1, Data Processing: Processing text data and its corresponding labels to construct training sample data; S2, Low-rank projection matrix learning: Freeze the parameters of the large language model for text processing to be fine-tuned, use mini-batch training sample data, and update the low-rank projection matrix through zero-order optimization. When the model parameters are updated, the low-rank projection matrix will be multiplied with the low-dimensional random matrix to construct a perturbation of the same dimension as the model parameters. By adjusting the low-rank perturbation space composed of the low-rank projection matrix, the perturbation direction in this space can be close to the descent direction of the loss function. S3, Model Parameter Update: Construct a low-rank perturbation matrix using the learned low-rank projection matrix, and combine all training sample data and the low-rank perturbation matrix to estimate and update the zero-order gradient of the large language model parameters.

[0044] The computing device provided in this embodiment, at the hardware level, includes not only a processor and memory, but also internal buses, network interfaces, memory, and other hardware required for business operations. The memory is non-volatile memory. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the zero-order fine-tuning method for large language models based on low-rank projection matrix learning described in S1-S3 above. Of course, besides software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0045] Based on the same inventive concept, the embodiments also provide a computer-readable storage medium storing a program that, when executed by a processor, implements the above-described zero-order fine-tuning method for large language models based on low-rank projection matrix learning, specifically including the following steps: S1, Data Processing: Processing text data and its corresponding labels to construct training sample data; S2, Low-rank projection matrix learning: Freeze the parameters of the large language model for text processing to be fine-tuned, use mini-batch training sample data, and update the low-rank projection matrix through zero-order optimization. When the model parameters are updated, the low-rank projection matrix will be multiplied with the low-dimensional random matrix to construct a perturbation of the same dimension as the model parameters. By adjusting the low-rank perturbation space composed of the low-rank projection matrix, the perturbation direction in this space can be close to the descent direction of the loss function. S3, Model Parameter Update: Construct a low-rank perturbation matrix using the learned low-rank projection matrix, and combine all training sample data and the low-rank perturbation matrix to estimate and update the zero-order gradient of the large language model parameters.

[0046] In this embodiment, the computer-readable medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data.

[0047] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A zero-order fine-tuning method for a large language model based on low-rank projection matrix learning, characterized in that, Includes the following steps: Data processing: Processing text data and its corresponding labels to construct training sample data; Low-rank projection matrix learning: Freeze the parameters of the large language model for text processing to be fine-tuned, and update the low-rank projection matrix through zero-order optimization using mini-batch training sample data. The low-rank projection matrix is ​​multiplied with a low-dimensional random matrix during model parameter updates, constructing a perturbation of the same dimension as the model parameters, including: in each iteration t At that time, the parameters of each layer of the large language model are... Independently sampled Gaussian random matrix ,Will , perturbation amplitude multiplied by the low-rank projection matrix Then add and subtract to the low-rank projection matrix of each level respectively. and Finally, the global positive parameter perturbation is obtained. and inverse parameter perturbation The superscript T denotes matrix transpose. Indicates the total number of model layers. For model layer index, This represents the set of all parameter perturbations; By adjusting the low-rank perturbation space composed of low-rank projection matrices, the perturbation direction within this space is made closer to the descent direction of the loss function. This includes: adjusting the sampled mini-batch training data... Calculate the difference between the forward and backward parameter perturbation losses using a large language model as input, and update the low-rank projection matrix of each layer based on the difference results. and : in, The parameters of the large language model after t iterations. Indicates based on training sample data The calculated loss function and These are the forward and reverse parameter perturbation losses, respectively. and These are low-rank projection matrices. and gradient, The learning rate is represented by the symbol. For update operation; Model parameter update: Construct a low-rank perturbation matrix using the learned low-rank projection matrix, and combine all training sample data and the low-rank perturbation matrix to estimate and update the zero-order gradient of the large language model parameters.

2. The zero-order fine-tuning method for large language models based on low-rank projection matrix learning according to claim 1, characterized in that, Constructing a low-rank perturbation matrix using the learned low-rank projection matrix includes: For low-rank projection matrix and Perform QR decomposition separately to obtain orthogonal matrices. and As an orthogonal base and ,in, For model layer index; For each layer, an independently sampled Gaussian random matrix and combined with orthogonal bases and Construct the low-rank perturbation matrix for each layer The superscript T stands for matrix transpose.

3. The zero-order fine-tuning method for large language models based on low-rank projection matrix learning according to claim 2, characterized in that, The zero-order gradient of the large language model parameters is estimated and updated by combining all training sample data and the low-rank perturbation matrix, including: Through the low-rank perturbation matrix Perturbation magnitude of model parameter updates The product of these factors constructs the positive and negative parameter perturbations for each layer, resulting in the global positive parameter perturbation. Inverse parameter perturbation ,in, Indicates the total number of model layers. This represents the set of all parameter perturbations; All training sample data Calculate the positive parameter perturbation loss from the input large language model and inverse parameter perturbation loss : Estimation of loss difference using forward and reverse parameter perturbation gradient Then update the model parameters : in, This is the learning rate.

4. The zero-order fine-tuning method for large language models based on low-rank projection matrix learning according to claim 1, characterized in that, Processing text data includes word segmentation, truncation, and padding. The tags corresponding to the text data are the target token sequence.

5. A zero-order fine-tuning system for a large language model based on low-rank projection matrix learning, characterized in that, include: The data processing module is used to process text data and its corresponding labels to construct training sample data. The low-rank projection matrix learning module is used to freeze the parameters of a large language model for text processing that is to be fine-tuned, and to update the low-rank projection matrix through zero-order optimization using mini-batch training sample data. The low-rank projection matrix is ​​multiplied with a low-dimensional random matrix during model parameter updates, constructing a perturbation of the same dimension as the model parameters, including: in each iteration t At that time, the parameters of each layer of the large language model are... Independently sampled Gaussian random matrix ,Will , perturbation amplitude multiplied by the low-rank projection matrix Then add and subtract to the low-rank projection matrix of each level respectively. and Finally, the global positive parameter perturbation is obtained. and inverse parameter perturbation The superscript T denotes matrix transpose. Indicates the total number of model layers. For model layer index, This represents the set of all parameter perturbations; By adjusting the low-rank perturbation space composed of low-rank projection matrices, the perturbation direction within this space is made closer to the descent direction of the loss function. This includes: adjusting the sampled mini-batch training data... Calculate the difference between the forward and backward parameter perturbation losses using a large language model as input, and update the low-rank projection matrix of each layer based on the difference results. and : in, The parameters of the large language model after t iterations. Indicates based on training sample data The calculated loss function and These are the forward and reverse parameter perturbation losses, respectively. and These are low-rank projection matrices. and gradient, The learning rate is represented by the symbol. For update operation; The model parameter update module is used to construct a low-rank perturbation matrix using the learned low-rank projection matrix, and to estimate and update the zero-order gradient of the large language model parameters by combining all training sample data and the low-rank perturbation matrix.

6. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the one or more processors execute the executable code, they are used to implement the zero-order fine-tuning method for large language models based on low-rank projection matrix learning as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the zero-order fine-tuning method for large language models based on low-rank projection matrix learning as described in any one of claims 1-4.

8. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the zero-order fine-tuning method for large language models based on low-rank projection matrix learning as described in any one of claims 1-4.

Images