Language model federated fine-tuning method and device, and electronic device

Abstract

This application provides a language model federated fine-tuning method, apparatus, and electronic device. The method includes: receiving first information sent by N clients, the first information including data importance indicators of the clients; determining the ideal rank of the N clients based on the data importance indicators and a preset total rank budget; determining the assigned rank of the N clients satisfying preset constraints based on the ideal rank of the N clients; the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint; sending the corresponding assigned rank to each of the N clients, the assigned rank being used to train the local low-rank adaptive matrix of the clients; receiving the trained local low-rank adaptive matrix sent by the N clients; and adjusting the weight matrix of the language model based on the trained local low-rank adaptive matrix of the N clients. This can improve the performance of the language model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus and electronic device for federated fine-tuning of a language model. Background Technology

[0002] Large Language Models (LLMs) demonstrate exceptional generalization capabilities across various natural language processing tasks. However, despite their powerful general representation capabilities, they often fail to achieve ideal performance on specific downstream tasks without targeted adaptation. While direct full-parameter fine-tuning of the entire LLM can improve task adaptability, it suffers from high computational costs. To address these issues, Parameter-Efficient Fine-Tuning (PEFT) methods have emerged, among which Low-Rank Adaptation (LoRA) has become one of the mainstream PEFT techniques due to its superior performance and lower resource overhead.

[0003] To achieve privacy-preserving fine-tuning of language models, related technologies have attempted to combine LoRA with federated learning. This requires assigning LoRA ranks to each client participating in federated learning. However, the current common practice in assigning ranks to clients is to assign the same rank to each client, which can easily lead to poor performance of the trained language model. Summary of the Invention

[0004] This application provides a language model federated fine-tuning method, apparatus, and electronic device to address the problem of poor performance of existing language models.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a language model federated fine-tuning method, applied to a server, the method comprising:

[0007] Receive first information sent by N clients, the first information including the data importance index of the client, the data importance index of the client is used to represent the importance of the client's local training data to the language model, and N is an integer greater than 1;

[0008] Based on the data importance index and the preset total rank budget, the ideal rank of the N clients is determined;

[0009] Based on the ideal rank of the N clients, determine the allocation rank of the N clients that satisfies the preset constraints; wherein the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint;

[0010] The corresponding allocation rank is sent to each of the N clients. The allocation rank is used to train the local low-rank adaptive matrix of the client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of input dimensions of the language model, m is the number of output dimensions of the language model, and r is the allocation rank of the client. r is less than the minimum value of d and m.

[0011] Receive the trained local low-rank adaptive matrix sent by the N clients;

[0012] The weight matrix of the language model is adjusted based on the local low-rank adaptive matrices trained on the N clients.

[0013] Secondly, embodiments of this application provide a language model federated fine-tuning method, applied to a first client, the method comprising:

[0014] Send first information to the server, the first information including the data importance index of the first client, the data importance index of the first client is used to represent the importance of the local training data of the first client to the language model;

[0015] The server sends an allocation rank assigned to the first client; the allocation rank of the first client is an allocation rank that satisfies preset constraints, determined by the ideal ranks of N clients; wherein, the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint, the ideal rank of the N clients is the rank determined by the data importance index of the N clients and the preset total rank budget, N is an integer greater than 1, and the first client is one of the N clients;

[0016] Based on the assigned rank of the first client and the local training data of the first client, a local low-rank adaptive matrix of the first client is trained. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of input dimensions of the language model, m is the number of output dimensions of the language model, r is the assigned rank of the first client, and r is less than the minimum value of d and m.

[0017] The trained local low-rank adaptive matrix of the first client is sent to the server. The trained local low-rank adaptive matrix of the first client is used to adjust the weight matrix of the language model.

[0018] Thirdly, embodiments of this application provide a language model federated fine-tuning device, applied to a server, the device comprising:

[0019] The first receiving module is used to receive first information sent by N clients. The first information includes the data importance index of the client. The data importance index of the client is used to represent the importance of the client's local training data to the language model. N is an integer greater than 1.

[0020] The first determining module is used to determine the ideal rank of the N clients based on the data importance index and the preset total rank budget;

[0021] The second determining module is used to determine the allocation rank of the N clients that satisfies preset constraints based on the ideal rank of the N clients; wherein the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint.

[0022] The first sending module is used to send the corresponding allocation rank to each of the N clients. The allocation rank is used to train the local low-rank adaptive matrix of the client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of input dimensions of the language model, m is the number of output dimensions of the language model, and r is the allocation rank of the client. r is less than the minimum value of d and m.

[0023] The second receiving module is used to receive the trained local low-rank adaptive matrix sent by the N clients;

[0024] The adjustment module is used to adjust the weight matrix of the language model based on the trained local low-rank adaptive matrix of the N clients.

[0025] Fourthly, embodiments of this application provide a language model federated fine-tuning device, applied to a first client, the device comprising:

[0026] The second sending module is used to send first information to the server. The first information includes the data importance index of the first client. The data importance index of the first client is used to represent the importance of the local training data of the first client to the language model.

[0027] The third receiving module is used to receive the allocation rank assigned to the first client sent by the server; the allocation rank of the first client is an allocation rank that satisfies preset constraints and is determined by the ideal ranks of N clients; wherein, the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint and rank range constraint, the ideal rank of the N clients is the rank determined by the data importance index of the N clients and the preset total rank budget, N is an integer greater than 1, and the first client is one of the N clients;

[0028] The training module is used to train the local low-rank adaptive matrix of the first client based on the assigned rank of the first client and the local training data of the first client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns, where d is the number of input dimensions of the language model, m is the number of output dimensions of the language model, r is the assigned rank of the first client, and r is less than the minimum value of d and m.

[0029] The third sending module is used to send the trained local low-rank adaptive matrix of the first client to the server. The trained local low-rank adaptive matrix of the first client is used to adjust the weight matrix of the language model.

[0030] Fifthly, embodiments of this application provide an electronic device, characterized in that it includes a transceiver and a processor.

[0031] The processor is used for:

[0032] Receive first information sent by N clients, the first information including the data importance index of the client, the data importance index of the client is used to represent the importance of the client's local training data to the language model, and N is an integer greater than 1;

[0033] Based on the data importance index and the preset total rank budget, the ideal rank of the N clients is determined;

[0034] Based on the ideal rank of the N clients, determine the allocation rank of the N clients that satisfies the preset constraints; wherein the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint;

[0035] The corresponding allocation rank is sent to each of the N clients. The allocation rank is used to train the local low-rank adaptive matrix of the client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the allocation rank of the client. r is less than the minimum value of d and m.

[0036] Receive the trained local low-rank adaptive matrix sent by the N clients;

[0037] The weight matrix of the language model is adjusted based on the local low-rank adaptive matrices trained on the N clients.

[0038] Sixthly, embodiments of this application provide an electronic device, characterized in that it includes a transceiver and a processor.

[0039] The processor is used for:

[0040] Send first information to the server, the first information including the data importance index of the first client, the data importance index of the first client is used to represent the importance of the local training data of the first client to the language model;

[0041] The server sends an allocation rank assigned to the first client; the allocation rank of the first client is an allocation rank that satisfies preset constraints, determined by the ideal ranks of N clients; wherein, the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint, the ideal rank of the N clients is the rank determined by the data importance index of the N clients and the preset total rank budget, N is an integer greater than 1, and the first client is one of the N clients;

[0042] Based on the assigned rank of the first client and the local training data of the first client, a local low-rank adaptive matrix of the first client is trained. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the assigned rank of the first client. r is less than the minimum value of d and m.

[0043] The trained local low-rank adaptive matrix of the first client is sent to the server. The trained local low-rank adaptive matrix of the first client is used to adjust the weight matrix of the language model.

[0044] In a seventh aspect, embodiments of this application provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the steps of the language model federated fine-tuning method described in the first aspect, or implements the steps of the language model federated fine-tuning method described in the second aspect.

[0045] Eighthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the language model federated fine-tuning method described in the first aspect, or implements the steps of the language model federated fine-tuning method described in the second aspect.

[0046] Ninthly, embodiments of this application provide a computer program product including computer instructions that, when executed by a processor, implement the steps of the method as described in the first or second aspect above.

[0047] In the language model federated fine-tuning method of this application embodiment, the data importance index of the local training data sent by each client can be obtained. Based on the data importance index and preset constraints, a corresponding rank is assigned to each client. That is, in the process of assigning ranks to each client, the data importance index of each client is considered, as well as at least one of the constraints of efficiency constraint, rank budget constraint and rank range constraint. The assigned ranks of N clients that meet the constraints are determined, which improves the rationality of the assigned ranks of the clients. Afterwards, each client can understand and receive the assigned ranks to train the local low-rank adaptive matrix locally, which can improve the local training effect. Subsequently, the server can adjust the weight matrix of the language model based on the trained local low-rank adaptive matrix sent by N clients to complete the language model training, which can improve the training effect of the language model and thus improve the performance of the trained language model. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is one of the flowcharts of a language model federated fine-tuning method provided in the embodiments of this application;

[0050] Figure 2 This is the second flowchart of a language model federated fine-tuning method provided in the embodiments of this application;

[0051] Figure 3 This is an overall schematic diagram of a language model federated fine-tuning method provided in an embodiment of this application;

[0052] Figure 4 This is a schematic diagram of the stacked aggregation principle based on Singular Value Decomposition (SVD) in a language model federated fine-tuning method provided in this application embodiment;

[0053] Figure 5 This is one of the structural schematic diagrams of a language model federated fine-tuning device provided in the embodiments of this application;

[0054] Figure 6 This is a second schematic diagram of the structure of a language model federated fine-tuning device provided in the embodiments of this application;

[0055] Figure 7 This is one of the structural schematic diagrams of an electronic device provided in the embodiments of this application;

[0056] Figure 8 This is a second schematic diagram of the structure of an electronic device provided in the embodiments of this application. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] Introduction to the relevant technologies involved in this application:

[0059] LoRA technology maintains the original weight matrix W0 (d rows, m columns) of the model being trained unchanged, while introducing two low-rank matrices A (r rows, m columns, also known as low-rank adaptive matrices) and B (d rows, r columns), where r represents the rank (also known as the LoRA rank), which is less than the minimum of d and m. Compared to the weight matrix W, the dimensions of the low-rank matrices A and B are reduced. In the multi-round fine-tuning (training) process based on LoRA, only A and B are updated (training / fine-tuning). After fine-tuning A and B, the first weight increment matrix (ΔW=BA) is determined based on the trained A and B and added to the original weight matrix W, thus completing the overall training of the model. In this way, in the LoRA-based model fine-tuning process, each round of fine-tuning only requires updating A and B, without adjusting W in each round. The adjustment of W0 is completed simply by adding the first weight increment matrix to the entire W0 after fine-tuning A and B, reducing the computational load during model training.

[0060] Federated learning can be understood as training a model through the collaborative efforts of multiple participants, such as servers and clients. Through collaboration between the server and clients, a global model is trained. Clients train locally, and the server aggregates the model parameters obtained from each client's local training to update the global model parameters on the server. The updated global model parameters are then distributed to each client, which performs the next round of training locally until training stops (e.g., model convergence or reaching a preset maximum training threshold). At this point, training ceases, and the final global model is obtained, completing the federated learning process.

[0061] Applying LoRA to model training requires the use of r. Federated learning involves multiple clients, requiring an assigned r for each client so that each client can perform local training using the received r and its local training data. However, current techniques often assign the same r to all clients, which can easily lead to poor performance of the trained language model.

[0062] Based on this, this application provides a language model federated fine-tuning method. It can obtain the data importance index of the local training data sent by each client, and assign a corresponding rank to each client based on the data importance index and preset constraints. Specifically, in the process of assigning ranks to each client, the data importance index of each client is considered, along with at least one of the constraints: efficiency constraint, rank budget constraint, and rank range constraint. This determines the assigned ranks of N clients that meet the constraints, improving the rationality of the assigned ranks. Subsequently, each client can understand and receive the assigned rank and use it to train its local low-rank adaptive matrix locally, improving the local training effect. The server can then adjust the weight matrix of the language model based on the trained local low-rank adaptive matrices sent by the N clients to complete the language model training, further improving the training effect and thus enhancing the performance of the trained language model.

[0063] See Figure 1 , Figure 1 This is a flowchart of a language model federated fine-tuning method provided in an embodiment of this application, applied to a server, such as... Figure 1 As shown, the language model federated fine-tuning method provided in this embodiment includes the following steps:

[0064] Step 101: Receive the first information sent by N clients. The first information includes the client's data importance index, which is used to represent the importance of the client's local training data to the language model. N is an integer greater than 1.

[0065] In this embodiment, the N clients can be understood as clients participating in the federated fine-tuning of the language model. The server and the N clients collaborate to complete the fine-tuning of the language model, i.e., to achieve federated training of the language model. In the server, a global weight increment matrix (also called a global LoRA adapter) with dimensions of d rows and m columns can be initialized first. This global weight increment matrix can be distributed to each client. After each round of fine-tuning, the server can distribute the fine-tuned global weight increment matrix to each client, and each client can update its local global weight increment matrix to determine the data importance index of its local training data. For example, each client can store the global weight increment matrix sent by the server. After each round of fine-tuning, the corresponding round's global weight increment matrix is ​​obtained. As an example, in each round of fine-tuning, the client can calculate the multi-level gradients (e.g., weight gradients) of its local training data on the global weight increment matrix of the previous round of fine-tuning. The sum of the norms of at least some of the multi-level gradients can be determined as the data importance index of the local training data, etc. It should be understood that the global weight increment matrix can be represented as the product of a first global low-rank adaptive matrix with dimensions of d rows and Rmax columns, and a second global low-rank adaptive matrix with dimensions of Rmax rows and m columns, where Rmax is a preset rank. The value of Rmax can be preset according to actual needs or historical experience, and the global weight increment matrix has dimensions of d rows and m columns.

[0066] Step 102: Determine the ideal rank for N clients based on the data importance index and the preset total rank budget.

[0067] The preset total rank budget, also known as the preset new total rank budget, can be pre-set based on actual needs or historical experience. As an example, a client's ideal rank can be positively correlated with the data importance index and the preset total rank budget, and negatively correlated with the sum of the data importance indices of N clients. As another example, a client's ideal rank can be the result of rounding a first value, which can be the product of the preset total rank budget and a first ratio, where the first ratio is the ratio of the client's data importance index to the total data importance index, and the total data importance index is the sum of the data importance indices of N clients.

[0068] Step 103: Determine the allocation rank of the N clients that satisfy the preset constraints based on the ideal rank of the N clients; wherein the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint.

[0069] In the process of rank allocation for clients, constraints are introduced in addition to the ideal rank to determine the allocation rank of N clients under the preset constraints.

[0070] Step 104: Send the corresponding assigned rank to each of the N clients. The assigned rank is used to train the local low-rank adaptive matrix of the client.

[0071] The local low-rank adaptive matrix consists of a first local low-rank adaptive matrix with d rows and r columns, and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows in the weight matrix of the language model, m is the number of columns in the weight matrix of the language model, and r is the assigned rank of the client. r is less than the minimum of d and m.

[0072] Step 105: Receive the trained local low-rank adaptive matrix sent by N clients.

[0073] After receiving the corresponding assigned rank, the client can combine the assigned rank and local training data to train its own local low-rank adaptive matrix, and then send the trained local low-rank adaptive matrix to the server. In this way, the server can receive trained local low-rank adaptive matrices sent by N clients.

[0074] Step 106: Adjust the weight matrix of the language model based on the local low-rank adaptive matrix trained on N clients.

[0075] The server can aggregate the trained local low-rank adaptive matrices from N clients to adjust the weight matrix of the language model later.

[0076] In the language model federated fine-tuning method of this application embodiment, the data importance index of the local training data sent by each client can be obtained. Based on the data importance index and preset constraints, a corresponding rank is assigned to each client. That is, in the process of assigning ranks to each client, the data importance index of each client is considered, as well as at least one of the constraints of efficiency constraint, rank budget constraint and rank range constraint. The assigned ranks of N clients that meet the constraints are determined, which improves the rationality of the assigned ranks of the clients. Then, each client can understand and receive the assigned ranks to train the local low-rank adaptive matrix locally, which can improve the local training effect. Subsequently, the server can adjust the weight matrix of the language model based on the trained local low-rank adaptive matrix sent by N clients to complete the language model training, which can improve the training effect of the language model and thus improve the performance of the trained language model.

[0077] In one embodiment, the weight matrix of the language model is adjusted based on the trained local low-rank adaptive matrix from N clients, including:

[0078] Based on the local low-rank adaptive matrix trained by N clients, determine the contribution of N clients in this round of fine-tuning;

[0079] The aggregate weight of the N clients is determined based on their contribution in this round of fine-tuning.

[0080] Based on the aggregate weights of N clients, the first local low-rank adaptive matrix trained by the N clients is weighted and stacked to obtain the first aggregated low-rank adaptive matrix, and the second local low-rank adaptive matrix trained by the N clients is weighted and stacked to obtain the aggregated second aggregated low-rank adaptive matrix.

[0081] The first weight increment matrix is ​​determined based on the first aggregated low-rank adaptive matrix and the second aggregated low-rank adaptive matrix.

[0082] Adjust the weight matrix of the language model based on the first weight increment matrix.

[0083] In this embodiment, the contribution of each of the N clients in this round of fine-tuning can be determined using the local low-rank adaptive matrices trained on the N clients. Thus, the aggregate weights of the N clients are determined based on their contributions in this round of fine-tuning, and the aggregate weights of each client are positively correlated with their contributions. Then, the first aggregated low-rank adaptive matrix is ​​obtained by weighted stacking of the first local low-rank adaptive matrices trained on the N clients using the aggregated weights. Similarly, the second aggregated low-rank adaptive matrix is ​​obtained by weighted stacking of the second local low-rank adaptive matrices trained on the N clients using the aggregated weights. The first weight increment matrix is ​​determined using the first and second aggregated low-rank adaptive matrices, and this is used to adjust the weight matrix of the language model, achieving federated training of the language model and improving its performance.

[0084] The language model in this application can be applied to multiple scenarios, such as, but not limited to, question-and-answer scenarios.

[0085] As an example, the first weight increment matrix can be the product of the first aggregated low-rank adaptive matrix and the second aggregated low-rank adaptive matrix.

[0086] As an example, before receiving the first information sent by N clients, the process may further include: receiving evaluation datasets sent by each of the N clients. The client's evaluation dataset is generated by the client based on the statistical characteristics of its local training data. The statistical characteristics of the client's evaluation dataset match (e.g., are the same) the statistical characteristics of its local training data. The client can send the generated evaluation dataset to the server. This ensures that the local training data does not leave the local machine. By reporting the evaluation dataset with matching statistical characteristics to the server, the privacy and security of the local training data are ensured, while also ensuring the accuracy of evaluating the client's contribution in this round of fine-tuning through the evaluation dataset.

[0087] As an example, before receiving the first information sent by N clients and after receiving the evaluation dataset sent by each of the N clients, the method may further include: sending the global weight increment matrix of the previous round of fine-tuning to the N clients.

[0088] In some embodiments, adjusting the weight matrix of the language model according to the first weight increment matrix includes:

[0089] Add the first weight increment matrix to the global weight increment matrix of the previous round of fine-tuning in the server to obtain the intermediate matrix. The global weight increment matrix of the previous round of fine-tuning is the product of the first global low-rank adaptive matrix and the second global low-rank adaptive matrix of the previous round of fine-tuning.

[0090] Perform singular value decomposition (SVD) on the intermediate matrix to obtain the first singular vector matrix, the singular value matrix, and the second singular vector matrix. The singular value matrix is ​​a diagonal matrix, and the elements on the diagonal of the singular value matrix are singular values, with the singular values ​​decreasing sequentially.

[0091] Extract the first Rmax rows and Rmax columns of the singular value matrix, the first Rmax columns of the first singular vector matrix, and the first Rmax columns of the second singular vector matrix, where Rmax is a preset rank.

[0092] Matrix reconstruction is performed based on the singular submatrix, the first Rmax column of the first singular vector matrix, and the first Rmax column of the second singular vector matrix to obtain the global weight increment matrix for this round of fine-tuning; wherein, if the fine-tuning stopping condition is not met, the global weight increment matrix for this round of fine-tuning is used for the next round of fine-tuning.

[0093] Until the fine-tuning stopping condition is met, the global weight increment matrix of the Lth round of fine-tuning is obtained, where L is a positive integer and L is the number of fine-tuning rounds until the fine-tuning stopping condition is met. The global weight increment matrix of the 0th round of fine-tuning is the initialized global weight increment matrix.

[0094] The weight matrix of the language model is adjusted based on the global weight increment matrix of the Lth round of fine-tuning.

[0095] Understandably, in each round of fine-tuning, the system receives initial information from N clients, determines the ideal rank of the N clients, determines the assigned rank of the N clients satisfying preset constraints, sends the corresponding assigned rank to each of the N clients, receives the trained local low-rank adaptive matrix from the N clients, determines the first weight increment matrix, and performs matrix reconstruction to obtain the global weight increment matrix for that round of fine-tuning. Obtaining the global weight increment matrix for each round signifies the completion of one round of fine-tuning. If the fine-tuning stopping condition is not met, the next round of fine-tuning begins. The process for each round of fine-tuning is similar and will not be elaborated further. If the fine-tuning stopping condition is met, the global weight increment matrix for the Lth round of fine-tuning (i.e., the latest round of fine-tuning) is obtained. Subsequently, the weight matrix of the language model can be adjusted using the global weight increment matrix of the Lth round of fine-tuning to complete the training of the language model. As an example, adjusting the weight matrix of the language model based on the global weight increment matrix of the Lth round of fine-tuning can include adding the weight matrix of the language model to the global weight increment matrix of the Lth round of fine-tuning to adjust the weight matrix of the language model. As an example, in the process of adjusting the weight matrix of the language model based on the first weight increment matrix, after obtaining the global weight increment matrix for this round of fine-tuning, the process also includes: sending the global weight increment matrix for this round of fine-tuning to N clients.

[0096] Additionally, it should be noted that performing Singular Value Decomposition (SVD) on the intermediate matrix yields the first singular vector matrix (U, also known as the left singular vector matrix), the singular value matrix (Σ), and the second singular vector matrix (V, also known as the right singular vector matrix). The intermediate matrix can be represented as UΣV T V T This represents the transpose of V. Extracting the first Rmax columns of V is equivalent to extracting V. T The first Rmax rows of the singular value matrix. The singular value submatrix of the first Rmax rows and Rmax columns of the singular value matrix is ​​also the submatrix of the top left corner of the singular value matrix with a dimension of Rmax rows and Rmax columns. The singular values ​​on the diagonal of the singular value matrix decrease sequentially. This can be understood as the singular value of the p-th row and p-th column of the singular value matrix being greater than the singular value of the (p+1)-th row and p+1-th column.

[0097] In this embodiment, the first singular vector matrix, the singular value matrix, and the second singular vector matrix can be obtained by performing singular value decomposition on the intermediate matrix. The singular value matrix is ​​a diagonal matrix. Then, the first Rmax rows and Rmax columns of the singular value matrix, the first Rmax columns of the first singular vector matrix, and the first Rmax columns of the second singular vector matrix can be extracted for matrix reconstruction. This yields the global weight increment matrix for this round of fine-tuning, thus solving the problem of rank growth caused by weighted stacking of low-rank adaptive matrices.

[0098] In some embodiments, the first information also includes a delay prediction method related to the rank parameter;

[0099] The efficiency constraints include that the estimated global wait time is greater than or equal to the estimated delay of each client, the estimated delay is the estimated delay determined by the client's delay estimation method under the client's rank parameter, and the estimated global wait time is the maximum value among the estimated delays of N clients.

[0100] The rank budget constraint includes the sum of the rank parameters of N clients being equal to the preset total rank budget;

[0101] The rank range constraint includes that the rank parameter of each client is within a preset rank range. The upper limit of the preset rank range is the first preset rank threshold, and the lower limit is the second preset rank threshold. The second preset rank threshold is less than the first preset rank threshold.

[0102] In this embodiment, the delay estimation method can also be called a delay estimation model or a delay estimation function. The estimated delay of the client can be determined by substituting the client's rank parameter into the delay estimation model or delay estimation function. The estimated global waiting time is the maximum value among the estimated delays of N clients. The determination of the estimated delay is related to the client's rank parameter and is related to the client's rank parameter; therefore, the estimated global waiting time is also related to the client's rank parameter. In the process of rank allocation for clients, at least one of the above-mentioned efficiency constraints, rank budget constraints, and rank range constraints is introduced to determine the allocation rank that satisfies at least one constraint condition, thereby improving the rationality of rank allocation.

[0103] In some embodiments, determining the allocation rank of N clients satisfying preset constraints based on the ideal rank includes:

[0104] With the goal of minimizing the first parameter, the values ​​of the rank parameters of N clients are solved under preset constraints. The allocated rank of the N clients is the value of the rank parameters of the N clients obtained by solving. The first parameter is the sum of the estimated global waiting time and the second parameter. The second parameter is the first preset coefficient multiple of the rank difference sum. The rank difference sum is the sum of the N rank differences. The rank difference represents the difference between the rank parameter of the client and the ideal rank of the client.

[0105] The rank parameter of the client is a variable in the first parameter set and is the parameter to be solved. By minimizing the first parameter, the values ​​of the rank parameters of the N clients under preset constraints can be obtained. In essence, an objective function can be constructed that minimizes the first parameter. Solving the objective function yields the values ​​of the rank parameters of the N clients, i.e., the assigned rank of the N clients.

[0106] The first parameter represents the sum of the estimated global waiting time and the second parameter. The second parameter represents the sum of the differences between the client's rank parameter and the client's ideal rank. With the goal of minimizing the first parameter, the values ​​of the rank parameters of N clients are solved under preset constraints to achieve the rank allocation of N clients, which can improve the rationality of rank allocation for clients.

[0107] See Figure 2 , Figure 2 This is a flowchart of a language model federated fine-tuning method provided in an embodiment of this application, applied to a first client. For example... Figure 2 As shown, the language model federated fine-tuning method provided in this embodiment includes the following steps:

[0108] Step 201: Send the first information to the server. The first information includes the data importance index of the first client. The data importance index of the first client is used to represent the importance of the local training data of the first client to the language model.

[0109] Step 202: Receive the allocation rank assigned to the first client by the server;

[0110] The allocation rank of the first client is the allocation rank that satisfies the preset constraints, which is determined by the ideal rank of N clients. The preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint. The ideal rank of the N clients is the rank determined by the data importance index of the N clients and the preset total rank budget. N is an integer greater than 1. The first client is one of the N clients.

[0111] Step 203: Based on the allocated rank of the first client and the local training data of the first client, train the local low-rank adaptive matrix of the first client;

[0112] The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the assigned rank of the first client. r is less than the minimum of d and m.

[0113] Step 204: Send the trained local low-rank adaptive matrix of the first client to the server. The trained local low-rank adaptive matrix of the first client is used to adjust the weight matrix of the language model.

[0114] In some embodiments, before sending the first information to the server, the method further includes: extracting statistical features of the local training data of the first client, generating an evaluation dataset of the first client based on the statistical features of the local training data of the first client, and sending the evaluation dataset of the first client to the server.

[0115] In some embodiments, before sending the first information to the server and after sending the evaluation dataset of the first client to the server, the method may further include: receiving the global weight increment matrix of the previous round of fine-tuning sent by the server.

[0116] The process of the above scheme will be specifically described below with some specific embodiments.

[0117] In related technologies, the aggregation weight allocation mechanism lacks fairness and struggles to handle data heterogeneity, leading to slower model convergence and impacting model performance. Research shows that heterogeneous data from different clients hinders model convergence and reduces model performance. To address the data heterogeneity problem in federated learning, the aggregation strategies of related technologies primarily allocate aggregation weights based on the size of the client's local training data to accelerate convergence. This weighting method fails to reflect the true marginal contribution of the client's local training data to model performance improvement. Holders of high-quality or scarce data do not receive sufficient weight, while updates containing noisy or low-quality data may dominate model training due to their large volume, failing to effectively address the data heterogeneity problem.

[0118] In related technologies, aggregation bias leads to a decline in model performance: global LoRA updates are constructed by independently averaging LoRA modules (local low-rank adaptive matrices) uploaded by clients. However, the product of the aggregated matrices is not mathematically equal to the expected value of the locally updated product. This mathematical inconsistency introduces severe aggregation noise or bias, hindering model convergence and reducing final performance.

[0119] In related technologies, system heterogeneity leads to training inefficiency: Aggregation requires all clients to have the same LoRA rank. This contradicts the real-world need for heterogeneous client systems (such as varying memory and computing power), causing resource-constrained devices to be unable to participate or to become "fallen behind," thus impacting system training efficiency.

[0120] To address the issues of low training efficiency, insufficient aggregation accuracy, and unfair aggregation weight allocation caused by system heterogeneity, this application proposes a federated LoRA fine-tuning scheme based on stacked aggregation and adaptive rank allocation. It utilizes a stacked aggregation method based on singular value decomposition, avoiding the direct aggregation bias caused by weighted averaging. A high-quality contribution evaluation strategy is designed, employing a server-generated evaluation dataset that preserves the statistical characteristics (statistical features) of client data to calculate scientific and fair aggregation weights, achieving fair and effective aggregation while reducing the risk of data leakage. Furthermore, a dual adaptive rank allocation strategy is designed, solving a composite optimization problem to balance client system latency (efficiency) and data importance (performance).

[0121] like Figure 3As shown, the process of the embodiment of this application is as follows.

[0122] S0. Initialization: The client generates an evaluation dataset based on local private data (local training data) and uploads it to the server. The server initializes the global LoRA adapter, that is, initializes the global weight increment matrix.

[0123] S1. Client-side pre-analysis and index upload: Calculate the sum of gradient norms of all layers, estimate the parameters of the delay model (delay prediction model / local delay model) and upload them;

[0124] S2, Server Dual Adaptive Rank Allocation: Executes a dual adaptive rank allocation strategy, assigning a rank to each client and sending it;

[0125] S3, Client-side Local Training: Receives the rank allocated to the client and completes local LoRA fine-tuning;

[0126] S4. Server Contribution Assessment: Execute a high-quality contribution assessment strategy to obtain the contribution of each client and calculate the aggregate weight of each client.

[0127] S5, Server Aggregation: Performs stacked aggregation based on singular value decomposition according to the aggregation weight.

[0128] The above steps S1 to S5 constitute a complete federated fine-tuning round (i.e., a federated fine-tuning communication round). After S5 is completed, the system determines whether the global model (language model) meets the preset convergence condition or whether the fine-tuning round has reached the preset maximum number of training rounds. If it does not meet or reach the preset maximum number of training rounds, the updated global LoRA adapter generated in S5 is used as the basis to return to S1 and start a new round of iterative training until the model converges or the preset maximum number of training rounds is reached.

[0129] In S0 above, taking client n as an example, client n uses a pre-trained generative model to generate an evaluation dataset. And upload it to the server all at once, the generative model can be based on the local training data D of client n. n generate The server collects data from all clients participating in the federated fine-tuning task (e.g., N clients). Construct a global evaluation dataset This is used for contribution evaluation in all subsequent rounds, and the global evaluation dataset can be a collection of evaluation datasets from all clients participating in the federated fine-tuning task. Furthermore, the server can initialize a global LoRA adapter. = , rank is , Yes The initialized second global low-rank adaptive matrix obtained after low-rank decomposition. Yes The first global low-rank adaptive matrix obtained after low-rank decomposition. It can be represented as .

[0130] In S1 above, when client n fine-tunes in the i-th round (where i is an integer greater than or equal to 1), it first adjusts the global weight increment matrix from the previous round on the local training data. Calculate multi-layer weight gradient To assess the "importance" of local training data, the client calculates the norm of the weight gradients of all or some layers, and sums the norms of the weight gradients to obtain the data importance index for client n. The formula is as follows:

[0131] ;

[0132] l Representation layer index, express The norm of .

[0133] Additionally, client n determines the latency estimation model for local computation (local training) and communication. They were uploaded to the server together.

[0134] In S2 above, a balance is struck between client-side system latency (efficiency) and data importance (performance), ensuring that important data achieves high rank while minimizing training time. The server collects data importance metrics from all clients and sets the budget for the new total rank in this round of fine-tuning. (i.e., preset total rank budget or preset new total rank budget), assuming there are N clients, the server first calculates the ideal rank of the nth client based on the importance index of the data uploaded by S1:

[0135] ;

[0136] This is a rounding function. Let be the data importance index for client k out of N clients.

[0137] Subsequently, to balance the ideal rank (effect) and system latency (efficiency), the server solves for the rank parameter of client n in the i-th round of fine-tuning. The value of is given by the objective function:

[0138] ;

[0139] in The global wait time is, i.e. , For the latency estimation model of client n in The estimated delay is as follows, with the efficiency constraint being: The budget constraint is: The range constraint is: The server will ultimately obtain the solution. The value (i.e., the assigned rank of client n in the i-th round of fine-tuning) is sent to client n. The second preset rank threshold, The first preset rank threshold, Let be the ideal rank of the nth client. This is the first preset coefficient.

[0140] In S3 above, client n performs LoRA fine-tuning; specifically, client n receives... The value is initialized and the rank is set to 0. New local adapter That is, the local low-rank adaptive matrix of client n in the i-th round of fine-tuning, including the first local low-rank adaptive matrix in the i-th round of fine-tuning. Second local low-rank adaptive matrix The client-side (n) freezes the original weight matrix W0 of the language model and the global adapter from the previous round of fine-tuning. , This is the second global low-rank adaptive matrix from the previous round of fine-tuning, i.e., the (i-1)th round of fine-tuning. This is the first global low-rank adaptive matrix from the previous round of fine-tuning, trained only on local training data. After training, it is uploaded to the server.

[0141] In S4 above, the server uses the evaluation dataset for client n. Calculate the Shapley score to evaluate the client's contribution in this round of training, i.e., the i-th round of fine-tuning. :

[0142] ;

[0143] Among them, utility function Used to measure a client consortium S (i.e., client set) The value of this proposal lies in its innovative use of the Federated LoRA method, which defines the utility function as an update of a client federation S and applies it to the global evaluation dataset after the previous round of global model updates. This is the second preset coefficient, which can be preset based on prior knowledge, etc.

[0144] in X: The set of all clients participating in federated tuning (e.g., N clients). : Not including all possible client alliances S of client n. Marginal contribution of client n after joining client alliance S. : Combination coefficients used for weighting, representing the above set X It does not include the number of all possible client alliances (combinations) for the current client n.

[0145] Model of Alliance S Defined as:

[0146] ;

[0147] ;

[0148] Define the valid function as:

[0149] ;

[0150] Represents the accuracy calculation function, effective function express In the global evaluation dataset To achieve the desired accuracy, the server needs to evaluate [the required parameters] to calculate the precise Shapley value. Different alliances S are used, which involves a large computational load when the number of clients N is relatively large. This embodiment of the application uses truncated Monte Carlo (TMC-Shapley) approximation. Further design of temperature parameters (Also known as adjustment coefficient) Contribution to completion Aggregated weights in the i-th round of fine-tuning Conversion:

[0151] ;

[0152] The temperature parameter τ is used to adjust the smoothness of the weight distribution. At higher values, the polymerization process becomes "smooth," and the effect of the Shapley value is weakened. At lower temperatures, the weight distribution becomes "sharper" to more aggressively filter out updates from low-contribution or malicious clients, aggregating only valuable information. Temperature parameter Temperature parameters can be preset based on experience or dynamically set based on data heterogeneity (data importance index).

[0153] In S5 above, a stacked aggregation method is used to eliminate aggregation noise introduced by traditional aggregation methods, while SVD projection is used to address the adverse effects of rank growth. In the stacked aggregation method, the server processes the matrix transmitted from client n... For each n ≤ N, the first local low-rank adaptive matrix and the second local low-rank adaptive matrix sent by N clients are weighted and stacked (weighted concatenation) according to the aggregation weights calculated in S4 above. The stacking method is as follows: Figure 4 As shown, the matrix obtained after stacking for:

[0154] ;

[0155] ;

[0156] Let be the first aggregated low-rank adaptive matrix in the i-th round of fine-tuning. This is the second aggregated low-rank adaptive matrix in the i-th round of fine-tuning; This indicates a stacking symbol, i.e., a splicing symbol; ;

[0157] Then update the matrix = , That is, the first weight increment matrix in the i-th round of fine-tuning;

[0158] Calculate the complete update matrix (i.e., the global weight increment matrix of the i-th round of fine-tuning). ):

[0159] ;

[0160] right Perform SVD decomposition, retaining the previous... Find the largest singular value and its corresponding singular vector, reconstruct the matrix, and obtain the global adapter for this round of fine-tuning, i.e., the i-th round of fine-tuning. , This is the second global low-rank adaptive matrix in this round of fine-tuning. This is the first global low-rank adaptive matrix in this round of fine-tuning. This represents the global weight increment matrix for the i-th round of fine-tuning. The server saves the global adapter and returns to execute S1 to S5 to enter the next round of fine-tuning. This process is repeated until the fine-tuning stop condition is met, at which point the fine-tuning stops, and the final global adapter is obtained. This adapter is then superimposed on the weight matrix of the language model to complete the training of the language model. The trained language model can be applied to, but is not limited to, question-and-answer scenarios to provide corresponding answers to user questions.

[0161] In summary, this application proposes a federated LoRA fine-tuning scheme based on stacked aggregation and adaptive rank allocation. A stacked aggregation method based on singular value decomposition is designed to achieve theoretically noise-free aggregation while addressing the rank growth problem through SVD projection. A high-quality contribution evaluation mechanism is designed, using synthetic data that preserves statistical properties to calculate client contribution values, achieving fair and efficient aggregation while avoiding data leakage risks. A dual adaptive rank allocation strategy is also designed, solving a composite optimization problem to balance client system latency (efficiency) and data importance (performance), achieving the optimal balance between training efficiency and model performance. This effectively solves the problems of low training efficiency, insufficient aggregation accuracy, and unfair client aggregation weights caused by data heterogeneity and system heterogeneity.

[0162] The solutions provided in the embodiments of this application can achieve the following:

[0163] To improve the fairness of aggregation weights and effectively address data heterogeneity issues, this application proposes a high-quality contribution evaluation strategy to assess the actual contribution of each client update to the global model performance. This determines scientific and fair aggregation weights, encouraging high-quality updates. Directly accessing local client data to evaluate client contributions carries the risk of data leakage. Therefore, this application uses an evaluation dataset with the same statistical characteristics as the local dataset (local training data) to evaluate client contributions, avoiding direct access to the local dataset and thus reducing the risk of data leakage.

[0164] To reduce aggregation bias and improve model performance, this application proposes a weighted stacking approach. Stacked aggregation mathematically guarantees a noise-free aggregation process, significantly improving the accuracy of global model updates. However, each stacking step introduces a rank growth problem; infinite rank growth can lead to architectural instability and parameter explosion. Therefore, this proposal innovatively combines SVD projection to decompose and truncate the total update matrix, stably reconstructing the increased rank after aggregation back to the preset maximum global rank (Rmax), thereby solving the problems of architectural instability and parameter explosion.

[0165] To improve training efficiency in heterogeneous systems, this application proposes a dual adaptive rank allocation strategy. This strategy solves a composite optimization problem while simultaneously balancing client-side system latency (efficiency) and data importance (performance). This ensures that important data achieves higher ranks while minimizing training time.

[0166] High-quality contribution evaluation mechanism: This mechanism introduces an evaluation dataset with the same statistical characteristics as the local training data to evaluate the actual contribution of each client update to the global model performance. This dataset is then used to calculate the aggregation weights, improving aggregation fairness, achieving high-quality updates, and avoiding data leakage.

[0167] like Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of a language model federated fine-tuning device 500 provided in an embodiment of this application, as shown below. Figure 5 As shown, the language model federated fine-tuning device 500, applied to a server, includes:

[0168] The first receiving module 501 is used to receive first information sent by N clients. The first information includes the client's data importance index, which is used to represent the importance of the client's local training data to the language model. N is an integer greater than 1.

[0169] The first determining module 502 is used to determine the ideal rank of N clients based on data importance indicators and a preset total rank budget;

[0170] The second determining module 503 is used to determine the allocation rank of N clients that satisfies preset constraints based on the ideal rank of N clients; wherein the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint.

[0171] The first sending module 504 is used to send the corresponding allocation rank to N clients respectively. The allocation rank is used to train the local low-rank adaptive matrix of the client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of input dimensions of the language model, m is the number of output dimensions of the language model, and r is the allocation rank of the client. r is less than the minimum value of d and m.

[0172] The second receiving module 505 is used to receive the trained local low-rank adaptive matrix sent by N clients;

[0173] Adjustment module 506 is used to adjust the weight matrix of the language model based on the local low-rank adaptive matrix trained by N clients.

[0174] In some embodiments, the weight matrix of the language model is adjusted based on the trained local low-rank adaptive matrix of N clients, including:

[0175] Based on the local low-rank adaptive matrix trained by N clients, determine the contribution of N clients in this round of fine-tuning;

[0176] The aggregate weight of the N clients is determined based on their contribution in this round of fine-tuning.

[0177] Based on the aggregate weights of N clients, the first local low-rank adaptive matrix trained by the N clients is weighted and stacked to obtain the first aggregated low-rank adaptive matrix, and the second local low-rank adaptive matrix trained by the N clients is weighted and stacked to obtain the aggregated second aggregated low-rank adaptive matrix.

[0178] The first weight increment matrix is ​​determined based on the first aggregated low-rank adaptive matrix and the second aggregated low-rank adaptive matrix.

[0179] Adjust the weight matrix of the language model based on the first weight increment matrix.

[0180] In some embodiments, adjusting the weight matrix of the language model according to the first weight increment matrix includes:

[0181] Add the first weight increment matrix to the global weight increment matrix of the previous round of fine-tuning in the server to obtain the intermediate matrix. The global weight increment matrix of the previous round of fine-tuning is the product of the first global low-rank adaptive matrix and the second global low-rank adaptive matrix of the previous round of fine-tuning.

[0182] Perform singular value decomposition (SVD) on the intermediate matrix to obtain the first singular vector matrix, the singular value matrix, and the second singular vector matrix. The singular value matrix is ​​a diagonal matrix, and the elements on the diagonal of the singular value matrix are singular values, with the singular values ​​decreasing sequentially.

[0183] Extract the first Rmax rows and Rmax columns of the singular value matrix, the first Rmax columns of the first singular vector matrix, and the first Rmax columns of the second singular vector matrix, where Rmax is a preset rank.

[0184] Matrix reconstruction is performed based on the singular submatrix, the first Rmax column of the first singular vector matrix, and the first Rmax column of the second singular vector matrix to obtain the global weight increment matrix for this round of fine-tuning; wherein, if the fine-tuning stopping condition is not met, the global weight increment matrix for this round of fine-tuning is used for the next round of fine-tuning.

[0185] Until the fine-tuning stopping condition is met, the global weight increment matrix of the Lth round of fine-tuning is obtained, where L is a positive integer and L is the number of fine-tuning rounds until the fine-tuning stopping condition is met. The global weight increment matrix of the 0th round of fine-tuning is the initialized global weight increment matrix.

[0186] The weight matrix of the language model is adjusted based on the global weight increment matrix of the Lth round of fine-tuning.

[0187] In some embodiments, the first information also includes a delay prediction method related to the rank parameter;

[0188] The efficiency constraints include that the estimated global wait time is greater than or equal to the estimated delay of each client, the estimated delay is the estimated delay determined by the client's delay estimation method under the client's rank parameter, and the estimated global wait time is the maximum value among the estimated delays of N clients.

[0189] The rank budget constraint includes the sum of the rank parameters of N clients being equal to the preset total rank budget;

[0190] The rank range constraint includes that the rank parameter of each client is within a preset rank range. The upper limit of the preset rank range is the first preset rank threshold, and the lower limit is the second preset rank threshold. The second preset rank threshold is less than the first preset rank threshold.

[0191] In some embodiments, determining the allocation rank of N clients satisfying preset constraints based on the ideal rank includes:

[0192] With the goal of minimizing the first parameter, the values ​​of the rank parameters of N clients are solved under preset constraints. The allocated rank of the N clients is the value of the rank parameters of the N clients obtained by solving. The first parameter is the sum of the estimated global waiting time and the second parameter. The second parameter is the first preset coefficient multiple of the rank difference sum. The rank difference sum is the sum of the N rank differences. The rank difference represents the difference between the rank parameter of the client and the ideal rank of the client.

[0193] As an example, the language model federated fine-tuning device 500 may also include: a data receiving module for receiving evaluation datasets sent by N clients respectively.

[0194] As an example, the language model federated fine-tuning device 500 may also include: a fourth sending module for sending the global weight increment matrix of the previous round of fine-tuning to N clients.

[0195] The language model federated fine-tuning device 500 provided in this embodiment can realize each process of the above-described embodiments of the language model federated fine-tuning method applied to the server. The technical features are one-to-one and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0196] See Figure 6 , Figure 6 This is a schematic diagram of the structure of a language model federated fine-tuning device 600 provided in an embodiment of this application, as shown below. Figure 6 As shown, the language model federated fine-tuning device 600, applied to the first client, includes:

[0197] The second sending module 601 is used to send first information to the server. The first information includes a data importance index of the first client, which is used to represent the importance of the local training data of the first client to the language model.

[0198] The third receiving module 602 is used to receive the allocation rank assigned to the first client sent by the server; the allocation rank of the first client is the allocation rank that satisfies the preset constraints determined by the ideal rank of N clients; wherein, the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint and rank range constraint, the ideal rank of N clients is the rank determined by the data importance index of N clients and the preset total rank budget, N is an integer greater than 1, and the first client is one of the N clients;

[0199] Training module 603 is used to train the local low-rank adaptive matrix of the first client based on the assigned rank of the first client and the local training data of the first client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the assigned rank of the first client. r is less than the minimum value of d and m.

[0200] The third sending module 604 is used to send the trained local low-rank adaptive matrix of the first client to the server. The trained local low-rank adaptive matrix of the first client is used to adjust the weight matrix of the language model.

[0201] As an example, the language model federated fine-tuning device 600 may also include:

[0202] The feature extraction module is used to extract statistical features from the local training data of the first client.

[0203] The data generation module is used to generate the evaluation dataset for the first client based on the statistical features of the local training data of the first client.

[0204] The data sending module is used to send the evaluation dataset of the first client to the server.

[0205] As an example, the language model federated fine-tuning device 600 may also include: a fourth receiving module for receiving the global weight increment matrix of the previous round of fine-tuning sent by the server.

[0206] The language model federated fine-tuning device 600 provided in this embodiment can realize the various processes of the above-described language model federated fine-tuning method applied to the first client. The technical features are one-to-one and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0207] This application also provides an electronic device, which can be a server. The electronic device includes a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the various processes of the above-described language model federated fine-tuning method embodiment applied to the server and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0208] For details, see Figure 7 This application also provides an electronic device, which is a server, including a bus 701, a transceiver 702, an antenna 703, a bus interface 704, a processor 705, and a memory 706.

[0209] The processor 705 is used for:

[0210] Receive first information sent by N clients. The first information includes the client's data importance index, which is used to represent the importance of the client's local training data to the language model. N is an integer greater than 1.

[0211] Based on data importance indicators and a preset total rank budget, determine the ideal rank for N clients;

[0212] Based on the ideal rank of N clients, determine the allocation rank of N clients that satisfy the preset constraints; wherein the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint.

[0213] Send the corresponding assigned rank to each of the N clients. The assigned rank is used to train the local low-rank adaptive matrix of the client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the assigned rank of the client. r is less than the minimum of d and m.

[0214] Receive trained local low-rank adaptive matrices sent by N clients;

[0215] The weight matrix of the language model is adjusted based on the local low-rank adaptive matrix trained on N clients.

[0216] In some embodiments, the weight matrix of the language model is adjusted based on the trained local low-rank adaptive matrix of N clients, including:

[0217] Based on the local low-rank adaptive matrix trained by N clients, determine the contribution of N clients in this round of fine-tuning;

[0218] The aggregate weight of the N clients is determined based on their contribution in this round of fine-tuning.

[0219] Based on the aggregate weights of N clients, the first local low-rank adaptive matrix trained by the N clients is weighted and stacked to obtain the first aggregated low-rank adaptive matrix, and the second local low-rank adaptive matrix trained by the N clients is weighted and stacked to obtain the aggregated second aggregated low-rank adaptive matrix.

[0220] The first weight increment matrix is ​​determined based on the first aggregated low-rank adaptive matrix and the second aggregated low-rank adaptive matrix.

[0221] Adjust the weight matrix of the language model based on the first weight increment matrix.

[0222] In some embodiments, adjusting the weight matrix of the language model according to the first weight increment matrix includes:

[0223] Add the first weight increment matrix to the global weight increment matrix of the previous round of fine-tuning in the server to obtain the intermediate matrix. The global weight increment matrix of the previous round of fine-tuning is the product of the first global low-rank adaptive matrix and the second global low-rank adaptive matrix of the previous round of fine-tuning.

[0224] Perform singular value decomposition (SVD) on the intermediate matrix to obtain the first singular vector matrix, the singular value matrix, and the second singular vector matrix. The singular value matrix is ​​a diagonal matrix, and the elements on the diagonal of the singular value matrix are singular values, with the singular values ​​decreasing sequentially.

[0225] Extract the first Rmax rows and Rmax columns of the singular value matrix, the first Rmax columns of the first singular vector matrix, and the first Rmax columns of the second singular vector matrix, where Rmax is a preset rank.

[0226] Matrix reconstruction is performed based on the singular submatrix, the first Rmax column of the first singular vector matrix, and the first Rmax column of the second singular vector matrix to obtain the global weight increment matrix for this round of fine-tuning; wherein, if the fine-tuning stopping condition is not met, the global weight increment matrix for this round of fine-tuning is used for the next round of fine-tuning.

[0227] Until the fine-tuning stopping condition is met, the global weight increment matrix of the Lth round of fine-tuning is obtained, where L is a positive integer and L is the number of fine-tuning rounds until the fine-tuning stopping condition is met. The global weight increment matrix of the 0th round of fine-tuning is the initialized global weight increment matrix.

[0228] The weight matrix of the language model is adjusted based on the global weight increment matrix of the Lth round of fine-tuning.

[0229] In some embodiments, the first information also includes a delay prediction method related to the rank parameter;

[0230] The efficiency constraints include that the estimated global wait time is greater than or equal to the estimated delay of each client, the estimated delay is the estimated delay determined by the client's delay estimation method under the client's rank parameter, and the estimated global wait time is the maximum value among the estimated delays of N clients.

[0231] The rank budget constraint includes the sum of the rank parameters of N clients being equal to the preset total rank budget;

[0232] The rank range constraint includes that the rank parameter of each client is within a preset rank range. The upper limit of the preset rank range is the first preset rank threshold, and the lower limit is the second preset rank threshold. The second preset rank threshold is less than the first preset rank threshold.

[0233] In some embodiments, determining the allocation rank of N clients satisfying preset constraints based on the ideal rank includes:

[0234] With the goal of minimizing the first parameter, the values ​​of the rank parameters of N clients are solved under preset constraints. The allocated rank of the N clients is the value of the rank parameters of the N clients obtained by solving. The first parameter is the sum of the estimated global waiting time and the second parameter. The second parameter is the first preset coefficient multiple of the rank difference sum. The rank difference sum is the sum of the N rank differences. The rank difference represents the difference between the rank parameter of the client and the ideal rank of the client.

[0235] As an example, the processor 705 is also used to receive evaluation datasets sent by N clients.

[0236] As an example, the processor 705 is also used to send the global weight increment matrix of the previous round of fine-tuning to N clients.

[0237] exist Figure 7 In this document, a bus architecture (represented by bus 701) is used. Bus 701 can include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 705 and memory represented by memory 706. Bus 701 can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 704 provides an interface between bus 701 and transceiver 702. Transceiver 702 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 705 is transmitted over a wireless medium via antenna 703, which further receives data and transmits data to processor 705.

[0238] Processor 705 manages bus 701 and general processing, and also provides various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Memory 706 can be used to store data used by processor 705 during operation.

[0239] Optionally, the processor 705 can be a CPU, ASIC, FPGA, or CPLD.

[0240] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes described in the above-described embodiment of the language model federated fine-tuning method applied to a server, and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0241] This application also provides an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the various processes of the above-described language model federated fine-tuning method embodiment applied to the first client and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0242] For details, see Figure 8 As shown in the figure, this application embodiment also provides an electronic device, which is a first client, including a bus 801, a transceiver 802, an antenna 803, a bus interface 804, a processor 805, and a memory 806.

[0243] The processor 805 is used for:

[0244] Send the first information to the server. The first information includes the data importance index of the first client. The data importance index of the first client is used to represent the importance of the local training data of the first client to the language model.

[0245] The server receives the allocation rank assigned to the first client; the allocation rank of the first client is the allocation rank that satisfies the preset constraints, determined by the ideal rank of N clients; wherein the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint and rank range constraint, the ideal rank of N clients is the rank determined by the data importance index of N clients and the preset total rank budget, N is an integer greater than 1, and the first client is one of the N clients;

[0246] Based on the assigned rank of the first client and the local training data of the first client, train the local low-rank adaptive matrix of the first client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the assigned rank of the first client. r is less than the minimum value of d and m.

[0247] The first client sends its trained local low-rank adaptive matrix to the server. This local low-rank adaptive matrix is ​​used to adjust the weight matrix of the language model.

[0248] As an example, the 805 processor is also used for:

[0249] Extract statistical features from the local training data of the first client;

[0250] The evaluation dataset for the first client is generated based on the statistical features of the local training data of the first client.

[0251] Send the evaluation dataset from the first client to the server.

[0252] As an example, the 805 processor is also used for:

[0253] Receive the global weight increment matrix from the previous round of fine-tuning sent by the server.

[0254] exist Figure 8 In this document, a bus architecture (represented by bus 801) is used. Bus 801 can include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 805 and memory represented by memory 806. Bus 801 can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 804 provides an interface between bus 801 and transceiver 802. Transceiver 802 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 805 is transmitted over a wireless medium via antenna 803, which further receives data and transmits data to processor 805.

[0255] The processor 805 manages the bus 801 and handles general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 806 can be used to store data used by the processor 805 during operation.

[0256] Optionally, the processor 805 can be a CPU, ASIC, FPGA, or CPLD.

[0257] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes described in the above-described embodiment of the language model federated fine-tuning method applied to the first client, and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be, for example, ROM, RAM, magnetic disk, or optical disk.

[0258] This application provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the various processes of the method described in the embodiment. The technical features are one-to-one and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0259] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0260] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0261] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for federated fine-tuning of a language model, characterized in that, Applied to a server, the method includes: Receive first information sent by N clients, the first information including the data importance index of the client, the data importance index of the client is used to represent the importance of the client's local training data to the language model, and N is an integer greater than 1; Based on the data importance index and the preset total rank budget, the ideal rank of the N clients is determined; Based on the ideal rank of the N clients, determine the allocation rank of the N clients that satisfies the preset constraints; wherein the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint; The corresponding allocation rank is sent to each of the N clients. The allocation rank is used to train the local low-rank adaptive matrix of the client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the allocation rank of the client. r is less than the minimum value of d and m. Receive the trained local low-rank adaptive matrix sent by the N clients; Based on the local low-rank adaptive matrices trained from the N clients, the weight matrix of the language model is adjusted. The first information also includes a delay prediction method related to the rank parameter; The efficiency constraint includes an estimated global wait time greater than or equal to the estimated latency of each client, wherein the estimated latency is the estimated latency determined by the client's latency estimation method under the client's rank parameter, and the estimated global wait time is the maximum value among the estimated latencies of the N clients. The rank budget constraint includes the fact that the sum of the rank parameters of the N clients is equal to the preset total rank budget; The rank range constraint includes the fact that the rank parameters of each client are all within a preset rank range, the upper limit of the preset rank range is a first preset rank threshold, the lower limit is a second preset rank threshold, and the second preset rank threshold is less than the first preset rank threshold; The step of determining the allocation rank of the N clients satisfying the preset constraints based on the ideal rank includes: With the goal of minimizing the first parameter, the values ​​of the rank parameters of the N clients are solved under the preset constraints. The allocated rank of the N clients is the value of the rank parameters of the N clients obtained by solving. The first parameter is the sum of the estimated global waiting time and the second parameter. The second parameter is a first preset coefficient multiple of the sum of rank differences. The sum of rank differences is the sum of N rank differences. The rank difference represents the difference between the rank parameter of the client and the ideal rank of the client.

2. The method according to claim 1, characterized in that, The adjustment of the language model's weight matrix based on the trained local low-rank adaptive matrix from the N clients includes: Based on the trained local low-rank adaptive matrix of the N clients, determine the contribution of the N clients in this round of fine-tuning; The aggregate weight of the N clients is determined based on their contribution in this round of fine-tuning; Based on the aggregate weights of the N clients, the first local low-rank adaptive matrices trained by the N clients are weighted and stacked to obtain the first aggregated low-rank adaptive matrix, and the second local low-rank adaptive matrices trained by the N clients are weighted and stacked to obtain the aggregated second aggregated low-rank adaptive matrix. Based on the first aggregated low-rank adaptive matrix and the second aggregated low-rank adaptive matrix, the first weight increment matrix is ​​determined; The weight matrix of the language model is adjusted based on the first weight increment matrix.

3. The method according to claim 2, characterized in that, The step of adjusting the weight matrix of the language model according to the first weight increment matrix includes: The first weight increment matrix is ​​added to the global weight increment matrix of the previous round of fine-tuning in the server to obtain the intermediate matrix. The global weight increment matrix of the previous round of fine-tuning is the product of the first global low-rank adaptive matrix and the second global low-rank adaptive matrix of the previous round of fine-tuning. The intermediate matrix is ​​decomposed by Singular Value Decomposition (SVD) to obtain a first singular vector matrix, a singular value matrix, and a second singular vector matrix. The singular value matrix is ​​a diagonal matrix, and the elements on the diagonal of the singular value matrix are singular values, which decrease sequentially. Extract the first Rmax rows and Rmax columns of the singular value matrix, the first Rmax columns of the first singular vector matrix, and the first Rmax columns of the second singular vector matrix, where Rmax is a preset rank size; Matrix reconstruction is performed based on the singular value submatrix, the first Rmax column of the first singular vector matrix, and the first Rmax column of the second singular vector matrix to obtain the global weight increment matrix for this round of fine-tuning; wherein, if the fine-tuning stopping condition is not met, the global weight increment matrix for this round of fine-tuning is used for the next round of fine-tuning. Until the fine-tuning stopping condition is met, the global weight increment matrix of the Lth round of fine-tuning is obtained, where L is a positive integer and L is the number of fine-tuning rounds until the fine-tuning stopping condition is met. The global weight increment matrix of the 0th round of fine-tuning is the initialized global weight increment matrix. Based on the global weight increment matrix of the Lth round of fine-tuning, the weight matrix of the language model is adjusted.

4. A method for federated fine-tuning of a language model, characterized in that, Applied to a first client, the method includes: Send first information to the server, the first information including the data importance index of the first client, the data importance index of the first client is used to represent the importance of the local training data of the first client to the language model; The server sends an allocation rank assigned to the first client; the allocation rank of the first client is an allocation rank that satisfies preset constraints, determined by the ideal ranks of N clients; wherein, the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint, the ideal rank of the N clients is the rank determined by the data importance index of the N clients and the preset total rank budget, N is an integer greater than 1, and the first client is one of the N clients; Based on the assigned rank of the first client and the local training data of the first client, a local low-rank adaptive matrix of the first client is trained. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the assigned rank of the first client. r is less than the minimum value of d and m. The trained local low-rank adaptive matrix of the first client is sent to the server. The trained local low-rank adaptive matrix of the first client is used to adjust the weight matrix of the language model. The first information also includes a delay prediction method related to the rank parameter; The efficiency constraint includes an estimated global wait time greater than or equal to the estimated latency of each client, wherein the estimated latency is the estimated latency determined by the client's latency estimation method under the client's rank parameter, and the estimated global wait time is the maximum value among the estimated latencies of the N clients. The rank budget constraint includes the fact that the sum of the rank parameters of the N clients is equal to the preset total rank budget; The rank range constraint includes the fact that the rank parameters of each client are all within a preset rank range, the upper limit of the preset rank range is a first preset rank threshold, the lower limit is a second preset rank threshold, and the second preset rank threshold is less than the first preset rank threshold; The allocation rank of the N clients is determined in the following way: With the goal of minimizing the first parameter, the values ​​of the rank parameters of the N clients are solved under the preset constraints. The allocated rank of the N clients is the value of the rank parameters of the N clients obtained by solving. The first parameter is the sum of the estimated global waiting time and the second parameter. The second parameter is a first preset coefficient multiple of the sum of rank differences. The sum of rank differences is the sum of N rank differences. The rank difference represents the difference between the rank parameter of the client and the ideal rank of the client.

5. A language model federated fine-tuning device, characterized in that, Applied to a server, the device includes: The first receiving module is used to receive first information sent by N clients. The first information includes the data importance index of the client. The data importance index of the client is used to represent the importance of the client's local training data to the language model. N is an integer greater than 1. The first determining module is used to determine the ideal rank of the N clients based on the data importance index and the preset total rank budget; The second determining module is used to determine the allocation rank of the N clients that satisfies preset constraints based on the ideal rank of the N clients; wherein the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint. The first sending module is used to send the corresponding allocation rank to each of the N clients. The allocation rank is used to train the local low-rank adaptive matrix of the client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of input dimensions of the language model, m is the number of output dimensions of the language model, and r is the allocation rank of the client. r is less than the minimum value of d and m. The second receiving module is used to receive the trained local low-rank adaptive matrix sent by the N clients; The adjustment module is used to adjust the weight matrix of the language model based on the trained local low-rank adaptive matrix of the N clients. The first information also includes a delay prediction method related to the rank parameter; The efficiency constraint includes an estimated global wait time greater than or equal to the estimated latency of each client, wherein the estimated latency is the estimated latency determined by the client's latency estimation method under the client's rank parameter, and the estimated global wait time is the maximum value among the estimated latencies of the N clients. The rank budget constraint includes the fact that the sum of the rank parameters of the N clients is equal to the preset total rank budget; The rank range constraint includes the fact that the rank parameters of each client are all within a preset rank range, the upper limit of the preset rank range is a first preset rank threshold, the lower limit is a second preset rank threshold, and the second preset rank threshold is less than the first preset rank threshold; The step of determining the allocation rank of the N clients satisfying the preset constraints based on the ideal rank includes: With the goal of minimizing the first parameter, the values ​​of the rank parameters of the N clients are solved under the preset constraints. The allocated rank of the N clients is the value of the rank parameters of the N clients obtained by solving. The first parameter is the sum of the estimated global waiting time and the second parameter. The second parameter is a first preset coefficient multiple of the sum of rank differences. The sum of rank differences is the sum of N rank differences. The rank difference represents the difference between the rank parameter of the client and the ideal rank of the client.

6. A language model federated fine-tuning device, characterized in that, Applied to a first client, the device includes: The second sending module is used to send first information to the server. The first information includes the data importance index of the first client. The data importance index of the first client is used to represent the importance of the local training data of the first client to the language model. The third receiving module is used to receive the allocation rank assigned to the first client sent by the server; the allocation rank of the first client is an allocation rank that satisfies preset constraints and is determined by the ideal ranks of N clients; wherein, the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint and rank range constraint, the ideal rank of the N clients is the rank determined by the data importance index of the N clients and the preset total rank budget, N is an integer greater than 1, and the first client is one of the N clients; The training module is used to train the local low-rank adaptive matrix of the first client based on the assigned rank of the first client and the local training data of the first client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns, where d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the assigned rank of the first client, where r is less than the minimum value of d and m. The third sending module is used to send the trained local low-rank adaptive matrix of the first client to the server. The trained local low-rank adaptive matrix of the first client is used to adjust the weight matrix of the language model. The first information also includes a delay prediction method related to the rank parameter; The efficiency constraint includes an estimated global wait time greater than or equal to the estimated latency of each client, wherein the estimated latency is the estimated latency determined by the client's latency estimation method under the client's rank parameter, and the estimated global wait time is the maximum value among the estimated latencies of the N clients. The rank budget constraint includes the fact that the sum of the rank parameters of the N clients is equal to the preset total rank budget; The rank range constraint includes the fact that the rank parameters of each client are all within a preset rank range, the upper limit of the preset rank range is a first preset rank threshold, the lower limit is a second preset rank threshold, and the second preset rank threshold is less than the first preset rank threshold; The allocation rank of the N clients is determined in the following way: With the goal of minimizing the first parameter, the values ​​of the rank parameters of the N clients are solved under the preset constraints. The allocated rank of the N clients is the value of the rank parameters of the N clients obtained by solving. The first parameter is the sum of the estimated global waiting time and the second parameter. The second parameter is a first preset coefficient multiple of the sum of rank differences. The sum of rank differences is the sum of N rank differences. The rank difference represents the difference between the rank parameter of the client and the ideal rank of the client.

7. An electronic device, a server, characterized in that, The electronic device includes a transceiver and a processor. The processor is used for: Receive first information sent by N clients, the first information including the data importance index of the client, the data importance index of the client is used to represent the importance of the client's local training data to the language model, and N is an integer greater than 1; Based on the data importance index and the preset total rank budget, the ideal rank of the N clients is determined; Based on the ideal rank of the N clients, determine the allocation rank of the N clients that satisfies the preset constraints; wherein the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint; The corresponding allocation rank is sent to each of the N clients. The allocation rank is used to train the local low-rank adaptive matrix of the client. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the allocation rank of the client. r is less than the minimum value of d and m. Receive the trained local low-rank adaptive matrix sent by the N clients; Based on the local low-rank adaptive matrices trained from the N clients, the weight matrix of the language model is adjusted. The first information also includes a delay prediction method related to the rank parameter; The efficiency constraint includes an estimated global wait time greater than or equal to the estimated latency of each client, wherein the estimated latency is the estimated latency determined by the client's latency estimation method under the client's rank parameter, and the estimated global wait time is the maximum value among the estimated latencies of the N clients. The rank budget constraint includes the fact that the sum of the rank parameters of the N clients is equal to the preset total rank budget; The rank range constraint includes the fact that the rank parameters of each client are all within a preset rank range, the upper limit of the preset rank range is a first preset rank threshold, the lower limit is a second preset rank threshold, and the second preset rank threshold is less than the first preset rank threshold; The step of determining the allocation rank of the N clients satisfying the preset constraints based on the ideal rank includes: With the goal of minimizing the first parameter, the values ​​of the rank parameters of the N clients are solved under the preset constraints. The allocated rank of the N clients is the value of the rank parameters of the N clients obtained by solving. The first parameter is the sum of the estimated global waiting time and the second parameter. The second parameter is a first preset coefficient multiple of the sum of rank differences. The sum of rank differences is the sum of N rank differences. The rank difference represents the difference between the rank parameter of the client and the ideal rank of the client.

8. An electronic device, serving as a first client, characterized in that: The electronic device includes a transceiver and a processor. The processor is used for: Send first information to the server, the first information including the data importance index of the first client, the data importance index of the first client is used to represent the importance of the local training data of the first client to the language model; Receive the allocation rank assigned to the first client by the server; The allocation rank of the first client is an allocation rank that satisfies preset constraints, determined by the ideal rank of N clients; wherein, the preset constraints include at least one of the following constraints: efficiency constraint, rank budget constraint, and rank range constraint, the ideal rank of the N clients is the rank determined by the data importance index of the N clients and the preset total rank budget, N is an integer greater than 1, and the first client is one of the N clients; Based on the assigned rank of the first client and the local training data of the first client, a local low-rank adaptive matrix of the first client is trained. The local low-rank adaptive matrix includes a first local low-rank adaptive matrix with d rows and r columns and a second local low-rank adaptive matrix with r rows and m columns. d is the number of rows of the weight matrix of the language model, m is the number of columns of the weight matrix of the language model, and r is the assigned rank of the first client. r is less than the minimum value of d and m. The trained local low-rank adaptive matrix of the first client is sent to the server. The trained local low-rank adaptive matrix of the first client is used to adjust the weight matrix of the language model. The first information also includes a delay prediction method related to the rank parameter; The efficiency constraint includes an estimated global wait time greater than or equal to the estimated latency of each client, wherein the estimated latency is the estimated latency determined by the client's latency estimation method under the client's rank parameter, and the estimated global wait time is the maximum value among the estimated latencies of the N clients. The rank budget constraint includes the fact that the sum of the rank parameters of the N clients is equal to the preset total rank budget; The rank range constraint includes the fact that the rank parameters of each client are all within a preset rank range, the upper limit of the preset rank range is a first preset rank threshold, the lower limit is a second preset rank threshold, and the second preset rank threshold is less than the first preset rank threshold; The allocation rank of the N clients is determined in the following way: With the goal of minimizing the first parameter, the values ​​of the rank parameters of the N clients are solved under the preset constraints. The allocated rank of the N clients is the value of the rank parameters of the N clients obtained by solving. The first parameter is the sum of the estimated global waiting time and the second parameter. The second parameter is a first preset coefficient multiple of the sum of rank differences. The sum of rank differences is the sum of N rank differences. The rank difference represents the difference between the rank parameter of the client and the ideal rank of the client.

Images