Big federal model fine tuning method and device based on mutual learning and medium

Abstract

The invention discloses a federal large model fine tuning method and device based on mutual learning and a medium, and relates to the technical field of large language models. The method comprises the following steps: finely adjusting a first language model through a client and a local private data set to obtain a second language model; calculating the public data set through a second language model to obtain a second result; calculating a first loss value of the second result and the first result, and forming a first knowledge set by the second result and the first loss value; obtaining a knowledge set, and calculating the public data set through a third language model to obtain a third result; a second loss value of the third result is calculated, and the first loss value and the second loss value are compared to fine tune the third language model. According to the method, the knowledge of the small language model of the client side is fully utilized to enrich the large language model of the server side, and meanwhile the knowledge of the large model of the server side is used for improving the capacity of the small language model of the client side.

Description

Technical Field

[0001] The present application relates to the technical field of large language models, and in particular to a method, device and medium for fine-tuning a federated large model based on mutual learning. Background Art

[0002] In recent years, large language models have received widespread attention and have been applied in various fields, but they still face challenges in real-world scenarios. These challenges stem from the scarcity of public domain data and the need to maintain privacy in private domain data. In order to solve these problems, federated large language models came into being.

[0003] Research on federated large language models mainly focuses on enabling clients to collaboratively fine-tune their respective locally deployed smaller large language models, or transferring the knowledge of the server-side large large language model to the client's small large language model. However, these methods ignore the problem that the insights of the client-side small large language model into its own unique domain knowledge can also enrich the server-side large language model.

[0004] Through the above analysis, the problems and defects of the prior art are as follows: The existing large federated language models mainly focus on the knowledge transfer between the client and the server, but ignore the role of the unique domain knowledge of the small client model in enriching the large server model. Summary of the invention

[0005] The embodiments of the present application provide a method, device and medium for fine-tuning a federated large model based on mutual learning, which can solve the problem in the prior art that the federated large language model mainly focuses on the knowledge transfer between the client and the server, but ignores the role of the unique domain knowledge of the client's small model in enriching the large model on the server.

[0006] In a first aspect, an embodiment of the present application provides a method for fine-tuning a federated large model based on mutual learning, the method comprising: fine-tuning a first language model through a client and a local private data set to obtain a second language model; calculating a public data set through the second language model to obtain a second result, wherein the public data set already has a first result, and the second result includes a feature vector, an attention distribution, and gradient information; calculating a first loss value of the second result and the first result, and forming the second result and the first loss value into a first knowledge set; obtaining a knowledge set, and calculating the public data set through a third language model to obtain a third result; calculating a second loss value of the third result, and comparing the first loss value and the second loss value to fine-tune the third language model.

[0007] In one implementation of the present application, a second loss value of the third result is calculated, and the first loss value and the second loss value are compared to fine-tune the third language model, specifically including: obtaining the minimum first loss value, and comparing the minimum first loss value and the second loss value according to a vocabulary mapping table with the client; if it is less than the second loss value, obtaining the first knowledge set of the client and fine-tuning the third language model.

[0008] In one implementation of the present application, after calculating the second loss value of the third result and comparing the first loss value and the second loss value to fine-tune the third language model, the method also includes: combining the third result and the second loss value into a second knowledge set; and sending the third result to the client according to a vocabulary mapping table with the client.

[0009] In one implementation of the present application, before fine-tuning the first language model through the client and the local private data set to obtain the second language model, the method also includes: collecting the local private data set and the public data set, extracting vocabulary, and counting the frequency of occurrence and context information of the vocabulary; based on the similarity and context information of the vocabulary, determining the mapping relationship between the vocabulary to obtain a vocabulary mapping table.

[0010] In one implementation of the present application, the method also includes: deduplicating repeated words, merging words whose similarity is greater than a preset threshold, and regularly updating the vocabulary mapping table; establishing a user feedback mechanism and a sharing mechanism to allow the first knowledge set to be shared between different clients.

[0011] In one implementation of the present application, a first language model is fine-tuned through a client and a local private dataset to obtain a second language model, specifically including: introducing two low-rank matrices in the attention mechanism layer and the feedforward neural network layer of the first language model; initializing the low-rank matrix, and training the low-rank matrix using a local private dataset to obtain a fine-tuned model.

[0012] In one implementation of the present application, the method further includes: evaluating the accuracy and efficiency of the fine-tuned model through synonym replacement; when the accuracy and efficiency reach a preset threshold, saving the current low-rank matrix to obtain a second language model.

[0013] In one implementation of the present application, a minimum first loss value is obtained, and the minimum first loss value and the second loss value are compared based on a vocabulary mapping table with the client, specifically including: based on the first result, including calculating the first loss value of the feature vector, attention distribution and gradient information, and comparing the first loss values ​​of the feature vector, attention distribution and gradient information respectively; respectively selecting the output form with the minimum first loss value of the feature vector, attention distribution and gradient information, and combining them to obtain the minimum first loss value.

[0014] In a second aspect, an embodiment of the present application further provides a federated large model fine-tuning device based on mutual learning, the device comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: fine-tune the first language model through the client and the local private data set to obtain the second language model; calculate the public data set through the second language model to obtain the second result, wherein the public data set already has the first result, and the second result includes the feature vector, the attention distribution and the gradient information; calculate the first loss value of the second result and the first result, and form the second result and the first loss value into a first knowledge set; obtain the knowledge set, and calculate the public data set through the third language model to obtain the third result; calculate the second loss value of the third result, and compare the first loss value and the second loss value to fine-tune the third language model.

[0015] In a third aspect, an embodiment of the present application further provides a non-volatile computer storage medium for fine-tuning a federated large model based on mutual learning, storing computer executable instructions, wherein the computer executable instructions are configured to: fine-tune a first language model through a client and a local private data set to obtain a second language model; calculate a public data set through the second language model to obtain a second result, wherein the public data set already has a first result, and the second result includes a feature vector, an attention distribution, and gradient information; calculate a first loss value of the second result and the first result, and form a first knowledge set with the second result and the first loss value; obtain a knowledge set, and calculate the public data set through a third language model to obtain a third result; calculate a second loss value of the third result, and compare the first loss value and the second loss value to fine-tune the third language model.

[0016] The embodiments of the present application provide a method, device and medium for fine-tuning a federated large model based on mutual learning, which fully utilizes the knowledge of a small language model on the client side to enrich a large language model on the server side, and at the same time uses the knowledge of the large model on the server side to enhance the capabilities of the small language model on the client side; the original output of the fine-tuned model on a public data set is used as a medium for transferring knowledge, and by establishing a vocabulary mapping table for the server-side and client-side language models, the original outputs of the two models are aligned to ensure efficient knowledge transfer between the two models. During the knowledge transfer, the original outputs of the two models on the public data set are compared to select the original output with better performance for transfer. This method can adaptively transfer the knowledge of the large language model on the server side to the small language model on the client side without leaking the client's privacy data set, and at the same time enrich the unique insights of the small language model on the client side into the large language model on the server side, thereby achieving a common improvement in the performance of the server-side and client-side language models. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a method for fine-tuning a federated large model based on mutual learning provided in an embodiment of the present application; Figure 2 A schematic diagram of test set results of a federated large model fine-tuning method based on mutual learning provided in an embodiment of the present application; Figure 3 A schematic diagram of a first loss value of a federated large model fine-tuning method based on mutual learning provided in an embodiment of the present application; Figure 4 A schematic diagram of component relationships of a method for fine-tuning a federated large model based on mutual learning provided in an embodiment of the present application; Figure 5 A schematic diagram for comparing a first loss value and a second loss value of a federated large model fine-tuning method based on mutual learning provided in an embodiment of the present application; Figure 6 A schematic diagram of a first loss value of a feature vector, attention distribution, and gradient information of a federated large model fine-tuning method based on mutual learning provided in an embodiment of the present application; Figure 7 A schematic diagram of comparing the server-side and local model losses of a federated large model fine-tuning method based on mutual learning provided in an embodiment of the present application; Figure 8 An overall logic diagram of a federated large model fine-tuning method based on mutual learning provided in an embodiment of the present application; Fig. 9A schematic diagram before and after model fine-tuning of a federated large model fine-tuning method based on mutual learning provided in an embodiment of the present application; Fig.10 A schematic diagram of the internal structure of a federated large model fine-tuning device based on mutual learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0019] The embodiments of the present application provide a method, device and medium for fine-tuning a federated large model based on mutual learning, which solves the problem in the prior art that the federated large language model mainly focuses on the knowledge transfer between the client and the server, but ignores the role of the unique domain knowledge of the client's small model in enriching the server-side large model.

[0020] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0021] Figure 1 A flow chart of a method for fine-tuning a federated large model based on mutual learning provided in an embodiment of the present application. Figure 1 As shown, the embodiment of the present application provides a method for fine-tuning a federated large model based on mutual learning, which specifically includes the following steps: Step 10: Fine-tune the first language model using the client and local private dataset to obtain a second language model; In this step, several clients use local private datasets to fine-tune their respective small language models. For example, the client private dataset contains 10,245 maintenance conversation records in the electronics field, involving questions and answers between maintenance personnel and equipment holders, covering various electronic issues, including electronic equipment card return and specific instructions for use. The private dataset has been cleaned and preprocessed to remove irrelevant information and noise.

[0022] As an optional embodiment, fine-tuning the first language model through the client and the local private data set to obtain the second language model may specifically include: Step 101: introducing two low-rank matrices in the attention mechanism layer and the feedforward neural network layer of the first language model; In this step, two low-rank matrices A and B are introduced in the decisive layers of the model, such as the attention mechanism layer of the Transformer or the feedforward neural network layer. The ranks of these matrices are usually much lower than the ranks of the original matrices, so they have fewer parameters.

[0023] Step 102: Initialize the low-rank matrix, use the local private data set to train the low-rank matrix, and obtain a fine-tuned model.

[0024] In this step, the private data set of each client is input into the first language model, and the loss of the model on the task is calculated. During the back propagation process, only the low-rank matrices A and B are updated, while other parameters of the original model remain unchanged. Through multiple iterative training, the low-rank matrix will gradually learn the specific characteristics of the electronic device conversation, thereby achieving fine-tuning of the model. For example, the original model parameter W can be decomposed into the product of two low-rank matrices A and B plus a very small residual matrix ΔW, that is, W = A * B^T +ΔW. First of all, we can understand that ΔW represents the residual matrix, which is a very small matrix used to capture the part of A*B^T that cannot fully represent W; the letter T represents the transpose operation, which represents the transpose matrix of matrix B, in order to enable matrices A and B to be multiplied. Then, during the fine-tuning process, only A, B and ΔW are updated, while most of the parameters of the original model remain unchanged. It is assumed that the size of the original weight matrix W is d×d, and two low-rank matrices A (size is d×r) and B (size is r×d) are defined. The original weight matrix W is a 4×4 square matrix (ie, d=4). A smaller rank r=2 is selected, and low-rank matrices A (size is 4×2) and B (size is 2×4). The residual matrix ΔW is also a 4×4 square matrix, but its element value is very small at the beginning, close to zero, and the learning rate is set to 0.0001. Furthermore, the predetermined number of iterations is 1000. During the training process, each iteration uses a batch of conversation records to calculate the gradient and update the parameters of the low-rank matrices A, B and the residual matrix ΔW. Fine-tuning can also include updating the weights of the model, adding additional layers, or adjusting the architecture of the model. The embodiment of the present application is to update the parameters, which can be determined according to the actual situation. As the iteration proceeds, the loss value will gradually decrease, indicating that the model's ability to understand the conversation of the electronic device is gradually improving.

[0025] As an optional embodiment, the method may further include: Step 103: evaluating the accuracy and efficiency of the fine-tuned model through synonym replacement; In this step, for example, replace "mobile phone" with "mobile phone", "screen" with "display screen", etc., and calculate the accuracy between the model prediction results and the actual labels. For example, the test set contains 100 samples, 50 of which belong to the electronic product category. After synonym replacement, the model correctly predicts 48 samples of the electronic product category, and the accuracy is: 48 / 50 = 96%; for generation tasks, indicators such as the quality, fluency and match with the input text of the generated text can be evaluated. For example, the BLEU (Bilingual Evaluation Understudy) score is used to evaluate the quality of the generated text. Assume that the BLEU score ranges from 0 to 1, where 1 indicates a perfect match. After evaluation, the BLEU score of the generated text is 0.85, indicating that the generated text has a high similarity with the reference text. At the same time, the time required for the model to process the test set is recorded to evaluate its efficiency. For example, if the total time for the model to process 100 samples is 10 seconds, the throughput of the model is: 100 / 10 = 10 samples / second, or the average time to process each sample is: 10 / 100 = 0.1 seconds / sample, such as Figure 2 shown.

[0026] Step 104: When the accuracy and efficiency reach a preset threshold, the current low-rank matrix is ​​saved to obtain a second language model.

[0027] In this step, for example, for prediction tasks (predictive maintenance step), the accuracy may be required to be no less than 90%; for generation tasks (generative maintenance step), the quality score of the generated text may be required to be no less than 85 points and the processing time may not exceed 1 second.

[0028] Step 20: Calculate the public data set through the second language model to obtain a second result, wherein the public data set already has the first result, and the second result includes feature vectors, attention distribution, and gradient information; That is to say, the purpose of the embodiment of the present application is to input the maintenance requirements and obtain the maintenance steps. Each client has its own dedicated field. In this step, the public data set is the maintenance process dialogue of all items that have been declassified, and each dialogue is accompanied by a manually revised reference answer. The first result is the reference answer, and its feature vector, etc. are known. All dialogues are input into the second language model, and through natural language processing, the following are obtained: 1. The feature vector of the current dialogue is obtained. The feature representation generated inside the model can be used for subsequent tasks such as classification, clustering or similarity calculation; 2. The distribution of attention weights of each part of the model when processing the input text helps to understand the model's focus on the input data; 3. The gradient value calculated during the model training process can be used to analyze the training dynamics and optimization direction of the model.

[0029] Step 30: Calculate the first loss value of the second result and the first result, and combine the second result and the first loss value into a first knowledge set; In this step, the mean square error (MSE) can be used as the loss function to calculate the loss value of the above feature vector. MSE =1 / n , n is the dimension of the feature vector, yi Is the first result in i eigenvalues, y ^ i The second result is i eigenvalues, assuming the first result is [1,2,3,4,5,6,7,8,9,10], the second result is [1.1,1.9,3.2,3.8,5.1,6.3,6.8,8.2,8.9,10.1], the first loss value MSE =101[(1-1.1) 2 +(2-1.9) 2 +…+(10-10.1) 2 ]=0.05, such as Figure 3 As shown in , after the client fine-tuning is completed, each client calculates a knowledge set on the same public dataset, such as Figure 4 As shown, each element in the set is a loss-original output pair, and the total number of elements in the set is the number of data in the dataset.

[0030] Step 40: Obtain a knowledge set, and calculate the public data set through a third language model to obtain a third result; In this step, for example, the third language model may be Llama2-13B, which is a pre-trained global model that has been pre-trained on a large amount of text data and then outputs a feature vector; each client uploads its calculated knowledge set to the server; the server converts the original output in the knowledge set uploaded by the client into the original output of the server model based on the vocabulary mapping table between the models of each client.

[0031] Step 50: Calculate a second loss value of the third result, and compare the first loss value with the second loss value to fine-tune the third language model.

[0032] In this step, if Figure 5 As shown, for example, the third result feature vector is [0.9, 2.1, 2.9, 4.2, 4.9, 6.1, 7.2, 7.8, 9.1, 9.8], MSE = 101 [(1-0.9) 2 +(2-2.1) 2 +…+(10-9.8)2 ], assuming the calculated MSE is 0.08, if the second loss value (0.08) is greater than the first loss value (0.05), you can consider using the knowledge of the second language model (through knowledge distillation) to fine-tune the third language model.

[0033] As an optional embodiment, calculating the second loss value of the third result and comparing the first loss value with the second loss value to fine-tune the third language model may specifically include: Step 501: obtaining the minimum first loss value, and comparing the minimum first loss value with the second loss value according to a vocabulary mapping table with the client; As an optional embodiment, Figure 6 As shown, obtaining the minimum first loss value and comparing the minimum first loss value with the second loss value according to the vocabulary mapping table with the client may specifically include: Step 5011: based on the first result including calculating the first loss value of the feature vector, the attention distribution and the gradient information, respectively comparing the first loss values ​​of the feature vector, the attention distribution and the gradient information; In this step, for example, if the first loss value of the feature vector of client 1 is the smallest and the first loss value of the attention distribution of client 2 is the smallest, the feature vector of client 1 and the attention distribution of client 2 are selected respectively.

[0034] Step 5012: Select the output forms with the smallest first loss value of the feature vector, attention distribution and gradient information respectively, and combine them to obtain the smallest first loss value.

[0035] In this step, the feature vector of client 1, the attention distribution of client 2, etc. are combined to obtain the minimum first loss value.

[0036] Step 502: If it is less than the second loss value, obtain the first knowledge set of the client and fine-tune the third language model.

[0037] In this step, the server compares the loss of each client for each data inference, takes the client loss value with the minimum loss, and compares it with the loss calculated by the server-side model. If it is less than the loss value calculated by the server, the original output converted by the client on the data is included in the server-side knowledge base, and the server-side model is fine-tuned using these knowledge bases.

[0038] As an optional embodiment, after calculating the second loss value of the third result and comparing the first loss value with the second loss value to fine-tune the third language model, the method may further include: Step 60: The third result and the second loss value form a second knowledge set; In this step, the server calculates the loss value and the original output of the local large language model inference for each data in the public dataset, and also forms a knowledge set of loss-original output pairs.

[0039] Step 70: Send the third result to the client according to the vocabulary mapping table with the client.

[0040] In this step, the server sends the knowledge set on the public dataset to each client; each client converts the original output in the knowledge set sent by the server into the original output of its own model according to the vocabulary mapping table between itself and the server; each client compares the loss of the server on each data with the loss calculated by the local model. If the loss of the server is smaller, the converted original output of the server on this data is incorporated into the knowledge base of this client, and the local model is fine-tuned using these knowledge bases. As Figure 7 shown, for example, on data item A of client 1, the loss of the server L_server_A = 0.025, while the loss of the local model L_local_A = 0.03. Since L_server_A < L_local_A, client 1 incorporates the converted original output of the server on data item A into its knowledge base and fine-tunes the local model using this knowledge. Refer to Figure 8 , in this way, finally, the knowledge set, parameters, and knowledge with smaller loss values (through knowledge distillation) are selected to fine-tune the large language model on the local or server side. As Fig. 9 shown, the accuracy rate has increased.

[0041] As an optional embodiment, before fine-tuning the first language model through the client and the local private dataset to obtain the second language model, the method may further include: Step 01: Collect the local private dataset and the public dataset, extract the vocabulary, and count the occurrence frequency and context information of the vocabulary; Step 02: Determine the mapping relationship between the vocabulary based on the similarity of the vocabulary and the context information to obtain the vocabulary mapping table.

[0042] As an optional embodiment, the method may further include: Step 03: Remove duplicate vocabulary and merge vocabulary with similarity greater than a preset threshold to update the vocabulary mapping table regularly; Step 04: Establish a user feedback mechanism and a sharing mechanism to allow different clients to share the first knowledge set.

[0043] The above is the method embodiment proposed by this application. Based on the same inventive concept, the embodiment of this application also provides a device for fine-tuning a federated large model based on mutual learning, and its structure is as Fig.10 shown.

[0044] Fig.10A schematic diagram of the internal structure of a federated large model fine-tuning device based on mutual learning provided in an embodiment of the present application. Fig.10 As shown, the device includes: at least one processor 1001; and, a memory 1002 communicatively connected to at least one processor; The memory 1002 stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor 1001 so that the at least one processor 1001 can: fine-tune the first language model through the client and the local private data set to obtain the second language model; calculate the public data set through the second language model to obtain the second result, wherein the public data set already has the first result, and the second result includes the feature vector, the attention distribution and the gradient information; calculate the first loss value of the second result and the first result, and form the second result and the first loss value into a first knowledge set; obtain the knowledge set, and calculate the public data set through the third language model to obtain the third result; calculate the second loss value of the third result, and compare the first loss value with the second loss value to fine-tune the third language model.

[0045] Some embodiments of the present application provide corresponding Figure 1 A non-volatile computer storage medium for fine-tuning a federated large model based on mutual learning, storing computer executable instructions, wherein the computer executable instructions are configured to: fine-tune a first language model through a client and a local private data set to obtain a second language model; calculate a public data set through the second language model to obtain a second result, wherein the public data set already has a first result, and the second result includes a feature vector, an attention distribution, and gradient information; calculate a first loss value of the second result and the first result, and form a first knowledge set with the second result and the first loss value; obtain a knowledge set, and calculate the public data set through a third language model to obtain a third result; calculate a second loss value of the third result, and compare the first loss value with the second loss value to fine-tune the third language model.

[0046] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the IoT device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0047] The system and medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the system and medium also have similar beneficial technical effects to the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the system and medium will not be repeated here.

[0048] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0049] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0050] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0051] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0052] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0053] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0054] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0055] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0056] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for fine-tuning a federated large model based on mutual learning, characterized in that: The method comprises: Fine-tune the first language model through the client and local private dataset to obtain the second language model; Calculate the public data set by using the second language model to obtain a second result, wherein the public data set already has the first result, and the second result includes a feature vector, attention distribution, and gradient information; Calculate the first loss value of the second result and the first result, and form the second result and the first loss value into a first knowledge set; Acquire the knowledge set, and calculate the public data set using a third language model to obtain a third result; A second loss value of the third result is calculated, and the first loss value is compared with the second loss value to fine-tune the third language model.

2. According to the mutual learning-based federated large model fine-tuning method of claim 1, it is characterized in that: Calculating a second loss value of the third result, and comparing the first loss value with the second loss value to fine-tune the third language model, specifically comprising: Obtaining a minimum first loss value, and comparing the minimum first loss value with a second loss value according to a vocabulary mapping table with the client; If the first loss value is less than the second loss value, the first knowledge set of the client is obtained, and the third language model is fine-tuned.

3. According to the mutual learning-based federated large model fine-tuning method of claim 1, it is characterized in that: After calculating the second loss value of the third result and comparing the first loss value with the second loss value to fine-tune the third language model, the method further includes: Combining the third result and the second loss value into a second knowledge set; The third result is sent to the client according to the vocabulary mapping table with the client.

4. According to the mutual learning-based federated large model fine-tuning method of claim 1, it is characterized in that: Before fine-tuning the first language model through the client and the local private data set to obtain the second language model, the method further includes: Collect local private data sets and public data sets, extract vocabulary, and count the occurrence frequency and context information of the vocabulary; Based on the similarity of the words and context information, a mapping relationship between the words is determined to obtain the word mapping table.

5. According to claim 4, a method for fine-tuning a federated large model based on mutual learning is characterized in that: The method further comprises: Deduplication processing is performed on repeated words, and words with similarity greater than a preset threshold are merged to regularly update the word mapping table; A user feedback mechanism and a sharing mechanism are established to allow the first knowledge set to be shared between different clients.

6. The method for fine-tuning a federated large model based on mutual learning according to claim 1, characterized in that: The first language model is fine-tuned through the client and the local private dataset to obtain the second language model, which includes: In the attention mechanism layer and feedforward neural network layer of the first language model, two low-rank matrices are introduced; Initialize the low-rank matrix, and use the local private data set to train the low-rank matrix to obtain a fine-tuned model.

7. A method for fine-tuning a federated large model based on mutual learning according to claim 6, characterized in that: The method further comprises: By synonym replacement, evaluating the accuracy and efficiency of the fine-tuned first language model; When the accuracy and efficiency reach a preset threshold, the current low-rank matrix is ​​saved to obtain the second language model.

8. The method for fine-tuning a federated large model based on mutual learning according to claim 1, characterized in that: Obtaining a minimum first loss value, and comparing the minimum first loss value with a second loss value according to a vocabulary mapping table with the client, specifically includes: Based on the first result, the first loss values ​​of the feature vector, the attention distribution and the gradient information are calculated, and the first loss values ​​of the feature vector, the attention distribution and the gradient information are compared respectively; The output forms with the smallest first loss values ​​of the feature vector, attention distribution, and gradient information are respectively selected, and combined to obtain the minimum first loss value.

9. A federated large model fine-tuning device based on mutual learning, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Fine-tune the first language model through the client and local private dataset to obtain the second language model; Calculate the public data set by using the second language model to obtain a second result, wherein the public data set already has the first result, and the second result includes a feature vector, attention distribution, and gradient information; Calculate the first loss value of the second result and the first result, and form the second result and the first loss value into a first knowledge set; Acquire the knowledge set, and calculate the public data set using a third language model to obtain a third result; A second loss value of the third result is calculated, and the first loss value is compared with the second loss value to fine-tune the third language model.

10. A non-volatile computer storage medium for fine-tuning a federated large model based on mutual learning, storing computer executable instructions, characterized in that: The computer executable instructions are configured to: fine-tune the first language model through the client and the local private data set to obtain a second language model; Calculate the public data set by using the second language model to obtain a second result, wherein the public data set already has the first result, and the second result includes a feature vector, attention distribution, and gradient information; Calculate the first loss value of the second result and the first result, and form the second result and the first loss value into a first knowledge set; Acquire the knowledge set, and calculate the public data set using a third language model to obtain a third result; A second loss value of the third result is calculated, and the first loss value is compared with the second loss value to fine-tune the third language model.

Images