Fine-tuning method and device for federated learning large model

Abstract

The application discloses a federal learning large model fine-tuning method and device, the method comprising: a server end divides a pre-trained model into a first model and a second model; a client constructs a local model consistent with the first model structure, and fixes the local model parameters according to the first model parameters; in each round of communication, the client obtains embedded data, and encrypts the embedded data according to a differential privacy mechanism to obtain encrypted embedded data; the server end inputs the encrypted embedded data into the second model to obtain output data; the client calculates model loss according to the output data to obtain corresponding gradients; the server end calculates the gradients corresponding to the parameters in the second model according to the gradients, and updates the second model according to the gradients corresponding to the parameters in the second model, and repeats the communication training until the second model converges; thus, the federal separation and differential privacy encryption avoid the risk of private domain data and model parameter leakage.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method for fine-tuning a federated learning large model, a computer-readable storage medium, a computer device, and a device for fine-tuning a federated learning large model. Background Technology

[0002] In related technologies, large-scale deep learning models have demonstrated outstanding performance in various tasks, including natural language processing, computer vision, and speech recognition. By pre-training on large amounts of data, these models can extract useful general representations for a wide range of downstream tasks. Although these pre-trained large models already have a certain predictive ability, this zero-shot predictive ability cannot meet the actual needs of certain specific tasks. Therefore, transfer learning methods such as model fine-tuning are needed to adapt the pre-trained large models to specific downstream tasks.

[0003] The current approach typically involves training a pre-trained large model on a dataset specific to a particular task, updating some or all of the model's parameters to enable it to solve that specific task. The advantage of fine-tuning large models lies in saving time and resources associated with large-scale training while achieving a more specialized and better-performing customized model for that specific task. However, in real-world scenarios, the specific task dataset (private domain data) and the pre-trained large model (model parameters) are usually data assets held by different companies. Due to privacy risks, companies holding private domain data cannot directly share their original data. Furthermore, because pre-training large models is time-consuming and labor-intensive, companies holding model parameters are unwilling to directly share them. In addition, fine-tuning the model requires significant computing resources, posing a significant challenge to enabling large models across various industries, especially for small and micro-enterprises lacking sufficient computing power.

[0004] In summary, directly applying current model fine-tuning algorithms to customized large-scale industry models presents the following three challenges: 1. Pre-training large models is time-consuming and labor-intensive. As a new type of data asset, companies holding model parameters are unwilling to share them directly; 2. Companies holding private domain data need to provide the original private domain data to the companies holding model parameters for centralized fine-tuning, which poses a risk of data privacy leakage; 3. Large models have a massive number of parameters, and fine-tuning large models also requires certain computing resources. Even if companies holding model parameters are willing to provide pre-trained model parameters, companies holding private domain data may not have the computing resources for fine-tuning large models. Summary of the Invention

[0005] This invention aims to at least partially address one of the technical problems in the aforementioned technologies. To this end, one objective of this invention is to propose a method for fine-tuning large federated learning models, which avoids the risk of leakage of private domain data and model parameters through federated separation and differential privacy encryption.

[0006] A second objective of this invention is to provide a computer-readable storage medium.

[0007] The third objective of this invention is to provide a computer device.

[0008] The fourth objective of this invention is to provide a fine-tuning device for federated learning large models.

[0009] To achieve the above objectives, a first aspect of the present invention proposes a method for fine-tuning a federated learning large model, comprising the following steps: The server divides a pre-trained model into a first model and a second model according to module functions, and sends the model parameters corresponding to the first model to the client; the client constructs a local model with the same structure as the first model, loads the model parameters corresponding to the first model into the local model, and fixes the parameters of the local model; in each round of communication, the client obtains sequence data from a private domain dataset, and inputs the sequence data into the local model to obtain corresponding embedded data; the client adds to the embedded data according to a differential privacy mechanism. Random noise is added to obtain encrypted embedded data, and the encrypted embedded data is sent to the server. The server inputs the encrypted embedded data into the second model to obtain model output data, and sends the model output data to the client. The client calculates the model loss based on the model output data to obtain the corresponding gradient, and sends the gradient to the server. The server calculates the gradient corresponding to the parameters in the second model based on the gradient, and updates the second model based on the gradient corresponding to the parameters in the second model to obtain the updated second model, so as to perform the next round of model update until the second model converges.

[0010] According to the federated learning large model fine-tuning method of this invention, the server first divides the pre-trained model into a first model and a second model according to the module functions, and sends the model parameters corresponding to the first model to the client. Then, the client constructs a local model with the same structure as the first model, loads the model parameters corresponding to the first model into the local model, and fixes the parameters of the local model. In each round of communication, the client obtains sequence data from the private domain dataset and inputs the sequence data into the local model to obtain the corresponding embedded data. Next, the client adds random noise to the embedded data according to a differential privacy mechanism to obtain encrypted embedded data, and sends the encrypted embedded data to the server. Then, the server inputs the encrypted embedded data into the second model to obtain model output data, and sends the model output data to the client. Subsequently, the client calculates the model loss based on the model output data to obtain the corresponding gradient, and sends the gradient to the server. Finally, the server calculates the gradient corresponding to the parameters in the second model based on the gradient, and updates the second model according to the gradient corresponding to the parameters in the second model to obtain the updated second model, so as to perform the next round of model updates until the second model converges. Thus, the risk of leakage of private domain data and model parameters is avoided through federated separation and differential privacy encryption.

[0011] In addition, the federated learning large model fine-tuning method proposed in the above embodiments of the present invention may also have the following additional technical features:

[0012] Optionally, the first model includes an embedding block, which consists of multiple embedding layers and is used to convert text sequences into embedded data. The second model includes a transformer block and an application block, which consists of multiple transformer layers stacked together and is used to learn a generalized representation of the text. The application block is constructed according to the attributes of the downstream task and processes the generalized representation output by the transformer block into the model output according to the specific task requirements.

[0013] Optionally, the client acquires sequence data from the private domain dataset to input the sequence data into the local model to obtain corresponding embedding data, including: the client acquires a batch of training data and its corresponding labels from the private domain dataset; segments each text sequence in the training data into words and assigns words in the text sequence to tokens according to the bag-of-words system, thereby obtaining a token sequence; and inputs the token sequence into the local model so that, through forward propagation of the local model, the embedding data corresponding to the current batch can be obtained.

[0014] Optionally, the client adds random noise to the embedded data according to a differential privacy mechanism to obtain encrypted embedded data, including: the client uses a Gaussian mechanism or a random mechanism that satisfies dχ-privacy to generate random noise for the embedded data, and adds the random noise to the embedded data to obtain encrypted embedded data.

[0015] To achieve the above objectives, a second aspect of the present invention provides a computer-readable storage medium storing a federated learning large model fine-tuning program, which, when executed by a processor, implements the federated learning large model fine-tuning method as described above.

[0016] According to embodiments of the present invention, a computer-readable storage medium stores a federated learning large model fine-tuning program so that when a processor executes the federated learning large model fine-tuning, it implements the federated learning large model fine-tuning method as described above. Thus, the risk of leakage of private domain data and model parameters is avoided through federated separation and differential privacy encryption.

[0017] To achieve the above objectives, a third aspect of the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the federated learning large model fine-tuning method as described above.

[0018] According to the computer device of the present invention, a federated learning large model fine-tuning program is stored in a memory, so that when the processor executes the federated learning large model fine-tuning program, the federated learning large model fine-tuning method as described above is implemented. Thus, the risk of leakage of private domain data and model parameters is avoided through federated separation and differential privacy encryption.

[0019] To achieve the above objectives, a fourth aspect of the present invention proposes a federated learning large model fine-tuning device, comprising: a model partitioning module, which is used to partition a pre-trained model into a first model and a second model according to the module function, and send the model parameters corresponding to the first model to a client; a construction module, which is used to construct a local model with the same structure as the first model, load the model parameters corresponding to the first model into the local model, and fix the parameters of the local model; a local training module, which is used to acquire sequence data from a private domain dataset, so as to input the sequence data into the local model to obtain corresponding embedded data; and an encryption module, which is used to add encryption to the embedded data according to a differential privacy mechanism. The system includes: a random noise source to obtain encrypted embedded data, which is then sent to the server; a model training module to input the encrypted embedded data into the second model to obtain model output data, which is then sent to the client; a calculation module to calculate the model loss based on the model output data to obtain the corresponding gradient, which is then sent to the server; and a gradient update module to calculate the gradient corresponding to the parameters in the second model based on the gradient, and update the second model based on the gradient corresponding to the parameters in the second model to obtain the updated second model, which is then used for the next round of model updates until the second model converges.

[0020] According to an embodiment of the present invention, a federated learning large model fine-tuning device avoids the risk of leakage of private domain data and model parameters through federated separation and differential privacy encryption.

[0021] In addition, the federated learning large model fine-tuning device proposed in the above embodiments of the present invention may also have the following additional technical features:

[0022] Optionally, the first model includes an embedding block, which consists of multiple embedding layers and is used to convert text sequences into embedded data. The second model includes a transformer block and an application block, which consists of multiple transformer layers stacked together and is used to learn a generalized representation of the text. The application block is constructed according to the attributes of the downstream task and processes the generalized representation output by the transformer block into the model output according to the specific task requirements.

[0023] Optionally, the client acquires sequence data from the private domain dataset to input the sequence data into the local model to obtain corresponding embedding data, including: the client acquires a batch of training data and its corresponding labels from the private domain dataset; segments each text sequence in the training data into words and assigns words in the text sequence to tokens according to the bag-of-words system, thereby obtaining a token sequence; and inputs the token sequence into the local model so that, through forward propagation of the local model, the embedding data corresponding to the current batch can be obtained.

[0024] Optionally, the client adds random noise to the embedded data according to a differential privacy mechanism to obtain encrypted embedded data, including: the client uses a Gaussian mechanism or a random mechanism that satisfies dχ-privacy to generate random noise for the embedded data, and adds the random noise to the embedded data to obtain encrypted embedded data. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the federated learning large model fine-tuning method according to an embodiment of the present invention.

[0026] Figure 2 This is a block diagram of a federated learning large model fine-tuning device according to an embodiment of the present invention. Detailed Implementation

[0027] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0028] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the invention to those skilled in the art.

[0029] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0030] It should be noted that model fine-tuning is a collaborative effort between the client and the server. The client is the data provider, possessing a private text dataset D, while the server is the model provider, possessing a pre-trained large model M. The client completes the model fine-tuning process by remotely requesting the server to execute the corresponding service, thereby obtaining a large industry expert model with specific domain knowledge.

[0031] In this specific embodiment of the invention, it is assumed that company A is the data provider, possessing a private text dataset D, where D consists of text sequences X∈R. n×l and label Y∈R n×c The system consists of two parts, where l and c represent the number of words and the number of tag types, respectively; company B is the model provider, possessing a pre-trained large language model M; the model customization and fine-tuning aims to use company A's dataset D to perform supervised training on company B's pre-trained model M, so that the fine-tuned model M′ has better performance on dataset D. The optimization process is formally described as argmin. θ L(X,Y,θ), where θ are the model parameters of M, and L(x) is the loss function.

[0032] Figure 1 This is a flowchart illustrating the federated learning large model fine-tuning method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method for fine-tuning the federated learning large model includes the following steps:

[0033] S101, the server divides the pre-trained model into a first model and a second model according to the module function, and sends the model parameters corresponding to the first model to the client.

[0034] In other words, before each round of communication model training, the server-side processing needs to divide the pre-trained model into a first model and a second model, and send the model parameters corresponding to the first model to the client.

[0035] As an example, the first model includes an embedding block, which consists of multiple embedding layers and is used to transform text sequences into embedded data. The second model includes a transformer block and an application block. The transformer block consists of multiple transformer layers stacked together and is used to learn a generalized representation of the text. The application block is constructed according to the attributes of the downstream task and processes the generalized representation output by the transformer block into the model output according to the specific task requirements.

[0036] In other words, the pre-trained model can be divided into three parts according to the module functions: the embedding block, the transformer block, and the application block. The embedding block consists of multiple embedding layers (such as fully connected layers) and is used to transform text sequences into embedded data. The transformer block consists of multiple transformer layers stacked together and is the core structure of the large language model, used to learn the generalized representation of text. The application block is constructed according to the attributes of the downstream task and processes the generalized representation output by the transformer block into the model output according to the specific task requirements. In the separate fine-tuning process, the pre-trained model M on the server side is divided into two parts, M1 (containing the embedding block) and M2 (containing the transformer block and the application block), and the model parameters of M1 and M2 are θ1 and θ2, respectively.

[0037] S102, the client constructs a local model with the same structure as the first model, loads the model parameters corresponding to the first model into the local model, and fixes the parameters of the local model.

[0038] In other words, before each round of communication model training, the client needs to build a local model with the same structure as the first model, load the model parameters corresponding to the first model into the local model, and fix the parameters of the local model.

[0039] In other words, the client builds a model M1′ locally with the same structure as the server-side model M1, loads the parameters θ1 of M1 obtained from the server into M1′, and fixes the parameters of M1′ throughout the training process.

[0040] S103, in each round of communication, the client obtains sequence data from the private domain dataset so that the sequence data can be input into the local model to obtain the corresponding embedded data.

[0041] As one example, the client obtains sequence data from a private domain dataset to input the sequence data into a local model to obtain corresponding embedded data, including: the client obtaining a batch of training data (text) from the private domain dataset. B and its corresponding label Y B ∈R n×c Each text sequence in the training data is segmented into words, and the words in the text sequence are mapped to tokens according to the bag-of-words algorithm, thus obtaining the token sequence X. B ∈R n×l ; Translate the word sequence X B ∈R n×l The data is input into the local model M1′ so that it can be propagated forward through the local model to obtain the embedded data E corresponding to the current batch. B ∈R B×l×h :

[0042] E B =forward(X B ,θ1)

[0043] Here, forward(·) represents the forward propagation of the model.

[0044] S104, the client adds random noise to the embedded data according to the differential privacy mechanism to obtain encrypted embedded data, and sends the encrypted embedded data to the server.

[0045] As an example, the client adds random noise to the embedded data according to the differential privacy mechanism to obtain encrypted embedded data, including: the client uses a Gaussian mechanism or a random mechanism that satisfies dχ-privacy to generate random noise for the embedded data, and adds the random noise to the embedded data to obtain encrypted embedded data.

[0046] In other words, the client uses a Gaussian mechanism or a random mechanism that satisfies dχ-privacy for E. B Generate random noise N B ∈R B×l×h The encrypted embedded data E′ is obtained by adding the two data together. B ∈R n×l×h :

[0047] E′ B =E B +N B

[0048] It should be noted that, in order to ensure that the embedded data transmitted from the client to the server does not expose the privacy of the original text data, differential privacy technology is used to encrypt the embedded data; different differential privacy mechanisms can be used to encrypt the embedded data according to the level of privacy restriction, and this invention does not make specific limitations on this.

[0049] For example, Gaussian differential privacy encryption can be used to embed data, where differential privacy is defined as: for a given domain random mechanism If for all All database pairs that satisfy |xy| ≤ 1 Yes, it satisfies:

[0050]

[0051] Then random mechanism It is (ε,δ)-differential privacy, when δ=0, It is ε-differential privacy.

[0052] As an example, for an embedded data E B ∈R B×l×h For each element e in the dataset, adding a random number sampled from a Gaussian distribution yields encrypted data E′ that satisfies the (ε,δ)-differential privacy constraint. B :

[0053]

[0054]

[0055] in, The mean is 0 and the variance is σ. 2 The Gaussian distribution sampling results are used, where s is the sensitivity of e; thus, the encrypted embedded data E′ generated after adding noise through the Gaussian mechanism is obtained. B This can effectively protect the privacy of the original data.

[0056] Alternatively, for example, a dχ-privacy noise addition mechanism can be used to encrypt embedded data, where dχ-privacy is defined as: for a given domain random mechanism If for all All database pairs that satisfy |xy| ≤ 1 satisfy:

[0057]

[0058] Then random mechanism It satisfies dχ-privacy.

[0059] As an example, for an embedded data E B ∈R B×l×h Each word embedding vector w∈R h Add random noise N∈R sampled from an h-dimensional distribution with probability density p(N)∝exp(-ε‖N‖). h This will yield encrypted data E′ that satisfies the dχ- privacy constraint. B :

[0060] w′=w+N

[0061] To sample N from the noise distribution, consider N as a combination (r, p) of vector length r and vector direction p:

[0062] N = rp

[0063] r∈R can represent the distance from the original word embedding vector w to the encrypted word embedding vector w′, p∈R hIt can be viewed as a point on an h-dimensional unit hypersphere, representing the direction of the noise vector N; subsequently, from the gamma distribution... By sampling r and randomly generating an h-dimensional unit vector, the random noise N can be calculated using N = rp.

[0064] S105, the server inputs the encrypted embedded data into the second model to obtain the model output data, and then sends the model output data to the client.

[0065] In other words, the server will use E′ B As input to model M2, M2 first uses a transformer block from E′ B A generalized representation of the original text data in the current batch is extracted. Then, depending on the dataset task (such as text classification), the generalized representation is input into different application blocks to calculate the model's output Y′. B ∈R n×c :

[0066] Y′ B =forward(E′) B ,θ2)

[0067] S106, the client calculates the model loss based on the model output data to obtain the corresponding gradient, and then sends the gradient to the server.

[0068] In other words, the client uses Y according to the loss function calculation formula. B and Y′ B In text classification, the loss of the model is calculated using the following formula:

[0069]

[0070] The client calculates the loss L according to the loss function formula. B For Y′ B gradient

[0071] S107, the server calculates the gradients corresponding to the parameters in the second model based on the gradients, and updates the second model based on the gradients corresponding to the parameters in the second model to obtain the updated second model, so as to carry out the next round of model updates until the second model converges.

[0072] In other words, the client will Y′ B The gradient is sent to the server, which then calculates the gradient of parameter θ2 in model M2 using backpropagation according to the chain rule. And update the model:

[0073]

[0074]

[0075] Where backward(·) represents the backpropagation of the model, and η is the learning rate of the model training.

[0076] Repeat steps S103-S107 until the model converges.

[0077] In summary, according to the federated learning large model fine-tuning method of this invention, the locally input sequence data X∈R is processed by the client. n×d After forward propagation of the local model, the data is transformed into embedded data E∈R. n×l×h Then, random noise N∈R is added to the embedded data according to a random mechanism that satisfies differential privacy or dχ-privacy. n×l×h The encrypted embedded data E′∈R is obtained. n×l×h The encrypted embedded data E′ is then sent to the server, which inputs it into the pre-trained model for forward propagation to calculate the model output Y′∈R. n×c The output Y′ is sent to the client, and the client determines the value based on Y′ and Y∈R. n×c Calculate the model loss L and the gradient of the loss with respect to the output. And The data is sent back to the server, and the server then... Backpropagation is performed to obtain the update gradient of the server-side model. The model is updated; thus, enterprises holding private domain data can use cloud services to fine-tune large models by leveraging the computing resources of cloud service providers (usually enterprises holding model parameters); furthermore, differential privacy is used to encrypt the feature representation of the original data during the federated fine-tuning process, avoiding the risk of privacy leakage of private domain data; in addition, the core part of the pre-trained model parameters does not leave the domain during the federated fine-tuning process, avoiding the risk of model parameter leakage; thus, this invention can effectively avoid the risk of leakage of original data and model parameters while ensuring the performance of the fine-tuned model.

[0078] To implement the above embodiments, this invention proposes a computer-readable storage medium storing a federated learning large model fine-tuning program, which, when executed by a processor, implements the federated learning large model fine-tuning method as described above.

[0079] According to embodiments of the present invention, a computer-readable storage medium stores a federated learning large model fine-tuning program so that when a processor executes the federated learning large model fine-tuning, it implements the federated learning large model fine-tuning method as described above. Thus, the risk of leakage of private domain data and model parameters is avoided through federated separation and differential privacy encryption.

[0080] To implement the above embodiments, this invention proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the federated learning large model fine-tuning method as described above.

[0081] According to the computer device of the present invention, a federated learning large model fine-tuning program is stored in a memory, so that when the processor executes the federated learning large model fine-tuning program, the federated learning large model fine-tuning method as described above is implemented. Thus, the risk of leakage of private domain data and model parameters is avoided through federated separation and differential privacy encryption.

[0082] To implement the above embodiments, this invention also proposes a fine-tuning device for federated learning large models, such as... Figure 2 As shown, the federated learning large model fine-tuning device includes: a model partitioning module 10, a construction module 20, a local training module 30, an encryption module 40, a model training module 50, a computation module 60, and a gradient update module 70.

[0083] The system comprises the following modules: Model partitioning module 10, which divides the pre-trained model into a first model and a second model according to the module's function, and sends the model parameters corresponding to the first model to the client; Construction module 20, which constructs a local model with the same structure as the first model, loads the model parameters corresponding to the first model into the local model, and fixes the parameters of the local model; Local training module 30, which obtains sequence data from the private domain dataset and inputs the sequence data into the local model to obtain the corresponding embedded data; Encryption module 40, which adds random noise to the embedded data according to the differential privacy mechanism to obtain encrypted embedded data and sends the encrypted embedded data to the server; Model training module 50, which inputs the encrypted embedded data into the second model to obtain model output data and sends the model output data to the client; Calculation module 60, which calculates the model loss based on the model output data to obtain the corresponding gradient and sends the gradient to the server; and Gradient update module 70, which calculates the gradient corresponding to the parameters in the second model based on the gradient and updates the second model based on the gradient corresponding to the parameters in the second model to obtain the updated second model for the next round of model updates until the second model converges.

[0084] As an example, the first model includes an embedding block, which consists of multiple embedding layers and is used to transform text sequences into embedded data. The second model includes a transformer block and an application block. The transformer block consists of multiple transformer layers stacked together and is used to learn a generalized representation of the text. The application block is constructed according to the attributes of the downstream task and processes the generalized representation output by the transformer block into the model output according to the specific task requirements.

[0085] As an example, the client obtains sequence data from a private domain dataset to input the sequence data into a local model to obtain the corresponding embedding data. This includes: the client obtaining a batch of training data and its corresponding labels from the private domain dataset; segmenting each text sequence in the training data into words and mapping the words in the text sequence to tokens according to the bag-of-words system to obtain a token sequence; and inputting the token sequence into the local model so that the embedding data corresponding to the current batch can be obtained through forward propagation of the local model.

[0086] As an example, the client adds random noise to the embedded data according to the differential privacy mechanism to obtain encrypted embedded data, including: the client uses a Gaussian mechanism or a random mechanism that satisfies dχ-privacy to generate random noise for the embedded data, and adds the random noise to the embedded data to obtain encrypted embedded data.

[0087] It should be noted that the above description of the fine-tuning method for federated learning large models also applies to this fine-tuning device for federated learning large models, and will not be repeated here.

[0088] In summary, the federated learning large model fine-tuning apparatus according to an embodiment of the present invention avoids the risk of leakage of private domain data and model parameters through federated separation and differential privacy encryption.

[0089] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0091] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0092] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0093] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0094] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0095] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0096] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0097] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0098] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0099] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0100] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for fine-tuning a large federated learning model, characterized in that, Includes the following steps: The server divides the pre-trained model into a first model and a second model according to the module function, and sends the model parameters corresponding to the first model to the client. The client constructs a local model with the same structure as the first model, loads the model parameters corresponding to the first model into the local model, and fixes the parameters of the local model. In each round of communication, the client obtains sequence data from the private domain dataset so as to input the sequence data into the local model to obtain the corresponding embedded data; The client adds random noise to the embedded data according to the differential privacy mechanism to obtain encrypted embedded data, and sends the encrypted embedded data to the server. The server inputs the encrypted embedded data into the second model to obtain model output data, and then sends the model output data to the client. The client calculates the model loss based on the model output data to obtain the corresponding gradient, and sends the gradient to the server. The server calculates the gradient corresponding to the parameters in the second model based on the gradient, and updates the second model based on the gradient corresponding to the parameters in the second model to obtain the updated second model, so as to perform the next round of model update until the second model converges.

2. The method for fine-tuning a large federated learning model as described in claim 1, characterized in that, The first model includes an embedding block, which consists of multiple embedding layers and is used to convert text sequences into embedded data. The second model includes a transformer block and an application block. The transformer block consists of multiple transformer layers stacked together and is used to learn a generalized representation of the text. The application block is constructed according to the attributes of the downstream task and processes the generalized representation output by the transformer block into the model output according to the task requirements.

3. The method for fine-tuning a large federated learning model as described in claim 1, characterized in that, The client obtains sequence data from the private domain dataset to input the sequence data into the local model to obtain corresponding embedded data, including: The client obtains a batch of training data and its corresponding labels from the private domain dataset; Each text sequence in the training data is segmented into words, and the words in the text sequence are mapped to lexical units according to the bag of words, thereby obtaining a lexical unit sequence; The word sequence is input into the local model so that the embedding data corresponding to the current batch can be obtained through forward propagation of the local model.

4. The method for fine-tuning a large federated learning model as described in claim 3, characterized in that, The client adds random noise to the embedded data according to a differential privacy mechanism to obtain encrypted embedded data, including: The client uses a Gaussian mechanism or satisfies... - A privacy-preserving random mechanism generates random noise for the embedded data, and adds the random noise to the embedded data to obtain encrypted embedded data.

5. A computer-readable storage medium, characterized in that, It stores a federated learning large model fine-tuning program, which, when executed by the processor, implements the federated learning large model fine-tuning method as described in any one of claims 1-4.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the federated learning large model fine-tuning method as described in any one of claims 1-4.

7. A device for fine-tuning a federated learning large model, characterized in that, include: The model partitioning module is used to partition the pre-trained model into a first model and a second model according to the module function, and send the model parameters corresponding to the first model to the client. A construction module is used to construct a local model with the same structure as the first model, load the model parameters corresponding to the first model into the local model, and fix the parameters of the local model. A local training module is used to acquire sequence data from a private domain dataset so that the sequence data can be input into the local model to obtain corresponding embedded data. An encryption module is used to add random noise to the embedded data according to a differential privacy mechanism to obtain encrypted embedded data, and then send the encrypted embedded data to the server. A model training module is used to input the encrypted embedded data into the second model to obtain model output data, and send the model output data to the client; The calculation module is used to calculate the model loss based on the model output data in order to obtain the corresponding gradient, and send the gradient to the server. The gradient update module is used to calculate the gradient corresponding to the parameters in the second model based on the gradient, and update the second model based on the gradient corresponding to the parameters in the second model to obtain the updated second model, so as to perform the next round of model update until the second model converges.

8. The federated learning large model fine-tuning apparatus as described in claim 7, characterized in that, The first model includes an embedding block, which consists of multiple embedding layers and is used to convert text sequences into embedded data. The second model includes a transformer block and an application block. The transformer block consists of multiple transformer layers stacked together and is used to learn a generalized representation of the text. The application block is constructed according to the attributes of the downstream task and processes the generalized representation output by the transformer block into the model output according to the task requirements.

9. The federated learning large model fine-tuning device as described in claim 7, characterized in that, The client obtains sequence data from the private domain dataset to input the sequence data into the local model to obtain corresponding embedded data, including: The client obtains a batch of training data and its corresponding labels from the private domain dataset; Each text sequence in the training data is segmented into words, and the words in the text sequence are mapped to lexical units according to the bag of words, thereby obtaining a lexical unit sequence; The word sequence is input into the local model so that the embedding data corresponding to the current batch can be obtained through forward propagation of the local model.

10. The federated learning large model fine-tuning apparatus as described in claim 9, characterized in that, The client adds random noise to the embedded data according to a differential privacy mechanism to obtain encrypted embedded data, including: The client uses a Gaussian mechanism or satisfies... - A privacy-preserving random mechanism generates random noise for the embedded data, and adds the random noise to the embedded data to obtain encrypted embedded data.

Images