Method, system and device for jointly training large language model based on privacy protection

Abstract

The embodiment of the specification provides a method, system and device for jointly training a large language model based on privacy protection. The large language model is divided into a first part network and a second part network. The first part network is deployed on each client and has fixed parameters. The second part network is deployed in a TEE of a server. The method comprises: the server receiving, from each client, a respective data set ciphertext. A single data set ciphertext is obtained by encrypting the embedding features and label data of the private sample of the client. The embedding features are obtained by processing the first part network deployed by the client. In the TEE, a model update is performed. The model update comprises: decrypting each data set ciphertext to obtain each data set plaintext; obtaining the prediction data corresponding to each private sample by using each embedding feature in each data set plaintext through the second part network; and adjusting the specified parameters in the second part network by using the difference between the prediction data corresponding to each private sample and the label data.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a method, system, and apparatus for jointly training a large language model based on privacy protection. Background Technology

[0002] Currently, Large Language Models (LLMs) are one of the hottest research areas in the field of artificial intelligence in recent years. LLMs have wide applications in natural language processing, machine translation, and dialogue systems. For example, in automatic question answering systems, LLMs can be used to generate answers related to user input; in machine translation systems, LLMs can be used to translate user-input source language into target language; and in dialogue systems, LLMs can be used to generate responses related to user input. When LLMs are applied to specific industries (such as healthcare and news), the LLMs need to be fine-tuned using data from that specific industry to obtain an LLM model applicable to that industry (also known as an industry-specific LLM model).

[0003] Within certain industries, organizations or enterprises possess limited data specific to that industry (hereinafter referred to as business data). Therefore, in the context of big data, to improve the prediction accuracy of the trained large language model, it is necessary to jointly utilize business data from multiple data providers (e.g., organizations within a specific industry) as training data to comprehensively train the large language model for that specific industry.

[0004] However, considering that the business data of each data provider often involves their own privacy data, how to provide a method for multi-party joint training of large language models that can protect the privacy data of all parties has become an urgent problem to be solved. Summary of the Invention

[0005] This specification provides one or more embodiments of a method, system, and apparatus for jointly training a large language model based on privacy protection, so as to protect the privacy data security of each party during the joint training of a large language model by multiple parties.

[0006] According to a first aspect, a method for jointly training a large language model based on privacy protection is provided, wherein the large language model is divided into a first part network and a second part network, the first part network is deployed on each client of a plurality of clients with fixed parameters, and the second part network is deployed in a TEE on the server side; the method is applied to the server side, including:

[0007] Each client receives its own encrypted dataset, which is obtained by encrypting the embedding features and label data of its private samples; the embedding features are processed by the first part of the network deployed by the client.

[0008] In the TEE, a model update is performed, which includes:

[0009] Decrypt the ciphertext of each dataset to obtain the plaintext of each dataset;

[0010] By utilizing the embedded features in the plaintext of each dataset, the predicted data corresponding to each private sample is obtained through the second part of the network;

[0011] By utilizing the differences between the predicted data and label data corresponding to each private sample, the specified parameters in the second part of the network are adjusted.

[0012] According to the second aspect, a system for jointly training a large language model based on privacy protection is provided. The system includes a server and several clients. The large language model is divided into a first part network and a second part network. The first part network is deployed on each client and has fixed parameters. The second part network is deployed in the TEE of the server.

[0013] Each client is configured to obtain the embedded features and label data corresponding to each private sample it holds, wherein a single embedded feature is obtained by processing the private sample through the first part of the network deployed by the client itself; the embedded features and label data corresponding to each private sample are encrypted to obtain the dataset ciphertext, and the dataset ciphertext is sent to the server;

[0014] The server is configured to receive encrypted datasets from each client; within the TEE, a model update is performed, the model update including:

[0015] Decrypt the ciphertext of each dataset to obtain the plaintext of each dataset; use the embedding features in the plaintext of each dataset to obtain the predicted data corresponding to each private sample through the second part of the network; use the difference between the predicted data and the label data corresponding to each private sample to adjust the specified parameters in the second part of the network.

[0016] According to a third aspect, an apparatus for jointly training a large language model based on privacy protection is provided. The large language model is divided into a first part network and a second part network. The first part network is deployed on each client among several clients and has fixed parameters. The second part network is deployed in a TEE (Transmission Equipment Environment) on the server side. The apparatus is deployed on the server side and includes:

[0017] The receiving module is configured to receive the encrypted datasets of each client. The encrypted dataset is obtained by encrypting the embedding features and label data of the client's private samples. The embedding features are obtained by processing the first part of the network deployed by the client.

[0018] An execution module, configured to perform model updates within the TEE, includes:

[0019] The decryption unit is configured to decrypt the ciphertext of each dataset to obtain the plaintext of each dataset.

[0020] The unit is configured to utilize the embedded features in the plaintext of each dataset to obtain the prediction data corresponding to each private sample through the second part of the network;

[0021] The adjustment unit is configured to adjust specified parameters in the second part of the network by utilizing the differences between the predicted data and label data corresponding to each private sample.

[0022] According to a fourth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0023] According to a fifth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method of the first aspect.

[0024] According to the privacy-preserving joint training method, system, and apparatus for large language models provided in the embodiments of this specification, the large language model is divided into a first part network and a second part network. The first part network is deployed on each of several clients with fixed parameters, and the second part network is deployed in the TEE of the server. The server receives the encrypted datasets of each client. The encrypted dataset of a single dataset is obtained by encrypting the embedding features and label data of its private samples by the client. The embedding features are processed by the first part network deployed on the client. The server performs model updates in the TEE. The model update includes: decrypting the encrypted datasets to obtain the plaintext datasets; using the embedding features in the plaintext datasets, obtaining the prediction data corresponding to each private sample through the second part network deployed on the client; and adjusting the specified parameters in the second part network based on the difference between the prediction data and the label data corresponding to each private sample.

[0025] In the above process, the server receives encrypted datasets from each client and decrypts them in the TEE to ensure the security of each client's private data, namely private samples and their labels, which will not be exposed at the server. Furthermore, the server uses the differences between the predicted data and label data corresponding to each private sample to adjust specified parameters in the second part of the network. That is, when fine-tuning (training) the large language model, only some parameters in the second part of the network deployed by the server are fine-tuned. Data between the clients does not interact, and the parameters of the first part of the network deployed by each client are fixed. This can better ensure the security of private data between the clients and prevent the private data of one client from being exposed at another client. This achieves the protection of the privacy data security of all parties during the joint training of the large language model by multiple parties. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0027] Figure 1 This is a schematic diagram illustrating the implementation framework of one embodiment disclosed in this specification;

[0028] Figure 2 A schematic flowchart illustrating a privacy-preserving method for jointly training a large language model, provided in an embodiment.

[0029] Figure 3A A schematic diagram of the structure of the second part of the network provided in the embodiment;

[0030] Figure 3B A schematic diagram of yet another structure of the second part of the network provided in the embodiment.

[0031] Figure 4 A schematic diagram of the structure of the first part of the network provided in the embodiment;

[0032] Figure 5 This is another flowchart illustrating a privacy-preserving method for jointly training a large language model, as provided in this embodiment.

[0033] Figure 6 A schematic diagram of a privacy-preserving system for jointly training a large language model, provided as an example;

[0034] Figure 7 This is a schematic block diagram of an apparatus for jointly training a large language model based on privacy protection, provided in an embodiment. Detailed Implementation

[0035] The technical solutions of the embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0036] This specification discloses a method, system, and apparatus for jointly training a large language model based on privacy protection. The application scenarios and technical concepts of the method are first introduced below:

[0037] As mentioned earlier, within certain specific industries, the data (hereinafter referred to as business data) possessed by various institutions (organizations or enterprises) is limited. Therefore, in the context of big data, to improve the prediction accuracy of large language models in a specific industry (such as healthcare or education), it is necessary to jointly utilize the business data held by multiple data providers (i.e., institutions within that specific industry) to comprehensively train the large language model for that industry. However, considering that the business data held by each data provider often involves their private data, how to provide a method for multi-party joint training of large language models that can protect the privacy and security of all parties' data becomes an urgent problem to be solved.

[0038] In view of this, the inventors propose a privacy-preserving method for jointly training large language models. Figure 1 This illustration shows an implementation scenario according to an embodiment disclosed in this specification. In this exemplary implementation scenario, it illustrates the multiple parties involved in the process of jointly training a large language model, including a server (also referred to as an intermediary) and several clients (also referred to as data owners, such as...). Figure 1 The diagram shows clients 1, 2, ..., T, where each client holds samples (which can be called private samples) used for jointly training the large language model, along with the corresponding label data for each private sample. The private samples and label data held by different clients can be the same or different. The server is equipped with a TEE (Trusted Execution Environment). In one implementation, this TEE can be based on an SGX chip.

[0039] The large language model trained by multiple parties is divided into two parts: a first-part network and a second-part network. In this scenario, the first-part network is deployed on each client with fixed parameters. This means that during the joint training of the large language model, the parameters of the first-part network deployed on each client are not adjusted, and the clients do not interact with each other to prevent data from one client (i.e., private samples and their corresponding label data) from being exposed on other clients. The second-part network is deployed in a TEE on the server to protect it and prevent data leakage from each client at the server during the joint training of the large language model.

[0040] During the joint training of a large language model by multiple parties, the server receives the encrypted datasets (such as...) from each client. Figure 1 As shown, client 1's dataset ciphertext 1, client 2's dataset ciphertext 2, ..., client T's dataset ciphertext T), a single dataset ciphertext is obtained by the client encrypting the embedding features and label data of its private samples; the embedding features are obtained by the first part of the network deployed by the client; then, the server performs a model update in its deployed TEE, wherein the model update includes:

[0041] The encrypted data of each dataset is decrypted to obtain the plaintext data of each dataset. Then, the embedded features in the plaintext data of each dataset are used to obtain the prediction data corresponding to each private sample through the second part of the network. Next, the difference between the prediction data and the label data corresponding to each private sample is used to adjust the specified parameters in the second part of the network to achieve joint training of the large language model.

[0042] In one implementation, the clients may include two or more clients. In another implementation, the large language model may include multiple network layers (e.g., N network layers), and the large language model is divided into two parts by layer, including a first network part and a second network part. Wherein, as... Figure 1 As shown, the first part of the network may include the first N1 layers of the N-layer network, and the second part of the network may include the last N2 (N-N1) layers of the multi-layer network, where N1 and N2 are both positive integers. To reduce the number of parameters that need to be fine-tuned during the joint training (fine-tuning) of the large language model, N1 is greater than N2. In one case, the large language model can be a network model based on a transformer structure.

[0043] In the above process, the server receives encrypted datasets from each client and decrypts them in the TEE to ensure the security of each client's private data, namely private samples and their labels, which will not be exposed at the server. Furthermore, the server uses the differences between the predicted data and label data corresponding to each private sample to adjust specified parameters in the second part of the network. That is, when fine-tuning (training) the large language model, only some parameters in the second part of the network deployed by the server are fine-tuned. Data between the clients does not interact, and the parameters of the first part of the network deployed by each client are fixed. This can better ensure the security of private data between the clients and prevent the private data of one client from being exposed at another client. It achieves the protection of the privacy data security of all parties in the process of multi-party joint training of the large language model, and reduces the number of fine-tuning parameters, which can improve the training efficiency of joint training of the large language model to a certain extent.

[0044] The following detailed description, with reference to specific embodiments, of the method, system, and apparatus for joint training of large language models based on privacy protection provided in this specification.

[0045] Figure 2 A flowchart illustrating a privacy-preserving method for jointly training a large language model according to one embodiment of this specification is shown. The method can be applied to a privacy-preserving system for jointly training a large language model, which includes a server and several clients. The server and clients can be implemented using any device, equipment, platform, device cluster, etc., with computing and processing capabilities. During the joint training of the large language model, the large language model (i.e., the jointly trained large language model) is divided into a first part network and a second part network. The first part network is deployed on each of the clients with fixed parameters, and the second part network is deployed in the TEE of the server, such as... Figure 2 As shown, the method includes the following steps S210-S250:

[0046] In step S210, each client obtains the embedding features and label data corresponding to each private sample it holds, wherein a single embedding feature is obtained by processing the private sample through the first part of the network deployed by the client itself.

[0047] Understandably, each client participating in the joint training of the large language model has a training set pre-stored in its corresponding storage device. Taking any client i as an example, the storage device corresponding to client i pre-stores training set i, which includes several samples and their corresponding label data. In this embodiment, the samples held by each client are referred to as private samples. In one case, the private samples and their label data can be in text format.

[0048] In one implementation, the private samples and their label data held by each client are related to the task type that the large language model needs to perform. For example, if the task type that the jointly trained large language model needs to perform is automatic question answering, then the private samples held by each client can be questions, and the label data corresponding to the private samples can be the answers corresponding to those private samples. As another example, if the task type that the jointly trained large language model needs to perform is machine translation, then the private samples held by each client can be first text of the first language class, and the label data corresponding to those private samples can be any second text of a non-first language class, where the second text and the first text express the same semantics.

[0049] In another implementation, the private samples and their labels held by each client can be data corresponding to a specific industry (such as healthcare or education). Accordingly, the large language model obtained through joint training can perform corresponding tasks for that specific industry.

[0050] For example, in the specific industry of healthcare, the private samples held by each client can be healthcare-related questions, and the label data corresponding to the private samples can be the healthcare-related answers for those samples. Accordingly, the large language model obtained through joint training can output the corresponding answers to healthcare-related questions input by users.

[0051] For example, in the education industry, the private samples held by each client can be education-related questions, and the label data corresponding to the private samples can be education-related answers. Accordingly, the large language model obtained through joint training can output corresponding answers to education-related questions input by users, and so on.

[0052] In one implementation, the large language model may include multiple network layers (e.g., N network layers). The large language model is divided into two parts by layers, including a first part network and a second part network. The first part network may include the first N1 layers of the N-layer network, and the first part network can be represented as M. f The second part of the network may include the subsequent N2 (N-N1) layers of the multi-layer network, and the second part of the network can be represented as M. lWhere N1 and N2 are both positive integers. To reduce the number of parameters that need to be fine-tuned during the joint training and fine-tuning of the large language model, N1 is greater than N2. For example, if N equals 28, the first part of the network can include the first 24 layers of the large language model, and the second part of the network can include the last 4 layers of the large language model. Accordingly, the number of parameters in the first part of the network is greater than the number of parameters in the second part of the network. In one case, the large language model can be a network model based on a transformer structure. This large language model is a model pre-trained on a large amount of training text.

[0053] In the process of jointly training a large language model, the implementation principle of each client obtaining the embedding features and label data corresponding to each private sample it holds is similar. The following describes the process of obtaining the embedding features and label data corresponding to each private sample it holds, taking any client i among several clients as an example. For the process of obtaining the embedding features and label data corresponding to each private sample it holds for other clients, please refer to the process of obtaining the embedding features and label data corresponding to each private sample it holds for client i.

[0054] In one implementation, client i can obtain each private sample and its corresponding label data from the aforementioned training set i, and then transfer the private samples it holds (hereinafter referred to as private sample X) to the client i. j (Using an example as an illustration), the input is the first part of its own deployed network, which processes the private sample X input within it. j To obtain the private sample X j Corresponding embedding feature E fj In this way, the embedded features and label data corresponding to each private sample held can be obtained through the aforementioned method.

[0055] In another implementation, the training set i held by client i may further include at least some embedded features corresponding to private samples. The embedded features corresponding to each private sample are features obtained by client i using the first part of its deployed network to process the corresponding private samples. Accordingly, client i can obtain the embedded features corresponding to the private samples and their corresponding label data from the aforementioned training set i.

[0056] After each client obtains the embedded features and label data corresponding to each private sample through the above implementation method, in step S220, each client encrypts the embedded features and label data corresponding to each private sample to obtain the encrypted dataset.

[0057] In one implementation, each client can use a pre-negotiated encryption key with the server to encrypt the embedded features and label data corresponding to each private sample, thus obtaining the encrypted dataset.

[0058] To better protect the private samples (and their corresponding embedded features) and tag data held by each client, in one scenario, the encryption keys between clients can be different. In another scenario, the encryption key is a one-time key. In one implementation, the encryption key can be a one-time password generated based on the OTP (One-Time Password) principle. That is, for a single client, each time it sends data to the server, it can use a different encryption key negotiated with the server to encrypt the data to be sent, thereby better protecting the client's data.

[0059] After each client obtains its own encrypted dataset, in step S230, each client sends its encrypted dataset to the server. Correspondingly, in step S240, the server receives the encrypted dataset from each client. In one implementation, the server can receive the encrypted dataset sent by each client from its REE (Rich Execution Environment) or TEE.

[0060] Subsequently, in order to protect the privacy data of each client, namely the embedded features and label data corresponding to each private sample, and to prevent them from being leaked at the server, in step S250, the server performs a model update in its deployed TEE.

[0061] In one scenario, after receiving the encrypted datasets from each client in its REE (Remote Access Provider), considering the limited storage space of the TEE (Trusted Execution Environment), the server can transmit the received encrypted datasets to the server's TEE in batches to better ensure the operation of the TEE. This allows the server to execute step S250 within the TEE. For example, if there are A encrypted datasets, the server can transmit the received encrypted datasets to the server's TEE in B batches in its REE, i.e., transmit A / B encrypted datasets to the server's TEE at a time. Correspondingly, the server can perform model updates in the TEE each time it receives A / B encrypted datasets, i.e., based on the received A / B encrypted datasets. Here, A and B are both positive integers, and B can be adjusted as needed.

[0062] In another scenario, after receiving the encrypted datasets sent by each client from their respective clients in its REE, the server can send all the encrypted datasets together to the server's TEE so that the server can execute step S250 in the TEE.

[0063] Specifically, in step S250, the server performs a model update within its deployed TEE, which may include the following steps S251-S253:

[0064] In step S251, the server decrypts the ciphertext of each dataset within its deployed TEE to obtain the plaintext of each dataset. In this step, the server, within the TEE, can use the decryption key corresponding to the encryption key negotiated with the client to decrypt the client's dataset ciphertext to obtain the plaintext of each dataset. The plaintext of a single dataset includes the embedded features and label data corresponding to each private sample of the corresponding client.

[0065] Next, in step S252, the server, in its deployed TEE, uses the embedded features in the plaintext of each dataset to obtain the prediction data corresponding to each private sample through the deployed second part of the network.

[0066] In one implementation, the server, within a TEE (Transparent Environment Execution Environment), for each dataset plaintext, extracts each embedded feature E from the plaintext dataset. f The inputs are then fed into the second part of the network, where the embedded features E are processed. f This yields the corresponding prediction data, and further, the prediction data for each private sample.

[0067] Subsequently, in step S253, the server, within its deployed TEE, adjusts the specified parameters in the second part of the network by utilizing the differences between the predicted data and label data corresponding to each private sample. In this step, after obtaining the predicted data corresponding to each private sample in the TEE, the server can use a preset loss function to determine the model loss by utilizing the differences between the predicted data and label data corresponding to each private sample. Then, with the goal of minimizing the model loss, the server adjusts the specified parameters in the second part of the network deployed on the server until the large language model reaches the preset convergence condition, thus obtaining the jointly trained large language model.

[0068] The preset loss function may include, but is not limited to, cross-entropy loss function and mean squared error loss function. The preset convergence condition may include, but is not limited to, the number of adjustments exceeding a preset threshold, the determined model loss being less than a preset loss threshold, and the training time exceeding a preset duration.

[0069] As mentioned earlier, in one implementation, the large language model may include multiple network layers (e.g., N network layers), and the large language model is divided into a first part network M by layer. f Second part network M l The first part of the network M fThis can include the first N1 layers of the N-layer network, and the first part of the network can be represented as the second part of the network M. l This can include the N2 (N-N1) subsequent network layers of the N-layer network. Correspondingly, in one embodiment, the second part of the network M... l It includes several network layers (i.e., the N2 layers following the aforementioned N-layer network), wherein each network layer includes at least a network sub-layer based on a self-attention mechanism, such as... Figure 3A As shown. The specified parameters in the aforementioned second part of the network may include the parameters of the first linear layer in the self-attention mechanism-based network sublayer.

[0070] In the first part of the network, the structures of each network layer are similar. Figure 4 This diagram illustrates a structural schematic of a single network layer in the first part of the network. Figure 4 In the case where the network layer shown is the first network layer of the first part of the network, its input features are features obtained by encoding private samples from the client. Figure 4 In the case of a network layer shown that is not the first network layer in a large language model, its input features are the outputs of the preceding network layer. The structures of the network layers in the second part of the network are similar. Figure 3A and Figure 3B The diagrams show a structural schematic of a single network layer in the second part of the network. Figure 3B When the network layer shown is the first network layer of the second part of the network, its input features are the embedded features corresponding to the private samples in the plaintext of the dataset obtained by decrypting the ciphertext of the dataset sent by the client. Figure 3B When the network layer shown is a second part of the network that is not the first network layer, the feature of its input is the output of the previous network layer.

[0071] Understandably, a network sublayer based on the self-attention mechanism can include not only the first linear layer but also other linear and non-linear layers. Given that the first linear layer can better capture key information in the input features (i.e., information that helps the large language model improve the accuracy of prediction results), the specified parameters mentioned above in this implementation include the parameters of the first linear layer. In this implementation, during the joint training of the large language model, the parameters of the other layers in the network sublayer based on the self-attention mechanism, excluding the first linear layer, can be frozen. This reduces the number of parameters that need to be fine-tuned (i.e., adjusted) while ensuring the accuracy of the large language model's prediction results in a specific industry (the industry involved in the private samples and their label data held by each client).

[0072] In one implementation, the first linear layer may include a QKV linear layer ( Figure 3B The QKV Linear layer and stacked linear layers shown are shown. Figure 3B The dense linear layer shown is used to process the input of this layer using its QKV weight matrix, calculating the corresponding query Q matrix, key K matrix, and value V matrix. The dense linear layer is used to perform linear processing on its input. The network sublayer based on the self-attention mechanism may also include a third linear layer and a first nonlinear layer. The third linear layer may include, for example, a third linear layer such as: Figure 3B shown A layer is used to process the input of that layer using its processing functions, obtaining intermediate values ​​for calculating the attention value, where d k This represents the number of dimensions of the key K matrix; for example... Figure 3B The "A·V" layer shown is used to calculate the product of the output matrix "A" of the corresponding "Softmax" layer and the aforementioned value matrix V to obtain the attention value. This "Softmax" layer is the first nonlinear layer of the network sublayer based on the self-attention mechanism. Figure 3B As shown, the first nonlinear layer may also include a "layer norm" layer, which is used to normalize the input using the normalization function of the layer; and a "Rotary_emb" layer, which is used to rotate the input using the rotation matrix of the layer, for feature enhancement of the input.

[0073] In another embodiment, the aforementioned single network layer may further include a feedforward network sublayer. In this implementation, to better reduce the number of fine-tuning parameters, the feedforward network sublayer includes at least a second linear layer and a fine-tuning structure, and the fine-tuning structure is configured in parallel with the second linear layer, such as... Figure 3A As shown. The specified parameters also include the parameters of the fine-tuning structure, wherein the number of parameters in the fine-tuning structure is much smaller than the number of parameters in the second linear layer. In other words, in this implementation, the parameters of the second linear layer can be frozen, and the parameters of the first linear layer and the fine-tuning structure, whose number of parameters is much smaller than that of the second linear layer, can be adjusted to better improve the accuracy of the model prediction results and reduce the number of model fine-tuning parameters, thereby improving the efficiency of model fine-tuning.

[0074] In one implementation, the second linear layer may include an MLP (Multilayer Perceptron) layer, i.e., as shown below. Figure 3B The diagram shows mlp linear1 (MLP layer 1) and mlp linear2 (MLP layer 2). Figure 3B As shown, the feedforward network sublayer may further include a second nonlinear layer, which may include, for example: Figure 3B The layernorm layer shown is used to normalize the input using the normalization function of this layer, and as... Figure 3BThe “GLUE” layer shown is used to activate the input using the layer’s activation function.

[0075] In one embodiment, the aforementioned fine-tuning structure includes: a low-rank adapter LoRA, such as Figure 3B As shown. Figure 3B The “Add” layer shown can represent the superposition of the output of the fine-tuning structure LoRA and the output of its parallel second nonlinear layer (mlp linear1 or mlp linear2).

[0076] In another embodiment, the second part of the network includes several network layers. Each network layer includes a self-attention-based sub-layer and a feedforward sub-layer. The self-attention-based sub-layer includes a first linear layer, and the feedforward sub-layer includes a second linear layer and a fine-tuning structure. The fine-tuning structure is set in parallel with the second linear layer, and the specified parameters may include the parameters of the fine-tuning structure. That is, in this implementation, during the joint adjustment of the large language model, only the parameters of the fine-tuning structure in each network layer of the second part of the network can be adjusted to better reduce the number of fine-tuning parameters and improve the model training efficiency.

[0077] In another embodiment, the second part of the network includes several network layers. Each network layer includes a self-attention-based sub-layer and a feedforward sub-layer. The self-attention-based sub-layer includes a first linear layer, and the feedforward sub-layer includes a second linear layer. The specified parameters may include the parameters of the first and second linear layers, respectively. In this implementation, during the joint adjustment of the large language model, only the parameters of the first and second linear layers in each network layer of the second part of the network can be adjusted, while the other parameters of each network layer of the second part of the network are frozen, so as to reduce the number of fine-tuning parameters and improve the model training efficiency.

[0078] In one implementation, such as Figure 4 As shown, the first part of the network M f A single network layer can also include a self-attention-based network sublayer and a feedforward network sublayer, wherein the first part of the network M f The specific structure of the self-attention-based sublayer in a single network layer in the first part of the network is similar to that of the self-attention-based sublayer in a single network layer in the second part of the network. For example... Figure 4 As shown, the first part of the network M f The feedforward sublayer in a single network layer in the second part of the network may not include the aforementioned fine-tuning structure; other structures are similar to those in the feedforward sublayer in a single network layer in the second part of the network.

[0079] In this embodiment, the server receives encrypted datasets from each client and decrypts them in the TEE to ensure the security of each client's private data, namely private samples and their labels, which will not be exposed at the server. Furthermore, the server uses the differences between the predicted data and label data corresponding to each private sample to adjust specified parameters in the second part of the network. That is, when fine-tuning (training) the large language model, only some parameters in the second part of the network deployed by the server are fine-tuned. Data between the clients does not interact, and the parameters of the first part of the network deployed by each client are fixed. This can better ensure the security of private data between the clients and prevent the private data of one client from being exposed at another client. It realizes the protection of the privacy data security of all parties in the process of multi-party joint training of the large language model.

[0080] In one embodiment, after the large language model is jointly trained, the first part of the network deployed on each client and the second part of the network deployed on the server can be combined to provide data inference services to users on each client. Specifically, the data inference process may include:

[0081] When any user of the first client among the aforementioned clients has a data inference requirement, the first client receives the data S input by its user that needs to be inferred; and processes the data S1 using the first part of its deployed network to obtain the embedding feature S2 of the data S1; the first client uses the encryption key E1 pre-negotiated with the server to encrypt the embedding feature S2 to obtain the ciphertext [S2] of the embedding feature S2. E1 ; and then embed the feature S2 into the ciphertext [S2]. E1 Send to the server.

[0082] The server receives the embedded feature S2 ciphertext [S2] from the first client. E1 Within its deployed TEE, the embedded feature S2 ciphertext [S2] is decrypted using the decryption key D1, which is pre-negotiated with the first client and corresponds to the encryption key E1. E1 The embedded feature S2 (plaintext) is obtained; then the server processes the embedded feature S2 in the TEE using the second part of the network it deploys to obtain the inference result.

[0083] In one implementation, the server can directly send the inference result to the first client, allowing the first client to display the result to the user. In another implementation, to better protect the privacy of the first client, the server, within its TEE (Trusted Execution Environment), can encrypt the inference result using the encryption key E2 negotiated with the first client, obtaining ciphertext, and then send the ciphertext to the first client. Upon receiving the ciphertext, the first client uses the decryption key D2 corresponding to the encryption key E2 negotiated with the server to decrypt the ciphertext, obtaining the plaintext inference result, which the first client then displays to the user. Note that the encryption key E2 and its corresponding decryption key D2 are different from the aforementioned encryption key E1 and its corresponding decryption key D1.

[0084] Corresponding to the above method embodiments, this specification provides another method for jointly training a large language model based on privacy protection. The jointly trained language model is divided into a first part network and a second part network. The first part network is deployed on each of several clients with fixed parameters, and the second part network is deployed in the TEE of the server. This method is applied to the server, and the server and each client can be implemented through any device, equipment, platform, device cluster, etc. with computing and processing capabilities.

[0085] like Figure 5 The diagram shown illustrates another process for a privacy-preserving method of jointly training a large language model. Figure 5 As shown, the method includes the following steps S510-S520:

[0086] In step S510, each client receives its own encrypted dataset. A single encrypted dataset is obtained by encrypting the embedding features and label data of its private samples; the embedding features are processed by the first part of the network deployed by that client. The implementation principle of step S510 is the same as described above. Figure 2 The implementation principle of step S240 and the implementation process of step S510 can be found in [reference needed]. Figure 2 The implementation process of step S240 shown is as follows. The process by which each client obtains its own encrypted dataset can be found in the previously described embodiment, which illustrates the process of a client obtaining its encrypted dataset (e.g., ...). Figure 2 The steps S210-S230 shown are not described in detail here.

[0087] In step S520, a model update is performed in the TEE, and the model update process includes the following steps S521-S523:

[0088] In step S521, the ciphertext of each dataset is decrypted to obtain the plaintext of each dataset.

[0089] In step S522, the embedded features in the plaintext of each dataset are used to obtain the prediction data corresponding to each private sample through the second part of the network.

[0090] In step S523, the specified parameters in the second part of the network are adjusted by utilizing the differences between the predicted data and label data corresponding to each private sample.

[0091] The implementation principle of step S520 (including steps S521-S523) is the same as described above. Figure 2 The implementation principle of step S250 (i.e., steps S251-S253) is similar. The implementation process of step S520 (including steps S521-S523) can be found in [reference needed]. Figure 2 The implementation process of step S250 (including steps S251-S253) shown will not be described in detail here.

[0092] In one embodiment, the second part of the network includes several network layers, each network layer including at least a network sublayer based on a self-attention mechanism, and the specified parameters include the parameters of the first linear layer in the network sublayer based on the self-attention mechanism.

[0093] In one embodiment, the aforementioned single network layer further includes a feedforward network sublayer, which includes at least a second linear layer and a fine-tuning structure. The fine-tuning structure is configured in parallel with the second linear layer, and the specified parameters also include parameters of the fine-tuning structure.

[0094] In one embodiment, the aforementioned fine-tuning structure includes a low-rank adapter LoRA.

[0095] In one embodiment, the encrypted dataset of each client is obtained by encrypting each sample data and its data characteristics based on the encryption key that the client has pre-negotiated with the server.

[0096] The aforementioned decryption of the ciphertext of each dataset yields the plaintext of each dataset, including:

[0097] Using the decryption key corresponding to the encryption key pre-negotiated with each client, the ciphertext of each dataset is decrypted to obtain the plaintext of each dataset.

[0098] In one embodiment, the aforementioned encryption key is a one-time key.

[0099] In one embodiment, the number of parameters in the first part of the network is greater than the number of parameters in the second part of the network.

[0100] The foregoing description describes specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than those shown in the embodiments, and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily need to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0101] Corresponding to the above method embodiments, this specification provides a system for jointly training a large language model based on privacy protection, such as... Figure 6 As shown, the system includes a server 610 and several clients 620. The large language model is divided into a first part network and a second part network. The first part network is deployed on each client and has fixed parameters, while the second part network is deployed in the TEE of the server 610.

[0102] Each client 620 is configured to obtain the embedded features and label data corresponding to each private sample it holds. The individual embedded features are obtained by processing the private samples through the first part of the network deployed by the client itself. The embedded features and label data corresponding to each private sample are encrypted to obtain the dataset ciphertext, and the dataset ciphertext is sent to the server.

[0103] The server 610 is configured to receive the encrypted datasets of each client; within the TEE, a model update is performed, the model update including:

[0104] Decrypt the ciphertext of each dataset to obtain the plaintext of each dataset; use the embedding features in the plaintext of each dataset to obtain the predicted data corresponding to each private sample through the second part of the network; use the difference between the predicted data and the label data corresponding to each private sample to adjust the specified parameters in the second part of the network.

[0105] Corresponding to the above method embodiments, this specification provides an apparatus 700 for jointly training a large language model based on privacy protection. The large language model is divided into a first part network and a second part network. The first part network is deployed on each of several clients with fixed parameters, and the second part network is deployed in the TEE of the server. The apparatus is deployed on the server, and its schematic block diagram is shown below. Figure 7 As shown, it includes:

[0106] The receiving module 710 is configured to receive the respective dataset ciphertext from each client. The ciphertext of a single dataset is obtained by encrypting the embedding features and label data of its private samples by the client. The embedding features are obtained by processing a first part of the network deployed by the client.

[0107] Execution module 720, configured to perform model updates in the TEE, includes:

[0108] Decryption unit 721 is configured to decrypt the ciphertext of each dataset to obtain the plaintext of each dataset.

[0109] Unit 722 is obtained and configured to use the embedded features in the plaintext of each dataset to obtain the prediction data corresponding to each private sample through the second part of the network;

[0110] The adjustment unit 723 is configured to adjust specified parameters in the second part of the network by utilizing the differences between the predicted data and label data corresponding to each private sample.

[0111] In one optional implementation, the second part of the network includes several network layers, each network layer including at least a network sub-layer based on a self-attention mechanism, and the specified parameters include the parameters of the first linear layer in the network sub-layer based on the self-attention mechanism.

[0112] In one optional implementation, a single network layer further includes a feedforward network sublayer, which includes at least a second linear layer and a fine-tuning structure, wherein the fine-tuning structure is configured in parallel with the second linear layer, and the specified parameters also include parameters of the fine-tuning structure.

[0113] In one alternative implementation, the fine-tuning structure includes a low-rank adapter LoRA.

[0114] In one optional implementation, the encrypted dataset of each client is obtained by encrypting each sample data and its data characteristics based on the encryption key pre-negotiated between the client and the server.

[0115] The decryption unit 721 is specifically configured to use the decryption key corresponding to the encryption key pre-negotiated with each client to decrypt the ciphertext of each dataset and obtain the plaintext of each dataset.

[0116] In one alternative implementation, the encryption key is a one-time key.

[0117] In one alternative implementation, the number of parameters in the first part of the network is greater than the number of parameters in the second part of the network.

[0118] The above system and device embodiments correspond to the method embodiments, and detailed descriptions can be found in the description of the method embodiments section, which will not be repeated here. The system and device embodiments are derived based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments; detailed descriptions can be found in the corresponding method embodiments.

[0119] This specification also provides a computer-readable storage medium storing a computer program that, when executed in a computer, causes the computer to perform the privacy-preserving joint training method for large language models provided in this specification.

[0120] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the privacy-preserving joint training method for large language models provided in this specification.

[0121] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for storage media and computing devices are basically similar to the method embodiments, so they are described more simply; relevant parts can be referred to the descriptions of the method embodiments.

[0122] Those skilled in the art will recognize that the functions described in the embodiments of the present invention in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0123] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made based on the technical solutions of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for jointly training a large language model based on privacy protection, wherein the large language model is divided into a first part network and a second part network, the first part network is deployed on each client of a plurality of clients with fixed parameters, and the second part network is deployed in a TEE on the server side; The method is applied to the server and includes: Each client receives its own encrypted dataset, which is obtained by encrypting the embedding features and label data of its private samples; the embedding features are obtained by processing the private samples by the first part of the network deployed by the client. The received datasets are transmitted in encrypted form to the server's TEE in batches. In the TEE, a model update is performed, which includes: Decrypt the ciphertext of each dataset to obtain the plaintext of each dataset; By utilizing the embedded features in the plaintext of each dataset, the predicted data corresponding to each private sample is obtained through the second part of the network; By utilizing the differences between the predicted data and label data corresponding to each private sample, the specified parameters in the second part of the network are adjusted.

2. The method as described in claim 1, wherein, The second part of the network includes several network layers, and each network layer includes at least a network sub-layer based on a self-attention mechanism. The specified parameters include the parameters of the first linear layer in the network sub-layer based on the self-attention mechanism.

3. The method as described in claim 2, wherein, A single network layer also includes a feedforward network sublayer, which includes at least a second linear layer and a fine-tuning structure. The fine-tuning structure is configured in parallel with the second linear layer, and the specified parameters also include the parameters of the fine-tuning structure.

4. The method of claim 3, wherein, The fine-tuning structure includes: a low-rank adapter LoRA.

5. The method of claim 1, wherein, Each client's encrypted dataset is obtained by encrypting each sample data and its data characteristics based on the encryption key that the client has pre-negotiated with the server. The process of decrypting the ciphertext of each dataset to obtain the plaintext of each dataset includes: Using the decryption key corresponding to the encryption key pre-negotiated with each client, the ciphertext of each dataset is decrypted to obtain the plaintext of each dataset.

6. The method of claim 5, wherein, The encryption key is a one-time key.

7. The method of claim 1, wherein, The number of parameters in the first part of the network is greater than the number of parameters in the second part of the network.

8. A system for jointly training a large language model based on privacy protection, the system comprising a server and several clients, wherein the large language model is divided into a first part network and a second part network, the first part network is deployed on each client with fixed parameters, and the second part network is deployed in the TEE of the server; Each client is configured to obtain the embedded features and label data corresponding to each private sample it holds, wherein, Each embedded feature is obtained by processing private samples through the first part of the network deployed by the client itself; the embedded features and label data corresponding to each private sample are encrypted to obtain the dataset ciphertext, and the dataset ciphertext is sent to the server; The server is configured to receive the encrypted datasets from each client. The received datasets are transmitted in encrypted form to the server's TEE in batches. In the TEE, a model update is performed, which includes: Decrypt the ciphertext of each dataset to obtain the plaintext of each dataset; By utilizing the embedded features in the plaintext of each dataset, the predicted data corresponding to each private sample is obtained through the second part of the network; the specified parameters in the second part of the network are adjusted by utilizing the difference between the predicted data and the label data corresponding to each private sample.

9. An apparatus for jointly training a large language model based on privacy protection, wherein the large language model is divided into a first part network and a second part network, the first part network is deployed on each client of a plurality of clients and the parameters are fixed, and the second part network is deployed in a TEE on the server side; The device is deployed on the server and includes: The receiving module is configured to receive the encrypted datasets of each client. The encrypted dataset is obtained by encrypting the embedding features and label data of the client's private samples. The embedding features are obtained by processing the private samples by the first part of the network deployed by the client. The execution module is configured to transmit the received encrypted dataset to the server's TEE in batches; within the TEE, model updates are performed. The execution module includes: The decryption unit is configured to decrypt the ciphertext of each dataset to obtain the plaintext of each dataset. The unit is configured to utilize the embedded features in the plaintext of each dataset to obtain the prediction data corresponding to each private sample through the second part of the network; The adjustment unit is configured to adjust specified parameters in the second part of the network by utilizing the differences between the predicted data and label data corresponding to each private sample.

10. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-7.

Images