A homomorphic encryption-based large model federated split privacy protection method

Abstract

The present application relates to a kind of big model federal split privacy protection methods based on homomorphic encryption, belong to big language model and privacy protection technical field.The method will big language model split into client sub-model and server sub-model;Client compresses the intermediate activation value generated by each client sub-model after completing each round of local training using compressed sensing technology;Then the activation value of compression processing is encrypted and transmitted using homomorphic encryption algorithm, thereby reducing the calculation overhead of homomorphic encryption algorithm and the communication overhead between client and server, and providing privacy protection capability to the sensitive data transmitted;Server sub-model is updated using the gradient of client, and each client fine-tunes server sub-model using local private data, realizes that the original data of client is not out of domain, and can be invisible, thereby protecting data privacy security.

Description

Technical Field

[0001] This invention relates to a privacy protection method for federated splitting of large language models based on homomorphic encryption, belonging to the field of large language models and privacy protection technology. Background Technology

[0002] In the current era of rapid development in artificial intelligence, data and models have become key drivers of technological progress. Massive amounts of data contain rich information and knowledge, enabling Large Language Models (LLMs) to achieve remarkable results in key areas such as natural language understanding, multimodal reasoning, and complex decision-making. Their performance gains depend on the quality of the supply of multimodal, cross-domain, heterogeneous data resources, and it is estimated that available public data sources will be exhausted around 2026. To overcome the performance bottlenecks of general-purpose models in vertical domains, the industry is exploring collaborative training using private data; however, this model carries the risk of sensitive information leakage. Federated learning allows for cross-departmental model training without data leaving the local machine, and can utilize distributed data sources to provide large models with massive amounts of data, alleviating the problem of data source depletion for large models.

[0003] However, due to the large parameter size of LLMs, directly training or fine-tuning LLMs in the federated learning paradigm incurs high communication overhead and poses significant challenges to the storage and computing resources of edge devices. Furthermore, during the distributed training and fine-tuning of large models based on federated learning (FL), although the server cannot directly access the training samples, attacks on FL demonstrate that it still faces threats such as inference attacks. These issues present significant challenges to improving the performance of LLMs in specific domains, hindering the development and application of large models.

[0004] Split Learning (SL), as an emerging distributed training paradigm, demonstrates great potential for training models on resource-constrained devices and overcomes the shortcomings of Functional Learning (FL). SL partitions the model between a client and a central server. The client retains the input layer (such as word embeddings) and the output layer, while the server hosts most of the intermediate layers. This design effectively reduces the computational burden on the client by offloading intensive computation to the server and exchanging only activation values / gradients rather than the original data or complete model parameters. Simultaneously, SL incorporates the advantages of parallel learning in FL, further improving training efficiency. In addition to model partitioning, SL also features periodic server-side and client-side sub-model aggregation, enabling model synchronization after multiple rounds of training, consistent with the design principles of FL. Summary of the Invention

[0005] The purpose of this invention is to provide a privacy protection method for federated splitting of large models based on homomorphic encryption, which aims to solve the technical problems of high communication overhead and storage and computing resource limitations on edge clients when directly training or fine-tuning LLMs in the federated learning paradigm.

[0006] To achieve the above objectives, this invention provides a privacy-preserving method for federated splitting of large language models based on homomorphic encryption. This method uses compressed sensing technology to compress transmitted activation values ​​and simultaneously uses the homomorphic encryption algorithm CKKS to encrypt the transmitted activation values, reducing the computational overhead of the homomorphic encryption algorithm and the communication overhead between the client and server. By splitting the model into client-side sub-models and server-side sub-models, large language models can run efficiently in a distributed environment, fully utilizing the data resources of each participant, improving the model's training speed and inference capabilities. This provides a solution for privacy protection requirements in splitting and fine-tuning of LLMs in specific domains, including the following steps:

[0007] Step 1: The server splits the large language model into client sub-models and server sub-models, and distributes the client sub-models to each client;

[0008] Step 2: After the split is completed, each client uses local private data to train the client sub-model. After each round of training, the weights generated by the client sub-model are compressed by two-dimensional discrete cosine transform, compressing the high-dimensional model parameters into one-dimensional parameters.

[0009] Step 3: After compression, the client uses a homomorphic encryption algorithm to protect the privacy of the compressed one-dimensional parameters. After each round of training, each client encrypts the weights transmitted between the client and the server and sends the encrypted weights to the server. After receiving the encrypted weights from each client, the server merges them and updates the server sub-model, thereby updating the client and completing the adjustment of the large language model using the client's local data.

[0010] The client sub-model contains the first N layers of Transformers of the large language model, and the server sub-model contains the last M layers of Transformers of the large language model.

[0011] The principle behind the large language model decomposition is to migrate the intensive computations of the large language model to the server side, thereby reducing the demand for storage and computing resources on edge clients. Simultaneously, data transmission and communication efficiency are considered to avoid excessive data transfer between the client and server.

[0012] Step 2 specifically includes the following steps:

[0013] Step 2.1: Define the variables in the training process, with the number of clients being K. For the global initial model gradient, This represents the set of trainable LoRA adapters for a pre-trained model on the server side, where... This represents the decomposition matrix of the k-th LoRA adapter. This represents the total number of trainable LoRA adapters for the pre-trained model on the server side. These are the local model weights of client i during the t-th round of communication, and the perceptual basis matrix is... The sparse orthogonal basis matrix is ​​Ψ, and the compression ratio is r;

[0014] Step 2.2: Based on the defined variables, after receiving the first N layers of the large language model's Transformer, each client uses its local private data for local training to obtain the weights generated by the client's sub-model. ;

[0015] Step 2.3: For an n×n two-dimensional matrix Two-dimensional discrete cosine transform coefficients for:

[0016]

[0017] Where G(u) and G(v) are normalization coefficients, expressed as:

[0018]

[0019] The clients will output their respective weights. Transforming it into a sparse vector s, the expression is:

[0020]

[0021] in, It is an n x n sparse orthogonal basis matrix;

[0022] Step 2.4: Compressed into one-dimensional parameters :

[0023]

[0024] in, It is an m x n perceptual basis matrix, and the compression ratio is expressed as r = m / n.

[0025] Based on step 2 above, the high-dimensional vector parameters can be compressed into a one-dimensional vector, which greatly reduces the computation and communication burden of subsequent homomorphic encryption, making it more efficient for the client to fine-tune the server-side model.

[0026] Step 3 specifically includes the following steps:

[0027] Step 3.1: The key generation center generates the public key pk, private key sk, and relinearization key rlk for the homomorphic encryption algorithm CKKS (Cheon-Kim-Kim-Song), and distributes the private key to each client;

[0028] in, CKKS.Enc() is the encryption algorithm. This represents the ciphertext obtained after encryption;

[0029] in, CKKS.Dec() is the decryption algorithm, and p is the plaintext obtained after decryption;

[0030] in, This is homomorphic addition. This is homomorphic multiplication;

[0031] Step 3.2: Based on the obtained one-dimensional parameters One-dimensional parameters are encrypted using the CKKS.Enc() algorithm. get:

[0032]

[0033] The weight of the encrypted ciphertext and client data real tags Uploaded to the server via wired or wireless channels;

[0034] Step 3.3: The server receives the encrypted ciphertext weights from all clients. Using its own model parameters LoRA adapter for server in round ti Perform forward propagation and calculate the predicted value. :

[0035]

[0036] in, Indicates the given model parameters and trainable LoRA adapter set Input data The mapping relationship between the predicted value and the actual value, and based on the predicted value and real labels Calculate the loss function L:

[0037]

[0038] Where D is the model batch size. The loss function is then used, and the server performs backpropagation. Weights of the generated server terminal model ;

[0039] Step 3.4: The server will It is sent to each client, and the client receives it. Then, the plaintext information is obtained by using the CKKS.Dec() decryption operation. :

[0040]

[0041] Step 3.5: Each client receives the plaintext. Then, the complete weight values ​​are reconstructed using the two-dimensional discrete cosine inverse transform:

[0042]

[0043] in, It is a two-dimensional discrete cosine inverse transform matrix, expressed as:

[0044]

[0045] Based on complete weight values Update the weights of the client sub-model and proceed with the next round of training until the model converges.

[0046] The beneficial effects of this invention are:

[0047] (1) The present invention proposes a privacy-preserving method for large model federated splitting based on homomorphic encryption. By splitting LLMs into client sub-models and server sub-models, the intensive computation is migrated to the server side, allowing the server side with powerful computing capabilities to host most of the intermediate layers. The client only has the first N layers of the large language model's Transformer, effectively solving the storage and computing resource limitations of edge clients. At the same time, by utilizing the distributed parallel training mechanism, the server-side model is fine-tuned efficiently using private data.

[0048] (2) In the process of training large-scale model tasks, the present invention compresses the intermediate activation values ​​generated by the edge client sub-model through compressed sensing technology, compresses the high-dimensional model parameters into a one-dimensional vector, and avoids the reduction of training efficiency due to insufficient client resources or communication delay by reducing the amount of data that needs to be transmitted between the client and the server.

[0049] (3) This invention employs the homomorphic encryption algorithm CKKS to counter inference attacks by malicious attackers in the splitting and fine-tuning of large language models. The CKKS encryption algorithm is used to encrypt and transmit the compressed activation values ​​from the client side, while the server side updates the sub-model using homomorphic computation. This securely and efficiently protects the private data of edge clients, greatly enhancing privacy protection capabilities in splitting and fine-tuning.

[0050] (4) In the training of large-scale model tasks, especially when facing privacy-sensitive data, this invention can solve the problems of data privacy and data source scarcity through federated split learning, while avoiding the problem of insufficient client resources. It provides a new idea for further integration and innovation of large language model splitting and fine-tuning with privacy protection. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the process of this invention;

[0052] Figure 2 This is a schematic diagram of the model splitting and client-side fine-tuning of the server model in this invention. Detailed Implementation

[0053] To make the uses, technical solutions, and advantages of this invention clearer and easier to understand, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0054] The illustrations and specific parameter values ​​provided in the following examples are mainly for illustrating the basic concept of the present invention and for simulation verification of the present invention. In specific application environments, appropriate adjustments can be made according to the actual scenario and requirements.

[0055] Example 1: A privacy-preserving method for large-scale federated splitting based on homomorphic encryption, such as... Figure 1 As shown, it includes:

[0056] Step 1: The server splits the large language model into client sub-models and server sub-models, and distributes the client sub-models to each client.

[0057] The client sub-model contains the first N layers of Transformers of the large language model, and the server sub-model contains the last M layers of Transformers of the large language model.

[0058] Specifically, in this embodiment, the server first splits the LLMs into client sub-models and server sub-models. The client sub-model contains the first N layers, and the server sub-model contains the last M layers. The client sub-model is then distributed to each client. This embodiment uses GPT2-s splitting as an example: the client sub-model contains the first 3 Transformer layers of GPT2-s, and the server sub-model contains the last 9 Transformer layers of GPT2-s. Three edge clients are used to participate in the fine-tuning of the LLMs model.

[0059] Step 2: After the split is completed, each client uses local private data to train the client sub-model. After each round of training, the weights generated by the client sub-model are compressed by two-dimensional discrete cosine transform, compressing the high-dimensional model parameters into one-dimensional parameters.

[0060] Step 2.1: Define the variables in the training process, with the number of clients being K. For the global initial model gradient, This represents the set of trainable LoRA adapters for a pre-trained model on the server side, where... This represents the decomposition matrix of the k-th LoRA adapter. This represents the total number of trainable LoRA adapters for the pre-trained model on the server side. These are the local model weights of client i during the t-th round of communication, and the perceptual basis matrix is... The sparse orthogonal basis matrix is ​​Ψ, and the compression ratio is r;

[0061] Step 2.2: Based on the defined variables, after receiving the first N layers of the large language model's Transformer, each client uses its local private data for local training to obtain the weights generated by the client's sub-model. ;

[0062] Step 2.3: For an n×n two-dimensional matrix The two-dimensional discrete cosine transform coefficients Ψ(u,v) are:

[0063]

[0064] Where G(u) and G(v) are normalization coefficients, expressed as:

[0065]

[0066] The clients will output their respective weights. Transforming it into a sparse vector s, the expression is:

[0067]

[0068] in, It is an n x n sparse orthogonal basis matrix;

[0069] Step 2.4: Compressed into one-dimensional parameters :

[0070]

[0071] in, It is an m x n perceptual basis matrix, and the compression ratio is expressed as r = m / n.

[0072] Specifically, in this embodiment, client 1 uses local private data to train the client sub-model and generate weight values. Using a sparse orthogonal basis matrix Ψ, the weight values ​​are... The vector is transformed into a sparse vector s, and then the sparsified weight values ​​are processed using a compressed sensing algorithm. Compression processing is performed to obtain compressed one-dimensional parameters. .

[0073] Furthermore, let the weight values ​​be... After compression, one-dimensional parameters are output. .

[0074] Step 3: After compression, the client uses a homomorphic encryption algorithm to protect the privacy of the compressed one-dimensional parameters. After each round of training, each client encrypts the weights transmitted between the client and the server and sends the encrypted weights to the server. After receiving the encrypted weights from each client, the server merges them and updates the server sub-model, thereby updating the client and completing the adjustment of the large language model using the client's local data.

[0075] Step 3.1: The key generation center generates the public key pk, private key sk, and relinearization key rlk for the homomorphic encryption algorithm CKKS, and distributes the private key to each client;

[0076] in, CKKS.Enc() is the encryption algorithm. This represents the ciphertext obtained after encryption;

[0077] in, CKKS.Dec() is the decryption algorithm, and p is the plaintext obtained after decryption;

[0078] in, This is homomorphic addition. This is homomorphic multiplication;

[0079] Step 3.2: Based on the obtained one-dimensional parameters One-dimensional parameters are encrypted using the CKKS.Enc() algorithm. get:

[0080]

[0081] The weight of the encrypted ciphertext and client data real tags Uploaded to the server via wired or wireless channels;

[0082] Step 3.3: The server receives the encrypted ciphertext weights from all clients. Using its own model parameters LoRA adapter for server in round ti Perform forward propagation and calculate the predicted value. :

[0083]

[0084] in, Indicates the given model parameters and trainable LoRA adapter set Input data The mapping relationship between the predicted value and the actual value, and based on the predicted value and real labels Calculate the loss function L:

[0085] , )

[0086] Where D is the model batch size. The loss function is then used, and the server performs backpropagation. Weights of the generated server terminal model ;

[0087] Step 3.4: The server will It is sent to each client, and the client receives it. Then, the plaintext information is obtained by using the CKKS.Dec() decryption operation. :

[0088]

[0089] Step 3.5: Each client receives the plaintext. Then, the complete weight values ​​are reconstructed using the two-dimensional discrete cosine inverse transform:

[0090]

[0091] in, It is a two-dimensional discrete cosine inverse transform matrix, expressed as:

[0092]

[0093] Based on complete weight values Update the weights of the client sub-model and proceed with the next round of training until the model converges.

[0094] Specifically, in this embodiment, client 1 uses the public key pk generated by the key generation center and the encryption algorithm CKKS.Enc() to compress the one-dimensional parameters. Encryption is performed to obtain ciphertext:

[0095]

[0096] Will The encrypted data is sent to the server via wired or wireless channels. for:

[0097] .

[0098] Furthermore, the remaining clients 2 and 3 execute steps 2 and 3 sequentially, and the server receives... Then, the CKKS.HomAdd() algorithm is executed to obtain the aggregation result. :

[0099]

[0100] Furthermore, through aggregation results Update the service terminal model and obtain .

[0101] Furthermore, Send to client 1, client 2, and client 3;

[0102] Among them, the weight of client 2 for:

[0103]

[0104] Among them, the weight of client 3 for:

[0105]

[0106] .

[0107] Furthermore, each client receives Decrypt using the private key sk Then, the complete weight values ​​are reconstructed using two-dimensional discrete cosine transform. The client then performs the next round of training based on the complete weight values ​​until the model converges.

[0108] Specifically, in this embodiment, the complete weight value for:

[0109] .

[0110] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A privacy-preserving method for large-scale federated splitting based on homomorphic encryption, characterized in that, Includes the following steps: Step 1: The server splits the large language model into client sub-models and server sub-models, and distributes the client sub-models to each client; Step 2: After the split is completed, each client uses local private data to train the client sub-model. After each round of training, the weights generated by the client sub-model are compressed by two-dimensional discrete cosine transform, compressing the high-dimensional model parameters into one-dimensional parameters. Step 3: After compression, the client uses a homomorphic encryption algorithm to protect the privacy of the compressed one-dimensional parameters. After each round of training, each client encrypts the weights transmitted between the client and the server and sends the encrypted weights to the server. After receiving the encrypted weights from each client, the server merges them and updates the server sub-model, thereby updating the client and completing the adjustment of the large language model using the client's local data.

2. The privacy protection method for large-scale federated splitting based on homomorphic encryption according to claim 1, characterized in that, The client sub-model contains the first N layers of Transformers of the large language model, and the server sub-model contains the last M layers of Transformers of the large language model.

3. The privacy protection method for large-scale federated splitting based on homomorphic encryption according to claim 1, characterized in that, Step 2 specifically includes the following steps: Step 2.1: Define the variables in the training process, with the number of clients being K. For the global initial model gradient, } represents the set of trainable LoRA adapters for the pre-trained model on the server side, where, This represents the decomposition matrix of the k-th LoRA adapter. This represents the total number of trainable LoRA adapters for the pre-trained model on the server side. These are the local model weights of client i during the t-th round of communication, and the perceptual basis matrix is... The sparse orthogonal basis matrix is ​​Ψ, and the compression ratio is r; Step 2.2: Based on the defined variables, after receiving the first N layers of the large language model's Transformer, each client uses its local private data for local training to obtain the weights generated by the client's sub-model. ; Step 2.3: For an n×n two-dimensional matrix (x, y), where x = 0, 1, ..., n-1, y = 0, 1, ..., n-1, and the two-dimensional discrete cosine transform coefficients Ψ(u, v) are: ； Where G(u) and G(v) are normalization coefficients, expressed as: ； ； The clients will output their respective weights. Transforming it into a sparse vector s, the expression is: ； Among them, Ψ It is an n x n sparse orthogonal basis matrix; Step 2.4: Compressed into one-dimensional parameters : ； in, It is an m x n perceptual basis matrix, and the compression ratio is expressed as r = m / n.

4. The privacy protection method for large-scale federated splitting based on homomorphic encryption according to claim 1, characterized in that, Step 3 specifically includes the following steps: Step 3.1: The key generation center generates the public key pk, private key sk, and relinearization key rlk for the homomorphic encryption algorithm CKKS, and distributes the private key to each client; in, =CKKS.Enc(pk, p), where CKKS.Enc() is the encryption algorithm. This represents the ciphertext obtained after encryption; Where p=CKKS.Dec(sk, ), CKKS.Dec() is the decryption algorithm, and p is the plaintext obtained after decryption; Among them, CKKS.HomAdd( ) represents homomorphic addition, CKKS.HomMul( This is homomorphic multiplication; Step 3.2: Based on the obtained one-dimensional parameters One-dimensional parameters are encrypted using the CKKS.Enc() algorithm. get: ； The weight of the encrypted ciphertext and client data real tags Uploaded to the server via wired or wireless channels; Step 3.3: The server receives the encrypted ciphertext weights from all clients. Using its own model parameters LoRA adapter for server in round t-1 Perform forward propagation and calculate the predicted value. : ； in, Indicates the given model parameters and trainable LoRA adapter set Input data The mapping relationship between the predicted value and the actual value, and based on the predicted value and real labels Calculate the loss function L: ； Where D is the model batch size. The loss function is used, and then the server performs backpropagation to generate the weights of the server-side sub-model. ; Step 3.4: The server will It is sent to each client, and the client receives it. Then, the plaintext information is obtained by using the CKKS.Dec() decryption operation. : ； Step 3.5: Each client receives the plaintext. Then, the complete weight values ​​are reconstructed using the two-dimensional discrete cosine inverse transform: ； in, It is a two-dimensional discrete cosine inverse transform matrix, expressed as: ； Based on complete weight values Update the weights of the client sub-model and proceed with the next round of training until the model converges.

Images