Private large model fine-tuning training method and device based on split learning

Abstract

The present application relates to a method and apparatus for fine-tuning a private large model based on split learning, wherein the method comprises: splitting a pre-trained large model into a head model segment, an intermediate model segment, and a tail model segment; splitting the original data into private data and non-private data; collaboratively training the head model segment and the tail model segment in the client, and the first intermediate model segment and the second intermediate model segment in the server based on the private data and the non-private data, to obtain a target large model; wherein, during the collaborative training process, the private intermediate activations related to the private data transmitted between the client and the server are homomorphically encrypted; and the target large model outputs a document response on a specific topic. Through this application, the problem in the related art of needing to transmit intermediate activations to the server and the risk of inferring or recovering the original data through model inversion attacks is solved; the protection of private data is achieved, and the risk of private data leakage is reduced.

Description

Technical Field

[0001] The present application relates to the field of large model technology, and in particular to a method and device for fine-tuning and training a privatized large model based on split learning. Background Art

[0002] With the development and rise of large model technology, many small and medium-sized enterprises want to use their private data sets to train and deploy their own private large models, but they face the problem of insufficient local computing power. To solve this problem, a split learning strategy can be used. Specifically, by splitting the model layers, some layers are calculated on the local machine, while the intermediate calculation results and the calculation tasks of the other layers are sent to the remote server, thereby reducing the computing and storage burden of the local device and achieving collaborative training of large models between local and remote. The problem with this method is that although the current split learning method avoids the direct transmission of original data and partially protects privacy, the client still needs to transmit intermediate activations to the server, which poses the risk of inferring or recovering the original data through model inversion attacks.

[0003] Regarding the need to transmit intermediate activation amounts to the server in related technologies, there is a risk of inferring or recovering the original data through model inversion attacks. No effective solution has been proposed so far. Summary of the Invention

[0004] In this embodiment, a private large model fine-tuning training method and device based on split learning are provided to solve the problem in related technologies that intermediate activation amounts need to be transmitted to the server, and there is a risk of inferring or recovering the original data through model inversion attacks.

[0005] First, in this embodiment, a private large model fine-tuning training method based on split learning is provided, including:

[0006] Splitting the pre-trained large model into a head model segment, an intermediate model segment, and a tail model segment; deploying the head model segment and the tail model segment on the client; deploying the intermediate model segment on the server; the intermediate model segment includes a first intermediate model segment and a second intermediate model segment;

[0007] Splitting original data into private data and non-private data; the original data is document data;

[0008] Based on the private data and the non-private data, the head model segment and the tail model segment in the client and the first intermediate model segment and the second intermediate model segment in the server are collaboratively trained to obtain a target large model; wherein, during the collaborative training process, the private intermediate activations related to the private data transmitted between the client and the server are homomorphically encrypted; the first intermediate model segment is used to process the non-private data; the second intermediate model segment is used to process the private data; and the target large model outputs a document response on a specific topic.

[0009] In some embodiments, the pre-trained large model is split into a head model segment, a middle model segment, and a tail model segment, including:

[0010] The pre-trained large model is split according to the hierarchical structure of the Transformer block to obtain the head model segment, the intermediate model segment and the tail model segment; wherein the head model segment and the intermediate model segment respectively include a small number of blocks; and the intermediate model segment includes a large number of blocks.

[0011] In some embodiments, the method further comprises:

[0012] The nonlinear layer in the first intermediate model segment is replaced by a preset polynomial or point activation to obtain the second intermediate model segment.

[0013] In some embodiments, based on the private data and the non-private data, collaboratively training the head model segment and the tail model segment in the client and the intermediate model segment in the server to obtain a target large model includes:

[0014] forward propagating the private data and the non-private data through the header model segment in the client to obtain a first private intermediate activation and a first non-private intermediate activation; homomorphically encrypting the first private intermediate activation to obtain ciphertext data; transmitting the ciphertext data and the first non-private intermediate activation to the server; the private intermediate activations include the first private intermediate activation and the second private intermediate activation;

[0015] forwarding the first non-private intermediate activation amount and the ciphertext data through the first intermediate model segment and the second intermediate model segment in the server, respectively, and returning the obtained second non-private intermediate activation amount and second private intermediate activation amount to the client;

[0016] Performing forward propagation and aggregation on the second private intermediate activation and the second non-private intermediate activation through the tail model segment in the client to obtain an intermediate gradient; performing backward propagation in the tail model segment using the intermediate gradient and updating the model parameters of the tail model segment; and transmitting the intermediate gradient to the server;

[0017] Backward propagating the intermediate gradient through the first intermediate model segment and the second intermediate model segment in the server, transmitting the generated reverse intermediate gradient to the client, and updating the model parameters of the first intermediate model segment and the second intermediate model segment;

[0018] updating the model parameters of the head model segment in the client based on the reverse intermediate gradient;

[0019] When the large model meets the convergence condition or reaches the number of training rounds, the target large model is obtained.

[0020] In some embodiments, forwarding the first non-private intermediate activation amount and the ciphertext data through the first intermediate model segment and the second intermediate model segment in the server, and returning the obtained second non-private intermediate activation amount and second private intermediate activation amount to the client, includes:

[0021] forward transmitting the first non-private intermediate activation amount through the first intermediate model segment in the server to obtain a second non-private intermediate activation amount;

[0022] forward transmitting the ciphertext data through the second intermediate model segment in the server to obtain a second privacy intermediate activation;

[0023] The second non-private intermediate activation amount and the second private intermediate activation amount are transmitted to the client.

[0024] In some embodiments, forward propagating and aggregating the second private intermediate activation and the second non-private intermediate activation through the tail model segment in the client to obtain an intermediate gradient includes:

[0025] Decrypting the second private intermediate activation amount to obtain a third private intermediate activation amount;

[0026] In the tail model segment of the client, forward propagation is performed on the third private intermediate activation and the second non-private intermediate activation to obtain a predicted value and a corresponding loss function value;

[0027] Perform back propagation according to the loss function value to obtain the gradient;

[0028] The gradient is polymerized to obtain the intermediate gradient.

[0029] In some embodiments, back-propagating the intermediate gradient through the first intermediate model segment and the second intermediate model segment in the server, transmitting the generated reverse intermediate gradient to the client, and updating the model parameters of the first intermediate model segment and the second intermediate model segment includes:

[0030] In the first intermediate model segment in the server, backpropagating the intermediate gradient to generate a reverse intermediate gradient; and updating the model parameters of the first intermediate model segment;

[0031] transmitting the reverse intermediate gradient to the client;

[0032] When the first intermediate model segment completes a preset number of rounds, the model parameters of the first intermediate model segment are synchronized to the second intermediate model segment to update the model parameters of the second intermediate model segment.

[0033] In a second aspect, a privatized large model fine-tuning training device based on split learning is provided in this embodiment, comprising: a first fine-tuning module, a second fine-tuning module, and a training module;

[0034] The first fine-tuning module is used to split the pre-trained large model into a head model segment, an intermediate model segment, and a tail model segment; deploy the head model segment and the tail model segment on the client; deploy the intermediate model segment on the server; the intermediate model segment includes a first intermediate model segment and a second intermediate model segment;

[0035] The second fine-tuning module is configured to split the original data into private data and non-private data; the original data is document data;

[0036] The training module is used to collaboratively train the head model segment and the tail model segment in the client, and the first intermediate model segment and the second intermediate model segment in the server based on the private data and the non-private data to obtain a target large model; wherein, during the collaborative training process, the private intermediate activations related to the private data transmitted between the client and the server are homomorphically encrypted; the first intermediate model segment is used to process the non-private data; the second intermediate model segment is used to process the private data; and the target large model outputs a document response on a specific topic.

[0037] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the privatized large model fine-tuning training method based on split learning described in the first aspect is implemented.

[0038] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored. When the program is executed by a processor, the privatized large model fine-tuning training method based on split learning described in the first aspect above is implemented.

[0039] Compared with the related art, the privatized large model fine-tuning training method and device based on split learning provided in this embodiment is achieved by splitting the pre-trained large model into a head model segment, an intermediate model segment and a tail model segment; deploying the head model segment and the tail model segment on the client; deploying the intermediate model segment on the server; the intermediate model segment includes a first intermediate model segment and a second intermediate model segment; splitting the original data into private data and non-private data; the original data is document data; based on the private data and non-private data, the head model segment and the tail model segment in the client, and the first intermediate model segment and the second intermediate model segment in the server are collaboratively trained to obtain a target large model; wherein the target large model outputs a document response on a specific topic; wherein During the collaborative training process, the privacy intermediate activations related to the private data transmitted between the client and the server are homomorphically encrypted, so that the private data in the client is transmitted to the second intermediate model segment of the server under homomorphic encryption, so that the first intermediate model segment processes the non-private data, thereby protecting the private data and reducing the risk of privacy data leakage; and the non-private data in the client is directly transmitted to the first intermediate model segment of the server for related propagation, so that the second intermediate model segment processes the private data, so that the training of the second intermediate model segment can feedback and optimize the training effect of the first intermediate model segment; thereby solving the problem in related technologies that the intermediate activations need to be transmitted to the server, and there is a risk of inferring or recovering the original data through model inversion attacks.

[0040] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 This is a hardware structure block diagram of a terminal device for a privatized large model fine-tuning training method based on split learning provided in one embodiment of the present application;

[0043] Figure 2 This is a flowchart of a privatized large model fine-tuning training method based on split learning provided in one embodiment of the present application;

[0044] Figure 3is a flowchart of step S230;

[0045] Figure 4 This is a schematic diagram of the training process of a privatized large model fine-tuning training method based on split learning provided in one embodiment of the present application;

[0046] Figure 5 This is a structural block diagram of a privatized large model fine-tuning training device based on split learning provided in one embodiment of the present application.

[0047] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 210, first fine-tuning module; 220, second fine-tuning module; 230, training module. DETAILED DESCRIPTION

[0048] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0049] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "an", "a", "the", "these" and the like in this application do not indicate quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Generally, the character " / " indicates that the related objects are in an "or" relationship. The terms "first," "second," "third," etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 This is a hardware block diagram of the terminal of the private large model fine-tuning training method based on split learning in this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0051] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the privatized large model fine-tuning training method based on split learning in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0052] The transmission device 106 is used to receive or send data via a network. The network may include a wireless network provided by the terminal's telecommunications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0053] In this embodiment, a private large model fine-tuning training method based on split learning is provided. Figure 2 This is a flowchart of the privatized large model fine-tuning training method based on split learning in this embodiment. Figure 2 As shown, the process includes the following steps:

[0054] Step S210: Split the pre-trained large model into a head model segment, an intermediate model segment, and a tail model segment; deploy the head model segment and the tail model segment on the client; deploy the intermediate model segment on the server; the intermediate model segment includes a first intermediate model segment and a second intermediate model segment;

[0055] Step S220: splitting the original data into private data and non-private data; the original data is document data;

[0056] Step S230: Based on the private data and the non-private data, the head model segment and the tail model segment in the client and the first intermediate model segment and the second intermediate model segment in the server are collaboratively trained to obtain a target large model; wherein, during the collaborative training process, the private intermediate activations related to the private data transmitted between the client and the server are homomorphically encrypted; the first intermediate model segment is used for non-processing private data; the second intermediate model segment is used for processing private data; and the target large model outputs a document response on a specific topic.

[0057] Specifically, a large model refers to a machine learning model with large-scale parameters and complex computing structure. It is pre-trained and can be obtained from a third-party database or local system. It includes but is not limited to LLaMA, Palm, GLM, BLOOM, and GPT, etc., and there is no restriction on this. Among them, the structure of the large model includes layers and blocks; a layer is a neural network component that converts input data into output data. Each layer performs certain operations on the input data, such as linear transformation, nonlinear activation function, etc., to generate output data. A block is a collection of a series of layers, which usually share the same parameters and perform a specific task together.

[0058] In this embodiment, it can be considered as splitting by layers or blocks to obtain a head model segment, an intermediate model segment, and a tail model segment. The head model segment and the tail model segment are deployed on the client; the intermediate model segment is deployed on the server; the intermediate model segment includes a first intermediate model segment and a second intermediate model segment; the model parameters and network structure of the first intermediate model segment and the second intermediate model segment are the same; but the operators are inconsistent. By splitting the model, the head model segment and the tail model segment are calculated on the local client, and the calculation task of the intermediate model segment is sent to the remote server, thereby reducing the computing and storage burden of the local client and realizing the collaborative completion of the training of the large model between the local and remote clients. Among them, the client can be a mobile terminal, a fixed terminal or a portable terminal, such as a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device or any combination thereof, including accessories and peripherals of these devices or any combination thereof.

[0059] Raw data refers to the user's private data; private data and non-private data are determined by the application scenario. For example, in the medical field, if you agree to disclose your health data, then the health data is non-private data; if you do not agree to disclose your health data, then the health data is non-private data. Another example: in the transportation field, if you agree to disclose your travel data, then the travel data is non-private data; if you do not agree to disclose your travel data, then the travel data is non-private data.

[0060] The collaborative training process involves collaboratively training the head and tail model segments in the client, and the first and second intermediate model segments in the server, based on private and non-private data, using relevant training methods to obtain the target large model. This can be considered as training the head, first, second, and tail model segments through forward and backward propagation of private and non-private data, updating the model parameters of each model segment, and obtaining the target large model when convergence conditions are met or the number of training rounds is reached. Because the private intermediate activations related to the private data transmitted between the client and server are homomorphically encrypted during the collaborative training process, private data can be protected and the risk of private data leakage can be reduced.

[0061] Through the above steps, the pre-trained large model is split into a head model segment, an intermediate model segment and a tail model segment; the head model segment and the tail model segment are deployed on the client; the intermediate model segment is deployed on the server; the intermediate model segment includes a first intermediate model segment and a second intermediate model segment; the original data is split into private data and non-private data; based on the private data and non-private data, the head model segment and the tail model segment in the client, and the first intermediate model segment and the second intermediate model segment in the server are collaboratively trained to obtain the target large model; wherein, during the collaborative training process, the private intermediate activations related to the private data transmitted between the client and the server are homomorphically encrypted; the first intermediate model segment is used to process non-private data; the second intermediate model segment is used to process private data; this solves the problem in related technologies that intermediate activations need to be transmitted to the server, and there is a risk of inferring or recovering the original data through model inversion attacks; homomorphic encryption is used in conjunction with the second intermediate model segment to protect private data and reduce the risk of private data leakage; and the training of the second intermediate model segment can feedback and optimize the training effect of the first intermediate model segment.

[0062] The following is a detailed description of each of the above steps:

[0063] It should be noted that the target macro model can be applied to fields such as medicine, transportation, and finance, without limitation. Specifically, raw data from the medical, transportation, or financial fields is used to fine-tune and train the corresponding macro model based on the aforementioned method embodiment to obtain a target macro model for the corresponding field. When applying the target macro model, simply input document data into the target macro model to output document responses on a specific topic.

[0064] In some embodiments, step S210 of splitting the pre-trained large model into a head model segment, a middle model segment, and a tail model segment includes the following steps:

[0065] The pre-trained large model is split according to the hierarchical structure of the Transformer block to obtain the head model segment, the intermediate model segment and the tail model segment; among them, the head model segment and the intermediate model segment each include a small number of blocks; the intermediate model segment includes a large number of blocks.

[0066] Current split learning strategies are primarily designed for optimizing traditional deep learning models and have not yet been applied to fine-tuning large Transformer architecture models. In this embodiment, the large model is a Transformer architecture model, which is split according to the hierarchical structure of the Transformer blocks, using a U-shaped split method to obtain the head model segment, the middle model segment, and the tail model segment, thereby achieving fine-tuning suitable for large Transformer architecture models.

[0067] The head model segment and the intermediate model segment each contain a small number of blocks, while the intermediate model segment contains a large number of blocks. Specifically, the ratio of blocks in the head model segment, the intermediate model segment, and the tail model segment can be determined based on the client's computing power. It can be assumed that the higher the client's computing power, the greater the number of blocks in the head model segment and the tail model segment.

[0068] In some embodiments, the privatized large model fine-tuning training method based on split learning further includes the following steps:

[0069] The nonlinear layer in the first intermediate model segment is replaced by a preset polynomial or point activation to obtain a second intermediate model segment.

[0070] Specifically, homomorphic encryption technology is used in large-scale model inference: Due to the complex operation of large Transformer-based models, which contain a large number of nonlinear layers (such as activation functions, Softmax layers, and LayerNorm layers), homomorphic encryption only supports addition and multiplication, making it difficult to directly apply to pre-trained large models. Existing technologies apply homomorphic encryption to large-scale model inference in the Transformer architecture to protect privacy. To address the difficulty of performing nonlinear operator operations on homomorphically encrypted ciphertext data, some work approximates the nonlinear operators in the forward calculation process of large-scale model inference with polynomial linear operators or estimation networks with addition and multiplication. At the same time, the user device is included in the inference, and the server passes the ciphertext input to the user for local max calculation, thereby performing the forward propagation operation of large-scale model inference on the homomorphically encrypted ciphertext data. This approach still requires additional training to ensure the accuracy of the model.

[0071] In this embodiment, the network model of the intermediate model segment is copied into a first intermediate model segment and a second intermediate model segment with identical model parameters but inconsistent operators. The nonlinear layer in the first intermediate model segment is replaced with polynomial or point activation. Point activation involves activating individual elements rather than the entire vector of the nonlinear layer. Polynomials are operators that homomorphic encryption can apply to. These methods enable homomorphic encryption to be applied to large-model inference, ensuring model training accuracy without increasing the amount of training required.

[0072] In some of these embodiments, Figure 3 As shown, step S230 includes the following steps:

[0073] Step S231: forward propagating the private data and the non-private data through the header model segment in the client to obtain a first private intermediate activation and a first non-private intermediate activation; homomorphically encrypting the first private intermediate activation to obtain ciphertext data; transmitting the ciphertext data and the first non-private intermediate activation to the server; the private intermediate activations include the first private intermediate activation and the second private intermediate activation;

[0074] Step S232: forward the first non-private intermediate activation amount and the ciphertext data through the first intermediate model segment and the second intermediate model segment in the server, and return the obtained second non-private intermediate activation amount and the second private intermediate activation amount to the client;

[0075] Step S233: forward propagating and aggregating the second private intermediate activation and the second non-private intermediate activation through the tail model segment in the client to obtain an intermediate gradient; performing backward propagation in the tail model segment using the intermediate gradient and updating the model parameters of the tail model segment; and transmitting the intermediate gradient to the server;

[0076] Step S234: back-propagate the intermediate gradient through the first intermediate model segment and the second intermediate model segment in the server, transmit the generated reverse intermediate gradient to the client, and update the model parameters of the first intermediate model segment and the second intermediate model segment;

[0077] Step S235 , updating the model parameters of the head model segment in the client based on the reverse intermediate gradient;

[0078] Step S236: When the large model meets the convergence condition or reaches the number of training rounds, the target large model is obtained.

[0079] Specifically, before executing step S230, the pre-trained large model is first split into a head model segment, an intermediate model segment, and a tail model segment. The head model segment and the tail model segment contain a small number of model blocks and are stored on the client, while the intermediate model segment contains a large number of model blocks and is stored on the server. Next, the raw data input for a training batch is divided into private data and non-private data.

[0080] Next, step S230 is executed. The process is as follows: the batch of raw data is collectively passed through the client's header model segment for forward propagation calculations. The intermediate activations associated with the private data in the raw data are homomorphically encrypted into ciphertext data. The intermediate activations associated with the non-private data in the raw data are not homomorphically encrypted and remain plaintext data. Both the ciphertext and plaintext data are transmitted to the server. The server is configured with intermediate model segments: the first intermediate model segment and the second intermediate model segment. The first intermediate model segment handles forward and backward propagation of the plaintext data; the second intermediate model segment handles forward and backward propagation of the ciphertext data. Subsequently, the forward propagation results of both the ciphertext and plaintext data are transmitted to the client. After decrypting the ciphertext data, the client uses the two parts of data to complete the forward propagation of the tail model segment, ultimately generating two gradients (one generated by the private data and the other generated by the non-private data). After aggregating the two gradients, the aggregated gradient is used to perform backpropagation on the tail model segment and update its parameters. The reverse intermediate gradient generated by the non-private data is then transmitted back to the server to complete the subsequent backpropagation and parameter update of the intermediate model end and the head model segment. After a certain number of rounds of training, the model parameters of the two intermediate model segments on the server are synchronized, and additional training is performed to make the sub-network used for ciphertext forward propagation applicable to the ciphertext environment.

[0081] This embodiment encrypts the intermediate activations associated with private data transmitted to the server (including the first and second private intermediate activations). Furthermore, the gradients corresponding to the private data are only used to update the tail model segment and do not need to be transmitted to the server. This approach protects private data from being leaked during large model training. Furthermore, by leveraging the correctness of the forward propagation of homomorphically encrypted ciphertext, non-sensitive data is used to update the intermediate and head model segments, thereby ensuring the effectiveness and accuracy of the large model.

[0082] The following combination Figure 4 Step S230 is described in detail:

[0083] Among them, the large model is divided into the head model segment W h , the first intermediate model segment W S , the second intermediate model segment W S ′ and tail model segment W t . Original data D.

[0084] Step S231 involves forward propagation and homomorphic encryption of the local header model segment. This involves forward propagating the private data and non-private data through the client's header model segment to obtain a first private intermediate activation and a first non-private intermediate activation. Homomorphically encrypting the first private intermediate activation yields ciphertext data. The ciphertext data and the first non-private intermediate activation are transmitted to the server, including the following steps:

[0085] For the kth batch of original data D in the original data D k Perform forward propagation of the head model segment to obtain the intermediate activation amount a of this batch h ; Original data D k Including private data and non-private data, the corresponding intermediate activation a h , which is divided into the first privacy intermediate activation corresponding to the private data The first non-private intermediate activation corresponding to the non-private data

[0086] Generate private keys k and public key p k ;

[0087] Use private keys k The first privacy intermediate activation Perform homomorphic encryption to obtain encrypted ciphertext data

[0088] The encrypted ciphertext data of the private data and the first non-private intermediate activation of the non-private data Transmit to the server.

[0089] Through the above process, privacy data can be protected and the risk of privacy data leakage can be reduced.

[0090] The process of forwarding the first non-private intermediate activation amount and the ciphertext data through the first intermediate model segment and the second intermediate model segment in the server, and returning the obtained second non-private intermediate activation amount and the second private intermediate activation amount to the client in step S232 includes the following steps:

[0091] forward transmitting the first non-private intermediate activation through the first intermediate model segment in the server to obtain a second non-private intermediate activation;

[0092] The encrypted data is forwarded through the second intermediate model segment in the server to obtain a second private intermediate activation;

[0093] The second non-private intermediate activation amount and the second private intermediate activation amount are transmitted to the client.

[0094] Specifically, this process involves processing and returning data on the server side. The specific process is as follows:

[0095] Since the ciphertext data after homomorphic encryption can only be added and multiplied, it cannot pass the nonlinear layers (ReLU, Softmax, LayerNorm layers) in the Transformer architecture model. Therefore, a second intermediate model segment is set up, which uses polynomials to replace the nonlinear layers in the first intermediate model segment; for example: ReLU(x i ) is approximately f3(x i )=0.47+0.50*x i +0.09*x i 2 -1.7e -10 *x i 3 ; Softmax(x i ) is approximately S(x i )=x i *T(∑ j ReLU(((x j ) / 2+1) 3 )), the LayerNorm layer is approximated as γ+β, where γ, is a learnable parameter and is the Hadamard product.

[0096] Use the client's head model segment to forward propagate the non-private data, and get the first non-private intermediate activation Pass it through the server's first intermediate model segment W s The forward propagation of the second non-private intermediate activation is obtained

[0097] Use the client's header model segment to forward propagate the private data and homomorphically encrypt it to obtain the ciphertext data. Pass it through the second intermediate model segment W on the server side s ′’s forward propagation to obtain the secondary privacy intermediate activation This step can be performed in parallel with the previous step;

[0098] The second non-private intermediate activation (plaintext data) and ciphertext data Transmit to the client.

[0099] Through the above steps, a separately set second intermediate model segment is used to ensure the forward transmission of the encrypted data; and combined with the first intermediate model segment, they work together to reduce the error caused by the approximate method in the second intermediate model segment, thereby avoiding the resulting poor training effect.

[0100] The step S233 of forward propagating and aggregating the second private intermediate activation and the second non-private intermediate activation through the tail model segment in the client to obtain the intermediate gradient includes the following steps:

[0101] Decrypt the second privacy intermediate activation amount to obtain the third privacy intermediate activation amount;

[0102] In the client's tail model segment, forward propagation is performed on the third private intermediate activation and the second non-private intermediate activation to obtain the predicted value and the corresponding loss function value;

[0103] Back propagation is performed according to the loss function value to obtain the gradient;

[0104] The gradients were aggregated to obtain an intermediate gradient.

[0105] Specifically, this process includes client decryption, forward propagation, backward propagation of the tail model segment, and gradient aggregation. The specific process is:

[0106] The client receives the second non-private intermediate activation from the server and ciphertext data Using the public key p k Ciphertext data Decrypt and get the third privacy intermediate activation amount

[0107] In the client's tail model segment, the second non-private intermediate activation and the activation amount between the third privacy Perform forward propagation to obtain the predicted value and predicted values And generate two parts of the loss function value:

[0108] Back propagation is performed based on the loss function values ​​of these two parts to obtain the gradient z of the last layer p and the gradient z c , aggregate the gradient of the last layer to get the intermediate gradient This allows the gradients generated by private data and non-private data to work together in back propagation and model parameter updates;

[0109] On this basis, the aggregated intermediate gradient is used to perform back propagation in the tail model segment of the client and update the model parameters of the tail model segment, and obtain the intermediate gradient g t ; The client will intermediate gradient g t Transmit to the server.

[0110] Through the above process, the gradients generated by private data and non-private data work together for backpropagation and model parameter updates, avoiding the risk of gradient leakage. The gradients corresponding to the private data generated by the client are only used to update the model parameters of the local tail model segment, thus preventing the private information carried in the gradient from being leaked to the server.

[0111] The step S234 of backpropagating the intermediate gradient through the first intermediate model segment and the second intermediate model segment in the server, transmitting the generated reverse intermediate gradient to the client, and updating the model parameters of the first intermediate model segment and the second intermediate model segment includes the following steps:

[0112] In the first intermediate model segment in the server, backpropagating the intermediate gradient to generate a reverse intermediate gradient; and updating the model parameters of the first intermediate model segment;

[0113] Transmit the reverse intermediate gradient to the client;

[0114] When the first intermediate model segment completes a preset number of rounds, the model parameters of the first intermediate model segment are synchronized to the second intermediate model segment to update the model parameters of the second intermediate model segment.

[0115] Specifically, this process involves server-side backpropagation and parameter update. The specific process is as follows:

[0116] The server receives the reverse intermediate gradient g from the client t ;

[0117] Use the intermediate gradient g t The first intermediate model segment W on the server side s Perform backpropagation and update the first intermediate model segment W s The model parameters, when back-propagated to the first layer of the server-side model, generate the reverse intermediate gradient g c ;

[0118] The server will reverse the intermediate gradient g c Passed to the client;

[0119] When the first intermediate model segment completes the preset rounds, the server will s The updated model parameters are synchronized to the second intermediate model segment W S ′, to update the model parameters of the second intermediate model segment. This process only requires a small amount of additional training to make the sub-network used for ciphertext forward propagation applicable to the ciphertext environment.

[0120] Through the above process, the gradients generated for non-private data are used to update the model parameters of the intermediate model segment and the tail model segment to ensure the overall training effect and convergence of the model.

[0121] On this basis, based on the reverse intermediate gradient, the model parameters of the head model segment in the client are updated; and steps S231 to S235 are repeated to obtain the target large model when the large model meets the convergence conditions or reaches the number of training rounds.

[0122] Through this embodiment, local-remote collaborative model fine-tuning training can be achieved while ensuring data privacy and security, reducing local computing and storage burdens, and enabling local devices with limited resources to fine-tune large models based on private data sets. At the same time, it provides more possibilities for further optimizing distributed large model training and improving privacy protection levels.

[0123] The present embodiment is described and illustrated below through preferred embodiments.

[0124] For example, a company wants to fine-tune a pre-trained big model using its private medical dataset to obtain a private target big model. The private dataset contains personal health data, which is private, while all other data is non-private. The target big model is used to output responses to documents related to health topics.

[0125] The pre-trained large model Llama3-70b (with a total of 80 layers) is split into a head model segment containing 4 layers, a tail model segment containing 4 layers, which are deployed on the client, and an intermediate model segment containing 72 layers, which is deployed on a remote server.

[0126] Split the private dataset into private data and non-private data. Divide the private dataset into several batches, each containing 32 data items, 16 of which are private data and 16 are non-private data.

[0127] After the original data of the current batch passes through the header model segment, the first private intermediate activation generated by the private data of this batch is homomorphically encrypted to generate ciphertext data; and the ciphertext data of the private data and the first non-private intermediate activation of the non-private data are transmitted to the server.

[0128] The server contains two sub-networks (the first intermediate model segment and the second intermediate model segment). The first intermediate model segment processes non-private data; the second intermediate model segment processes private data. The two intermediate model segments propagate forward to generate the second non-private intermediate activation and the second private intermediate activation.

[0129] After receiving the two activations (the second non-private intermediate activation and the second private intermediate activation), the client decrypts the encrypted second private intermediate activation and completes the forward propagation of the tail model segment of these two activations. This generates two losses and gradients, aggregates the two gradients, and performs backpropagation using the aggregated intermediate gradients to jointly update the model parameters of the tail model segment. Simultaneously, the loss generated by the non-private data is backpropagated to obtain the reverse intermediate gradient, which is transmitted to the server. The intermediate model segment and the head model segment then update their model parameters based on these gradients.

[0130] After several batches of training, the model parameters of the two sub-networks on the server side are synchronized, and additional training is performed to ensure that the sub-network processing ciphertext data can perform forward propagation normally.

[0131] Repeat the above training steps until the convergence condition is met or the number of training rounds is reached, and the target large model is obtained.

[0132] Through this preferred embodiment, the private data is homomorphically encrypted and used for fine-tuning training of the large model; during the fine-tuning training of the large model, the private data is homomorphically encrypted on the client; and the forward propagation correctness of the homomorphically encrypted ciphertext data is utilized on the server side, and two sub-networks are used to perform parallel forward propagation on the activation of the ciphertext and the activation of the plaintext respectively; a sophisticated gradient management strategy is adopted in the backpropagation process, and forward propagation is performed on the tail model end on the client and the gradient is calculated, and the two parts of the gradient are used for gradient aggregation to jointly update the tail model segment parameters; the gradient generated by the plaintext part is used to update the intermediate model segment and the head model segment to ensure the accuracy and effectiveness of the fine-tuned model.

[0133] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0134] In this embodiment, a privatized large model fine-tuning training device based on split learning is also provided. The device is used to implement the above-mentioned embodiments and preferred embodiments, and the details that have been described will not be repeated. The terms "module", "unit", "sub-unit", etc. used below can implement a combination of software and / or hardware that implements the predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0135] Figure 5 This is a structural block diagram of the privatized large model fine-tuning training device based on split learning in this embodiment, such as Figure 5As shown, the apparatus includes: a first fine-tuning module 210, a second fine-tuning module 220 and a training module 230;

[0136] The first fine-tuning module 210 is configured to split the pre-trained large model into a head model segment, an intermediate model segment, and a tail model segment; deploy the head model segment and the tail model segment on the client; and deploy the intermediate model segment on the server; the intermediate model segment includes a first intermediate model segment and a second intermediate model segment;

[0137] The second fine-tuning module 220 is used to split the original data into private data and non-private data; the original data is document data;

[0138] The training module 230 is used to collaboratively train the head model segment and the tail model segment in the client, and the first intermediate model segment and the second intermediate model segment in the server based on the private data and the non-private data to obtain a target large model; wherein, during the collaborative training process, the private intermediate activations related to the private data transmitted between the client and the server are homomorphically encrypted; the first intermediate model segment is used to process the non-private data; the second intermediate model segment is used to process the private data; and the target large model outputs a document response on a specific topic.

[0139] The above-mentioned device solves the problem in related technologies of needing to transmit intermediate activation amounts to the server, which poses the risk of inferring or recovering the original data through model inversion attacks; homomorphic encryption is used in conjunction with the second intermediate model segment to protect private data and reduce the risk of private data leakage.

[0140] In some embodiments, the first fine-tuning module 210 is also used to split the pre-trained large model according to the hierarchical structure of the Transformer block to obtain a head model segment, an intermediate model segment and a tail model segment; wherein the head model segment and the intermediate model segment each include a small number of blocks; and the intermediate model segment includes a large number of blocks.

[0141] In some of the embodiments, the privatized large model fine-tuning training device based on split learning further includes a replacement module;

[0142] The replacement module is used to replace the nonlinear layer in the first intermediate model segment with a preset polynomial or point activation to obtain a second intermediate model segment.

[0143] In some embodiments, the training module 230 is further configured to perform forward propagation on the private data and the non-private data through the header model segment in the client to obtain a first private intermediate activation and a first non-private intermediate activation; perform homomorphic encryption on the first private intermediate activation to obtain ciphertext data; and transmit the ciphertext data and the first non-private intermediate activation to the server; the private intermediate activation includes the first private intermediate activation and the second private intermediate activation.

[0144] Forwarding the first non-private intermediate activation and the ciphertext data through the first intermediate model segment and the second intermediate model segment in the server, respectively, and returning the obtained second non-private intermediate activation and the second private intermediate activation to the client;

[0145] Through the tail model segment in the client, forward propagate and aggregate the second private intermediate activation and the second non-private intermediate activation to obtain the intermediate gradient; use the intermediate gradient to perform backward propagation in the tail model segment and update the model parameters of the tail model segment; transmit the intermediate gradient to the server;

[0146] Back-propagating the intermediate gradient through the first intermediate model segment and the second intermediate model segment in the server, transmitting the generated reverse intermediate gradient to the client, and updating the model parameters of the first intermediate model segment and the second intermediate model segment;

[0147] Based on the reverse intermediate gradient, update the model parameters of the head model segment in the client;

[0148] When the large model meets the convergence conditions or reaches the number of training rounds, the target large model is obtained.

[0149] In some embodiments, the training module 230 is further configured to forward transmit the first non-private intermediate activation through the first intermediate model segment in the server to obtain a second non-private intermediate activation;

[0150] The encrypted data is forwarded through the second intermediate model segment in the server to obtain a second private intermediate activation;

[0151] The second non-private intermediate activation amount and the second private intermediate activation amount are transmitted to the client.

[0152] In some embodiments, the training module 230 is further configured to decrypt the second private intermediate activation amount to obtain a third private intermediate activation amount;

[0153] In the client's tail model segment, forward propagation is performed on the third private intermediate activation and the second non-private intermediate activation to obtain the predicted value and the corresponding loss function value;

[0154] Back propagation is performed according to the loss function value to obtain the gradient;

[0155] The gradients were aggregated to obtain an intermediate gradient.

[0156] In some embodiments, the training module 230 is further configured to perform back propagation on the intermediate gradient in the first intermediate model segment in the server to generate a reverse intermediate gradient; and update the model parameters of the first intermediate model segment;

[0157] Transmit the reverse intermediate gradient to the client;

[0158] When the first intermediate model segment completes a preset number of rounds, the model parameters of the first intermediate model segment are synchronized to the second intermediate model segment to update the model parameters of the second intermediate model segment.

[0159] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0160] This embodiment further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0161] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0162] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0163] S1: Split the pre-trained large model into a head model segment, an intermediate model segment, and a tail model segment; deploy the head model segment and the tail model segment on the client; deploy the intermediate model segment on the server; the intermediate model segment includes a first intermediate model segment and a second intermediate model segment;

[0164] S2, splits the original data into private data and non-private data; the original data is document data;

[0165] S3, based on the private data and non-private data, collaboratively train the head model segment and the tail model segment in the client, and the first intermediate model segment and the second intermediate model segment in the server to obtain the target large model; wherein, during the collaborative training process, the private intermediate activations related to the private data transmitted between the client and the server are homomorphically encrypted; the first intermediate model segment is used to process the non-private data; the second intermediate model segment is used to process the private data; the target large model outputs a document response on a specific topic.

[0166] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0167] In addition, in conjunction with the split learning-based privatized large model fine-tuning training method provided in the above embodiments, this embodiment may also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the split learning-based privatized large model fine-tuning training methods described in the above embodiments.

[0168] It should be noted that the information and data involved in this application are all authorized by the user or fully authorized by all parties and will be used legally.

[0169] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0170] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0171] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0172] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A private large model fine-tuning training method based on split learning, characterized by: include: Splitting the pre-trained large model into a head model segment, an intermediate model segment, and a tail model segment; deploying the head model segment and the tail model segment on the client; deploying the intermediate model segment on the server; the intermediate model segment includes a first intermediate model segment and a second intermediate model segment; Splitting original data into private data and non-private data; the original data is document data; Collaboratively training the head model segment and the tail model segment in the client, and the first intermediate model segment and the second intermediate model segment in the server according to the private data and the non-private data to obtain a target large model, which includes: collaboratively training the head model segment and the tail model segment in the client according to the private data and the non-private data, and updating model parameters of the head model segment and the tail model segment; collaboratively training the first intermediate model segment and the second intermediate model segment in the server according to the private data and the non-private data, and updating model parameters of the first intermediate model segment and the second intermediate model segment; obtaining the target large model when a convergence condition is met or the number of training rounds is reached; During the collaborative training process, the privacy intermediate activations related to the privacy data transmitted between the client and the server are homomorphically encrypted; the first intermediate model segment is used to process the non-privacy data; the second intermediate model segment is used to process the privacy data; and the target large model outputs a document response on a specific topic.

2. The privatized large model fine-tuning training method based on split learning according to claim 1 is characterized in that: Split the pre-trained large model into the head model segment, the middle model segment, and the tail model segment, including: The pre-trained large model is split according to the hierarchical structure of the Transformer block to obtain the head model segment, the intermediate model segment and the tail model segment; wherein the head model segment and the tail model segment include fewer model blocks than the intermediate model segment.

3. The privatized large model fine-tuning training method based on split learning according to claim 1 is characterized in that: The method further comprises: The nonlinear layer in the first intermediate model segment is replaced by a preset polynomial or point activation to obtain the second intermediate model segment.

4. The privatized large model fine-tuning training method based on split learning according to any one of claims 1 to 3, characterized in that: The head model segment and the tail model segment in the client and the intermediate model segment in the server are collaboratively trained based on the private data and the non-private data to obtain a target large model, including: forward propagating the private data and the non-private data through the header model segment in the client to obtain a first private intermediate activation and a first non-private intermediate activation; homomorphically encrypting the first private intermediate activation to obtain ciphertext data; transmitting the ciphertext data and the first non-private intermediate activation to the server; the private intermediate activations include the first private intermediate activation and the second private intermediate activation; forwarding the first non-private intermediate activation amount and the ciphertext data through the first intermediate model segment and the second intermediate model segment in the server, respectively, and returning the obtained second non-private intermediate activation amount and second private intermediate activation amount to the client; Performing forward propagation and aggregation on the second private intermediate activation and the second non-private intermediate activation through the tail model segment in the client to obtain an intermediate gradient; performing backward propagation in the tail model segment using the intermediate gradient and updating the model parameters of the tail model segment; and transmitting the intermediate gradient to the server; Backward propagating the intermediate gradient through the first intermediate model segment and the second intermediate model segment in the server, transmitting the generated reverse intermediate gradient to the client, and updating the model parameters of the first intermediate model segment and the second intermediate model segment; updating the model parameters of the head model segment in the client based on the reverse intermediate gradient; When the large model meets the convergence condition or reaches the number of training rounds, the target large model is obtained.

5. The privatized large model fine-tuning training method based on split learning according to claim 4 is characterized in that: Forward transmitting the first non-private intermediate activation amount and the ciphertext data respectively through the first intermediate model segment and the second intermediate model segment in the server, and returning the obtained second non-private intermediate activation amount and second private intermediate activation amount to the client, comprising: forward transmitting the first non-private intermediate activation amount through the first intermediate model segment in the server to obtain a second non-private intermediate activation amount; forward transmitting the ciphertext data through the second intermediate model segment in the server to obtain a second privacy intermediate activation; The second non-private intermediate activation amount and the second private intermediate activation amount are transmitted to the client.

6. The privatized large model fine-tuning training method based on split learning according to claim 4 is characterized in that: Performing forward propagation and aggregation on the second private intermediate activation and the second non-private intermediate activation through the tail model segment in the client to obtain an intermediate gradient, including: Decrypting the second private intermediate activation amount to obtain a third private intermediate activation amount; In the tail model segment of the client, forward propagation is performed on the third private intermediate activation and the second non-private intermediate activation to obtain a predicted value and a corresponding loss function value; Perform back propagation according to the loss function value to obtain the gradient; The gradient is polymerized to obtain the intermediate gradient.

7. The privatized large model fine-tuning training method based on split learning according to claim 4 is characterized in that: Backward propagating the intermediate gradient through the first intermediate model segment and the second intermediate model segment in the server, transmitting the generated reverse intermediate gradient to the client, and updating model parameters of the first intermediate model segment and the second intermediate model segment, including: In the first intermediate model segment in the server, backpropagating the intermediate gradient to generate a reverse intermediate gradient; and updating the model parameters of the first intermediate model segment; transmitting the reverse intermediate gradient to the client; When the first intermediate model segment completes a preset number of rounds, the model parameters of the first intermediate model segment are synchronized to the second intermediate model segment to update the model parameters of the second intermediate model segment.

8. A private large model fine-tuning training device based on split learning, characterized in that: include: A first fine-tuning module, a second fine-tuning module, and a training module; The first fine-tuning module is used to split the pre-trained large model into a head model segment, an intermediate model segment, and a tail model segment; deploy the head model segment and the tail model segment on the client; deploy the intermediate model segment on the server; the intermediate model segment includes a first intermediate model segment and a second intermediate model segment; The second fine-tuning module is configured to split the original data into private data and non-private data; the original data is document data; The training module is configured to collaboratively train the head model segment and the tail model segment in the client, and the first intermediate model segment and the second intermediate model segment in the server based on the private data and the non-private data, to obtain a target large model. The training module includes: collaboratively training the head model segment and the tail model segment in the client based on the private data and the non-private data, and updating the model parameters of the head model segment and the tail model segment; collaboratively training the first intermediate model segment and the second intermediate model segment in the server based on the private data and the non-private data, and updating the model parameters of the first intermediate model segment and the second intermediate model segment; and obtaining the target large model when a convergence condition is met or the number of training rounds is reached. During the collaborative training process, the privacy intermediate activations related to the privacy data transmitted between the client and the server are homomorphically encrypted; the first intermediate model segment is used to process the non-privacy data; the second intermediate model segment is used to process the privacy data; and the target large model outputs a document response on a specific topic.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps of the privatized large model fine-tuning training method based on split learning according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the privatized large model fine-tuning training method based on split learning according to any one of claims 1 to 7 are implemented.

Images