Method, device and product for homomorphic fine-tuning and inference of pre-trained language models

Abstract

The application discloses a method, device and product for pre-training language model-oriented secure fine-tuning and reasoning, wherein a participant includes a user and a service provider; the service provider possesses a pre-training language model and participates in secure fine-tuning and secure reasoning, parameters of the pre-training language model being privacy of the service provider; the user possesses a fine-tuning data set and input and output of reasoning, being privacy data; the secure fine-tuning fine-tunes the language model on the premise of protecting the pre-training language model of the service provider and the fine-tuning data set of the user, and the user obtains a non-frozen part of a fine-tuned model; the secure reasoning performs reasoning of a frozen layer of the fine-tuned model by the service provider on the premise of protecting input and output of the user for reasoning, and delivers a reasoning result to the user, and the user performs reasoning of a non-frozen layer and obtains a reasoning result. The application reduces time and communication overhead of secure fine-tuning and reasoning, and improves accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of information security technology and relates to a method, device and product for dense fine-tuning and reasoning of pre-trained language models. Specifically, it relates to a method, device and product for fine-tuning and reasoning of pre-trained language models with low overhead and high accuracy while protecting the privacy of service providers and users. Background Art

[0002] Pre-trained language models have demonstrated excellent performance in many domains. A common approach to using these models is fine-tuning, which addresses the need to customize new models without extensive computing resources. In practice, platforms like ChatGPT provide remote APIs for fine-tuning, allowing users to upload their training data to train and deploy customized models for inference.

[0003] Despite their success, this fine-tuning and inference service can infringe on user privacy. For example, social data often contains sensitive personal information that users do not want to disclose to service providers. Furthermore, service providers can access user queries and predictions in plaintext during inference, which is unsuitable for applications such as medical diagnosis. While service providers can provide pre-trained models to users for local fine-tuning and inference, this compromises their intellectual property and poses a variety of potential attacks.

[0004] Existing privacy-preserving schemes for pre-trained language models are costly. This is primarily due to two reasons: First, in dense inference on deep neural networks, security protocols are invoked layer by layer. However, language models have more complex architectures and larger parameters. In particular, they require multiple calls to the matrix multiplication protocol. Second, existing research uses an approximation of the Gaussian Error Linear Unit (GELU), a unique nonlinear function in language models. However, using a uniform piecewise linear approximation without considering curvature results in unnecessary cost in low-curvature segments and loss of accuracy in high-curvature segments. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a fine-tuning and reasoning method, device and product with low overhead, high accuracy and protection of the privacy of service providers and users.

[0006] The technical solution adopted by the method of the present invention is: a dense fine-tuning and inference method for pre-trained language models, in which the participants include users and service providers;

[0007] The language model is composed of a stack of multiple transformer modules, each of which is divided into a linear layer and a nonlinear layer. The linear layer includes an embedding layer, a matrix multiplication layer, and a fully connected layer. The nonlinear layer includes a normalized exponential function layer Softmax layer, a Gaussian error linear unit layer GELU layer, and a layer normalization layer LayerNorm layer.

[0008] The service provider owns the pre-trained language model and participates in the confidential fine-tuning and confidential inference. The parameters of the pre-trained language model are private to the service provider. The user owns the fine-tuning dataset and the input and output of the inference, which are private data.

[0009] The confidential fine-tuning fine-tunes the language model while protecting the service provider's pre-trained language model and the user's fine-tuning dataset, and the user obtains the non-frozen part of the fine-tuned model;

[0010] The dense inference is carried out by the service provider on the premise of protecting the user's inference input and output, and the obtained inference results are delivered to the user, who then performs inference on the non-frozen layer and obtains the inference results.

[0011] Preferably, the dense state fine-tuning repeatedly executes four protocols, including a secure computation protocol for forward propagation and derivatives, a secure computation protocol for derivatives of the loss function, a secure computation protocol for backpropagation, and an optimizer protocol, until the language model converges to obtain a fine-tuned model. The service provider delivers its share of the unfrozen portion of the fine-tuned model to the user. At this point, the user owns the unfrozen portion of the fine-tuned model, and the service provider owns the frozen portion of the fine-tuned model.

[0012] The secure computation protocol for forward propagation and derivatives, the user inputs the matrix X, and the service provider inputs the pre-trained language model M P , output the result share of forward propagation <z>and the derivatives of each layer of the neural network <d>;

[0013] The loss function derivative secure calculation protocol, user input matrix Y, user and service provider input forward propagation result share <z>, output share in represents the derivative of the cross entropy loss function;

[0014] The secure computation protocol of the back-propagation, user and service provider input matrix <d>, output gradient share <g>;

[0015] The optimizer protocol, user and service provider input matrices <g>and the current language model, output the updated language model <M T >.

[0016] As a preference, the dense state reasoning, the reasoning of the frozen part, uses the forward propagation protocol to calculate the result share <z>0 and <z>1, <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and then the service provider sends <z>1 to the user; for the inference of the unfrozen part, the user inputs Z and the final prediction result is calculated locally.

[0017] As an optimization, the linear layer calculation relies on the matrix multiplication protocol M2OFL(X,W), the user inputs the matrix X, the service provider inputs the matrix W, and outputs the secret sharing share<X·W> , where · represents matrix multiplication;

[0018] The linear computation Q·K in the first transformer module is frozen T =X·W·X T ,in W w 、W p is the embedding layer parameter owned by the service provider, W Q 、W K are the fully connected layer parameters owned by the service provider;

[0019] For the first transformer module freeze-friendly linear computation security protocol EMM(), the specific implementation includes offline phase steps A1-A3 and online phase steps A4-A6:

[0020] Step A1: The user generates a random matrix R;

[0021] Step A2: The user and the service provider use the matrix multiplication protocol M3OFL(R,W) and obtain the secret sharing share <R·W·R T >

[0022] Step A3: The user and the service provider use the matrix multiplication protocol M2OFL(R,W), M2OFL(R,W T ) and obtain the secret share<R·W> 、 <R·W T >

[0023] Step A4: The user sends XR to the service provider;

[0024] Step A5: User calculates and obtains share <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0025] Step A6: Service provider calculates and obtains share Among them, Q·K T = <Q·K T >0+ <Q·K T >1, <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1.

[0026] As a preference, for linear computations in subsequent transformer modules that are frozen in ∈, ζ are the parameters of the LayerNorm layer owned by the service provider, W Q 、W K are the parameters of the attention layer owned by the service provider, X * is the intermediate result of the LayerNorm layer, <X # >0= <X * >0 owned by user, Owned by the service provider, <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0027] The freeze-friendly linear computation safety protocol for subsequent transformer modules is implemented in the offline phases B1-B2 and the online phases B3-B5:

[0028] Step B1: User and service provider use EMM (W * ) offline phase to obtain R # 、 where R # =R;

[0029] Step B2: User and service provider use matrix multiplication protocol and obtain a secret share

[0030] Step B3: User and service provider use EMM (W * ) online phase, to obtain < <X # >0· Service providers also receive <X # >0-R # ;

[0031] Step B4: User calculates and obtains share <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0032] Step B5: The service provider calculates and obtains the share Among them, Q·K T = <Q·K T >0+ <Q·K T >1.

[0033] Preferably, in step A2, the matrix multiplication protocol M3OFL(R,W), the user inputs the matrix R, and the service provider inputs the matrix W to obtain the secret sharing share <R·W·R T >

[0034] The specific implementation calculation includes the following steps:

[0035] Step C1: The user follows m w ×n w 、n w ×m w The window specification will be the matrix R with the specification of m×n and the matrix R with the specification of n×m T Divide into small matrix blocks R α,β 、 Among them, α, δ∈[m′], The subscript w indicates window;

[0036] Step C2: User passes R α,β 、 Encoded as a polynomial; π A , π C Represents the mapping function from matrix to polynomial;

[0037] Step C3: The service provider follows n w ×n w Divide the n×n matrix W into small matrix blocks W β,γ and through W β,γ Encoded as a polynomial, π B Represents the mapping function from matrix to polynomial;

[0038] Step C4: User sends Ciphertext to service providers;

[0039] Step C5: Service Provider Calculation in is a homomorphic addition calculation, It is a homomorphic multiplication calculation;

[0040] Step C6: The service provider uses Clean(a 0,α,γ ,n w ) to obtain in n w The coefficients and a corresponding to the integer multiple terms 0,α,γ The coefficients of the other terms are 0;

[0041] Step C7: Service Provider Calculation And from S 3,α,δ Recover from <R·W·R T >1, where For homomorphic subtraction calculation, S 3,α,δ is a mask polynomial randomly generated by the service provider;

[0042] Step C8: If m w ≥n w , the service provider sends a 3,α,δ Otherwise, each n w / m w Polynomials are combined into one and then sent out;

[0043] Step C9: User decrypts and recovers the plaintext <R·W·R T >0.

[0044] Preferably, the matrix is ​​encoded as a function of a multinomial in In addition, if i=1 and j∈[2,n], calculate If i∈

[0045] [2,m] and j∈[n], then calculate N is the degree of the polynomial;

[0046] The matrix is ​​encoded as a function of multiple numbers Among them, if i∈[n] and j∈[n], calculate

[0047] The matrix is ​​encoded as a function of multiple numbers When m <n时,如果i=1且j∈[m],就计算 If i∈[2,n] and j∈[m], calculate When m≥n, if i=1 and j∈[n], or i∈[n] and j∈[n+1,m], calculate If i∈[2,n] and j∈[n], calculate

[0048] Preferably, for the GELU(x) layer of the nonlinear layer, the user input share <x>0, service provider input share <x>1. To get a share <y>、<y′> , where y=GELU(x), y′ represents the derivative of y;

[0049] The specific implementation of the calculation includes the following steps:

[0050] Step D1: User and service provider local computing in i∈[6], L is the number of bits of the language model left shift, t i is the segmentation when approximating the GELU(x) curve, <>0 means that the share belongs to the user, <>1 means that the share belongs to the service provider, and <> means <>0 and <>1;

[0051] Step D2: User and Service Provider Extraction <x i Low To obtain is a hyperparameter;

[0052] Step D3: User and service provider use phase agreement calculation interception High S i Position i∈[3],Π tr To cut off the agreement;

[0053] Step D4: The user and the service provider use the lookup table protocol to calculate i∈[3], and set <w4>= <1>, Π lit is a lookup table protocol, <w i >, <η i > are shares of coefficients and bias values of a linear approximation stored in a lookup table T i ;

[0054] Step D5: The user and the service provider compute <w i x> = Π mut (<w i >, <x>), i∈[3], Π mut It is a multiplication protocol;

[0055] Step D6: User and service provider compute locally i >= <w i x>+<η o >, and set <u4> ＝ <x>;

[0056] Step D7: User and service provider compute using comparison protocol i∈[6],Π cmp is a comparison protocol, <ρ i > denotes <ρ o >0 and <ρ i >1, ρ =1 otherwise, where i denotes the XOR computation;

[0057] Step D8: User and service provider locally compute where denotes the XOR computation and set <z4>=<ρ6>;

[0058] Step D9: The user and the service provider use the multiplexer protocol to calculate <v i >=Π mux ( <z i >, i >)、 <v′ i >=Π mux ( <z i >, <w i >), then, calculate and output locally <y>=∑ <v i >、<y′> =∑ <v′ i >, i∈[4], Π mux For a multi-way selection protocol, when z i =1 when output u i , when z i =0, output 0, <v i >、 <v′ i > is the share of the results of the agreement.

[0059] The technical solution adopted by the device of the present invention is: a dense state fine-tuning and inference device for pre-trained language models, comprising:

[0060] one or more processors;

[0061] A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the dense fine-tuning and inference method for the pre-trained language model.

[0062] The technical solution adopted by the product of the present invention is: a dense state fine-tuning and reasoning product for pre-trained language models, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the dense state fine-tuning and reasoning method for pre-trained language models.

[0063] Compared with the prior art, the beneficial effects of the present invention include:

[0064] The present invention designs a dense state fine-tuning and reasoning system for pre-trained language models, in which the user's data and the service provider's pre-trained model are protected, and the scheme reduces time and communication overhead and improves accuracy. By proposing a freezing-friendly matrix multiplication protocol, the protocol optimizes the frozen layer of the transformer model as a whole, thereby reducing computational and communication costs and complexity. For calculating Gaussian Error Linear Units (GELU), a derivative-friendly GELU protocol is proposed, which not only achieves higher accuracy and lower cost, but also makes it easy to derive derivatives during the fine-tuning process. Compared with previous solutions, the present invention achieves dense state fine-tuning and reasoning for pre-trained language models with low time, communication overhead and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The technical solution of this invention is further illustrated below using embodiments and specific implementation methods. In addition, some drawings are used in the process of illustrating the technical solution. Those skilled in the art can also derive other drawings and the intent of the present invention based on these drawings without making any creative efforts.

[0066] Figure 1 A schematic diagram of a method according to an embodiment of the present invention;

[0067] Figure 2 This is a flow chart of the dense state fine-tuning according to an embodiment of the present invention;

[0068] Figure 3 Schematic diagram of a matrix coding layer polynomial in an embodiment of the present invention;

[0069] Figure 4 This is an example diagram of the GELU approximation scheme of an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to facilitate the understanding and implementation of the present invention by those skilled in the art, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0071] Please see Figure 1 ,This embodiment provides a dense fine-tuning and inference method for a pre-trained language model, ,participants include users and service providers;

[0072] The language model is composed of a stack of multiple transformer modules, each of which is divided into a linear layer and a nonlinear layer. The linear layer includes an embedding layer, a matrix multiplication layer, and a fully connected layer. The nonlinear layer includes a normalized exponential function layer Softmax layer, a Gaussian error linear unit layer GELU layer, and a layer normalization layer LayerNorm layer.

[0073] The service provider owns the pre-trained language model and participates in the confidential fine-tuning and confidential inference. The parameters of the pre-trained language model are private to the service provider. The user owns the fine-tuning dataset and the input and output of the inference, which are private data.

[0074] The confidential fine-tuning fine-tunes the language model while protecting the service provider's pre-trained language model and the user's fine-tuning dataset, and the user obtains the non-frozen part of the fine-tuned model;

[0075] The dense inference is carried out by the service provider on the premise of protecting the user's inference input and output, and the obtained inference results are delivered to the user, who then performs inference on the non-frozen layer and obtains the inference results.

[0076] Please see Figure 2 In one embodiment, the dense state fine-tuning repeatedly executes four protocols, including a secure computation protocol for forward propagation and derivatives, a secure computation protocol for loss function derivatives, a secure computation protocol for backpropagation, and an optimizer protocol, until the language model converges to obtain a fine-tuned model. The service provider then delivers its share of the unfrozen portion of the fine-tuned model to the user. At this point, the user owns the unfrozen portion of the fine-tuned model, and the service provider owns the frozen portion of the fine-tuned model.

[0077] In one embodiment, for the secure computation protocol of forward propagation and derivatives, the user inputs the matrix X and the service provider inputs the pre-trained language model M P , output the result share of forward propagation <z>and the derivatives of each layer of the neural network <d>.

[0078] In one embodiment, for the secure computation of forward propagation and derivatives, the present invention provides a freeze-friendly matrix multiplication protocol M2OFL(X,W) and a derivative-friendly GELU computation protocol.

[0079] In one embodiment, the freeze-friendly matrix multiplication protocol M2OFL(X,W) includes an online|offline sub-protocol for the first transformer module and an online|offline sub-protocol for subsequent transformer modules;

[0080] For the online|offline sub-protocol of the first transformer module, the specific calculation steps are as follows:

[0081] Step 1: The user generates a random matrix R;

[0082] Step 2: The user and service provider use the matrix multiplication protocol M3OFL(R,W) and obtain the secret sharing share <R·W·R T >

[0083] Step 3: The user and the service provider use the matrix multiplication protocol M2OFL(R,W), M2OFL(R,W T ) and obtain the secret share<R·W> 、 <R·W T >

[0084] Step 4: The user sends XR to the service provider (- is a subtraction operation);

[0085] Step 5: User calculates and gets share <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0086] Step 6: The service provider calculates and obtains the share Among them, Q·K T = <Q·K T >0+ <Q·K T >1, <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0087] For the linear calculations in the subsequent transformer modules that are frozen Where W * =∈ 2 W Q W K 、 W Q 、W K are the parameters of the attention layer owned by the service provider, ∈, ζ are the parameters of the LayerNorm layer owned by the service provider, X * is the intermediate result of the LayerNorm layer, <X # >0= <X * >0 owned by user, Owned by the service provider.

[0088] In one embodiment, for the subsequent transformer module's online|offline sub-protocol, the specific implementation calculation steps include:

[0089] Step 1: User and service provider use EMM (W * ) offline phase to obtain where R # =R;

[0090] Step 2: The user and the service provider use the matrix multiplication protocol M2OFL (R # ,W1 * ), and obtain a secret share Represents matrix multiplication;

[0091] Step 3: User and service provider use EMM (W * ) online phase, to obtain < <X # >0· Service providers also receive <X # >0-R # ;

[0092] Step 4: User calculates and gets share <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0093] Step 5: The service provider calculates and obtains the share Among them, Q·K T = <Q·K T >0+ <Q·K T >1.

[0094] Please see Figure 3 This embodiment provides a matrix multiplication protocol M3OFL(R,W), where the user inputs the matrix R and the service provider inputs the matrix W to obtain the secret sharing share. <R·W·R T >, the specific calculation steps include:

[0095] Step 1: User clicks m w ×n w 、n w ×m w The window specification will be the matrix R with the specification of m×n and the matrix R with the specification of n×m T Divide into small matrix blocks R α,β 、 Among them, α, δ∈[m′], β, γ∈[n′], The subscript w indicates window;

[0096] Step 2: User passes R α,β 、 Encoded as a polynomial; π A , π C Represents the mapping function from matrix to polynomial;

[0097] Step 3: The service provider follows n w ×n w Divide the n×n matrix W into small matrix blocks W β,γ and through W β,γ Encoded as a polynomial; v B Represents the mapping function from matrix to polynomial;

[0098] Step 4: User sends Ciphertext to service providers;

[0099] Step 5: Service Provider Calculation in is a homomorphic addition calculation, It is a homomorphic multiplication calculation;

[0100] Step 6: Service Provider uses Clean(a 0,α,γ ,n w ) to obtain in n w The coefficients and a corresponding to the integer multiple terms 0,α,γ The coefficients of the other terms are 0;

[0101] Step 7: Service Provider Calculation And from S 3,α,δ Recover from <R·W·R T >1, where For homomorphic subtraction calculation, S 3,α,δ is a mask polynomial randomly generated by the service provider;

[0102] Step 8: If m w ≥n w , the service provider sends a 3,α,δ Otherwise, each n w / m w Polynomials are combined into one and then sent out;

[0103] Step 9: User decrypts and recovers the plaintext <R·W·R T >0.

[0104] In one embodiment, for the matrix multiplication protocol, this embodiment provides an encoding function of a matrix encoding polynomial:

[0105] Functions encoding matrices into polynomials in In addition, if i=1 and j∈[2,n], calculate If i∈[2,m] and j∈

[0106] [n], then calculate N is the degree of the polynomial;

[0107] Functions encoding matrices into polynomials Among them, if i∈[n] and j∈[n], calculate

[0108] Functions encoding matrices into polynomials When m <n时,如果i=1且j∈[m],就计算 If i∈[2,n] and j∈[m], calculate When m≥n, if i=1 and j∈[n], or i∈[n] and j∈[n+1,m], calculate If i∈

[0109] [2,n] and j∈[n], then calculate

[0110] For the nonlinear layer GELU(x), this embodiment provides a derivative-friendly GELU secure calculation protocol, and the user input share <x>0, service provider input share <x>1. To get a share <y>、<y′> , where y=GELU(x), y′ represents the derivative of y, and the specific calculation steps include:

[0111] Step 1: User and service provider local computing in i∈[6], L is the number of bits of the model left shift, t i is the segmentation when approximating the GELU(x) curve, <>0 means that the share belongs to the user, <>1 means that the share belongs to the service provider, and <> means <>0 and <>1;

[0112] Step 2: User and Service Provider Extraction <x′ i >Low To obtain i∈[3], is a hyperparameter;

[0113] Step 3: User and service provider use phase agreement calculation interception High S i Position i∈[3],Π tr To cut off the agreement;

[0114] Step 4: User and service provider use lookup table protocol to calculate i∈[3], and set <w4>= <1> , Π lut is a lookup table protocol, <w i >,<η i > is stored in the lookup table T i The coefficients and bias values ​​of the linear approximation in ;

[0115] Step 5: User and service provider use multiplication protocol to calculate <w i x>=Π mut ( <w i >, <x>), i ∈ [3], Π mut is a multiplication protocol;

[0116] Step 6: The user and the service provider locally compute u i > = <w i x> + <η i >, and set <u4> ＝ <x>;

[0117] Step 7: User and service provider use comparison protocol calculation i∈[6],Π cmp is the comparison agreement, <ρ i > means <ρ i >0 and <ρ i >1, when When ρ i =1, otherwise it is 0, where Indicates exclusive OR calculation;

[0118] Step 8: User and Service Provider Local Computing Where ⊕ represents XOR calculation, and sets <z4>=<ρ6>;

[0119] Step 9: User and service provider use multiplexer protocol calculation <v i >=Π mux ( <z i >, i >)、 <v′ i >=Π mux ( <z i >, <w i >), then, calculate and output locally <y>=∑ <v i >、<y′> =∑ <v′ i >, i∈[4], Π mux For a multi-way selection protocol, when z i =1 when output u i , when z i =0, output 0, <v i >、 <v′ i > is the share of the results of the agreement;

[0120] Please see Figure 4 , is an example diagram of the GELU approximation scheme of an embodiment of the present invention. Figure 4 As can be seen from the figure, this embodiment divides the curve into five parts according to the curvature of the function.

[0121] This embodiment also provides a dense state fine-tuning and inference device for a pre-trained language model, including:

[0122] one or more processors;

[0123] A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the dense fine-tuning and inference method for the pre-trained language model.

[0124] This embodiment also provides a dense state fine-tuning and reasoning product for a pre-trained language model, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the dense state fine-tuning and reasoning method for a pre-trained language model.

[0125] The following is a detailed explanation of the present invention through experiments. Experiments were conducted on the BERT and GPT2 models, using the WNLI, CoLA, and SST-2 datasets. Comparisons with the current best-performing Iron and CipherGPT models demonstrated improvements in time and communication overhead. Comparisons with the current best-performing Iron, CipherGPT, and Plaintext models also demonstrated improvements in accuracy.

[0126] Table 1 Comparative experimental results of the present invention in terms of time and communication overhead

[0127]

[0128] Table 2 Comparative experimental results of the present invention on accuracy

[0129]

[0130] The present invention can provide:

[0131] 1. A low-overhead, high-accuracy, and confidential fine-tuning and inference system for pre-trained language models that protects both model and user privacy: Neither model nor user data is leaked during this process.

[0132] 2. Freeze-friendly matrix multiplication protocol: This protocol optimizes the frozen layers of the transformer model as a whole, thereby reducing computational and communication costs and complexity.

[0133] 3. Derivative-friendly GELU computation protocol: This protocol not only achieves higher accuracy and lower cost, but also makes it easy to derive derivatives during fine-tuning.

[0134] It should be understood that the embodiments described above are only some of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention may be arbitrarily combined with each other to form a feasible technical solution. Such combination is not restricted by the order of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is mutually inconsistent or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0135] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.< / y> ​ < / x> < / u4> < / x> < / y> < / x> < / x> < / d> < / z> < / y> ​ ​< / x> < / u4> ​< / x> < / y> < / x> < / x> < / z> < / z> < / z> < / g> < / g> < / d> < / z> < / d> < / z>

Claims

1. A dense fine-tuning and inference method for pre-trained language models, involving users and service providers; characterized by: The language model is composed of a stack of multiple transformer modules, each of which is divided into a linear layer and a nonlinear layer. The linear layer includes an embedding layer, a matrix multiplication layer, and a fully connected layer. The nonlinear layer includes a normalized exponential function layer (Softmax layer), a Gaussian error linear unit layer (GELU layer), and a layer normalization layer (LayerNorm layer); The service provider owns a pre-trained language model and participates in closed-state fine-tuning and closed-state inference. The parameters of the pre-trained language model are private to the service provider. The user owns the fine-tuning dataset and the input and output of the inference, which are private data. The confidential fine-tuning fine-tunes the language model while protecting the service provider's pre-trained language model and the user's fine-tuning dataset, and the user obtains the non-frozen part of the fine-tuned model; The encrypted fine-tuning repeatedly executes four protocols, including the secure computation protocol for forward propagation and derivatives, the secure computation protocol for loss function derivatives, the secure computation protocol for backpropagation, and the optimizer protocol, until the language model converges to obtain a fine-tuned model. The service provider then delivers its share of the unfrozen portion of the fine-tuned model to the user. At this point, the user owns the unfrozen portion of the fine-tuned model, and the service provider owns the frozen portion of the fine-tuned model. The secure computation protocol for forward propagation and derivatives, the user inputs the matrix X, and the service provider inputs the pre-trained language model M P , output the result share of forward propagation <z>and the derivatives of each layer of the neural network <d>, where, for the meaning of share <>, the result share of forward propagation <z>For example, <z>express <z>0 and <z> 1， <z> 0+ <z> 1=Z, <> subscript 0 means the share belongs to the user, <> subscript 1 means the share belongs to the service provider;< / z> < / z> < / z> < / z> < / z> < / z> < / d> < / z> The loss function derivative secure calculation protocol, user input matrix Y, user and service provider input forward propagation result share <z>, output share in represents the derivative of the cross entropy loss function;< / z> The secure computation protocol of the back-propagation, user and service provider input matrix <d>, output gradient share <g> ；< / g> < / d> The optimizer protocol, user and service provider input matrices <g>and the current language model, output the updated language model <M T >< / g> The encrypted inference is performed by the service provider on the frozen layer of the fine-tuned model while protecting the user's inference input and output. The obtained inference results are delivered to the user, who then performs inference on the unfrozen layer and obtains the inference results. The dense reasoning, the reasoning of the frozen part, uses the forward propagation protocol to calculate the result share <z>0 and <z>1, <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and then the service provider sends <z> 1 to the user; for the inference of the unfrozen part, the user inputs Z and the final prediction result is calculated locally.< / z> < / z> < / z> 2. The dense state fine-tuning and inference method for a pre-trained language model according to claim 1, characterized in that: The linear layer calculation relies on the matrix multiplication protocol M2OFL(X,W), the user inputs the matrix X, the service provider inputs the matrix W, and outputs the secret sharing share<X·W> , where · represents matrix multiplication; The linear computation Q·K in the first transformer module is frozen T =X·W·X T ,in W w 、W p is the embedding layer parameter owned by the service provider, W Q 、W K are the fully connected layer parameters owned by the service provider; For the first transformer module, the freeze-friendly linear computation security protocol EMM() is implemented in the offline phase steps A1-A3 and the online phase steps A4-A6: Step A1: The user generates a random matrix R; Step A2: The user and the service provider use the matrix multiplication protocol M3OFL(R,W) and obtain the secret sharing share <R·W·R T > Step A3: The user and the service provider use the matrix multiplication protocol M2OFL(R,W), M2OFL(R,W T ) and obtain the secret share<R·W> 、 <R·W T > Step A4: The user sends XR to the service provider; Step A5: User calculates and obtains share <>0 means the share belongs to the user, <>1 means it belongs to the service provider; Step A6: Service provider calculates and obtains share Among them, Q·K T = <Q·K T >0+ <Q·K T >1, <>0 means the share belongs to the user, <>1 means it belongs to the service provider.

3. The dense state fine-tuning and inference method for a pre-trained language model according to claim 2, characterized in that: For the linear calculations in the subsequent transformer modules that are frozen Where W * =∈ 2 W Q W K 、 ∈, ζ are the parameters of the LayerNorm layer owned by the service provider, W Q 、W K is the parameter of the attention mechanism layer owned by the service provider, X * is the intermediate result of the LayerNorm layer, <X # >0= <X * >0 owned by user, Owned by the service provider, <>0 means the share belongs to the user, <>1 means it belongs to the service provider; The freeze-friendly linear computation safety protocol for subsequent transformer modules is implemented in the offline phases B1-B2 and the online phases B3-B5: Step B1: User and service provider use EMM (W * ) offline phase to obtain R # 、 <R # W * >、 where R # =R; Step B2: User and service provider use matrix multiplication protocol and obtain a secret share and in Step B3: User and service provider use EMM (W * ) online stage, to obtain Service providers also receive <X # >0-R # , <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1; Step B4: User calculates and obtains share <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1; Step B5: The service provider calculates and obtains the share Among them, Q·K T = <Q·K T >0+ <Q·K T >1.

4. The dense state fine-tuning and inference method for a pre-trained language model according to claim 3, characterized in that: Step A2: The matrix multiplication protocol M3OFL(R,W), the user inputs the matrix R, the service provider inputs the matrix W, to obtain the secret sharing share <R·W·R T > The specific implementation of the calculation includes the following steps: Step C1: The user follows m w ×n w 、n w ×m w The window specification will be the matrix R with the specification of m×n and the matrix R with the specification of n×m T Divide into small matrix blocks R α,β 、 Among them, α, δ∈[m′], β, γ∈[n′], The subscript w indicates the window, n w and m w It is a hyperparameter; are there any special requirements for the values ​​of m and n? Step C2: User passes R α,β 、 Encoded as a polynomial; π A , π C Represents a mapping function from matrix to polynomial; Step C3: The service provider follows n w ×n w Divide the n×n matrix W into small matrix blocks W β,γ and through W β,γ Encoded as a polynomial; π B Represents a mapping function from matrix to polynomial; Step C4: User sends Ciphertext to service providers; Step C5: Service Provider Calculation in is a homomorphic addition calculation, It is a homomorphic multiplication calculation; Step C6: The service provider uses Clean(a 0,α,γ ,n w ) to obtain in n w The coefficients and a corresponding to the integer multiple terms 0,α,γ The coefficients of the other terms are 0; Step C7: Service Provider Calculation And from S 3,α,δ Recover from <R·W·R T >1, where For homomorphic subtraction calculation, S 3,α,δ is a mask polynomial randomly generated by the service provider; Step C8: If m w ≥n w , the service provider sends a 3,α,δ Otherwise, each n w / m w Polynomials are combined into one and then sent out; Step C9: User decrypts and recovers the plaintext <R·W·R T >0.

5. The dense state fine-tuning and inference method for a pre-trained language model according to claim 4, characterized in that: The matrix is ​​encoded as a function of multiple numbers in In addition, if i=1 and j∈[2,n], calculate If i∈[2,m] and j∈[n], calculate N is the degree of the polynomial; The matrix is ​​encoded as a function of multiple numbers Among them, if i∈[n] and j∈[n], calculate The matrix is encoded into a function of multiple numbers When m < n, if i = 1 and j ∈ [m], calculate If i ∈ [2, n] and j ∈ [m], calculate When m ≥ n, if i = 1 and j ∈ [n], or i ∈ [n] and j ∈ [n + 1, m], calculate If i ∈ [2,n] and j∈[n], then calculate 6. The dense state fine-tuning and inference method for a pre-trained language model according to claim 1, characterized in that: For the GELU(x) layer of the nonlinear layer, the user input share <x>0, service provider input share <x>1. To get a share <y> 、<y′> , where y=GELU(x), y′ represents the derivative of y;< / y> < / x> < / x> The specific implementation calculation includes the following steps: Step D1: User and service provider local computing in L is the number of bits of the language model left shift, t i is the segmentation when approximating the GELU(x) curve; Step D2: User and Service Provider Extraction <x′ i >Low To obtain is a hyperparameter; Step D3: User and service provider use phase agreement calculation interception High S i Position Π tr To cut off the agreement; Step D4: The user and the service provider use the lookup table protocol to calculate And set <w4>= <1> , Π lut is a lookup table protocol, <w i >,<η i > is stored in the lookup table T i The coefficients and bias values ​​of the linear approximation in ; Step D5: The user and the service provider use the multiplication protocol to calculate <w i x>=Π mut ( <w i >, <x>), i∈[3], Π mut It is a multiplication protocol;< / x> Step D6: User and service provider compute locally i >= <w i x>+<η i >, and set <u4> ＝ <x> ；< / x> < / u4> ​ Step D7: The user and the service provider use the comparison protocol to calculate Π cmp is the comparison agreement, <ρ i > means <ρ i >0 and <ρ i >1,<ρ i >0⊕<ρ i >1=ρ i ,when When ρ i =1, otherwise it is 0, where ⊕ represents XOR calculation; Step D8: User and service provider local computing <z1>=<ρ3>⊕<ρ4>, <z2>=<ρ2>⊕<ρ3>⊕<ρ4>⊕<ρ5>, <z3>=<ρ1>⊕<ρ2>⊕<ρ5>⊕<ρ6>, where ⊕ represents XOR calculation, and sets <z4> =<ρ6>; Step D9: The user and the service provider use the multiplexer protocol to calculate <v i >=Π mux ( <z i >, i >)、 <v′ i >=Π mux ( <z i >, <w i >), then calculate and output locally <y>=∑ <v i >、<y′> =∑ <v′ i >, i∈[4], Π mux For a multi-way selection protocol, when z i =1 when output u i , when z i =0, output 0, <v i >、 <v′ i > is the output of the protocol.< / y> ​ 7. A dense state fine-tuning and inference device for pre-trained language models, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the dense fine-tuning and inference method for a pre-trained language model as described in any one of claims 1 to 6.

8. A dense state fine-tuning and inference product for a pre-trained language model, comprising computer program instructions, characterized in that: When the computer program instructions are executed on a computer, the computer is caused to execute the dense fine-tuning and reasoning method for a pre-trained language model according to any one of claims 1 to 6.

Images