Pre-training language model-oriented secret state fine tuning and reasoning method, device and product

Abstract

The invention discloses a pre-training language model-oriented secret state fine tuning and reasoning method, device and product. A participant comprises a user and a service provider; the service provider has a pre-training language model and participates in secret state fine tuning and secret state reasoning, and parameters of the pre-training language model are privacy of the service provider; the user has input and output for fine tuning of a data set and reasoning and is private data; in the secret state fine tuning, on the premise that a pre-training language model of a service provider and a fine tuning data set of a user are protected, fine tuning is performed on the language model, and the user obtains a non-frozen part of the fine tuning model; according to the secret state reasoning, on the premise that user reasoning input and output are protected, a service provider conducts reasoning of a fine-tuning model freezing layer, an obtained reasoning result is delivered to a user, and the user conducts reasoning of a non-freezing layer and obtains a reasoning result. According to the invention, the time and communication overhead of fine tuning and reasoning of the secret state are reduced, and the accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the field of information security technology, and relates to a method, device and product for dense state fine-tuning and reasoning of a pre-trained language model, and specifically to a method, device and product for fine-tuning and reasoning of a pre-trained language model with low overhead and high accuracy while protecting the privacy of service providers and users. Background Art

[0002] Pre-trained language models have shown excellent performance in many fields. A common way to use these models is fine-tuning, which meets the need to customize new models without a lot of computing resources. In practice, platforms like ChatGPT provide remote APIs for fine-tuning, allowing users to upload their training data to train and deploy customized models for inference.

[0003] Despite their success, such fine-tuning and inference services may infringe on user privacy. For example, social data often contains sensitive information about individuals, which users do not want to disclose to service providers. In addition, service providers can access user queries and predictions in plain text during inference, which is not suitable for applications such as medical diagnosis. Although service providers can provide pre-trained models to users for local fine-tuning and inference, this will compromise their intellectual property rights and hide various attacks.

[0004] Existing privacy protection schemes for pre-trained language models are costly. There are two main reasons: On the one hand, in dense inference on deep neural networks, security protocols are called layer by layer. However, language models have more complex architectures and larger parameters. In particular, they require multiple calls to the matrix multiplication protocol. On the other hand, existing studies use an approximation of the Gaussian Error Linear Unit (GELU), a unique nonlinear function in language models. However, using a uniform piecewise linear approximation without considering curvature results in unnecessary costs for low-curvature segments and precision loss for high-curvature segments. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a fine-tuning and reasoning method, device and product with low overhead, high accuracy and privacy protection for service providers and users.

[0006] The technical solution adopted by the method of the present invention is: a dense state fine-tuning and reasoning method for a pre-trained language model, and the participating entities include users and service providers;

[0007] The language model is formed by stacking multiple transformer modules, each transformer module is divided into a linear layer and a nonlinear layer, the linear layer includes an embedding layer, a matrix multiplication layer and a fully connected layer, and the nonlinear layer includes a normalized exponential function layer Softmax layer, a Gaussian error linear unit layer GELU layer and a layer normalization layer LayerNorm layer;

[0008] The service provider owns the pre-trained language model and participates in the dense fine-tuning and dense inference. The parameters of the pre-trained language model are the privacy of the service provider. The user owns the fine-tuning dataset and the input and output of the inference, which are private data.

[0009] The secret fine-tuning is to fine-tune the language model under the premise of protecting the pre-trained language model of the service provider and the fine-tuning dataset of the user, and the user obtains the non-frozen part of the fine-tuning model;

[0010] The dense inference is performed by the service provider on the frozen layer of the fine-tuned model under the premise of protecting the user's inference input and output, and the obtained inference result is delivered to the user, who performs inference on the non-frozen layer and obtains the inference result.

[0011] Preferably, the dense state fine-tuning repeatedly executes four protocols including a secure calculation protocol of forward propagation and derivatives, a secure calculation protocol of derivatives of loss functions, a secure calculation protocol of back propagation, and an optimizer protocol until the language model converges to obtain a fine-tuning model, and the service provider delivers its share of the unfrozen part of the fine-tuning model to the user. At this time, the user owns the unfrozen part of the fine-tuning model, and the service provider owns the frozen part of the fine-tuning model.

[0012] The secure computation protocol of the forward propagation and derivatives, the user inputs the matrix X, and the service provider inputs the pre-trained language model M P , output the result share of forward propagation <z>And the derivative of each layer of the neural network <d>;

[0013] The derivative of the loss function is securely calculated using the user input matrix Y, and the user and service provider input forward propagation result share <z>, output share in represents the derivative of the cross entropy loss function;

[0014] The secure computation protocol of back propagation, the user and service provider input matrix <d>, output gradient share <g>;

[0015] The optimizer protocol, user and service provider input matrix <g>and the current language model, output the updated language model <M T >.

[0016] As a preference, the dense state reasoning freezes part of the reasoning and uses the forward propagation protocol to calculate the result share. <z>0 and <z>1, <>0 means the share belongs to the user, <>1 means it belongs to the service provider, then the service provider sends <z>1 to the user; for the reasoning of the unfrozen part, the user inputs Z and the final prediction result is calculated locally.

[0017] As a preference, the linear layer calculation relies on the matrix multiplication protocol M2OFL(X,W), where the user inputs the matrix X, the service provider inputs the matrix W, and outputs the secret sharing shares<X·W> , where · represents matrix multiplication;

[0018] The linear computation Q·K in the first transformer module is frozen T =X·W·X T ,in W w , W p is the embedding layer parameter owned by the service provider, W Q , W K are the fully connected layer parameters owned by the service provider;

[0019] For the first transformer module freeze-friendly linear computation safety protocol EMM(), the specific implementation includes offline phase steps A1-A3 and online phase steps A4-A6:

[0020] Step A1: The user generates a random matrix R;

[0021] Step A2: The user and the service provider use the matrix multiplication protocol M3OFL(R,W) and obtain the secret sharing share <R·W·R T >

[0022] Step A3: The user and the service provider use the matrix multiplication protocol M2OFL(R,W), M2OFL(R,W T ) and obtain the secret share<R·W> , <R·W T >

[0023] Step A4: The user sends the XR to the service provider;

[0024] Step A5: User calculates and obtains share <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0025] Step A6: The service provider calculates and obtains the share Among them, Q·K T = <Q·K T >0+ <Q·K T >1, <>0 means that the share belongs to the user, <>1 means that the share belongs to the service provider, and <> means <>0 and <>1.

[0026] As a preference, linear computations in subsequent transformer modules are frozen in ∈, ζ are the parameters of the LayerNorm layer owned by the service provider, W Q , W K is the parameter of the attention layer owned by the service provider, X * It is the intermediate result of the LayerNorm layer. <X # >0= <X * >0 owned by user, Owned by the service provider, <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0027] The specific implementation of the freeze-friendly linear computation safety protocol for the subsequent transformer module includes offline stages B1-B2 and online stages B3-B5:

[0028] Step B1: User and service provider use EMM (W * ) offline phase to obtain R # , Where R # =R;

[0029] Step B2: User and service provider use matrix multiplication protocol and obtain the secret share

[0030] Step B3: User and service provider use EMM (W * ) online phase to obtain < <X # >0· Service providers also gain <X # >0-R # ;

[0031] Step B4: User calculates and obtains share <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0032] Step B5: The service provider calculates and obtains the share Among them, Q·K T = <Q·K T >0+ <Q·K T >1.

[0033] Preferably, in step A2, the matrix multiplication protocol M3OFL(R,W), the user inputs the matrix R, and the service provider inputs the matrix W to obtain the secret sharing share <R·W·R T >

[0034] The specific implementation of the calculation includes the following steps:

[0035] Step C1: The user follows m w ×n w 、n w ×m w The window specification of the matrix R with a specification of m×n and the matrix R with a specification of n×m T Divide into small matrix blocks R α,β , Among them, α, δ∈[m′], The subscript w indicates window;

[0036] Step C2: User passes R α,β , Encoded as a polynomial; π A , π C Represents the mapping function from matrix to polynomial;

[0037] Step C3: The service provider follows n w ×n w Divide the n×n matrix W into small matrix blocks W β,γ and through W β,γ Encoded as a polynomial, π B Represents the mapping function from matrix to polynomial;

[0038] Step C4: User sends The ciphertext to service providers;

[0039] Step C5: Service Provider Calculation in is a homomorphic addition calculation, It is a homomorphic multiplication calculation;

[0040] Step C6: The service provider uses Clean(a 0,α,γ ,n w ) to obtain in n w The coefficients and a corresponding to the integer multiples of the terms 0,α,γ The coefficients of the other terms are 0;

[0041] Step C7: Service Provider Calculation and recover <R·W·R 3,α,δ from S T >1, where is the homomorphic subtraction calculation, and S 3,α,δ is a masked polynomial randomly generated by the service provider;

[0042] Step C8: If m w ≥n w , the service provider sends a 3,α,δ , otherwise, every n w / m w polynomials are combined into one and sent out;

[0043] Step C9: The user decrypts and recovers <R·W·R T >0 from the plaintext.

[0044] Preferably, the matrix is encoded into a function of polynomials where In addition, if i = 1 and j ∈ [2, n], calculate If i ∈

[0045] [2, m] and j ∈ [n], calculate N is the degree of the polynomial;

[0046] The matrix is encoded into a function of polynomials where, if i ∈ [n] and j ∈ [n], calculate

[0047] The matrix is encoded into a function of polynomials When m < n, if i = 1 and j ∈ [m], calculate If i ∈ [2, n] and j ∈ [m], calculate When m ≥ n, if i = 1 and j ∈ [n], or i ∈ [n] and j ∈ [n + 1, m], calculate If i ∈ [2, n] and j ∈ [n], calculate

[0048] Preferably, for the GELU(x) layer of the non-linear layer, the user input share <x>0, service provider input share <x>1. To get a share <y>,<y′> , wherein y=GELU(x), y′ represents the derivative of y;

[0049] The specific implementation of the calculation includes the following steps:

[0050] Step D1: User and service provider local computing in i∈[6], L is the number of bits of the language model left shift, t i is the segmentation when the GELU(x) curve is approximated, <>0 means that the share belongs to the user, <>1 means that the share belongs to the service provider, and <> means <>0 and <>1;

[0051] Step D2: User and Service Provider Extraction <x i ′> low To obtain is a hyperparameter;

[0052] Step D3: User and service provider use phase agreement calculation Interception High S i Position i∈[3],Π tr To cut off the agreement;

[0053] Step D4: The user and the service provider use the lookup table protocol to calculate i∈[3], and set <w4>= <1> , Π lit is a lookup table protocol, <w i >,<η i > is stored in the lookup table T i The shares of the coefficients and bias values ​​of the linear approximation in ;

[0054] Step D5: The user and the service provider use the multiplication protocol to calculate <w i x>=Π mut ( <w i >, <x>), i∈[3], Π mut It is a multiplication protocol;

[0055] Step D6: User and service provider compute locally i >= <w i x>+<η o >, and set <u4> ＝ <x>;

[0056] Step D7: The user and the service provider use the comparison protocol to calculate i∈[6],Π cmp is the comparison protocol, <ρ i > means <ρ o >0 and <ρ i >1, when When i =1, otherwise 0, where Indicates XOR calculation;

[0057] Step D8: User and service provider local computing in Indicates XOR calculation and sets <z4>= <ρ6>;

[0058] Step D9: The user and the service provider use the multiplexer protocol to calculate <v i >=Π mux ( <z i >, i >), <v′ i >=Π mux ( <z i >, <w i >), then calculate and output locally <y>=∑ <v i >、<y′> =∑ <v′ i >, i∈[4], Π mux For a multi-way selection protocol, when z i =1 when output u i , when z i =0, output 0, <v i >、 <v′ i >Share of the results of the agreement.

[0059] The technical solution adopted by the device of the present invention is: a dense state fine-tuning and reasoning device for a pre-trained language model, comprising:

[0060] one or more processors;

[0061] A storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the dense state fine-tuning and reasoning method for the pre-trained language model.

[0062] The technical solution adopted by the product of the present invention is: a dense state fine-tuning and reasoning product for a pre-trained language model, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the dense state fine-tuning and reasoning method for a pre-trained language model.

[0063] Compared with the prior art, the beneficial effects of the present invention include:

[0064] The present invention designs a dense state fine-tuning and reasoning system for pre-trained language models, in which the user's data and the service provider's pre-trained model are protected, and the scheme reduces time and communication overhead and improves accuracy. By proposing a freezing-friendly matrix multiplication protocol, the protocol optimizes the frozen layer of the transformer model as a whole, thereby reducing the computational and communication costs and complexity. For calculating Gaussian error linear units (GELU), a derivative-friendly GELU protocol is proposed, which not only achieves higher accuracy and lower cost, but also makes it easy to derive derivatives during the fine-tuning process. Compared with previous schemes, the present invention achieves low time, communication overhead, and high accuracy for dense state fine-tuning and reasoning for pre-trained language models. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The technical solution of this invention is further described below using embodiments and specific implementation methods. In addition, some drawings are also used in the process of describing the technical solution. For those skilled in the art, other drawings and the intention of the invention can also be obtained based on these drawings without paying creative labor.

[0066] Figure 1 A schematic diagram of a method according to an embodiment of the present invention;

[0067] Figure 2 A flow chart of the dense state fine-tuning according to an embodiment of the present invention;

[0068] Figure 3 An example diagram of a matrix coding layer polynomial in an embodiment of the present invention;

[0069] Figure 4 This is an example diagram of the GELU approximation scheme of an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to facilitate the understanding and implementation of the present invention by those skilled in the art, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0071] Please see Figure 1 ,This embodiment provides a dense state fine-tuning and reasoning method for a pre-trained language model, and the participating entities include users and service providers;

[0072] The language model is formed by stacking multiple transformer modules, each transformer module is divided into a linear layer and a nonlinear layer, the linear layer includes an embedding layer, a matrix multiplication layer and a fully connected layer, and the nonlinear layer includes a normalized exponential function layer Softmax layer, a Gaussian error linear unit layer GELU layer and a layer normalization layer LayerNorm layer;

[0073] The service provider owns the pre-trained language model and participates in the dense fine-tuning and dense inference. The parameters of the pre-trained language model are the privacy of the service provider. The user owns the fine-tuning dataset and the input and output of the inference, which are private data.

[0074] The secret fine-tuning is to fine-tune the language model under the premise of protecting the pre-trained language model of the service provider and the fine-tuning dataset of the user, and the user obtains the non-frozen part of the fine-tuning model;

[0075] The dense inference is performed by the service provider on the frozen layer of the fine-tuned model under the premise of protecting the user's inference input and output, and the obtained inference result is delivered to the user, who performs inference on the non-frozen layer and obtains the inference result.

[0076] Please see Figure 2 In one embodiment, the dense state fine-tuning repeatedly executes four protocols including a forward propagation and derivative secure calculation protocol, a loss function derivative secure calculation protocol, a back propagation secure calculation protocol, and an optimizer protocol until the language model converges to obtain a fine-tuning model, and the service provider delivers its share of the unfrozen part of the fine-tuning model to the user. At this time, the user owns the unfrozen part of the fine-tuning model, and the service provider owns the frozen part of the fine-tuning model.

[0077] In one embodiment, for the secure computation protocol of forward propagation and derivatives, the user inputs the matrix X and the service provider inputs the pre-trained language model M P , output the result share of forward propagation <z>And the derivative of each layer of the neural network <d>.

[0078] In one embodiment, for the secure computation of forward propagation and derivatives, the present invention provides a freeze-friendly matrix multiplication protocol M2OFL(X,W) and a derivative-friendly GELU computation protocol.

[0079] In one embodiment, the freeze-friendly matrix multiplication protocol M2OFL(X,W) includes an online|offline sub-protocol for the first transformer module and an online|offline sub-protocol for subsequent transformer modules;

[0080] For the online|offline sub-protocol of the first transformer module, the specific calculation steps are as follows:

[0081] Step 1: The user generates a random matrix R;

[0082] Step 2: The user and the service provider use the matrix multiplication protocol M3OFL(R,W) and obtain the secret sharing share <R·W·R T >

[0083] Step 3: The user and the service provider use the matrix multiplication protocol M2OFL(R,W), M2OFL(R,W T ) and obtain the secret share<R·W> , <R·W T >

[0084] Step 4: The user sends XR to the service provider (- is a subtraction operation);

[0085] Step 5: User calculates and gets share <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0086] Step 6: The service provider calculates and obtains the share Among them, Q·K T = <Q·K T >0+ <Q·K T >1, <>0 means the share belongs to the user, <>1 means it belongs to the service provider, <> means <>0 and <>1;

[0087] For the linear calculations in the subsequent transformer modules that are frozen Where W * =∈ 2 W Q ·W K , W Q , W K are the parameters of the attention layer owned by the service provider, ∈, ζ are the parameters of the LayerNorm layer owned by the service provider, and X * It is the intermediate result of the LayerNorm layer. <X # >0= <X * >0 owned by user, Owned by the service provider.

[0088] In one implementation, for the online|offline sub-protocol of the subsequent transformer module, the specific implementation calculation steps include:

[0089] Step 1: Users and service providers use EMM (W * ) offline phase to obtain Where R # =R;

[0090] Step 2: Users and service providers use the matrix multiplication protocol M2OFL (R # ,W1 * ), and obtain the secret share Represents matrix multiplication;

[0091] Step 3: Users and service providers use EMM (W * ) online phase to obtain < <X # >0· Service providers also gain <X # >0-R # ;

[0092] Step 4: User calculates and gets share <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1;

[0093] Step 5: The service provider calculates and obtains the share Among them, Q·K T = <Q·K T >0+ <Q·K T >1.

[0094] Please see Figure 3 This embodiment provides a matrix multiplication protocol M3OFL(R,W), where the user inputs the matrix R and the service provider inputs the matrix W to obtain the secret sharing share. <R·W·R T >, the specific calculation steps include:

[0095] Step 1: The user clicks m w ×n w 、n w ×m w The window specification of the matrix R with a specification of m×n and the matrix R with a specification of n×m T Divide into small matrix blocks R α,β , Among them, α, δ∈[m′], β, γ∈[n′], The subscript w indicates window;

[0096] Step 2: User passes R α,β , Encoded as a polynomial; π A , π C Represents the mapping function from matrix to polynomial;

[0097] Step 3: The service provider follows n w ×n w Divide the n×n matrix W into small matrix blocks W β,γ and through W β,γ Encoded as a polynomial; v B Represents the mapping function from matrix to polynomial;

[0098] Step 4: User sends The ciphertext to service providers;

[0099] Step 5: Service Provider Calculation in is a homomorphic addition calculation, It is a homomorphic multiplication calculation;

[0100] Step 6: Service provider uses Clean(a 0,α,γ ,n w ) to obtain in n w The coefficients and a corresponding to the integer multiple terms 0,α,γ The coefficients of the other terms are 0;

[0101] Step 7: Service Provider Calculation And from S 3,α,δ Recover from <R·W·R T >1, where is the homomorphic subtraction calculation, S 3,α,δ is a mask polynomial randomly generated by the service provider;

[0102] Step 8: If m w ≥n w , the service provider sends a 3,α,δ , otherwise, every n w / m w polynomials are combined into one and then sent out;

[0103] Step 9: The user decrypts and recovers <R·W·R T >0 from the plaintext.

[0104] In one implementation, for the matrix chain multiplication protocol, this embodiment provides an encoding function for matrix-encoded polynomials:

[0105] Function for encoding a matrix into a polynomial where Additionally, if i = 1 and j ∈ [2, n], calculate If i ∈ [2, m] and j ∈

[0106] [n], calculate N is the degree of the polynomial;

[0107] Function for encoding a matrix into a polynomial where, if i ∈ [n] and j ∈ [n], calculate

[0108] Function for encoding a matrix into a polynomial When m < n, if i = 1 and j ∈ [m], calculate If i ∈ [2, n] and j ∈ [m], calculate When m ≥ n, if i = 1 and j ∈ [n], or i ∈ [n] and j ∈ [n + 1, m], calculate If i ∈

[0109] [2, n] and j ∈ [n], calculate

[0110] For the non-linear layer GELU(x), this embodiment provides a derivative-friendly GELU secure computing protocol, and the user inputs shares <x>0, service provider input share <x>1. To get a share <y>,<y′> , where y=GELU(x), y′ represents the derivative of y, and the specific calculation steps include:

[0111] Step 1: User and service provider local computing in i∈[6], L is the number of bits of the model left shift, t i is the segmentation when the GELU(x) curve is approximated, <>0 means that the share belongs to the user, <>1 means that the share belongs to the service provider, and <> means <>0 and <>1;

[0112] Step 2: User and Service Provider Extraction <x′ i > Low To obtain i∈[3], is a hyperparameter;

[0113] Step 3: User and service provider use phase agreement to calculate Interception High S i Position i∈[3],Π tr To cut off the agreement;

[0114] Step 4: The user and service provider use the lookup table protocol to calculate i∈[3], and set <w4>= <1> , Π lut is a lookup table protocol, <w i >,<η i > is stored in the lookup table T i The shares of the coefficients and bias values ​​of the linear approximation in ;

[0115] Step 5: User and service provider use multiplication protocol to calculate <w i x>=Π mut ( <w i >, <x>), i∈[3], Π mut It is a multiplication protocol;

[0116] Step 6: Users and service providers compute locally i >= <w i x>+<η i >, and set <u4> ＝ <x>;

[0117] Step 7: User and service provider use comparison protocol calculation i∈[6],Π cmp is the comparison protocol, <ρ i > means <ρ i >0 and <ρ i >1, when When i =1, otherwise 0, where Indicates XOR calculation;

[0118] Step 8: User and Service Provider Local Computing Where ⊕ represents XOR calculation, and sets <z4>= <ρ6>;

[0119] Step 9: User and service provider use multiplexer protocol calculation <v i >=Π mux ( <z i >, i >), <v′ i >=Π mux ( <z i >, <w i >), then calculate and output locally <y>=∑ <v i >、<y′> =∑ <v′ i >, i∈[4], Π mux For a multi-way selection protocol, when z i =1 when output u i , when z i =0, output 0, <v i >、 <v′ i >Share of the results of the agreement;

[0120] Please see Figure 4 , is a diagram showing an example of a GELU approximation scheme according to an embodiment of the present invention. Figure 4 It can be seen from the figure that this implementation divides the curve into five parts according to the curvature of the function.

[0121] This embodiment also provides a dense state fine-tuning and reasoning device for a pre-trained language model, including:

[0122] one or more processors;

[0123] A storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the dense state fine-tuning and reasoning method for the pre-trained language model.

[0124] This embodiment also provides a dense state fine-tuning and reasoning product for a pre-trained language model, including computer program instructions. When the computer program instructions are executed on a computer, the computer executes the dense state fine-tuning and reasoning method for a pre-trained language model.

[0125] The present invention is further described below through specific experiments. Experiments were conducted on the BERT and GPT2 models, and the WNLI, CoLA, and SST-2 datasets were selected. By comparing with the current best performing Iron and CipherGPT, the improvement of the present invention in time and communication overhead was verified. By comparing with the current best performing Iron, CipherGPT, and plain text, the improvement of the present invention in accuracy was verified.

[0126] Table 1 Comparative experimental results of the present invention in terms of time and communication overhead

[0127]

[0128] Table 2 Comparative experimental results of the present invention on accuracy

[0129]

[0130] The present invention can provide:

[0131] 1. A low-overhead, high-accuracy, and private-protection system for pre-trained language models that protects model and user privacy. Neither model nor user data is leaked in this process.

[0132] 2. Freeze-friendly matrix multiplication protocol: This protocol optimizes the frozen layers of the transformer model as a whole, thereby reducing computational and communication costs and complexity.

[0133] 3. Derivative-friendly GELU computation protocol: This protocol not only achieves higher accuracy and lower cost, but also makes it easy to derive derivatives during fine-tuning.

[0134] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0135] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.< / y> ​ < / x> < / u4> ​< / x> < / y> < / x> < / x> < / d> < / z> < / y> ​ < / x> < / u4> ​< / x> < / y> < / x> < / x> < / z> < / z> < / z> < / g> < / g> < / d> < / z> < / d> < / z>

Claims

1. A method for fine-tuning and reasoning a pre-trained language model, wherein the participants include users and service providers; the method is characterized by: The language model is formed by stacking multiple transformer modules, each transformer module is divided into a linear layer and a nonlinear layer, the linear layer includes an embedding layer, a matrix multiplication layer and a fully connected layer, and the nonlinear layer includes a normalized exponential function layer Softmax layer, a Gaussian error linear unit layer GELU layer and a layer normalization layer LayerNorm layer; The service provider owns a pre-trained language model and participates in dense fine-tuning and dense inference. The parameters of the pre-trained language model are private to the service provider. The user owns a fine-tuning dataset and the input and output of the inference, which are private data. The secret fine-tuning is to fine-tune the language model under the premise of protecting the pre-trained language model of the service provider and the fine-tuning dataset of the user, and the user obtains the non-frozen part of the fine-tuning model; The dense inference is performed by the service provider on the frozen layer of the fine-tuned model under the premise of protecting the user's inference input and output, and the obtained inference result is delivered to the user, who performs inference on the non-frozen layer and obtains the inference result.

2. The dense state fine-tuning and reasoning method for a pre-trained language model according to claim 1, characterized in that: The dense state fine-tuning repeatedly executes four protocols including the secure calculation protocol of forward propagation and derivatives, the secure calculation protocol of the derivatives of the loss function, the secure calculation protocol of back propagation, and the optimizer protocol until the language model converges to obtain a fine-tuning model, and the service provider delivers the share of the unfrozen part of the fine-tuning model it owns to the user. At this time, the user owns the unfrozen part of the fine-tuning model, and the service provider owns the frozen part of the fine-tuning model; The secure computation protocol of the forward propagation and derivatives, the user inputs the matrix X, and the service provider inputs the pre-trained language model M P , output the result share of forward propagation <z>and the derivatives of each layer of the neural network <d>, where the meaning of share <> is the result share of the forward propagation <z>For example, <z>express <z>0 and <z> 1， <z> 0+ <z> 1=Z, <> subscript 0 means the share belongs to the user, <> subscript 1 means the share belongs to the service provider;< / z> < / z> < / z> < / z> < / z> < / z> < / d> < / z> The derivative of the loss function is securely calculated using the user input matrix Y, and the user and service provider input forward propagation result share <z>, output share in represents the derivative of the cross entropy loss function;< / z> The secure computation protocol of back propagation, the user and service provider input matrix <d>, output gradient share <g> ；< / g> < / d> The optimizer protocol, user and service provider input matrix <g>and the current language model, output the updated language model <M T >.< / g> 3. The dense state fine-tuning and reasoning method for a pre-trained language model according to claim 2, characterized in that: The dense reasoning freezes part of the reasoning and uses the forward propagation protocol to calculate the result share <z>0 and <z>1, <>0 means the share belongs to the user, <>1 means it belongs to the service provider, then the service provider sends <z> 1 to the user; for the reasoning of the unfrozen part, the user inputs Z and the final prediction result is calculated locally.< / z> < / z> < / z> 4. The dense state fine-tuning and reasoning method for a pre-trained language model according to claim 1, characterized in that: The linear layer calculation relies on the matrix multiplication protocol M2OFL(X,W), where the user inputs the matrix X, the service provider inputs the matrix W, and outputs the secret sharing share<X·W> , where · represents matrix multiplication; The linear computation Q·K in the first transformer module is frozen T =X·W·X T ,in W w , W p is the embedding layer parameter owned by the service provider, W Q , W K are the fully connected layer parameters owned by the service provider; For the first transformer module freeze-friendly linear computation safety protocol EMM(), the specific implementation includes offline phase steps A1-A3 and online phase steps A4-A6: Step A1: The user generates a random matrix R; Step A2: The user and the service provider use the matrix multiplication protocol M3OFL(R,W) and obtain the secret sharing share <R·W·R T > Step A3: The user and the service provider use the matrix multiplication protocol M2OFL(R,W), M2OFL(R,W T ) and obtain the secret share<R·W> , <R·W T > Step A4: The user sends the XR to the service provider; Step A5: User calculates and obtains share <>0 means the share belongs to the user, <>1 means it belongs to the service provider; Step A6: The service provider calculates and obtains the share Among them, Q·K T = <Q·K T >0+ <Q·K T >1, <>0 means the share belongs to the user, <>1 means it belongs to the service provider.

5. The dense state fine-tuning and reasoning method for a pre-trained language model according to claim 4, characterized in that: For the linear calculations in the subsequent transformer modules that are frozen in ∈, ζ are the parameters of the LayerNorm layer owned by the service provider, W Q , W K is the parameter of the attention mechanism layer owned by the service provider, X * It is the intermediate result of the LayerNorm layer. <X # >0= <X * >0 owned by user, Owned by the service provider, <>0 means the share belongs to the user, <>1 means it belongs to the service provider; The specific implementation of the freeze-friendly linear computation safety protocol for the subsequent transformer module includes offline stages B1-B2 and online stages B3-B5: Step B1: User and service provider use EMM (W * ) offline phase to obtain R # , <R # ·W * >、 <R # ·W *T >, where R # =R; Step B2: User and service provider use matrix multiplication protocol and obtain the secret share and in Step B3: User and service provider use EMM (W * ) online phase, to obtain Service providers also gain <X # >0-R # , <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1; Step B4: User calculates and obtains share <>0 means the share belongs to the user, <>1 means it belongs to the service provider, and <> means <>0 and <>1; Step B5: The service provider calculates and obtains the share Among them, Q·K T = <Q·K T >0+ <Q·K T >1.

6. The dense state fine-tuning and reasoning method for a pre-trained language model according to claim 4, characterized in that: Step A2: the matrix multiplication protocol M3OFL(R,W), the user inputs the matrix R, the service provider inputs the matrix W, to obtain the secret sharing share <R·W·R T > The specific implementation of the calculation includes the following steps: Step C1: The user follows m w ×n w 、n w ×m w The window specification of the matrix R with a specification of m×n and the matrix R with a specification of n×m T Divide into small matrix blocks R α,β , Among them, α, δ∈[m′], β, γ∈[n′], The subscript w indicates the window, n w and m w is a hyperparameter; Step C2: User passes R α,β , Encoded as a polynomial; π A , π C Represents a mapping function from matrix to polynomial; Step C3: The service provider follows n w ×n w Divide the n×n matrix W into small matrix blocks W β,γ and through W β,γ Encoded as a polynomial; π B Represents a mapping function from matrix to polynomial; Step C4: User sends The ciphertext to service providers; Step C5: Service Provider Calculation in is a homomorphic addition calculation, It is a homomorphic multiplication calculation; Step C6: The service provider uses Clean(a 0,α,γ ,n w ) to obtain in n w The coefficients and a corresponding to the integer multiples of the terms 0,α,γ The coefficients of the other terms are 0; Step C7: Service Provider Calculation And from S 3,α,δ Recover from <R·W·R T >1, where is the homomorphic subtraction calculation, S 3,α,δ is a mask polynomial randomly generated by the service provider; Step C8: If m w ≥n w , the service provider sends a 3,α,δ Otherwise, each n w m w polynomials are combined into one and then sent out; Step C9: User decrypts and recovers the plaintext <R·W·R T >0.

7. The dense state fine-tuning and reasoning method for a pre-trained language model according to claim 6, characterized in that: The matrix is ​​encoded as a function of a multinomial in In addition, if i = 1 and j ∈ [2, n], calculate If i∈[2,m] and j∈[n], calculate N is the degree of the polynomial; The matrix is ​​encoded as a function of a multinomial Among them, if i∈[n] and j∈[n], calculate The matrix is encoded into a function of multiple numbers When m < n, if i = 1 and j ∈ [m], then calculate If i ∈ [2, n] and j ∈ [m], then calculate When m ≥ n, if i = 1 and j ∈ [n], or i ∈ [n] and j ∈ [n + 1, m], then calculate If i ∈ [2, n] and j ∈ [n], then calculate 8. The dense state fine-tuning and reasoning method for a pre-trained language model according to claim 1, characterized in that: For the GELU(x) layer of the nonlinear layer, the user input share <x>0, service provider input share <x>1. To get a share <y> ,<y′> , wherein y=GELU(x), y′ represents the derivative of y;< / y> < / x> < / x> The specific implementation of the calculation includes the following steps: Step D1: User and service provider local computing in i∈[6], L is the number of bits of the language model left shift, t i is the segmentation when approximating the GELU(x) curve; Step D2: User and Service Provider Extraction <x′ i > Low To obtain is a hyperparameter; Step D3: The user and the service provider use the phase agreement to calculate Interception High S i Position i∈[3],Π tr To cut off the agreement; Step D4: The user and the service provider use the lookup table protocol to calculate i∈[3], and set <w4>= <1> , Π lut is a lookup table protocol, <w i >,<η i > is stored in the lookup table T i The shares of the coefficients and bias values ​​of the linear approximation in ; Step D5: The user and the service provider use the multiplication protocol to calculate <w i x>=Π mut ( <w i >, <x>), i∈[3], Π mut It is a multiplication protocol;< / x> Step D6: User and service provider compute locally i >= <w i x>+<η i >, and set <u4> ＝ <x> ；< / x> < / u4> ​ Step D7: The user and the service provider use the comparison protocol to calculate i∈[6],Π cmp is the comparison protocol, <ρ i > means <ρ i>0 and <ρ i>1 , <ρ i>0 ⊕<ρ i>1 =ρ i ,when When i =1, otherwise 0, where ⊕ represents XOR calculation; Step D8: User and service provider local computing <z1>=<ρ3>⊕<ρ4>, <z2>=<ρ2>⊕<ρ3>⊕<ρ4>⊕<ρ5>, <z3>=<ρ1>⊕<ρ2>⊕<ρ5>⊕<ρ6>, where ⊕ represents XOR calculation, and sets <z4> = <ρ6>; Step D9: The user and the service provider use the multiplexer protocol to calculate <v i >=Π mux ( <z i >, i >), <v′ i >=Π mux ( <z i >, <w i >), then calculate and output locally <y>=∑ <v i >、<y′> =∑ <v′ i >, i∈[4], Π mux For a multi-way selection protocol, when z i =1 when output u i , when z i =0 when the output is 0, <v i >、 <v′ i > is the output of the protocol.< / y> ​ 9. A dense state fine-tuning and reasoning device for a pre-trained language model, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the dense fine-tuning and reasoning method for a pre-trained language model as described in any one of claims 1 to 8.

10. A dense state fine-tuning and reasoning product for a pre-trained language model, comprising computer program instructions, characterized in that: When the computer program instructions are executed on a computer, the computer executes the dense fine-tuning and reasoning method for a pre-trained language model as described in any one of claims 1 to 8.

Images