A method for ensuring data privacy and verifiable computation results of a nonlinear GELU function

Abstract

The application provides a method for ensuring data privacy and verifiable calculation results of a nonlinear GELU function. The method ensures the transparency of the calculation process by committing the local model parameters and private data on the server side. After the client-side encrypts the local private data, it is sent to the server for GELU operation to ensure the security of the data during transmission. The client verifies the validity of the server commitment to ensure the reliability of the server data processing. After receiving the GELU operation result, the client decrypts it and verifies the validity of the nonlinear function GELU operation result and the prediction result. The cloud server of the application provides powerful storage and computing capacity to support large-scale model inference services. The client is responsible for encrypting local data and sending data to the cloud server for inference and prediction. Through this method, user data privacy can be effectively protected, and the correctness and verifiability of the calculation results can be ensured.

Description

A method to ensure data privacy and verifiable computation results for nonlinear GELU functions Technical Field

[0001] This invention belongs to the field of data processing technology, and further relates to information security technology, specifically a method for ensuring data privacy and verifiability of calculation results for nonlinear GELU functions. Background Technology

[0002] With the advent of the big data era, people's lives are now inextricably linked to data, making data privacy and security issues increasingly prominent. Ensuring efficient data use while guaranteeing data security is a crucial problem to solve. In large-scale model inference scenarios, clients often outsource local data to the server for inference on their private data. Data owners may need to outsource their local data to untrusted cloud servers, and client data, being sensitive and containing a large amount of user privacy information, can be problematic. When performing privacy-preserving inference on user private data in the cloud, the correctness of the inference results is often difficult to verify. For example, to save local computing resources, the server can forge false inference results and send them to the client, making it impossible for the client to verify the validity of the results. Designing an efficient privacy-preserving and verifiable nonlinear function, GELU (Gaussian Error Linear Unit), can effectively guarantee client data security and the verifiability of inference results. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a method for ensuring data privacy and verifiable computation results of nonlinear GELU functions. This method can be used in large language model transformer architectures to implement nonlinear GELU function operations on the server side while ensuring client data security and privacy, and to ensure the verifiable prediction results of matrix GELU functions.

[0004] This invention is achieved through the following technical solution:

[0005] A method for ensuring data privacy and verifiable computation results of nonlinear GELU functions is implemented using a server-side module and a client-side module. The server-side module utilizes a cloud server provider for storage and computation, offering large-scale model inference services to users. The client-side module allows users to encrypt their local private data and send it to the cloud server for inference and prediction. The method specifically includes the following steps:

[0006] (1) The server makes a commitment to the local model parameters and private data;

[0007] (2) The client encrypts the local private data and sends it to the server for GELU operation;

[0008] (3) The client verifies the validity of the server's commitment;

[0009] (4) The client decrypts the GELU operation result and verifies the GELU operation result of the nonlinear function to verify the validity of the prediction result.

[0010] Specifically, step (1) generates a commitment value and sends it to the client for the client to verify its validity. Specifically, the server performs a polynomial replacement on the GELU function and commits to the corresponding model parameters; the GELU function expression is: ;

[0011] The server uses the following piecewise function to perform a polynomial approximation of the GELU function:

[0012] ;

[0013] in, For function variables; It is the hyperbolic tangent function;

[0014] Introducing indicator variables , is used to identify the segment interval to which variable t belongs, and its value takes the following rules:

[0015] belong Fragmentation; otherwise, 0;

[0016] Based on indicator variables The piecewise function can be uniformly represented as:

[0017] ;

[0018] ;

[0019] Further organized into information about polynomial form: ;

[0020] The server makes a commitment to the polynomial parameters and generates the corresponding commitment value:

[0021] ;

[0022] in, The function variable t can take the values ​​of 0 and 1. and It is the constant term in the polynomial approximation;

[0023] The server will Send to the client.

[0024] Further, step (2) includes the following: the client encrypts the local private data and sends it to the server for GELU operation; specifically, the client encrypts the local private data X=[ Encryption processing is performed:

[0025] Client-side private matrix X is composed of It consists of several private data elements; the expression is as follows:

[0026] ;

[0027] The client encrypts X using a fully homomorphic encryption algorithm to obtain Enc(X), and then sends Enc(X) to the server:

[0028] .

[0029] Furthermore, step (2) involves sending the data to the server for GELU computation; specifically, this includes the following:

[0030] The server obtains the encrypted data from the client, decompresses the encrypted data, and obtains the encrypted data. The server calls the ciphertext comparison algorithm to calculate :

[0031] ;

[0032] The server calculates the polynomial coefficients based on the comparison results. to The coefficients are determined by combining the comparison results with preset coefficients; the server calculates the polynomial coefficient expression as follows:

[0033] ;

[0034] Based on the polynomial coefficients and the encrypted data, the server calculates the encrypted polynomial result of the GELU function. The server-calculated GELU function polynomial is as follows:

[0035] ;

[0036] The server will Send to the client.

[0037] Furthermore, in step (3), the client verifies the validity of the server's commitment; specifically, the client uses a commitment verification operation to verify the validity of the coefficient commitment, hash value, and random number provided by the server.

[0038] ;

[0039] ;

[0040] ;

[0041] If the above equation equals 1, the client trusts the validity of the server's commitment. If the above equation != 1, meaning that any verification operation does not return a successful result, the client rejects the server's reasoning result.

[0042] Furthermore, in step (4), the client decrypts the GELU operation result and verifies the validity of the nonlinear function GELU operation result, specifically as follows:

[0043] The client receives the encrypted result. Decryption is performed to obtain the calculation result: Dec ( );

[0044] The client compares the original data X with a preset value and obtains the comparison result. ;

[0045] Furthermore, in step (4), the client decrypts the GELU operation result and verifies the validity of the nonlinear function GELU operation result, specifically as follows:

[0046] The client receives the encrypted result. Decryption is performed to obtain the calculation result: Dec ( );

[0047] The client compares the original data X with a preset value and obtains the comparison result. ;

[0048] The client performs hash calculations and verifies the validity of each coefficient in the GELU operation result of the nonlinear function and the final inference result based on homomorphic hashing and commitment verification mechanisms.

[0049] Specifically, the validity of the hash calculation and verification is determined by the following expression:

[0050] ;

[0051] If all the above hash verifications are true, the client considers the GELU operation result returned by the server to be valid; otherwise, the result is considered invalid.

[0052] Specifically, the client and server ensure the transparency of the computation process and provide support for result verification by utilizing a pre-agreed protocol.

[0053] Specifically, the nonlinear function GELU is the Gaussian Error Linear Unit function, used for activation function calculation in deep learning models. When verifying the validity of the nonlinear function GELU operation result, the client compares the decryption result with the value within the expected range to ensure the validity and correctness of the operation result.

[0054] The present invention also provides an electronic device, including a memory and a processor, characterized in that the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the method for ensuring data privacy and verifiable calculation results of the nonlinear GELU function.

[0055] The beneficial effects of this invention are as follows:

[0056] The system of this invention mainly includes two modules: a cloud server and a client. The cloud server provides powerful storage and computing capabilities, supporting large-scale model inference services; the client is responsible for encrypting local data and sending data to the cloud server for inference and prediction. This method not only ensures the correctness and verifiability of the calculation results but also effectively protects user data privacy and security. Attached Figure Description

[0057] Figure 1 is a flowchart of the verification method of the present invention;

[0058] Figure 2 is a schematic diagram of the module reasoning scenario of the present invention. Detailed Implementation

[0059] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0060] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0061] Referring to Figures 1 and 2, the application scenarios of the method of the present invention include clients and cloud servers. The functions of each entity are described below:

[0062] Client U: The client has its own local dataset, and the client's local data is private.

[0063] Cloud server: The cloud server has local model parameters, which are private.

[0064] This invention is a technical solution for ensuring user data privacy and security, as well as verifying the calculation results of the nonlinear function GELU operation. This method ensures the verification of the nonlinear function GELU operation results while guaranteeing user data security and privacy.

[0065] To achieve the above objectives, the technical solution of the present invention includes four steps: (1) the server makes a commitment to the private data of local model parameters; (2) the client encrypts the local private data and sends it to the server for GELU operation; (3) the client verifies the data commitment of the server and verifies the validity of the commitment; (4) the client decrypts the GELU operation result, verifies the GELU operation result of the nonlinear function, and verifies the validity of the prediction result.

[0066] This invention mainly consists of two modules: a cloud server provider, which possesses strong storage and computing capabilities to provide users with large-scale model inference services; and a client, where users encrypt their local private data and send it to the cloud server for inference and prediction.

[0067] The specific steps of this invention are as follows:

[0068] Step 1: The server performs polynomial replacement on the GELU function and commits to the corresponding model parameters. The GELU function expression is:

[0069] ;

[0070] The server uses the following piecewise function to approximate the polynomial:

[0071] ;

[0072] in, For function variables; It is the hyperbolic tangent function;

[0073] Introducing indicator variables , is used to identify the segment interval to which variable t belongs, and its value takes the following rules: belong Fragmentation; otherwise, 0;

[0074] Based on indicator variables The piecewise function can be uniformly represented as:

[0075] ;

[0076] ;

[0077] Further organized into information about polynomial form:

[0078] ;

[0079] The server makes a commitment to the polynomial parameters and generates the corresponding commitment value:

[0080] ;

[0081] in, The function variable t can take the values ​​of 0 and 1. and It is the constant term in the polynomial approximation;

[0082] The server will Send to the client.

[0083] Step 2: The client processes local private data X=[ Encryption is performed; the private matrix X is... It consists of a private data element;

[0084] The client-side private matrix X expression is as follows:

[0085] ;

[0086] The client encrypts X using a fully homomorphic encryption algorithm to obtain Enc(X), and then sends Enc(X) to the server:

[0087] .

[0088] Step 3: The server obtains the encrypted data from the client, decompresses the encrypted data, and obtains the encrypted data. The server calls the ciphertext comparison algorithm to calculate :

[0089]

[0090] The server calculates the polynomial coefficients based on the comparison results. to The coefficients are determined by combining the comparison results with preset coefficients; the server calculates the polynomial coefficient expression as follows:

[0091] ;

[0092] Based on the polynomial coefficients and the encrypted data, the server calculates the encrypted polynomial result of the GELU function. The server-calculated GELU function polynomial is as follows:

[0093] ;

[0094] The server will Send to the client.

[0095] Step 4: The client uses the commitment verification operation to verify the validity of the commitment provided by the server using the coefficient commitment, hash value, and random number respectively.

[0096] ;

[0097] If the above equation equals 1, the client trusts the validity of the server's commitment. If the above equation !=1, meaning that any verification operation does not return a successful result, the client rejects the server's reasoning result.

[0098] Step 5: The client decrypts the GELU inference result and verifies its validity.

[0099] The client receives the encrypted result. Decryption is performed to obtain the calculation result: Dec ( ).

[0100] The client compares the original data X with the preset value. Size obtained .

[0101] The client uses homomorphic hashing and commitment verification mechanisms to perform hash calculations and verify the validity of each coefficient in the GELU operation result of the nonlinear function and the final inference result.

[0102] ;

[0103] If all the above hash verifications are true, the client considers the GELU operation result returned by the server to be valid; otherwise, the result is considered invalid.

[0104] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the method described above for ensuring data privacy and verifiability of calculation results for nonlinear GELU functions; and provides an electronic device including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the method described above for ensuring data privacy and verifiability of calculation results for nonlinear GELU functions.

[0105] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0107] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for ensuring data privacy and verifiable computation results of nonlinear GELU functions, characterized in that, This method utilizes a server-side module and a client-side module. The server-side module is a cloud server provider used for storage and computation, providing users with large model inference services. The client-side module allows users to encrypt their local private data and send it to the cloud server for inference and prediction. The method specifically includes the following steps: (1) The server makes a commitment to the local model parameters and private data; generates a commitment value and sends it to the client for the client to verify its validity. Specifically, the server performs a polynomial replacement on the GELU function and makes a commitment to the corresponding model parameters; the GELU function expression is: The server uses the following piecewise function to perform a polynomial approximation of the GELU function: Where t is a function variable; tanh() is the hyperbolic tangent function; and an indicator variable b is introduced. i b is used to identify the segment interval to which variable t belongs, and its value takes the following rules: i =1 if t belongs to the i-th segment; otherwise 0; based on the indicator variable b i The piecewise function can be uniformly represented as: Further rearranged into a polynomial form in t: GELU(t) = a0 + a1t 2 +a2t 2 +a3t 3 +a4t 4 +a5t 5 +a6t 6 The server makes commitments to the polynomial parameters and generates corresponding commitment values. Among them, b i The function variable t can take the values ​​0 and 1; c i and d i It is the constant term in the polynomial approximation; the server will... (1) Send to the client; (2) The client encrypts the local private data and sends it to the server for GELU operation; (3) The client verifies the validity of the server's commitment; (4) The client decrypts the GELU operation result and verifies the nonlinear function GELU operation result to verify the validity of the prediction result.

2. The method for ensuring data privacy and verifiability of computation results for nonlinear GELU functions according to claim 1, characterized in that, The client encrypts its local private data and sends it to the server for GELU computation; specifically, the client encrypts the local private data X = [x1, x2, ..., x...]. n Encryption processing is performed: The client's private matrix X consists of n private data elements; the expression is as follows: The client encrypts X using a fully homomorphic encryption algorithm to obtain Enc(X), and then sends Enc(X) to the server: Enc(X1)=[Enc(x 1,0 ),Enc(x 1,1 ),…,Enc(x 1,m-1 )]…Enc(X n-1 )=[Enc(x n-1,0 ),Enc(x n-1,1 ),…,Enc(x n-1,m-1 )]。 3. The method for ensuring data privacy and verifiability of computation results for nonlinear GELU functions according to claim 2, characterized in that, The step of sending data to the server for GELU computation specifically includes the following: the server receives the encrypted data from the client, the server decompresses the encrypted data, and obtains the encrypted data Enc(x). ij The server calls the ciphertext comparison algorithm to calculate b0, b1, b2, b3: b0, b1, b2, b3 ← comp(Enc(x), -4, -1.95, 3); the server calculates the polynomial coefficients a0 to a6 based on the comparison result, where each coefficient is determined by combining the comparison result with preset coefficients; the server calculates the polynomial coefficient expression as follows: Based on the polynomial coefficients and the encrypted data, the server calculates the encrypted polynomial result of the GELU function. The server-calculated GELU function polynomial is as follows: The server sends Enc(GELU(X)) to the client.

4. The method for ensuring data privacy and verifiability of computation results for nonlinear GELU functions according to claim 3, characterized in that, The client verifies the validity of the server's commitment; specifically, the client uses commitment verification operations to verify the validity of the coefficient commitment, hash value, and random number provided by the server. If the above equation equals 1, the client trusts the validity of the server's commitment; if the above equation ! = 1, meaning that any verification operation does not return a successful result, the client rejects the server's reasoning result.

5. The method for ensuring data privacy and verifiability of computation results for nonlinear GELU functions according to claim 4, characterized in that, The client decrypts the GELU operation result and verifies the validity of the nonlinear function GELU operation result. Specifically, the client decrypts the received encrypted result Enc(GELU(X)) to obtain the calculation result: GELU(X)=Dec(Enc(GELU(X))). The client compares the original data X with the preset values ​​to obtain the comparison results b0, b1, b2, b3; the client performs hash calculations and verifies the validity of each coefficient in the nonlinear function GELU operation result and the final inference result based on the homomorphic hash and commitment verification mechanism.

6. The method for ensuring data privacy and verifiability of computation results for nonlinear GELU functions according to claim 5, characterized in that, The validity of the hash calculation and verification is determined by the following expression: If all the above hash verifications are true, the client considers the GELU operation result returned by the server to be valid; otherwise, the result is considered invalid.

7. The method for ensuring data privacy and verifiability of computation results for nonlinear GELU functions according to claim 1, characterized in that, The client and server use a pre-agreed protocol to ensure the transparency of the computation process and to provide support for result verification.

8. The method for ensuring data privacy and verifiability of computation results for nonlinear GELU functions according to claim 1, characterized in that, The nonlinear function GELU is the Gaussian Error Linear Unit function, used for activation function calculation in deep learning models. When verifying the validity of the GELU operation result, the client compares the decryption result with the value within the expected range to ensure the validity and correctness of the operation result.

9. An electronic device comprising a memory and a processor, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the method for ensuring data privacy and verifiable calculation results of the nonlinear GELU function as described in any one of claims 1-8.

Images