Hypothesis testing method and device for privacy-preserving generalized linear models

Through the secret sharing protocol, the hypothesis test of the generalized linear model is carried out in multi-party computing, which solves the privacy protection problem in the prior art and realizes data privacy protection in the hypothesis testing process of the generalized linear model.

CN118504038BActive Publication Date: 2024-11-08ANT GROUP CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202410957854.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-08
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

In the prior art, how to conduct hypothesis test of generalized linear models under the premise of privacy protection to avoid leakage of input feature values ​​and parameter values ​​of model parameters, and the lack of corresponding solutions.

Method used

A secret sharing protocol is used to obtain Heisen matrix and target statistics through joint calculations of multiple parties, reflect the confidence interval of model parameters, and use statistical processing to obtain the probability value of hypothesis test to ensure privacy protection during the calculation process.

Benefits of technology

It realizes hypothesis testing of generalized linear models under the premise of privacy protection, avoids the leakage of input feature values ​​and parameter values ​​of model parameters, and ensures the security of data privacy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118504038B_ABST
    Figure CN118504038B_ABST
Patent Text Reader

Abstract

The embodiments of the present specification provide a hypothesis testing method and device for a privacy-protected generalized linear model. The method includes: any party among multiple parties performs a first multi-party joint calculation with other parties based on its own slice of a feature matrix composed of each input feature value of multiple samples and its own slice of a parameter vector composed of parameter values ​​of each model parameter, to obtain its own slice of a Hessian matrix; the elements of the Hessian matrix are the second-order derivatives of the loss function with respect to the model parameters; based on its own slice of the Hessian matrix and its own slice of the parameter vector, it performs a second multi-party joint calculation with other parties to obtain its own slice of a target statistic; the target statistic reflects the confidence interval of the model parameter; based on its own slice of the target statistic, through interactive processing with other parties, obtains the plain text of the target statistic, and obtains the probability value of the hypothesis test based on the plain text using statistical processing, which serves as the basis for accepting or rejecting the null hypothesis of the parameter value of each model parameter.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the computer field, and more particularly, to a hypothesis testing method and apparatus for a privacy-preserving generalized linear model. Background Art

[0002] The data held by different data holders may contain the user's private information, and data sharing between data holders may infringe the user's privacy. In order to facilitate the data flow between multiple parties, it is proposed to use secure multi-party computation (MPC) technology to support joint computation between multiple parties, tap the value of data, and ensure that the plaintext information of each party's private data will not be leaked during multi-party interaction.

[0003] Secure multi-party computation is a cryptographic technology that enables multiple non-trusting parties to jointly compute a target function without exposing any information other than the output, thus protecting the inputs of the parties and the intermediate results of the computation. Secure multi-party computation technology includes technologies such as obfuscated circuits, secret sharing, and homomorphic encryption. Among them, the secret sharing protocol is widely used for secure computing in various scenarios due to its high efficiency in arithmetic and linear algebra operations.

[0004] Generalized linear models are widely used in finance and other scenarios. Compared with neural network models, their parameters are more interpretable, and hypothesis testing of model parameters is often performed based on statistical processing. In the prior art, there is a lack of corresponding solutions on how to perform hypothesis testing of generalized linear models under the premise of privacy protection and avoid leaking input feature values ​​and parameter values ​​of model parameters. Summary of the invention

[0005] One or more embodiments of the present specification describe a hypothesis testing method and device for a privacy-preserving generalized linear model, which can perform hypothesis testing of a generalized linear model under the premise of privacy protection and avoid leaking input feature values ​​and parameter values ​​of model parameters.

[0006] In a first aspect, a hypothesis testing method for a privacy-preserving generalized linear model is provided, wherein parameter values ​​of each model parameter and input feature values ​​of the generalized linear model are distributed to multiple parties in a secret sharing manner, and the method is performed by any one of the multiple parties, including:

[0007] Based on the slice of the feature matrix formed by each input feature value of multiple samples and the slice of the parameter vector formed by the parameter values ​​of each model parameter, a first multi-party joint calculation is performed with other parties to obtain the slice of the Hessian matrix; the elements of the Hessian matrix are the second-order derivatives of the loss function with respect to the model parameters;

[0008] Based on the local slice of the Hessian matrix and the local slice of the parameter vector, a second multi-party joint calculation is performed with other parties to obtain the local slice of the target statistic; the target statistic reflects the confidence interval of the model parameter;

[0009] Based on the local shard of the target statistic, the plain text of the target statistic is obtained through interactive processing with other parties, and the probability value of the hypothesis test is obtained based on the plain text using statistical processing. The probability value is used as the basis for accepting or rejecting the null hypothesis of the parameter value of each model parameter.

[0010] In a possible implementation, the first multi-party joint computing includes:

[0011] Based on the local slice of the feature matrix and the local slice of the parameter vector, a first MPC calculation is performed with other parties to obtain the local slice of the prediction vector composed of the prediction values ​​of each sample output by the generalized linear model;

[0012] At least based on the own slice of the feature matrix and the own slice of the prediction vector, a second MPC calculation is performed with the other party to obtain the own slice of the Hessian matrix.

[0013] Further, the first MPC calculation includes:

[0014] Based on the local slice of the feature matrix and the local slice of the parameter vector, a secure multiplication calculation of matrices and vectors is performed with other parties to obtain the local slice of the first multiplication result of the feature matrix and the parameter vector, and the first multiplication result is used as the prediction vector.

[0015] Further, the second MPC calculation includes:

[0016] When the data distribution of the output of the generalized linear model belongs to the Poisson distribution, a diagonal matrix constructed based on the local slice of the transposed matrix of the feature matrix, the local slice of the prediction vector, and the local slice of the feature matrix are subjected to matrix-matrix safe multiplication operations with other parties to obtain the local slice of the second multiplication result of the transposed matrix, the diagonal matrix and the feature matrix, and the second multiplication result is used as the Hessian matrix.

[0017] Further, the second MPC calculation includes:

[0018] When the data distribution of the output of the generalized linear model belongs to the gamma distribution or the composite distribution of the Poisson distribution and the gamma distribution, obtaining the local slice of the label vector composed of the label values ​​of each sample;

[0019] Determine the local slice of the dispersion parameter jointly with other parties according to the local slice of the label vector, the local slice of the prediction vector and the distribution parameter of the data distribution; the distribution parameter is used to indicate whether the data distribution is a gamma distribution or a composite distribution;

[0020] Based on the diagonal matrix constructed based on the local slices of the transposed matrix of the characteristic matrix, the local slices of the prediction vector and the local slices of the dispersion parameter, and the local slices of the characteristic matrix, a matrix-matrix secure multiplication operation is performed with other parties to obtain the local slice of the third multiplication result of the transposed matrix, the diagonal matrix and the characteristic matrix, and the third multiplication result is used as the Hessian matrix.

[0021] Furthermore, the sharding of the party jointly determining the dispersion parameter with other parties includes:

[0022] Locally calculating the difference between the local slice of the label vector and the local slice of the prediction vector to obtain the local slice of the difference vector;

[0023] Based on the slice of the difference vector, perform a matrix-matrix secure multiplication operation with the other party to obtain the slice of the square result of the difference vector;

[0024] Based on the local slice of the prediction vector and the distribution parameter of the data distribution, perform a matrix-matrix secure multiplication operation with the other party to obtain the local slice of the power operation result of the distribution parameter of the prediction vector;

[0025] Based on the local slice of the square result and the local slice of the power operation result, a safe division operation is performed with the other party to obtain the local slice of the first division result of the square result divided by a constant multiple of the power operation result, and the first division result is used as the dispersion parameter.

[0026] Furthermore, the diagonal matrix is ​​constructed in the following manner:

[0027] When the data distribution of the output of the generalized linear model belongs to the gamma distribution, the square of the local slice of the prediction vector is divided by the local slice of the dispersion parameter to obtain a second division result, and each element value of the second division result is sequentially used as an element of the diagonal of the diagonal matrix to obtain the diagonal matrix; or,

[0028] When the data distribution of the output of the generalized linear model belongs to a composite distribution, a third division result is obtained by subtracting 2 of the local slice of the prediction vector minus the power of the distribution parameter and dividing it by the local slice of the dispersion parameter, and each element value of the third division result is sequentially used as the diagonal element of the diagonal matrix to obtain the diagonal matrix.

[0029] In a possible implementation manner, the second multi-party joint computing includes:

[0030] Based on the slice of the Hessian matrix, perform a security matrix inverse operation in conjunction with other parties to obtain the slice of the first inverse matrix of the Hessian matrix;

[0031] The values ​​of the diagonal elements of the local slice of the first inverse matrix are retained to obtain a diagonal matrix, which is used as the local slice of the second inverse matrix;

[0032] Based on the local slice of the parameter vector and the local slice of the second inverse matrix, perform a safe multiplication operation and a safe division operation with other slices to obtain a local slice of an intermediate result of dividing the square of the parameter vector by the second inverse matrix;

[0033] Based on the own slice of the intermediate result, a secure square root calculation is performed with other parties to obtain the own slice of the square root result of the intermediate result, and the slice is used as the own slice of the target statistic.

[0034] In a possible implementation manner, obtaining a probability value of a hypothesis test based on the plaintext by using statistical processing includes:

[0035] Calling a statistical function based on the plaintext to calculate a cumulative probability of a normal distribution;

[0036] The probability value is obtained by local calculation based on the cumulative probability of normal distribution.

[0037] In a possible implementation, the multiple parties are three parties, wherein the feature matrix and the parameter vector are both divided into three slices, and each of the three parties holds two of the three slices in a non-repetitive manner.

[0038] In a second aspect, a hypothesis testing device for a privacy-preserving generalized linear model is provided, wherein parameter values ​​of each model parameter and input feature values ​​of the generalized linear model are distributed to multiple parties in a secret sharing manner, and the device is arranged at any one of the multiple parties, including:

[0039] A first joint calculation unit is used to perform a first multi-party joint calculation with other parties based on the slice of the feature matrix formed by each input feature value of multiple samples and the slice of the parameter vector formed by the parameter values ​​of each model parameter, so as to obtain the slice of the Hessian matrix; the elements of the Hessian matrix are the second-order derivatives of the loss function with respect to the model parameters;

[0040] A second joint calculation unit is used to perform a second multi-party joint calculation with other parties based on the local slice of the Hessian matrix and the local slice of the parameter vector obtained by the first joint calculation unit to obtain the local slice of the target statistic; the target statistic reflects the confidence interval of the model parameter;

[0041] A statistical processing unit is used to obtain the plain text of the target statistic based on the local slice of the target statistic obtained by the second joint computing unit through interactive processing with other parties, and obtain the probability value of the hypothesis test based on the plain text by statistical processing, and the probability value is used as the basis for accepting or rejecting the null hypothesis of the parameter value of each model parameter.

[0042] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method of the first aspect.

[0043] According to a fourth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of the first aspect is implemented.

[0044] Through the method and device provided in the embodiments of this specification, any one of the multiple parties first performs a first multi-party joint calculation with other parties based on its own slice of the feature matrix composed of each input feature value of multiple samples and its own slice of the parameter vector composed of the parameter values ​​of each model parameter to obtain its own slice of the Hessian matrix; the elements of the Hessian matrix are the second-order derivatives of the loss function with respect to the model parameters; then, based on its own slice of the Hessian matrix and its own slice of the parameter vector, a second multi-party joint calculation is performed with other parties to obtain its own slice of the target statistic; the target statistic reflects the confidence interval of the model parameter; finally, based on its own slice of the target statistic, through interactive processing with other parties, the plain text of the target statistic is obtained, and based on the plain text, the probability value of the hypothesis test is obtained by statistical processing, and the probability value is used as the basis for accepting or rejecting the null hypothesis of the parameter value of each model parameter. During the entire processing process, the cryptographic secret sharing protocol was used to realize the secret calculation of the probability value of the hypothesis test, so that the hypothesis test of the generalized linear model can be performed under the premise of privacy protection, avoiding the leakage of input eigenvalues ​​and parameter values ​​of model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0046] Figure 1 A schematic diagram of an implementation scenario of an embodiment disclosed in this specification;

[0047] Figure 2A flow chart showing a hypothesis testing method for a privacy-preserving generalized linear model according to an embodiment is shown;

[0048] Figure 3 A schematic block diagram of a hypothesis testing apparatus for a privacy-preserving generalized linear model according to an embodiment is shown. DETAILED DESCRIPTION

[0049] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0050] Figure 1 This is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification. The implementation scenario involves hypothesis testing of a privacy-preserving generalized linear model. Among them, the generalized linear model (GLM) is a generalization of the linear regression model, which is used to deal with situations where the dependent variable (i.e., the response variable) does not satisfy the normal distribution or there are some variables that are not linearly related to the dependent variable. The generalized linear model can model different types of distributions under a unified framework, such as binomial distribution, Poisson distribution, and exponential distribution. In hypothesis testing, the calculation of probability values ​​is involved, and the probability value can be called a p-value. P-value: It is a statistic used for hypothesis testing, which measures the probability of observing the current result or a more extreme result under the condition that the null hypothesis is true. The null hypothesis, also known as the original hypothesis, refers to a hypothesis established in advance when performing a hypothesis test, usually expressed as a hypothesis that there is no statistical effect or difference. The concept of p-value is usually used with specific statistical tests to determine whether there is sufficient evidence to reject the null hypothesis. The above-mentioned specific statistical tests may include z-tests, etc.

[0051] like Figure 1 As shown, the scenario of hypothesis testing of the privacy-preserving generalized linear model involves participants A, B, and C, or referred to as the first party, the second party, and the third party, or referred to as party A, party B, and party C, or referred to as P0, P1, and P2. Each participant can be implemented as any device, platform, server, or device cluster with computing and processing capabilities. All parties should jointly implement hypothesis testing of the generalized linear model while protecting data privacy.

[0052] Actual computing scenarios, such as machine learning, are usually based on floating-point operations. However, using secret sharing to implement secure floating-point operations will result in high overhead, and the efficiency is difficult to meet actual computing needs. Therefore, the usual practice is to use fixed-point numbers to approximate floating-point numbers, achieving greater efficiency optimization under the premise of a certain loss of precision. Secret sharing protocols are usually defined on rings or domains, each with its own advantages and disadvantages. Calculations on rings are more efficient because their modulus operations can be implicitly handled by hardware, compared to domain calculations that require manual modulus.

[0053] In the embodiments of this specification, fixed-point numbers can be mapped into a ring for calculation. The mapping construction process is as follows: Assume is a rational number, we need to Mapping to the integer domain, let , where f is the number of digits of precision, that is, the number of digits in the decimal part, To round up. Perform modular operation so that , where k represents the number of bits of x, mapping the input to On the ring.

[0054] Among them, fixed-point arithmetic is shared in the loop In addition, Boolean sharing is also combined, which is calculated in The arithmetic sharing is expressed as , Boolean sharing is represented by .

[0055] In the embodiment of this specification, three computing parties P0, P1, and P2 are involved, and the secret input x is split into three fragments (x0, x1, x2), satisfying P i Hold(x i ,x i+1 ),as well as .

[0056] The embodiments of this specification may correspond to a secure generalized linear regression p-value calculation protocol based on a secret sharing protocol, which implements a p-value calculation method for multiple distributions, including Poisson, Gamma, and Tweedie distributions. For example, the input of the p-value calculation protocol includes: the feature matrix X composed of the input feature values ​​of multiple samples, whose size is n×d; the label vector y composed of the label values ​​of multiple samples, whose size is n×1; the parameter vector composed of the parameter values ​​of each model parameter of the generalized linear regression model is , whose size is d×1; data distribution dist∈{´Poisson´,´Gamma´, ´Tweedie´}; and distribution parameter p of Tweedie distribution. The output vector of the p-value calculation protocol is P, whose elements correspond to the p-values ​​of each model parameter, and the p-values ​​are plain text.

[0057] Among them, the above p-value calculation protocol is implemented by multiple parties, and any party has a slice of the feature matrix X , the slices of the label vector y , parameter vector Shard , data distribution dist, and distribution parameter p of Tweedie distribution, and obtain output vector P based on secure multi-party computation. It can be understood that label vector y is an optional input, which is required in some data distributions and not required in some data distributions.

[0058] The embodiments of this specification propose corresponding solutions for hypothesis testing of privacy-preserving generalized linear models, in order to avoid leaking input feature values ​​and parameter values ​​of model parameters.

[0059] Figure 2 A flowchart of a hypothesis testing method for a privacy-preserving generalized linear model according to an embodiment is shown. The method can be based on Figure 1 In the implementation scenario shown, the parameter values ​​of each model parameter of the generalized linear model and the input feature value are distributed to multiple parties in a secret sharing manner, and the method is executed by any one of the multiple parties. Figure 2 As shown, the hypothesis testing method of the privacy-preserving generalized linear model in this embodiment includes the following steps: Step 21, based on the local slice of the feature matrix composed of each input feature value of multiple samples, and the local slice of the parameter vector composed of the parameter value of each model parameter, the first multi-party joint calculation is performed with other parties to obtain the local slice of the Hessian matrix; the elements of the Hessian matrix are the second-order derivatives of the loss function with respect to the model parameters; Step 22, based on the local slice of the Hessian matrix and the local slice of the parameter vector, the second multi-party joint calculation is performed with other parties to obtain the local slice of the target statistic; the target statistic reflects the confidence interval of the model parameter; Step 23, based on the local slice of the target statistic, through interactive processing with other parties, the plain text of the target statistic is obtained, and the probability value of the hypothesis test is obtained based on the plain text by using statistical processing, and the probability value is used as the basis for accepting or rejecting the null hypothesis of the parameter value of each model parameter. The specific execution method of each of the above steps is described below.

[0060] First, in step 21, based on the local slice of the feature matrix composed of each input feature value of multiple samples and the local slice of the parameter vector composed of the parameter value of each model parameter, a first multi-party joint calculation is performed with other parties to obtain the local slice of the Hessian matrix; the elements of the Hessian matrix are the second-order derivatives of the loss function with respect to the model parameters. It can be understood that the specific calculation method of the Hessian matrix is ​​related to the data distribution. When the data distribution is different, the calculation method of the Hessian matrix is ​​also different.

[0061] In one example, the multiple parties are three parties, wherein the feature matrix and the parameter vector are both divided into three slices, and each of the three parties holds two of the three slices in a non-repetitive manner.

[0062] This example corresponds to a typical application scenario, but the embodiments of this specification are not limited thereto. In another example, the multiple parties are two parties, wherein the feature matrix and the parameter vector are both divided into two slices, and the two parties each hold one of the two slices.

[0063] In one example, the first multi-party joint computing includes:

[0064] Based on the local slice of the feature matrix and the local slice of the parameter vector, a first MPC calculation is performed with other parties to obtain the local slice of the prediction vector composed of the prediction values ​​of each sample output by the generalized linear model;

[0065] At least based on the own slice of the feature matrix and the own slice of the prediction vector, a second MPC calculation is performed with the other party to obtain the own slice of the Hessian matrix.

[0066] In this example, the local slice of the prediction vector is calculated first, and then the local slice of the Hessian matrix is ​​calculated based on the local slice of the prediction vector, so as to achieve safe calculation of the Hessian matrix. The elements of the Hessian matrix are the observed values ​​of the second-order derivative of the loss function with respect to the model parameters at the current model parameters.

[0067] Further, the first MPC calculation includes:

[0068] Based on the local slice of the feature matrix and the local slice of the parameter vector, a secure multiplication calculation of matrices and vectors is performed with other parties to obtain the local slice of the first multiplication result of the feature matrix and the parameter vector, and the first multiplication result is used as the prediction vector.

[0069] In this example, secure multiplication is used to achieve secure calculation of the predicted value. It is understandable that the plain text of the predicted value can be directly obtained according to the standard generalized linear model calculation formula.

[0070] For example, any of the parties may jointly execute , thus obtaining the local slice of the prediction vector composed of the prediction values ​​of each sample. Among them, represents the slices of the feature matrix, represents the slices of the parameter vector, Represents a slice of the prediction vector.

[0071] Further, the second MPC calculation includes:

[0072] When the data distribution of the output of the generalized linear model belongs to the Poisson distribution, a diagonal matrix constructed based on the local slice of the transposed matrix of the feature matrix, the local slice of the prediction vector, and the local slice of the feature matrix are subjected to matrix-matrix safe multiplication operations with other parties to obtain the local slice of the second multiplication result of the transposed matrix, the diagonal matrix and the feature matrix, and the second multiplication result is used as the Hessian matrix.

[0073] In this example, the data distribution belongs to Poisson distribution, and the secure calculation of the Hessian matrix is ​​achieved through secure multiplication. It can be understood that the above diagonal matrix can be constructed locally without interacting with other parties.

[0074] For example, any of the parties may jointly execute , thus obtaining the local slice of the Hessian matrix when the data distribution belongs to the Poisson distribution. Among them, represents the slices of the feature matrix, represents the slices of the transposed matrix of the feature matrix, represents the slice of the prediction vector, represents a diagonal matrix.

[0075] Further, the second MPC calculation includes:

[0076] When the data distribution of the output of the generalized linear model belongs to the gamma distribution or the composite distribution of the Poisson distribution and the gamma distribution, obtaining the local slice of the label vector composed of the label values ​​of each sample;

[0077] Determine the local slice of the dispersion parameter jointly with other parties according to the local slice of the label vector, the local slice of the prediction vector and the distribution parameter of the data distribution; the distribution parameter is used to indicate whether the data distribution is a gamma distribution or a composite distribution;

[0078] Based on the diagonal matrix constructed based on the local slices of the transposed matrix of the characteristic matrix, the local slices of the prediction vector and the local slices of the dispersion parameter, and the local slices of the characteristic matrix, a matrix-matrix secure multiplication operation is performed with other parties to obtain the local slice of the third multiplication result of the transposed matrix, the diagonal matrix and the characteristic matrix, and the third multiplication result is used as the Hessian matrix.

[0079] In this example, for data distribution belonging to gamma distribution or composite distribution, the local shards of the dispersion parameters are first determined jointly with other parties, and then the secure multiplication is performed to achieve secure calculation of the Hessian matrix. It can be understood that the above diagonal matrix can be constructed locally without interacting with other parties.

[0080] Further, the local shard for jointly determining the dispersion parameter with other parties includes:

[0081] Locally calculate the difference between the local shard of the label vector and the local shard of the prediction vector to obtain the local shard of the difference vector;

[0082] Based on the local shard of the difference vector, perform a secure matrix multiplication operation with other parties to obtain the local shard of the squared result of the difference vector;

[0083] Based on the local shard of the prediction vector and the distribution parameter of the data distribution, perform a secure matrix multiplication operation with other parties to obtain the local shard of the result of the power operation of the prediction vector to the power of the distribution parameter;

[0084] Based on the local shard of the squared result and the local shard of the power operation result, perform a secure division operation with other parties to obtain the local shard of the first division result of the squared result divided by a constant multiple of the power operation result, and the first division result is used as the dispersion parameter.

[0085] In this example, the secure calculation of the dispersion parameter is achieved through secure multiplication calculation and secure division operation. Here, it is calculated according to the Pearson chi-square statistic and the squared loss.

[0086] For example, any party among multiple parties can jointly execute , to obtain the local shard of the dispersion parameter. Among them, represents the shard of the label vector, represents the shard of the prediction vector. The dimension of the feature matrix X is n×d, p represents the distribution parameter. For the composite distribution of the Poisson distribution and the gamma distribution, when p = 1, it is the Poisson distribution; when p = 2, it is the gamma distribution; when 1 < p < 2, it is the composite distribution of the Poisson distribution and the gamma distribution.

[0087] Further, the diagonal matrix is constructed in the following manner:

[0088] When the data distribution output by the generalized linear model belongs to the gamma distribution, divide the square of the local shard of the prediction vector by the local shard of the dispersion parameter to obtain the second division result, and use the respective element values of the second division result as the elements of the diagonal of the diagonal matrix in sequence to obtain the diagonal matrix; or,

[0089] When the data distribution output by the generalized linear model belongs to the composite distribution, divide the 2 minus the power of the distribution parameter of the local shard of the prediction vector by the local shard of the dispersion parameter to obtain the third division result, and use the respective element values of the third division result as the elements of the diagonal of the diagonal matrix in sequence to obtain the diagonal matrix.

[0090] In the embodiment of this specification, a diagonal matrix can be constructed based on a vector. For example, a vector is [1, 2, 3], and the diagonal matrix constructed based on the vector is , where the elements of the vector are used as the elements on the diagonal of the matrix, thus obtaining a diagonal matrix; a diagonal matrix can also be constructed based on a matrix, for example, a matrix is , the diagonal matrix constructed based on this matrix is , where the elements on the diagonal of the matrix are retained and the remaining elements are set to 0, thus obtaining a diagonal matrix.

[0091] The embodiments of this specification can be compatible with multiple data distributions, and the corresponding Hessian matrix calculation method is selected according to each data distribution. For example, the secure calculation of the Hessian matrix can be implemented through the following code:

[0092] “if dist==´Poisson´ then

[0093]

[0094] else if dist==´Gamma´ then

[0095]

[0096] else if dist==´Tweedie´ then

[0097]

[0098] end if”

[0099] It can be understood that ´Poisson´ is the aforementioned Poisson distribution, ´Gamma´ is the aforementioned gamma distribution, and ´Tweedie´ is the aforementioned composite distribution.

[0100] Then, in step 22, based on the local slice of the Hessian matrix and the local slice of the parameter vector, a second multi-party joint calculation is performed with other parties to obtain the local slice of the target statistic; the target statistic reflects the confidence interval of the model parameter. It can be understood that the target statistic can include but is not limited to the z statistic.

[0101] In the embodiments of this specification, corresponding to multiple model parameters, target statistics corresponding to each model parameter can be obtained respectively, and each target statistic constitutes a vector, which can be referred to as a statistical vector.

[0102] In one example, the second multi-party joint computing includes:

[0103] Based on the slice of the Hessian matrix, perform a security matrix inverse operation in conjunction with other parties to obtain the slice of the first inverse matrix of the Hessian matrix;

[0104] The values ​​of the diagonal elements of the local slice of the first inverse matrix are retained to obtain a diagonal matrix, which is used as the local slice of the second inverse matrix;

[0105] Based on the local slice of the parameter vector and the local slice of the second inverse matrix, perform a safe multiplication operation and a safe division operation with other slices to obtain a local slice of an intermediate result of dividing the square of the parameter vector by the second inverse matrix;

[0106] Based on the own slice of the intermediate result, a secure square root calculation is performed with other parties to obtain the own slice of the square root result of the intermediate result, and the slice is used as the own slice of the target statistic.

[0107] In this example, the secure calculation of the target statistic is achieved through secure matrix inverse operation, secure multiplication calculation, secure division operation, and secure square root calculation. It can be understood that the above diagonal matrix can be constructed locally without interacting with other parties.

[0108] For example, any of the parties may jointly execute , thus obtaining the slice of the second inverse matrix, and then jointly executing with other parties , thus obtaining the local fragment of the target statistic. is the slice of the Hessian matrix, inverse represents the safe inverse operation, and diag represents the construction of a diagonal matrix. represents the slice of the second inverse matrix, represents the slice of the parameter vector, sqrt represents the safe square root calculation, Represents the shard of the target statistic.

[0109] Finally, in step 23, based on the local shard of the target statistic, the plain text of the target statistic is obtained through interactive processing with other parties, and the probability value of the hypothesis test is obtained based on the plain text by statistical processing, and the probability value is used as the basis for accepting or rejecting the null hypothesis of the parameter value of each model parameter. It can be understood that the probability value is the p value.

[0110] In this embodiment of the specification, the above interaction process can be expressed as .

[0111] In one example, obtaining a probability value of a hypothesis test based on the plaintext by using statistical processing includes:

[0112] Calling a statistical function based on the plaintext to calculate a cumulative probability of a normal distribution;

[0113] The probability value is obtained by local calculation based on the cumulative probability of the normal distribution.

[0114] For example, any of the multiple parties locally executes , thus obtaining the probability value of the hypothesis test. Among them, norm.cdf represents the normal distribution cumulative distribution function, which is used to calculate the cumulative probability of the normal distribution, and P represents the probability value of the hypothesis test. Specifically, each model parameter has a corresponding p value, and P is a vector composed of the p values ​​corresponding to each model, referred to as a probability vector.

[0115] The embodiments of this specification can complete the p-value calculation safely, correctly and efficiently. The intermediate calculation results in the entire calculation process are confidential, and only the final z statistic and p-value will be decrypted into plain text, which protects the privacy of the generalized linear model to the greatest extent and can be applied in actual application scenarios, especially in multi-party collaborative modeling scenarios.

[0116] Compared with the neural network model, the parameters of the generalized linear model are more interpretable, and the calculation of the p-value is also widely used in hypothesis testing of model parameters. The p-value can be used to determine whether the model parameters are significant, that is, whether the corresponding independent variables can effectively predict the variation of the dependent variable, thereby determining whether the corresponding explanatory variables should be included in the model.

[0117] Through the method provided in the embodiments of this specification, a cryptographic secret sharing protocol is utilized to realize the secret calculation of the probability value of the hypothesis test, so that the hypothesis test of the generalized linear model can be performed under the premise of privacy protection to avoid the leakage of input feature values ​​and parameter values ​​of model parameters.

[0118] The following is a brief description of the conventional computing primitives used in the aforementioned embodiments of this specification, including:

[0119] Addition and constant multiplication operations can be completed directly according to the original addition secret sharing protocol. Each participant only needs to perform calculations locally;

[0120] Multiplication operation, a fixed-point multiplication operation includes two key calculations. First, the participants use the standard multiplication protocol to complete the integer multiplication operation. After the multiplication calculation is completed, due to the limited precision of the fixed-point data, the calculation result needs to be truncated. The Truncation protocol is used to truncate the data after the precision is doubled: the last f bits of the data are truncated, that is, divided by 2. f ;

[0121] For division operations, Goldschmidt's iteration method is used to approximate the division operation. An initial guess value is estimated based on the input, and then an approximate solution for the division is obtained through multiple rounds of iterative calculations.

[0122] Square root operation (sqrt) is approximated using Goldschmidt's iterative method. First, an initial guess value rsqrt is estimated based on the input, and then the approximate solution of sqrt is obtained through multiple rounds of iterative calculations.

[0123] Matrix inverse operation (inverse) avoids exact solution and uses Newton iteration method for multiple rounds of iterative approximation.

[0124] According to another aspect of the embodiment, there is also provided a hypothesis testing device for a privacy-preserving generalized linear model, which is used to execute the method provided in the embodiment of this specification, wherein the parameter values ​​of each model parameter of the generalized linear model and the input feature values ​​are distributed to multiple parties in a secret sharing manner, and the device is set at any one of the multiple parties. Figure 3 A schematic block diagram of a hypothesis testing apparatus for a privacy-preserving generalized linear model according to an embodiment is shown. Figure 3 As shown, the device 300 includes:

[0125] A first joint calculation unit 31 is used to perform a first multi-party joint calculation with other parties based on the local slice of the feature matrix composed of each input feature value of multiple samples and the local slice of the parameter vector composed of the parameter values ​​of each model parameter, so as to obtain the local slice of the Hessian matrix; the elements of the Hessian matrix are the second-order derivatives of the loss function with respect to the model parameters;

[0126] A second joint calculation unit 32 is used to perform a second multi-party joint calculation with other parties based on the local slice of the Hessian matrix and the local slice of the parameter vector obtained by the first joint calculation unit 31 to obtain the local slice of the target statistic; the target statistic reflects the confidence interval of the model parameter;

[0127] The statistical processing unit 33 is used to obtain the plain text of the target statistic based on the local slice of the target statistic obtained by the second joint calculation unit 32 through interactive processing with other parties, and obtain the probability value of the hypothesis test based on the plain text using statistical processing, and the probability value is used as the basis for accepting or rejecting the null hypothesis of the parameter value of each model parameter.

[0128] Optionally, as an embodiment, the first joint calculation unit 31 includes:

[0129] A first computing subunit is configured to perform a first MPC calculation with other parties based on the local slice of the feature matrix and the local slice of the parameter vector to obtain the local slice of the prediction vector composed of the prediction values ​​of each sample output by the generalized linear model;

[0130] The second computing subunit is used to perform a second MPC calculation with other parties based on at least the own slice of the feature matrix and the own slice of the prediction vector obtained by the first computing subunit to obtain the own slice of the Hessian matrix.

[0131] Furthermore, the first computing subunit is specifically used to perform secure matrix and vector multiplication calculations with other parties based on the local slice of the feature matrix and the local slice of the parameter vector, to obtain the local slice of the first multiplication result of the feature matrix and the parameter vector, and the first multiplication result is used as the prediction vector.

[0132] Furthermore, the second computing subunit is specifically used to perform matrix-matrix safe multiplication operations with other parties based on the local slices of the transposed matrix of the feature matrix, the diagonal matrix constructed by the local slices of the prediction vector, and the local slices of the feature matrix when the data distribution of the output of the generalized linear model belongs to the Poisson distribution, to obtain the local slice of the second multiplication result of the transposed matrix, the diagonal matrix and the feature matrix, and the second multiplication result is used as the Hessian matrix.

[0133] Furthermore, the second computing subunit is specifically configured to:

[0134] When the data distribution of the output of the generalized linear model belongs to the gamma distribution or the composite distribution of the Poisson distribution and the gamma distribution, obtaining the local slice of the label vector composed of the label values ​​of each sample;

[0135] Determine the local slice of the dispersion parameter jointly with other parties according to the local slice of the label vector, the local slice of the prediction vector and the distribution parameter of the data distribution; the distribution parameter is used to indicate whether the data distribution is a gamma distribution or a composite distribution;

[0136] Based on the diagonal matrix constructed based on the local slices of the transposed matrix of the characteristic matrix, the local slices of the prediction vector and the local slices of the dispersion parameter, and the local slices of the characteristic matrix, a matrix-matrix secure multiplication operation is performed with other parties to obtain the local slice of the third multiplication result of the transposed matrix, the diagonal matrix and the characteristic matrix, and the third multiplication result is used as the Hessian matrix.

[0137] Furthermore, the sharding of the party jointly determining the dispersion parameter with other parties includes:

[0138] Locally calculating the difference between the local slice of the label vector and the local slice of the prediction vector to obtain the local slice of the difference vector;

[0139] Based on the slice of the difference vector, perform a matrix-matrix secure multiplication operation with the other party to obtain the slice of the square result of the difference vector;

[0140] Based on the local slice of the prediction vector and the distribution parameter of the data distribution, perform a matrix-matrix secure multiplication operation with the other party to obtain the local slice of the power operation result of the distribution parameter of the prediction vector;

[0141] Based on the local slice of the square result and the local slice of the power operation result, a safe division operation is performed with the other party to obtain the local slice of the first division result of the square result divided by a constant multiple of the power operation result, and the first division result is used as the dispersion parameter.

[0142] Furthermore, the diagonal matrix is ​​constructed in the following manner:

[0143] When the data distribution of the output of the generalized linear model belongs to the gamma distribution, the square of the local slice of the prediction vector is divided by the local slice of the dispersion parameter to obtain a second division result, and each element value of the second division result is sequentially used as an element of the diagonal of the diagonal matrix to obtain the diagonal matrix; or,

[0144] When the data distribution of the output of the generalized linear model belongs to a composite distribution, a third division result is obtained by subtracting 2 of the local slice of the prediction vector minus the power of the distribution parameter and dividing it by the local slice of the dispersion parameter, and each element value of the third division result is sequentially used as the diagonal element of the diagonal matrix to obtain the diagonal matrix.

[0145] Optionally, as an embodiment, the second joint calculation unit 32 includes:

[0146] A matrix inverse operation subunit, configured to perform a security matrix inverse operation on the slice of the Hessian matrix on the local side in conjunction with other parties to obtain the slice of the first inverse matrix of the Hessian matrix on the local side;

[0147] The diagonal matrix construction subunit is used to retain the values ​​of the diagonal elements of the local slice of the first inverse matrix obtained by the matrix inverse operation subunit, obtain a diagonal matrix, and use it as the local slice of the second inverse matrix;

[0148] an intermediate processing subunit, configured to perform a safe multiplication operation and a safe division operation with other slices of the parameter vector and the second inverse matrix obtained by the diagonal matrix construction subunit, so as to obtain a slice of the intermediate result of dividing the square of the parameter vector by the second inverse matrix;

[0149] The square root calculation subunit is used to perform secure square root calculation with other parties based on the local slice of the intermediate result obtained by the intermediate processing subunit, to obtain the local slice of the square root result of the intermediate result, and use it as the local slice of the target statistic.

[0150] Optionally, as an embodiment, the statistical processing unit 33 includes:

[0151] A first calculation subunit, configured to call a statistical function based on the plain text to calculate a cumulative probability of a normal distribution;

[0152] The second calculation subunit is used to locally calculate the probability value based on the cumulative probability of the normal distribution obtained by the first calculation subunit.

[0153] Optionally, as an embodiment, the multiple parties are three parties, wherein the feature matrix and the parameter vector are both divided into three slices, and each of the three parties holds two of the three slices in a non-repetitive manner.

[0154] Through the device provided by the embodiments of the present specification, any party among the multiple parties first performs a first multi-party joint calculation with other parties based on its own slice of the feature matrix composed of each input feature value of multiple samples and its own slice of the parameter vector composed of the parameter values ​​of each model parameter by a first joint calculation unit 31 to obtain its own slice of the Hessian matrix; the elements of the Hessian matrix are the second-order derivatives of the loss function with respect to the model parameters; then a second joint calculation unit 32 performs a second multi-party joint calculation with other parties based on its own slice of the Hessian matrix and its own slice of the parameter vector to obtain its own slice of the target statistic; the target statistic reflects the confidence interval of the model parameter; finally, a statistical processing unit 33 obtains the plain text of the target statistic based on its own slice of the target statistic through interactive processing with other parties, and obtains the probability value of the hypothesis test based on the plain text using statistical processing, and the probability value serves as the basis for accepting or rejecting the null hypothesis of the parameter value of each model parameter. During the entire processing process, the cryptographic secret sharing protocol was used to realize the secret calculation of the probability value of the hypothesis test, so that the hypothesis test of the generalized linear model can be performed under the premise of privacy protection, avoiding the leakage of input eigenvalues ​​and parameter values ​​of model parameters.

[0155] According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 2 The method described.

[0156] According to another embodiment of the present invention, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the Figure 2 The method described.

[0157] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0158] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A hypothesis testing method for a privacy-preserving generalized linear model, wherein parameter values ​​of each model parameter and input feature values ​​of the generalized linear model are distributed to multiple parties in a secret sharing manner, and the method is performed by any one of the multiple parties, comprising: Based on the slice of the feature matrix formed by each input feature value of multiple samples and the slice of the parameter vector formed by the parameter values ​​of each model parameter, a first multi-party joint calculation is performed with other parties to obtain the slice of the Hessian matrix; the elements of the Hessian matrix are the second-order derivatives of the loss function with respect to the model parameters; Based on the local slice of the Hessian matrix and the local slice of the parameter vector, a second multi-party joint calculation is performed with other parties to obtain the local slice of the target statistic; the target statistic reflects the confidence interval of the model parameter; Based on the target statistic, the local shard obtains the plain text of the target statistic through interactive processing with other parties, and obtains the probability value of the hypothesis test based on the plain text by statistical processing, and the probability value is used as the basis for accepting or rejecting the null hypothesis of the parameter value of each model parameter; Wherein, the first multi-party joint computing includes: Based on the local slice of the feature matrix and the local slice of the parameter vector, a first MPC calculation is performed with other parties to obtain the local slice of the prediction vector composed of the prediction values ​​of each sample output by the generalized linear model; Performing a second MPC calculation with other parties based at least on the own slice of the feature matrix and the own slice of the prediction vector to obtain the own slice of the Hessian matrix; Wherein, the second multi-party joint computing includes: Based on the slice of the Hessian matrix, perform a security matrix inverse operation in conjunction with other parties to obtain the slice of the first inverse matrix of the Hessian matrix; The values ​​of the diagonal elements of the local slice of the first inverse matrix are retained to obtain a diagonal matrix, which is used as the local slice of the second inverse matrix; Based on the local slice of the parameter vector and the local slice of the second inverse matrix, perform a safe multiplication operation and a safe division operation with other slices to obtain a local slice of an intermediate result of dividing the square of the parameter vector by the second inverse matrix; Based on the own slice of the intermediate result, a secure square root calculation is performed with other parties to obtain the own slice of the square root result of the intermediate result, and the slice is used as the own slice of the target statistic.

2. The method of claim 1, wherein: The first MPC calculation includes: Based on the local slice of the feature matrix and the local slice of the parameter vector, a secure multiplication calculation of matrices and vectors is performed with other parties to obtain the local slice of the first multiplication result of the feature matrix and the parameter vector, and the first multiplication result is used as the prediction vector.

3. The method of claim 1, wherein: The second MPC calculation includes: When the data distribution of the output of the generalized linear model belongs to the Poisson distribution, a diagonal matrix constructed based on the local slice of the transposed matrix of the feature matrix, the local slice of the prediction vector, and the local slice of the feature matrix are subjected to matrix-matrix safe multiplication operations with other parties to obtain the local slice of the second multiplication result of the transposed matrix, the diagonal matrix and the feature matrix, and the second multiplication result is used as the Hessian matrix.

4. The method of claim 1, wherein: The second MPC calculation includes: When the data distribution of the output of the generalized linear model belongs to the gamma distribution or the composite distribution of the Poisson distribution and the gamma distribution, obtaining the local slice of the label vector composed of the label values ​​of each sample; Determine the local slice of the dispersion parameter jointly with other parties according to the local slice of the label vector, the local slice of the prediction vector and the distribution parameter of the data distribution; the distribution parameter is used to indicate whether the data distribution is a gamma distribution or a composite distribution; Based on the diagonal matrix constructed based on the local slices of the transposed matrix of the characteristic matrix, the local slices of the prediction vector and the local slices of the dispersion parameter, and the local slices of the characteristic matrix, a matrix-matrix secure multiplication operation is performed with other parties to obtain the local slice of the third multiplication result of the transposed matrix, the diagonal matrix and the characteristic matrix, and the third multiplication result is used as the Hessian matrix.

5. The method of claim 4, wherein: The shards of the party jointly determining the dispersion parameters with other parties include: Locally calculating the difference between the local slice of the label vector and the local slice of the prediction vector to obtain the local slice of the difference vector; Based on the slice of the difference vector, perform a matrix-matrix secure multiplication operation with the other party to obtain the slice of the square result of the difference vector; Based on the local slice of the prediction vector and the distribution parameter of the data distribution, perform a matrix-matrix secure multiplication operation with the other party to obtain the local slice of the power operation result of the distribution parameter of the prediction vector; Based on the local slice of the square result and the local slice of the power operation result, a safe division operation is performed with the other party to obtain the local slice of the first division result of the square result divided by a constant multiple of the power operation result, and the first division result is used as the dispersion parameter.

6. The method of claim 4, wherein: The diagonal matrix is ​​constructed as follows: When the data distribution of the output of the generalized linear model belongs to the gamma distribution, the square of the local slice of the prediction vector is divided by the local slice of the dispersion parameter to obtain a second division result, and each element value of the second division result is sequentially used as an element of the diagonal of the diagonal matrix to obtain the diagonal matrix; or, When the data distribution of the output of the generalized linear model belongs to a composite distribution, a third division result is obtained by subtracting 2 of the local slice of the prediction vector minus the power of the distribution parameter and dividing it by the local slice of the dispersion parameter, and each element value of the third division result is sequentially used as the diagonal element of the diagonal matrix to obtain the diagonal matrix.

7. The method of claim 1, wherein: The method of obtaining a probability value of a hypothesis test based on the plaintext by using statistical processing includes: Calling a statistical function based on the plaintext to calculate a cumulative probability of a normal distribution; The probability value is obtained by local calculation based on the cumulative probability of the normal distribution.

8. The method of claim 1, wherein: The multiple parties are three parties, wherein the feature matrix and the parameter vector are both divided into three slices, and each of the three parties holds two of the three slices in a non-repetitive manner.

9. A hypothesis testing device for a privacy-preserving generalized linear model, wherein the parameter values ​​of each model parameter and the input feature value of the generalized linear model are distributed to multiple parties in a secret sharing manner, and the device is set at any one of the multiple parties, comprising: A first joint calculation unit is used to perform a first multi-party joint calculation with other parties based on the slice of the feature matrix formed by each input feature value of multiple samples and the slice of the parameter vector formed by the parameter values ​​of each model parameter, so as to obtain the slice of the Hessian matrix; the elements of the Hessian matrix are the second-order derivatives of the loss function with respect to the model parameters; A second joint calculation unit is used to perform a second multi-party joint calculation with other parties based on the local slice of the Hessian matrix and the local slice of the parameter vector obtained by the first joint calculation unit to obtain the local slice of the target statistic; the target statistic reflects the confidence interval of the model parameter; A statistical processing unit, configured to obtain a plain text of the target statistic based on the local shard of the target statistic obtained by the second joint computing unit through interactive processing with other parties, and obtain a probability value of a hypothesis test based on the plain text by using statistical processing, wherein the probability value serves as a basis for accepting or rejecting a null hypothesis of a parameter value of each model parameter; Wherein, the first joint computing unit comprises: A first computing subunit is configured to perform a first MPC calculation with other parties based on the local slice of the feature matrix and the local slice of the parameter vector to obtain the local slice of the prediction vector composed of the prediction values ​​of each sample output by the generalized linear model; A second computing subunit is configured to perform a second MPC calculation with other parties based on at least the own slice of the feature matrix and the own slice of the prediction vector obtained by the first computing subunit, so as to obtain the own slice of the Hessian matrix; Wherein, the second joint computing unit comprises: A matrix inverse operation subunit, configured to perform a security matrix inverse operation on the slice of the Hessian matrix on the local side in conjunction with other parties to obtain the slice of the first inverse matrix of the Hessian matrix on the local side; The diagonal matrix construction subunit is used to retain the values ​​of the diagonal elements of the local slice of the first inverse matrix obtained by the matrix inverse operation subunit, obtain a diagonal matrix, and use it as the local slice of the second inverse matrix; an intermediate processing subunit, configured to perform a safe multiplication operation and a safe division operation with other slices of the parameter vector and the second inverse matrix obtained by the diagonal matrix construction subunit, so as to obtain a slice of the intermediate result of dividing the square of the parameter vector by the second inverse matrix; The square root calculation subunit is used to perform secure square root calculation with other parties based on the local slice of the intermediate result obtained by the intermediate processing subunit, to obtain the local slice of the square root result of the intermediate result, and use it as the local slice of the target statistic.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 8.

11. A computing device, comprising a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 8 is implemented.

Citation Information

Patent Citations

  • Method and device for performing significance test on private data by multi-party joint

    CN111506922A

  • Privacy-protecting method and device for determining feature effective value of business data

    CN113407987A

  • Classification model training method and device

    CN114048804A

  • Probability truncation method and device for protecting privacy data

    CN116821962A

  • System and method for privacy-preserving distributed training of neural network models on distributed datasets

    EP4205344A1