A privacy protection based logistic regression scheme in cloud environment

Abstract

This invention proposes a privacy-preserving logistic regression scheme for cloud environments. Logistic regression, as a classification algorithm in machine learning, has wide applications in real life. With the development of cloud computing, the training of logistic regression models can be outsourced to cloud servers, but this brings the problem of privacy leakage of user training data. To solve this problem, this invention proposes a privacy-preserving logistic regression scheme based on distributed double trapdoor public-key cryptography (DT-PKC), suitable for outsourced training of logistic regression models in cloud environments. This invention uses a specified multiplication, vector inner product, and activation function operation protocol, through the interaction of two cloud servers, to ensure that the data is correctly operated in an encrypted state, thereby protecting user training data from leakage and obtaining an accurate training model.

Description

Technical Field

[0001] This invention relates to the fields of cryptography and machine learning, specifically to a privacy-preserving logistic regression machine learning method. Background Technology

[0002] In recent years, artificial intelligence (AI) has received widespread attention, with machine learning playing a crucial role. With the development of cloud computing, an increasing number of machine learning tasks are being performed by cloud servers. Logistic regression, as one of the widely used classification methods in machine learning, can be outsourced to cloud servers for training on massive amounts of data. While the training tasks are efficient for cloud computing capabilities, data privacy breaches are unavoidable.

[0003] Common solutions to this problem include secure multi-party computation, differential privacy, and homomorphic encryption. In 2019, Mandal et al. proposed PrivFL, a method for global gradient computation on active user data. PrivFL guarantees the privacy of user data to the server and the privacy of the server's model data to the user. However, this method requires users to remain active and cannot be implemented offline after users provide data. In 2019, Yu et al. used batch processing to perform formal and fine-grained privacy loss analysis on two different datasets, solving the privacy problem of sensitive data and improving the accuracy of centralized differential privacy models. However, this method has a large computational load, reducing model training efficiency. In 2019, SergiuCarpov et al. proposed a general solution based on fully homomorphic encryption (FHE). This method can be applied to different parameter sets without re-encrypting the database, improving model quality. However, this solution also requires a large amount of computational resources and is inefficient. To address this issue, this invention designs a method based on partially homomorphic encryption (PFE) and proposes a privacy-preserving logistic regression scheme based on distributed double trapdoor public-key cryptography (DT-PKC), suitable for outsourced logistic regression training in cloud environments. Our scheme protects the privacy of the training data while ensuring the correctness of the model classification by using a specified protocol for multiplication, vector inner product, and activation function operations on ciphertext data. Summary of the Invention

[0004] The technical problem to be solved by this invention is to obtain an accurate logistic regression model without disclosing training data, so that cloud servers can provide efficient training services without compromising user privacy.

[0005] The technical approach adopted in this invention is to use dual servers for model training. The two servers interact to complete secure integer multiplication, vector inner product, and activation function operations without exposing the plaintext, thereby ensuring data security and model accuracy.

[0006] Compared with existing solutions, this invention uses dual servers for model training, reducing dependence on a single server and improving training efficiency and system reliability. By converting floating-point numbers, it overcomes the limitation of some homomorphic encryption schemes that only handle integer calculations. The encrypted operation protocol for multiplication, vector inner product, and activation function designed in this invention can be extended to other types of model training.

[0007] This invention is a privacy-preserving logistic regression method in a cloud environment. The participants in the method include a Key Generation Center (KGC), a Data Provider (DP), a Cloud Storage Server (CSS), a Cryptographic Service Provider (CSP), a Service Forwarder (SF), and a Service Requester (SR). Detailed Implementation

[0008] The embodiments of the present invention include the following steps:

[0009] [x] is used in this specification. k This represents the ciphertext encrypted with public key k, where "×" represents multiplication and "." represents vector dot product.

[0010] (1) System initialization: KGC generates system parameter para by inputting a random security parameter κ and running the algorithm DT-PKC.Setup(κ);

[0011] (2) Key generation: KGC executes the algorithm DT-PKC.KeyGen(para) to generate a public-private key pair para=(N,g) and the system master key λ for all participants in the system. Then, KGC executes the algorithm DT-PKC.KeySplit(para,λ) to randomly assign an additional partial decryption key λ1 and λ2 to CSS and CSP respectively, so that CSS holds λ1 and CSP holds λ2. After completing this step, KGC can be offline.

[0012] (3) Floating-point conversion:

[0013] Given a positive integer E, representing the precision of a floating-point number, input a floating-point number. The main conversion method is: if the floating-point number x to be converted is positive, then use x... * =x×2 E As the integer converted from the floating-point number x; otherwise, x * =Nx×2 E As the integer obtained by converting the floating-point number x;

[0014] (4) Safe integer multiplication:

[0015] Data encrypted using keys PK1 and PK2 and Calculate the ciphertext under the new key PK3 without decryption. This operation is completed collaboratively by two cloud servers, CSS and CSP. The specific steps are as follows: (a) CSS first retrieves data from group Z. N Select four random numbers r x ,r y ,R x ,R y The four random numbers are then encrypted using public keys PK1 and PK2 respectively to obtain the corresponding ciphertext. Then, the ciphertext is calculated using the homomorphic property. Next, use part of its decryption key λ1 to decrypt the ciphertext. Perform the first step of decryption and obtain the intermediate result. Finally, the ciphertext and part of the decrypted ciphertext are... Send to CSP;

[0016] (b) The CSP receives the message Then, using a portion of the decryption key λ2, a second decryption step is performed, first yielding the ciphertext. Next, CSP can obtain the plaintext:

[0017]

[0018] Then, calculate h = x' × y', and encrypt S, T, and h respectively using the public key PK3 required by the system. Finally, CSP will output the result. Send to CSS;

[0019] (c) CSS receives ciphertext Then, calculate To remove random number r x r y R x R y Finally, calculate and output. As the ciphertext of the product;

[0020] (5) Secure vector dot product operation:

[0021] Input two n-dimensional vectors encrypted using keys PK1 and PK2 and Calculate the ciphertext encrypted with the new key PK3 without revealing the plaintext. This was accomplished collaboratively by the CSS and CSP on two cloud servers, and the specific steps are as follows:

[0022] (a) CSS performs the following steps for each i (1≤i≤n): First from group Z N Select four random numbers The four random numbers are then encrypted using public keys PK1 and PK2 respectively, yielding the corresponding ciphertext. Then, the ciphertext is calculated using the homomorphic property. Next, use part of its decryption key λ1 to decrypt the ciphertext. The first step is decryption, yielding intermediate results. After performing the above steps n times, all data will be packaged into a message. Send to CSP;

[0023] (b) The CSP receives the message Then, for each i (1≤i≤n), As input, the second step of decryption is performed using a portion of its decryption key λ2 to obtain the ciphertext. The plaintext message is calculated as follows:

[0024]

[0025]

[0026] Perform a multiplication operation h on the obtained plaintext i =x i ′×y i Then, use the public key PK3 to encrypt h respectively. i S i T i Received news Finally, all the encrypted results Send to CSS;

[0027] (c) CSS performs the following steps for each i (1≤i≤n): Calculate

[0028] To remove each random number, we get Then, calculate and output.

[0029] (6) Safe activation function Sigmoid operation:

[0030] Input two ciphertext vectors encrypted with different public keys. Without revealing the plaintext of X and W, calculate the Sigmoid function z(X,W) = 1 / (1+e^(-1 / W)) under the new key PK3. -<X.W> ) ciphertext This was accomplished collaboratively by the CSS and CSP on two cloud servers, and the specific steps are as follows:

[0031] (a) First, CSS and CSP perform a secure vector inner product operation on the input ciphertext vector. and Calculate the inner product, then CSS will output the result [v] = [<X.W> The first step is to decrypt [v'] to obtain [v], and then ([v], [v']) is sent to CSP;

[0032] (b) After receiving the message ([v],[v']), the CSP performs a second-step decryption to obtain the result v; then it uses the Taylor expansion e -v ≈1-v+1 / 2×v 2 -1 / 6×v 3 +1 / 24×v 4 -1 / 120×v 5 Approximate calculations are performed, resulting in Z = 1 / (1+e) -v Finally, the result Z is encrypted using the public key PK3 to obtain... And send it to CSS;

[0033] (7) Training of privacy-preserving logistic regression model:

[0034] In logistic regression training, all operations are performed on encrypted data. The gradient descent algorithm is used to update the model parameters w to obtain the optimal solution that minimizes the loss function. First, the model parameter vector w is initialized to 1. The encrypted form of the model parameters w and the training sample x is then input, and their inner product is calculated using secure vector inner product operations.<x.w> Then, the Sigmoid value is calculated from the inner product result using the Sigmoid safe activation function. Next, the difference between the predicted label and the true label is calculated based on the homomorphism of the DT-PKC algorithm, and the loss value of the loss function is calculated accordingly. Finally, the model parameters w are adjusted and updated according to the direction of the difference between the predicted label and the true label.

Claims

1. A privacy-preserving logistic regression method in a cloud environment, characterized in that, The participants in the method include a Key Generation Center (KGC), a Data Provider (DP), a Cloud Storage Server (CSS), a Cryptographic Service Provider (CSP), a Service Forwarder (SF), and a Service Requester (SR); the method uses DT-PKC (Distributed Two Trapdoors). A public-key cryptosystem (a distributed double-trapdoor public-key cryptosystem) includes the following steps: (1) System initialization: KGC generates system parameter para by inputting a random security parameter κ and running the algorithm DT-PKC.Setup(κ); (2) Key generation: KGC executes the algorithm DT-PKC.KeyGen(para) to generate a public-private key pair para=(N,g) and the system master key λ for all participants in the system. Then, KGC executes the algorithm DT-PKC.KeySplit(para,λ) to randomly assign an additional partial decryption key λ1 and λ2 to CSS and CSP respectively, so that CSS holds λ1 and CSP holds λ2. After completing this step, KGC can be offline. (3) Floating-point conversion: Given a positive integer E, representing the precision of a floating-point number, input a floating-point number. The main conversion method is: if the floating-point number x to be converted is positive, then use x... * =x×2 E As the integer converted from the floating-point number x; otherwise, x * =Nx×2 E As the integer obtained by converting the floating-point number x; (4) Safe integer multiplication: Use [x] k This represents the ciphertext encrypted with public key k, given plaintext x. The input keys PK1 and PK2 are used to encrypt the data. and Calculate the ciphertext under the new key PK3 without decryption. This operation is completed collaboratively by two cloud servers: CSS and CSP. The specific steps are as follows: (a) CSS first starts from group Z N Select four random numbers r x ,r y ,R x ,R y The four random numbers are then encrypted using public keys PK1 and PK2 respectively to obtain the corresponding ciphertext. Then, the ciphertext is calculated using the homomorphic property. Next, use part of its decryption key λ1 to decrypt the ciphertext. Perform the first step of decryption and obtain the intermediate result. Finally, the ciphertext and part of the decrypted ciphertext are... Send to CSP; (b) The CSP receives the message Then, using a portion of the decryption key λ2, a second decryption step is performed, first yielding the ciphertext. Next, CSP can obtain the plaintext. Then, calculate h = x' × y', and encrypt S, T, and h respectively using the public key PK3 required by the system. Finally, CSP will output the result. Send to CSS; (c) CSS receives encrypted text Then, calculate To remove random number r x r y R x R y Finally, calculate and output. As the ciphertext of the product; (5) Secure vector dot product operation: Input two n-dimensional vectors encrypted using keys PK1 and PK2 and Calculate the ciphertext encrypted with the new key PK3 without revealing the plaintext. The "." indicates a vector inner product operation, which is performed collaboratively by the CSS and CSP of two cloud servers. The specific operation is as follows: (a) CSS performs the following steps for each i (1≤i≤n): First from group Z N Select four random numbers The four random numbers are then encrypted using public keys PK1 and PK2 respectively, yielding the corresponding ciphertext. Then, the ciphertext is calculated using the homomorphic property. Next, use part of its decryption key λ1 to decrypt the ciphertext. The first step is decryption, yielding intermediate results. After performing the above steps n times, all data will be packaged into a message. Send to CSP; (b) The CSP receives the message Then, for each i (1≤i≤n), As input, the second step of decryption is performed using a portion of its decryption key λ2 to obtain the ciphertext. The plaintext message is calculated as follows: Perform a multiplication operation h on the obtained plaintext i =x i ′×y i Then, use the public key PK3 to encrypt h respectively. i S i T i Received news Finally, all the encrypted results Send to CSS; (c) CSS performs the following steps for each i (1≤i≤n): Calculate To remove each random number, we get Then, calculate and output. (6) Safe activation function Sigmoid operation: Input two ciphertext vectors encrypted with different public keys. Without revealing the plaintext of X and W, calculate the Sigmoid function z(X,W) = 1 / (1+e^(-1 / W)) under the new key PK3. -<X.W> ) ciphertext This was accomplished collaboratively by the CSS and CSP on two cloud servers, and the specific steps are as follows: (a) First, CSS and CSP perform a secure vector inner product operation on the input ciphertext vector. and Calculate the inner product, then CSS will output the result: [v] = [<X.W> The first step is to decrypt [v'] to obtain [v], and then ([v], [v']) is sent to CSP; (b) After receiving the message ([v],[v']), the CSP performs a second-step decryption to obtain the result v; then it uses the Taylor expansion e -v ≈1-v+1 / 2×v 2 -1 / 6×v 3 +1 / 24×v 4 -1 / 120×v 5 Approximate calculations are performed, resulting in Z = 1 / (1+e) -v Finally, the result Z is encrypted using the public key PK3 to obtain... And send it to CSS; (7) Training of privacy-preserving logistic regression model: In logistic regression training, all operations are performed on encrypted data. Gradient descent is used to update the model parameters w to obtain the optimal solution that minimizes the loss function. First, the model parameter vector w is initialized to 1. The encrypted form of the model parameters w and the training sample x is then input, and their inner product is calculated using secure vector inner product operations.<x.w> Then, the Sigmoid value is calculated from the inner product result using the Sigmoid safe activation function. Next, the difference between the predicted label and the true label is calculated based on the homomorphism of the DT-PKC algorithm, and the loss value of the loss function is calculated accordingly. Finally, the model parameters w are adjusted and updated according to the direction of the difference between the predicted label and the true label.

Images