Efficient pearson coefficient calculation method based on third party in federated learning environment

Abstract

This invention presents an efficient Pearson coefficient calculation method based on a third party in a federated learning environment. The method utilizes the open-source FATE framework for overall computational communication of the Pearson coefficients. The two parties involved in calculating the feature correlation coefficients are party A and party B, with party C serving as the semi-honest third party. Existing schemes require numerous homomorphic encryption operations to securely generate Beaver Triplets. The new scheme, through a semi-honest third party, eliminates homomorphic encryption operations without sacrificing security, securely generating (a,b,c) Beaver Triplets while allowing each party to share the addition secret of (a,b,c). Since the numerous large integer modular exponentiation operations in the original Paillier homomorphic encryption are eliminated, and only tensor dot products and addition / subtraction operations are used, efficiency is significantly improved.

Description

Technical Field

[0001] This invention relates to a method for calculating efficient Pearson coefficients based on third parties in a federated learning environment. Background Technology

[0002] The following are some basic concepts in this field:

[0003] Federated machine learning: The application of privacy computing in the field of machine learning, which can integrate data from multiple parties without disclosing the privacy data of each party, and use machine learning algorithms to train models and make predictions.

[0004] The security models for federated learning are mostly semi-honest models.

[0005] Feature engineering: A data preprocessing method in machine learning engineering that filters and discretizes the feature data of samples in order to train better machine learning models later.

[0006] Pearson coefficient: A method for calculating data correlation, which can be used in feature engineering to calculate the correlation between features in sample data, in order to filter out irrelevant and redundant features that have little effect on the overall model's prediction. This facilitates better and more efficient training of machine learning models later.

[0007] Semi-honest security model: A model used when analyzing the security of a computing or communication protocol. In this security model, the participants in the protocol strictly adhere to the relevant steps and requirements of the protocol, but attackers may attempt to obtain other data that they should not know through the data acquired during the protocol process.

[0008] Secure multi-party computation: Secure multi-party computation is a cryptographic technique and a popular research area in cryptography, falling under the category of privacy-preserving computation. It is primarily used when two or more parties wish to collaboratively compute a function based on their input data without revealing their own private inputs. In secure multi-party computation, apart from the final result, the inputs of each party remain private.

[0009] This research direction originated from the Millionaire Problem posed by Turing machine laureate Andrew Yao. Early solutions were inefficient and impractical. In recent years, with the continuous development of this technology, efficiency has greatly improved, and it has begun to see practical applications.

[0010] Fully homomorphic encryption: Fully homomorphic encryption is an encryption algorithm in which any calculation can be performed on the ciphertext (either arithmetic addition, multiplication, or logical bit XOR and AND), and the decryption result is the same as the result of the calculation on the plaintext.

[0011] Semi-homomorphic encryption and finite-level fully homomorphic encryption: Currently, due to the efficiency and storage limitations of fully homomorphic encryption in applications, semi-homomorphic encryption or finite-level fully homomorphic encryption is more widely used. Semi-homomorphic encryption algorithms only support homomorphic operations of ciphertext addition or multiplication, while finite-level fully homomorphic encryption supports homomorphic addition and multiplication of finite levels.

[0012] Federated learning, proposed by Google in 2017, combines data from multiple users and uses machine learning algorithms to train more accurate models, thereby providing better recommendations and other services to users. To better protect user privacy, users do not need to transfer private data to intermediary service providers like Google. All machine learning training calculations are completed locally on the user's machine. Users only need to transmit the final results, such as gradients, to Google. Google then integrates the results and sends the new model to all users. All users then undergo a new round of learning and training, repeating the process until a satisfactory model is output.

[0013] Federated learning, by eliminating data silos and combining data from various sources to train better predictive models and provide better services while protecting user data privacy, has attracted widespread attention since its inception, with many internet and financial companies venturing into the field.

[0014] WeBank is one such company. Its federated learning framework, FATE, is designed for the financial sector, enabling secure data sharing between enterprises for machine learning training. Due to its simplicity, practicality, efficiency, and open-source nature, it has attracted a large user base, boasting approximately 9,000 stars and 1,000 branches on GitHub.

[0015] Feature engineering is a crucial component of the FATE technical framework. It removes irrelevant features and discretizes feature data to train more accurate machine learning models that better suit real-world applications. FATE (Federated AI Technology Enabler) is an open-source project initiated by WeBank's AI department, providing a reliable and secure computing framework for the federated learning ecosystem.

[0016] The Pearson coefficient between features is used to determine the linear correlation between features, and is used to remove features with high linear correlation in order to better perform machine learning.

[0017] However, the calculation of Pearson coefficients for feature data among the participants in FATE references the multi-party computation framework SPDZ, where the generation of Beaver Triplets involves a large number of Paillier homomorphic encryption operations. This computational process makes feature correlation calculation extremely slow for large-scale data volumes; for example, with tens of millions of data points, the computation task cannot be completed in a day, making FATE's data correlation calculation impractical.

[0018] Currently, FATE uses a multi-party computation framework similar to SPDZ, which calculates the correlation between the characteristics of the participants through secret sharing and Paillier homomorphic encryption. At the same time, it removes the ciphertext encryption under malicious adversaries and MAC verification in SPDZ to improve the efficiency of computation.

[0019] However, this method still uses a large number of Paillier homomorphic public-key cryptographic operations to generate BeaverTriplets (triplets), resulting in relatively inefficient calculation of the overall Pearson correlation coefficient for FATE, making it impractical for large-scale data. For example, with tens of millions of data points, the calculation task cannot be completed in a day, making FATE's data correlation calculation very impractical. Summary of the Invention

[0020] This invention addresses the current FATE technical framework, which uses numerous Paillier homomorphic public-key encryption operations to generate Beaver Triplets. It provides a third-party efficient Pearson coefficient calculation method for federated learning environments. This method eliminates the large number of large integer modular exponentiation operations in the original Paillier homomorphic encryption scheme, using only tensor dot products and addition / subtraction operations, thus significantly improving efficiency.

[0021] The technical solution adopted by this invention to achieve its technical objective is: a third-party efficient Pearson coefficient calculation method in a federated learning environment. In this method, the open-source FATE framework is selected as the overall communication framework for calculating the Pearson coefficient. The two parties involved in the feature correlation coefficient calculation are party A and party B, and the semi-honest third party is party C. The method includes the following steps:

[0022] Step S1: The two parties A and B involved in calculating the correlation coefficient will perform an addition secret sharing of their respective feature data tensors x and y. Each party will receive an addition secret sharing share of the other party's data tensor.

[0023] Step S2: After obtaining the addition secret shared share of each other's data tensors x and y, the two parties A and B involved in calculating the correlation coefficient each generate a tensor a of the same size as the feature data tensors x and y locally. i ,b i , which is a secret shared as one of the triples a and b;

[0024] Step S3: The two parties involved in calculating the correlation coefficient, A and B, will each generate tensor a. i ,b i Send to a semi-honest third party C;

[0025] Step S4: Semi-honest third party C uses the received tensor a i ,b i Add them together to get a and b in the triplet, and further calculate c in the triplet, where c is the dot product between tensors a and b;

[0026] Step S5: The semi-honest third party C performs an addition operation on c and secretly shares it to generate c1 and c2, and sends them to the two parties A and B who are participating in the calculation of the correlation coefficient, respectively.

[0027] Step S6: Both parties A and B involved in calculating the correlation coefficient obtain c respectively. i Then, each person uses their existing triplet share a. i ,b i ,c i and tensor x i ,y i z is obtained through interactive calculation with the other party. i , z i It is equal to the secret shared share of the addition of the dot product z between tensors x and y;

[0028] Step S7: The two parties involved in calculating the correlation coefficient, A and B, interact with each other. i The two values ​​are summed to obtain z, which is used as the output of the Pearson correlation coefficient tensor for both features.

[0029] Furthermore, in the above-mentioned efficient Pearson coefficient calculation method based on a third party in the federated learning environment, the method includes the following steps before step S1:

[0030] Step S0: The two parties involved in calculating the correlation coefficient, A and B, perform sample alignment based on their respective input data.

[0031] Furthermore, in the above-mentioned federated learning environment based on a third-party efficient Pearson coefficient calculation method, the sample alignment algorithm adopts a secure privacy intersection algorithm.

[0032] Furthermore, in the above-mentioned federated learning environment based on the efficient Pearson coefficient calculation method of a third party: the sample alignment algorithm used is the privacy-preserving intersection algorithm based on RSA that comes with FATE.

[0033] Furthermore, in the above-mentioned federated learning environment based on the efficient Pearson coefficient calculation method of a third party: step S0 further includes: the two parties A and B involved in calculating the correlation coefficient need to preprocess their respective datasets.

[0034] Furthermore, in the aforementioned federated learning environment based on a third-party efficient Pearson coefficient calculation method, the preprocessing of the dataset includes:

[0035] For a dataset X of A that participates in calculating the correlation coefficient, for each x i ∈X, first calculate sum_x=∑x i ,sum_x 2 =∑x i 2

[0036] avg_x = sum_x / n, where n is the number of data points X.

[0037] So

[0038] Find all x in X i Replace with x i And output it as a new dataset, denoted as x, where x i '=(x i -avg_x) / α. Similarly, dataset Y of side B is processed in the same way, outputting dataset y.

[0039] In this invention, a semi-honest third party is introduced to generate BeaverTriplets in the multi-party computation framework for calculating Pearson coefficients. Compared with the original FATE federated learning framework which uses Paillier fully homomorphic encryption to generate BeaverTriplets, the computational efficiency is greatly improved.

[0040] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0041] Appendix Figure 1 This is a flowchart of Embodiment 1 of the present invention. Detailed Implementation

[0042] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described examples are only a part of the embodiments of the present application, and not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0043] Example 1, such as Figure 1 The diagram illustrates an efficient Pearson coefficient calculation method based on a third party in a federated learning environment. This method uses the open-source FATE framework for overall computation and communication of the Pearson coefficients. The two parties involved in calculating the feature correlation coefficients are party A and party B, with party C serving as the semi-honest third party. The specific steps include:

[0044] (1) First, select a framework for overall computation and communication that needs to calculate Pearson coefficients. Here, we choose the open-source FATE.

[0045] (2) Assume that the two parties involved in the calculation of the feature correlation coefficient are party A and party B, and the semi-honest third party is party C.

[0046] (3) Both A and B need to perform sample alignment based on their respective input data (filtering data with the same id but different features). The sample alignment algorithm can be any secure privacy intersection algorithm. Here we choose the privacy intersection algorithm based on RSA that comes with FATE.

[0047] (4) After sample alignment, A and B each output datasets X and Y.

[0048] (5) In order to calculate the feature correlation in the next step, both A and B need to preprocess their respective datasets. Specifically:

[0049] First, for dataset X of A, for each x... i ∈X, first calculate sum_x=∑x i ,sum_x 2 =∑x i 2

[0050] avg_x = sum_x / n, where n is the number of data points X.

[0051] So

[0052] Find all x in X i Replace with x i And output it as a new dataset, denoted as x, where x i '=(x i-avg_x) / α

[0053] Similarly, dataset Y from side B is processed in the same way, and dataset y is output.

[0054] The final required feature data Pearson correlation coefficient is calculated by dot(x,y), where dot represents the dot product between tensors.

[0055] (6) The two parties, A and B, involved in calculating the correlation coefficient, secretly share their respective feature data tensors x and y by addition. Each party will receive a secret share of the other party's data tensor by addition. Specifically: Party A secretly shares the dataset x by addition x = x1 + x2 and sends x2 to Party B. Similarly, Party B secretly shares y by addition y = y1 + y2 and sends y2 to Party A.

[0056] (7) After obtaining each other's data x and y shares, the two parties involved in calculating the correlation coefficient, A and B, respectively, generate a tensor (a) of the same size as the feature data tensor x and y locally. i ,b i As a secret shared by (a,b) in Beaver Triplets, each (a) i ,b i The data is sent to a semi-honest third party C. Specifically, party A generates random tensors (a1, b1) of the same size as x and y locally and sends them to the semi-honest third party C. Party B also generates random tensors (a2, b2) of the same size as x and y locally and sends them to the semi-honest third party C.

[0057] (8) C uses the received (a) i ,b i The tensors are added together to obtain (a,b) in Beaver Triplets, and then c = dot(a,b) is calculated in Beaver Triplets, where dot represents the dot product between tensors. A semi-honest third party C performs an addition operation on c calculated in the previous step to secretly share and generate c1 and c2, and sends them to A and B respectively. Specifically, C calculates a = a1 + a2, b = b1 + b2, and simultaneously calculates c = dot(a,b), performs an addition operation on c to secretly share c = c1 + c2, and sends it to A and B respectively.

[0058] (9) Both A and B received c i Then, each uses Beaver Triplets' share(a) function. i ,b i ,c i ), and (x i ,y i z is obtained through interactive calculation with the other party.i =dot i (x, y). Specifically, both A and B obtained (x...). i ,y i ),(a i ,b i ,c i ), which are the datasets (x,y) and (a,b,c)beaver trplets triplet addition secret share (share), respectively. For the convenience of the following description, the share of the data can also be represented as [a],[b],[c].

[0059] (10) Both parties interact z i And add them together to get z, which is output as the tensor of the correlation coefficient between the two features. Specifically, A and B calculate their respective [x+a] and [y+b] and send them to each other. After adding their own shares to the shares received from the other party, they get the complete k = x+a; j = y+b. A and B calculate their respective [z] = [a*b] - k[b] - j[a] + k*j, that is, z i The data is then sent to the other party. The received share is added to the original share to obtain the complete z = dot(x,y), which is the final output of the correlation data between the features of both parties.

[0060] In this embodiment: In order to securely generate Beaver Triplets, a large number of homomorphic encryption operations are required. In the new scheme, a semi-honest third party can eliminate the homomorphic encryption operations without sacrificing security and securely generate (a,b,c) Beaver Triplets, while allowing both parties to obtain the addition secret share of (a,b,c).

[0061] Since the original Paillier homomorphic encryption scheme no longer involves a large number of large integer modular exponentiation operations, and only tensor dot products and addition and subtraction operations are used, the efficiency will be greatly improved.

Claims

1. A method for efficient calculation of Pearson coefficient based on a third party in a federated learning environment, wherein open-source FATE is selected as a framework for overall calculation and communication of Pearson coefficient, two parties participating in calculation of feature correlation coefficient are A party and B party, and a semi-honest third party is C party; and the method is characterized in that: Comprise the following steps: Step S1, the two parties A, B participating in calculating the correlation coefficient make an additive secret sharing of the feature data tensor x of A and the feature data tensor y of B, the A party obtains the additive secret sharing share y of the data tensor y of the B party i , and the B party obtains the additive secret sharing share x of the data tensor x of the A party i ; Step S2, the two parties A, B participating in the calculation of the correlation coefficient, after obtaining the additive secret sharing of the data tensor x, y of the other party, generate a tensor a of the same size as the characteristic data tensor x, y locally i ,b i , as one of the additive secret sharing in the triple a, b; Step S3, party A generates a random tensor [a1, b1] of the same size as x, y locally and sends it to the semi-honest third party C, and party B also generates a random tensor [a2, b2] of the same size as x, y locally and sends it to the semi-honest third party C; Step S4, the semi-honest third party C uses the received tensor a i b i The a, b in the triple are added, and the c in the triple is further calculated, and the c is the dot product between the tensors a and b; Step S5, the semi-honest third party C makes a secret sharing of c to generate c1 and c2, and sends them to parties A and B participating in the calculation of the correlation coefficient respectively; Step S6, after obtaining c1, c2, the two parties A, B participating in the calculation of the correlation coefficient respectively, A uses the existing triple shares a i ,b i ,c i , and the tensor x i ,y i , obtains z i by interacting with the other party, B uses the existing triple shares a i ,b i ,c i , and the tensor x i ,y i , obtains z i by interacting with the other party, z i is equal to the additive secret sharing share of the dot product z between the data tensors x, y; Step S7, the two parties A, B participating in the calculation of the correlation coefficient each interact z i , and add up to get z as the correlation coefficient tensor output of the two parties.

2. The method of claim 1, wherein the method is based on a third-party efficient Pearson correlation coefficient calculation in a federated learning environment. Before step S1, it also includes: Step S0, parties A and B participating in the calculation of the correlation coefficient perform sample alignment according to their respective input data.

3. The method of claim 2, wherein the method is based on a third-party efficient Pearson correlation coefficient calculation in a federated learning environment. The sample alignment algorithm adopts a secure private intersection algorithm.

4. The method of claim 3, wherein the method is based on a third-party efficient Pearson correlation coefficient calculation in a federated learning environment. The algorithm adopted by the sample alignment algorithm is a private intersection algorithm based on RSA provided by FATE.

5. The method of claim 2, wherein the method is based on a third-party efficient Pearson correlation coefficient calculation in a federated learning environment. In the step S0, parties A and B participating in the calculation of the correlation coefficient also need to preprocess their respective data sets.

6. The method of claim 5, wherein the method is based on a third-party efficient Pearson correlation coefficient calculation in a federated learning environment. The preprocessing of the data set includes: For the dataset x of A participating in the computation of the correlation coefficient, for each x j ∈x, first compute sum_x = ∑x j ,sum_x 2 = ∑x j 2 avg_x = sum_x / n, where n is the number of elements of the data set x; So Replace all x in X with j avg_x j ' and output as x, where x j ' = (x j - avg_x) / alpha; the same is done for the data set y of party B, outputting data set y.

Images