System and method for performing differential private principal component analysis (PCA) on vertically partitioned data

By adopting a differential private principal component analysis system in federated learning, using Skellam noise and secret sharing protocol to process vertical partition data, the problem of vertical partition data privacy protection is solved, and the dual aspects of data and client privacy in federated learning are realized. Protect.

CN119004089AInactive Publication Date: 2024-11-22SHOPEE IP SINGAPORE PTE LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410607594.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-05-14
Filing Date
2024-05-16
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In federated learning, traditional differential privacy methods are mainly used to partition data horizontally, and have not yet effectively solved the privacy protection problem of vertical partition data. Secure multi-party computing can only prevent privacy leakage during the calculation process, but cannot prevent the calculation results from Infer private input.

Method used

Using a differential private principal component analysis (PCA) system, the first client and the second client discrete their respective column vectors, and introduce Skellam noise, combined with the secret sharing protocol BGW protocol, calculate without revealing private inputs. PCA results and simplify the results into covariance matrix to reduce the data size.

Benefits of technology

It realizes efficient privacy protection for vertical partitioned data, ensures local partition privacy of data sets and clients, significantly improves the level of data privacy protection, and avoids attackers from inferring private input from calculation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119004089A_ABST
    Figure CN119004089A_ABST
Patent Text Reader

Abstract

Systems and methods for performing differential private principal component analysis (PCA) on vertically partitioned data are disclosed. Various embodiments relate to a computer system for performing differential private principal component analysis (PCA) on vertically partitioned data. The system includes a first client having a first column vector and a second client having a second column vector. The first client is configured to discretize a first column vector to obtain a first discretized column vector, and the second client is configured to discretize a second column vector to obtain a second discretized column vector. The first client is configured to introduce a first noise to the first discretized column vector, and the second client is configured to introduce a second noise to the second discretized column vector to obtain a PCA result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Various aspects of the present disclosure are directed to systems and methods for performing differential private principal component analysis (PCA) on vertically partitioned data. Background Art

[0002] Data privacy is a key issue in federated learning (FL), where a server and multiple clients collaboratively train models on distributed data containing sensitive information. As a rigorous framework for protecting sensitive information, differential privacy (DP) has been widely adopted in FL to address privacy issues. However, traditionally, current approaches for DP in FL mainly focus on horizontally partitioned data, where sensitive datasets are partitioned among clients by records (i.e., rows). Federated learning naturally preserves a certain degree of data privacy by keeping data local to the client and providing limited access to local data to the server and other clients. Meanwhile, secure multi-party computation (MPC) has been applied to FL to further enhance data privacy, in the sense that clients can use MPC to jointly compute functions without directly exposing their sensitive inputs to others. However, MPC itself can only prevent privacy leakage during the computation process and cannot prevent attackers from inferring private inputs from the sensitive results of the computation. In other words, MPC solves the problem of how to compute sensitive results without revealing private inputs, but the content (i.e., results) of the MPC computation still leaks information about the inputs.

[0003] Currently, the use of DP in FL for vertically partitioned data (vertical FL), where a data set is partitioned by attributes (eg, columns), has not been accomplished. Accordingly, efficient and reliable methods for DP in FL for vertically partitioned data are desired. Summary of the invention

[0004] Various embodiments relate to a computer system for performing differential private principal component analysis (PCA) on vertically partitioned data. The system includes a first client having a first column vector and a second client having a second column vector. The first client is configured to discretize the first column vector to obtain a first discretized column vector, and the second client is configured to discretize the second column vector to obtain a second discretized column vector. The first client is configured to introduce a first noise to the first discretized column vector, and the second client is configured to introduce a second noise to the second discretized column vector to obtain a PCA result.

[0005] According to one embodiment, the PCA result is a combination of a first discretized column vector having a first noise and a second discretized column vector having a second noise.

[0006] According to one embodiment, the first client and the second client are configured to send the PCA results to the server for further processing using a secret sharing protocol.

[0007] According to one embodiment, the secret sharing protocol is the BGW protocol.

[0008] According to one embodiment, the BGW protocol is used to calculate a first discretized column vector with a first noise and a second discretized column vector with a second noise as a PCA result when the first client does not know the second discretized column vector and the second client does not know the first discretized column vector.

[0009] According to one embodiment, the server is configured to simplify the PCA results into a covariance matrix to reduce the size of the data.

[0010] According to one embodiment, the first client is configured to introduce a first noise and the second client is configured to introduce a second noise, using predetermined noise parameters.

[0011] According to one embodiment, both the first noise and the second noise are Skellan noise.

[0012] According to one embodiment, the first client is configured to discretize the first column vector and the second client is configured to discretize the second column vector using a predetermined discretization algorithm to reduce the size of the first column vector and the second column vector.

[0013] Various embodiments relate to a computer-implemented method for performing differential private principal component analysis (PCA) on vertically partitioned data, comprising: discretizing a first column vector from a first client to obtain a first discretized column vector; discretizing a second column vector from a second client to obtain a second discretized column vector; introducing a first noise to the first discretized column vector using the first client, and introducing a second noise to the second discretized column vector using the second client to obtain a PCA result.

[0014] According to one embodiment, the PCA result is a combination of a first discretized column vector having a first noise and a second discretized column vector having a second noise.

[0015] According to one embodiment, the first client and the second client are configured to send the PCA results to the server for further processing using a secret sharing protocol.

[0016] According to one embodiment, the secret sharing protocol is the BGW protocol.

[0017] According to one embodiment, the BGW protocol is used to calculate a first discretized column vector with a first noise and a second discretized column vector with a second noise as a PCA result when the first client does not know the second discretized column vector and the second client does not know the first discretized column vector.

[0018] According to one embodiment, the server is configured to simplify the PCA results into a covariance matrix to reduce the size of the data.

[0019] According to one embodiment, the first client is configured to introduce a first noise and the second client is configured to introduce a second noise, using predetermined noise parameters.

[0020] According to one embodiment, both the first noise and the second noise are Skellan noise.

[0021] According to one embodiment, the first client is configured to discretize the first column vector and the second client is configured to discretize the second column vector using a predetermined discretization algorithm to reduce the size of the first column vector and the second column vector.

[0022] According to one embodiment, a computer program element comprising program instructions is provided, which, when executed by one or more processors, causes the one or more processors to perform the above method.

[0023] According to one embodiment, a computer readable medium containing program instructions is provided. When the program instructions are executed by one or more processors, the one or more processors are caused to execute the above method.

[0024] It should be noted that embodiments described in the context of a method using a system for performing differential private principal component analysis (PCA) on vertically partitioned data are analogously valid for the system, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The invention will be better understood with reference to the detailed description when considered in conjunction with the non-limiting examples and accompanying drawings, in which:

[0026] Figure 1 An exemplary system for performing differential private principal component analysis (PCA) on vertically partitioned data is shown in accordance with an embodiment.

[0027] Figure 2 An exemplary method for performing differential private principal component analysis (PCA) on vertically partitioned data according to an embodiment is shown.

[0028] Figure 3 Graphs are shown illustrating the performance of PCA under various conditions according to an embodiment.

[0029] Figure 4 Graphs are shown that illustrate performance results under various conditions according to embodiments. Specific embodiments

[0030] The following detailed description refers to the accompanying drawings, which illustrate by way of illustration the specific details and embodiments in which the present disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present disclosure. Other embodiments may also be utilized, and structural and logical changes may be made without departing from the scope of the present disclosure. The various embodiments are not necessarily mutually exclusive, as some embodiments may be combined with one or more other embodiments to form new embodiments.

[0031] Embodiments described in the context of one of a system or a method are analogously valid for the other system or method. Similarly, embodiments described in the context of a system are analogously valid for the method, and vice versa.

[0032] Features described in the context of one embodiment may be correspondingly applicable to the same or similar features in other embodiments. Features described in the context of one embodiment may be correspondingly applicable to other embodiments, even if not explicitly described in other embodiments. In addition, additions and / or combinations and / or substitutions described for features in the context of one embodiment may be correspondingly applicable to the same or similar features in other embodiments.

[0033] In the context of various embodiments, the articles “a,” “an,” and “the” used for features or elements include reference to one or more of the features or elements.

[0034] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0035] The embodiments will be described in detail below.

[0036] Figure 1 An exemplary system for performing differential private principal component analysis (PCA) on vertically partitioned data is shown in accordance with an embodiment.

[0037] In various embodiments, data privacy is a key issue in federated learning (FL), where a server and multiple clients collaboratively train models on distributed data.

[0038] Federated learning naturally preserves a degree of data privacy by keeping the client's data local and providing limited access to local data to the server and other clients. At the same time, secure multi-party computation (MPC) has been applied to FL to further enhance data privacy, in the sense that clients can use MPC to jointly compute functions without directly exposing their sensitive inputs to others. However, MPC itself can only prevent privacy leakage during the computation process and cannot prevent attackers from inferring private inputs from sensitive results of the computation.

[0039] In other words, MPC solves the problem of how to compute sensitive results without revealing private inputs, but the content of the MPC computation (i.e., the result) still leaks information about the inputs.

[0040] In various embodiments, it is desirable to design a mechanism that (approximately) computes the principal components of an input dataset while simultaneously preserving two levels of differential privacy. These privacy guarantees include individual records in the dataset and local partitions of each participating client in the FL process.

[0041] The system (which can be called SSS-PCA (Skellam Secret Sharing-PCA)) cleverly integrates Skellam noise to enable DP in FL and adopts Secret Sharing, which further enhances privacy. More details of the system are introduced below.

[0042] In various embodiments, the system 100 includes a first client 101 , a second client 111 , and a server 121 .

[0043] In various embodiments, the first client 101 may have a first column vector 102 containing data of the first client 101. The first client may include a first client processor 103 for performing any computer-implemented functions with respect to the first column vector 102.

[0044] In various embodiments, the second client 111 may have a second column vector 112 containing data for the second client 111. The second client 111 may include a second client processor 113 for performing any computer-implemented functions with respect to the second column vector 112.

[0045] In various embodiments, the server 121 may include a server processor 123 for performing any computer-implemented functions for communicating information or data with respect to the first client 101 and the second client 111 .

[0046] In various embodiments, there is a processor 130 for processing information from the first client 101 and the second client 111. The processor 130 may be part of the server 121 or may be separate from the server 121.

[0047] In various embodiments, the first client is configured to discretize the first column vector to obtain a first discretized column vector.

[0048] In various embodiments, the second client is configured to discretize the second column vector to obtain a second discretized column vector.

[0049] In various embodiments, the first client is configured to introduce a first noise into the first discretized column vector, and the second client is configured to introduce a second noise into the second discretized column vector, to obtain a PCA result.

[0050] In various embodiments, there is a processor 130 for processing information from the first client 101 and the second client 111 .

[0051] In various embodiments, the processor 130 may receive a first discretized column vector and a first noise and / or a first discretized column vector having a first noise from a first client, and receive a second discretized column vector and a second noise and / or a second discretized column vector having a second noise from a second client. The processor 130 may obtain a PCA result using the first discretized column vector and the first noise and / or the first discretized column vector having the first noise, and the second discretized column vector and the second noise and / or the second discretized column vector having the second noise.

[0052] In various embodiments, the PCA result is a combination of a first discretized column vector having a first noise and a second discretized column vector having a second noise.

[0053] In various embodiments, the first client and the second client are configured to send the PCA results to a server for further processing using a secret sharing protocol.

[0054] In various embodiments, the secret sharing protocol is the BGW protocol.

[0055] In various embodiments, the BGW protocol is used to calculate a first discretized column vector with a first noise and a second discretized column vector with a second noise as a PCA result when the first client is unaware of the second discretized column vector and the second client is unaware of the first discretized column vector.

[0056] In various embodiments, the server is configured to simplify the PCA results into a covariance matrix to reduce the size of the data.

[0057] In various embodiments, the first client is configured to introduce a first noise, and the second client is configured to introduce a second noise, using predetermined noise parameters.

[0058] In various embodiments, the first noise and the second noise are Skellan noises.

[0059] In various embodiments, PCA is a dimensionality reduction method used to reduce the dimensionality of a large data set by converting a large set of variables into a smaller set of variables that still contains most of the information in the large set. Therefore, PCA has the ability to reduce the data size of information (such as the first column vector 102 and the second column vector 112).

[0060] Advantageously, PCA aims to reduce the number of attributes of a dataset D while capturing its covariance by performing a linear transformation. Covariance measures the total variation of two random variables relative to their expected values. Using covariance, we can only measure the direction of the relationship (whether the variables tend to move together (in tandem) or show an inverse relationship).

[0061] In various embodiments, for a data set (matrix) m represents the number of records, and n represents the number of attributes in D. In various embodiments, the rank k subspace (e.g., the matrix ), whose goal is to maximize the variance in DA, which is equivalent to finding the covariance matrix C = D T The first k eigenvectors of D.

[0062] The covariance matrix C is calculated as:

[0063]

[0064] For i, j∈[n]. Given the optimal rank k subspace, V k (For example, the k-dimensional principal singular subspace of C), the error of the rank k subspace A on the description matrix D is defined as:

[0065]

[0066] where ||·||F represents the Frobenius norm of the matrix. The error reaches a minimum (0) when the columns of A match the principal rank-k right singular subspace of D.

[0067] In various embodiments, differential privacy is a mathematical framework for ensuring the privacy of individuals in a dataset. It can provide strong privacy guarantees by allowing data to be analyzed without revealing sensitive information about any client in the dataset.

[0068] In various embodiments, differential privacy (DP) quantifies the indistinguishability of the output distribution of a mechanism over adjacent datasets, where D and D′ are called adjacent datasets (denoted as D~D′) if they differ by one record.

[0069] In various embodiments, there may be three types of DP (with different definitions).

[0070] In various embodiments, for Definition 1((∈,δ)-A, the random mechanism M:D→Y satisfies (∈,δ) differential privacy (DP) in the following case, i.e., for any It maintains:

[0071]

[0072] Here, smaller ∈ or δ means a higher difficulty in distinguishing the distributions, and hence a stronger privacy guarantee. Given a mechanism (function) of interest, the canonical way to make it differentially private is to perturb its results by injecting random noise, whose size is calibrated to the sensitivity of the mechanism.

[0073] In various embodiments, for Definition 2, let ||·|| represent the norm metric of interest (e.g., the L2 norm), then the function M: D→R n The sensitivity S(M) (denoted as S(M)) is defined as:

[0074]

[0075] In various embodiments, for Definition 3, the random mechanism M satisfies (α,τ) Renyi differential privacy (RDP) for some α∈(0,1)∪(1,∞) under the following conditions, i.e. where Dα(P||Q) represents the Renyi divergence of P and Q defined on the same domain.

[0076] In various embodiments, secret sharing is a method of distributing a secret among a group of parties in a manner such that: 1) no single party learns any non-trivial information about the secret; and 2) when a sufficiently large number of parties combine their information, the secret is reconstructed.

[0077] Lagrange interpolation theorem states that N points on a polynomial uniquely define a polynomial of degree less than N. With this in mind, the secret holder divides the secret into N parts by computing N points on a polynomial of degree N-1, whose constant term is exactly the original secret, and the other N-1 coefficients are randomly selected by the secret holder and kept secret. Therefore, any subset with fewer than N members cannot learn any non-trivial information about the secret. On the other hand, N members can reconstruct the secret by polynomial interpolation.

[0078] In various embodiments, one type of secret sharing may be the BGW (Ben-Or, Goldwasser, and Widgerson) protocol.

[0079] In various embodiments, in the BGW protocol, the result of a polynomial function f(s1, ..., sN), where the function is known to all members without revealing private inputs to others. The three steps of BGW are as follows:

[0080] 1. First, each client distributes its private input as a secret share to other clients using the Shamir algorithm. Note that no client can infer the private input of any other client, which is enforced by the security properties of the Shamir algorithm.

[0081] 2. Next, each client simulates the calculation of function f using a digital circuit, while keeping the value of each secret as a secret shared by all member parties. Similar to the first step, no member party can infer the value of each secret.

[0082] 3. Finally, all clients use their shared secrets to reconstruct the true result of f.

[0083] The BGW protocol implements the notion of information-theoretic security, i.e., each client must never learn more about the private inputs of other clients than it does from the output itself (i.e., f(s1,...,sN)). Note that this guarantee of BGW is different from DP, since an attacker can still extract information about the private inputs from the sensitive result of f.

[0084] In various embodiments, noise is introduced into the first vertical column and the second vertical column for additional privacy. The noise can be Gaussian noise or Skellam noise. Skellam noise is noise using a Skellam distribution.

[0085] If a random variable Z is the difference between two independent Poisson random variables with the same parameter μ, then the random variable Z follows a symmetric Skellam distribution with respect to the parameter μ. By the linearity of the Poisson distribution, the sum of two independent Skellam noises sampled from Sk(μ1) and Sk(μ2) has distribution Sk(μ1+μ2). Sk(μ) is the set of n independent random variables sampled from Sk(μ). The following divergence guarantees are the results of perturbing an integer-valued function with noise sampled from Sk(μ).

[0086] For any integer α > 1, and any n-dimensional integer-valued function F with bounded output range, it holds that:

[0087]

[0088] In various embodiments, for principal component analysis in vertical federated learning, let Represents sensitive data, where each line Represents the i-th (i∈[m]) record vector consisting of n real-valued attributes, with each column A vector representing attributes j∈[n] from all records.

[0089] For vertical federated learning (VFL), each client has a set of attributes (e.g., a subset of columns in D). More specifically, if there are N clients, client q has a subset Dq (q∈[N]) of D. Here Dq consists of a set of attributes Aq such that Dq={D[:,j]:j∈Aq}. Furthermore, assume that the set of columns (respectively, attributes) {Dq:q=1,...,N} (respectively, {Aq:q=1,...,N}) is mutually exclusive. Given an integer k (usually k<<n), the untrusted server aims to (approximately) learn the rank k subspace This subspace retains most of the variance in the original dataset D, that is, maximizes The disclosed privacy-preserving mechanism is able to find such a V k , while preserving differential privacy with respect to any individual record such as D[i,:], and any portion of the dataset of a client such as Dq.

[0090] In various embodiments, the threat model under vertical federated learning involves three entities: an individual record D[i,:] (i∈[m]), a client q with a portion of the dataset Dq (q∈[N]), and an untrusted server that wants to learn about the dataset D.

[0091] First, assume that each individual trusts each client but does not trust the server. This means that the client q has full access to the corresponding private part Dq, which contains sensitive information of the individual record (including information used to identify the individual) that needs to be protected from the server. The validity of this assumption can be explained by an example. For example, in an online shopping scenario, given the same group of customers, each online retailer regarded as a client has the purchase or browsing history of each individual customer, which is regarded as an attribute. However, in addition to these trusted clients, individuals may not want any other entity (including the server) to know their own private records (including information used to identify the individual), which also means that for the server, it is regarded as a private record. The above discussion leads to the following DP definition.

[0092] In Definition 4, ((∈,δ)-record-server DP). The random mechanism M satisfies (∈,δ)-record-server DP in the following case, that is, for any set of output It maintains:

[0093]

[0094] Here, “~” indicates the adjacent relationship between two data sets (correspondingly, matrices), that is, one can be obtained from the other by adding / deleting a record (correspondingly, a row).

[0095] Using the online retailer example, a client (e.g., an online retailer) also has an incentive to prevent its competitors from learning about its private data partitions (e.g., a customer's purchase or browsing history) because otherwise, competitors could send personalized advertisements to the client's customers, which could cause damage to the client's business, which leads to the following DP definition.

[0096] In Definition 5, the random mechanism M satisfies (∈,δ)-partition 210 clients DP if, for each client q and private partition Dq and any set of output It maintains:

[0097]

[0098] Here “~” indicates the adjacency relation, that is, a set can be obtained from another set by replacing one row. For any q = 1, ..., N, we use To represent the complement matrix of Dq taken from the complete matrix D. We write to represent the inverse operation, that is, by substituting Dq and Concatenate to construct the matrix.

[0099] Note that the adjacency relation in the partition-client DP is defined on the action of replacing a row, not on the action of adding / deleting a row in the record-server DP. This is because each client has full access to their private data portion, including the information needed to identify the individual. Therefore, assuming that the records are aligned in the client's partition, and m is considered non-private to the client, this means that the only information the client is curious about is the value of the attribute in the partitions of other clients. Each member (server or client) strictly follows the execution of M when trying to infer the private data in the partitions of other clients.

[0100] In various embodiments, each client first independently perturbs its local data (e.g., a submatrix) with random Gaussian noise. Next, all clients reveal their perturbed local data to one of the clients (without loss of generality, e.g., Client 1), which then reconstructs the entire perturbed matrix; computes the covariance matrix; and sends the first k eigenvectors of the covariance matrix to the server. The privacy guarantee of this baseline is derived from the DP guarantee of additive Gaussian noise, and post-processing (computing eigenvectors) preserves DP.

[0101] In various embodiments, SSS-PCA builds on the classic DP algorithm for PCA and injects independent noise into the upper diagonal of the covariance matrix C=DTD, which is defined as follows: for i, j∈[n]. The attacker can then compute the k-dimensional singular subspace of the perturbed covariance matrix C~ using a standard non-DP algorithm. The privacy guarantee stems from the fact that C~ is already differentially private and the post-processing preserves DP.

[0102] First, the exact C[i,j] is computed from two columns D[:,i] and D[:,j], which can be stored in different clients. In this case, the computation of C[i,j] cannot be done by a single client alone, which requires the use of secret sharing to perform this computation without directly revealing the private inputs (e.g., the columns of D). However, secret sharing does not prevent curious parties from inferring the private inputs from the computation results, so injecting noise into the inputs preserves differential privacy.

[0103] SSS-PCA is shown in Algorithm 1 below.

[0104] The entry at the jth column of the i-th row of the covariance matrix needs to be computed while maintaining privacy. The first case is when D[:,i] and D[:,j] are two columns (including the case when i=j). In this case, the client p with (one or more) columns discretizes the (one or more) column vectors using Algorithm 2 and obtains D[:,i] (and D[:,j]) (line 5 in Algorithm 1), where the overline indicates the discretized version. Next, the client samples the Skellam noise zp from Sk(2μ) (line 6) and computes:

[0105]

[0106]

[0107]

[0108] The client reports the result to the server (line 7). Another case is when columns D[:,i] and D[:,j] are owned by different clients (e.g., client p and client q). In this case, client p (respectively, client q) having column vector D[:,i] (respectively, vector D[:,j]) privately and independently discretizes the vector, obtaining an integer-valued column vector D[:,i] (respectively, vector D[:,j]) (line 9). Next, the two clients privately and independently generate two Skellan noises zp and zq from Sk(2μ) (line 10). Then, the two clients use the BGW protocol to calculate:

[0109]

[0110] The two clients report the results to the server (line 11). After completing the calculation of all upper diagonal terms, the server obtains the perturbed covariance matrix by scaling down the results (line 12) And calculate its first k eigenvectors by itself.

[0111] In various embodiments, the noise injection to R[i, j] in equation (6) must be contributed by two clients (zp and zq). Otherwise, a single client (or any other member party) performing the noise injection can infer private information about the basis D from the exact result.

[0112] SSS-PCA satisfies DP, both with respect to records and clients.

[0113] Lemma 2. Given a discretization parameter γ and a noise parameter μ, Algorithm 1 satisfies the (α, τserver) record-server RDP, where:

[0114]

[0115] Algorithm 1 also satisfies (α,τclient)-partition-client RDP, where

[0116]

[0117] in

[0118] Lemma 3. Let Vk be the rank k subspace of the original matrix D, and let For the matrix The main rank k subspace of (computed with additive Skellam noise) has high probability

[0119]

[0120] Lemma 3 quantifies To what extent R captures the variance of D. To prove Lemma 3, it suffices to quantify to what extent R captures the variance of γD. The problem simplifies to bounding the maximum eigenvalue (spectral norm) of the matrix R-γ2DTD in the worst case, which can be decomposed into Ed+Ep, where Ed is a symmetric random matrix whose entries represent the randomness due to discretization (refer again to line 4 in Algorithm 1), and Ep is a symmetric random matrix whose entries represent the randomness due to DP (e.g., each entry is sampled from Sk(2μ)). The entries in Ed (in the upper diagonal) are independent random variables uniformly bounded with finite variance, and the entries in Ep (in the upper diagonal) are subexponential, which means that E d and E p The largest eigenvalue of has the order with high probability 1-o(1 / n)

[0121] Figure 2 An exemplary method for performing differential private principal component analysis (PCA) on vertically partitioned data according to an embodiment is shown.

[0122] like Figure 2 As shown, there may be a method 200 for performing differential private principal component analysis (PCA) on vertical partitioned data. In the method 200, a first step 202 may include discretizing a first column vector from a first client to obtain a first discretized column vector. A second step 204 may include discretizing a second column vector from a second client to obtain a second discretized column vector. A third step 206 may include introducing a first noise to the first discretized column vector using the first client, and introducing a second noise to the second discretized column vector using the second client to obtain a PCA result.

[0123] It is understandable that the above Figure 2 The operations involved are not limited to this particular order and any suitable, modified order of operations may be used.

[0124] Figure 3 Graphs are shown illustrating the performance of PCA under various conditions according to an embodiment. Figure 4 Graphs are shown that illustrate performance results under various conditions according to embodiments.

[0125] The solution is compared with existing solutions on the KDDCUP dataset (m=195666 and n=817) and the California ACSIncome dataset (m=145585 and n=117), as shown in Figure 3 and Figure 4 shown.

[0126] Figure 3The average utility over 20 independent runs in , from which it can be clearly seen that the solution consistently outperforms the baseline solution under various parameter settings and is close to the strongly centralized DP baseline, confirming the efficacy of SSS-PCA. In particular, for KDDCUP, the performance of SSS-PCA is almost the same as the strongly centralized DP baseline. For ACSIncome, the difference between SSS-PCA and the centralized baseline is only small when γ ≥ 6, and note that even for γ = 4, SSS-PCA still performs significantly better than the distributed baseline.

[0127] Figure 4 Shown is a plot of KDDCUP for k=2, where both the centralized DP algorithm and the solution obtain a high-quality two-dimensional subspace for the input data set because the normal points and non-normal points are significantly separated when projected onto the subspace.

[0128] The methods described in the present disclosure may be executed, and the various processing or computing units and devices and computing entities described in the present disclosure may be implemented by one or more circuits. In an embodiment, a "circuit" may be understood as any type of logic implementation entity, which may be hardware, software, firmware, or any combination thereof. Therefore, in an embodiment, a "circuit" may be a hardwired logic circuit or a programmable logic circuit, such as a programmable processor, such as a microprocessor. A "circuit" may also be software implemented or executed by a processor, such as any type of computer program, such as a computer program using virtual machine code. Any other type of implementation of the functions described herein may also be understood as a "circuit" according to alternative embodiments.

[0129] Although the present disclosure has been particularly shown and described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the scope of the present invention is indicated by the appended claims, and therefore, it is intended to include all changes within the equivalent meaning and range of the claims.

Claims

1. A computer system for performing differential private principal component analysis (PCA) on vertically partitioned data, comprising: a first client having a first column vector; a second client having a second column vector; wherein the first client is configured to discretize the first column vector to obtain a first discretized column vector, and the second client is configured to discretize the second column vector to obtain a second discretized column vector; The first client is configured to introduce a first noise into the first discretized column vector, and the second client is configured to introduce a second noise into the second discretized column vector, so as to obtain a PCA result.

2. The computer system of claim 1, wherein the PCA result is generated based on a combination of the first discretized column vector having the first noise and the second discretized column vector having the second noise.

3. The computer system of claims 1-2, wherein at least one of the first client and the second client is configured to send the PCA results to a server for further processing using a secret sharing protocol. The computer system of claim 3 , wherein the secret sharing protocol is a BGW protocol.

5. A computer system according to claim 4, wherein the BGW protocol is used to calculate the first discretized column vector with the first noise and the second discretized column vector with the second noise into the PCA result when the first client is unaware of the second discretized column vector and the second client is unaware of the first discretized column.

6. The computer system of claims 3-5, wherein the server is configured to simplify the PCA results into a covariance matrix to reduce the size of the data.

7. The computer system of claims 1-6, wherein the first client is configured to introduce the first noise, and the second client is configured to introduce the second noise, using predetermined noise parameters.

8. The computer system of claims 1-7, wherein the first noise and the second noise are Skellan noises.

9. A computer system according to claims 1-8, wherein the first client is configured to discretize the first column vector and the second client is configured to discretize the second column vector, using a predetermined discretization algorithm to reduce the size of the first column vector and the second column vector.

10. A computer-implemented method for performing differential private principal component analysis (PCA) on vertically partitioned data, comprising: discretizing a first column vector from a first client to obtain a first discretized column vector; discretizing a second column vector from a second client to obtain a second discretized column vector; A first noise is introduced into the first discretized column vector using the first client, and a second noise is introduced into the second discretized column vector using the second client to obtain a PCA result. 11 . The computer-implemented method of claim 10 , wherein the PCA result is generated based on a combination of the first discretized column vector having the first noise and the second discretized column vector having the second noise.

12. The computer-implemented method of claims 10-11, wherein at least one of the first client and the second client is configured to send the PCA results to a server for further processing using a secret sharing protocol.

13. The computer-implemented method of claim 12, wherein the secret sharing protocol is a BGW protocol.

14. A computer-implemented method according to claim 13, wherein the BGW protocol is used to calculate the first discretized column vector with the first noise and the second discretized column vector with the second noise into the PCA result when the first client is unaware of the second discretized column vector and the second client is unaware of the first discretized column.

15. The computer-implemented method of claims 12-14, wherein the server is configured to simplify the PCA results into a covariance matrix to reduce the size of the data.

16. The computer-implemented method of claims 10-15, wherein the first client is configured to introduce the first noise, and the second client is configured to introduce the second noise, using predetermined noise parameters.

17. The computer-implemented method of claims 10-16, wherein the first noise and the second noise are Skellan noises.

18. The computer-implemented method of claims 10-17, wherein the first client is configured to discretize the first column vector and the second client is configured to discretize the second column vector using a predetermined discretization algorithm to reduce the size of the first column vector and the second column vector.

19. A computer program element comprising program instructions which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 10 to 17.

20. A computer readable medium comprising program instructions which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 10 to 17.