Method and apparatus for privacy-preserving multi-party joint dimensionality reduction processing

Abstract

The embodiment of the specification provides a method and device for multi-party joint dimension reduction processing for privacy protection. The method comprises the following steps: reading local features of M samples in a current batch according to a predetermined sample sequence to construct a local original matrix; locally calculating a first matrix obtained by multiplying a transpose matrix of the local original matrix with itself, and multiplying the transpose matrix of the local original matrix with original matrices of other parties by using multi-party secure calculation to obtain a second matrix and multiplying transpose matrices of the original matrices of the other parties with the local original matrix to obtain a third matrix; splicing the first matrix, the second matrix, the third matrix and first matrices of the other parties to form a merged matrix; performing mean value processing on the merged matrix to obtain a center matrix; accumulating the center matrices of each batch to obtain a covariance matrix; and performing multi-party joint dimension reduction processing on sample feature data of each batch by using the covariance matrix. The processing speed can be improved.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of computers, and more particularly to methods and apparatus for multi-party collaborative dimensionality reduction processing to protect privacy. Background Technology

[0002] In machine learning, principal component analysis (PCA) is a commonly used unsupervised data analysis algorithm. By exploring high-dimensional data structures, PCA can perform dimensionality reduction and visualization, thereby accelerating the processing of valuable information from samples.

[0003] However, sample feature data is often distributed among multiple data owners, and the feature data of the samples owned by each data owner is considered their private data. During PCA collaboration among multiple data owners, their private data must not be disclosed. In existing technologies, to protect privacy data, PCA processing speeds for large datasets are typically low.

[0004] Therefore, we hope to find an improved solution that can increase processing speed in the multi-party collaborative dimensionality reduction process that protects privacy. Summary of the Invention

[0005] This specification describes one or more embodiments of a method and apparatus for multi-party collaborative dimensionality reduction processing that protects privacy and can improve processing speed.

[0006] Firstly, a privacy-preserving multi-party joint dimensionality reduction method is provided, wherein each party stores different feature parts of N samples; the method is executed by any one of the parties, including:

[0007] Using a batch processing approach, the local features of M samples in the current batch are read according to a predetermined sample order to construct the original matrix; where M is less than N.

[0008] The first matrix is ​​obtained by multiplying the transpose of the original matrix of the local party by itself. Then, using secure multi-party computation (MPC), the transpose of the original matrix of the local party is multiplied by the original matrix of the other party to obtain the second matrix. The transpose of the original matrix of the other party is multiplied by the original matrix of the local party to obtain the third matrix. Based on the first matrix, the second matrix, the third matrix, and the first matrix of the other party, the merged matrix of the current batch is formed.

[0009] The merged matrix is ​​averaged to obtain the center matrix;

[0010] The covariance matrices of each batch are summed to obtain the covariance matrices of N samples.

[0011] Using the covariance matrix, multi-party joint dimensionality reduction processing is performed on the sample feature data of each batch.

[0012] In one possible implementation, the step of averaging the merged matrix to obtain a central matrix includes:

[0013] Obtain the mean matrix, which is formed based on the mean vectors corresponding to the original matrices of each party. The mean vectors are determined by taking the mean in the dimension corresponding to the feature.

[0014] The center matrix is ​​obtained by performing a difference operation on the merged matrix and the mean matrix.

[0015] Furthermore, one column of the original matrix corresponds to a feature; obtaining the mean matrix includes:

[0016] The first mean submatrix is ​​obtained by multiplying the transpose of the mean row vector of the original matrix with the mean row vector. Then, using Multi-Party Secure Computation (MPC), the transpose of the transpose of the matrix is ​​multiplied with the mean row vector of the other matrix to obtain the second mean submatrix. Finally, the transpose of the other matrix is ​​multiplied with the mean row vector to obtain the third mean submatrix.

[0017] The mean matrix is ​​formed by concatenating the first mean submatrix, the second mean submatrix, the third mean submatrix, and the first mean submatrix of the other parties.

[0018] Further, the multiple parties are two parties; the process of concatenating the first mean submatrix, the second mean submatrix, the third mean submatrix, and the first mean submatrix of the other parties to form the mean matrix includes:

[0019] The mean matrix to be formed is divided into square matrices of 2*2 blocks;

[0020] The first mean submatrix of this side is used to fill the block in the first row and first column of the square matrix, the second mean submatrix of the square matrix is ​​used to fill the block in the first row and second column of the square matrix, the third mean submatrix of the square matrix is ​​used to fill the block in the second row and first column of the square matrix, and the first mean submatrix of the other side is used to fill the block in the second row and second column of the square matrix. The filled square matrix is ​​then used as the mean matrix.

[0021] In one possible implementation, the sample is a user, and the local features include user attributes as privacy data; in the local original matrix, one column corresponds to one user attribute, and one row corresponds to one user.

[0022] In one possible implementation, the multiple parties are two parties; the process of concatenating the first matrix, the second matrix, the third matrix, and the first matrix of the other parties to form the merge matrix for the current batch includes:

[0023] The merge matrix to be formed is divided into a square matrix of 2*2 blocks;

[0024] The first matrix of this party is used to fill the block in the first row and first column of the square matrix, the second matrix is ​​used to fill the block in the first row and second column of the square matrix, the third matrix is ​​used to fill the block in the second row and first column of the square matrix, and the first matrix of the other party is used to fill the block in the second row and second column of the square matrix. The filled square matrix is ​​then used as the merged matrix.

[0025] Secondly, a privacy-preserving multi-party joint dimensionality reduction processing apparatus is provided, wherein each of the parties stores different feature parts of N samples; the apparatus is disposed at any of the parties and includes:

[0026] The construction unit is used to read the local features of M samples in the current batch in a predetermined sample order using a batch processing method to construct the original matrix of the local sample; where M is less than N;

[0027] The merging processing unit is used to locally calculate the first matrix obtained by multiplying the transpose of the original matrix constructed by the construction unit with itself, and using Multi-Party Secure Computation (MPC) to multiply the transpose of the original matrix by the original matrix of the other party to obtain the second matrix, and multiply the transpose of the original matrix of the other party by the original matrix of the unit to obtain the third matrix; based on the first matrix, the second matrix, the third matrix and the first matrix of the other party, the merging matrix of the current batch is formed.

[0028] The mean processing unit is used to perform mean processing on the merged matrix obtained by the merge processing unit to obtain the center matrix;

[0029] The accumulation unit is used to accumulate the center matrix obtained by the mean processing unit of each batch to obtain the covariance matrix corresponding to N samples.

[0030] The dimensionality reduction processing unit is used to perform multi-party joint dimensionality reduction processing on the sample feature data of each batch using the covariance matrix obtained by the accumulation unit.

[0031] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of the first aspect.

[0032] Fourthly, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method of the first aspect.

[0033] The method and apparatus provided in the embodiments of this specification allow each party to first use a batch processing approach, reading the local features of M samples in the current batch according to a predetermined sample order, and constructing its own original matrix; where M is less than N. Then, based on its own original matrix, a merged matrix for the current batch is formed through local computation and multi-party secure computation with other parties. Next, the merged matrix is ​​averaged to obtain a central matrix. Then, the central matrices of each batch are summed to obtain the covariance matrix corresponding to N samples. Finally, the covariance matrix is ​​used to perform multi-party joint dimensionality reduction processing on the sample feature data of each batch. As can be seen from the above, the embodiments of this specification differ from the usual method of calculating the covariance matrix. It does not require obtaining the local features of all N samples to calculate the mean of each dimension feature. Instead, it performs a series of processes on the data of each batch to obtain a central matrix, and then sums the central matrices of each batch to obtain the covariance matrix corresponding to N samples. Therefore, the dataset consisting of N samples only needs to be traversed once, instead of twice, thereby improving processing speed. Attached Figure Description

[0034] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification;

[0036] Figure 2 A schematic diagram illustrating the execution process of a principal component analysis (PCA) method according to one embodiment is shown.

[0037] Figure 3 A flowchart illustrating a privacy-preserving multi-party collaborative dimensionality reduction method according to one embodiment is shown.

[0038] Figure 4 A schematic block diagram of an apparatus for privacy-preserving multi-party collaborative dimensionality reduction processing according to one embodiment is shown. Detailed Implementation

[0039] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0040] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification. This implementation scenario involves multi-party collaborative dimensionality reduction processing to protect privacy. For example... Figure 1 As shown, in a shared learning scenario, training data is provided by multiple holders 1, 2, ..., k, each possessing a portion of the training data. The training data is vertically distributed, also known as a longitudinal distribution; for example, each holder possesses data on different attribute items of the same batch of samples. To ensure the security of privacy data, each holder needs to keep its privacy data locally, without outputting plaintext data or performing plaintext aggregation.

[0041] In one embodiment of this specification, the aforementioned multiple holders jointly employ Principal Component Analysis (PCA) to reduce the dimensionality of the training data. As those skilled in the art know, the core step of the PCA method is to form a covariance matrix based on the data matrix to be reduced in dimensionality, and then solve for the eigenvalues ​​and eigenvectors of this covariance matrix. In the embodiments of this specification, to improve processing speed, a batch processing approach is adopted, supporting the loading of data from large datasets in batches. By calculating the centrality matrix of each batch and then accumulating the centrality matrices of each batch, the covariance matrix corresponding to the large dataset is obtained. To ensure the privacy and security of each holder's data, local computation and secure multi-party computation (MPC) are used when calculating the centrality matrix of each batch. In this way, each holder can reduce the dimensionality of their local data based on the covariance matrix and ultimately form the overall dimensionality-reduced data.

[0042] Principal Component Analysis (PCA) is used to transform data from its original coordinate system to a new one derived from the data itself. The first new coordinate axis is chosen along the direction of maximum variance in the original data. The second new coordinate axis intersects the first and also follows the direction of maximum variance. This process is repeated a number of times, equal to the number of features in the original data. Principal components are extracted by selecting the first few dimensions that contain the most information, thus performing dimensionality reduction or noise reduction.

[0043] To describe the above process more clearly, let's first combine... Figure 2 This describes the execution process of the principal component analysis (PCA) method.

[0044] exist Figure 2 In this context, assume there are D-dimensional feature data for N samples, where each feature data corresponds to the attribute value of one of the sample's attributes. Suppose we want to reduce the dimensionality of the D-dimensional features to d' dimensions, where d'... <D。

[0045] First, in step 201, an N*D dimensional original matrix Y can be formed based on the D-dimensional feature data of the N samples mentioned above. For example, a row can represent a sample, and a column can represent an attribute, thus forming an N-row, D-column original matrix Y.

[0046] Next, in step 202, the original matrix is ​​subjected to zero-mean normalization, or centering, to obtain the central matrix X. The goal of centering is to ensure that, for any attribute in the D dimensions, the mean of the attribute values ​​of all N samples for that attribute is 0. Operationally, for attribute j, the mean of the attribute values ​​of all N samples for attribute j can be calculated first, and then the element corresponding to attribute j in the original matrix can be subtracted from this mean. For example, if a column represents an attribute, for each column of the original matrix, the mean of that column can be calculated, and then each element in that column can be subtracted from the corresponding mean to obtain the zero-mean central matrix X.

[0047] Then, in step 203, the covariance matrix A is calculated, specifically, A = X. T X, where X T Let X be the transpose of matrix X.

[0048] Next, in step 204, the eigenvalues ​​λ and eigenvectors v of the covariance matrix A are solved. Mathematically, the eigenvalues ​​λ and eigenvectors v of the covariance matrix A satisfy:

[0049] Av=λv.

[0050] The covariance matrix is ​​a symmetric matrix with multiple eigenvalues ​​and corresponding eigenvectors, and these eigenvectors are orthogonal to each other. The eigenvalues ​​and eigenvectors of the covariance matrix can be obtained using various algorithms. These eigenvectors can then form the eigenmatrix.

[0051] Based on the multiple eigenvalues ​​and eigenvectors obtained above, the dimension reduction transformation matrix T is determined in step 204.

[0052] It's important to understand that, physically, an eigenvector represents a projection direction in the original D-dimensional space. Multiple eigenvectors being orthogonal to each other implies that these projection directions are also orthogonal. The essence of PCA feature reduction is to find d' mutually orthogonal projection directions in the original D-dimensional space as d' coordinate directions, and project the original sample points onto the d'-dimensional mapping space formed by these d' coordinate directions, maximizing the variance of the projected sample points. The variance after projection in each direction can be represented by eigenvalues.

[0053] Based on this idea, for the multiple eigenvalues ​​obtained, they can be sorted in descending order, and the first d' eigenvalues ​​can be taken as target eigenvalues. This allows us to determine the d' eigenvectors corresponding to these d' target eigenvalues. These d' eigenvectors correspond to the d' coordinate directions selected for dimensionality reduction. These d' eigenvectors constitute the dimensionality reduction transformation matrix T.

[0054] Therefore, in step 205, by applying a dimension reduction transformation matrix T to the original matrix Y, the dimension-reduced matrix Y′ can be obtained, which has N*d' dimensions. Thus, transforming the original matrix Y into matrix Y′ is equivalent to reducing the original D-dimensional features to d' dimensions.

[0055] The above describes the process of reducing the dimensionality of the original N*D matrix in the PCA method by decomposing the eigenvalues ​​of the covariance matrix. However, in Figure 1 In the shared learning scenario shown, each data holder possesses a portion of the sample data, and plaintext aggregation is not possible. Therefore, it is impossible to directly form a full matrix representing all sample data, making it difficult to directly obtain the covariance matrix and solve for it. Furthermore, since step 202 requires zero-mean normalization of the original matrix before calculating the covariance matrix in step 203, it is equivalent to traversing the dataset twice, which is slow for large datasets.

[0056] Therefore, in the embodiments of this specification, a multi-party joint dimensionality reduction method combining local computation and secure multi-party computation (MPC) is used to protect privacy data. Batch processing is employed, and the dataset only needs to be traversed once, thus improving processing speed.

[0057] For example, given a sample set X = (x1, x2, ..., x... N The sample set is shared by two parties, referred to as L and R. The two parties possess the sample set in a vertical manner, meaning they have the same number of samples but different features. L constructs its original matrix X based on the local features of its samples. L R constructs its original matrix X based on the local features of the samples it possesses. R .

[0058] In the typical processing method, L-squared and R-squared first need to perform zero-mean normalization on their respective original matrices. This process requires traversing their own data once. The central matrix obtained by L-squared is denoted as X. L The central matrix obtained by R squared is denoted as X. R Then, based on the central matrix privacy calculation, the covariance matrix X is calculated. T X = (X L ′,X R ′) T (XL ′,X R This process also requires iterating through the data on this side once, where (X) L ′) T X L 'Calculated locally in L, (X R ′) T X R 'Calculated locally in R, for (X L ′) T X R ′ and (X) R ′) T X L This paper utilizes the multiplication triplet protocol for privacy-preserving computation. Understandably, privacy-preserving computation requires protecting private data from leakage; that is, R cannot obtain X. L ′, L cannot obtain X R The multiplication triplet protocol is an existing multi-party secure computation protocol.

[0059] The embodiments in this specification employ a batch processing method, eliminating the need to traverse the dataset twice. L and R read data in batches, with the i-th batch read by L denoted as X. L i Let X be the i-th batch of data read by R. R i It is understandable that X L i and X R i This corresponds to different features of the same batch of samples. The covariance matrix is ​​obtained by accumulating the calculation results for each batch and then dividing the cumulative result by the sample size N. The above cumulative result can be expressed as sum((X... L i ,X R i ) T (X L i ,X R i ))-sum(E(X L i ,X R i ) T E(X L i ,X R i This process only requires traversing the data on this side once. It's understandable that E(X) L i ,X R i ), representing the matrix (X)L i ,X R i Take the mean of each column of X, and obtain a row vector composed of the mean values ​​of all columns, E(X). L i ,X R i ) T This represents transposing the above row vector to obtain the column vector. Where (X... L i ) T X L i and E(X) L i ) T E(X L i ) Calculated locally in L, (X R i ) T X R i and E(X) R i ) T E(X R i ) Calculated locally in R, for (X L i ) T X R i 、(X R i ) T X L i E(X) L i ) T E(X R i ) and E(X R i ) T E(X L i Privacy computation is performed using the multiplication triplet protocol.

[0060] The following describes the specific implementation steps of joint dimensionality reduction for the case of vertical data distribution.

[0061] Figure 3 This diagram illustrates a privacy-preserving multi-party collaborative dimensionality reduction method according to one embodiment, which can be based on... Figure 1 In the implementation scenario shown, each of the multiple parties stores different feature parts of N samples; the method is executed by any one of the multiple parties, which can be understood as... Figure 1 Multiple holders in the agreement. For example... Figure 3 As shown, the privacy-preserving multi-party joint dimensionality reduction method in this embodiment includes the following steps: Step 31, using batch processing, read the local features of M samples in the current batch according to a predetermined sample order to construct the original matrix of the local party; where M is less than N; Step 32, locally calculate the first matrix obtained by multiplying the transpose of the original matrix of the local party by itself, and use multi-party secure computation (MPC) to multiply the transpose of the original matrix of the local party by the original matrix of the other party to obtain the second matrix, and multiply the transpose of the original matrix of the other party by the original matrix of the local party to obtain the third matrix; based on the first matrix, the second matrix, the third matrix and the first matrix of the other party, concatenate to form the merged matrix of the current batch; Step 33, perform mean processing on the merged matrix to obtain the center matrix; Step 34, perform cumulative operation on the center matrices of each batch to obtain the covariance matrix corresponding to N samples; Step 35, use the covariance matrix to perform multi-party joint dimensionality reduction processing on the sample feature data of each batch. The specific execution method of each of the above steps is described below.

[0062] First, in step 31, a batch processing method is used to read the local features of M samples in the current batch according to a predetermined sample order, and construct the original matrix of this party; where M is less than N. It should be understood that since each holder has different feature data of the same batch of N samples, in order to facilitate subsequent processing, each holder needs to align the samples in the dimension, that is, to unify the order of the samples and arrange the feature data according to the predetermined sample order.

[0063] In one example, the sample is a user, and the local features include user attributes as privacy data; in the local original matrix, one column corresponds to one user attribute, and one row corresponds to one user.

[0064] In one embodiment, when forming the original matrix, each row represents a sample and each column represents a feature. Thus, the k-th holder forms M*D. k The original matrix of dimension 1. The following descriptions are all based on this matrix arrangement.

[0065] It is understandable that each holder forms a similar original matrix. If the original matrices of each holder are concatenated horizontally, the original full matrix Y of the current batch can be formed:

[0066] Y = (Y1, Y2, ..., Y) k ).

[0067] Where Y1 represents the original matrix of the first holder, Y2 represents the original matrix of the second holder, ..., Y k The original matrix representing the k-th holder.

[0068] The original full matrix is ​​an M*D dimensional matrix, where each column represents a feature, with a total of D columns; each row represents a sample, with a total of M rows, where D is the total number of feature terms:

[0069]

[0070] Furthermore, the rows have been aligned according to the sample order.

[0071] However, it should be understood that since the holders do not directly aggregate the original data in plaintext, the original full matrix is ​​only a matrix assumed for the convenience of description.

[0072] Then, in step 32, the transpose of the original matrix is ​​locally calculated and multiplied by itself to obtain a first matrix. Using Multi-Party Secure Computation (MPC), the transpose of the original matrix is ​​multiplied by the original matrix of the other party to obtain a second matrix. The transpose of the original matrix of the other party is multiplied by the original matrix of the original party to obtain a third matrix. Based on the first matrix, the second matrix, the third matrix, and the first matrix of the other party, the merged matrix for the current batch is formed. It can be understood that the first matrix of the other party is obtained by the other party through local calculation and can be sent to the original party, i.e., the executor of this method. Furthermore, the second and third matrices obtained using MPC are initially in fragmented form, and can be obtained by merging the fragments.

[0073] In one example, the multiple parties are two parties; the process of concatenating the first matrix, the second matrix, the third matrix, and the first matrix of the other parties to form the merge matrix for the current batch includes:

[0074] The merge matrix to be formed is divided into a square matrix of 2*2 blocks;

[0075] The first matrix of this party is used to fill the block in the first row and first column of the square matrix, the second matrix is ​​used to fill the block in the first row and second column of the square matrix, the third matrix is ​​used to fill the block in the second row and first column of the square matrix, and the first matrix of the other party is used to fill the block in the second row and second column of the square matrix. The filled square matrix is ​​then used as the merged matrix.

[0076] For example, let's say the two sides are L and R, with our side being L. The i-th batch of data read by L constructs the original matrix X. L i The original matrix X is constructed from the i-th batch of data read by R. R i The first matrix of L is (XL i ) T X L i The first matrix of R is (X R i ) T X R i The second matrix is ​​(X L i ) T X R i The third matrix is ​​(X R i ) T X L i The merged matrix is:

[0077]

[0078] Next, in step 33, the merged matrix is ​​averaged to obtain the center matrix. It is understood that the averaging process targets the merged matrix corresponding to each batch of data, and its processing method differs from the usual zero-mean averaging.

[0079] In one example, the step of averaging the merged matrix to obtain a central matrix includes:

[0080] Obtain the mean matrix, which is formed based on the mean vectors corresponding to the original matrices of each party. The mean vectors are determined by taking the mean in the dimension corresponding to the feature.

[0081] The center matrix is ​​obtained by performing a difference operation on the merged matrix and the mean matrix.

[0082] For example, let's say the two sides are L and R, with our side being L. The i-th batch of data read by L constructs the original matrix X. L i The original matrix X is constructed from the i-th batch of data read by R. R i The mean vector of L squared is represented as E(X). L i The mean vector of R-squared is represented as E(X). R i ).

[0083] Furthermore, one column of the original matrix corresponds to a feature; obtaining the mean matrix includes:

[0084] The first mean submatrix is ​​obtained by multiplying the transpose of the mean row vector of the original matrix with the mean row vector. Then, using Multi-Party Secure Computation (MPC), the transpose of the transpose of the matrix is ​​multiplied with the mean row vector of the other matrix to obtain the second mean submatrix. Finally, the transpose of the other matrix is ​​multiplied with the mean row vector to obtain the third mean submatrix.

[0085] The mean matrix is ​​formed by concatenating the first mean submatrix, the second mean submatrix, the third mean submatrix, and the first mean submatrix of the other parties.

[0086] For example, let's say the two sides are L and R, with our side being L. The i-th batch of data read by L constructs the original matrix X. L i The original matrix X is constructed from the i-th batch of data read by R. R i The mean row vector of L squared is represented as E(X). L i The mean row vector of R-squared is represented as E(X). R i The first mean submatrix of L squared is E(X). L i ) T E(X L i The first mean submatrix of R squared is E(X). R i ) T E(X R i The second mean submatrix is ​​E(X). L i ) T E(X R i The third mean submatrix is ​​E(X). R i ) T E(X L i ).

[0087] Further, the multiple parties are two parties; the process of concatenating the first mean submatrix, the second mean submatrix, the third mean submatrix, and the first mean submatrix of the other parties to form the mean matrix includes:

[0088] The mean matrix to be formed is divided into square matrices of 2*2 blocks;

[0089] The first mean submatrix of this side is used to fill the block in the first row and first column of the square matrix, the second mean submatrix of the square matrix is ​​used to fill the block in the first row and second column of the square matrix, the third mean submatrix of the square matrix is ​​used to fill the block in the second row and first column of the square matrix, and the first mean submatrix of the other side is used to fill the block in the second row and second column of the square matrix. The filled square matrix is ​​then used as the mean matrix.

[0090] For example, let L be the two sides and R be the two sides, and let L be the side. Let the first mean submatrix of L be E(X). L i ) T E(X L i The first mean submatrix of R squared is E(X). R i ) T E(X R i The second mean submatrix is ​​E(X). L i ) T E(X R i The third mean submatrix is ​​E(X). R i ) T E(X L i The mean matrix is:

[0091]

[0092] The central matrix corresponding to the i-th batch of data is:

[0093]

[0094] In step 34, the center matrices of each batch are summed to obtain the covariance matrices corresponding to the N samples. It can be understood that the summation result is divided by the number of samples N to obtain the covariance matrices corresponding to the N samples.

[0095] The above cumulative result can be expressed as sum((X) L i ,X R i ) T (X L i ,X R i ))-sum(E(X L i ,X R i ) T E(X Li ,X R i When there are two parties involved, the above cumulative result can also be expressed as:

[0096]

[0097] Finally, in step 35, the covariance matrix is ​​used to perform multi-party joint dimensionality reduction processing on the sample feature data of each batch. It is understood that the covariance matrix can be obtained by any of the parties. In this embodiment, the privacy data to be protected is the local features of the samples held by each party; that is, the original matrix of any one party cannot be obtained by the other parties, but the covariance matrix can be provided to all parties.

[0098] The embodiments in this specification can effectively support large datasets through batch processing mode, and only require traversing the dataset once. The test on 8430 sample data points took only 0.57 seconds.

[0099] The method provided in the embodiments of this specification involves each party first using a batch processing approach. Following a predetermined sample order, it reads the local features of M samples in the current batch to construct its own original matrix, where M is less than N. Then, based on its own original matrix, it performs local computation and multi-party secure computation with other parties to form a merged matrix for the current batch. Next, it performs mean processing on the merged matrix to obtain a central matrix. Then, it performs an accumulation operation on the central matrices of each batch to obtain the covariance matrix corresponding to N samples. Finally, it uses the covariance matrix to perform multi-party joint dimensionality reduction processing on the sample feature data of each batch. As can be seen, the embodiments of this specification differ from the usual method of calculating the covariance matrix. It does not require obtaining the local features of all N samples to calculate the mean of each dimension feature. Instead, it performs a series of processes on the data of each batch to obtain a central matrix, and then performs an accumulation operation on the central matrices of each batch to obtain the covariance matrix corresponding to N samples. Therefore, it only needs to traverse the dataset consisting of N samples once, instead of twice, thereby improving processing speed.

[0100] According to another embodiment, a privacy-preserving multi-party joint dimensionality reduction processing apparatus is also provided, wherein each of the parties stores different feature portions of N samples; the apparatus is disposed at any of the parties, and the apparatus is used to execute this specification. Figure 3 The method provided in the illustrated embodiment. Figure 4 A schematic block diagram of an apparatus for privacy-preserving multi-party collaborative dimensionality reduction processing according to one embodiment is shown. Figure 4 As shown, the device 400 includes:

[0101] Construction unit 41 is used to read the local features of M samples in the current batch in a predetermined sample order using a batch processing method to construct the original matrix of the local side; where M is less than N;

[0102] The merging processing unit 42 is used to locally calculate the first matrix obtained by multiplying the transpose of the original matrix constructed by the construction unit 41 by itself, and using Multi-Party Secure Computation (MPC) to multiply the transpose of the original matrix by the original matrix of the other party to obtain the second matrix, and multiply the transpose of the original matrix of the other party by the original matrix of the own party to obtain the third matrix; based on the first matrix, the second matrix, the third matrix and the first matrix of the other party, the merging matrix of the current batch is formed by concatenating them.

[0103] The mean processing unit 43 is used to perform mean processing on the merged matrix obtained by the merge processing unit 42 to obtain the center matrix;

[0104] The accumulation unit 44 is used to accumulate the center matrix obtained by the mean processing unit 43 for each batch to obtain the covariance matrix corresponding to N samples.

[0105] The dimensionality reduction processing unit 45 is used to perform multi-party joint dimensionality reduction processing on the sample feature data of each batch using the covariance matrix obtained by the accumulation unit 44.

[0106] Optionally, as an embodiment, the mean processing unit 43 includes:

[0107] Obtain sub-units for obtaining mean matrices, which are formed based on the mean vectors corresponding to the original matrices of each party, and the mean vectors are determined by averaging in the dimension corresponding to the feature.

[0108] The operation subunit is used to perform a difference operation on the merged matrix and the mean matrix obtained by the acquisition subunit to obtain the center matrix.

[0109] Furthermore, one column of the original matrix corresponds to a feature; the acquisition sub-unit includes:

[0110] The calculation module is used to locally calculate the product of the transpose of the mean row vector of the original matrix and the mean row vector to obtain a first mean submatrix. Then, using Multi-Party Secure Computation (MPC), the transpose of the transpose of the matrix is ​​multiplied by the mean row vector of the other party to obtain a second mean submatrix. Finally, the transpose of the other party is multiplied by the mean row vector to obtain a third mean submatrix.

[0111] The splicing module is used to splice the first mean submatrix, the second mean submatrix, the third mean submatrix, and the first mean submatrix of the other parties obtained by the calculation module to form the mean matrix.

[0112] Further, the multiple parties are two parties; the splicing module is specifically used to divide the mean matrix to be formed into a square matrix formed by 2*2 blocks; fill the block at the first row and first column position of the square matrix with the first mean submatrix of its own party, fill the block at the first row and second column position of the square matrix with the second mean submatrix, fill the block at the second row and first column position of the square matrix with the third mean submatrix, fill the block at the second row and second column position of the square matrix with the first mean submatrix of the other party, and use the filled square matrix as the mean matrix.

[0113] Optionally, as an embodiment, the sample is a user, and the local features include user attributes as privacy data; in the local original matrix, one column corresponds to one user attribute, and one row corresponds to one user.

[0114] Optionally, as an embodiment, the multiple parties are two parties; the merging processing unit 42 is specifically used to divide the merging matrix to be formed into a square matrix formed by 2*2 blocks; fill the block at the first row and first column position of the square matrix with the first matrix of its own party, fill the block at the first row and second column position of the square matrix with the second matrix, fill the block at the second row and first column position of the square matrix with the third matrix, fill the block at the second row and second column position of the square matrix with the first matrix of the other party, and use the filled square matrix as the merging matrix.

[0115] Using the apparatus provided in the embodiments of this specification, each of the multiple parties first uses the construction unit 41 to read the local features of M samples in the current batch in a batch processing manner according to a predetermined sample order, and constructs its own original matrix; where M is less than N; then the merging processing unit 42 uses its own original matrix to form a merged matrix for the current batch through local calculation and multi-party secure calculation with other parties; next, the mean processing unit 43 performs mean processing on the merged matrix to obtain a center matrix; then the accumulation unit 44 performs accumulation operation on the center matrices of each batch to obtain the covariance matrix corresponding to N samples; finally, the dimensionality reduction processing unit 45 uses the covariance matrix to perform multi-party joint dimensionality reduction processing on the sample feature data of each batch. As can be seen from the above, the embodiments in this specification differ from the usual method of calculating the covariance matrix. Instead of obtaining the local features of all N samples to calculate the mean of each dimension feature, a series of processes are performed on each batch of data to obtain the center matrix. Then, the center matrices of each batch are summed to obtain the covariance matrix corresponding to the N samples. Thus, the dataset consisting of N samples only needs to be traversed once, instead of twice, thereby improving the processing speed.

[0116] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a combination Figure 2 or Figure 3 The method described.

[0117] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements a combination... Figure 2 or Figure 3 The method described.

[0118] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0119] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A privacy-preserving multi-party joint dimensionality reduction method, wherein each of the parties stores different feature parts of N samples, wherein the samples are users; the method is executed by any one of the parties, comprising: Using a batch processing approach, local features of M samples in the current batch are read according to a predetermined sample order to construct the original matrix; wherein M is less than N, the local features include user attributes as privacy data, and one column in the original matrix corresponds to one user attribute, and one row corresponds to one user; The system locally calculates a first matrix by multiplying the transpose of its own original matrix by itself. Then, using Multi-Party Secure Computation (MPC), it multiplies the transpose of the original matrix by the original matrices of the other parties to obtain a second matrix. Finally, it multiplies the transpose of the original matrices of the other parties by the original matrix to obtain a third matrix. Based on the first matrix, the second matrix, the third matrix, and the first matrix from the other parties, the system concatenates them to form the merged matrix for the current batch. The first matrix from the other parties is calculated locally by the other parties and then sent to the system by the other parties. The merged matrix is ​​averaged to obtain the center matrix; The covariance matrices of each batch are summed to obtain the covariance matrices of N samples. Using the covariance matrix, multi-party joint dimensionality reduction processing is performed on the sample feature data of each batch.

2. The method as described in claim 1, wherein, The step of averaging the merged matrix to obtain the central matrix includes: Obtain the mean matrix, which is formed based on the mean vectors corresponding to the original matrices of each party. The mean vectors are determined by taking the mean in the dimension corresponding to the feature. The center matrix is ​​obtained by performing a difference operation on the merged matrix and the mean matrix.

3. The method as described in claim 2, wherein, Each column of the original matrix corresponds to a feature; The process of obtaining the mean matrix includes: The first mean submatrix is ​​obtained by multiplying the transpose of the mean row vector of the original matrix with the mean row vector. Then, using Multi-Party Secure Computation (MPC), the transpose of the transpose of the matrix is ​​multiplied with the mean row vector of the other matrix to obtain the second mean submatrix. Finally, the transpose of the other matrix is ​​multiplied with the mean row vector to obtain the third mean submatrix. The mean matrix is ​​formed by concatenating the first mean submatrix, the second mean submatrix, the third mean submatrix, and the first mean submatrix of the other parties.

4. The method according to claim 3, wherein, The multiple parties refer to two parties; the process of concatenating the first mean submatrix, the second mean submatrix, the third mean submatrix, and the first mean submatrix of the other parties to form the mean matrix includes: The mean matrix to be formed is divided into square matrices of 2*2 blocks; The first mean submatrix of this side is used to fill the block in the first row and first column of the square matrix, the second mean submatrix of the square matrix is ​​used to fill the block in the first row and second column of the square matrix, the third mean submatrix of the square matrix is ​​used to fill the block in the second row and first column of the square matrix, and the first mean submatrix of the other side is used to fill the block in the second row and second column of the square matrix. The filled square matrix is ​​then used as the mean matrix.

5. The method according to claim 1, wherein, The "multiple parties" refers to two parties; the process of concatenating the first matrix, the second matrix, the third matrix, and the first matrix of the other parties to form the merge matrix for the current batch includes: The merge matrix to be formed is divided into a square matrix of 2*2 blocks; The first matrix of this party is used to fill the block in the first row and first column of the square matrix, the second matrix is ​​used to fill the block in the first row and second column of the square matrix, the third matrix is ​​used to fill the block in the second row and first column of the square matrix, and the first matrix of the other party is used to fill the block in the second row and second column of the square matrix. The filled square matrix is ​​then used as the merged matrix.

6. A device for privacy-preserving multi-party joint dimensionality reduction processing, wherein each of the parties stores different feature parts of N samples, wherein the samples are users; The device is disposed in any one of the parties, including: The construction unit is used to read the local features of M samples in the current batch in a predetermined sample order using a batch processing method to construct the local original matrix; wherein M is less than N, the local features include user attributes as privacy data, and one column in the local original matrix corresponds to one user attribute, and one row corresponds to one user; The merging processing unit is used to locally calculate a first matrix obtained by multiplying the transpose of the original matrix constructed by the building unit with itself, and using Multi-Party Secure Computation (MPC), multiplying the transpose of the original matrix of the local party with the original matrix of the other party to obtain a second matrix, and multiplying the transpose of the original matrix of the other party with the original matrix of the local party to obtain a third matrix; based on the first matrix, the second matrix, the third matrix, and the first matrix of the other party, the merging matrix of the current batch is formed by concatenating them; the first matrix of the other party is obtained by the other party through local calculation and sent to the local party by the other party; The mean processing unit is used to perform mean processing on the merged matrix obtained by the merge processing unit to obtain the center matrix; The accumulation unit is used to accumulate the center matrix obtained by the mean processing unit of each batch to obtain the covariance matrix corresponding to N samples. The dimensionality reduction processing unit is used to perform multi-party joint dimensionality reduction processing on the sample feature data of each batch using the covariance matrix obtained by the accumulation unit.

7. The apparatus of claim 6, wherein, The mean processing unit includes: Obtain sub-units for obtaining mean matrices, which are formed based on the mean vectors corresponding to the original matrices of each party, and the mean vectors are determined by averaging in the dimension corresponding to the feature. The operation subunit is used to perform a difference operation on the merged matrix and the mean matrix obtained by the acquisition subunit to obtain the center matrix.

8. The apparatus of claim 7, wherein, One column of the original matrix corresponds to a feature; the acquisition sub-unit includes: The calculation module is used to locally calculate the product of the transpose of the mean row vector of the original matrix and the mean row vector to obtain a first mean submatrix. Then, using Multi-Party Secure Computation (MPC), the transpose of the transpose of the matrix is ​​multiplied by the mean row vector of the other party to obtain a second mean submatrix. Finally, the transpose of the other party is multiplied by the mean row vector to obtain a third mean submatrix. The splicing module is used to splice the first mean submatrix, the second mean submatrix, the third mean submatrix, and the first mean submatrix of the other parties obtained by the calculation module to form the mean matrix.

9. The apparatus according to claim 8, wherein, The multiple parties refer to two parties; the splicing module is specifically used to divide the mean matrix to be formed into a square matrix formed by 2*2 blocks; fill the block at the first row and first column position of the square matrix with the first mean submatrix of its own party, fill the block at the first row and second column position of the square matrix with the second mean submatrix, fill the block at the second row and first column position of the square matrix with the third mean submatrix, fill the block at the second row and second column position of the square matrix with the first mean submatrix of the other party, and use the filled square matrix as the mean matrix.

10. The apparatus according to claim 6, wherein, The multiple parties refer to two parties; the merging processing unit is specifically used to divide the merging matrix to be formed into a square matrix formed by 2*2 blocks; fill the block at the first row and first column position of the square matrix with the first matrix of its own party, fill the block at the first row and second column position of the square matrix with the second matrix, fill the block at the second row and first column position of the square matrix with the third matrix, fill the block at the second row and second column position of the square matrix with the first matrix of the other party, and use the filled square matrix as the merging matrix.

11. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-5.

12. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-5.

Images