Principal component analysis method and device based on distributed differential privacy protection

Abstract

The invention discloses a principal component analysis method and device based on distributed differential privacy protection, and the method comprises the steps: obtaining user data and the number of users of each distributed server node, adding differential privacy noise to the user data, and generating an attribute mean value based on the noise disturbance data of a global node; performing de-mean processing on the user data according to the attribute mean value to obtain de-mean data of each node, performing data mapping on the de-mean data, dividing a disturbance data interval corresponding to the mapping data, and enabling a disturbance result to be located in a high-functionality interval with a higher probability; calculating a covariance matrix corresponding to the disturbance de-averaging data according to the disturbance de-averaging data in each distributed server node; and finally, converging the covariance matrixes to a central server, and extracting principal components according to the covariance matrixes. According to the method, unified interference is carried out on the data in the distributed scene, user privacy leakage during data aggregation is prevented, the differential privacy noise is reduced, and the effectiveness of the data is improved.

Description

Technical Field

[0001] The present invention relates to the field of information installation technology, and in particular to a principal component analysis method and device based on distributed differential privacy protection. Background Art

[0002] In recent years, with the widespread application of smart devices, medical systems, and financial systems, massive amounts of user data have been continuously collected, resulting in an exponential growth in data dimensions, which has significantly increased the complexity of data analysis. Principal component analysis (PCA) projects the original data into a new coordinate system through linear transformation, which can effectively reduce the data dimension while retaining the original information of the data as much as possible, thereby simplifying the difficulty of data analysis. Therefore, PCA has become one of the most commonly used tools for handling complex data analysis tasks. However, PCA relies on the server to aggregate user data from each node. If the process is not properly protected and principal component analysis is performed directly, the analysis results may indirectly leak privacy information, thereby posing serious risks to user privacy.

[0003] Differential privacy is a widely used privacy protection technology. Its basic principle is to add noise or perturb data to protect the user's privacy information. In this way, even if the attacker obtains some results or statistical data, it is impossible to accurately restore or infer the individual's private information, thereby reducing the risk of privacy leakage. Since PCA relies on a large amount of user data collected by the server, the use of differential privacy technology can ensure that the server publishes perturbed data, thereby preventing user privacy leakage.

[0004] Jiang et al. proposed a differential privacy protection algorithm PPDP based on the Laplace mechanism. This method adds a symmetric Laplace noise matrix to the covariance matrix and performs eigenvalue decomposition on the perturbation result to obtain the principal components. However, the symmetric Laplace noise matrix still destroys the semi-positive definiteness of the covariance matrix. Jiang Wuxuan proposed a principal component analysis method based on the Wishart mechanism. This method adds Wishart noise to the covariance matrix and uses eigenvalue decomposition to extract the principal components from the perturbed covariance matrix, ensuring that the perturbed covariance matrix remains semi-positive definite. Zhang Jiuxing proposed another solution, the principal component differential privacy algorithm CMPDP based on the covariance matrix. This method adds Laplace noise to the main diagonal of the covariance matrix and performs eigenvalue decomposition on the perturbation result to obtain the principal components, aiming to ensure the semi-positive definiteness of the covariance matrix while reducing the noise intensity. For distributed scenarios, Zhu Xiao proposed a horizontal federated PCA differential privacy protection algorithm HFedPCA. The method first adds symmetric Gaussian noise to the local covariance matrix at each distributed server node and publishes it to the central server. Secondly, the central server aggregates the perturbation covariance matrix of all distributed nodes. Finally, the method performs eigenvalue decomposition on the aggregated covariance matrix to obtain the principal components.

[0005] The above methods all assume that the central server is trustworthy. However, in the scenario of untrustworthy servers, Balcan et al. proposed a distributed differential privacy QR matrix decomposition method for principal component analysis. The method first generates a random Gaussian matrix by the server and performs QR decomposition to obtain the initial eigenvalues ​​and eigenvectors, and then distributes these initial eigenvectors to each node. Each node multiplies its locally stored covariance matrix with the received eigenvector, adds Gaussian noise to the result, and then sends the calculation result back to the server. The server aggregates the data of all nodes and performs QR decomposition again to obtain the eigenvalues ​​and eigenvectors of the next round. Finally, the server publishes the current eigenvalues ​​and eigenvectors to each node, and the node repeats the above steps to update until the gap between the eigenvalues ​​is less than the set threshold. However, in order to speed up the convergence speed, this method increases the gap between the eigenvalues ​​by increasing the number of principal components, resulting in the generated principal component dimensions being inconsistent with the required dimensions. To solve the above problem, Wang et al. proposed a local Gaussian mechanism method. LGM first generates a covariance matrix based on the node data and adds Gaussian noise to the result. Secondly, LGM aggregates data from all nodes on the server side and performs feature decomposition to extract the principal components. However, there are two problems with LGM. First, this method only protects privacy in the covariance matrix calculation stage, but the calculation of the covariance matrix depends on the mean-free data, which may lead to privacy leakage of user privacy due to mean calculation. Second, LGM protects user privacy through the Gaussian noise mechanism, but the Gaussian mechanism can only achieve approximate differential privacy protection, resulting in insufficient privacy protection. Summary of the invention

[0006] In view of the above-mentioned deficiencies of the prior art, the present invention provides a principal component analysis method and device based on distributed differential privacy protection, which reduces the impact of differential privacy noise on data utility rows by generating local sensitivity and adding differential privacy noise, as well as mapping processing and perturbation of the covariance matrix, ensuring strict local differential privacy protection in both the mean calculation and covariance matrix calculation stages, while improving the accuracy of principal component extraction.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a principal component analysis method based on distributed differential privacy protection, comprising the following steps:

[0009] S1. Obtain user data and the number of users of each distributed server node, and generate local sensitivity according to data attributes of the user data;

[0010] S2. Based on the local sensitivity, determine the differential privacy noise according to the Laplace mechanism, add the differential privacy noise to each data of the user data, obtain noise disturbance data, and publish it to the central server;

[0011] S3, the central server calculates the attribute mean of the global distributed server nodes according to the noise disturbance data and the number of users, and returns the attribute mean to each distributed server node;

[0012] S4, performing mean removal processing on the user data of each distributed server node according to the attribute mean, to obtain mean removal data of each distributed server node;

[0013] S5, performing data mapping processing on each de-averaged data of each distributed server node to obtain mapping data of each distributed server node;

[0014] S6. Determine a disturbance interval according to the size of the mapped data, and then perform random disturbance to select disturbance data, perform inverse mapping processing on the disturbance data, so as to obtain disturbance de-meaned data of each distributed server node, and calculate a covariance matrix according to the de-meaned data;

[0015] S7. Aggregate the disturbance covariance matrices of the distributed server nodes to the central server to calculate a global covariance matrix, and then perform principal component extraction on the global covariance matrix.

[0016] In the principal component analysis method based on distributed differential privacy protection, as a preferred solution, in step S2, the noise disturbance data is obtained by the following formula:

[0017]

[0018] In the formula, is the noise perturbation data; For each distributed server node X t user data; i is the i-th attribute in any distributed server node, i = 1, 2, ..., m, m is the number of attributes; t is the t-th node in any distributed server node, t = 1, 2, ..., T, T is the number of distributed server nodes; u is the u-th user number of each distributed server node, u = 1, 2, ..., n, n is the number of users of each distributed server node; is the Laplace noise; Δf i t is the local sensitivity; ε 1 Budget for privacy.

[0019] In the principal component analysis method based on distributed differential privacy protection, as a preferred solution, in step S3, the attribute mean of the global distributed server node is calculated by the following formula:

[0020]

[0021] In the formula, μ i is the attribute mean of the global distributed server nodes.

[0022] In the principal component analysis method based on distributed differential privacy protection, as a preferred solution, in step S4, the attribute mean calculated by the central server is subtracted from the user data of each distributed server node, so as to obtain the de-meaned data of each distributed server node, that is:

[0023]

[0024] In the formula, It is the mean value data of each distributed server node.

[0025] In the principal component analysis method based on distributed differential privacy protection, as a preferred solution, step S5 specifically includes:

[0026] S501: De-averaging the data generated by each distributed server node Perform absolute value processing and determine the maximum value corresponding to each attribute based on the averaged data after absolute value processing As a mapping parameter;

[0027] S502: Mapping the de-averaged data of each distributed server node according to the mapping parameters to obtain the mapping data corresponding to each distributed server node, which is calculated in the following way:

[0028]

[0029] In the formula, Mapping data for any distributed server node; is the mean data of each distributed server node, h=1,2,...,m, j=1,2,...,m, t=1,2,...,T.

[0030] In the principal component analysis method based on distributed differential privacy protection, as a preferred solution, step S6 specifically includes:

[0031] S601, initialization parameters And determine the disturbance interval [-C, C], where C is the size of the disturbance interval, p is the probability of selecting a high-functionality interval during the disturbance process, and ε 2 Budget for privacy;

[0032] S602: According to any mapping data in each distributed server node according to The size of the disturbance interval is divided into high-functional intervals and low functional range And randomly select a value in the corresponding interval as the perturbation data

[0033] S603, randomly generate a natural number q between 0 and 1;

[0034] S604: Any mapping data for each distributed server node When q<p is satisfied, the disturbance data is determined to be Located in the high functionality range, otherwise the perturbation data Located in the low functional range;

[0035] S605, performing inverse mapping processing on any of the disturbance data by using the following formula to restore the original range of the data:

[0036]

[0037] In the formula, is the disturbance data after inverse mapping;

[0038] S606: Repeat steps S601 to S605 until the mean-averaged data of all distributed server nodes are disturbed, thereby obtaining the disturbed mean-averaged data of each distributed server node.

[0039] In the principal component analysis method based on distributed differential privacy protection, as a preferred solution, the covariance matrix of each distributed server node is calculated by the following formula:

[0040]

[0041] In the formula, S t is the covariance matrix of each distributed server node; for The transposed matrix of .

[0042] In the principal component analysis method based on distributed differential privacy protection, as a preferred solution, in step S7, the global covariance matrix is ​​calculated by the following formula:

[0043]

[0044] Where S is the global covariance matrix.

[0045] In the principal component analysis method based on distributed differential privacy protection, as a preferred solution, in step S7, principal component extraction is performed on the global covariance matrix, including:

[0046] S701, performing eigenvalue decomposition on the global covariance matrix to obtain the eigenvalues ​​and eigenvectors of the global covariance matrix, namely:

[0047] Sv i =λ i v i ;

[0048] In the formula, λ i is the eigenvalue of the global covariance matrix, v i is the eigenvector of the global covariance matrix, i=1,2,...,m;

[0049] S702: Arrange the eigenvalues ​​in descending order according to their magnitudes into λ 1 >λ 2 >...>λ m , then the corresponding eigenvector is v 1 ,v 2 ,...,v m ;

[0050] S703, extract the eigenvectors corresponding to the first k eigenvalues ​​as the principal components, i.e., V k =[v 1 ,v 2 ,...,v k ].

[0051] In a second aspect, the present invention further provides a principal component analysis device based on distributed local differential privacy protection, which is used to implement the above-mentioned principal component analysis method based on distributed differential privacy protection. The principal component analysis device based on distributed local differential privacy protection includes:

[0052] A mean calculation module, used to obtain user data and the number of users of each distributed server node, and generate local sensitivity according to the data attributes of the user data, determine differential privacy noise based on the local sensitivity according to the Laplace mechanism, add the differential privacy noise to each data of the user data, obtain noise disturbance data, and calculate the attribute mean of the global distributed server node based on the noise disturbance data and the number of users;

[0053] A data segmented disturbance module is used to perform mean removal processing on the user data of each distributed server node according to the attribute mean, obtain mean removal data of each distributed server node, and perform mapping processing on each mean removal data in each distributed server node to produce mapping data of each distributed server node, determine the disturbance interval according to the size of the mapping data, and then perform random disturbance to select disturbance data, perform inverse mapping processing on the disturbance data, obtain disturbance mean removal data of each distributed server node, and then calculate the covariance matrix according to the mean removal data, and publish it to the central server;

[0054] The principal component extraction module is used to aggregate the disturbance covariance matrices of the distributed server nodes to the central server to calculate the global covariance matrix, and then perform principal component extraction on the global covariance matrix.

[0055] Compared with the prior art, the present invention has the following technical effects:

[0056] (1) In the principal component analysis process of the present invention, each distributed server node will first perturb any data before publishing it. Therefore, each distributed server node only publishes the perturbation-processed data. This ensures that even if the data is stored in different nodes, user privacy can still be effectively protected, and reduces the risk of privacy leakage when the central server aggregates data.

[0057] (2) In the mean calculation stage, the present invention adaptively adjusts the sensitivity of the data according to the attribute changes, thereby improving the accuracy of the mean calculation; in the covariance matrix calculation stage, the present invention first performs homogenization mapping on the de-meaned data to prevent different degrees of impact when the same degree of differential privacy noise is added to different attributes of the user's de-meaned data; secondly, the present invention outputs perturbation data from the high functionality interval with a higher probability, thereby maintaining the correlation between data while protecting privacy, and further improving the accuracy of principal component analysis.

[0058] (3) The present invention perturbs the covariance matrix to ensure that the differential privacy protection method can be better deployed in the existing distributed system without changing the principal component analysis calculation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0060] Figure 1 The present invention discloses a flow chart of a principal component analysis method based on distributed differential privacy protection;

[0061] Figure 2It is a flow chart of the principal component analysis method based on distributed differential privacy protection of the present invention;

[0062] Figure 3 This is a structural block diagram of a principal component analysis device based on distributed differential privacy protection in the present invention;

[0063] Figure 4 The relationship between the subspace distance and the privacy budget of the embodiment of the present invention and the existing method in the Covertype and Communities and Crime datasets when the number of principal components is 2;

[0064] Figure 5 The figure shows the relationship between the subspace distance and the number of principal components of the embodiment of the present invention and the existing method in the Covertype and Communities and Crime datasets when the privacy budget is 0.5. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0066] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0067] In a distributed scenario, principal component analysis (PCA) relies on the server to aggregate user data from each node. If the process is not properly protected and the principal component analysis is performed directly, the analysis results may indirectly leak private information, thereby posing a serious risk to user privacy. In the prior art, traditional differential privacy technology protects user privacy by adding noise, but this noise may destroy the structure and correlation of the data and reduce the utility of the data.

[0068] To solve the above problems, the present invention proposes a principal component analysis method based on distributed differential privacy protection. Figure 1 As shown, the method comprises the following steps:

[0069] S1. Obtain user data and the number of users of each distributed server node, and generate local sensitivity according to data attributes of the user data;

[0070] S2. Based on the local sensitivity, determine the differential privacy noise according to the Laplace mechanism, add the differential privacy noise to each data of the user data, obtain noise disturbance data, and publish it to the central server;

[0071] S3, the central server calculates the attribute mean of the global distributed server nodes according to the noise disturbance data and the number of users, and returns the attribute mean to each distributed server node;

[0072] S4, performing mean removal processing on the user data of each distributed server node according to the attribute mean, to obtain mean removal data of each distributed server node;

[0073] S5, performing data mapping processing on each de-averaged data of each distributed server node to obtain mapping data of each distributed server node;

[0074] S6. Determine a disturbance interval according to the size of the mapped data, and then perform random disturbance to select disturbance data, perform inverse mapping processing on the disturbance data, so as to obtain disturbance de-meaned data of each distributed server node, and calculate a covariance matrix according to the de-meaned data;

[0075] S7. Aggregate the disturbance covariance matrices of the distributed server nodes to the central server to calculate a global covariance matrix, and then perform principal component extraction on the global covariance matrix.

[0076] The present invention uniformly interferes with data in distributed scenarios to prevent user privacy leakage when data is aggregated, and on this basis, reduces the size of differential privacy noise, thereby improving the utility of data. The following is a further detailed description of each link in the present invention. Figure 2 shown.

[0077] In step S1, for each distributed server node X t (t=1,2,...,T), the present invention calculates the difference between the maximum and minimum values ​​of the attribute based on the value range of the user data attribute to generate the local sensitivity Δf i t (i=1,2,...,m,t=1,2,...,T).

[0078] In step S2, the noise disturbance data is obtained by the following formula:

[0079]

[0080] In the formula, is the noise perturbation data; For each distributed server node X t user data; i is the i-th attribute in any distributed server node, i = 1, 2, ..., m, m is the number of attributes; t is the t-th node in any distributed server node, t = 1, 2, ..., T, T is the number of distributed server nodes; u is the u-th user number of each distributed server node, u = 1, 2, ..., n, n is the number of users of each distributed server node; is the Laplace noise; Δf i t is the local sensitivity; ε 1 Budget for privacy.

[0081] In step S3, the data disturbed by the Laplace mechanism will be published to the central server. At the same time, the distributed server node will also publish the local user quantity information. The central server aggregates the disturbance data and user quantity information of all distributed server nodes to calculate the attribute mean U of all nodes = [μ 1 ,μ 2 ,...,μ m ], the attribute mean of the global distributed server node is calculated by the following formula:

[0082]

[0083] In the formula, μ i is the attribute mean of the global distributed server nodes.

[0084] In this embodiment, the mean calculation stage is performed as described in steps S1 to S3. The present invention adds Laplace noise to the user data on each distributed server node, thereby ensuring that the user data is privacy protected before being transmitted to the central server, thereby reducing the risk of privacy leakage when the central server aggregates data; at the same time, the present invention adaptively adjusts the sensitivity of the data according to the attribute changes, thereby improving the accuracy of the mean calculation.

[0085] In step S4, the attribute mean value calculated by the central server is subtracted from the user data of each distributed server node, thereby obtaining the averaged data of each distributed server node, that is:

[0086]

[0087] In the formula, It is the mean value data of each distributed server node.

[0088] In step S5, it specifically includes:

[0089] S501: De-averaging the data generated by each distributed server node Perform absolute value processing and determine the maximum value corresponding to each attribute based on the averaged data after absolute value processing As a mapping parameter;

[0090] S502: Mapping the de-averaged data of each distributed server node according to the mapping parameters to obtain the mapping data corresponding to each distributed server node, which is calculated in the following way:

[0091]

[0092] In the formula, Mapping data for any distributed server node; is the mean data of each distributed server node, h=1,2,...,m, j=1,2,...,m, t=1,2,...,T.

[0093] For any distributed server node, the present invention realizes data normalization and privacy protection by processing and mapping the covariance matrix data in absolute value on the distributed server node based on the locally stored de-mean attribute data. By determining the mapping parameters and mapping their values ​​to the interval of -1 to 1, it not only unifies the scales of different attributes and improves the calculation stability, but also provides preparation for the subsequent perturbation steps, ensuring that the utility of the data is maximized while protecting privacy. In addition, the mapped data is easier to aggregate, which simplifies the data processing flow on the central server side and reduces the resources required for storing and transmitting data.

[0094] In step S6, it specifically includes:

[0095] S601, initialization parameters And determine the disturbance interval [-C, C], where C is the size of the disturbance interval, p is the probability of selecting a high-functionality interval during the disturbance process, and ε 2 Budget for privacy;

[0096] S602: According to any mapping data in each distributed server node according to The size of the disturbance interval is divided into high-functional intervals and low functional range And randomly select a value in the corresponding interval as the perturbation data

[0097] S603, randomly generate a natural number q between 0 and 1;

[0098] S604: Any mapping data for each distributed server node When q<p is satisfied, the disturbance data is determined to be Located in the high functionality range, otherwise the perturbation data Located in the low functional range;

[0099] S605, performing inverse mapping processing on any of the disturbance data by using the following formula to restore the original range of the data:

[0100]

[0101] In the formula, is the disturbance data after inverse mapping;

[0102] S606: Repeat steps S601 to S605 until the mean-averaged data of all distributed server nodes are disturbed, thereby obtaining the disturbed mean-averaged data of each distributed server node.

[0103] In this embodiment, the disturbance interval is determined by initializing parameters C and p, and according to the mapping data The size of the data is divided into high-functionality intervals and low-functionality intervals, and the data in the high-functionality interval are selected for perturbation with a higher probability, which enhances the correlation of the data and further improves the accuracy of principal component analysis.

[0104] The covariance matrix is ​​calculated independently on each distributed server node. Specifically, the covariance matrix of each distributed server node is calculated using the following formula:

[0105]

[0106] In the formula, S t is the covariance matrix of each distributed server node; for The transposed matrix of .

[0107] In step S7, the global covariance matrix is ​​calculated by the following formula:

[0108]

[0109] Where S is the global covariance matrix.

[0110] Then, principal component extraction is performed on the global covariance matrix, including:

[0111] S701, performing eigenvalue decomposition on the global covariance matrix to obtain the eigenvalues ​​and eigenvectors of the global covariance matrix, namely:

[0112] Sv i =λi v i ;

[0113] In the formula, λ i is the eigenvalue of the global covariance matrix, v i is the eigenvector of the global covariance matrix, i=1,2,...,m;

[0114] S702: Arrange the eigenvalues ​​in descending order according to their magnitudes into λ 1 >λ 2 >...>λ m , then the corresponding eigenvector is v 1 ,v 2 ,...,v m ;

[0115] S703, extract the eigenvectors corresponding to the first k eigenvalues ​​as the principal components, i.e., V k =[v 1 ,v 2 ,...,v k ].

[0116] As another aspect of the present invention, a principal component analysis device based on distributed local differential privacy protection is provided, which is used to implement the principal component analysis method based on distributed differential privacy protection as described above, wherein: Figure 3 As shown, the principal component analysis device based on distributed local differential privacy protection includes:

[0117] The mean calculation module 100 is used to obtain user data and the number of users of each distributed server node, generate local sensitivity according to the data attributes of the user data, determine differential privacy noise according to the local sensitivity according to the Laplace mechanism, add the differential privacy noise to each data of the user data, obtain noise disturbance data, and calculate the attribute mean of the global distributed server node based on the noise disturbance data and the number of users;

[0118] The data segmented disturbance module 200 is used to perform mean removal processing on the user data of each distributed server node according to the attribute mean, obtain the mean removal data of each distributed server node, and perform mapping processing on each mean removal data in each distributed server node to produce the mapping data of each distributed server node, determine the disturbance interval according to the size of the mapping data, and then perform random disturbance to select the disturbance data, perform inverse mapping processing on the disturbance data, obtain the disturbance mean removal data of each distributed server node, and then calculate the covariance matrix according to the mean removal data, and publish it to the central server;

[0119] The principal component extraction module 300 is used to aggregate the disturbance covariance matrices of the distributed server nodes to the central server to calculate the global covariance matrix, and then perform principal component extraction on the global covariance matrix.

[0120] In order to further verify the principal component analysis method and device based on distributed differential privacy protection provided by the present invention, the present invention is further illustrated by the following embodiments.

[0121] This example is conducted on a server running Windows 10 operating system, equipped with an Intel(R) Core(TM) i7-10700 processor with 8 cores. The example uses Python 3.7 programming language to run the algorithm. In this example, Covertype and Communities and Crime datasets are used to conduct experimental comparisons with LGM and its improved method. Before the experiment, we set ε 1 / ε 2 Set to 0.1, the number of nodes T is set to 100, and the number of users of Covertype and Communities and Crime datasets is set to 10,000 and 1,900 respectively, to ensure that the relationship between the privacy budget and the number of principal components can be fully evaluated under the same experimental parameters. Due to the randomness of data perturbations, this embodiment and the existing method are independently run 50 times and evaluated by subspace distance, which is calculated as follows:

[0122]

[0123] Where V k ' and V k They represent the principal components obtained by the non-privacy protection method and the privacy protection method respectively. The smaller the subspace distance, the higher the accuracy of principal component extraction, and vice versa.

[0124] Tables 1 and 2 show the relationship between the subspace distance and the number of nodes in this embodiment when the privacy budget is 0.5 in the Covertype and Communities and Crime datasets, respectively. Specifically, as shown in Tables 1 and 2, taking the Covertype dataset as an example, when the number of principal components is set to 20, as the number of nodes increases from 20 to 100, the subspace distance decreases from 3.81 to 3.73; in the Communities and Crime dataset, under the same experimental conditions, the subspace distance decreases from 5.3 to 5.07. The reason for this trend is that when the number of users is fixed, the fewer the number of nodes in the distributed server, the more user data each node contains. In this way, each node can obtain more sample data, thereby more accurately calculating the mean of the attribute, which makes the extraction of the principal component more accurate. It is worth noting that the number of users in the Covertype dataset is large, and even if the data is distributed to different server nodes, the amount of data that a single node can obtain is still sufficient. This means that each node can effectively estimate the accurate attribute mean and covariance matrix, so that the calculated principal component is more stable, which leads to little change in the subspace distance.

[0125] Table 1. Relationship between subspace distance and number of nodes in this embodiment when the privacy budget is 0.5 in the Covertype dataset.

[0126]

[0127] Table 2 Relationship between subspace distance and number of nodes in this embodiment when the privacy budget is 0.5 in the Communities and Crime dataset

[0128]

[0129] Figure 4 Shows the Covertype (such as Figure 4 (a)) and Communities and Crime (as Figure 4 (b) shows the relationship between the subspace distance and the privacy budget of this embodiment and the existing method when the number of principal components is 2. In these two datasets, the subspace distance of this embodiment reaches a balance when the privacy budget is 0.5; while the subspace distance of LGM-Lap reaches a balance when the privacy budget is 3.5 and 2.5, respectively, and the subspace distance of LGM reaches a balance when the privacy budget is 4 and 2.5, respectively.

[0130] The accuracy of principal component analysis will gradually converge with the increase of privacy budget. The above results show that this embodiment can converge to a lower subspace distance faster, thereby better balancing the privacy of users and the utility of services.

[0131] Figure 5 Shows the Covertype (such as Figure 5 (a)) and Communities and Crime (as Figure 5 (a) shows the relationship between the subspace distance and the number of principal components of the present embodiment and the existing method when the privacy budget is 0.5. Specifically, in the Covertype and Communities and Crime datasets, the subspace distance of the present embodiment increases from 0.25 to 3.77 and from 0.05 to 5.08 respectively; the subspace distance of LGM-Lap increases from 0.99 to 4.63 and from 1.33 to 5.59; the subspace distance of LGM increases from 1.31 to 4.66 and from 1.29 to 5.64. These results show that the subspace distance of the present embodiment is significantly smaller than that of the existing method. This is because the present embodiment divides the perturbed data into high-functionality intervals and low-functionality intervals according to the user's de-meaned data, and ensures that the perturbation results are in the high-functionality interval with a higher probability, thereby effectively maintaining the correlation between the data.

[0132] In summary, in the principal component analysis process of the present invention, each distributed server node will first perturb any data before publishing it. Therefore, each distributed server node only publishes the perturbation-processed data, which ensures that even if the data is dispersed and stored in different nodes, user privacy can still be effectively protected, and reduces the risk of privacy leakage when the central server collects data. In the mean calculation stage, the present invention adaptively adjusts the sensitivity of the data according to the attribute change, thereby improving the accuracy of the mean calculation; in the covariance matrix calculation stage, the present invention first performs homogenization mapping on the de-meaned data to prevent different degrees of influence when adding the same degree of differential privacy noise to different user attribute data. Secondly, the present invention outputs perturbed data from a high-functionality interval with a higher probability, thereby maintaining the correlation between data under the premise of protecting privacy, and further improving the accuracy of principal component analysis. The present invention perturbs the covariance matrix to ensure that the differential privacy protection method can be better deployed in an existing distributed system without changing the principal component analysis calculation process.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described with reference to the preferred embodiments of the present invention, it should be understood by those skilled in the art that various changes may be made in form and details without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A principal component analysis method based on distributed differential privacy protection, characterized in that: The steps include: S1. Obtain user data and the number of users of each distributed server node, and generate local sensitivity according to data attributes of the user data; S2. Based on the local sensitivity, determine the differential privacy noise according to the Laplace mechanism, add the differential privacy noise to each data of the user data, obtain noise disturbance data, and publish it to the central server; S3, the central server calculates the attribute mean of the global distributed server nodes according to the noise disturbance data and the number of users, and returns the attribute mean to each distributed server node; S4, performing mean removal processing on the user data of each distributed server node according to the attribute mean, to obtain mean removal data of each distributed server node; S5, performing data mapping processing on each de-averaged data of each distributed server node to obtain mapping data of each distributed server node; S6. Determine a disturbance interval according to the size of the mapped data, and then perform random disturbance to select disturbance data, perform inverse mapping processing on the disturbance data, so as to obtain disturbance de-meaned data of each distributed server node, and calculate a covariance matrix according to the de-meaned data; S7. Aggregate the disturbance covariance matrices of the distributed server nodes to the central server to calculate a global covariance matrix, and then perform principal component extraction on the global covariance matrix.

2. The principal component analysis method based on distributed differential privacy protection according to claim 1, characterized in that: In step S2, the noise disturbance data is obtained by the following formula: In the formula, is the noise perturbation data; For each distributed server node X t user data; i is the i-th attribute in any distributed server node, i = 1, 2, ..., m, m is the number of attributes; t is the t-th node in any distributed server node, t = 1, 2, ..., T, T is the number of distributed server nodes; u is the u-th user number of each distributed server node, u = 1, 2, ..., n, n is the number of users of each distributed server node; is the Laplace noise; Δf i t is the local sensitivity; ε1 is the privacy budget.

3. The principal component analysis method based on distributed differential privacy protection according to claim 2, characterized in that: In step S3, the attribute mean of the global distributed server node is calculated by the following formula: In the formula, μ i is the attribute mean of the global distributed server nodes.

4. The principal component analysis method based on distributed differential privacy protection according to claim 3 is characterized in that: In step S4, the attribute mean calculated by the central server is subtracted from the user data of each distributed server node, thereby obtaining the averaged data of each distributed server node, that is: In the formula, It is the mean value data of each distributed server node.

5. The principal component analysis method based on distributed differential privacy protection according to claim 4 is characterized in that: Step S5 specifically includes: S501: De-averaging the data generated by each distributed server node Perform absolute value processing and determine the maximum value corresponding to each attribute based on the averaged data after absolute value processing As a mapping parameter; S502: Mapping the de-averaged data of each distributed server node according to the mapping parameters to obtain the mapping data corresponding to each distributed server node, which is calculated in the following way: In the formula, Mapping data for any distributed server node; is the mean data of each distributed server node, h=1,2,...,m, j=1,2,...,m, t=1,2,...,T.

6. The principal component analysis method based on distributed differential privacy protection according to claim 5, characterized in that: Step S6 specifically includes: S601, initialization parameters And determine the perturbation interval [-C, C], where C is the size of the perturbation interval, p is the probability of selecting a high-functionality interval during the perturbation process, and ε2 is the privacy budget; S602: According to any mapping data in each distributed server node according to The size of the disturbance interval is divided into high-functional intervals and low functional range And randomly select a value in the corresponding interval as the perturbation data S603, randomly generate a natural number q between 0 and 1; S604: Any mapping data for each distributed server node When q<p is satisfied, the disturbance data is determined to be Located in the high functionality range, otherwise the perturbation data Located in the low functional range; S605, performing inverse mapping processing on any of the disturbance data by using the following formula to restore the original range of the data: In the formula, is the disturbance data after inverse mapping; S606: Repeat steps S601 to S605 until the mean-averaged data of all distributed server nodes are disturbed, thereby obtaining the disturbed mean-averaged data of each distributed server node.

7. The principal component analysis method based on distributed differential privacy protection according to claim 6, characterized in that: The covariance matrix of each distributed server node is calculated by the following formula: In the formula, S t is the covariance matrix of each distributed server node; for The transposed matrix of .

8. The principal component analysis method based on distributed differential privacy protection according to claim 7, characterized in that: In step S7, the global covariance matrix is ​​calculated by the following formula: Where S is the global covariance matrix.

9. The principal component analysis method based on distributed differential privacy protection according to claim 8, characterized in that: In step S7, principal component extraction is performed on the global covariance matrix, including: S701, performing eigenvalue decomposition on the global covariance matrix to obtain the eigenvalues ​​and eigenvectors of the global covariance matrix, namely: Sun i =λ i v i ; In the formula, λ i is the eigenvalue of the global covariance matrix, v i is the eigenvector of the global covariance matrix, i=1,2,...,m; S702: Arrange the eigenvalues ​​in descending order according to their magnitudes: λ1>λ2>...>λ m , then the corresponding eigenvectors are v1,v2,...,v m ; S703, extract the eigenvectors corresponding to the first k eigenvalues ​​as the principal components, i.e., V k =[v1,v2,...,v k ].

10. A principal component analysis device based on distributed local differential privacy protection, used to implement the principal component analysis method based on distributed differential privacy protection as claimed in any one of claims 1 to 9, characterized in that: The principal component analysis device based on distributed local differential privacy protection includes: A mean calculation module, used to obtain user data and the number of users of each distributed server node, and generate local sensitivity according to the data attributes of the user data, determine differential privacy noise based on the local sensitivity according to the Laplace mechanism, add the differential privacy noise to each data of the user data, obtain noise disturbance data, and calculate the attribute mean of the global distributed server node based on the noise disturbance data and the number of users; A data segmented disturbance module is used to perform mean removal processing on the user data of each distributed server node according to the attribute mean, obtain mean removal data of each distributed server node, and perform mapping processing on each mean removal data in each distributed server node to produce mapping data of each distributed server node, determine the disturbance interval according to the size of the mapping data, and then perform random disturbance to select disturbance data, perform inverse mapping processing on the disturbance data, obtain disturbance mean removal data of each distributed server node, and then calculate the covariance matrix according to the mean removal data, and publish it to the central server; The principal component extraction module is used to aggregate the disturbance covariance matrices of the distributed server nodes to the central server to calculate the global covariance matrix, and then perform principal component extraction on the global covariance matrix.

Images