Data principal acquisition method based on transverse federated learning
By employing a horizontal federated learning approach, and utilizing random number vectors and noise to protect data privacy, the problem of obtaining principal components among different participating entities is solved, thereby improving the accuracy of principal component analysis and the scope of data application.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUZHOU XINYUN TECH CO LTD
- Filing Date
- 2022-09-15
- Publication Date
- 2026-05-15
AI Technical Summary
The lack of a method for obtaining principal components that takes into account the privacy of different participants leads to insufficient data collection, which affects the accuracy of principal component analysis.
A horizontal federated learning-based approach is adopted, which generates random number vectors and noise through negotiation by a trusted coordinator. The covariance matrix is then calculated by combining local sample data to obtain the global covariance matrix and extract eigenvalues and eigenvectors, thereby achieving data preservation and privacy protection among all participating parties.
While protecting data privacy, the data from multiple participating entities can be integrated to improve the accuracy of principal component analysis and allow its use in more scenarios.
Smart Images
Figure CN115496139B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of information technology, and specifically to a method for obtaining data principal components based on horizontal federated learning. Background Technology
[0002] Principal component analysis (PCA) is an important unsupervised learning task in machine learning. It's commonly used to reduce the dimensionality of data and further for data visualization. A dataset typically consists of a set of data samples, each composed of a certain number of feature values. A dataset can be represented by a matrix. PCA seeks k linearly independent orthogonal bases such that the variance of the projection of the data matrix onto the first orthogonal base is maximized, and when choosing the i-th orthogonal base, after removing the influence of the previous i-1 orthogonal bases, the variance of the projection onto that orthogonal base is also maximized. PCA projects the original data matrix onto these k selected orthogonal bases, reducing the feature dimension of the data to k dimensions while preserving as much of the variation between data points as possible.
[0003] When applying principal component analysis (PCA), participating entities, including institutions and individuals, first need to collect data for the analysis. Often, this data comes from different data owners. This data may be sensitive. Due to privacy concerns, these data owners may be unwilling or legally prohibited from disclosing this sensitive data. In such cases, the analyzing institution or individual may not be able to collect enough data, thus affecting the accuracy of PCA. Therefore, it is necessary to study methods to reduce data dimensionality that can protect the data privacy and security of each participating entity. Summary of the Invention
[0004] The technical problem this invention aims to solve is the current lack of a method for obtaining principal components (PCS) across different participating entities while ensuring data privacy. This invention proposes a PCS extraction method based on horizontal federated learning, which can fuse data from multiple participating entities to complete PCS extraction while protecting data privacy.
[0005] To solve the above technical problems, the present invention adopts the following technical solution: a data principal component acquisition method based on lateral federated learning, comprising:
[0006] The participants select a trusted coordinator, and each participant P k Sum the features of each feature value in the local sample data X to obtain the feature vector W. k , n k x represents the number of samples in the local sample data. k，i Indicates participant P k The i-th row of data, x k，iIt has m eigenvalues;
[0007] The participants negotiate to generate a random number vector R. k , satisfying ∑R k =0;
[0008] The random number vector R k As features and vectors of noise and local sample data, W k Add, T k =W k +R k , will T k and n k Send to a trusted coordinator;
[0009] The trusted coordinator calculates the mean of each feature value and sends the mean to each participant.
[0010] The participants calculate the difference between each feature value and the mean of each sample data, and then calculate the covariance matrix U of the local sample data. k ;
[0011] The participants negotiate to generate a random number vector R again. k The random number vector and the covariance matrix U k Add, C k =U k +R k The local covariance matrix C with noise k Send to a trusted coordinator;
[0012] The trusted coordinator calculates the global covariance matrix C = ∑C k / N, where N is the total number of all sample data;
[0013] Using (λI-C)*p=0, we obtain the m eigenvalues and eigenvectors of the covariance matrix, where I is an m*m identity matrix;
[0014] Select d eigenvalues λ from m eigenvalues in descending order of their magnitude. i，i∈[1，d] and the corresponding eigenvector p i，i∈[1，d] , establish λ = (λ1, λ2,..., λ d ), P = (p1, p2, ..., p d ), where P is an n x d orthogonal matrix, and each column represents an orthogonal basis, i.e., an eigenvector p. i Send λ and P to each participant P. k ;
[0015] Participant P k Projecting the local sample data X onto the d-dimensional space formed by P, we obtain the projection L of the local sample data X in the d-dimensional space. L is the principal component of the local sample data.
[0016] As a preferred option, the participants negotiate to generate a random number vector R. k The methods include:
[0017] Assign an order to all participants: P1, P2, ..., P K K represents the number of participants;
[0018] For each pair of participants P k and P j Use a cryptographic exchange protocol to obtain the same private key, denoted as c. kj ;
[0019] After agreeing on the positive or negative attribute of the private key according to the order of the participants, the private key c kj As a random number vector R k One of the elements;
[0020] Determine the random number vector R k The required number of elements to be generated is obtained by repeatedly executing the aforementioned steps to obtain the required number of private keys c. kj , forming a random number vector R k .
[0021] As a preferred option, determine the random number vector R. k The method to determine the required number of elements is: obtain the features and vector W. k The number of elements is the random number vector R. k The number of elements to be generated.
[0022] Preferably, the cryptographic exchange protocol is the Diffe-Hellman key exchange protocol.
[0023] As a preferred option, each participant P k Multiple private keys c are obtained by pairing with other participants. kj Calculate r kj =PRNG(c kj ), where PRNG() is a pseudo-random generator function;
[0024] Then calculate r. k =-∑ j＜k r kj +∑ k＜j r kj r k As a random number vector R k One of the elements;
[0025] Repeat the above steps multiple times to obtain the required number of r. k , forming a random number vector R k .
[0026] As the preferred option, participant P k The method for projecting local sample data X onto the d-dimensional space formed by P is as follows: calculate L = X k *P, L are n rows and d columns matrices.
[0027] Preferably, the values of d are set from large to small to obtain the projection L of multiple local sample data X in d-dimensional space. d Calculate the covariance matrix U for each projection. kd The rank of the projection L with the largest rank. d The corresponding value d is the recommended principal component dimension.
[0028] As a preferred option, the participants will compare each feature value of each sample data with the mean. Calculate the difference, that is, let Calculate the covariance matrix
[0029] The beneficial technical effects of this invention include: by adopting a principal component analysis method based on horizontal federated learning, on the one hand, data is allowed to remain within each participating party without disclosing the original data to the outside world, thereby protecting data privacy; on the other hand, while protecting data privacy, it allows the use of data from more sources, and as the available data increases, the accuracy of model analysis is improved, enabling the principal component analysis method to be used in more scenarios.
[0030] Other features and advantages of the present invention will be disclosed in detail in the following detailed description and accompanying drawings. Attached Figure Description
[0031] The invention will be further described below with reference to the accompanying drawings:
[0032] Figure 1 This is a schematic diagram of the data principal component acquisition method according to an embodiment of the present invention.
[0033] Figure 2 A schematic diagram illustrating the process of generating random number vectors through negotiation among participating parties. Detailed Implementation
[0034] The technical solutions of the embodiments of the present invention will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of the present invention and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of the present invention.
[0035] In the following description, terms such as “inner,” “outer,” “upper,” “lower,” “left,” and “right” are used only to indicate orientation or positional relationship for the convenience of describing the embodiments and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.
[0036] For a method to obtain principal components of data based on horizontal federated learning, please refer to the appendix. Figure 1 ,include:
[0037] Step A01) The participants select a trusted coordinator, and each participant P k Sum the features of each feature value in the local sample data X to obtain the feature vector W. k , n k x represents the number of samples in the local sample data. k，i This represents the i-th row of data for participant Pk, x k，i It has m feature values. Assume there are K data owners, each possessing a portion of the sample set. These owners use federated learning to perform full principal component analysis on this data, aiming to project these samples into a d-dimensional space and preserve as much information as possible from the dimensionality reduction. Assume the k-th owner possesses n... k There are m sample data points, and all participants' samples have the same m numerical features. The dataset of the k-th participant is represented as follows: Where x k，i ∈R m This is the m-dimensional feature value data of the i-th sample from the k-th participant. Here, x k，i and W k These are all m-dimensional eigenvectors, and the summation is performed separately for each eigenvalue.
[0038] Step A02) Participants negotiate to generate a random number vector R k , satisfying ∑R k =0;
[0039] Step A03) Transfer the random number vector R k As features and vectors of noise and local sample data, W k Add, T k =W k +R k , will T k and n k Send to a trusted coordinator;
[0040] Step A04) The trusted coordinator calculates the mean of each feature value and sends the mean to each participant.
[0041] Calculating the mean includes: calculating the sum of all samples from all participants. Calculate the total number of samples from all participants: Calculate the mean of each feature of the sample:
[0042] Step A05) The participants will compare each feature value of each sample data with the mean. Calculate the difference Calculate the covariance matrix U of the local sample data k The participants compared each feature value of each sample data with the mean. Calculate the difference, that is, let Calculate the covariance matrix
[0043] Step A06) The participants negotiate to generate a random number vector R again. k The random number vector and the covariance matrix U k Add, C k =U k +R k The local covariance matrix C with noise k Send to a trusted coordinator;
[0044] Step A07) The trusted coordinator calculates the global covariance matrix C = ∑C k / N, where N is the total number of all sample data;
[0045] Step A08) Use (λI-C)*p=0 to obtain the m eigenvalues and eigenvectors of the covariance matrix, where I is an m*m identity matrix;
[0046] Step A09) Select d eigenvalues λ from m eigenvalues in descending order. i，i∈[1，d] and the corresponding eigenvector P i，i∈[1，d] , establish λ = (λ1, λ2,..., λ d ), P = (p1, p2, ..., p d ), where P is an n x d orthogonal matrix, and each column represents an orthogonal basis, i.e., an eigenvector p. i Send λ and P to each participant P. k ;
[0047] Step A10) Participant P k Projecting the local sample data X onto the d-dimensional space formed by P yields the projection L of the local sample data X into the d-dimensional space. L is the principal component of the local sample data. (Participant P) k The method for projecting local sample data X onto the d-dimensional space formed by P is as follows: calculate L = X k *P, L are n rows and d columns matrices.
[0048] By setting the value of d from largest to smallest, we obtain the projection L of multiple local sample data X onto the d-dimensional space. d Calculate the covariance matrix U for each projection. ka The rank of the projection L with the largest rank. d The corresponding value d is the recommended principal component dimension.
[0049] In steps A02) and A06), please refer to the appendix. Figure 2 The participating parties negotiate to generate a random number vector R. k The methods include:
[0050] Step B01) Assign an order to all participants: P1, P2, ..., P K K represents the number of participants;
[0051] Step B02) For each pair of participants P k and P j Use a cryptographic exchange protocol to obtain the same private key, denoted as c. kj ;
[0052] Step B03) After agreeing on the positive and negative attributes of the private key according to the order of the participants, the private key c kj As a random number vector R k One of the elements;
[0053] Step B04) Determine the random number vector R k The required number of elements to be generated is obtained by repeatedly executing the aforementioned steps to obtain the required number of private keys c. kj , forming a random number vector R k .
[0054] Determine the random number vector R k The method to determine the required number of elements is: obtain the features and vector W. k The number of elements is the random number vector R. k The number of elements to be generated. The cryptographic exchange protocol is the Diffe-Hellman key exchange protocol.
[0055] More advantageously, in this embodiment, the obtained key c kj Further transformation is required, specifically including: each participant P k Multiple private keys c are obtained by pairing with other participants. kj Calculate r kj =PRNG(c kj ), where PRNG() is a pseudo-random generator function; then r is calculated. k =-∑ j＜k r kj +∑ k＜j rkj r k As a random number vector R k One element; repeat the aforementioned steps multiple times to obtain the required number of r. k , forming a random number vector R k .
[0056] The beneficial technical effects of this embodiment include: by adopting the principal component analysis method based on horizontal federated learning, on the one hand, data is allowed to remain within each participant without disclosing the original data to the outside world, thereby protecting data privacy; on the other hand, while protecting data privacy, it allows the use of data from more sources, and as the available data increases, the accuracy of model analysis is improved, enabling the principal component analysis method to be used in more scenarios.
[0057] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art should understand that the present invention includes, but is not limited to, the contents described in the accompanying drawings and the specific embodiments above. Any modifications that do not depart from the functional and structural principles of the present invention will be included within the scope of the claims.
Claims
1. A method for obtaining principal components of data based on horizontal federated learning, characterized in that, include: The participants select a trusted coordinator, and each participant P k Sum the features of each feature value in the local sample data X to obtain the feature vector W. k , n k x represents the number of samples in the local sample data. k,i Indicates participant P k The i-th row of data, x k,i It has m eigenvalues; The participants negotiate to generate a random number vector R. k , satisfying ∑R k =0; The random number vector R k As features and vectors of noise and local sample data, W k Add, T k =W k +R k , will T k and n k Send to a trusted coordinator; The trusted coordinator calculates the mean of each feature value and sends the mean to each participant. The participants calculate the difference between each feature value and the mean of each sample data, and then calculate the covariance matrix U of the local sample data. k ; The participants negotiate to generate a random number vector R again. k The random number vector and the covariance matrix U k Add, C k =U k +R k The local covariance matrix C with noise k Send to a trusted coordinator; The trusted coordinator calculates the global covariance matrix C = ∑C k / N, where N is the total number of all sample data; Using (λI-C)*p=0, we obtain the m eigenvalues and eigenvectors of the covariance matrix, where I is an m*m identity matrix; from the m eigenvalues, we select d eigenvalues λ in descending order. i,i∈[1,d] and the corresponding eigenvector p i,i∈[1,d] Establish λ = (λ1, λ2, ..., λ d ), P = (p1, p2, ..., p d ), where P is an n x d orthogonal matrix, and each column represents an orthogonal basis, i.e., an eigenvector p. i Send λ and P to each participant P. k ; Participant P k Projecting the local sample data X onto the d-dimensional space formed by P, we obtain the projection L of the local sample data X in the d-dimensional space, and L is the principal component of the local sample data. The participants negotiate to generate a random number vector R. k The methods include: Assign an order to all participants: P1, P2, ..., P K K represents the number of participants; For each pair of participants P k and P j Use a cryptographic exchange protocol to obtain the same private key, denoted as c. kj ; After agreeing on the positive or negative attribute of the private key according to the order of the participants, the private key c kj As a random number vector R k One of the elements; Determine the random number vector R k The required number of elements to be generated is obtained by repeatedly executing the aforementioned steps to obtain the required number of private keys c. kj , forming a random number vector R k .
2. The method for obtaining principal components of data based on lateral federated learning according to claim 1, characterized in that, Determine the random number vector R k The method to determine the required number of elements is: obtain the features and vector W. k The number of elements is the random number vector R. k The number of elements to be generated.
3. The method for obtaining principal components of data based on lateral federated learning according to claim 1 or 2, characterized in that, The cryptographic exchange protocol is the Diffe-Hellman key exchange protocol.
4. The method for obtaining principal components of data based on lateral federated learning according to claim 1 or 2, characterized in that, Each participant P k Multiple private keys c are obtained by pairing with other participants. kj Calculate r kj =PRNG(c kj ), where PRNG() is a pseudo-random generator function; Then calculate r. k =-∑ j<k r kj +∑ k<j r kj r k As a random number vector R k One of the elements; Repeat the above steps multiple times to obtain the required number of r. k , forming a random number vector R k .
5. The method for obtaining principal components of data based on lateral federated learning according to claim 1 or 2, characterized in that, Participant P k The method for projecting local sample data X onto the d-dimensional space formed by P is as follows: calculate L = X k *P, L are n rows and d columns matrices.
6. The method for obtaining principal components of data based on lateral federated learning according to claim 1 or 2, characterized in that, By setting the value of d from largest to smallest, we obtain the projection L of multiple local sample data X onto the d-dimensional space. d Calculate the covariance matrix U for each projection. kd The rank of the projection L with the largest rank. d The corresponding value d is the recommended principal component dimension.
7. The method for obtaining principal components of data based on lateral federated learning according to claim 1 or 2, characterized in that, The participants will compare each feature value of each sample data with the mean. Calculate the difference, that is, let Calculate the covariance matrix