Linear regression method for longitudinal federated learning

By constructing an encrypted data matrix and sending the product with the orthogonal transformation to the calculation node, multiplication and encrypted linear regression coefficient vector is calculated, which solves the problem of high complexity in the existing technology, and realizes simple collaboration and data sharing among users, reducing the number of users. Implementation complexity.

CN119066625AInactive Publication Date: 2024-12-03WUYI UNIV

Patent Information

Application Number
CN202411090648.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the existing vertical federated learning technology that implements linear regression, secure multi-party computing requires the use of complex obfuscation circuits and trusted initializers and encryption service providers, which leads to high implementation complexity and it is difficult to simply collaborate and share data between enterprises.

Method used

By constructing an encrypted data matrix and sending the product with the orthogonal transformation to the calculation node, the multiplication encrypted linear regression coefficient vector is calculated, which avoids the need to introduce a trusted initializer and a high-complexity obfuscating circuit, and realizes simple collaboration between users .

Benefits of technology

Reduces implementation complexity, allowing vertical federated learning to be completed only by users who own data, simplifying data sharing and model training processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066625A_ABST
    Figure CN119066625A_ABST
Patent Text Reader

Abstract

The invention relates to a linear regression method for longitudinal federated learning, which comprises the following steps that: a first participant constructs an original data matrix as a column encryption data matrix of a sub-matrix according to the original data matrix formed by private original data, a product of the column encryption data matrix and an orthogonal transformation is sent to a first computing node; and the first computing node cooperates with a label data owner to compute a multiplied encrypted linear regression coefficient vector. Compared with the prior art, the embodiment of the invention does not need to introduce a credible initializer and an encryption service provider, and does not need to realize a mixed dish circuit with very high complexity, so that the implementation complexity can be reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a linear regression method for vertical federated learning. Background Art

[0002] The development of artificial intelligence based on machine learning is inseparable from the massive data that supports its training models. However, artificial intelligence technology faces many challenges in its practical application in various industries, including privacy, security and efficiency. Its data is an important factor affecting the development of applications. Due to the privacy issues of training data and the trust issues between participants, it is difficult for private data to circulate between enterprises, that is, the potential application value cannot be brought into play and there is a lack of valuable data cooperation, which restricts their further development.

[0003] As a new branch of artificial intelligence technology, federated learning can share data, train models, and release data value among multiple participants under the premise of ensuring privacy protection and data security, thus helping industrial innovation and promoting the development of enterprises and industries. The application scenarios of federated learning can be divided into horizontal federated learning and vertical federated learning. Horizontal federated learning is also called sample-based federated learning. Each participant has different data samples and improves the performance of the training model through cooperation. Each participant may be an enterprise with data or a user's client, etc. Vertical federated learning is also called feature-based federated learning. Each participant has different features of the sample. Each participant is usually an enterprise or large organization with different fields and directions.

[0004] In the existing vertical federated learning technology for linear regression, secure multi-party computing requires the use of obfuscated circuits, as well as the introduction of trusted initializers, encryption service providers, and evaluators, and the complexity of technical implementation is very high. Summary of the invention

[0005] Based on this, it is necessary to provide a linear regression method for vertical federated learning for the above technical problems, which can be completed by collaboration between users who own data, and is simple to implement; it can also be completed by introducing computing and storage nodes (i.e., evaluators) to collaborate. The present invention does not require the introduction of a trusted initializer and encryption service provider, nor does it require the implementation of a very complex obfuscation circuit, thereby reducing the complexity of implementation.

[0006] An embodiment of the present invention provides a linear regression method for vertical federated learning, the method comprising the following steps:

[0007] The first participant constructs a column encrypted data matrix with the original data matrix as a submatrix according to the original data matrix formed by the private original data, and sends the product of the column encrypted data matrix and an orthogonal transformation to the first computing node; or, there are m participants with their own private original data, wherein a total of m-1 participants from the second participant to the mth participant all construct a column encrypted data matrix with the original data matrix as a submatrix, and send the product of the column encrypted data matrix and an orthogonal transformation to the first computing node;

[0008] The first computing node cooperates with the tag data owner to calculate and obtain the multiplied encrypted linear regression coefficient vector.

[0009] Further, the first participant constructs a column encrypted data matrix with the original data matrix as a submatrix according to the original data matrix formed by the private original data, and sends the product of the column encrypted data matrix and an orthogonal transformation to the first computing node, including:

[0010] The first participant constructs a column encrypted data matrix with the original data matrix as a submatrix from the original data matrix formed by the private original data, and sends the product of the column encrypted data matrix and the orthogonal transformation to the first computing node; after the first computing node receives the product of the column encrypted data matrix and the orthogonal transformation, it calculates and stores the multiplication encryption result of the square root matrix of the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix, that is, the multiplication encryption factor matrix, and the column encrypted data and factor product matrix, wherein the multiplication encryption factor matrix is ​​the product of the square root matrix of the multiplication encryption mask matrix and the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix, and the column encrypted data and factor product matrix is ​​the product of the square root matrix of the column encrypted data matrix and the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix.

[0011] Furthermore, there are m participants having their own private original data, wherein m-1 participants from the second participant to the mth participant all construct a column encrypted data matrix with the original data matrix as a submatrix, and send the product of the column encrypted data matrix and an orthogonal transformation to the first computing node, including:

[0012] The m participants respectively construct a column encrypted data matrix with the original data matrix as a submatrix from the original data matrix formed by their own private original data, and send the product of the above column encrypted data matrix and the orthogonal transformation to the first computing node, so that after the first computing node receives the product of the column encrypted data matrix and the orthogonal transformation sent by the m participants respectively, it cooperates with the label data owner to calculate the multiplied encrypted linear regression coefficient vector, where m is an integer greater than or equal to 2.

[0013] Furthermore, the first computing node cooperates with the tag data owner to calculate the multiplication encrypted linear regression coefficient vector, including:

[0014] The first computing node calculates and obtains the multiplication encryption result of the square root matrix of the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix of a total of m participants, that is, the multiplication encryption factor matrix, and the column encrypted data and factor product matrix corresponding to the column encrypted data matrix of a total of m participants, wherein the multiplication encryption factor matrix is ​​the product of the square root matrix of the multiplication encryption mask matrix and the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix, and the column encrypted data and factor product matrix is ​​the product of the square root matrix of the column encrypted data matrix and the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix; the label data owner uses the private label column vector to cooperate with the first computing node that owns the multiplication encryption factor matrix of a total of m participants and the column encrypted data and factor product matrix of a total of m participants to calculate the multiplication encryption linear regression coefficient vector, and the multiplication encryption linear regression coefficient vector is the product of the multiplication encryption mask matrix and the linear regression coefficient vector.

[0015] Furthermore, the method further comprises:

[0016] In the process of calculating the multiplication encryption factor matrix and the column encrypted data and factor product matrix of a total of m participants by the first computing node, the multiplication encryption factor matrix and the column encrypted data and factor product matrix of a total of i-1 participants are used to obtain the multiplication encryption factor matrix and the column encrypted data and factor product matrix of a total of i participants. Starting from i=2, iterative calculations are performed on i=2, 3, ..., m in turn, and finally the multiplication encryption factor matrix and the column encrypted data and factor product matrix of a total of i=m participants are obtained.

[0017] Furthermore, the first computing node is composed of a plurality of logical nodes, and each of the plurality of logical nodes stores part of the data.

[0018] Furthermore, the constraint of the square root matrix obtained by the first participant is:

[0019]

[0020] Among them, F 1 is the square root matrix, F 1 The transpose or conjugate transpose of Encrypt the data matrix for the first party's columns, for The transpose or conjugate transpose of , λ is a regularization coefficient greater than or equal to zero, and I is the unit matrix;

[0021] The constraints of the L factor matrix and the D factor matrix obtained by the first participant are:

[0022]

[0023] Among them, D 1 is a D-factor matrix, a diagonal matrix; L 1 is the L factor matrix, which is a square matrix.

[0024] Furthermore, the column encrypted data matrix is ​​one of a matrix consisting of an original data matrix of a participant, an original data matrix of a participant and a column-added encrypted mask matrix; the column-added encrypted mask matrix includes one or more columns.

[0025] Furthermore, when the column encrypted data matrix of the first participant is a matrix composed of the original data matrix of the first participant and the column-added encrypted mask matrix, after the multiplication encrypted linear regression coefficient vector is obtained by calculation, the method further includes:

[0026] The label data owner or the first computing node sends the multiplication encrypted linear regression coefficient vector to the first participant or forwards it to the first participant by other participants, so that the first participant can at least partially eliminate the influence of its own column-added encryption mask matrix on the multiplication encrypted linear regression coefficient vector based on its own private multiplication encrypted mask matrix, and update the multiplication encrypted linear regression coefficient vector.

[0027] Furthermore, when the column encrypted data matrix of the other participants is a matrix composed of the original data matrix of the other participants and the column-added encrypted mask matrix, after the multiplication encrypted linear regression coefficient vector is obtained by calculation, the method further includes:

[0028] The label data owner or the first computing node sends the multiplication encrypted linear regression coefficient vector to the other participants or forwards it to the other participants by any participant except the other participants, so that the other participants can at least partially eliminate the influence of their own column addition encryption mask matrix on the multiplication encrypted linear regression coefficient vector based on their own private multiplication encryption mask matrix, and update the multiplication encrypted linear regression coefficient vector.

[0029] Furthermore, the method further comprises:

[0030] The label data owner, the first computing node, the first participant or the other participants send several multiplication encrypted linear regression coefficients corresponding to several data features to the corresponding participants, so that the corresponding participants decrypt the received several multiplication encrypted linear regression coefficients according to their own multiplication encrypted mask matrix to obtain the corresponding several linear regression coefficients.

[0031] Furthermore, the calculation formula of the multiplied encrypted linear regression coefficient vector is:

[0032]

[0033] Where m represents the number of parties providing private data; is the label vector, T m is the column encrypted data and factor product matrix received by the tag data owner from the first participant or other participants, P m is the multiplication factor matrix received or calculated by the first computing node;

[0034] The multiplied encrypted linear regression coefficient vector with the linear regression coefficient vector Satisfy between:

[0035]

[0036] Among them, i The multiplication encryption mask matrix for the i-th party providing private data.

[0037] In the above linear regression method for vertical federated learning, the first participant constructs the column encrypted data matrix of the original data matrix as a submatrix based on the original data matrix formed by the private original data, and sends the product of the column encrypted data matrix and an orthogonal transformation to the first computing node; or, there are m participants with their own private original data, among which the total m-1 participants from the second participant to the mth participant all construct the column encrypted data matrix of the original data matrix as a submatrix, and send the product of the column encrypted data matrix and an orthogonal transformation to the first computing node; then the first computing node cooperates with the label data owner to calculate the multiplication encrypted linear regression coefficient vector. Compared with the prior art, the embodiment of the present invention can be completed only by collaboration between users who own the data, and the implementation is simple; computing and storage nodes (i.e., evaluation parties) can also be introduced. The present invention does not need to introduce a trusted initializer and encryption service provider, nor does it need to implement a very complex confusion circuit, thereby reducing the implementation complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flowchart of a linear regression method for vertical federated learning provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this technical field without creative work are within the scope of protection of the present invention.

[0040] It should be noted that the step numbers in the text are only for the convenience of explaining the specific embodiment and do not limit the order of execution of the steps. The method provided in this embodiment can be executed by a related server, and the following description will be given by taking the server as the execution subject as an example.

[0041] like Figure 1 As shown, the linear regression method for vertical federated learning provided by the first embodiment of the present invention includes steps S11 to S12:

[0042] Step S11: The first participant constructs a column encrypted data matrix with the original data matrix as a submatrix according to the original data matrix formed by the private original data, and sends the product of the column encrypted data matrix and an orthogonal transformation to the first computing node; or, there are m participants with their own private original data, wherein a total of m-1 participants from the second participant to the mth participant all construct a column encrypted data matrix with the original data matrix as a submatrix, and send the product of the column encrypted data matrix and an orthogonal transformation to the first computing node;

[0043] Step S12: The first computing node cooperates with the tag data owner to calculate and obtain the multiplied encrypted linear regression coefficient vector.

[0044] The first participant constructs a column encrypted data matrix with the original data matrix as a submatrix from the original data matrix formed by the private original data, and sends the product of the column encrypted data matrix and the orthogonal transformation to the first computing node; after receiving the product of the column encrypted data matrix and the orthogonal transformation, the first computing node calculates and stores the multiplication encryption result of the square root matrix of the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix, i.e., the multiplication encryption factor matrix, and the column encrypted data and factor product matrix. The multiplication encryption factor matrix is ​​the product of the square root matrix of the multiplication encryption mask matrix and the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix, and the column encrypted data and factor product matrix is ​​the product of the square root matrix of the column encrypted data matrix and the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix.

[0045] More generally, m (m is greater than or equal to 1) participants respectively construct a column encrypted data matrix with the original data matrix as a submatrix from the original data matrix formed by their own private original data, and send the product of the above column encrypted data matrix and the orthogonal transformation to the first computing node; after the first computing node receives the product of the column encrypted data matrix and the orthogonal transformation sent by the above m participants respectively, it calculates the multiplication encryption result of the square root matrix of the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix of the m participants, that is, the multiplication encryption factor matrix, and the column encrypted data and factor product matrix corresponding to the column encrypted data matrix of the m participants.

[0046] Then, the owner of the label data, i.e., the m+1th participant, uses the private label column vector to collaborate with the first computing node that owns the multiplication encryption factor matrix of the m participants and the column encryption data and factor product matrix of the m participants to calculate the multiplication encryption linear regression coefficient vector, i.e., the product of the multiplication encryption mask matrix and the linear regression coefficient vector.

[0047] In the process of calculating the multiplication encryption factor matrix and the column encrypted data and factor product matrix of m participants in total at the first computing node, the multiplication encryption factor matrix and the column encrypted data and factor product matrix of i-1 participants in total are used to obtain the multiplication encryption factor matrix and the column encrypted data and factor product matrix of i participants in total. Starting from i=2, iterative calculations are performed on i=2, 3, ..., m in turn, and finally the multiplication encryption factor matrix and the column encrypted data and factor product matrix of i=m participants are obtained.

[0048] The technical solution of the present invention includes step S11, and step S11 further includes step S111 and step S112, which are described as follows. The following first introduces sub-steps S111-a to S111-e included in step S111:

[0049] Sub-step S111-a: This sub-step is an optional sub-step for adding mask columns. In this sub-step, the first participant adds a mask column to its original data matrix X 1 Adding one or more columns of masks at the end is an optional step. If the first participant hopes to achieve a better privacy protection effect, he / she can choose this step. Usually, a single column of mask vector can achieve the privacy protection effect. The first participant can also use multiple columns of masks to achieve a better privacy protection effect by increasing the number of columns of the mask. The first participant forms the original data matrix X based on the private original data 1 And add the encrypted mask matrix Δ 1 Generate a data matrix with added mask columns It is also referred to as the column-encrypted data matrix.

[0050] Specifically, the column-encrypted data matrix is ​​the data matrix with mask columns added The original data matrix X of the first participant 1 There is τ 1 The column contains τ 1 The original data of features, and the encrypted mask matrix Δ 1 have The columns can be generated by the first party, usually adding columns to the encrypted mask matrix Δ 1 is randomly generated. Indicates that when there is τ 1 The columns of the original data matrix X 1 Added later Column-added encryption mask matrix Δ 1 To obtain common Column-by-column encrypted data matrix Notice Can be equal to 0, which is equivalent to the first party not adding columns to the encryption mask matrix Δ 1 to encrypt the original data; in addition, it is usually If it is equal to 1, it can have a good encryption effect. Column-added encryption mask matrix Δ 1 In fact, it is a mask column vector with only one column. If the first party does not choose this optional step of adding a mask column, then there will be Note that in the above description, Column-added encryption mask matrix Δ 1 Put it in the original data matrix X 1 The following is just for the sake of simplicity; in actual implementation, the encrypted mask matrix Δ 1 of Columns can be shared Column-by-column encrypted data matrix Anywhere in Columns can also be separated or non-contiguous List.

[0051] Sub-step S111-b: In this sub-step, the first participant uses the original data matrix X containing itself 1 Matrix as a submatrix (i.e. column-encrypted data matrix) Multiply by an orthogonal transformation Ω 1 get Then Sent to the first computing node.

[0052] Sub-step S111-c: In this sub-step, the first computing node receives the Obtain the corresponding multiplication-encrypted square root matrix P 1, the multiplication of the encrypted square root matrix P 1 satisfy Here I represents the identity matrix, and λ represents a scalar, called the regularization coefficient. Note that the first computation node can be replaced by an arbitrary orthogonal matrix Θ 1 Multiply right by P 1 Get the updated P 1 =P 1 Θ 1 The multiplication of the encrypted square root matrix P 1 , is to satisfy The square root matrix F 1 Do multiplication encryption, that is The square root matrix F 1 is the original data matrix X containing the first party 1 Matrix as a submatrix (i.e. column-encrypted data matrix) The regularized symmetric matrix The inverse matrix The square root matrix F 1 .

[0053] Sub-step S111-d: The first computing node uses the received And the multiplication encrypted square root matrix P obtained in the previous sub-step 1 , get the product of column encrypted data and square root matrix T 1 , by calculating The T 1 is an intermediate variable, which is the original data matrix X containing the first party 1 The matrix as a submatrix is ​​the column-encrypted data matrix Matrix with column encrypted data The square root matrix of the inverse matrix of the regularized symmetric matrix F 1 The product of 1 , that is, T 1 satisfy Note that in this embodiment, although the first computing node does not have a column encrypted data matrix and square root matrix F 1 , but can be calculated by Get Satisfied T 1 In the following, the first party's column encrypted data matrix and square root matrix F 1 The product of 1 , referred to as the first party's column encrypted data and factor product matrix T 1 , also known as the intermediate variable T 1 .

[0054] In the present invention, (·)T represents the transpose of a real matrix (including vectors), or represents the conjugate transpose of a complex matrix (including vectors), as described above In and F 1 F 1 T In Putting two matrices together, or a scalar and a matrix together, means multiplication, for example middle express take And λI means λ times I. Note that the regularized symmetric matrix In this case, λI is a matrix related to regularization. In some cases, λI can be in the form of The diagonal matrix of , where the entries on the diagonal can take different values; even λI can be an arbitrary matrix.

[0055] Sub-step S111-e: The control variable i corresponding to the above-mentioned first participant (first data owner) is equal to 1. In this sub-step, the value of i is first increased by 1; then, if the value of i is greater than m, step S12 is entered; if the value of i is less than or equal to m, step S112 is entered. As mentioned above, vertical federated learning has m participants who own their own private original data, also known as m data owners, where m is an integer greater than or equal to 1; in addition, there is a first computing node and the m+1th participant called the label owner. This step is explained as follows: The value of m determines the characteristics of the aforementioned second participant, and also determines which step to enter next. When m is equal to 1, the above-mentioned second participant is the label data owner, and step S12 is entered accordingly; when m is greater than or equal to 2, the above-mentioned second participant is the second data owner, and step S112 is entered accordingly.

[0056] The sub-steps S111-a to S111-e included in step S111 in the above step S11 can also have a second implementation method, and the corresponding main principle is: although in the scenario with the first computing node, the computing task is usually arranged on the first computing node first; however, for the scenario of the first participant, the computing task can also be arranged on the first participant. Accordingly, the second implementation method of sub-steps S111-a to S111-e included in step S111 in step S11 is introduced as follows:

[0057] Sub-step S111-a of the second implementation method: This sub-step is an optional sub-step of adding a mask column. In this sub-step, the first participant adds a mask column to its original data matrix X 1Adding one or more columns of masks at the end is an optional step. If the first participant hopes to achieve a better privacy protection effect, he / she can choose this step. Usually, a single column of mask vector can achieve the privacy protection effect. The first participant can also use multiple columns of masks to achieve a better privacy protection effect by increasing the number of columns of the mask. The first participant forms the original data matrix X based on the private original data 1 And add the encrypted mask matrix Δ 1 Generate a data matrix with added mask columns It is also referred to as the column-encrypted data matrix. Specifically, the column-encrypted data matrix is ​​the data matrix with mask columns added The original data matrix X of the first participant 1 There is τ 1 The column contains τ 1 The original data of features, and the encrypted mask matrix Δ 1 have The columns can be generated by the first party, usually adding columns to the encrypted mask matrix Δ 1 is randomly generated.

[0058] Sub-step S111-b of the second implementation method: In this sub-step, the first participant uses the original data matrix X containing itself 1 Matrix as a submatrix (i.e. column-encrypted data matrix) Obtain the corresponding multiplication-encrypted square root matrix P 1 . Find P 1 There are many ways to calculate , and two different methods are listed below: The first method is to use column encryption data matrix Multiply by an orthogonal transformation Ω 1 get Seek satisfaction again The multiplication of the encrypted square root matrix P 1 ; The second calculation method is to calculate the satisfaction The square root matrix F 1 , and then multiply the encryption mask matrix Ξ 1 Left multiply the square root matrix F 1 , get the multiplication encryption factor matrix P 1 =Ξ 1 F 1 Note that the first computation node can be represented by an arbitrary orthogonal matrix Θ 1 Multiply right by P 1 Get the updated P 1 =P 1 Θ 1 .

[0059] Sub-step S111-c of the second implementation method: In this sub-step, the first participant obtains the product T of the column encrypted data and the square root matrix 1 . Find T 1 There are many ways to calculate , and two different methods are listed below: The first method is to use the calculation method obtained in sub-step S111-b. and P 1 ,calculate The second calculation method is to use the column encryption data matrix The square root matrix F obtained in the second calculation method of sub-step S111-b 1 ,calculate

[0060] Sub-step S111-d of the second implementation method: The first participant multiplies the encrypted square root matrix P 1 Sum column encrypted data and square root matrix product T 1 Sent to the first computing node.

[0061] The present invention uses a concatenation matrix of encrypted data of i (i is less than or equal to m) participants 1, 2, ..., i. It is referred to as the column encrypted data matrix of the i-th participant. Note that the column encrypted data matrix of the j-th participant (j is greater than or equal to 1 and less than or equal to i) is is the original data matrix X consisting of the original data privately owned by the jth participant j Add an additional column to the encrypted mask matrix Δ j The obtained value can be obtained without increasing the number of columns of the encrypted mask matrix Δ j Thus there is It is easy to see that step S11 uses It can also be expressed as That is, the column encrypted data matrix of a common participant This representation method also covers the aforementioned step S111.

[0062] In step S112 included in step S11, the first computing node receives the encrypted data matrix of the i-th participant (i is greater than or equal to 2 and less than or equal to m), i.e., the i-th data owner. (Note Δ i can be an empty matrix as a special case) and the orthogonal transformation Ω i The product of The first computing node uses the received And the column encrypted data of i-1 participants stored by itself and the square root matrix product T i-1 and multiply the encryption factor matrix P i-1, calculate the column encrypted data of the total i participants and the square root matrix product T i and multiply the encryption factor matrix P i The step S112 included in the above step S11 includes the following sub-steps S112-a to S112-f, a total of 6 sub-steps:

[0063] Sub-step S112-a: This sub-step is an optional sub-step for adding mask columns. In this sub-step, the data matrix X i The i-th participant in his data matrix X i Adding one or more columns of masks at the end is an optional step. The data matrix X constructed by the i-th participant based on the private original data i And add the encrypted mask matrix Δ i The generated data matrix with added mask columns It is also referred to as the column-encrypted data matrix.

[0064] Specifically, the column-encrypted data matrix is ​​the data matrix with mask columns added The data matrix X of the i-th participant i have The columns contain The original data of features, and the encrypted mask matrix Δ i have The columns can be generated by the i-th participant, usually adding columns to the encrypted mask matrix Δ i is randomly generated. Indicates that there is The columns of the data matrix X i Added later Column-added encryption mask matrix Δ i To obtain common Column-by-column encrypted data matrix in Can be equal to 0, which is equivalent to the i-th participant not adding columns to the encryption mask matrix Δ i to encrypt the original data; in addition, it is usually If it is equal to 1, it can have a good encryption effect. Column-added encryption mask matrix Δ i In fact, it is a mask column vector with only one column. If the i-th participant does not choose this optional step of adding a mask column, then there will be Note that in the above description, Column-added encryption mask matrix Δ i Put it in the original data matrix X i The following is just for the sake of simplicity; in actual implementation, the encrypted mask matrix Δi of Columns can be shared Column-by-column encrypted data matrix Anywhere in Columns can also be separated or non-contiguous List.

[0065] Sub-step S112-b: In this sub-step, the i-th participant uses the original data matrix X containing itself i Matrix as a submatrix (i.e. column-encrypted data matrix) Multiply by an orthogonal transformation Ω i get Then Sent to the first computing node.

[0066] Sub-step S112-c: In this sub-step, the first computing node receives Calculate the square root matrix F for a total of i participants i The submatrix G in i The matrix obtained by multiplying and encrypting satisfies The matrix Satisfied by calculation The square root matrix At this step, we can use an orthogonal matrix Θ i Right multiplication To Updated to

[0067] The above submatrix G i is the square root matrix F of the total number of participants i A submatrix in As shown, × represents a submatrix that is not currently of interest. i satisfy It is easy to see that the square root matrix F i The submatrix G in i It is also a regularized symmetric matrix The inverse matrix The square root matrix of G i Do the multiplication and encryption to get the matrix satisfy Right now is an orthogonal matrix Multiply G i The result obtained.

[0068] Sub-step S112-d: In this sub-step, the first computing node calculates the column encryption data and factor product matrix T of the i participants in total i , but in fact the first computing node only needs to calculate Ti The product of the column encrypted data and the square root matrix T for a total of i-1 participants i-1 The specific implementation method is that the first computing node calculates Among them, only T i Relative T i-1 The added columns are Need to be calculated.

[0069] As mentioned above, T i is the column-encrypted data matrix of i participants With the square root matrix F i The product of The square root matrix F i is the column-encrypted data matrix of i participants The regularized symmetric matrix The inverse matrix The square root matrix, that is, the square root matrix F i satisfy Note that T i Is and F i As defined above, the first computing node does not have the and F i , we can still calculate T i .

[0070] Sub-step S112-e: In this sub-step, the first computing node converts P i-1 Update to P i As mentioned above, P i is the square root matrix F of the total number of participants i i The multiplication factor matrix is ​​obtained by multiplying a matrix on the left side for multiplication encryption, and the square root matrix F of the total i participants is i satisfy A specific implementation method may be that the first computing node obtains by calculation In P i-1 Add multiple columns to the right of In P i-1 Add a submatrix with all zeros just below P i-1 Add a submatrix to the lower right corner Thus, the multiplication encryption factor matrix P of the total i-1 participants is i-1 Update to the multiplication encryption factor matrix P of the total i participants i .

[0071] As mentioned before, the matrix is the square root matrix F of the i participants i The submatrix G in iDo the multiplication encryption to get the matrix, matrix satisfy Right now is an orthogonal matrix Multiplier matrix G i The result is obtained. Because only participant i knows the orthogonal matrix Ω i , that is, only participant i knows the orthogonal matrix Thus, the first computing node does not know the orthogonal matrix So it is impossible to get the matrix The square root matrix F of the i participants is derived i The submatrix G in i , so that for the first computing node, the submatrix G is realized i In other embodiments, the encryption matrix is ​​usually multiplied by Ξ i It can be seen from the above description that in this embodiment, the multiplication of the encryption matrix Accordingly, as mentioned above, multiply the encryption factor matrix Substitute it into Can get

[0072]

[0073] Sub-step S112-f (sub-step of iteration control): If the value of i is less than or equal to m-1, then increase the value of i by 1 and return to sub-step S112-a to start the next iteration; otherwise, when the value of i is equal to m, proceed to the following step S12. Note that in the above steps of this embodiment, a total of m participants including participants 1, 2, ...m all use their own original data matrix X i Matrix as a submatrix (i.e. column-encrypted data matrix) Multiply by an orthogonal transformation Ω i get Then Send to the first computing node; the above sending There is a sequence in the description, but there is no sequence in the actual implementation. In other words, when describing, Send in Before sending; when implementing, The sending can be Furthermore, the sequence numbers of the above participants 1, 2, ...m may not be pre-set, but the above m participants all send their matrices The message is sent to the first computing node, and the first computing node arbitrarily sets the m participants as participants 1, 2, ...m.

[0074] The steps of calculating the multiplication encrypted linear regression coefficient vector in the embodiment of the present invention are as follows: the tag data owner, i.e. the m+1th participant, uses the private tag column vector and the multiplication factor matrix P with a total of m participants m and the column encrypted data and factor product matrix T of m participants m The first computing node cooperates to calculate the multiplication of the encrypted linear regression coefficient vector That is, multiply the encrypted mask matrix by the linear regression coefficient vector. A specific implementation method may be that the first computing node calculates P m and T m Sent to the owner of the label data, who then calculates

[0075] Calculate the above multiplication encrypted linear regression coefficient vector Afterwards, the subsequent steps of the above embodiment of the present invention achieve the following effect: if there are several participants among the participants 1 to m who use the column-increasing encryption mask matrix with non-zero columns, then the several participants will multiply the encryption factor matrix P in turn. m Sum and multiply the encrypted linear regression coefficient vector At least partially eliminate the influence of its column-added encryption mask matrix Δ. When each participant using a non-zero column-added encryption mask matrix multiplies the encrypted linear regression coefficient vector The influence of the participant's added column encryption mask matrix Δ is at least partially eliminated, or when there is no participant using a non-zero column added column encryption mask matrix, then the most recently updated multiplied encrypted linear regression coefficient vector The participant or the first computing node The corresponding items of participant i (where i is greater than or equal to 1 and less than or equal to m) are sent to participant i, and the participant i uses its own private multiplication encryption mask matrix Ξ i Decryption obtains the corresponding items in the linear regression coefficient vector. The implementation method of eliminating the influence of the added column encryption mask matrix may not include the first computing node. Accordingly, it is necessary to multiply the encryption factor matrix P=P m Sum and multiply the encrypted linear regression coefficient vector Sent to the i-th participant, where i is a positive integer less than or equal to m-1, or i is a positive integer less than or equal to m. There are also other implementations for the step of eliminating the influence of adding columns to the encryption mask matrix, where P m and At least one of is stored in the first computing node.

[0076] The first computing node mentioned above can be independent of the m+1 participants mentioned above, or it can be set to any one of the m+1 participants mentioned above. The first computing node mentioned above can be a cryptographic service provider (CSP) or an evaluator (Evaluator) often mentioned in the literature in the field of federated learning, or the first computing node mentioned above can also include both a cryptographic service provider and an evaluator. The first computing node performs computing and storage functions, and may also provide encryption services. The first computing node is a logical node. As is known to all, the physical node corresponding to it can be located in any one of the m+1 participants mentioned above; in addition, the first computing node can also include multiple logical nodes, for example, including both the cryptographic service provider and the evaluator. Accordingly, the two physical nodes corresponding to the cryptographic service provider and the evaluator can be located in two of the m+1 participants mentioned above, respectively.

[0077] In the case where the first computing node includes at least two logical nodes, the above-mentioned i-th participant (i is greater than or equal to 1 and less than or equal to m) Sent to the first computing node and received by the first computing node Includes equivalent sending and receiving, for example: the first computing node contains a logical node sending a size and matrix The matrix Ψ i To the i-th participant, the i-th participant gives Add or subtract Ψ i The result is sent to another logical node contained in the first computing node. It is easy to see that a logical node contained in the first computing node knows Ψ i , and another logical node contained in the first computing node receives Add or subtract Ψ i As a result, as long as these two logical nodes cooperate, it can be simply Add or subtract Ψ i In the result, subtract or add Ψ accordingly. i To get The collaboration between the above two logical nodes can adopt the existing multi-party secure computing method.

[0078] Based on the implementation method of using square root decomposition to decompose the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix of the first participant in the embodiment of the present invention, another commonly used LDL is easily obtained: T For example, the above FF T =(X T X+λI) -1 The matrix F is the inverse matrix (X T X+λI) -1The square root matrix can be regarded as a symmetric matrix (X T X+λI) -1 A decomposition factor matrix, the corresponding symmetric matrix can also use another commonly used LDL T LDL in the process of breakdown T Factor matrix replacement, and LDL T The factor matrix includes a triangular L factor matrix and a diagonal D factor matrix. The L factor matrix can be expanded to a general square matrix, that is, the L factor matrix can be a non-triangular matrix or a triangular matrix, and the D factor matrix is ​​still a diagonal matrix. T The factor matrix can be slightly modified to obtain the LDL-based T Specific implementation of factorization matrix.

[0079] The factorization matrix of the inverse matrix of the regularized symmetric matrix of the column-encrypted data matrix used in the embodiment is the square root matrix F i , and satisfies For i greater than or equal to 1 and less than or equal to m. Obviously, there is another LDL as a factorization matrix T The factor matrix, i.e. L i and D i satisfy Where D is the factor matrix D i is a diagonal matrix, and the L factor matrix L i is a general square matrix, which can be a triangular matrix or a non-triangular matrix. In practical applications, the above L i and D i A simple method is to convert the matrix F in Example 2 i Set to L i , that is, L i =F i , and let D i is equal to the identity matrix, then the embodiment becomes based on the decomposition factor matrix L i and D i More generally, D i It may not be equal to the identity matrix. It can be based on Derived and in is a diagonal matrix, each entry on its diagonal is D i The square roots of the diagonal terms in the same position, and yes The inverse matrix of , each entry on its diagonal is The inverse of the item at the same position on the diagonal. Accordingly, a more general LDL-based TThe implementation example of the factor matrix will not be described in detail here.

[0080] It should be understood that, although the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0081] In summary, the linear regression method for vertical federated learning provided by the present invention is that m (m is greater than or equal to 1) participants respectively construct a column encrypted data matrix with the original data matrix as a submatrix from the original data matrix formed by their own private original data, and send the product of the above column encrypted data matrix and the orthogonal transformation to the first computing node; after the first computing node receives the product of the column encrypted data matrix and the orthogonal transformation sent by the above m participants, it calculates the multiplication encryption result of the square root matrix of the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix of the m participants, that is, the multiplication encryption factor matrix, and the column encrypted data and factor product matrix corresponding to the column encrypted data matrix of the m participants. Then, the label data owner, that is, the m+1th participant, uses the private label column vector to cooperate with the first computing node that owns the multiplication encryption factor matrix of the m participants and the column encrypted data and factor product matrix of the m participants, and calculates the multiplication encryption linear regression coefficient vector, that is, the product of the multiplication encryption mask matrix and the linear regression coefficient vector. Among them, in the process of calculating the multiplication encryption factor matrix of m participants and the column encrypted data and factor product matrix of m participants by the first computing node, the multiplication encryption factor matrix of i-1 participants and the column encrypted data and factor product matrix are used to obtain the multiplication encryption factor matrix of i participants and the column encrypted data and factor product matrix, and thus starting from i=2, i=2, 3, ..., m are iterated and calculated in turn, and finally the multiplication encryption factor matrix of i=m participants and the column encrypted data and factor product matrix are obtained. Compared with the prior art, the present invention can be completed only by collaboration between users who own data, and the implementation is simple; computing and storage nodes (i.e., evaluation parties) can also be introduced. The present invention does not need to introduce a trusted initializer and encryption service provider, nor does it need to implement a very complex confusion circuit, thereby reducing the implementation complexity.

[0082] It should be pointed out that more complex existing encryption technologies can be further adopted on the basis of the embodiments of the present invention to achieve better encryption effects.

[0083] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A linear regression method for vertical federated learning, characterized in that: The method comprises the following steps: The first participant constructs a column encrypted data matrix with the original data matrix as a submatrix according to the original data matrix formed by the private original data, and sends the product of the column encrypted data matrix and an orthogonal transformation to the first computing node; or, there are m participants with their own private original data, wherein a total of m-1 participants from the second participant to the mth participant all construct a column encrypted data matrix with the original data matrix as a submatrix, and send the product of the column encrypted data matrix and an orthogonal transformation to the first computing node; The first computing node cooperates with the tag data owner to calculate and obtain the multiplied encrypted linear regression coefficient vector.

2. The linear regression method for vertical federated learning according to claim 1, characterized in that: The first participant constructs a column encrypted data matrix with the original data matrix as a submatrix according to an original data matrix formed by the private original data, and sends the product of the column encrypted data matrix and an orthogonal transformation to the first computing node, including: The first participant constructs a column encrypted data matrix with the original data matrix as a submatrix from the original data matrix formed by the private original data, and sends the product of the column encrypted data matrix and the orthogonal transformation to the first computing node; after the first computing node receives the product of the column encrypted data matrix and the orthogonal transformation, it calculates and stores the multiplication encryption result of the square root matrix of the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix, that is, the multiplication encryption factor matrix, and the column encrypted data and factor product matrix, wherein the multiplication encryption factor matrix is ​​the product of the square root matrix of the multiplication encryption mask matrix and the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix, and the column encrypted data and factor product matrix is ​​the product of the square root matrix of the column encrypted data matrix and the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix.

3. The linear regression method for vertical federated learning according to claim 1, characterized in that: There are m participants with their own private original data, wherein m-1 participants from the second participant to the mth participant all construct a column encrypted data matrix with the original data matrix as a submatrix, and send the product of the column encrypted data matrix and an orthogonal transformation to the first computing node, including: The m participants respectively construct a column encrypted data matrix with the original data matrix as a submatrix from the original data matrix formed by their own private original data, and send the product of the above column encrypted data matrix and the orthogonal transformation to the first computing node, so that after the first computing node receives the product of the column encrypted data matrix and the orthogonal transformation sent by the m participants respectively, it cooperates with the label data owner to calculate the multiplied encrypted linear regression coefficient vector, where m is an integer greater than or equal to 2.

4. The linear regression method for vertical federated learning according to claim 3, characterized in that: The first computing node cooperates with the tag data owner to calculate the multiplication encrypted linear regression coefficient vector, including: The first computing node calculates and obtains the multiplication encryption result of the square root matrix of the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix of a total of m participants, that is, the multiplication encryption factor matrix, and the column encrypted data and factor product matrix corresponding to the column encrypted data matrix of a total of m participants, wherein the multiplication encryption factor matrix is ​​the product of the square root matrix of the multiplication encryption mask matrix and the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix, and the column encrypted data and factor product matrix is ​​the product of the square root matrix of the column encrypted data matrix and the inverse matrix of the regularized symmetric matrix of the column encrypted data matrix; the label data owner uses the private label column vector to cooperate with the first computing node that owns the multiplication encryption factor matrix of a total of m participants and the column encrypted data and factor product matrix of a total of m participants to calculate the multiplication encryption linear regression coefficient vector, and the multiplication encryption linear regression coefficient vector is the product of the multiplication encryption mask matrix and the linear regression coefficient vector.

5. The linear regression method for vertical federated learning according to claim 3, characterized in that: The method further comprises: In the process of calculating the multiplication encryption factor matrix and the column encrypted data and factor product matrix of a total of m participants by the first computing node, the multiplication encryption factor matrix and the column encrypted data and factor product matrix of a total of i-1 participants are used to obtain the multiplication encryption factor matrix and the column encrypted data and factor product matrix of a total of i participants. Starting from i=2, iterative calculations are performed on i=2, 3, ..., m in turn, and finally the multiplication encryption factor matrix and the column encrypted data and factor product matrix of a total of i=m participants are obtained.

6. The linear regression method for vertical federated learning according to claim 1, characterized in that: The first computing node is composed of a plurality of logical nodes, and each of the plurality of logical nodes stores part of the data.

7. The linear regression method for vertical federated learning according to claim 1, characterized in that: The constraints of the square root matrix obtained by the first participant are: Among them, F1 is the square root matrix, is the transpose or conjugate transpose of F1, Encrypt the data matrix for the first party's columns, for The transpose or conjugate transpose of , λ is a regularization coefficient greater than or equal to zero, and I is the unit matrix; The constraints of the L factor matrix and the D factor matrix obtained by the first participant are: Among them, D1 is the D factor matrix, which is a diagonal matrix; L1 is the L factor matrix, which is a square matrix.

8. The linear regression method for vertical federated learning according to claim 1, characterized in that: The column encrypted data matrix is ​​one of the matrices consisting of the original data matrix of the participants, the original data matrix of the participants and the column-added encryption mask matrix; the column-added encryption mask matrix contains one or more columns.

9. The linear regression method for vertical federated learning according to claim 8, characterized in that: When the column encrypted data matrix of the first participant is a matrix composed of the original data matrix of the first participant and the column-added encrypted mask matrix, after the multiplication encrypted linear regression coefficient vector is obtained by calculation, the method further includes: The label data owner or the first computing node sends the multiplication encrypted linear regression coefficient vector to the first participant or forwards it to the first participant by other participants, so that the first participant can at least partially eliminate the influence of its own column-added encryption mask matrix on the multiplication encrypted linear regression coefficient vector based on its own private multiplication encrypted mask matrix, and update the multiplication encrypted linear regression coefficient vector.

10. The linear regression method for vertical federated learning according to claim 8, characterized in that: When the column encrypted data matrix of the other participants is a matrix composed of the original data matrix of the other participants and the column-added encrypted mask matrix, after the multiplication encrypted linear regression coefficient vector is obtained by calculation, the method further includes: The label data owner or the first computing node sends the multiplication encrypted linear regression coefficient vector to the other participants or forwards it to the other participants by any participant except the other participants, so that the other participants can at least partially eliminate the influence of their own column addition encryption mask matrix on the multiplication encrypted linear regression coefficient vector based on their own private multiplication encryption mask matrix, and update the multiplication encrypted linear regression coefficient vector.

11. The linear regression method for vertical federated learning according to claim 1, 9 or 10, characterized in that: The method further comprises: The label data owner, the first computing node, the first participant or the other participants send several multiplication encrypted linear regression coefficients corresponding to several data features to the corresponding participants, so that the corresponding participants decrypt the received several multiplication encrypted linear regression coefficients according to their own multiplication encrypted mask matrix to obtain the corresponding several linear regression coefficients.

12. The linear regression method for vertical federated learning according to claim 11, characterized in that: The calculation formula of the multiplied encrypted linear regression coefficient vector is: Where m represents the number of parties providing private data; is the label vector, T m is the column encrypted data and factor product matrix received by the tag data owner from the first participant or other participants, P m is the multiplication factor matrix received or calculated by the first computing node; The multiplied encrypted linear regression coefficient vector with the linear regression coefficient vector Satisfy between: Among them, i The multiplication encryption mask matrix for the i-th party providing private data.

Citation Information

Patent Citations

  • Federal learning-based model training method and device, medium and program product

    CN114492850A

  • Linear regression longitudinal federated learning method based on homomorphic encryption

    CN114547643A

  • Data regression processing method, terminal and equipment

    CN115495709A

  • Privacy protection-based linear regression method and system, storage medium and equipment

    CN118195024A

  • Linear regression method based on privacy protection

    CN118313481A

Cited By

  • Linear regression method based on privacy protection

    CN120012038A

  • Privacy preserving linear regression method

    CN120012038B

  • Linear regression method for vertical federated learning

    WO2026032283A1

  • Privacy preserving-based linear regression method

    WO2026148840A1