Linear regression method for vertical federated learning

Abstract

The present application relates to a linear regression method for vertical federated learning, comprising: on the basis of a raw data matrix formed from private raw data, a first participant constructs a column-encrypted data matrix using the raw data matrix as a sub-matrix, and sends the product of the column-encrypted data matrix and an orthogonal transformation to a first computing node; and the first computing node collaborates with a label data owner to calculate a multiplicatively encrypted linear regression coefficient vector. Compared with the prior art, embodiments of the present application do not require a trusted initializer or an encryption service provider, and do not require a garbled circuit having high implementation complexity, thereby reducing implementation complexity.

Description

Linear Regression Methods for Vertical Federated Learning Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a linear regression method for longitudinal federated learning. Background Technology

[0002] The development of machine learning-based artificial intelligence relies heavily on the massive amounts of data supporting its training models. However, the practical application of AI technology across various industries faces numerous challenges, including privacy, security, and efficiency, with data being a crucial factor influencing application development. Privacy concerns regarding training data and trust issues among stakeholders hinder the flow of private data between enterprises, preventing the realization of potential application value and hindering valuable data collaboration, thus restricting their further development.

[0003] Federated learning, as a new branch of artificial intelligence technology, enables data sharing, model training, and the release of data value among multiple participants while ensuring privacy protection and data security. This supports industrial innovation and promotes enterprise and industry development. Federated learning applications can be divided into horizontal federated learning and vertical federated learning. Horizontal federated learning, also known as sample-based federated learning, involves participants possessing different data samples who collaborate to improve the performance of the trained model. These participants may be data-owning enterprises or user clients. Vertical federated learning, also known as feature-based federated learning, involves participants possessing different features of the samples. These participants are typically enterprises and large organizations with different domains and focuses.

[0004] In existing longitudinal federated learning techniques for implementing linear regression, secure multi-party computation requires the use of obfuscated circuits, as well as the introduction of trusted initializers, encryption service providers, and evaluators, making the technical implementation extremely complex. Summary of the Invention

[0005] Therefore, it is necessary to provide a linear regression method for longitudinal federated learning to address the aforementioned technical problems. This method can be implemented simply by collaboration among users who possess the data, or it can involve collaboration between computing and storage nodes (i.e., evaluators). This application does not require the introduction of a trusted initializer and encryption service provider, nor does it require the implementation of highly complex obfuscation circuits, thereby reducing implementation complexity.

[0006] This application provides a linear regression method for longitudinal federated learning, the method comprising the following steps:

[0007] The first participant constructs a column-encrypted data matrix of the original data matrix as a sub-matrix according to the original data matrix formed by the private original data, and sends a product of the column-encrypted data matrix and an orthogonal transformation to the first computing node; or, there are m participants with respective private original data, and m-1 participants from the second participant to the mth participant all construct a column-encrypted data matrix of the original data matrix as a sub-matrix, and send a product of the column-encrypted data matrix and an orthogonal transformation to the first computing node.

[0008] The first computing node cooperates with the label data owner to calculate the multiplication-encryption linear regression coefficient vector.

[0009] Further, the first participant constructs a column-encrypted data matrix of the original data matrix as a sub-matrix according to the original data matrix formed by the private original data, and sends a product of the column-encrypted data matrix and an orthogonal transformation to the first computing node, including:

[0010] The first participant constructs a column-encrypted data matrix of the original data matrix as a sub-matrix according to the original data matrix formed by the private original data, and sends a product of the column-encrypted data matrix and an orthogonal transformation to the first computing node; after the first computing node receives the product of the column-encrypted data matrix and the orthogonal transformation, the first computing node calculates and stores a multiplication-encryption result of a square root matrix of an inverse matrix of a regular symmetric matrix of the column-encrypted data matrix, that is, a multiplication-encryption factor matrix, and a column-encrypted data and factor product matrix, wherein the multiplication-encryption factor matrix is a product of a multiplication-encryption mask matrix and the square root matrix of the inverse matrix of the regular symmetric matrix of the column-encrypted data matrix, and the column-encrypted data and factor product matrix is a product of the column-encrypted data matrix and the square root matrix of the inverse matrix of the regular symmetric matrix of the column-encrypted data matrix.

[0011] Further, the m participants with respective private original data, and m-1 participants from the second participant to the mth participant all construct a column-encrypted data matrix of the original data matrix as a sub-matrix, and send a product of the column-encrypted data matrix and an orthogonal transformation to the first computing node, including:

[0012] The m participants with respective private original data construct a column-encrypted data matrix of the original data matrix as a sub-matrix according to the original data matrix formed by the private original data, and send a product of the column-encrypted data matrix and an orthogonal transformation to the first computing node, so that, after the first computing node receives the product of the column-encrypted data matrix and the orthogonal transformation sent by the m participants respectively, the first computing node cooperates with the label data owner to calculate the multiplication-encryption linear regression coefficient vector, wherein m is an integer greater than or equal to 2.

[0013] Further, the first computing node cooperates with the label data owner to calculate a multiplication-encryption linear regression coefficient vector, comprising:

[0014] The first computing node calculates a multiplication-encryption result of a square root matrix of an inverse matrix of a regularization symmetric matrix of a column-encryption data matrix of the m participants, i.e., a multiplication-encryption factor matrix, and a column-encryption data and factor product matrix corresponding to the column-encryption data matrix of the m participants, wherein the multiplication-encryption factor matrix is a product of a multiplication-encryption mask matrix and the square root matrix of the inverse matrix of the regularization symmetric matrix of the column-encryption data matrix, and the column-encryption data and factor product matrix is a product of the column-encryption data matrix and the square root matrix of the inverse matrix of the regularization symmetric matrix of the column-encryption data matrix; the label data owner cooperates with the first computing node that owns the multiplication-encryption factor matrix of the m participants and the column-encryption data and factor product matrix of the m participants by using a private label column vector to calculate a multiplication-encryption linear regression coefficient vector, which is a product of a multiplication-encryption mask matrix and a linear regression coefficient vector.

[0015] Further, the method further comprises:

[0016] In the process of calculating the multiplication-encryption factor matrix of the m participants and the column-encryption data and factor product matrix of the m participants by the first computing node, the multiplication-encryption factor matrix and the column-encryption data and factor product matrix of the i participants are obtained by using the multiplication-encryption factor matrix and the column-encryption data and factor product matrix of the i-1 participants, and then the iteration calculation is performed on i=2, 3, …, m in sequence from i=2, and finally the multiplication-encryption factor matrix and the column-encryption data and factor product matrix of the i=m participants are obtained.

[0017] Further, the first computing node is composed of a plurality of logical nodes, and each of the plurality of logical nodes stores part of data.

[0018] Further, the constraint of the square root matrix obtained by the first participant is:

[0019] wherein F1 is the square root matrix, is a transpose or a conjugate transpose of F1, is a column-encryption data matrix of the first participant, is a transpose or a conjugate transpose of , and λ is a regularization coefficient greater than or equal to zero, and I is a unit matrix.

[0020] The constraint of the L factor matrix and the D factor matrix obtained by the first participant is:

[0021] Wherein, D1 is a D factor matrix, which is a diagonal matrix; L1 is an L factor matrix, which is a square matrix.

[0022] Further, the column-encrypted data matrix is one of the original data matrix of the participant, the original data matrix of the participant and the augmented column-encrypted mask matrix; the augmented column-encrypted mask matrix contains one column or multiple columns.

[0023] Further, when the column-encrypted data matrix of the first participant is the matrix composed of the original data matrix of the first participant and the augmented column-encrypted mask matrix, after the multiplication-encrypted linear regression coefficient vector is obtained, the method further comprises:

[0024] The label data owner or the first computing node sends the multiplication-encrypted linear regression coefficient vector to the first participant or forwards it to the first participant by other participants, so that the first participant at least partially eliminates the influence of the augmented column-encrypted mask matrix of the first participant on the multiplication-encrypted linear regression coefficient vector according to the multiplication-encrypted mask matrix private to the first participant, and updates the multiplication-encrypted linear regression coefficient vector.

[0025] Further, when the column-encrypted data matrix of the other participant is the matrix composed of the original data matrix of the other participant and the augmented column-encrypted mask matrix, after the multiplication-encrypted linear regression coefficient vector is obtained, the method further comprises:

[0026] The label data owner or the first computing node sends the multiplication-encrypted linear regression coefficient vector to the other participant or forwards it to the other participant by any participant except the other participant, so that the other participant at least partially eliminates the influence of the augmented column-encrypted mask matrix of the other participant on the multiplication-encrypted linear regression coefficient vector according to the multiplication-encrypted mask matrix private to the other participant, and updates the multiplication-encrypted linear regression coefficient vector.

[0027] Further, the method further comprises:

[0028] The label data owner, the first computing node, the first participant or the other participant sends a plurality of multiplication-encrypted linear regression coefficients corresponding to a plurality of data features to the corresponding participant, so that the corresponding participant decrypts the plurality of multiplication-encrypted linear regression coefficients received according to the multiplication-encrypted mask matrix of the corresponding participant to obtain a plurality of linear regression coefficients corresponding to the plurality of data features.

[0029] Further, the calculation formula of the multiplication-encrypted linear regression coefficient vector is:

[0030] Wherein, m represents the number of participants providing private data; is a label vector, Tm is a column encryption data and factor product matrix received by the label data owner from the first participant or the other participants, P m is a multiplication encryption factor matrix received or calculated by the first computing node;

[0031] the multiplication encryption linear regression coefficient vector and the linear regression coefficient vector satisfy:

[0032] wherein Ξ i is a multiplication encryption mask matrix of the ith participant providing private data.

[0033] The linear regression method for vertical federated learning provided in the above embodiment can be implemented in the following way. The first participant constructs an original data matrix as a column encryption data matrix of a submatrix according to an original data matrix formed by private original data, and sends the column encryption data matrix and a product of an orthogonal transformation to the first computing node. Alternatively, there are m participants each having private original data, and the original data matrix is constructed as a column encryption data matrix of a submatrix by the m-1 participants from the second participant to the mth participant, and the column encryption data matrix and a product of an orthogonal transformation are sent to the first computing node. Then, the first computing node cooperates with the label data owner to calculate a multiplication encryption linear regression coefficient vector. Compared with the prior art, the embodiment of the present application can be completed only by cooperation between users who own data, and is simple to implement. The embodiment of the present application can also introduce a computing and storage node (i.e., an evaluator). The present application does not need to introduce a trusted initializer and an encryption service provider, and does not need to implement a complex garbled circuit, thereby reducing the implementation complexity. BRIEF DESCRIPTION OF DRAWINGS

[0034] FIG. 1 is a flowchart of a linear regression method for vertical federated learning provided by an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0036] It should be noted that the step numbers in the text are only for the convenience of explanation of the specific embodiments, and do not serve as the function of limiting the execution sequence of the steps. The method provided in the embodiment can be executed by a related server, and the server is taken as an example for description in the following.

[0037] As shown in FIG. 1, the linear regression method for vertical federated learning provided by Embodiment One of the present application comprises steps S11 to S12:

[0038] Step S11: The first participant constructs a column-encrypted data matrix of the original data matrix formed by the private original data as a submatrix and sends the product of the column-encrypted data matrix and an orthogonal transformation to the first computing node; or, there are m participants with respective private original data, wherein m-1 participants from the second participant to the mth participant all construct a column-encrypted data matrix of the original data matrix as a submatrix and send the product of the column-encrypted data matrix and an orthogonal transformation to the first computing node.

[0039] Step S12: The first computing node cooperates with the label data owner to calculate the multiplication-encrypted linear regression coefficient vector.

[0040] The first participant constructs a column-encrypted data matrix of the original data matrix formed by the private original data as a submatrix and sends the product of the column-encrypted data matrix and an orthogonal transformation to the first computing node; after receiving the product of the column-encrypted data matrix and the orthogonal transformation, the first computing node calculates and stores the multiplication-encrypted result of the square root matrix of the inverse matrix of the regularized symmetric matrix of the column-encrypted data matrix, i.e., the multiplication-encrypted factor matrix, and the column-encrypted data and factor product matrix. The multiplication-encrypted factor matrix is the product of the multiplication-encrypted mask matrix and the square root matrix of the inverse matrix of the regularized symmetric matrix of the column-encrypted data matrix, and the column-encrypted data and factor product matrix is the product of the column-encrypted data matrix and the square root matrix of the inverse matrix of the regularized symmetric matrix of the column-encrypted data matrix.

[0041] More generally, m (m is greater than or equal to 1) participants respectively construct a column-encrypted data matrix of the original data matrix formed by the respective private original data as a submatrix and send the product of the column-encrypted data matrix and an orthogonal transformation to the first computing node; after receiving the product of the column-encrypted data matrix and the orthogonal transformation sent by the m participants respectively, the first computing node calculates the multiplication-encrypted result of the square root matrix of the inverse matrix of the regularized symmetric matrix of the column-encrypted data matrix of the m participants, i.e., the multiplication-encrypted factor matrix, and the column-encrypted data and factor product matrix corresponding to the column-encrypted data matrix of the m participants.

[0042] Then, the label data owner, i.e., the m+1th participant, cooperates with the first computing node that has the multiplication-encrypted factor matrix of the m participants and the column-encrypted data and factor product matrix of the m participants to calculate the multiplication-encrypted linear regression coefficient vector, i.e., the product of the multiplication-encrypted mask matrix and the linear regression coefficient vector.

[0043] In the process of calculating the multiplication encryption factor matrix of the co-i participants and the column encryption data and factor product matrix of the co-i participants, the multiplication encryption factor matrix of the co-i participants and the column encryption data and factor product matrix of the co-i participants are obtained by using the multiplication encryption factor matrix of the co-i-1 participants and the column encryption data and factor product matrix of the co-i-1 participants. Thus, starting from i = 2, the iteration calculation is sequentially performed for i = 2, 3,..., m, and finally the multiplication encryption factor matrix of the co-i = m participants and the column encryption data and factor product matrix of the co-i = m participants are obtained.

[0044] The technical solution of the present application comprises step S11, and step S11 further comprises steps S111 and S112, which are introduced as follows. First, the sub-steps S111-a to S111-e included in step S111 in step S11 are introduced:

[0045] Sub-step S111-a: This sub-step is an optional sub-step of adding a mask column. In this sub-step, the first participant adds one or more columns of masks behind the original data matrix X1 of the first participant, which is an optional step. If the first participant wants to achieve better privacy protection effect, this step is selected. Usually, 1 column of mask vectors can achieve the privacy protection effect, and the first participant can also use multiple columns of masks to achieve better privacy protection effect by increasing the number of columns of masks. The first participant generates the data matrix with added mask columns , which is also referred to as a column encryption data matrix in the following

[0046] Specifically, the column encryption data matrix, i.e., the data matrix with added mask columns , is generated by the first participant according to the original data matrix X1 composed of private original data and the encryption mask matrix Δ1 with added columns. The encryption mask matrix Δ1 with added columns can be generated by the first participant, and the encryption mask matrix Δ1 with added columns is usually randomly generated. , which means that the encryption mask matrix Δ1 with added columns is added behind the original data matrix X1 with τ1 columns to obtain the column encryption data matrix with columns. Note that may be equal to 0, which means that the first participant does not add the encryption mask matrix Δ1 with added columns to encrypt the original data. In addition, usually is equal to 1 to achieve good encryption effect, and at this time, there are The column-adding encryption mask matrix Δ1 is actually a column vector of the column-adding encryption mask with only one column. If the first participant does not choose this optional column-adding mask step, then the following steps will be used... Note that in the above description, "having" is used. The column-adding encryption mask matrix Δ1 is placed after the original data matrix X1 for simplicity; in actual implementation, the column-adding encryption mask matrix Δ1... A column can be a shared column. Column-wise encrypted data matrix any position in Columns can also be separate, i.e., discontinuous. List.

[0047] Sub-step S111-b: In this sub-step, the first participant uses its own original data matrix X1 as the matrix of sub-matrices (i.e., the column-encrypted data matrix). Multiply by an orthogonal transformation Ω1 to obtain Then Send to the first computing node.

[0048] Sub-step S111-c: In this sub-step, the first computing node receives the message sent by the first participant. The corresponding multiplicative encrypted square root matrix P1 is obtained, wherein the multiplicative encrypted square root matrix P1 satisfies Here, I represents the identity matrix, and λ represents a scalar called the regularization coefficient. Note that the first computation node can be right-multiplied by P1 using any orthogonal matrix Θ1 to obtain the updated P1 = P1Θ1. The multiplication by the encrypted square root matrix P1 is performed on the matrix satisfying... Multiplication encryption is performed on the square root matrix F1, i.e. The square root matrix F1 is a matrix that includes the original data matrix X1 of the first participant as a submatrix (i.e., a column-encrypted data matrix). Regularized symmetric matrix inverse matrix The square root matrix F1.

[0049] Sub-step S111-d: The first computing node utilizes the data received from the first participant. And using the square root matrix P1 obtained in the previous sub-step to multiply the encrypted data, we obtain the product T1 of the column encrypted data and the square root matrix. This is then calculated... T1 is an intermediate variable, which is a column-encrypted data matrix containing the original data matrix X1 of the first participant as a submatrix. Encrypted data matrix with columns The product T1 of the square root matrix F1 of the inverse of the regularized symmetric matrix, i.e., T1 satisfies Note that in this embodiment, although the first computing node does not have a column encrypted data matrix. The square root matrix F1, but it can be calculated Satisfaction T1. In the following text, the first participant's column encryption data matrix... The product T1 of the square root matrix F1 is simply referred to as the column encrypted data and factor product matrix T1 of the first participant, also known as the intermediate variable T1.

[0050] In this application, (·) T This represents the transpose of a real matrix (including vectors), or the conjugate transpose of a complex matrix (including vectors), as described above. In and In Putting two matrices together, or a scalar and a matrix together, indicates multiplication. For example... middle express take And λI represents λ multiplied by I. Note that this is a regularized symmetric matrix. In this context, λI is a matrix related to regularization. In some cases, λI can take the form of... λI is a diagonal matrix, where the terms on the diagonal can take different values; even λI can be any matrix.

[0051] Sub-step S111-e: The control variable i corresponding to the first participant (first data owner) is equal to 1. In this sub-step, the value of i is first incremented by 1; then, if the value of i is greater than m, proceed to step S12; if the value of i is less than or equal to m, proceed to step S112. As mentioned earlier, vertical federated learning has m participants who own their own private original data, also called m data owners, where m is an integer greater than or equal to 1; in addition, there is a first computing node and a (m+1)th participant called the label owner. This step is explained as follows: The value of m determines the characteristics of the aforementioned second participant, and also determines which step to proceed to. When m equals 1, then the aforementioned second participant is the label data owner, and proceed to step S12 accordingly; when m is greater than or equal to 2, then the aforementioned second participant is the second data owner, and proceed to step S112 accordingly.

[0052] The sub-steps S111-a to S111-e included in the step S111 in the step S11 can also have a second implementation manner, and the corresponding main principle is that, although in the scenario with the first computing node, the computing task is usually arranged in the first computing node in priority, but for the scenario of the first participant, the computing task can also be arranged in the first participant. Correspondingly, the second implementation manner of the sub-steps S111-a to S111-e included in the step S111 in the step S11 is introduced as follows.

[0053] The sub-step S111-a of the second implementation manner is an optional sub-step of adding a mask column. In this sub-step, the first participant adds one or more columns of masks behind the original data matrix X1 of the first participant, which is an optional step. If the first participant wants to achieve better privacy protection effect, this step is selected. Usually, 1 column of mask vector can achieve the privacy protection effect, and the first participant can also use multiple columns of masks to achieve better privacy protection effect by increasing the number of columns of masks. The first participant generates the data matrix with added mask columns which is also referred to as column-encrypted data matrix in the following in particular, the column-encrypted data matrix is the data matrix with added mask columns where the original data matrix X1 of the first participant has τ1 columns containing τ1 original data features, and the column-encrypted mask matrix Δ1 has columns which can be generated by the first participant. Usually, the column-encrypted mask matrix Δ1 is randomly generated.

[0054] The sub-step S111-b of the second implementation manner is that the first participant uses the original data matrix X1 containing the original data of the first participant as a matrix (i.e., the column-encrypted data matrix to obtain the corresponding multiplication-encryption square root matrix P1. There can be multiple calculation methods for obtaining P1. Two different calculation methods are listed as follows: the first calculation method is to multiply the column-encrypted data matrix by an orthogonal transformation Ω1 to obtain and then obtain the multiplication-encryption square root matrix P1 that satisfies The second calculation method is to calculate the square root matrix F1 that satisfies and then multiply the square root matrix F1 by the multiplication-encryption mask matrix Ξ1 from the left to obtain the multiplication-encryption factor matrix P1=Ξ1F1. It is noted that the first computing node can multiply P1 by an arbitrary orthogonal matrix Θ1 from the right to obtain the updated P1=P1Θ1.

[0055] The sub-step S111-c of the second implementation manner: in this sub-step, the first participant obtains the column-encrypted data and the square root matrix product T1. There are multiple calculation methods for obtaining T1, and two different calculation methods are listed as follows: the first calculation method is to calculate the column-encrypted data matrix and P1, and then calculate The second calculation method is to calculate the column-encrypted data matrix and the square root matrix F1 obtained in the second calculation method of the sub-step S111-b, and then calculate

[0056] The sub-step S111-d of the second implementation manner: the first participant sends the multiplied encrypted square root matrix P1 and the column-encrypted data and the square root matrix product T1 to the first computing node.

[0057] In this application, the concatenation matrix of the column-encrypted data of the i (i is less than or equal to m) participants 1, 2, …, i is denoted as , which is referred to as the column-encrypted data matrix of the i participants, and it is noted that the column-encrypted data matrix of the j participant (j is greater than or equal to 1 and less than or equal to i) in the column-encrypted data matrix of the i participants is the original data matrix X composed of the original data private to the j participant. J The column-encrypted data matrix of the i participants is obtained by adding the column-encrypted mask matrix Δ j to the column-encrypted data matrix of the i participants, and the column-encrypted mask matrix Δ j may also not be added to the column-encrypted data matrix of the i participants. Therefore, the column-encrypted data matrix of the i participants may also be denoted as , that is, the column-encrypted data matrix of the i participants. This denotation method also covers the aforementioned step S111.

[0058] In the step S112 included in the step S11, the first computing node receives the column-encrypted data matrix of the i participant (i is greater than or equal to 2 and less than or equal to m), that is, the i data owner, and calculates the column-encrypted data matrix of the i participants (notice that Δ i may be an empty matrix, as a special case) and the orthogonal transformation Ω i , that is, The first computing node calculates the column-encrypted data matrix of the i participants by using the received column-encrypted data matrix of the i participant i-1 and the column-encrypted data and the square root matrix product T i-1 of the i-1 participants stored by the first computing node, and the multiplied encrypted factor matrix P i . iStep S112, which is included in step S11 above, includes the following sub-steps S112-a to S112-f, a total of 6 sub-steps:

[0059] Sub-step S112-a: This sub-step is optional and involves adding a mask column. In this sub-step, the data matrix X is... i The i-th participant in its own data matrix X i Adding one or more columns of masking is an optional step. The data matrix X constructed by the i-th participant based on its private original data... i and the addition of an encryption mask matrix Δ i The generated data matrix with added mask columns This will also be referred to as column-encrypted data matrix for short.

[0060] Specifically, a column-encrypted data matrix is ​​a data matrix with added mask columns. Where the data matrix X of the i-th participant i There is τ i The column contains τ i The original data of each feature, and the added encryption mask matrix Δ i have The column can be generated by the i-th participant, usually by adding an encryption mask matrix Δ. i It is randomly generated. Indicates that there is τ i Column data matrix X i Added later Column-adding encryption mask matrix Δ i To obtain shared ownership Column-wise encrypted data matrix in It can be equal to 0, which is equivalent to the i-th participant not adding an additional column to the encryption mask matrix Δ. i To encrypt the original data; in addition, it is usually made A value of 1 provides excellent encryption; in this case, there is... Column-adding encryption mask matrix Δ i In reality, it's a single-column incrementing encryption mask column vector. If the i-th participant doesn't choose this optional incrementing mask column step, then the following steps will be used... Note that in the above description, "having" is used. Column-adding encryption mask matrix Δ i Placed in the original data matrix X i The following is for simplicity of description; in actual implementation, an additional encryption mask matrix Δ is added. i of A column can have a total of τ i + Column-encrypted data matrix Any position in the matrix Column, which can also be separate, i.e. discontinuous Column.

[0061] Sub-step S112-b: In this sub-step, the i-th participant uses the original data matrix X i Matrix as a sub-matrix (i.e. column-encrypted data matrix) Multiplies an orthogonal transformation Ω i Obtains And then Sends to the first computing node.

[0062] Sub-step S112-c: In this sub-step, the first computing node receives Calculates the square root matrix F i of the i participants i Multiplies the obtained matrix to obtain a matrix that satisfies The matrix Calculates the square root matrix that satisfies The matrix In this step, an orthogonal matrix Θ i Right multiplies To update To

[0063] The above sub-matrix G i is a sub-matrix of the square root matrix F i of the i participants, as shown in , where × represents a sub-matrix that is not currently of interest. The matrix G i satisfies From this, it is easy to see that the sub-matrix G i in the square root matrix F i is also the inverse matrix of the square root matrix of the regular symmetric matrix The above multiplication of G i to obtain the matrix satisfies That is, is an orthogonal matrix Multiplied by the result of G i

[0064] Sub-step S112-d: In this sub-step, the first computing node calculates the column-encrypted data and factor product matrix T i of the i participants, while in fact the first computing node only needs to calculate T i ​The product T of the encrypted column data of i-1 participants and the square root matrix. i-1 Several additional columns were added. Specifically, the first computation node calculates... Only T i relative to T i-1 The added columns, namely It needs to be calculated.

[0065] As mentioned earlier, T i It is a column-encrypted data matrix with i participants. With the square root matrix F i The product of, i.e. And the square root matrix F i It is a column-encrypted data matrix with i participants. Regularized symmetric matrix inverse matrix The square root matrix, i.e., the square root matrix F i satisfy Note that T i It is by and F i As defined, while the first computing node does not have the above-mentioned... and F i T can still be calculated i .

[0066] Sub-step S112-e: In this sub-step, the first computation node sets P i-1 Updated to P i As mentioned earlier, P i It is the square root matrix F of i participants. i The left side is multiplied by a matrix and then encrypted to obtain the encryption factor matrix, while the square root matrix F of the i participants is... i satisfy A specific implementation method could be that the first computing node obtains the result through calculation. In P i-1 Add multiple columns to the right of In P i-1 Add a submatrix consisting entirely of zeros directly below P. i-1 Add a submatrix to the lower right. This allows the encryption factor matrix P to be multiplied by the i-1 participating parties. i-1 Updated to a encryption factor matrix P multiplied by i participants. i .

[0067] As mentioned earlier, matrix F is the square root matrix of i participants.i submatrix G in i The matrix obtained by multiplication and encryption, matrix satisfy Right now It is an orthogonal matrix Multiplier matrix G i The result obtained. Because only participant i knows the orthogonal matrix Ω. i That is, only participant i knows the orthogonal matrix. Therefore, the first computation node does not know the orthogonal matrix. Therefore, it is impossible to obtain from the matrix The square root matrix F of the i participants is derived. i submatrix G in i Thus, for the first computation node, the submatrix G is achieved. i To achieve the effect of encryption. In other embodiments, multiplying the encryption matrix is ​​typically done using Ξ. i As can be seen from the above description, in this embodiment, the encryption matrix is ​​multiplied... Accordingly, as mentioned earlier, multiplying the encryption factor matrix Substitute it into It can be obtained

[0068] Sub-step S112-f (iteration control sub-step): If the value of i is less than or equal to m-1, increment the value of i by 1 and return to sub-step S112-a to start the next iteration; otherwise, when the value of i is equal to m, proceed to step S12 below. Note that in the above steps of this embodiment, there are m participants, including participants 1, 2, ... m, each using their own original data matrix X. i A matrix that serves as a submatrix (i.e., a column-encrypted data matrix). Multiply by an orthogonal transformation Ω i get Then Send to the first compute node; the above sending... (i = 1, 2, ..., m), which have a specific order in the description, but no specific order in the actual implementation. In other words, in the description... Sending in Before sending; during implementation, The sending can be done After the transmission, furthermore, the sequence numbers of the aforementioned participants 1, 2, ... m can be not pre-set, but rather each of the m participants will send their respective matrices. The message is sent to the first computing node, which then arbitrarily assigns the aforementioned m participants as participant 1, 2, ..., m.

[0069] The steps for calculating the encrypted linear regression coefficient vector in this embodiment are as follows: The tag data owner, i.e., the (m+1)th participant, uses a private tag column vector. Multiply the encryption factor matrix P with a total of m participants m The encrypted data and factor product matrix T of m participating parties. m The first computing node collaborates to compute the multiplied encrypted linear regression coefficient vector. That is, multiplying the encryption mask matrix by the linear regression coefficient vector. A specific implementation could be that the first computing node multiplies P... m and T m It is sent to the tag data owner, who then calculates the result.

[0070] The above-mentioned multiplicative encrypted linear regression coefficient vector is calculated. Subsequently, the effect achieved by the following steps in the above embodiments of this application is: if there are several participants from participant 1 to m that use an incrementing encryption mask matrix with non-zero columns, then the several participants sequentially multiply the encryption factor matrix P. m Multiplication and encryption of linear regression coefficient vectors This at least partially eliminates the influence of its incrementing encryption mask matrix Δ. When each participant using a non-zero column incrementing encryption mask matrix multiplies the encryption linear regression coefficient vector... This at least partially eliminates the impact of the participant's added encryption mask matrix Δ, or, if there is no participant using a non-zero column added encryption mask matrix, then it possesses the most recently updated multiplied encryption linear regression coefficient vector. The participants or the first computing node, Several items corresponding to participant i (where i is greater than or equal to 1 and less than or equal to m) are sent to participant i, who then uses its own private multiplication encryption mask matrix Ξ i Decryption yields several terms corresponding to the linear regression coefficient vector. One implementation method to eliminate the influence of the added encryption mask matrix can exclude the first computation node; correspondingly, it requires multiplying by the encryption factor matrix P = P m Multiplication and encryption of linear regression coefficient vectors Send it to the i-th participant, where i is a positive integer less than or equal to m-1, or i is a positive integer less than or equal to m. There are also other implementation methods for the step of eliminating the influence of the added-column encryption mask matrix, where P... m and At least one of them is stored on the first computing node.

[0071] The first computing node can be independent of the m+1 participants or can be any one of the m+1 participants. The first computing node can be a Crypto Service Provider (CSP) or an Evaluator as often mentioned in the literature of federated learning, or the first computing node can include both a CSP and an Evaluator. The first computing node can perform computation and storage, and can also provide encryption services. The first computing node is a logical node, and it is known that the physical node corresponding to the first computing node can be located in any one of the m+1 participants. In addition, the first computing node can include multiple logical nodes, for example, the first computing node can include both a CSP and an Evaluator, and the two physical nodes corresponding to the CSP and the Evaluator can be located in two of the m+1 participants, respectively.

[0072] In the case where the first computing node includes at least two logical nodes, the i-th participant (i is greater than or equal to 1 and less than or equal to m) sends to the first computing node and the first computing node receives The sending and receiving are equivalent, for example, one logical node included in the first computing node sends a matrix of size Ψ i to the i-th participant, and the i-th participant sends Ψ i to another logical node included in the first computing node. It can be seen that one logical node included in the first computing node knows i Ψ i , and another logical node included in the first computing node receives Ψ i . As long as the two logical nodes cooperate, the result of Ψ i can be obtained by subtracting or adding i Ψ from or to the result of Ψ i . The cooperation of the two logical nodes can use existing multi-party secure computation methods.

[0073] Based on the implementation of square root decomposition of the inverse matrix of the regular symmetric matrix of the column-encrypted data matrix of the first participant in the embodiments of the present application, another commonly used LDL T decomposition implementation can be easily obtained. For example, the matrix F satisfying FF T = (X T X+ λI) -1 is the inverse matrix (X T X+ λI)-1 The square root matrix can be viewed as a symmetric matrix (X). T X+λI) -1 One method of factorization matrix, the corresponding symmetric matrix can also be derived using another commonly used LDL method. T LDL in decomposition T Factor matrix replacement, and LDL T The factor matrix includes a triangular L-factor matrix and a diagonal D-factor matrix. The L-factor matrix can be extended to a general square matrix; that is, the L-factor matrix can be either triangular or not. The D-factor matrix remains a diagonal matrix. Based on this extension, LDL... T The factor matrix can be slightly modified from the embodiments of this application to obtain an LDL-based model. T Specific implementation methods for factoring matrices.

[0074] The factorization matrix of the inverse of the regularized symmetric matrix of the column-encrypted data matrix used in the embodiment is the square root matrix F. i And satisfy For i greater than or equal to 1 and less than or equal to m, there clearly exists another LDL as a factor matrix. T Factor matrix, i.e., L i and D i satisfy Where the D factor matrix D i It is a diagonal matrix, while L is a factor matrix. i It is a general square matrix, which can be a triangular matrix or a non-triangular matrix. In practical applications, the above L is obtained. i and D i A simple method is to use matrix F in Example 2. i Let L be the value of L. i L i =F i And let D i Since it equals the identity matrix, the implementation example becomes based on the factorization matrix L. i and D i Examples of implementations. More generally, D i It may not be equal to the identity matrix. It can be based on... The derivation yields and in It is a diagonal matrix, and each element on its diagonal is D. i The square root of the term in the same position on the diagonal, and yes The inverse matrix, where each element on the diagonal is The inverse of the diagonal entries. Correspondingly, a more general LDL decomposition based on the inverse of the diagonal entries of the Cholesky factor matrix can be derived. T Embodiments of the factor matrix are not described again here.

[0075] It should be understood that, although the steps in the above flowcharts are shown in sequence according to the directions of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least some of the steps in the above flowcharts can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be alternated or rotated with at least part of other steps or sub-steps or stages of other steps.

[0076] In summary, the linear regression method for vertical federated learning provided in the present application is as follows: m (m is greater than or equal to 1) participants respectively form an original data matrix from their own private original data, construct a column-encrypted data matrix taking the original data matrix as a sub-matrix, and send the product of the column-encrypted data matrix and an orthogonal transformation to a first computing node; after receiving the products of the column-encrypted data matrix and the orthogonal transformation sent by the m participants respectively, the first computing node calculates the square root matrix of the inverse matrix of the regularization symmetric matrix of the column-encrypted data matrix of the m participants, the multiplication and encryption result of the square root matrix, i.e., the multiplication and encryption factor matrix, and the column-encrypted data and factor product matrix corresponding to the column-encrypted data matrix of the m participants. Then, the label data owner, i.e., the m+1th participant, cooperates with the first computing node that owns the multiplication and encryption factor matrix of the m participants and the column-encrypted data and factor product matrix of the m participants to calculate the multiplication and encryption linear regression coefficient vector, i.e., the product of the multiplication and encryption mask matrix and the linear regression coefficient vector. In the process of calculating the multiplication and encryption factor matrix and the column-encrypted data and factor product matrix of the m participants by the first computing node, the multiplication and encryption factor matrix and the column-encrypted data and factor product matrix of the i participants are obtained by using the multiplication and encryption factor matrix and the column-encrypted data and factor product matrix of the i-1 participants. Thus, starting from i=2, the iteration calculation is sequentially performed for i=2, 3, …, m, and finally the multiplication and encryption factor matrix and the column-encrypted data and factor product matrix of the i=m participants are obtained. Compared with the prior art, the present application can be completed by cooperation between users who own data, and is simple to implement; or a computing and storage node (i.e., an evaluator) can be introduced. The present application does not need to introduce a trusted initializer and an encryption service provider, and does not need to implement a complex garbled circuit, thereby reducing the implementation complexity.

[0077] It should be noted that further more complex existing encryption technology can be used on the basis of the embodiments of the present application to achieve better encryption effect.

[0078] The above only describes the preferred embodiments of the present application, and it should be noted that those skilled in the art can make several improvements and modifications without departing from the technical principles of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A linear regression method for federated learning in the longitudinal direction, characterized in that, The method comprises the following steps: The first participant constructs the original data matrix formed by the private original data as a column encryption data matrix of sub-matrix, and sends the product of the column encryption data matrix and an orthogonal transformation to the first computing node; or, there are m participants with respective private original data, and m-1 participants from the second participant to the mth participant all construct the original data matrix as a column encryption data matrix of sub-matrix, and send the product of the column encryption data matrix and an orthogonal transformation to the first computing node. The first computing node cooperates with the label data owner to calculate a multiplication encryption linear regression coefficient vector.

2. The linear regression method for federated learning in the vertical direction according to claim 1, wherein, The first participant constructs the original data matrix formed by the private original data as a column encryption data matrix of sub-matrix, and sends the product of the column encryption data matrix and an orthogonal transformation to the first computing node, comprising: The first participant constructs the original data matrix formed by the private original data as a column encryption data matrix of sub-matrix, and sends the product of the column encryption data matrix and an orthogonal transformation to the first computing node; after the first computing node receives the product of the column encryption data matrix and the orthogonal transformation, the first computing node calculates and stores the multiplication encryption factor matrix of the square root matrix of the inverse matrix of the regularization symmetric matrix of the column encryption data matrix, and the column encryption data and factor product matrix, wherein the multiplication encryption factor matrix is the product of the multiplication encryption mask matrix and the square root matrix of the inverse matrix of the regularization symmetric matrix of the column encryption data matrix, and the column encryption data and factor product matrix is the product of the column encryption data matrix and the square root matrix of the inverse matrix of the regularization symmetric matrix of the column encryption data matrix.

3. The linear regression method for federated learning in the vertical direction according to claim 1, wherein, The first participant constructs the original data matrix formed by the private original data as a column encryption data matrix of sub-matrix, and sends the product of the column encryption data matrix and an orthogonal transformation to the first computing node, comprising: The first participant constructs the original data matrix formed by the private original data as a column encryption data matrix of sub-matrix, and sends the product of the column encryption data matrix and an orthogonal transformation to the first computing node, comprising:

4. The linear regression method for federated learning in the vertical direction according to claim 3, wherein, The first participant constructs the original data matrix formed by the private original data as a column encryption data matrix of sub-matrix, and sends the product of the column encryption data matrix and an orthogonal transformation to the first computing node, comprising: The first participant constructs the original data matrix formed by the private original data as a column encryption data matrix of sub-matrix, and sends the product of the column encryption data matrix and an orthogonal transformation to the first computing node, comprising: The first computing node calculates a multiplication encryption result of a square root matrix of an inverse matrix of a regular symmetric matrix of a column encryption data matrix of the m participants, that is, a multiplication encryption factor matrix, and a column encryption data and factor product matrix corresponding to the column encryption data matrix of the m participants, wherein the multiplication encryption factor matrix is a product of a multiplication encryption mask matrix and the square root matrix of the inverse matrix of the regular symmetric matrix of the column encryption data matrix, and the column encryption data and factor product matrix is a product of the column encryption data matrix and the square root matrix of the inverse matrix of the regular symmetric matrix of the column encryption data matrix; the label data owner cooperates with the first computing node that owns the multiplication encryption factor matrix of the m participants and the column encryption data and factor product matrix of the m participants to calculate a multiplication encryption linear regression coefficient vector, which is a product of the multiplication encryption mask matrix and a linear regression coefficient vector.

5. The linear regression method for federated learning in the vertical direction according to claim 3, wherein, The method further comprises: In the process of calculating the multiplication encryption factor matrix of the m participants and the column encryption data and factor product matrix of the m participants by the first computing node, the multiplication encryption factor matrix and the column encryption data and factor product matrix of i participants are obtained by using the multiplication encryption factor matrix and the column encryption data and factor product matrix of i-1 participants, and thus iterative calculation is performed on i=2, 3, …, m in sequence, and finally the multiplication encryption factor matrix and the column encryption data and factor product matrix of i=m participants are obtained.

6. The linear regression method for federated learning in the vertical direction according to claim 1, wherein, The first computing node is composed of a plurality of logical nodes, and the plurality of logical nodes each store part of data.

7. The linear regression method for federated learning in the vertical direction according to claim 1, wherein, The constraint on the square root matrix acquired by the first participant is: wherein F1 is a square root matrix, transpose or conjugate transpose of F1, encrypting the column data matrix for the first participant, For is a transpose or a conjugate transpose of D, and λ is a regularization coefficient greater than or equal to zero, and I is a unit matrix; The constraint on the L-factor matrix and the D-factor matrix acquired by the first participant is: D1 is a D factor matrix, which is a diagonal matrix, and L1 is an L factor matrix, which is a square matrix.

8. The linear regression method for federated learning in the vertical direction according to claim 1, wherein, The column encryption data matrix is one of an original data matrix of a participant, a matrix composed of an original data matrix of a participant and an additional column encryption mask matrix, and the additional column encryption mask matrix contains one column or multiple columns.

9. The linear regression method for federated learning in the vertical direction according to claim 8, wherein, When the column encryption data matrix of the first participant is a matrix composed of an original data matrix of the first participant and an additional column encryption mask matrix, after the multiplication encryption linear regression coefficient vector is calculated, the method further comprises: The label data owner or the first computing node sends the multiplication encryption linear regression coefficient vector to the first participant or forwards the multiplication encryption linear regression coefficient vector to the first participant by other participants, so that the first participant at least partially eliminates the influence of the additional column encryption mask matrix of the first participant on the multiplication encryption linear regression coefficient vector according to a multiplication encryption mask matrix private to the first participant, and updates the multiplication encryption linear regression coefficient vector.

10. The linear regression method for federated learning in the vertical direction according to claim 8, wherein, When the column encryption data matrix of the other participant is a matrix composed of an original data matrix of the other participant and an additional column encryption mask matrix, after the multiplication encryption linear regression coefficient vector is calculated, the method further comprises: The label data owner or the first computing node sends the multiplied encryption linear regression coefficient vector to the other participants or forwards the multiplied encryption linear regression coefficient vector to the other participants by any participant other than the other participants, so that the other participants eliminate the influence of their own augmented encryption mask matrix on the multiplied encryption linear regression coefficient vector according to their own private multiplied encryption mask matrix, and update the multiplied encryption linear regression coefficient vector.

11. The linear regression method for federated learning in the vertical direction according to claim 1, 9 or 10, characterized in that, The method further comprises: The label data owner, the first computing node, the first participant or the other participants send a plurality of multiplied encryption linear regression coefficients corresponding to a plurality of data features to the corresponding participants, so that the corresponding participants decrypt the received plurality of multiplied encryption linear regression coefficients according to their own multiplied encryption mask matrix to obtain a plurality of corresponding linear regression coefficients.

12. The linear regression method for federated learning in the vertical direction according to claim 11, wherein, The calculation formula of the multiplication-adding linear regression coefficient vector is: where m represents the number of participants providing private data; is a label vector, T m is a column-encrypted data and factor product matrix received by the label data owner from the first participant or other participants, P m is a multiplication-encrypted factor matrix received or computed by the first computing node; the multiply-add-cryptographic linear regression coefficient vector with the linear regression coefficient vector between them: where Ξ i is the multiplication encryption mask matrix of the ith participant providing private data.

Images