Federated learning linear regression method, device, system, equipment, storage medium and product

Abstract

The application discloses a federated learning linear regression method, device, system, equipment, storage medium and product, encrypted data obtained by a series of encryption calculations on original data of each participant is obtained from each participant, an aggregation operation is performed, and an aggregation result is fed back to a target receiving party, the target receiving party decrypts the aggregation result to obtain a joint training result, in the whole training process, there is no leakage risk of the original data, and joint training of the model is realized.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a federated learning linear regression method, apparatus, system, device, storage medium, and product. Background Technology

[0002] With the advent of the artificial intelligence era, more and more users and enterprises are beginning to pay attention to the field of machine learning. Linear regression algorithm is a statistical learning method based on the assumption that there is a linear relationship between independent and dependent variables. It is a widely used algorithm in the field of machine learning. Through linear regression algorithm, existing data can be modeled to make predictions.

[0003] A high-performing linear regression model requires a large amount of data for training and optimization, which needs to be collected from multiple data providers. However, with the rapid development of the digital economy, digital assets are becoming increasingly important, and data providers are paying more and more attention to the privacy and security of their own data. The leakage of their own data is unacceptable to them. How to enable multiple data providers to collaborate and complete the modeling of a linear regression algorithm while ensuring the privacy and security of their own data is a very valuable research problem. Summary of the Invention

[0004] Based on this, the present invention provides a federated learning linear regression method, apparatus, system, device, storage medium and product, which can obtain the aggregation result by performing aggregation operation on the encrypted data, and obtain the final joint training result by decrypting the aggregation result, thus completing the joint training of the model while avoiding data leakage.

[0005] To achieve the above objectives, embodiments of the present invention provide a federated learning linear regression method, comprising:

[0006] Receive encrypted data sent by different participants; wherein, the encrypted data is obtained by sequentially performing blinding processing, matrix transpose multiplication processing, and public key encryption processing on the original data, the original data includes the original independent variable and the original dependent variable, the matrix transpose multiplication processing refers to multiplying the transpose of the blinded data obtained by the blinding processing with the blinded data, and the public key in the public key encryption processing is generated by the target receiver and made public to all the participants;

[0007] The encrypted data is aggregated to obtain an aggregation result, which is then fed back to the target receiver so that the target receiver can obtain a joint training result based on the aggregation result. The aggregation result includes an aggregation result of independent variables and an aggregation result of dependent variables. The joint training result is obtained by sequentially performing private key decryption processing (corresponding to the public key encryption processing), matrix inverse multiplication processing, and deblinding processing (corresponding to the blinding processing) on ​​the aggregation result. The private key in the private key decryption processing is generated by the target receiver. The matrix inverse multiplication processing refers to multiplying the inverse of the aggregation result of independent variables after private key decryption processing by the aggregation result of dependent variables after private key decryption processing.

[0008] To achieve the above objectives, embodiments of the present invention provide a federated learning linear regression method, comprising:

[0009] The acquired raw data is subjected to blinding processing, matrix transpose multiplication processing, and public key encryption processing to obtain encrypted data, which is then sent to the data aggregator. The data aggregator performs aggregation operations on the encrypted data to obtain the aggregation result and feeds it back to the target receiver, so that the target receiver can obtain the joint training result based on the aggregation result.

[0010] The original data includes original independent variables and original dependent variables. The matrix transpose multiplication process refers to multiplying the transpose of the blinded data obtained after the blinding process with the blinded data. The public key in the public key encryption process is generated by the target recipient and disclosed to the participants. The aggregation result includes the aggregation result of independent variables and the aggregation result of dependent variables. The joint training result is obtained by sequentially performing private key decryption processing corresponding to the public key encryption process, matrix inverse multiplication processing, and deblinding processing corresponding to the blinding process on the aggregation result. The private key in the private key decryption process is generated by the target recipient. The matrix inverse multiplication process refers to multiplying the inverse of the aggregation result of independent variables after the private key decryption process with the aggregation result of dependent variables after the private key decryption process.

[0011] To achieve the above objectives, embodiments of the present invention provide a federated learning linear regression method, comprising:

[0012] Receive the joint training results; wherein the joint training results are obtained by the federated learning linear regression method described in any of the above embodiments.

[0013] To achieve the above objectives, embodiments of the present invention also provide a federated learning linear regression apparatus, comprising:

[0014] A receiving module is used to receive encrypted data sent by different participants; wherein, the encrypted data is obtained by sequentially performing blinding processing, matrix transpose multiplication processing, and public key encryption processing on the original data, the original data includes the original independent variable and the original dependent variable, the matrix transpose multiplication processing refers to multiplying the transpose of the blinded data obtained by the blinding processing with the blinded data, and the public key in the public key encryption processing is generated by the target receiver and made public to all the participants;

[0015] The learning module is used to perform aggregation operations on the encrypted data, obtain the aggregation result, and feed it back to the target receiver so that the target receiver can obtain a joint training result based on the aggregation result. The aggregation result includes an aggregation result of independent variables and an aggregation result of dependent variables. The joint training result is obtained by sequentially performing private key decryption processing (corresponding to the public key encryption processing), matrix inverse multiplication processing, and deblinding processing (corresponding to the blinding processing) on ​​the aggregation result. The private key in the private key decryption processing is generated by the target receiver. The matrix inverse multiplication processing refers to multiplying the inverse of the aggregation result of independent variables after private key decryption processing by the aggregation result of dependent variables after private key decryption processing.

[0016] To achieve the above objectives, embodiments of the present invention also provide a federated learning linear regression system, comprising:

[0017] The participating party is responsible for: performing blinding processing, matrix transpose multiplication processing, and public key encryption processing on the acquired raw data to obtain encrypted data, and sending it to the data aggregator; wherein, the raw data includes the original independent variable and the original dependent variable, the matrix transpose multiplication processing refers to multiplying the transpose of the blinded data obtained by the blinding processing with the blinded data, and the public key in the public key encryption processing is generated by the target receiver and made public to the participating party;

[0018] The data aggregator is configured to: perform aggregation operations on the encrypted data to obtain an aggregation result and send it to the target receiver;

[0019] The target receiver is configured to: receive the aggregation result, and sequentially perform private key decryption processing corresponding to the public key encryption processing, matrix inversion multiplication processing, and deblinding processing corresponding to the blinding processing on the aggregation result to obtain a joint training result; wherein, the aggregation result includes an aggregation result of independent variables and an aggregation result of dependent variables, and the matrix inversion multiplication processing refers to multiplying the inverted aggregation result of independent variables after the private key decryption processing by the multiplication result of dependent variables after the private key decryption processing.

[0020] To achieve the above objectives, embodiments of the present invention also provide a federated learning linear regression device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the federated learning linear regression method as described in any of the above embodiments.

[0021] To achieve the above objectives, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the federated learning linear regression method as described in any of the above embodiments.

[0022] To achieve the above objectives, embodiments of the present invention also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the federated learning linear regression method as described in any of the above embodiments.

[0023] Compared with existing technologies, the federated learning linear regression method, apparatus, system, device, storage medium, and product disclosed in this invention perform aggregation operations on encrypted data sent by different participants to obtain an aggregation result, which is then fed back to the target receiver. This allows the target receiver to obtain a joint training result based on the aggregation result. The encrypted data is obtained by sequentially performing blinding processing, matrix transpose multiplication, and public-key encryption on the original data. The original data includes original independent variables and original dependent variables. The matrix transpose multiplication refers to the transpose of the blinded data obtained through the blinding processing and the original independent variables. The data is multiplied, and the public key in the public key encryption process is generated by the target recipient and made public to all participating parties. The aggregation result includes the aggregation result of independent variables and the aggregation result of dependent variables. The joint training result is obtained by sequentially performing private key decryption processing corresponding to the public key encryption process, matrix inverse multiplication processing, and deblinding processing corresponding to the blinding process on the aggregation result. The private key in the private key decryption process is generated by the target recipient, and the matrix inverse multiplication process refers to multiplying the inverse of the aggregation result of independent variables after private key decryption processing with the aggregation result of dependent variables after private key decryption processing. Therefore, in this embodiment of the invention, encrypted data obtained from each participating party after a series of encryption calculations on their own original data is obtained, aggregation operations are performed, and the aggregation result is fed back to the target recipient. The target recipient decrypts the aggregation result to obtain the joint training result. In the entire training process, there is no risk of leakage of original data, and joint training of the model is achieved. Attached Figure Description

[0024] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a federated learning linear regression method according to an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of the architecture of a federated learning linear regression system provided in an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of the structure of a federated learning linear regression device provided in an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of a federated learning linear regression device provided in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] See Figure 1 This is a flowchart illustrating a federated learning linear regression method according to an embodiment of the present invention. Specifically, the federated learning linear regression method includes steps S1 to S2:

[0031] S1. Receive encrypted data sent by different participants; wherein the encrypted data is obtained by sequentially performing blinding processing, matrix transpose multiplication processing, and public key encryption processing on the original data; wherein the original data includes the original independent variable and the original dependent variable, the matrix transpose multiplication processing refers to multiplying the transpose of the blinded data obtained by the blinding processing with the blinded data, and the public key in the public key encryption processing is generated by the target receiver and made public to all participants;

[0032] S2. Perform aggregation operations on the encrypted data to obtain the aggregation result and feed it back to the target receiver so that the target receiver can obtain the joint training result based on the aggregation result; wherein, the aggregation result includes the aggregation result of independent variables and the aggregation result of dependent variables, and the joint training result is obtained by sequentially performing private key decryption processing corresponding to the public key encryption processing, matrix inverse multiplication processing, and deblinding processing corresponding to the blinding processing on the aggregation result. The private key in the private key decryption processing is generated by the target receiver, and the matrix inverse multiplication processing refers to multiplying the inverse of the aggregation result of independent variables after the private key decryption processing by the aggregation result of dependent variables after the private key decryption processing.

[0033] Specifically, the principle of linear regression will be briefly introduced below:

[0034] A linear regression model is a function that predicts real-valued output labels as accurately as possible through a linear combination of attributes. Given a sample dataset D = {(x1,y1),(x2,y2),…,(x...}... m ,y m )}, where x i =(x i1 ;x i2 ;…;x id x ), y∈R. i It is the independent variable, y i It is the dependent variable, which the linear regression model attempts to learn. Make f(x) i )≈y i , where ω=(ω1,ω2,…,ω d ),b∈R).

[0035] Mean square error formula:

[0036] Based on the mean squared error formula, the objective function of the linear regression model can be derived as follows:

[0037]

[0038] Combine ω and b into a single vector. Represent the sample dataset D as an m×(d+1) matrix X, where each row corresponds to a sample. Assign the first d elements of each row to the d attribute values ​​of the sample, and always set the last element to 1.

[0039]

[0040] Then, write the labeled result y in vector form as y = (y1; 2; ...; m )

[0041]

[0042] Therefore, the objective of solving the linear regression model can be expressed as follows:

[0043]

[0044] Solution process:

[0045] make right Differentiating, we get:

[0046]

[0047] make We can obtain:

[0048]

[0049] Specifically, as the above analysis shows, the original data can be processed into encrypted data that cannot be parsed by performing data operations such as encryption and transposition. Then, model training can be performed, and finally, the training results (i.e., the aggregated results) can be inversely operated and decrypted to obtain the final joint training results. This achieves collaborative training of multi-party data, increases the sample data, makes the trained model more accurate, and avoids data leakage.

[0050] It is worth noting that the method can be applied to the target receiver, the data aggregator, and any participating party. The data aggregator is a computing resource independent of both the participating parties and the target receiver. The target receiver can be a participating party, meaning it also needs to provide encrypted data for model training; however, the target receiver may not be a participating party. Preferably, the target receiver is also a participating party. For example, assuming the target receiver is one of all participating parties, it initiates a model training request, directly or through the data aggregator, sending instructions to other participating parties. Each participating party processes its data to obtain encrypted data and uploads it to the data aggregator. The data aggregator aggregates all the encrypted data, obtains the aggregation result, and then feeds it back to the target receiver. The target receiver processes this data to obtain the joint training result. Alternatively, the data aggregator can initiate a model training request to all participating parties (in which case the target receiver can be specified by the data aggregator or pre-defined), and each participating party provides encrypted data for model training. Furthermore, each participating party can automatically upload encrypted data for model training, etc. It is understood that the joint training result is the trained model parameters.

[0051] Furthermore, multiple companies in the same industry hope to collaboratively train a linear regression model using their respective private data, ensuring all participants have the same data dimensions. Since federated learning demands significant computing power, large-scale model training often requires substantial computational and storage resources, resulting in substantial costs for enterprises. This is unacceptable for some small and medium-sized enterprises (SMEs). Due to limitations in physical hardware, some companies lack the computational capabilities for local model training. Therefore, they seek external computing power to assist in the federated learning modeling process. Thus, preferably, a data aggregator is employed as the implementing entity for this method.

[0052] Furthermore, to ensure their own data is not leaked, participants need to perform a series of processing steps to obtain encrypted data. Due to limited enterprise resources, they need to utilize computing power networks to assist participants in sharing some of the computing tasks. Specifically, participants include the first computing power device and its corresponding computing power space. That is, in addition to the enterprise / individual's private device (the first computing power device), participants also include the computing power space allocated to that enterprise / individual in the computing power network. The first computing power device performs blinding processing on the raw data before sending it to the corresponding computing power space. The computing power space uses its resources to perform matrix transpose processing and public key encryption processing on the blinded data to obtain encrypted data, which is then sent to the data aggregator for aggregation. The aggregation result is then fed back to the target recipient's computing power space. The target recipient's computing power space performs private key decryption processing and matrix inversion processing on the aggregation result and feeds the processing result back to the target recipient's first computing power device. The target recipient's first computing power device performs deblinding processing to obtain the federated learning result. It is worth noting that any of the processing methods in matrix transpose, public key encryption, private key decryption, and matrix inversion can be processed by the first computing power device or by the computing power space.

[0053] Compared with existing technologies, the method disclosed in this invention can effectively solve the problem of 'data silos' between enterprises through federated learning, ensuring that there is enough sample data for model training and improving the accuracy of the model. Federated learning can make enterprise private data 'usable but not visible', that is, the model training process can be completed without leaving the domain of the private data of all enterprises participating in the machine learning model training. All communication and interaction between enterprises is only encrypted intermediate results, which effectively guarantees the privacy and security of enterprise data. There is no risk of leakage of original data during the entire training process.

[0054] In one embodiment, the method further includes:

[0055] The joint training results are sent to the first participant; wherein the first participant is any participant among all the participants.

[0056] Specifically, the data receiver sends the joint training results to all participants via broadcast or other means, so that all participants who provided the training data can use the trained parameters.

[0057] In one implementation, the blinding process is as follows: blinding the original independent variable using a public key and a private key, and blinding the original dependent variable using the private key; wherein the public key is generated by the target recipient and made public to all the participants, and the private key is generated by the participant to which the original independent variable belongs.

[0058] It is understandable that some enterprises lack sufficient computing resources and do not have the ability to perform large-scale calculations. Therefore, they need to rely on the computing power provided by the computing power network. However, relying on external computing power can lead to plaintext data going out of the domain, which may pose security risks. Therefore, the data is now blinded in the first computing power device and then sent to the corresponding computing power space. After the federated learning is completed, the data can be unblinded locally.

[0059] Furthermore, the process of blinding the original independent variable using a public key and a private key, and blinding the original dependent variable using the private key, includes:

[0060] Multiply the private key, the original independent variable, and the public key to obtain blinded data for the independent variable;

[0061] Multiply the private key by the original dependent variable to obtain blinded dependent variable data.

[0062] Furthermore, the unblinding process is as follows: the public key is used to unblind the data obtained after the matrix inverse multiplication process.

[0063] For example, to make the process of the federated learning linear regression method clearer, the following is a brief introduction using the federated learning linear regression method applied in a federated learning linear regression system. The federated learning linear regression system includes a target receiver, participating parties, and a data aggregator. It is assumed that the target receiver is also one of the participating parties. See [link to relevant documentation]. Figure 2 The diagram shows the architecture of a federated learning linear regression system. There are three participants: company 1, company 2, and company n. Each participant has an allocated computing power space. Assume the total dataset X consists of n participating companies {P1, P2, ..., Pn}. n Jointly provided by party P i The sample data is (X i ,yi)(1≤i≤n). X iIt is a matrix composed of samples, where each row represents a sample data point, and the row number represents the number of samples; y i It is a column vector, where each row is X. i Given the true labeling results of the samples in the corresponding row, multiple participants wish to jointly train a federated learning linear regression algorithm. Based on the above description, the linear regression solution can be expressed as follows:

[0064]

[0065] All participating companies {P1, P2, ..., P} n} can be based on its own sample data as (X i ,y i ) Calculated separately (A) i ,b i The final joint training result can be obtained by aggregating the final calculation results.

[0066] The specific training process is as follows:

[0067] Step 1: Participant P1 (target receiver) first generates a public key P, which exists in matrix form. The construction method of P is as follows:

[0068] P = (I - 2pp) T )

[0069] Where I is the identity matrix, p is a unit vector, i.e., the inner product of p is 1, p T Let p be the transpose of p, and p satisfies the following property:

[0070] PP T =(I-2pp) T (I-2pp) T ) T

[0071] =(I-2pp) T (I-2pp) T )

[0072] =I-2pp T -2pp T +4pp T pp T

[0073] =I-4pp T +4pp T

[0074] =I

[0075] P is an orthogonal matrix (satisfying P = P) T =P -1Next, participant P1 will broadcast the public key P to all participants.

[0076] Step 2: All participants generate their own private key Q.

[0077] Q = (I - 2qq) T )

[0078] Q is an m*m orthogonal matrix (constructed in a similar manner to P).

[0079] Step 3: All participants use the public key P and private key Q to blind their own data (original data). The blinding method is as follows:

[0080] X i ′=X i P

[0081] Y i ′=y i

[0082] Step 4: All participants send the blinded input task to the computing power space U they applied for in the computing power network resource pool. i In the middle, and require the calculation of A. i ′=(X i ′ T X i ′), b i ′=(X i ′ T Y i The derivation process is as follows:

[0083]

[0084]

[0085] Participant P i The ciphertext result (A′) can be obtained by using a computing network to perform ciphertext computation. i b i Without knowing the public key P, an external adversary cannot obtain the true result (A) from the encrypted result. i b i Therefore, the company's private data and calculation results are fully protected at this step.

[0086] Step 5: The computing space U requested by participant P1 i Generate a public-private key pair (PK, SK) using the Paillier algorithm, and publish the public key PK. The Paillier algorithm is a homomorphic encryption algorithm.

[0087] Step 6: All participants allocate their respective computing power space U from the computing power network. i The public key PK is used to encrypt the calculation result obtained by itself, encrypting A1 and b1 into E(A′). i ),E(b i After encryption, the data is sent to the data aggregation center (data aggregator).

[0088] Step 7: After receiving all the encrypted data, the data aggregation center performs aggregation operations to obtain:

[0089] E(A)=E(A1′+A2′+A3′+….+A n ′),E(b)=E(b1′+b2′+b3′+….+b n ′)

[0090] Addition satisfies the following properties:

[0091]

[0092]

[0093] In the formula S2 (A) =A1'+A2', a 11 (1) This represents the first element of the first row in A1', a 11 (2 b represents the first element of the first row in A2'. 11 (1) This represents the first element of the first row in b1', and so on for a. ij (1) a ij (2) b ij (1) b ij (2) Let A, B, and C represent the j-th element in the i-th row of A1', A2', b1', and b2', respectively.

[0094] Step 8: The data aggregation center aggregates the results and returns them to the computing space U1 of participant P1. At this time, U1 uses the private key SK to decrypt and obtain the following result:

[0095] A′=(A1′+A2′+…+A n B′) and B′=(b1′+b2′+…+b n From A′ and B′, we can derive the following form:

[0096] A′=(A1′+A2′+…+A n ′)

[0097] =(P T A1P+P T A2P+…+P T A n P)

[0098] =P T (A1+A2+…+A n )P

[0099] B′=(b′1+b′2+…+b′ n )

[0100] =(P T b1+P T b2+…+P T b n )

[0101] =P T (b1+b2+…+b n )

[0102] Therefore, A′=P T AP and B=P T B, the real information A and B are blinded here by the public key matrix, ensuring that the company's private data is not leaked. The inverse of U1 with respect to A' yields A'. -1 =P T A -1 P, then calculate A′ -1 B', after calculation, is returned to participant P1.

[0103] Step 9: Participant P1 uses the public key P to decrypt A′ -1 B' obtains the joint training results:

[0104] (A1+A2+…+A n ) -1 (b1+b2+…+b n )=(X T X) -1 X T y.

[0105] Step 10: Participant P1 broadcasts to all participants, and all participants receive the model parameters to complete the federated learning modeling process.

[0106] Compared with existing technologies, this invention provides a multi-user collaborative horizontal federated learning linear regression scheme based on computing power networks. This scheme allows for the training of horizontal federated learning linear regression models using computing power networks while ensuring enterprise data privacy and security. In this scheme, all data sent by the enterprise to the external computing power platform is encrypted, thus preventing the external computing power platform from stealing any confidential information from the enterprise's messages. Simultaneously, this scheme enables the external computing power platform to perform specific encrypted calculations in encrypted form, effectively solving the problem of insufficient computing power for some enterprises and significantly reducing the computing power cost required for users to train models.

[0107] This invention also provides another federated learning linear regression method, including:

[0108] The acquired raw data is subjected to blinding processing, matrix transpose multiplication processing, and public key encryption processing to obtain encrypted data, which is then sent to the data aggregator. The data aggregator performs aggregation operations on the encrypted data to obtain the aggregation result and feeds it back to the target receiver, so that the target receiver can obtain the joint training result based on the aggregation result.

[0109] The original data includes original independent variables and original dependent variables. The matrix transpose multiplication process refers to multiplying the transpose of the blinded data obtained after the blinding process with the blinded data. The public key in the public key encryption process is generated by the target recipient and disclosed to the participants. The aggregation result includes the aggregation result of independent variables and the aggregation result of dependent variables. The joint training result is obtained by sequentially performing private key decryption processing corresponding to the public key encryption process, matrix inverse multiplication processing, and deblinding processing corresponding to the blinding process on the aggregation result. The private key in the private key decryption process is generated by the target recipient. The matrix inverse multiplication process refers to multiplying the inverse of the aggregation result of independent variables after the private key decryption process with the aggregation result of dependent variables after the private key decryption process.

[0110] Specifically, the method is applied to each participating party.

[0111] Compared with the prior art, in this embodiment of the invention, encrypted data is obtained from each participant after the participant performs a series of encrypted calculations on its own original data. After aggregation operation, the aggregation result is fed back to the target receiver, and the target receiver decrypts the aggregation result to obtain the joint training result. In the entire training process, there is no risk of leakage of the original data, and the joint training of the model is realized.

[0112] In one implementation, the participants include a first computing power device and its corresponding computing power space;

[0113] The process of blinding, matrix transpose multiplication, and public-key encryption of the acquired raw data to obtain encrypted data, and then sending it to the data aggregator, includes:

[0114] The computing power device performs blinding processing on the acquired raw data to obtain blinded data, which is then sent to the corresponding computing power space.

[0115] The blinded data is processed by matrix transpose multiplication and public key encryption using the computing power space.

[0116] It is worth noting that the specific working process of the federated learning linear regression method can be referred to the working process of the federated learning linear regression method in the above embodiments, and will not be repeated here.

[0117] Compared with the prior art, in this embodiment of the invention, encrypted data is obtained from each participant after the participant performs a series of encrypted calculations on its own original data. After aggregation operation, the aggregation result is fed back to the target receiver, and the target receiver decrypts the aggregation result to obtain the joint training result. In the entire training process, there is no risk of leakage of the original data, and the joint training of the model is realized.

[0118] This invention also provides another federated learning linear regression method, including:

[0119] Receive joint training results; wherein the joint training results are obtained by the data aggregator performing the federated learning linear regression method as described in any of the above embodiments.

[0120] Specifically, the method is applied to the data receiver.

[0121] It is worth noting that the specific working process of the federated learning linear regression method can be referred to the working process of the federated learning linear regression method in the above embodiments, and will not be repeated here.

[0122] Compared with the prior art, in this embodiment of the invention, encrypted data is obtained from each participant after the participant performs a series of encrypted calculations on its own original data. After aggregation operation, the aggregation result is fed back to the target receiver, and the target receiver decrypts the aggregation result to obtain the joint training result. In the entire training process, there is no risk of leakage of the original data, and the joint training of the model is realized.

[0123] See Figure 3 This invention also provides a federated learning linear regression apparatus, comprising:

[0124] The receiving module 11 is used to receive encrypted data sent by different participants; wherein the encrypted data is obtained by sequentially performing blinding processing, matrix transpose multiplication processing, and public key encryption processing on the original data; wherein the original data includes the original independent variable and the original dependent variable, the matrix transpose multiplication processing refers to multiplying the transpose of the blinded data obtained by the blinding processing with the blinded data, and the public key in the public key encryption processing is generated by the target receiver and made public to all the participants;

[0125] Learning module 12 is used to perform aggregation operations on the encrypted data, obtain the aggregation result, and feed it back to the target receiver so that the target receiver can obtain the joint training result based on the aggregation result; wherein, the aggregation result includes the aggregation result of independent variables and the aggregation result of dependent variables, and the joint training result is obtained by sequentially performing private key decryption processing corresponding to the public key encryption processing, matrix inverse multiplication processing, and deblinding processing corresponding to the blinding processing on the aggregation result of ...

[0126] In one embodiment, the device further includes:

[0127] A data sending module is used to send the joint training results to a first participant; wherein the first participant is any participant among all the participants.

[0128] In one implementation, the blinding process is as follows: blinding the original independent variable using a public key and a private key, and blinding the original dependent variable using the private key; wherein the public key is generated by the target recipient and made public to all the participants, and the private key is generated by the participant to which the original independent variable belongs.

[0129] In one implementation, the step of blinding the original independent variable using a public key and a private key, and blinding the original dependent variable using the private key, includes:

[0130] Multiply the private key, the original independent variable, and the public key to obtain blinded data for the independent variable;

[0131] Multiply the private key by the original dependent variable to obtain blinded dependent variable data.

[0132] In one embodiment, the unblinding process is as follows: the public key is used to unblind the data obtained after the matrix inverse multiplication process.

[0133] In one implementation, the target recipient is one of all the participating parties.

[0134] It is worth noting that the specific working process of the federated learning linear regression device can be referred to the working process of the federated learning linear regression method described in the above embodiments, and will not be repeated here.

[0135] Compared with the prior art, in this embodiment of the invention, encrypted data is obtained from each participant after the participant performs a series of encrypted calculations on its own original data. After aggregation operation, the aggregation result is fed back to the target receiver, and the target receiver decrypts the aggregation result to obtain the joint training result. In the entire training process, there is no risk of leakage of the original data, and the joint training of the model is realized.

[0136] This invention also provides a federated learning linear regression system, comprising:

[0137] The participating party is responsible for: performing blinding processing, matrix transpose multiplication processing, and public key encryption processing on the acquired raw data to obtain encrypted data, and sending it to the data aggregator; wherein, the raw data includes the original independent variable and the original dependent variable, the matrix transpose multiplication processing refers to multiplying the transpose of the blinded data obtained by the blinding processing with the blinded data, and the public key in the public key encryption processing is generated by the target receiver and made public to the participating party;

[0138] The data aggregator is configured to: perform aggregation operations on the encrypted data to obtain an aggregation result and send it to the target receiver;

[0139] The target receiver is configured to: receive the aggregation result, and sequentially perform private key decryption processing corresponding to the public key encryption processing, matrix inversion multiplication processing, and deblinding processing corresponding to the blinding processing on the aggregation result to obtain a joint training result; wherein, the aggregation result includes an aggregation result of independent variables and an aggregation result of dependent variables, and the matrix inversion multiplication processing refers to multiplying the inverted aggregation result of independent variables after the private key decryption processing by the multiplication result of dependent variables after the private key decryption processing.

[0140] Specifically, the specific structure of the system can be found in [reference needed]. Figure 2 .

[0141] It is worth noting that the specific working process of the federated learning linear regression system can be referred to the working process of the federated learning linear regression method described in the above embodiments, and will not be repeated here.

[0142] Compared with the prior art, the system provided by the embodiments of the present invention obtains encrypted data from each participant after the participant performs a series of encrypted calculations on its own original data, performs aggregation calculations, and feeds back the aggregation result to the target receiver. The target receiver decrypts the aggregation result to obtain the joint training result. In the entire training process, there is no risk of leakage of the original data, and the joint training of the model is realized.

[0143] See Figure 4 This invention also provides a federated learning linear regression device, including a processor 21, a memory 22, and a computer program stored in the memory 22 and configured to be executed by the processor 21. When the processor 21 executes the computer program, it implements the steps described in the above-described federated learning linear regression method embodiment, for example... Figure 1 The steps S1 to S2 described above; or, when the processor 21 executes the computer program, it implements the functions of each module in the above-described device embodiments.

[0144] For example, the computer program can be divided into one or more modules, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the federated learning linear regression device. For example, the computer program can be divided into multiple modules, each with the following specific functions:

[0145] The receiving module 11 is used to receive encrypted data sent by different participants; wherein the encrypted data is obtained by sequentially performing blinding processing, matrix transpose multiplication processing, and public key encryption processing on the original data; wherein the original data includes the original independent variable and the original dependent variable, the matrix transpose multiplication processing refers to multiplying the transpose of the blinded data obtained by the blinding processing with the blinded data, and the public key in the public key encryption processing is generated by the target receiver and made public to all the participants;

[0146] Learning module 12 is used to perform aggregation operations on the encrypted data, obtain the aggregation result, and feed it back to the target receiver so that the target receiver can obtain the joint training result based on the aggregation result; wherein, the aggregation result includes the aggregation result of independent variables and the aggregation result of dependent variables, and the joint training result is obtained by sequentially performing private key decryption processing corresponding to the public key encryption processing, matrix inverse multiplication processing, and deblinding processing corresponding to the blinding processing on the aggregation result of ...

[0147] The specific working process of each module can be referred to the working process of the federated learning linear regression device described in the above embodiments, and will not be repeated here.

[0148] The federated learning linear regression device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The federated learning linear regression device may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the federated learning linear regression device may also include input / output devices, network access devices, buses, etc.

[0149] The processor 21 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the federated learning linear regression device, connecting all parts of the device via various interfaces and lines.

[0150] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the federated learning linear regression device by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as image playback function), etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0151] If the modules integrated into the federated learning linear regression device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0152] This invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the federated learning linear regression method as described in any of the above embodiments.

[0153] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A federated learning linear regression method, characterized in that, include: Receive encrypted data sent by different participants; wherein, the encrypted data is obtained by sequentially performing blinding processing, matrix transpose multiplication processing, and public key encryption processing on the original data, the original data includes the original independent variable and the original dependent variable, the matrix transpose multiplication processing refers to multiplying the transpose of the blinded data obtained by the blinding processing with the blinded data, and the public key in the public key encryption processing is generated by the target receiver and made public to all the participants; The encrypted data is aggregated to obtain an aggregation result, which is then fed back to the target receiver so that the target receiver can obtain a joint training result based on the aggregation result. The aggregation result includes an aggregation result of independent variables and an aggregation result of dependent variables. The joint training result is obtained by sequentially performing private key decryption processing (corresponding to the public key encryption processing), matrix inversion multiplication processing, and deblinding processing (corresponding to the blinding processing) on ​​the aggregation result. The private key in the private key decryption processing is generated by the target receiver. The matrix inversion multiplication processing refers to multiplying the inverted independent variable aggregation result after private key decryption processing by the multiplied dependent variable aggregation result after private key decryption processing. The blinding process is as follows: the original independent variable is blinded using a public key and a private key, and the original dependent variable is blinded using the private key; wherein, the public key is generated by the target recipient and made public to all the participants, and the private key is generated by the participant to which the original independent variable belongs.

2. The federated learning linear regression method as described in claim 1, characterized in that, Also includes: The joint training results are sent to the first participant; wherein the first participant is any participant among all the participants.

3. The federated learning linear regression method as described in claim 1, characterized in that, The process of blinding the original independent variables using public and private keys, and blinding the original dependent variable using the private key, includes: Multiply the private key, the original independent variable, and the public key to obtain blinded data for the independent variable; Multiply the private key by the original dependent variable to obtain blinded dependent variable data.

4. The federated learning linear regression method as described in claim 1 or 3, characterized in that, The steps of the unblinding process are as follows: use the public key to unblind the data obtained after the matrix inverse multiplication process.

5. The federated learning linear regression method as described in claim 1, characterized in that, The target recipient is one of all the participating parties.

6. A federated learning linear regression method, characterized in that, include: The acquired raw data is subjected to blinding processing, matrix transpose multiplication processing, and public key encryption processing to obtain encrypted data, which is then sent to the data aggregator. The data aggregator performs aggregation operations on the encrypted data to obtain the aggregation result and feeds it back to the target receiver, so that the target receiver can obtain the joint training result based on the aggregation result. The original data includes original independent variables and original dependent variables. The matrix transpose multiplication process refers to multiplying the transpose of the blinded data obtained after the blinding process with the blinded data. The public key in the public key encryption process is generated by the target recipient and disclosed to the participants. The aggregation result includes the aggregation result of independent variables and the aggregation result of dependent variables. The joint training result is obtained by sequentially performing private key decryption processing corresponding to the public key encryption process, matrix inverse multiplication processing, and deblinding processing corresponding to the blinding process on the aggregation result. The private key in the private key decryption process is generated by the target recipient. The matrix inverse multiplication process refers to multiplying the inverse of the aggregation result of independent variables after the private key decryption process with the aggregation result of dependent variables after the private key decryption process. The blinding process is as follows: the original independent variable is blinded using a public key and a private key, and the original dependent variable is blinded using the private key; wherein, the public key is generated by the target recipient and made public to all the participants, and the private key is generated by the participant to which the original independent variable belongs.

7. The federated learning linear regression method as described in claim 6, characterized in that, The participating parties include the first computing power device and its corresponding computing power space; The process of blinding, matrix transpose multiplication, and public-key encryption of the acquired raw data to obtain encrypted data, and then sending it to the data aggregator, includes: The computing power device performs blinding processing on the acquired raw data to obtain blinded data, which is then sent to the corresponding computing power space. The blinded data is processed by matrix transpose multiplication and public key encryption using the computing power space.

8. A federated learning linear regression method, characterized in that, include: Receive joint training results; wherein the joint training results are obtained by the data aggregator performing the federated learning linear regression method as described in any one of claims 1 to 5.

9. A federated learning linear regression device, characterized in that, include: A receiving module is used to receive encrypted data sent by different participants; wherein, the encrypted data is obtained by sequentially performing blinding processing, matrix transpose multiplication processing, and public key encryption processing on the original data, the original data includes the original independent variable and the original dependent variable, the matrix transpose multiplication processing refers to multiplying the transpose of the blinded data obtained by the blinding processing with the blinded data, and the public key in the public key encryption processing is generated by the target receiver and made public to all the participants; The learning module is used to perform aggregation operations on the encrypted data, obtain the aggregation result, and feed it back to the target receiver so that the target receiver can obtain the joint training result based on the aggregation result. The aggregation result includes an aggregation result of independent variables and an aggregation result of dependent variables. The joint training result is obtained by sequentially performing private key decryption processing (corresponding to the public key encryption processing), matrix inversion multiplication processing, and deblinding processing (corresponding to the blinding processing) on ​​the aggregation result. The private key in the private key decryption processing is generated by the target receiver. The matrix inversion multiplication processing refers to multiplying the inverted independent variable aggregation result after private key decryption processing by the multiplied dependent variable aggregation result after private key decryption processing. The blinding process is as follows: the original independent variable is blinded using a public key and a private key, and the original dependent variable is blinded using the private key; wherein, the public key is generated by the target recipient and made public to all the participants, and the private key is generated by the participant to which the original independent variable belongs.

10. A federated learning linear regression system, characterized in that, include: The participating parties are responsible for: performing blinding processing, matrix transpose multiplication, and public-key encryption on the acquired raw data to obtain encrypted data, and then sending it to the data aggregator; wherein, the raw data includes raw independent variables and raw dependent variables, the matrix transpose multiplication refers to multiplying the transpose of the blinded data obtained through the blinding processing with the blinded data, and the public key in the public-key encryption processing is generated by the target receiver and made public to the participating parties; the blinding processing steps are as follows: using a public key and a private key to blind the raw independent variables, and using the private key to blind the raw dependent variables; wherein, the public key is generated by the target receiver and made public to all participating parties, and the private key is generated by the participating party to which the raw independent variables belong; The data aggregator is configured to: perform aggregation operations on the encrypted data to obtain an aggregation result and send it to the target receiver; The target receiver is configured to: receive the aggregation result, and sequentially perform private key decryption processing corresponding to the public key encryption processing, matrix inversion multiplication processing, and deblinding processing corresponding to the blinding processing on the aggregation result to obtain a joint training result; wherein, the aggregation result includes an aggregation result of independent variables and an aggregation result of dependent variables, and the matrix inversion multiplication processing refers to multiplying the inverted aggregation result of independent variables after the private key decryption processing by the multiplication result of dependent variables after the private key decryption processing.

11. A federated learning linear regression device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the federated learning linear regression method as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the federated learning linear regression method as described in any one of claims 1 to 8.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the federated learning linear regression method as described in any one of claims 1 to 8.

Images