Computing Method, Device, Equipment and Medium in Vertical Federated Learning

Abstract

The present application discloses a calculation method, device, equipment and medium in vertical federated learning, which relates to the field of federated learning in artificial intelligence. In the present application, multiple participating nodes deployed in a multi-way tree topology are adopted, where one upper-layer participating node has k lower-layer participating nodes. After the upper-layer participating node and the k lower-layer participating nodes exchange their public keys, the upper-layer participating node and the lower-layer participating nodes perform two-party joint secure calculation with the first public key and the second public key as encryption parameters to obtain k two-party joint outputs of the federated model. Furthermore, the upper-layer participating node merges the k two-party joint outputs to obtain the first joint model output corresponding to the federated model. Thus, a vertical federated learning architecture deployed in a multi-way tree topology is provided, which improves the equality of each participating node during the vertical federated learning process.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of machine learning, and particularly to a calculation method, device, equipment and medium in vertical federated learning. Background Art

[0002] Federated Learning can train machine learning models and provide model inference services by combining data sources of multiple participants without data leaving the domain. Federated Learning enables data cooperation across departments, companies, and even industries, while meeting the requirements of data protection laws and regulations.

[0003] In the related art, multi-party vertical federated learning depends on a trusted central node to participate, and the central node stores the private key, and each participant stores the public key, so as to realize the encrypted transmission of data between each participant.

[0004] However, since it is difficult to find a trusted central node to store the private key, the above solution is difficult to be applied in practice. Moreover, having a central node store the private key will cause a large centralized security risk for this single point of the central node. Summary of the Invention

[0005] Embodiments of the present application provide a calculation method, device, equipment and medium in vertical federated learning, which can provide a vertical federated learning architecture deployed with a multi-tree topology, and improve the equality of each participating node in the process of vertical federated learning. The technical solution is as follows:

[0006] According to one aspect of the present application, there is provided a calculation method in vertical federated learning, which is applied to an upper-layer participating node. The upper-layer participating node has k lower-layer participating nodes among multiple participating nodes deployed with a multi-tree topology. A sub-model of the federated model is locally deployed in each participating node, and k is an integer greater than 1. The method includes:

[0007] Distribute the first public key corresponding to the upper-layer participating node to the k lower-layer participating nodes, and obtain k second public keys respectively corresponding to the k lower-layer participating nodes;

[0008] Using the first public key and the second public key as encryption parameters, the upper-layer participating node performs two-party joint secure calculation with the k lower-layer participating nodes to obtain k two-party joint outputs of the federated model; the two-party joint secure calculation includes the upper-layer participating node and the lower-layer participating node performing forward calculation on the sub-models of both parties using their respective data based on the homomorphic encryption method;

[0009] Merge the k two-party joint outputs to obtain the first joint model output corresponding to the upper-layer participating node and the k lower-layer participating nodes.

[0010] According to one aspect of the present application, there is provided a calculation method in vertical federated learning, which is characterized in that it is applied to a lower-layer participating node. The lower-layer participating node has an upper-layer participating node among multiple participating nodes deployed in a multi-way tree topology. Each participating node locally deploys a sub-model in the federated model. The method includes:

[0011] Report the second public key of the lower-layer participating node to the upper-layer participating node, and obtain the first public key corresponding to the upper-layer participating node. The upper-layer participating node includes k lower-layer participating nodes, where k is an integer greater than 1;

[0012] Using the first public key and the second public key as encryption parameters, the lower-layer participating node performs two-party joint secure calculation with the upper-layer participating node to obtain a two-party joint output of the federated model. The two-party joint secure calculation includes the forward calculation performed by the upper-layer participating node and the lower-layer participating node on their respective sub-models based on the homomorphic encryption method using their respective data.

[0013] According to one aspect of the present application, there is provided an upper-layer participating node. The upper-layer participating node has k lower-layer participating nodes among multiple participating nodes deployed in a multi-way tree topology. Each participating node locally deploys a sub-model in the federated model, where k is an integer greater than 1. The upper-layer participating node includes:

[0014] A communication module for distributing the first public key corresponding to the upper-layer participating node to the k lower-layer participating nodes, and obtaining k second public keys respectively corresponding to the k lower-layer participating nodes;

[0015] A joint calculation module for using the first public key and the second public key as encryption parameters to perform two-party joint secure calculation with the k lower-layer participating nodes to obtain k two-party joint outputs of the federated model. The two-party joint secure calculation includes the forward calculation performed by the upper-layer participating node and the lower-layer participating node on their respective sub-models based on the homomorphic encryption method using their respective data;

[0016] A merging module for merging the k two-party joint outputs to obtain the first joint model output corresponding to the upper-layer participating node and the k lower-layer participating nodes.

[0017] According to one aspect of the present application, a lower-layer participating node is provided. The lower-layer participating node has an upper-layer participating node among a plurality of participating nodes deployed in a multi-way tree topology. A sub-model in a federated model is locally deployed in each participating node. The lower-layer participating node includes:

[0018] A communication module, configured to report a second public key of the lower-layer participating node to the upper-layer participating node, and obtain a first public key corresponding to the upper-layer participating node. The upper-layer participating node includes k lower-layer participating nodes, where k is a positive integer greater than 1;

[0019] A joint computing module, configured to perform two-party joint secure computing with the upper-layer participating node using the first public key and the second public key as encryption parameters, to obtain a two-party joint output of the federated model. The two-party joint secure computing includes forward computing performed by the upper-layer participating node and the lower-layer participating node on their respective sub-models of both parties based on a homomorphic encryption method using their respective data.

[0020] According to another aspect of the present application, a computer device is provided. The computer device includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the computing method in vertical federated learning as described in the above aspect.

[0021] According to another aspect of the present application, a computer-readable storage medium is provided. At least one instruction, at least one program, a code set, or an instruction set is stored in the computer-readable storage medium. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the computing method in vertical federated learning as described in the above aspect.

[0022] According to one aspect of the present application, a computer program product is provided. The computer program product includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the computing method in vertical federated learning as described in the above aspect.

[0023] The beneficial effects brought by the technical solutions provided in the embodiments of the present application at least include:

[0024] By adopting multiple participating nodes deployed in a multi - fork tree topology, where an upper - layer participating node has k lower - layer participating nodes. After the upper - layer participating node and the k lower - layer participating nodes exchange their public keys, the upper - layer participating node and the lower - layer participating nodes perform two - party joint secure computation using the first public key and the second public key as encryption parameters to obtain k two - party joint outputs of the federated model. Furthermore, the upper - layer participating node merges the k two - party joint outputs to obtain the first joint model output corresponding to the federated model. Thus, a vertical federated learning architecture with a multi - fork tree topology is provided, which improves the equality of each participating node in the process of vertical federated learning. Since the upper - layer participating node and the lower - layer participating nodes exchange their public keys with each other and each keeps its own private key, the security risks can be spread across each participating node. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following - described drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0026] Figure 1 is a schematic diagram of the principle of vertical federated learning provided by an exemplary embodiment of the present application;

[0027] Figure 2 is a block diagram of the structure of a computer system provided by an exemplary embodiment of the present application;

[0028] Figure 3 is a flowchart of a computing method in vertical federated learning provided by another exemplary embodiment of the present application;

[0029] Figure 4 is a flowchart of a method when an upper - layer participating node and a lower - layer participating node perform two - party joint secure computation provided by another exemplary embodiment of the present application;

[0030] Figure 5 is a block diagram of N participating nodes with a two - layer multi - fork tree topology provided by another exemplary embodiment of the present application;

[0031] Figure 6 is a block diagram of N participating nodes with a multi - layer multi - fork tree topology provided by another exemplary embodiment of the present application;

[0032] Figure 7 is a flowchart of a computing method in vertical federated learning provided by another exemplary embodiment of the present application;

[0033] Figure 8It is a flowchart of a calculation method in vertical federated learning provided by another exemplary embodiment of the present application;

[0034] Figure 9 It is a block diagram of a distributed system provided by an exemplary embodiment of the present application;

[0035] Figure 10 It is a schematic diagram of a block structure provided by an exemplary embodiment of the present application;

[0036] Figure 11 It is a block diagram of an upper-layer participating node provided by another exemplary embodiment of the present application;

[0037] Figure 12 It is a block diagram of a lower-layer participating node provided by another exemplary embodiment of the present application;

[0038] Figure 13 It is a block diagram of a computer device provided by another exemplary embodiment of the present application. Detailed implementation manners

[0039] To make the objectives, technical solutions and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0040] First, a brief introduction to several nouns related to the embodiments of the present application is given:

[0041] Artificial Intelligence (AI): It is to use a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, a theory, method, technology and application system that perceives the environment, acquires knowledge and uses knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

[0042] Artificial intelligence technology is an interdisciplinary subject, involving a wide range of fields, including both hardware-level technologies and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0043] Machine Learning (ML): It is an interdisciplinary subject that involves multiple fields such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rote learning.

[0044] Federated Learning: It is used to train machine learning models by combining data sources of multiple participants without the data leaving their respective domains and to provide model inference services. Federated learning can improve the performance of machine learning models by making full use of the data sources of multiple participants while protecting user privacy and data security. Federated learning makes it possible for cross-departmental, cross-company, and even cross-industry data cooperation, while meeting the requirements of data protection laws and regulations.

[0045] Federated learning can be divided into three categories: Horizontal Federated Learning, Vertical Federated Learning, and Federated Transfer Learning.

[0046] Vertical Federated Learning: It is a type of federated learning used when there is a relatively large overlap in the training sample identifiers (IDs) of the participants but a relatively small overlap in the data features. For example, a bank and an e-commerce company in the same region may each have different feature data of the same customer A. The bank may have the financial data of customer A, and the e-commerce company may have the shopping data of customer A. The word "vertical" comes from the "vertical partitioning" of the data. As Figure 1 shown, federated learning is carried out by combining different feature data of user samples with intersections among multiple participants, that is, the training samples of each participant are vertically partitioned.

[0047] Homomorphic Encryption: It is a cryptographic technology based on the computational complexity theory of mathematical problems. Processing data encrypted by homomorphic encryption yields an output, and decrypting this output gives the same result as processing the unencrypted original data using the same method.

[0048] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, smart customer service, digital credit, financial credit, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0049] This application provides a technical solution for vertical federated learning. This vertical federated learning does not require a central node that holds the private key, and each participating party can adopt a centerless node solution to complete the model training service. This vertical federated learning can support vertical federated learning of any number of parties and is used for model training services and model inference services for federated learning tasks.

[0050] Figure 2 The block diagram of a vertical federated learning system 100 provided by an exemplary embodiment of this application is shown. This vertical federated learning system supports vertical federated learning in which N participating nodes (also referred to as participating parties) cooperate together. This vertical federated learning system 100 includes: an upper-layer participating node P0, lower-layer participating nodes P1 to participating node P N-1 。

[0051] The upper-layer participating node P0 and the lower-layer participating nodes (participating nodes P1 to participating node P N-1 ) are deployed using a multi-way tree topology. Any one participating node can be a server, or multiple servers, or a logical computing module in cloud computing services. Any two participating nodes belong to different data sources, such as data sources of different companies, or data sources of different subsidiaries of the same company.

[0052] The upper-layer participating node P0 is the participating node that has the label information y. The upper-layer participating node P0 may have the feature data X0 or may not have the feature data X0. For example, the upper-layer participating node P0 is a bank and has the mobile phone number and whether there is an overdue record of each user. Among them, the mobile phone number of the user is used as the sample ID, and whether there is an overdue record is used as the label information y. Exemplarily, when there are multiple participating nodes with label information, these participating nodes take turns as the upper-layer participating node P0.

[0053] The lower-layer participating node P i each has the feature data X i , i = 1, 2,..., N - 1. The lower-layer participating node P i may have the label information y or may not have the label information y.

[0054] The above vertical federated learning system 100 supports N participating nodes to collaborate and securely train a neural network model. The neural network model includes, but is not limited to: a linear regression model (Linear Regression, LR), or a logistic regression model (Logistic Regression, LogR), or a support vector machine model (Support Vector Machine, SVM), where N≥3.

[0055] This neural network model is also called a federated model. The federated model includes N sub-models, and each participating node locally deploys a sub-model. The network parameters in each sub-model can be different. Taking the neural network model as a multiple linear regression model y = W0X0 + W1X1 + W2X2 + W3X3 + W4X4 + W5X5 as an example, the sub-model deployed by the first participating node has the first part of the network parameters W0 of the multiple linear regression model, the sub-model deployed by the second participating node has the second part of the parameters W1 and W2 of the multiple linear regression model, and the sub-model deployed by the third participating node has the third part of the network parameters W3 and W4 of the multiple linear regression model, and so on. The network parameters in the sub-models deployed by all participating nodes constitute all the network parameters of the multiple linear regression model.

[0056] The upper-level participating node P0 has the second public key PK i of the lower-level participating node P i , i = 1, 2, …, N - 1. The second public keys PK i of different lower-level participating nodes are different. The lower-level participating node P i (i = 1, 2, …, N - 1) has the first public key PK0 of the upper-level participating node P0. The first public key PK0 owned by each lower-level participating node is the same. No participating node will disclose its private key to other participating nodes. The public key is used to encrypt the intermediate calculation results during model training. The encryption algorithm adopted in this application is an additive homomorphic encryption algorithm. The additive homomorphic encryption algorithm can be the Paillier homomorphic encryption algorithm.

[0057] The upper-level participating node P0 generates the first random mask R i for the lower-level participating node P i , i = 1, 2, …, N - 1. The first random masks corresponding to different lower-level participating nodes are different. The lower-level participating node P i (i = 1, 2, …, N - 1) generates the second random mask R 0,i. The second random masks corresponding to different lower-layer participating nodes are different. No participating node will disclose the plaintext of any random mask to other participating nodes. The random mask is used to protect the intermediate calculation results during model training and prevent the network parameters of the neural network model from being inversely solved through the model output values of multiple groups of training samples.

[0058] During the model training process, the upper-layer participating node P0 and the lower-layer participating node P i cooperate to perform joint secure computation between the two parties without disclosing any training samples:

[0059]

[0060] Among them, {W i , i = 1, 2, …, N - 1} are the model parameters of the neural network model. The operation of performing joint secure computation {z i , i = 1, 2, …, N - 1} between the upper-layer participating node P0 and multiple lower-layer participating nodes P 0,i can be executed in parallel by (N - 1) parties to improve efficiency.

[0061] In another possible implementation, the calculation formula of the above z o，i is as follows:

[0062] z 0,i = (W0 + R 0,i )X0 + (W i + R i )X i , i = 1, 2, …, N - 1;

[0063] That is, the weights in the previous implementation are optional weights.

[0064] The upper-layer participating node P0 calculates the joint model output and the prediction model output and calculates and sends the residual (also called the error) to the lower-layer participating nodes.

[0065] The lower-layer participating node P i locally calculates the gradient g i according to the residual δ, and locally updates the model parameters W i , i = 1, 2, …, N - 1.

[0066] Optionally, this application also organizes multiple participating nodes through a topological structure such as a multi-way tree (including a binary tree) to reduce the requirements for the computing and communication capabilities of a single participating node P0.

[0067] Figure 3The flowchart of the computing method in vertical federated learning provided by an exemplary embodiment of the present application is shown. This method is applied to at least two layers (all layers or a part of the layers) of participating nodes deployed in a multi-way tree topology, and a star topology is formed between the upper-layer participating nodes and the lower-layer participating nodes. The at least two layers of participating nodes include upper-layer participating nodes and k lower-layer participating nodes. The method includes:

[0068] Step 301: The upper-layer participating node distributes the first public key corresponding to the upper-layer participating node to the k lower-layer participating nodes, and the k lower-layer participating nodes obtain the first public key corresponding to the upper-layer participating node;

[0069] The upper-layer participating node P0 generates a first public key PK0 and a first private key SK0, and sends the first public key PK0 to N - 1 lower-layer participating nodes P i The first public key PK0 and the first private key SK0 are a pair of key pairs. For the ciphertext encrypted by the first public key PK0, the first private key SK0 can decrypt the ciphertext to obtain the original text; for the ciphertext encrypted by the first private key SK0, the first public key PK0 can decrypt the ciphertext to obtain the original text. The first public key PK0 is visible to the lower-layer participating node P i It can be seen that the first private key SK0 is not visible to the lower-layer participating node P i invisible.

[0070] Assume that N is the total number of participating nodes, and k is a positive integer less than N.

[0071] Step 302: The k lower-layer participating nodes report the second public key to the upper-layer participating node, and the upper-layer participating node obtains the k second public keys corresponding to the k lower-layer participating nodes respectively;

[0072] Each lower-layer participating node P i generates a second public key PK i and a second private key SK i , and sends the second public key PK i to the upper-layer participating node P i . The second public key PK i and the second private key SK i are a pair of key pairs. For the ciphertext encrypted by the second public key PK i , the second private key SK i can decrypt the ciphertext to obtain the original text; for the ciphertext encrypted by the second private key SK i , the second public key PK i can decrypt the ciphertext to obtain the original text. The second public key PK i is visible to the lower-layer participating node, and the second private key SK i is not visible to the lower-layer participating node. The value range of i is from 1 to k.

[0073] The embodiments of the present application do not limit the timing of the above two steps. Step 301 and step 302 can be executed simultaneously, or step 302 can be executed before or after step 301.

[0074] Step 303: Using the first public key and the second public key as encryption parameters, the upper-layer participating node performs two-party joint secure computation with k lower-layer participating nodes to obtain k two-party joint outputs of the federated model.

[0075] Correspondingly, using the first public key and the second public key as encryption parameters, each lower-layer participating node performs two-party joint secure computation with the upper-layer participating node to obtain one two-party joint output of the federated model.

[0076] The first public key and the second public key are used to encrypt the intermediate computation results between the upper-layer participating node and the lower-layer participating nodes.

[0077] The two-party joint secure computation includes the forward computation performed by the upper-layer participating node and the lower-layer participating nodes on their respective sub-models based on the homomorphic encryption method using their respective data. In the model training stage, the data refers to the feature data X owned by the participating nodes; in the model prediction stage, the data refers to the data to be predicted by the participating nodes.

[0078] Step 304: The upper-layer participating node merges the k two-party joint outputs to obtain the first joint model output corresponding to the upper-layer participating node and the k lower-layer participating nodes.

[0079] The first joint model output is the output after performing forward computation on the k sub-models according to the feature data X corresponding to the upper-layer participating node and the k lower-layer participating nodes.

[0080] In summary, the method provided in this embodiment, by using multiple participating nodes deployed in a multi-way tree topology, where one upper-layer participating node has k lower-layer participating nodes, after the upper-layer participating node and the k lower-layer participating nodes exchange their public keys, the upper-layer participating node and the lower-layer participating nodes perform two-party joint secure computation using the first public key and the second public key as encryption parameters to obtain k two-party joint outputs of the federated model; and then the upper-layer participating node merges the k two-party joint outputs to obtain the first joint model output corresponding to the federated model. Thus, a vertical federated learning architecture with a multi-way tree topology is provided, which improves the equality of each participating node in the process of vertical federated learning. Since the upper-layer participating node and the lower-layer participating nodes exchange their public keys with each other, and the upper-layer participating node and the lower-layer participating nodes each keep their own private keys, the security risks can be evenly distributed among each participating node.

[0081] Figure 4The figure shows a flowchart for performing joint secure computation between two participating nodes provided by an exemplary embodiment of the present application. The upper-layer participating node P0 and the i-th lower-layer participating node P i ( Figure 4 taking the first lower-layer participating node as an example) perform two-party joint secure computation, where the value range of i is from 1 to k. The method includes the following sub-steps:

[0082] Step 1: The upper-layer participating node P0 generates a first public key PK0 and a first private key SK0, and the i-th lower-layer participating node P i generates a second public key PK i and a second private key SK i ;

[0083] Step 2: The upper-layer participating node P0 sends the first public key PK0 to the i-th lower-layer participating node P i , and the i-th lower-layer participating node P i sends the second public key PK i to the upper-layer participating node P0;

[0084] The i-th lower-layer participating node P i saves the first public key PK0, and the upper-layer participating node P0 saves the second public key PK i .

[0085] Steps 1 and 2 are the processes of the above-mentioned steps 301 and 302, which will not be elaborated here. Step 303 may optionally include the following sub-steps:

[0086] Step 3: The upper-layer participating node P0 randomly generates a first network parameter W0 and a first random mask R i , and the i-th lower-layer participating node P i randomly generates a second network parameter W i and a second random mask R 0,i ;

[0087] Among them, the first network parameter W0 is the network parameter of the sub-model locally deployed by the upper-layer participating node P0. The second network parameter W i is the network parameter of the sub-model locally deployed by the i-th lower-layer participating node P i ;

[0088] Since the first random masks R i are all generated by the upper-layer participating node P0, the first random masks generated by the upper-layer participating node P0 for different lower-layer participating nodes can be the same or different. Since the second random masks R 0,i are generated by each lower-layer participating node, the second random masks generated by different lower-layer participating nodes for the upper-layer participating node P0 are generally different.

[0089] Step 4: The upper-layer participating node P0 encrypts the first random mask R using the first public key PK0 based on the homomorphic encryption technology i to obtain the first encrypted value PK0(R i ); The i-th lower-layer participating node P i encrypts the second random mask R i using the second public key PK 0，i to obtain the second encrypted value PK i (R 0，i );

[0090] Step 5: The upper-layer participating node P0 sends the first encrypted value PK0(R i ) to the i-th lower-layer participating node P i , and the i-th lower-layer participating node sends the second encrypted value PK i (R 0，i ) to the upper-layer participating node;

[0091] The i-th lower-layer participating node P i receives the first encrypted value PK0(R i ) sent by the upper-layer participating node P0, and the upper-layer participating node P0 receives the second encrypted value PK i (R i ) sent by the i-th lower-layer participating node P 0，i .

[0092] Step 6: The upper-layer participating node P0 calculates the product of the second encrypted value PK i (R 0，i ) and the first network parameter W0, i.e., PK i (R 0，i )·W0. The i-th lower-layer participating node calculates the product of the first encrypted value PK0(R i ) and the second network parameter W i , i.e., PK i (R 0，i )·W i .

[0093] The second network parameter is the network parameter of the sub-model locally deployed by the i-th lower-layer participating node.

[0094] Step 7: The upper-layer participating node P0 generates the second random number r 0,i , and the i-th lower-layer participating node P i generates the first random number r i .

[0095] Since the second random numbers are all generated by the upper-layer participating node P0, the first random masks generated by the upper-layer participating node P0 for different lower-layer participating nodes can be the same or different. Since the first random numbers are generated by each lower-layer participating node, generally the first random numbers generated by different lower-layer participating nodes for the upper-layer participating node P0 are not the same.

[0096] Step 8: The upper-layer participating node P0 sends the third encrypted value to the i-th lower-layer participating node P i and receives the fourth encrypted value sent by the i-th lower-layer participating node P i .

[0097] The third encrypted value is the value after the first data X0 is processed using the second encrypted value PK i (R 0,i ) and the second random number r 0,1 . Exemplarily, the third encrypted value is PK i (R 0,i )·X0 - r 0,1 .

[0098] The fourth encrypted value is the value after the second data X i is encrypted using the first encrypted value PK0(R i ) and the first random number r i . Exemplarily, the fourth encrypted value is PK0(R i )·X i - r i - r i .

[0099] Step 9: The upper-layer participating node P0 decrypts the fourth encrypted value PK i (R 0,i )·X i - r i using the first private key to obtain the masked value R i X i - r i of the second data, and the i-th lower-layer participating node P i decrypts the third encrypted value using the second private key to obtain the masked value R0X0 - r 0,i .

[0100] Step 10: The upper-layer participating node P0 calculates the first local output s1, and the i-th lower-layer participating node P i calculates the second local output s2;

[0101] The upper-layer participating node P0 calculates the first local output s1;

[0102]

[0103] The i-th lower-layer participating node P i Calculate the second-party output s2;

[0104]

[0105] It should be noted that the weight 1 / k in the above formula is an optional weight. In some embodiments, it is also possible to achieve without this weight. That is:

[0106] The upper-layer participating node P0 calculates the first-party output s1;

[0107] s1 = (W0X0 + r 0，1 ) + R1X1 - r1;

[0108] The i-th lower-layer participating node P i Calculate the second-party output s2;

[0109] s2 = (R 0,1 X0 - r 0,1 ) + W1X1 + r1;

[0110] Step 11: The i-th lower-layer participating node P i Report the second-party output s2 to the upper-layer participating node P0;

[0111] Step 12: The upper-layer participating node P0 adds the first-party output and the second-party output to obtain the i-th two-party joint output z 0,i .

[0112]

[0113] Or,

[0114] z 0,i = (W0 + R 0,i )X0 + (W i + R i )X i , i = 1, 2,..., k;

[0115] In a possible design, the two-party joint secure computation between the upper-layer participating node and the k lower-layer participating nodes is executed in parallel.

[0116] In summary, the method provided in this embodiment provides a two-party joint secure computation method in which the relationship between the upper-layer participating node and the lower-layer participating node is basically equal by adopting a multiple encryption mechanism of public keys, random masks, and random numbers, and at the same time can ensure the security of the feature data between the upper-layer participating node and the lower-layer participating node.

[0117] Based on Figure 3In an alternative embodiment, when deploying N participating nodes according to a multi - fork tree topology, there are at least two deployment methods:

[0118] · Two - layer multi - fork tree topology

[0119] The upper - layer participating nodes are the root nodes among the two - layer participating nodes, and the lower - layer participating nodes are the leaf nodes among the two - layer participating nodes. Each leaf node is connected to the root node.

[0120] As Figure 5 shown, 1 participating node P0 serves as the upper - layer participating node, that is, the root node. N - 1 participating nodes P i serve as the lower - layer participating nodes, where i = 1, 2, …, N - 1. There is a network connection between the root node and each lower - layer participating node.

[0121] · Multi - layer multi - fork tree topology

[0122] The N participating nodes are divided into at least a three - layer multi - fork tree topology. Adjacent two - layer participating nodes include an upper - layer participating node and a lower - layer participating node. The upper - layer participating node is the participating node at the higher level among the adjacent two - layer participating nodes, and the lower - layer participating node is the participating node at the lower level among the adjacent two - layer participating nodes. Each lower - layer participating node is connected to its corresponding upper - layer participating node. Each upper - layer participating node includes at least two lower - layer participating nodes. As Figure 6 shown, taking the deployment of 7 participating nodes with a three - layer multi - fork tree topology as an example, the participating node P0 is the first - layer participating node, that is, the root node; the participating nodes P1 and P2 are the second - layer participating nodes; the participating nodes P3, P4, P5, and P6 are the third - layer participating nodes.

[0123] The participating node P0 is the upper - layer participating node of the participating nodes P1 and P2, and the participating nodes P1 and P2 are the lower - layer participating nodes of the participating node P0.

[0124] The participating node P1 is the upper - layer participating node of the participating nodes P3 and P4, and the participating nodes P3 and P4 are the lower - layer participating nodes of the participating node P1. The participating node P2 is the upper - layer participating node of the participating nodes P5 and P6, and the participating nodes P5 and P6 are the lower - layer participating nodes of the participating node P1.

[0125] For the model training process of the two - layer multi - fork tree topology, the following embodiments are provided:

[0126] Figure 7 shows a flowchart of a calculation method for vertical federated learning provided by an exemplary embodiment of the present application. This method is applied to Figure 5Among the upper-layer participating nodes and N - 1 lower-layer participating nodes shown. Taking the federated model as a Logistic Regression (LogR) model as an example, the method includes:

[0127] Phase 1: Public key distribution;

[0128] Step 701, the upper-layer participating node P0 generates the first public key PK0 and the first private key SK0, and sends the first public key PK0 to the N - 1 lower-layer participating nodes P i Send the first public key PK0;

[0129] The upper-layer participating node P0 generates a public key and private key pair (PK0, SK0), and sends the first public key PK0 to each lower-layer participating node P i . The N - 1 lower-layer participating nodes P i Receive the first public key PK0 sent by the upper-layer participating node P0, and the N - 1 lower-layer participating nodes P i Save the first public key PK0.

[0130] The first public key PK0 is used for additive homomorphic encryption of the intermediate calculation results. For example, the Paillier homomorphic encryption algorithm is used.

[0131] Step 702, each lower-layer participating node P i Generates the second public key PK i And the second private key SK i， Sends the second public key PK to the upper-layer participating node P0 i ;

[0132] The upper-layer participating node P0 saves the N - 1 public keys PK i , i = 1, 2,..., N - 1. Schematically, the N - 1 second public keys PK i Are all different.

[0133] The second public key PK i Is used for additive homomorphic encryption of the intermediate calculation results. For example, the Paillier homomorphic encryption algorithm is used.

[0134] Phase 2: Encrypted sample alignment;

[0135] Step 703, each participating node performs encrypted sample alignment;

[0136] Suppose the participating node P i Has the training feature data set X i , i = 0, 1, 2,..., n - 1. Among them, the participating node P0 has the label information y, and the participating node P0 may not have feature data, that is, X0 is empty.

[0137] The N participating nodes in vertical federated learning need to align the training samples they each possess, filter out the ID intersection of the training samples they each possess, that is, obtain the intersection of the training samples with the same sample ID in multiple training feature datasets X i , i = 0, 1, 2, …, N - 1, and cannot disclose the training samples in the non - intersection. This step is the alignment of the training samples of multiple participating nodes, and an algorithm based on the Freedman protocol can be used.

[0138] Phase 3: Forward model output:

[0139] Step 704, using the first public key and the second public key as encryption parameters, the upper - layer participating node P0 and each lower - layer participating node P i perform two - party joint secure computation to obtain the two - party joint output z i between the upper - layer participating node P0 and each lower - layer participating node P 0,i ;

[0140]

[0141] Among them, {W i , i = 0, 1, 2, …, N - 1} are the parameters of the federated model.

[0142] The upper - layer participating node P0 and each lower - layer participating node P i The algorithm for performing two - party joint secure computation is not limited, and any two - party joint secure computation method can be used. In this application, the two - party joint secure computation is exemplified by using the Figure 4 steps 3 to 12 shown.

[0143] It should be noted that the process of the upper - layer participating node P0 and each lower - layer participating node P i calculating {z 0,i , i = 1, 2, …, N - 1} can be executed in parallel on the (N - 1) lower - layer participating nodes P i to reduce the training time of the federated model and improve the efficiency of multi - party federated learning.

[0144] Step 705, the upper - layer participating node P0 calculates the multi - party joint output z 0,i ;

[0145] After the upper - layer participating node P0 obtains the plain - text form of {z i , i = 1, 2, …, N - 1} from each lower - layer participating node P 0,i , the participating node P0 can calculate the multi - party joint output z corresponding to the N participating nodes:

[0146]

[0147] In another possible implementation, the calculation formula of the above z is as follows:

[0148]

[0149] That is, the in the previous implementation is an optional weight.

[0150] Furthermore, the upper-layer participating node P0 can calculate the multi-party joint output of the federated model as:

[0151]

[0152] where sigmoid is the S-shaped function, also known as the activation function. E is the natural constant.

[0153] Phase 4: Backward error propagation:

[0154] Step 706, the upper-layer participating node P0 calculates the forward prediction error δ of the federated model according to the difference between the multi-party joint output and the label information y;

[0155]

[0156] Step 707, the upper-layer participating node P0 iteratively updates the first network parameter according to the forward prediction residual δ;

[0157] After obtaining the forward prediction residual δ, the upper-layer participating node P i locally calculates the gradient of the loss function of the federated model with respect to the first network parameter W0. For the federated logistic regression model, the gradient of the loss function with respect to the first network parameter W0 is: g0 = δX0.

[0158] The upper-layer participating node P0 locally updates the first network parameter of the federated model: W0 ← W0 - ηg0. Where η is the learning rate. For example, η = 0.01.

[0159] Step 708, the upper-layer participating node P0 sends the forward prediction residual δ to each lower-layer participating node P i ;

[0160] Step 709, the lower-layer participating node P i iteratively updates the second network parameter according to the forward prediction residual δ;

[0161] Schematically, the upper-layer participating node P0 sends the forward prediction residual to the lower-layer participating node P i , i = 1, 2,..., N - 1, in plaintext or ciphertext.

[0162] After obtaining the forward prediction residual δ, the lower-layer participating node Pi Calculate the gradient of the loss function of the federated model with respect to the second network parameter W i locally. For a logistic regression model, the gradient of the loss function with respect to the second network parameter W i is: g i = δX i , where i = 1, 2, …, N - 1.

[0163] The lower-level participating node P i updates the second network parameter of the federated model locally: W i ← W i - ηg i . Here, η is the learning rate. For example, η = 0.01.

[0164] Step 710, when the iteration end condition is not met, execute again from the above step 704;

[0165] The iteration end condition includes: the number of iterations is greater than the maximum number of iterations, or the network parameters of the federated model converge.

[0166] Step 711, when the iteration end condition is met, stop the above training process.

[0167] In summary, the method provided in this embodiment, by using multiple participating nodes deployed in a multi-way tree topology, where an upper-level participating node has k lower-level participating nodes. After the upper-level participating node and the k lower-level participating nodes exchange their public keys, the upper-level participating node and the lower-level participating nodes perform two-party joint secure computation using the first public key and the second public key as encryption parameters to obtain k two-party joint outputs of the federated model; then the upper-level participating node merges the k two-party joint outputs to obtain the first joint output corresponding to the federated model. Thus, a vertical federated learning architecture with a multi-way tree topology is provided, which improves the equality of each participating node in the vertical federated learning process. Since the upper-level participating node and the lower-level participating nodes exchange their public keys with each other and each keeps its own private key, the security risks can be evenly distributed among the participating nodes.

[0168] The method provided in this embodiment also constructs a root node and N - 1 leaf nodes in a star topology, which is easy to implement in practical scenarios. Just any participating node with label information can be used as the root node to communicate with other participating nodes, and the entire vertical federated learning process can be completed equally.

[0169] In an alternative embodiment based on Figure 7 , when an upper-level participating node cannot meet the computing and communication capabilities requirements, multiple different upper-level participating nodes can be used to share the computing and communication tasks, and a topology structure such as a multi-layer multi-way tree can be used to organize multiple participating nodes.

[0170] For the model training process of a multi - layer and multi - branch tree topology, the following embodiments are provided:

[0171] Figure 8 The flowchart of the calculation method for vertical federated learning provided by an exemplary embodiment of the present application is shown. This method is applied to Figure 6 the upper - layer participating nodes and lower - layer participating nodes shown. This method includes:

[0172] Phase 1: Public key distribution;

[0173] Step 801, the upper - layer participating node generates a first public key and a first private key, and sends the first public key to k lower - layer participating nodes;

[0174] Taking the second - layer participating node as the upper - layer participating node as an example, the upper - layer participating nodes include: participating node P1 and participating node P2.

[0175] For the upper - layer participating node P1, the upper - layer participating node P1 generates a public - key and private - key pair (PK1, SK1), and sends the first public key PK1 to the lower - layer participating nodes P3 and P4. The lower - layer participating nodes P3 and P4 receive the first public key PK1 sent by the upper - layer participating node P1, and the lower - layer participating nodes P3 and P4 save the first public key PK1.

[0176] The first public key PK1 is used for additive homomorphic encryption of the intermediate calculation result. For example, the Paillier homomorphic encryption algorithm is used.

[0177] For the upper - layer participating node P2, the upper - layer participating node P2 generates a public - key and private - key pair (PK2, SK2), and sends the first public key PK2 to the lower - layer participating nodes P5 and P6. The lower - layer participating nodes P5 and P6 receive the first public key PK2 sent by the upper - layer participating node P2, and the lower - layer participating nodes P5 and P6 save the first public key PK2.

[0178] The first public key PK2 is used for additive homomorphic encryption of the intermediate calculation result. For example, the Paillier homomorphic encryption algorithm is used.

[0179] Step 802, each lower - layer participating node P i generates a second public key PK i and a second private key SK i， and sends the second public key PK i to the upper - layer participating node;

[0180] For the upper-layer participating node P1, the lower-layer participating node P3 generates a public-private key pair (PK3, SK3) and sends the second public key PK1 to the upper-layer participating node P1. The lower-layer participating node P4 generates a public-private key pair (PK4, SK4) and sends the second public key PK4 to the upper-layer participating node P1. The upper-layer participating nodes P1 and P4 receive the second public key PK3 sent by the lower-layer participating node P3 and the second public key PK4 sent by the lower-layer participating node P4. The upper-layer participating node P1 stores the second public keys PK3 and PK4.

[0181] The second public keys PK3 and PK4 are used for additive homomorphic encryption of the intermediate calculation results. For example, the Paillier homomorphic encryption algorithm is used.

[0182] For the upper-layer participating node P2, the lower-layer participating node P5 generates a public-private key pair (PK5, SK5) and sends the second public key PK2 to the upper-layer participating node P2. The lower-layer participating node P6 generates a public-private key pair (PK6, SK6) and sends the second public key PK6 to the upper-layer participating node P2. The upper-layer participating nodes P2 and P6 receive the second public key PK5 sent by the lower-layer participating node P5 and the second public key PK6 sent by the lower-layer participating node P6. The upper-layer participating node P2 stores the second public keys PK5 and PK6.

[0183] The second public keys PK5 and PK6 are used for additive homomorphic encryption of the intermediate calculation results. For example, the Paillier homomorphic encryption algorithm is used.

[0184] Phase 2: Encrypted sample alignment;

[0185] Step 803, each participating node performs encrypted sample alignment;

[0186] Suppose the participating node P i owns the training feature dataset X i , i = 0, 1, 2, …, N - 1. Among them, the participating node P0 has the label information y, and the participating node P0 may not have feature data, that is, X0 is empty.

[0187] The N participating nodes in vertical federated learning need to align their respective owned training samples, filter out the ID intersection of their respective owned training samples, that is, find the intersection of the training samples with the same sample ID in multiple training feature datasets X i , i = 0, 1, 2, …, N - 1, and non-intersection training samples cannot be leaked. This step is the alignment of the training samples of multiple participating nodes, and an algorithm based on the Freedman protocol can be used.

[0188] Phase 3: Forward model output:

[0189] Step 804: Using the first public key and the second public key as encryption parameters, the upper-layer participating node and each lower-layer participating node perform two-party joint secure computation to obtain the two-party joint output z j between the upper-layer participating node P i and each lower-layer participating node P j,i ;

[0190]

[0191] where {W i , i = 0, 1, 2, …, k} are the parameters of the federated model. Similarly, W j are also the parameters of the federated model.

[0192] The upper-layer participating node P1 performs two-party joint secure computation with the lower-layer participating nodes P3 and P4 respectively to obtain the sub-model outputs z 1,3 and z 1,4 . Wherein:

[0193]

[0194] The upper-layer participating node P2 performs two-party joint secure computation with the lower-layer participating nodes P5 and P6 respectively to obtain the sub-model outputs z 25 and z 26 . Wherein:

[0195]

[0196] The algorithm for the upper-layer participating node P j to perform two-party joint secure computation with each lower-layer participating node P i is not limited, and any two-party joint secure computation method can be used. In this application, it is exemplified by using the steps 3 to 12 shown in Figure 4 .

[0197] It should be noted that the process of the upper-layer participating node and each lower-layer participating node calculating the two-party joint secure computation can be executed in parallel on the K lower-layer participating nodes to reduce the training time of the federated model and improve the efficiency of multi-party federated learning.

[0198] Step 805: The upper-layer participating node calculates the multi-party joint output;

[0199] The upper-layer participating node P1 combines the sub-model outputs z 1,3 and z 1,4 to obtain the multi-party joint output Z1 corresponding to the participating node P1, the participating node P3, and the participating node P4. The multi-party joint output Z1 represents the multi-party joint output of k + 1 = 3 participating parties.

[0200] The upper-layer participating node P2 combines the sub-model output z2,5 and z 2,6 to obtain the multi-party joint output Z2 corresponding to the participating node P2, the participating node P5, and the participating node P6. The multi-party joint output Z2 represents the multi-party joint output of k + 1 = 3 participating parties.

[0201] Step 806, when the upper-layer participating node is the lower-layer participating node of other upper-layer participating nodes in the multi-way tree topology, report the multi-party joint output to other upper-layer participating nodes;

[0202] Other upper-layer participating nodes are the upper-layer participating nodes of this upper-layer participating node. The upper-layer participating node reports the first joint output to other upper-layer participating nodes in plaintext or ciphertext.

[0203] The upper-layer participating node P1 reports the first joint output Z1 to other upper-layer participating nodes P0.

[0204] The upper-layer participating node P2 reports the first joint output Z2 to other upper-layer participating nodes P0.

[0205] Step 807, other upper-layer participating nodes merge according to their own single-party model output and the multi-party joint model output of each subordinate lower-layer participating node to obtain the multi-party joint model output corresponding to other upper-layer participating nodes;

[0206] After the upper-layer participating node P0 obtains z1 and z2 in plaintext form, it merges its own single-party model output W0X0 and calculates the multi-party joint output corresponding to the upper-layer participating node P0 and all subordinate lower-layer participating nodes: z = z1 + z2 + W0X0.

[0207] This multi-party joint output represents the joint output of 7 participating nodes.

[0208] Further, the upper-layer participating node P0 can calculate the multi-party joint output of the federated model as:

[0209]

[0210] where sigmoid is the S-shaped function, also known as the activation function. E is the natural constant.

[0211] Phase 4: Backward error propagation:

[0212] Step 808, when the upper-layer participating node is the root node with label information, the upper-layer participating node calculates the forward prediction error of the federated model according to the difference between the multi-party joint model output and the label information;

[0213] The upper-layer participating node P0 calculates the forward prediction error δ of the federated model according to the difference between the multi-party joint output and the label information y;

[0214]

[0215] Step 809. The upper-layer participating node P0 iteratively updates the first network parameter according to the forward prediction residual δ.

[0216] After obtaining the forward prediction residual δ, the upper-layer participating node P i locally calculates the gradient of the loss function of the federated model with respect to the first network parameter W0. For the federated model, the gradient of the loss function with respect to the first network parameter W0 is: g0 = δX0.

[0217] The upper-layer participating node P0 locally updates the first network parameter of the federated model: W0 ← W0 - ηg0. Here, η is the learning rate. For example, η = 0.01.

[0218] Step 810. The upper-layer participating node P0 sends the forward prediction residual δ to each lower-layer participating node P i ;

[0219] Step 811. The lower-layer participating node P i iteratively updates the second network parameter according to the forward prediction residual δ;

[0220] Illustratively, the upper-layer participating node P0 sends the forward prediction residual to the lower-layer participating node P i , i = 1, 2, …, N - 1.

[0221] After obtaining the forward prediction residual δ, the lower-layer participating node P i locally calculates the gradient of the loss function of the federated model with respect to the second network parameter W i For the federated model, the gradient of the loss function with respect to the second network parameter W i is: g i = δX i , i = 0, 1, 2, …, N - 1.

[0222] The lower-layer participating node P i locally updates the second network parameter of the federated model: W i ← W i - ηg i . Here, η is the learning rate. For example, η = 0.01.

[0223] Step 812. When the iteration end condition is not satisfied, execute again from the above step 806;

[0224] The iteration end condition includes: the number of iterations is greater than the maximum number of iterations, or the network parameters of the federated model converge.

[0225] Step 813, when the iteration end condition is met, stop the above training process.

[0226] It should be noted that since the outputs of participating nodes P1 and P2 already mix the outputs of multiple parties, in the above embodiments, there is no need for participating node P0 to share public keys with participating nodes P1 and P2, nor is it necessary to use additive homomorphic encryption or random masking. Plaintext calculation can be directly used. Of course, participating node P0 and participating nodes P1 and P2 can also choose to continue using the two-party joint secure computing method to calculate z. That is, participating node P0 and participating nodes P1 and P2 still share public keys and random masks and continue to use the two-party joint secure computing method as shown in Figure 4 to calculate z. The difference is only that participating node P1 regards the three-party joint output of "participating nodes P1, P3, P4" as its own model output, and participating node P2 regards the three-party joint output of "participating nodes P2, P5, P6" as its own model output.

[0227] In summary, the method provided in this embodiment can distribute the calculation and communication to different upper-level participating nodes by deploying N participating nodes using a multi-layer multi-way tree topology, thereby reducing the calculation and communication pressure on the participating node as the root node.

[0228] At the same time, since the upper-level participating nodes have aggregated the multi-party joint outputs of k + 1 parties, when communicating between upper-level participating nodes and other upper-level participating nodes, there is no need for encrypted communication and encrypted computing methods, thereby reducing the computing overhead and communication overhead of each upper-level participating node.

[0229] Figure 7 and Figure 8 The embodiment has been schematically described with the model training process. In the model prediction stage, only the feature data X needs to be replaced with the input data, and the same process as above is executed until the forward calculation process, and there is no need to execute the backward error backpropagation process.

[0230] In a schematic example, the above method is applied in the financial field, and the federated model is a financial risk control model constructed using a multiple linear regression equation. The label information is whether a user has a repayment overdue record. Each participating node represents a different company, such as a bank, an e-commerce company, an instant messaging company, and the enterprise where the user works. The feature data includes but is not limited to: the basic attribute information and deposit records of the user (bank), the shopping records of the user (e-commerce company), the social relationship chain of the user (instant messaging company), and the salary data of the user (employing enterprise).

[0231] Since vertical federated learning enables data cooperation across departments, companies, and even industries, while meeting the requirements of data protection laws and regulations, the application scenarios of the above vertical federated learning are not limited in this application.

[0232] Figure 9 A vertical federated learning system involved in an exemplary embodiment of this application can be a distributed system formed by connecting a client and multiple participating nodes (any form of computing device accessing the network, such as a server or a user terminal) through network communication. Taking the distributed system as a blockchain system as an example, refer to Figure 9 , Figure 9 FIG. 100 is a schematic structural diagram of a distributed system 100 applied to a blockchain system provided by an exemplary embodiment of this application, which is formed by multiple participating nodes (any form of computing device accessing the network, such as a server or a user terminal) 200 and a client 300. A peer-to-peer (P2P) network is formed among the participating nodes 200. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). In the distributed system, any machine such as a server or a terminal can join and become a participating node. The participating node includes a hardware layer, an intermediate layer, an operating system layer, and an application layer.

[0233] Refer to Figure 9 which shows the functions of each participating node 200 in the blockchain system. The functions involved include:

[0234] 1) Routing, which is a basic function of the participating node 200 and is used to support communication between the participating nodes 200.

[0235] In addition to the routing function, the participating node 200 may also have the following functions:

[0236] 2) Application, which is used to be deployed in the blockchain, implement specific services according to actual business requirements, record the data related to the implemented functions to form record data, carry a digital signature in the record data to indicate the source of the task data, and send the record data to other participating nodes 200 in the blockchain system. When other participating nodes 200 verify the source and integrity of the record data successfully, the record data is added to the temporary block.

[0237] For example, the services implemented by the application include but are not limited to:

[0238] 2.1) Wallet (or other services) for providing functions for conducting transactions of electronic currency, including initiating a transaction (i.e., sending the transaction record of the current transaction to other participating nodes in the blockchain system. After successful verification by other participating nodes, as a response acknowledging the validity of the transaction, the record data of the transaction is deposited into the temporary block of the blockchain. Of course, the wallet also supports querying the remaining electronic currency in the electronic currency address;

[0239] 2.2) Shared ledger for providing functions such as storage, query, and modification of account data. The record data of the operations on the account data is sent to other participating nodes in the blockchain system. After successful verification by other participating nodes, as a response acknowledging the validity of the account data, the record data is deposited into the temporary block, and a confirmation can also be sent to the participating node that initiated the operation.

[0240] 2.3) Smart contract, a computerized protocol that can execute the terms of a certain contract, implemented by code deployed on the shared ledger and executed when certain conditions are met. According to actual business requirements, the code is used to complete automated transactions, such as querying the logistics status of the goods purchased by the buyer and transferring the buyer's electronic currency to the merchant's address after the buyer signs for the goods. Of course, smart contracts are not limited to executing contracts for transactions but can also execute contracts for processing received information.

[0241] 3) Blockchain, including a series of blocks (Blocks) connected in sequence according to the chronological order of generation. Once a new block is added to the blockchain, it will not be removed again. The block records the record data submitted by participating nodes in the blockchain system.

[0242] See Figure 10 , Figure 10 is an optional schematic diagram of the block structure provided by an exemplary embodiment of the present application. Each block includes the hash value of the transaction record stored in this block (the hash value of this block) and the hash value of the previous block. The blocks are connected through hash values to form a blockchain. In addition, the block may also include information such as the timestamp when the block is generated. Blockchain, essentially a decentralized database, is a string of data blocks associated using cryptographic methods. Each data block contains relevant information for verifying the validity of its information (anti-counterfeiting) and generating the next block.

[0243] Figure 11The block diagram of an upper-layer participating node (or upper-layer participating device) provided by an exemplary embodiment of the present application is shown. The upper-layer participating node has k lower-layer participating nodes among a plurality of participating nodes deployed in a multi-way tree topology. A sub-model in the federated model is locally deployed in each participating node, and k is an integer greater than 1. The upper-layer participating node includes:

[0244] A communication module 1120, configured to distribute the first public key corresponding to the upper-layer participating node to the k lower-layer participating nodes, and obtain k second public keys respectively corresponding to the k lower-layer participating nodes;

[0245] A joint calculation module 1140, configured to perform two-party joint secure calculation with the k lower-layer participating nodes using the first public key and the second public key as encryption parameters, to obtain k two-party joint outputs of the federated model; the two-party joint secure calculation includes the forward calculation performed by the upper-layer participating node and the lower-layer participating node on the sub-model using their respective data based on the homomorphic encryption method; and merge the k two-party joint outputs to obtain the joint model output corresponding to the upper-layer participating node and the k lower-layer participating nodes.

[0246] In an alternative design of this embodiment, the joint calculation module 1140 is configured to:

[0247] Randomly generate first network parameters and a first random mask, where the first network parameters are the network parameters of the sub-model locally deployed by the upper-layer participating node;

[0248] Send a first encrypted value to the i-th lower-layer participating node, and receive a second encrypted value sent by the i-th lower-layer participating node. The first encrypted value is obtained by encrypting the first random mask using the first public key based on the homomorphic encryption technology, and the second encrypted value is obtained by encrypting the second random mask using the i-th second public key based on the homomorphic encryption technology. The value range of i is from 1 to k;

[0249] Send a third encrypted value to the i-th lower-layer participating node, and receive a fourth encrypted value sent by the i-th lower-layer participating node. The third encrypted value is the value obtained by encrypting the first data after encrypting the second encrypted value and the second random number, and the fourth encrypted value is the value obtained by encrypting the second data by the i-th lower-layer participating node using the first encrypted value and the first random number. The first data is the data in the upper-layer participating node, and the second data is the data in the i-th lower-layer participating node; the second random number is generated by the upper-layer participating node, and the first random number is generated by the lower-layer participating node;

[0250] Calculate the first local output, and receive the second local output sent by the $i$-th lower-layer participating node. The first local output is calculated based on the decrypted value of the fourth encrypted value, the data of the upper-layer participating node, the first network parameter, and the second random number. The second local output is calculated by the $i$-th lower-layer node using the decrypted value of the third encrypted value, the data of the $i$-th lower-layer participating node, the second network parameter, and the first random number;

[0251] Add the first local output and the second local output to obtain the $i$-th two-party joint output.

[0252] In an alternative design of this embodiment, the two-party joint secure computation between the upper-layer participating node and the $k$ lower-layer participating nodes is executed in parallel.

[0253] In an alternative design of this embodiment, the upper-layer participating node is a root node with label information. The apparatus further includes:

[0254] A backward error propagation module 1160, configured to calculate the forward prediction error of the federated model according to the difference between the multi-party joint model output and the label information; send the forward prediction error to the $k$ lower-layer participating nodes, where the forward prediction error is used for the $k$ lower-layer participating nodes to perform backward propagation to update the second network parameter of the sub-model in the $k$ lower-layer participating nodes.

[0255] In an alternative design of this embodiment, the upper-layer participating node is a lower-layer participating node of other upper-layer participating nodes in the multi-way tree topology. The communication module 1120 is further configured to report the multi-party joint model output to the other upper-layer participating nodes, and the other upper-layer participating nodes are configured to merge the multi-party joint model output corresponding to the other upper-layer participating nodes according to the multi-party joint model output, their own single-party model output, and the multi-party joint model output of other lower-layer participating nodes under them.

[0256] In an alternative design of this embodiment, the upper-layer participating node is a lower-layer participating node of other upper-layer participating nodes in the multi-way tree topology. The communication module 91120 is further configured to report the first public key to the other upper-layer participating nodes, and obtain the corresponding third public key of the other upper-layer participating nodes;

[0257] The joint computing module 1140 is configured to use the first public key and the third public key as encryption parameters. The upper-layer participating node performs two-party joint secure computation with the other upper-layer participating nodes with itself as a lower-layer participating node to obtain a two-party joint output of the federated model.

[0258] Figure 12 The block diagram of a lower-layer participating node (or lower-layer participating device) provided by an exemplary embodiment of the present application is shown. The lower-layer participating node has an upper-layer participating node among multiple participating nodes deployed in a multi-way tree topology. A sub-model in a federated model is locally deployed in each participating node. The lower-layer participating node includes:

[0259] A communication module 1220, configured to report a second public key of the lower-layer participating node to the upper-layer participating node, and obtain a first public key corresponding to the upper-layer participating node. The upper-layer participating node includes k lower-layer participating nodes, and k is a positive integer;

[0260] A joint computing module 1240, configured to perform two-party joint secure computing with the upper-layer participating node using the first public key and the second public key as encryption parameters to obtain a two-party joint output of the federated model. The two-party joint secure computing includes forward computing performed by the upper-layer participating node and the lower-layer participating node on the sub-model using their respective data based on a homomorphic encryption method.

[0261] In an alternative design of this embodiment, the joint computing module 1240 is configured to randomly generate a second network parameter and a second random mask. The second network parameter is the network parameter of the sub-model locally deployed by the lower-layer participating node;

[0262] Receive a first encrypted value sent by the upper-layer participating node, and send a second encrypted value to the upper-layer participating node. The first encrypted value is obtained by encrypting a first random mask using the first public key based on homomorphic encryption technology. The second encrypted value is obtained by encrypting the second random mask using the second public key based on homomorphic encryption technology;

[0263] Receive a third encrypted value sent by the upper-layer participating node, and send a fourth encrypted value to the upper-layer participating node. The third encrypted value is the value obtained by encrypting the first data after encrypting the second encrypted value and a second random number. The fourth encrypted value is the value obtained by encrypting the second data by the i-th lower-layer participating node using the first encrypted value and a first random number. The first data is the data in the upper-layer participating node, and the second data is the data in the i-th lower-layer participating node. The second random number is generated by the upper-layer participating node, and the first random number is generated by the lower-layer participating node;

[0264] Send the second local output to the upper-layer participating node, so that the upper-layer participating node adds the first local output and the second local output to obtain the i-th two-party joint output; the first local output is calculated based on the decryption value of the fourth encrypted value, the data of the upper-layer participating node, the first network parameter, and the second random number, and the second local output is calculated by the i-th lower-layer node using the decryption value of the third encrypted value, the data of the i-th lower-layer participating node, the second network parameter, and the first random number.

[0265] In an alternative design of this embodiment, the communication module 1220 is configured to receive the forward prediction error sent by the upper-layer participating node; the backward error propagation module 1260 is configured to perform backward propagation according to the forward prediction error to update the second network parameter of the sub-model in the lower-layer participating node.

[0266] It should be noted that the upper-layer participating node and the lower-layer participating node provided in the above embodiments are only illustrated by dividing the above functional modules. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the upper-layer participating node and the lower-layer participating node provided in the above embodiments belong to the same concept as the embodiments of the calculation method of vertical federated learning. The specific implementation process is detailed in the method embodiments and will not be repeated here.

[0267] This application also provides a computer device (terminal or server), which can be implemented as the above participating node. The computer device includes a processor and a memory, and at least one instruction is stored in the memory. The at least one instruction is loaded and executed by the processor to implement the calculation method of vertical federated learning provided in each of the above method embodiments. It should be noted that the computer device can be as follows Figure 13 the computer device provided.

[0268] Figure 13The structural block diagram of a computer device 1300 provided by an exemplary embodiment of the present application is shown. The computer device 1300 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a notebook computer, or a desktop computer. The computer device 1300 may also be referred to by other names such as a user device, a portable computer device, a laptop computer device, a desktop computer device, etc.

[0269] Generally, the computer device 1300 includes: a processor 1301 and a memory 1302.

[0270] The processor 1301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1301 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), PLA (Programmable Logic Array). The processor 1301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1301 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1301 may also include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0271] The memory 1302 may include one or more computer-readable storage media, which may be non-transitory. The memory 1302 may also include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1302 is used to store at least one instruction for being executed by the processor 1301 to implement the computing method of vertical federated learning provided in the method embodiments of this application.

[0272] In some embodiments, the computer device 1300 may further optionally include: a peripheral device interface 1303 and at least one peripheral device. The processor 1301, the memory 1302, and the peripheral device interface 1303 may be connected through a bus or signal lines. Each peripheral device may be connected to the peripheral device interface 1303 through a bus, signal lines, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 1304, a touch display screen 1305, a camera 1306, an audio circuit 1307, and a power supply 1308.

[0273] The peripheral device interface 1303 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1301 and the memory 1302. In some embodiments, the processor 1301, the memory 1302, and the peripheral device interface 1303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1301, the memory 1302, and the peripheral device interface 1303 may be implemented on a separate chip or circuit board, and this embodiment does not limit this.

[0274] The radio frequency circuit 1304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1304 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1304 converts electrical signals into electromagnetic signals for transmission, or converts the received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 1304 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so on. The radio frequency circuit 1304 can communicate with other computer devices through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1304 may further include a circuit related to NFC (Near Field Communication), which is not limited in this application.

[0275] The display screen 1305 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1305 is a touch display screen, the display screen 1305 also has the ability to collect touch signals on or above the surface of the display screen 1305. The touch signals can be input to the processor 1301 as control signals for processing. At this time, the display screen 1305 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1305, which is set on the front panel of the computer device 1300; in other embodiments, there may be at least two display screens 1305, which are respectively set on different surfaces of the computer device 1300 or are in a foldable design; in still other embodiments, the display screen 1305 may be a flexible display screen, which is set on a curved surface or a foldable surface of the computer device 1300. Even, the display screen 1305 can be set as an irregular non-rectangular shape, that is, a special-shaped screen. The display screen 1305 can be prepared using materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0276] The camera component 1306 is used to collect images or videos. Optionally, the camera component 1306 includes a front camera and a rear camera. Generally, the front camera is disposed on the front panel of the computer device, and the rear camera is disposed on the back of the computer device. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth camera, a wide-angle camera, and a telephoto camera, so as to implement the background blurring function by fusing the main camera and the depth camera, the panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions by fusing the main camera and the wide-angle camera. In some embodiments, the camera component 1306 may further include a flash. The flash may be a single-color temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0277] The audio circuit 1307 may include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals and input them to the processor 1301 for processing, or input them to the radio frequency circuit 1304 to implement voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively disposed at different parts of the computer device 1300. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signals from the processor 1301 or the radio frequency circuit 1304 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for uses such as ranging. In some embodiments, the audio circuit 1307 may further include a headphone jack.

[0278] The power supply 13098 is used to supply power to each component in the computer device 1300. The power supply 1308 may be alternating current, direct current, a disposable battery or a rechargeable battery. When the power supply 1308 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charging technology.

[0279] In some embodiments, the computer device 1300 further includes one or more sensors 1310. The one or more sensors 1310 include but are not limited to: an acceleration sensor 13131, a gyroscope sensor 1312, a pressure sensor 1313, an optical sensor 1314, and a proximity sensor 1315.

[0280] The acceleration sensor 1311 can detect the magnitudes of accelerations on the three coordinate axes of the coordinate system established by the computer device 1300. For example, the acceleration sensor 1311 can be used to detect the components of the gravitational acceleration on the three coordinate axes. The processor 1301 can control the touch display screen 1305 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signals collected by the acceleration sensor 1311. The acceleration sensor 1311 can also be used for collecting game or user's motion data.

[0281] The gyroscope sensor 1312 can detect the body orientation and rotation angle of the computer device 1300. The gyroscope sensor 1312 can cooperate with the acceleration sensor 1313 to collect the 3D actions of the user on the computer device 1300. Based on the data collected by the gyroscope sensor 1312, the processor 1301 can implement the following functions: motion sensing (such as changing the UI according to the user's tilting operation), image stabilization during shooting, game control, and inertial navigation.

[0282] The pressure sensor 1313 can be disposed on the side frame of the computer device 1300 and / or the lower layer of the touch display screen 1305. When the pressure sensor 1313 is disposed on the side frame of the computer device 1300, it can detect the holding signal of the user on the computer device 1300, and the processor 1301 can perform left / right hand recognition or shortcut operations according to the holding signal collected by the pressure sensor 1313. When the pressure sensor 1313 is disposed on the lower layer of the touch display screen 1305, the processor 1301 can control the operable controls on the UI interface according to the pressure operation of the user on the touch display screen 1305. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0283] The optical sensor 1314 is used to collect the ambient light intensity. In one embodiment, the processor 1301 can control the display brightness of the touch display screen 1305 according to the ambient light intensity collected by the optical sensor 1314. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 1305 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 1305 is decreased. In another embodiment, the processor 1301 can also dynamically adjust the shooting parameters of the camera module 1306 according to the ambient light intensity collected by the optical sensor 1314.

[0284] The proximity sensor 1315, also known as the distance sensor, is usually disposed on the front panel of the computer device 1300. The proximity sensor 1315 is used to collect the distance between the user and the front of the computer device 1300. In one embodiment, when the proximity sensor 1315 detects that the distance between the user and the front of the computer device 1300 is gradually decreasing, the touch display screen 1305 is controlled by the processor 1301 to switch from the lit screen state to the off screen state; when the proximity sensor 1315 detects that the distance between the user and the front of the computer device 1300 is gradually increasing, the touch display screen 1305 is controlled by the processor 1301 to switch from the off screen state to the lit screen state.

[0285] Those skilled in the art can understand that Figure 13 the structure shown in does not constitute a limitation on the computer device 1300, and may include more or fewer components than those shown, or combine some components, or adopt a different component arrangement.

[0286] The memory further includes one or more programs, and the one or more programs are stored in the memory. The one or more programs include a calculation method for performing vertical federated learning provided in the embodiments of the present application.

[0287] The present application provides a computer-readable storage medium, in which at least one instruction is stored, and the at least one instruction is loaded and executed by the processor to implement the calculation method in vertical federated learning provided in the above-mentioned various method embodiments.

[0288] The present application also provides a computer program product. When the computer program product runs on a computer, the computer is caused to execute the calculation method in vertical federated learning provided in the above-mentioned various method embodiments.

[0289] The serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the superiority or inferiority of the embodiments.

[0290] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disc, etc.

[0291] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A calculation method in vertical federated learning, characterized in that, Applied to upper-layer participating nodes, where the upper-layer participating nodes have k lower-layer participating nodes among multiple participating nodes deployed in a multi-way tree topology, and a sub-model in the federated model is locally deployed in each of the participating nodes, k is an integer greater than 1, and the method includes: Distribute the first public key corresponding to the upper-layer participating node to the k lower-layer participating nodes, and obtain k second public keys respectively corresponding to the k lower-layer participating nodes; Randomly generate first network parameters and a first random mask, where the first network parameters are the network parameters of the sub-model locally deployed by the upper-layer participating node; Send a first encrypted value to the i-th lower-layer participating node, and receive a second encrypted value sent by the i-th lower-layer participating node. The first encrypted value is obtained by encrypting the first random mask using the first public key based on the homomorphic encryption technology, and the second encrypted value is obtained by encrypting a second random mask using the i-th second public key based on the homomorphic encryption technology, where the value range of i is from 1 to k; Send a third encrypted value to the i-th lower-layer participating node, and receive a fourth encrypted value sent by the i-th lower-layer participating node. The third encrypted value is the value obtained by encrypting the first data using the second encrypted value and a second random number, and the fourth encrypted value is the value obtained by the i-th lower-layer participating node encrypting the second data using the first encrypted value and a first random number. The first data is the data in the upper-layer participating node, and the second data is the data in the i-th lower-layer participating node; the second random number is generated by the upper-layer participating node, and the first random number is generated by the lower-layer participating node; Calculate the first local output, and receive the second local output sent by the i-th lower-layer participating node. The first local output is calculated based on the decrypted value of the fourth encrypted value, the data of the upper-layer participating node, the first network parameters, and the second random number, and the second local output is calculated by the i-th lower-layer node using the decrypted value of the third encrypted value, the data of the i-th lower-layer participating node, second network parameters, and the first random number; Add the first local output and the second local output to obtain the i-th two-party joint output; the first local output and the second local output include the calculation results of the upper-layer participating node and the lower-layer participating node jointly performing forward calculation on the sub-model using their respective data based on the homomorphic encryption method; Merge the k two-party joint outputs to obtain the multi-party joint output corresponding to the upper-layer participating node and the k lower-layer participating nodes.

2. The method according to claim 1, wherein The two-party joint secure calculation between the upper-layer participating node and the k lower-layer participating nodes is executed in parallel.

3. The method according to claim 1 or 2, characterized in that, The multi-way tree topology includes: Two layers of participating nodes, where the upper-layer participating node is the root node among the two layers of participating nodes, and the lower-layer participating nodes are the leaf nodes among the two layers of participating nodes, and each leaf node is connected to the root node; Or, Participating nodes above three layers, where the upper-layer participating nodes are the participating nodes at the higher level among the adjacent two layers of participating nodes, and the lower-layer participating nodes are the participating nodes at the lower level among the adjacent two layers of participating nodes. Each of the lower-layer participating nodes is connected to its corresponding upper-layer participating node.

4. The method according to claim 1 or 2, characterized in that, The upper-layer participating node is a root node with label information. The method further includes: Calculating the forward prediction error of the federated model according to the difference between the output of the multi-party joint model and the label information; Sending the forward prediction error to the k lower-layer participating nodes, where the forward prediction error is used for the k lower-layer participating nodes to perform backpropagation to update the second network parameters of the sub-models in the k lower-layer participating nodes.

5. The method according to claim 1 or 2, characterized in that, The upper-layer participating node is a lower-layer participating node of other upper-layer participating nodes in the multi-way tree topology. The method further includes: The upper-layer participating node reports the output of the multi-party joint model to the other upper-layer participating nodes, and the other upper-layer participating nodes are used to merge the output of the multi-party joint model corresponding to the other upper-layer participating nodes according to the output of the multi-party joint model, the output of its own single-party model, and the output of the multi-party joint models of other lower-layer participating nodes under it.

6. The method according to claim 1 or 2, characterized in that, The upper-layer participating node is a lower-layer participating node of other upper-layer participating nodes in the multi-way tree topology. The method further includes: The upper-layer participating node reports the first public key to the other upper-layer participating nodes and obtains the corresponding third public key of the other upper-layer participating nodes; Using the first public key and the third public key as encryption parameters, the upper-layer participating node performs two-party joint secure computation with the other upper-layer participating node as a lower-layer participating node to obtain a two-party joint output of the federated model.

7. A calculation method in vertical federated learning, characterized in that, Applied to the lower-layer participating nodes, the lower-layer participating nodes have upper-layer participating nodes among multiple participating nodes deployed in a multi-way tree topology. A sub-model in the federated model is locally deployed in each of the participating nodes. The method includes: Reporting the second public key of the lower-layer participating node to the upper-layer participating node and obtaining the corresponding first public key of the upper-layer participating node. The upper-layer participating node includes k lower-layer participating nodes, and k is a positive integer; Randomly generating second network parameters and a second random mask, where the second network parameters are the network parameters of the sub-model locally deployed by the lower-layer participating node; Receiving the first encrypted value sent by the upper-layer participating node and sending the second encrypted value to the upper-layer participating node. The first encrypted value is obtained by encrypting the first random mask using the first public key based on the homomorphic encryption technology, and the second encrypted value is obtained by encrypting the second random mask using the second public key based on the homomorphic encryption technology; Receive the third encrypted value sent by the upper-layer participating node, and send the fourth encrypted value to the upper-layer participating node. The third encrypted value is the value obtained by encrypting the first data with the second encrypted value and the second random number. The fourth encrypted value is the value obtained by the i-th lower-layer participating node encrypting the second data with the first encrypted value and the first random number. The first data is the data in the upper-layer participating node, and the second data is the data in the i-th lower-layer participating node. The second random number is generated by the upper-layer participating node, and the first random number is generated by the lower-layer participating node. Send the second local output to the upper-layer participating node so that the upper-layer participating node adds the first local output and the second local output to obtain the i-th two-party joint output. The first local output is calculated based on the decrypted value of the fourth encrypted value, the data of the upper-layer participating node, the first network parameter, and the second random number. The second local output is calculated by the i-th lower-layer node based on the decrypted value of the third encrypted value, the data of the i-th lower-layer participating node, the second network parameter, and the first random number. And obtain a two-party joint output of the federated model. The first local output and the second local output include the calculation results of the forward calculation performed by the upper-layer participating node and the lower-layer participating node on the sub-model by jointly using their respective data based on the homomorphic encryption method.

8. The method according to claim 7, wherein The method further includes: Receive the forward prediction error sent by the upper-layer participating node; Perform backpropagation according to the forward prediction error to update the second network parameter of the sub-model in the lower-layer participating node.

9. An upper-layer participating node, characterized in that, The upper-layer participating node has k lower-layer participating nodes among the multiple participating nodes deployed in a multi-way tree topology. Each participating node locally deploys a sub-model in the federated model, where k is an integer greater than 1. The upper-layer participating node includes: A communication module, configured to distribute the first public key corresponding to the upper-layer participating node to the k lower-layer participating nodes, and obtain k second public keys respectively corresponding to the k lower-layer participating nodes; A joint calculation module, configured to randomly generate a first network parameter and a first random mask, where the first network parameter is the network parameter of the sub-model locally deployed by the upper-layer participating node; Send the first encrypted value to the i-th lower-layer participating node, and receive the second encrypted value sent by the i-th lower-layer participating node. The first encrypted value is obtained by encrypting the first random mask with the first public key based on the homomorphic encryption technology. The second encrypted value is obtained by encrypting the second random mask with the i-th second public key based on the homomorphic encryption technology, and the value range of i is from 1 to k; Send the third encrypted value to the i-th lower-layer participating node, and receive the fourth encrypted value sent by the i-th lower-layer participating node. The third encrypted value is the value obtained by encrypting the first data using the second encrypted value and the second random number. The fourth encrypted value is the value obtained by the i-th lower-layer participating node encrypting the second data using the first encrypted value and the first random number. The first data is the data in the upper-layer participating node, and the second data is the data in the i-th lower-layer participating node. The second random number is generated by the upper-layer participating node, and the first random number is generated by the lower-layer participating node. Calculate the first local output, and receive the second local output sent by the i-th lower-layer participating node. The first local output is calculated based on the decrypted value of the fourth encrypted value, the data of the upper-layer participating node, the first network parameter, and the second random number. The second local output is calculated by the i-th lower-layer node using the decrypted value of the third encrypted value, the data of the i-th lower-layer participating node, the second network parameter, and the first random number. Add the first local output and the second local output to obtain the i-th two-party joint output. The first local output and the second local output include the calculation results of the forward calculation of the sub-model performed by the upper-layer participating node and the lower-layer participating node using their respective data based on the homomorphic encryption method. The merging module is used to merge the k two-party joint outputs to obtain the multi-party joint output corresponding to the upper-layer participating node and the k lower-layer participating nodes.

10. A lower-layer participating node, characterized in that, The lower-layer participating node has an upper-layer participating node among the multiple participating nodes deployed in a multi-way tree topology. Each participating node locally deploys a sub-model in the federated model. The lower-layer participating node includes: A communication module for reporting the second public key of the lower-layer participating node to the upper-layer participating node and obtaining the first public key corresponding to the upper-layer participating node. The upper-layer participating node includes k lower-layer participating nodes, and k is an integer greater than 1. A joint calculation module for randomly generating a second network parameter and a second random mask. The second network parameter is the network parameter of the sub-model locally deployed by the lower-layer participating node. Receive the first encrypted value sent by the upper-layer participating node and send the second encrypted value to the upper-layer participating node. The first encrypted value is obtained by encrypting the first random mask using the first public key based on the homomorphic encryption technology. The second encrypted value is obtained by encrypting the second random mask using the second public key based on the homomorphic encryption technology. Receive the third encrypted value sent by the upper-layer participating node, and send the fourth encrypted value to the upper-layer participating node. The third encrypted value is the value obtained by encrypting the first data with the second encrypted value and the second random number. The fourth encrypted value is the value obtained by the i-th lower-layer participating node encrypting the second data with the first encrypted value and the first random number. The first data is the data in the upper-layer participating node, and the second data is the data in the i-th lower-layer participating node. The second random number is generated by the upper-layer participating node, and the first random number is generated by the lower-layer participating node. Send the second local output to the upper-layer participating node, so that the upper-layer participating node adds the first local output and the second local output to obtain the i-th two-party joint output. The first local output is calculated based on the decrypted value of the fourth encrypted value, the data of the upper-layer participating node, the first network parameter, and the second random number. The second local output is calculated by the i-th lower-layer node using the decrypted value of the third encrypted value, the data of the i-th lower-layer participating node, the second network parameter, and the first random number. And obtain a two-party joint output of the federated model. The first local output and the second local output include the calculation results of the forward calculation of the sub-model performed by the upper-layer participating node and the lower-layer participating node using their respective data based on the homomorphic encryption method.

11. A computer device, characterized in that, The computer device includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the calculation method in the vertical federated learning as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, At least one instruction, at least one program, a code set, or an instruction set is stored in the readable storage medium. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the calculation method in the vertical federated learning as described in any one of claims 1 to 8.

13. A computer program product, characterized in that, The computer program product includes computer instructions. The computer instructions are stored in a computer-readable storage medium. The processor reads and executes the computer instructions from the computer-readable storage medium to implement the calculation method in the vertical federated learning as described in any one of claims 1 to 5 above.

Images