A Vertical Federated Learning Method and System

Abstract

This invention discloses a longitudinal federated learning method and system, belonging to the field of artificial intelligence technology. It employs an asynchronous framework to implement longitudinal federated learning on the client side, executing the sending of embedding vectors and the receiving of gradient values ​​in two sub-threads respectively. In the main thread, the generation of embedding vectors and the updating of local model parameters are performed asynchronously. When generating embedding vectors, if no gradient from the previous round has been received and the model has reached its staleness upper bound, the generation of embedding vectors is paused to ensure that the model's staleness does not exceed a preset limit. This invention pipelines local computation and transmission to overlap remote communication with local computation, thereby minimizing communication overhead. Simultaneously, it utilizes finite small boundaries to limit model staleness, ensuring good learning performance and reducing communication overhead while maintaining model accuracy.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and more specifically, relates to a vertical federated learning method and system. Background Technology

[0002] In 2016, Google proposed Federated Learning (FL), a distributed approach that allows users to train models on local private data across various mobile devices. Each device then uploads its trained model to a server, where all uploaded models are aggregated. This approach effectively protects user privacy by keeping their private data on-premises while leveraging vast amounts of distributed data to train models with superior performance. Federated learning is generally categorized into three types: horizontal federated learning, vertical federated learning, and federated transfer learning. In vertical federated learning, different participants share the same sample instances but have different characteristics. For example, two companies (a bank and an e-commerce company) provide different services and possess different types of customer data, but their customer groups have significant overlap.

[0003] Vertical federated learning can be applied in retail, healthcare, and e-commerce scenarios; for example, e-commerce companies can collaborate with banks to train learning models that predict users' creditworthiness using their historical transaction and deposit information, respectively. Individual hospitals can leverage collected clinical data to understand patients' health conditions. In these endeavors, each party generates its own embedding vectors to mask private data, which is then sent to a parameter server (PS) and buffered for joint training. The PS then passes gradients with respect to the embeddings back to one party for local model updates; this typical vertical federated learning collaboration involves frequent data movements between party servers involving statistical exchanges.

[0004] Communication is considered a major bottleneck in longitudinal federated learning because parameter transmission over long-distance wide area networks (WANs) consumes significant time. To improve training efficiency, FedBCD mimics FedAvg by performing multiple local updates to reduce the frequency of statistical exchanges. Nevertheless, due to its synchronous framework, it is still affected by periodic WAN communication even without computational lag. As WAN speeds change, communication overhead consistently dominates computational consumption, especially under conditions of high network traffic and insufficient capacity. Despite significant expansions in network bandwidth in recent years, the ever-increasing cross-datacenter data flows offset bandwidth availability, severely prolonging WAN data movement.

[0005] Existing longitudinal federated learning methods have invested significant effort in reducing communication overhead during model training; in particular, caching and reusing outdated statistics to learn model gradients can help reduce the exchange of cross-client statistics while ensuring fast convergence. However, existing longitudinal federated learning methods are primarily applicable to two-client scenarios, which limits their scalability in more general cases. Furthermore, the root causes of WAN latency remain unresolved due to synchronization optimizations.

[0006] To accelerate collaboration over slow WANs, existing methods, such as the VAFL framework, propose asynchronous updates to the local models of each client. However, this approach employs sequential model updates and parameter transfers, and utilizes outdated statistics based on small samples. This results in costly communication and fails to guarantee adequate model staleness, severely impacting learning performance and prolonging the training process. Summary of the Invention

[0007] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a vertical federated learning method and system, the purpose of which is to reduce communication overhead while ensuring model accuracy.

[0008] To achieve the above objectives, in a first aspect, the present invention provides a vertical federated learning method, applied to each client participating in the vertical federated learning, comprising:

[0009] Start the client's main thread, the client's first child thread, and the client's second child thread;

[0010] Executed in the first child thread of the client: When the client's send buffer queue contains the embedded vector, the embedded vector is retrieved from it in a first-in-first-out manner and sent to the server.

[0011] Execute in the second child thread of the client: Receive the gradient value returned by the server and store it in the client's receive buffer queue;

[0012] In the client's main thread, the operations of generating embedding vectors and updating local model parameters are executed asynchronously.

[0013] The operation of generating embedding vectors includes: inputting the local sample dataset into the client's local model in batches; generating a corresponding embedding vector for each batch of local sample data and storing it in the sending buffer queue; and in this process, for each embedding vector generated, calculating the difference between the total number of embedding vectors generated from the initial time to the current time and the total number of received gradient values ​​as the stale value; if there are no gradient values ​​in the receiving buffer queue and the staleness is greater than or equal to the preset staleness upper bound, then pausing the input of the next batch of local sample data, and inputting the next batch of local sample data again after the receiving buffer queue has re-cached gradient values ​​or the staleness is less than the preset staleness upper bound.

[0014] The operation of updating local model parameters includes: when gradient values ​​are cached in the receive buffer queue, the gradient values ​​are retrieved in a first-in-first-out manner and then backpropagated to update the parameters in the client's local model.

[0015] More preferably, the embedded vector is compressed and encrypted before being sent to the server.

[0016] More preferably, a counter with an initial value of 0 is set to calculate stale values ​​in real time; wherein, the counter is incremented by one for each embedding vector generated by the client, and decremented by one for each gradient value received.

[0017] In a second aspect, the present invention provides a client comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the vertical federated learning method provided in the first aspect of the present invention.

[0018] Thirdly, this invention provides a vertical federated learning method, applied to a server participating in vertical federated learning, comprising:

[0019] Start the server-side main thread, the server-side first child thread, and the server-side second child thread respectively;

[0020] In the first sub-thread on the server side, the following is executed: receive the embedding vectors sent by each client, and form an embedding vector set from the embedding vectors of the same batch sent by each client, and store it in the receive buffer queue on the server side.

[0021] Execute in the main thread on the server side: retrieve the embedding vector set from the receive buffer queue in a first-in-first-out manner; for each embedded vector set retrieved, perform forward and backward propagation on the local model on the server to obtain the gradient value corresponding to each client, store the gradient value corresponding to each client in the send buffer queue on the server side, and update the local model on the server.

[0022] In the second sub-thread on the server side, the gradient values ​​corresponding to each client are retrieved from the sending buffer queue in a first-in-first-out manner and sent to each client accordingly.

[0023] More preferably, the gradient values ​​are compressed and encrypted before being sent to the corresponding client.

[0024] Fourthly, the present invention provides a server-side component, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the vertical federated learning method provided in the third aspect of the present invention.

[0025] Fifthly, the present invention provides a vertical federated learning system, comprising: a server and multiple clients;

[0026] The client is used to execute the longitudinal federated learning method provided in the first aspect of this invention;

[0027] The server is used to execute the vertical federated learning method provided in the third aspect of this invention.

[0028] In a sixth aspect, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed by a processor, controls the device in which the storage medium is located to execute the vertical federated learning method provided in the first aspect of the present invention and / or the vertical federated learning method provided in the third aspect of the present invention.

[0029] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0030] 1. This invention provides a longitudinal federated learning method, which uses an asynchronous framework to implement longitudinal federated learning on the client side. Embedding vector sending and gradient value reception are performed in two sub-threads respectively, while the main thread asynchronously executes the operations of generating embedding vectors and updating local model parameters. Specifically, when generating embedding vectors, if no gradient from the previous round has been received and the model has reached its staleness upper bound, embedding vector generation is paused to ensure that the model's staleness does not exceed a preset limit. This invention pipelines local computation and transmission to overlap remote communication with local computation, thereby minimizing communication overhead. Simultaneously, it utilizes a finite small boundary to limit model staleness, ensuring good learning performance and reducing communication overhead while maintaining model accuracy.

[0031] 2. This invention provides a vertical federated learning method that uses an asynchronous framework to implement vertical federated learning on the server side. The gradient value sending and the embedding vector receiving are executed in two sub-threads respectively, and the gradient calculation and local model update are implemented in the main thread. By overlapping remote communication with local calculation, the communication overhead is greatly reduced.

[0032] 3. Furthermore, the vertical federated learning method provided by this invention compresses and encrypts the data to be sent when communicating between each client and server, so as to further reduce the amount of communication and ensure communication security. Attached Figure Description

[0033] Figure 1 This is a flowchart of the vertical federated learning process provided by the present invention;

[0034] Figure 2 This is a schematic diagram of pipelined computation and wide area network communication for vertical federated learning provided by the present invention;

[0035] Figure 3 This is a client workflow diagram provided in an embodiment of the present invention;

[0036] Figure 4 This is a server-side workflow diagram provided in an embodiment of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0038] Vertical federated learning (VFL) allows multiple clients to train a global model by sharing data features while protecting data privacy. Communication overhead is a major bottleneck in VFL training because clients need to frequently exchange statistics over a wide area network (WAN). To address this issue, this invention proposes a bounded asynchronous vertical federated learning approach for handling model updates and data exchange. It pipelines local computation and communication transmission to overlap remote communication with local computation, thereby minimizing communication overhead while ensuring good learning performance. By reaching consensus on training samples, this invention ensures training is performed using large-scale data batches and leverages finite small boundaries to limit model staleness.

[0039] The main idea of ​​the bounded asynchronous longitudinal federated learning method provided by this invention is to pipeline parameter communication and model computation by keeping each client and server busy with forward and backward propagation. Similar to synchronous optimization, each party also generates embeddings and locally updates the model, but these embeddings and updates are based on outdated data.

[0040] Specifically, such as Figure 1 As shown, a longitudinal federated learning system is established between multiple clients (party1, ..., partyK) and a server. Each client has the same sample instances but different data features and is configured with a local model parameter generator and an embedding vector generator. The server maintains a global model and is configured with forward and backward propagators and a gradient calculator. Embedding vectors and gradients are transmitted between the multiple clients and the server via a communication network. The entire process includes:

[0041] On each client, a low-dimensional embedding vector is generated based on local data features and local model parameters, and then sent to the server.

[0042] On the server side, forward and backward propagation is performed based on the received embedding vectors and server model parameters, and the gradient corresponding to each embedding vector is calculated.

[0043] On the server side, the server model parameters are updated based on the gradient, and the gradient is returned to the corresponding client.

[0044] On each client, the local model parameters are updated based on the received gradients;

[0045] Repeat the above steps until the predetermined number of training rounds or convergence criteria are reached;

[0046] In this process, the generation of embedding vectors and the updating of local model parameters are performed asynchronously on each client. Furthermore, if no gradient from the previous round is received and the upper bound of obsolescence has been reached when generating embedding vectors, the generation of embedding vectors is paused to ensure that the model obsolescence does not exceed a preset limit.

[0047] An agreement is reached between each client and server to determine the batch of samples used in each round of training.

[0048] When communicating between each client and server, techniques such as compression, encoding, or encryption are used to reduce the amount of communication or protect communication security.

[0049] The pipelined computation and wide-area network communication of vertical federated learning provided by this invention are illustrated as follows: Figure 2 As shown.

[0050] Specifically, the aforementioned bounded asynchronous longitudinal federated learning method includes a bounded asynchronous longitudinal federated learning method applied to each client participating in the longitudinal federated learning and a longitudinal federated learning method applied to the server side participating in the longitudinal federated learning.

[0051] The bounded asynchronous longitudinal federated learning method provided by this invention, applicable to each client participating in longitudinal federated learning, includes:

[0052] Start the client's main thread, the client's first child thread, and the client's second child thread;

[0053] Executed in the first child thread of the client: When the client's send buffer queue contains the embedded vector, the embedded vector is retrieved from it in a first-in-first-out manner and sent to the server.

[0054] Execute in the second child thread of the client: Receive the gradient value returned by the server and store it in the client's receive buffer queue;

[0055] In the client's main thread, the operations of generating embedding vectors and updating local model parameters are executed asynchronously.

[0056] The operation of generating embedding vectors includes: inputting the local sample dataset into the client's local model in batches; generating a corresponding embedding vector for each batch of local sample data and storing it in the sending buffer queue; and in this process, for each embedding vector generated, calculating the difference between the total number of embedding vectors generated from the initial time to the current time and the total number of received gradient values ​​as the stale value; if there are no gradient values ​​in the receiving buffer queue and the staleness is greater than or equal to the preset staleness upper bound, then pausing the input of the next batch of local sample data, and inputting the next batch of local sample data again after the receiving buffer queue has re-cached gradient values ​​or the staleness is less than the preset staleness upper bound.

[0057] The operation of updating local model parameters includes: when gradient values ​​are cached in the receive buffer queue, the gradient values ​​are retrieved in a first-in-first-out manner and then backpropagated to update the parameters in the client's local model.

[0058] It should be noted that during the entire process, the main thread needs to wait to receive a corresponding gradient value for each embedded vector generated. The difference between the total number of embedded vectors generated from the initial time to the current time and the total number of gradient values ​​received is the staleness of the current model. For example, the staleness can be measured in the following way: set a counter with an initial value of 0, and the value of the counter is the current staleness; increment the counter by one for each embedded vector generated by the client; decrement the counter by one for each gradient value received. The staleness value can be calculated in real time through the counter. After each operation to generate embedding vectors, the main thread needs to make the following judgments: If gradient values ​​exist in the receiving buffer queue, all gradient values ​​are retrieved sequentially for model updates until all gradient values ​​in the receiving buffer queue are retrieved; if no gradient values ​​exist in the receiving buffer queue, it is determined whether the current staleness has reached the staleness upper bound. If the staleness is less than the preset staleness upper bound, the next batch of local sample data is input to perform the next round of embedding vector generation; if the staleness is greater than or equal to the staleness upper bound, the thread is blocked, the input of the next batch of local sample data is paused, and the receiving of gradient values ​​is waited for to update the model and reduce the model staleness.

[0059] This invention ensures that the model's staleness is not too high on the client side by using an upper bound on the staleness, thus avoiding the problem of excessively severe decrease in convergence accuracy. At the same time, the bounded asynchronous nature allows for pipelined communication and computation, avoiding the time spent by the server waiting for the embedding vectors of the next batch of data after returning the gradient. This overlaps communication and computation time and avoids a large amount of communication overhead.

[0060] Correspondingly, the vertical federated learning method provided by this invention, applied to the server side participating in vertical federated learning, includes:

[0061] Start the server-side main thread, the server-side first child thread, and the server-side second child thread respectively;

[0062] In the first sub-thread on the server side, the following is executed: receive the embedding vectors sent by each client, and form an embedding vector set from the embedding vectors of the same batch sent by each client, and store it in the receive buffer queue on the server side.

[0063] Execute in the main thread on the server side: retrieve the embedding vector set from the receive buffer queue in a first-in-first-out manner; for each embedded vector set retrieved, perform forward and backward propagation on the local model on the server to obtain the gradient value corresponding to each client, store the gradient value corresponding to each client in the send buffer queue on the server side, and update the local model on the server.

[0064] In the second sub-thread on the server side, the gradient values ​​corresponding to each client are retrieved from the sending buffer queue in a first-in-first-out manner and sent to each client accordingly.

[0065] To further illustrate the vertical federated learning method provided by this invention, the following detailed description is provided in conjunction with specific implementation methods:

[0066] Suppose there are three clients, A, B, and C, each with the same sample instances but different data features. For example, client A has basic information such as the user's age, gender, and occupation; client B has behavioral information such as the user's shopping history and preferences; and client C has financial information such as the user's credit score and repayment history. These three clients want to train a global model using a vertical federated learning method to predict the user's credit risk.

[0067] First, the three clients and one parameter server establish a longitudinal federated learning system and negotiate to determine the batch size B for each training round. Before each training round begins, the clients and the parameter server randomly select a batch of samples. And notify all clients.

[0068] Then, on each client, based on local data x with corresponding data characteristics k and local model parameters θ k Generate a low-dimensional embedding vector h k This is then sent to the parameter server. In this embodiment, it is assumed that each client uses a single-layer neural network to generate the embedding vector, i.e., h. k =σ(W k x k +b k ), where σ is the activation function, W k and b k These are the weight matrix and bias vector, which belong to the local model parameters θ, respectively. k Before sending the embedded vector, each client compresses and encrypts it before sending it to the server. Specifically, a hash function can be used to compress the vector, and a public-key encryption algorithm can be used to protect the communication security.

[0069] Next, on the parameter server, forward and backward propagation are performed based on the received embedding vectors and server model parameters θ0, and the gradient corresponding to each embedding vector is calculated. In this embodiment, it is assumed that the parameter server uses a multi-layer neural network to construct the global model, i.e., z = σ(W0[h1,···,h k ]+b0), where [·] denotes vector concatenation, σ is the activation function, and W0 and b0 are the weight matrix and bias vector, respectively, belonging to the server model parameters θ0. Then, the parameter server uses the cross-entropy loss function to calculate the global model loss value and uses the backpropagation algorithm to calculate the gradient corresponding to each embedding vector.

[0070] Next, on the parameter server, the server model parameters are updated based on the gradients, and the gradients are returned to the corresponding clients. In this embodiment, the parameter server uses the SGD algorithm to update the server model parameters and sets a hyperparameter with a learning rate of η. Simultaneously, the parameter server sends the gradient corresponding to each embedding vector to the corresponding client.

[0071] Finally, on each client, the local model parameters are updated based on the received gradients. In this embodiment, it is assumed that each client also uses the SGD algorithm to update its local model parameters and sets a hyperparameter with a learning rate of η.

[0072] The above steps are performed asynchronously on each client. When generating embedding vectors, if no gradient from the previous round has been received and the stale upper bound has been reached, the generation of embedding vectors is paused to ensure that the model's staleness does not exceed a preset stale upper bound D (which is set to 5 in this embodiment). This can prevent model divergence or reduced model accuracy.

[0073] Repeat the above steps until the predetermined number of training rounds or convergence criteria are reached.

[0074] Specifically, in this embodiment, the training time is assumed to be CT, and the accuracy on the test set is used as the convergence criterion. The system in this embodiment consists of a central parameter server (PS) and multiple clients. Each client is equipped with a GPU node. The PS simulates the central model, while each client stores and processes its own data share. The system is implemented on a cloud instance, using NVIDIA A10 and Tesla P4 GPUs. The WAN bandwidth is configured to 200Mbps, and Linux tc is used to control the WAN transmission rate and latency.

[0075] This embodiment uses the MNIST dataset as an example dataset. Each image is divided into three equal parts and stored on different clients. Each client implements convolutional layers locally to extract features. These features are then converted into embedding vectors. PS uses a ResNet-18 model to connect the client-generated embedding vectors and update the model.

[0076] Communication between the PS and the client is conducted over a wide area network (WAN). The client sends the generated embedding vectors to the PS, which receives and processes them. Due to the introduction of bounded stale asynchronization, the frequency of communication rounds is low, thus reducing communication overhead.

[0077] To maintain consistency across parameters, this embodiment specifies the following: total training time, number of local iterations, upper bound on staleness, data batch size, and sample reading order. In actual training, a communication process needs to be initiated to ensure parallel communication and computation. The sending and receiving processes are each responsible for data transmission and reception, and separate buffer queues for data to be sent and received need to be established to facilitate collaboration between the communication and computation processes. Therefore, before co-training begins, each participant needs to load the dataset, initialize the model, determine the parameters, establish data buffer queues, and initiate the communication process.

[0078] During training, the client and server perform asynchronous collaborative training using the following strategy:

[0079] like Figure 3 As shown, perform the following steps in the client:

[0080] Step 1: Read sample data from the local dataset in a pre-agreed reading order, transfer it to the computing device, and perform forward propagation to input the sample data into the model to generate embedded data. Transfer the embedded data to the data buffer queue to be sent. At this point, the main process can proceed to the next step. The sending process will retrieve the embedded data from the data buffer queue to be sent and send the embedded data to the server.

[0081] Step 2: Determine if the received data buffer queue is empty. If it is empty, it means that the gradient data returned by the server has not yet been received. If it is not empty, it means that the gradient data can be used for local iteration. Skip to step 5.

[0082] Step 3: Determine if the current obsolescence level has reached the upper limit of obsolescence. If it has, the obsolescence level needs to be maintained within a certain limit, and the work needs to be paused. If it has not reached the upper limit, the next batch of embedded data can be generated, and the process can be returned to Step 1.

[0083] Step 4: Wait to receive the gradient data returned from the server.

[0084] Step 5: At this point, the data buffer queue already contains gradient data returned by the server. The gradient data is retrieved, backpropagation is performed, and the model parameters are updated through multiple local iterations. Then, return to Step 1.

[0085] like Figure 4 As shown, perform the following steps on the server side:

[0086] Step 1: Read the sample labels from the local dataset in a pre-agreed reading order and transfer them to the computing device.

[0087] Step two: Wait to receive embedded data sent from each client.

[0088] Step 3: At this point, the embedded data sent by the client has been received in the data buffer queue. Each embedded data is retrieved, and forward propagation is performed to calculate the loss. Then, backpropagation is performed to calculate the gradient of each embedded data.

[0089] Step four: Transfer the gradient data to the data buffer queue to be sent. At this point, the main process can proceed to the next step. The sending process will retrieve the gradient data from the data buffer queue to be sent and return the gradient data to each client.

[0090] Step 5: Complete multiple local iterations to update the model parameters, then return to Step 1.

[0091] Repeat the above steps until the training time CT is reached. Through these steps, the three clients and the parameter server can train a global model using a longitudinal federated learning method to predict users' credit risk, while protecting their respective data privacy and minimizing communication overhead.

[0092] The present invention also provides a client comprising: a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform the bounded asynchronous longitudinal federated learning method described above for each client participating in longitudinal federated learning.

[0093] The present invention also provides a server-side component, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the aforementioned vertical federated learning method applied to the server-side component participating in vertical federated learning.

[0094] The present invention also provides a vertical federated learning system, comprising: a server and multiple clients;

[0095] The client is used to execute the bounded asynchronous longitudinal federated learning method described above for each client participating in the longitudinal federated learning;

[0096] The server-side is used to execute the aforementioned vertical federated learning method applied to the server-side participating in vertical federated learning.

[0097] The present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed by a processor, it controls the device where the storage medium is located to execute the above-described bounded asynchronous longitudinal federated learning method applied to each client participating in longitudinal federated learning, and / or the above-described longitudinal federated learning method applied to the server participating in longitudinal federated learning.

[0098] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vertical federated learning method, characterized in that, This applies to every client participating in longitudinal federated learning, including: Start the client's main thread, the client's first child thread, and the client's second child thread; Executed in the first child thread of the client: When the client's send buffer queue contains the embedded vector, the embedded vector is retrieved from it in a first-in-first-out manner and sent to the server. Execute in the second child thread of the client: Receive the gradient value returned by the server and store it in the client's receive buffer queue; In the client's main thread, the operations of generating embedding vectors and updating local model parameters are executed asynchronously. The process of generating embedding vectors includes: inputting the local sample dataset into the client's local model in batches; generating a corresponding embedding vector for each batch of local sample data and storing it in the sending buffer queue; and during this process, for each embedded vector generated, calculating the difference between the total number of embedded vectors generated from the initial time to the current time and the total number of received gradient values ​​as the aging value; if there are no gradient values ​​in the receiving buffer queue and the aging value is greater than or equal to a preset aging upper bound, then pausing the input of the next batch of local sample data, and waiting until the receiving buffer queue has cached gradient values ​​again or the aging value is less than the preset aging upper bound before inputting the next batch of local sample data; a counter with an initial value of 0 is set to calculate the aging value in real time; the counter is incremented by one for each embedded vector generated by the client and decremented by one for each gradient value received; The operation of updating local model parameters includes: when gradient values ​​are cached in the receive buffer queue, the gradient values ​​are retrieved in a first-in-first-out manner and then backpropagated to update the parameters in the client's local model.

2. The vertical federated learning method according to claim 1, characterized in that, The embedded vector is compressed and encrypted before being sent to the server.

3. A client application, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the vertical federated learning method according to any one of claims 1-2.

4. A vertical federated learning method, characterized in that, The server-side components used in vertical federated learning include: Start the server-side main thread, the server-side first child thread, and the server-side second child thread respectively; In the first sub-thread on the server side, the following steps are executed: receiving embedding vectors sent by each client, and forming an embedding vector set from the embedding vectors of the same batch sent by each client, which is then stored in the receiving buffer queue on the server side; the client is used to execute the vertical federated learning method as described in any one of claims 1-2; Execute in the main thread on the server side: retrieve the embedding vector set from the receive buffer queue in a first-in-first-out manner; for each embedded vector set retrieved, perform forward and backward propagation on the local model on the server to obtain the gradient value corresponding to each client, store the gradient value corresponding to each client in the send buffer queue on the server side, and update the local model on the server. In the second sub-thread on the server side, the gradient values ​​corresponding to each client are retrieved from the sending buffer queue in a first-in-first-out manner and sent to each client accordingly.

5. The vertical federated learning method according to claim 4, characterized in that, The gradient values ​​are compressed and encrypted before being sent to the corresponding client.

6. A server-side component, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the vertical federated learning method of claim 4 or 5.

7. A vertical federated learning system, characterized in that, include: Server-side and multiple clients; The client is used to execute the longitudinal federated learning method according to any one of claims 1-2; The server is used to execute the vertical federated learning method according to any one of claims 4-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed by a processor, it controls the device where the storage medium is located to perform the vertical federated learning method according to any one of claims 1-2, and / or the vertical federated learning method according to any one of claims 4-5.

Images