Asynchronous model training method and system based on federated learning and electronic device

Abstract

The present disclosure relates to an asynchronous model training method and system based on federated learning and an electronic device, which are used to implement federated learning-based asynchronous model training via a version controller and improve utilization of computing resources and model training efficiency. The asynchronous model training method comprises: obtaining a first embedding vector on the basis of a first bottom model and first feature data, and obtaining a second embedding vector from a version controller, wherein the version controller is used to perform version management of embedding vectors and gradient information obtained in a federated learning process; determining target gradient information on the basis of the first embedding vector, the second embedding vector, the first feature data, a label of the first feature data, and a top model; and updating the first bottom model on the basis of the target gradient information, and storing the target gradient information to the version controller, so that a second participant obtains the target gradient information from the version controller and updates a second bottom model on the basis of the target gradient information.

Description

Asynchronous model training methods, systems, and electronic devices based on federated learning

[0001] This application claims priority to Chinese Patent Application No. 202410704172.0, filed on May 31, 2024, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure relates to an asynchronous model training method, system, and electronic device based on federated learning. Background Technology

[0003] Vertical federated learning is a method for collaborative model training among multiple data owners. Participants in vertical federated learning each launch a large number of trainers for collaborative training. However, due to factors such as unbalanced load, inconsistent hardware and software, and unstable networks, the trainers may train at different times and in different sequences, and will not complete the collaborative task at exactly the same moment.

[0004] The synchronous training mechanism of vertical federated learning requires that each participant's trainer completes one batch of training tasks before it can execute the next batch of training tasks. This results in idle trainers, which cannot effectively utilize computing resources, leading to low overall training efficiency. Summary of the Invention

[0005] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] In a first aspect, this disclosure provides an asynchronous model training method based on federated learning, applied to a labeled first participant, the method comprising:

[0007] The first embedding vector is obtained based on the first bottom model and the first feature data, and the second embedding vector is obtained from the version controller. The second embedding vector is obtained by the unlabeled second participant based on the second bottom model and the second feature data. The version controller is used to manage the embedding vector and gradient information obtained in the federated learning process. The first feature data and the second feature data are feature data with the same sample space but different feature dimensions.

[0008] The target gradient information is determined based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model;

[0009] The first bottom model is updated based on the target gradient information, and the target gradient information is stored in the version controller so that the second participant can obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

[0010] Secondly, this disclosure provides an asynchronous model training method based on federated learning, applied to an unlabeled second participant, the method comprising:

[0011] The second embedding vector is obtained based on the second bottom model and the second feature data, and the second embedding vector is stored in the version controller so that the labeled first participant can perform the following operations: obtain the second embedding vector from the version controller, and determine the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, and store the target gradient information in the version controller. The version controller is used to manage the version of the embedding vector and gradient information obtained in the federated learning process. The first embedding vector is obtained by the first participant based on the first bottom model and the first feature data. The first feature data and the second feature data are feature data with the same sample space but different feature dimensions.

[0012] Obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

[0013] Thirdly, this disclosure provides an asynchronous model training system based on federated learning. The asynchronous model training system includes a labeled first participant, an unlabeled second participant, and a version controller. The version controller is used to manage the version of the embedding vectors and gradient information obtained during the federated learning process. The first participant is used to execute the steps of the method provided in the first aspect above, and the second participant is used to execute the steps of the method provided in the second aspect above.

[0014] Fourthly, this disclosure provides an asynchronous model training apparatus based on federated learning, applied to a labeled first participant, the apparatus comprising:

[0015] The acquisition module is used to obtain a first embedding vector based on a first bottom model and first feature data, and to obtain a second embedding vector from a version controller. The second embedding vector is obtained by an unlabeled second participant based on a second bottom model and second feature data. The version controller is used to manage the embedding vectors and gradient information obtained during the federated learning process. The first feature data and the second feature data are feature data with the same sample space but different feature dimensions.

[0016] The determination module is used to determine the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model;

[0017] The first bottom model update module is used to update the first bottom model based on the target gradient information and store the target gradient information in the version controller so that the second participant can obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

[0018] Fifthly, this disclosure provides an asynchronous model training apparatus based on federated learning, applied to an unlabeled second participant, the apparatus comprising:

[0019] The storage module is used to obtain a second embedding vector based on the second bottom model and the second feature data, and store the second embedding vector in the version controller so that the labeled first participant can perform the following operations: obtain the second embedding vector from the version controller, and determine the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, and store the target gradient information in the version controller. The version controller is used to manage the version of the embedding vector and gradient information obtained in the federated learning process. The first embedding vector is obtained by the first participant based on the first bottom model and the first feature data. The first feature data and the second feature data are feature data with the same sample space but different feature dimensions.

[0020] The second bottom model update module is used to obtain target gradient information from the version controller and update the second bottom model according to the target gradient information.

[0021] In a sixth aspect, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method provided in the first or second aspect described above.

[0022] In a seventh aspect, this disclosure provides an electronic device, comprising:

[0023] A storage device on which computer programs are stored;

[0024] A processing device for executing the computer program in the storage device to implement the steps of the method provided in the first or second aspect above.

[0025] Eighthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the methods provided in the first or second aspect above. Attached Figure Description

[0026] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings:

[0027] Figure 1 is a schematic diagram illustrating a vertical federated learning process according to an exemplary embodiment of the present disclosure;

[0028] Figure 2 is a schematic diagram illustrating asynchronous training of distributed machine learning according to an exemplary embodiment of the present disclosure;

[0029] Figure 3 is a flowchart illustrating an asynchronous model training method based on federated learning according to an exemplary embodiment of the present disclosure;

[0030] Figure 4 is a flowchart illustrating an asynchronous model training method based on federated learning according to another exemplary embodiment of the present disclosure;

[0031] Figure 5 is a schematic diagram of an asynchronous model training system based on federated learning according to an exemplary embodiment of the present disclosure;

[0032] Figure 6 is a flowchart illustrating an asynchronous model training method based on federated learning according to yet another exemplary embodiment of this disclosure;

[0033] Figure 7 is a flowchart illustrating a synchronous training process for longitudinal federated learning according to an exemplary embodiment of the present disclosure;

[0034] Figure 8 is a timing diagram of synchronous training for a longitudinal federated learning method according to an exemplary embodiment of the present disclosure;

[0035] Figure 9 is a schematic diagram of an asynchronous model training system based on federated learning according to another exemplary embodiment of the present disclosure;

[0036] Figure 10 is a timing diagram of asynchronous training for longitudinal federated learning according to an exemplary embodiment of the present disclosure;

[0037] Figure 11 is a structural block diagram of an asynchronous model training device based on federated learning according to an exemplary embodiment of the present disclosure;

[0038] Figure 12 is a structural block diagram of an asynchronous model training apparatus based on federated learning according to another exemplary embodiment of the present disclosure; and

[0039] Figure 13 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0040] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0041] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0042] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0043] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0044] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0045] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0046] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0047] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0048] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0049] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0050] Meanwhile, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0051] Vertical federated learning is a method for collaboratively training models among multiple data owners, achieving secure joint machine learning while protecting the data privacy and security of each participant. As shown in Figure 1, in vertical federated learning, data is typically partitioned according to features, with each participant possessing its own feature data. During training, the feature data of each participant are aligned using the same ID identifier. Taking two participants in vertical federated learning as an example, the participant providing data labels is called the Guest, and the participant without labels is called the Host. Each participant trains its own Bottom model, while the Guest, based on the vertical federated interaction layer, interacts with the Host's forward embeddings and backward gradient information to complete the training of the Top model.

[0052] In large-scale vertical federated learning, each participant launches a large number of trainers (workers) for collaborative training. However, due to factors such as load imbalance, inconsistent hardware and software, and network instability, the trainers may work at different speeds and not complete the collaborative tasks simultaneously. Furthermore, the synchronous training mechanism of vertical federated learning requires each participant's trainer to complete one batch of training tasks before executing the next, leading to idle trainers, inefficient use of computing resources, and ultimately, low overall training efficiency.

[0053] It's important to note that asynchronous training in distributed machine learning is typically used in unilateral machine learning scenarios. Taking the distributed PS (Parameter Server) architecture shown in Figure 2 as an example, the PS architecture consists of a PS client and worker clients. The PS client stores and updates model parameters, while the worker clients are the model training nodes. Training data is usually distributed across multiple worker nodes. Each worker node can independently process its local training data and update model parameters, updating gradients and sharing model information through asynchronous communication with the PS client to achieve asynchronous model training. However, large-scale longitudinal federated learning scenarios require connecting the forward embeddings and backward gradient information of all participants for cross-participant model training. Therefore, this approach is only suitable for unilateral machine learning scenarios and cannot perform asynchronous training across participants, making it unsuitable for large-scale longitudinal federated learning scenarios.

[0054] In view of this, this disclosure provides an asynchronous model training method, system, and electronic device based on federated learning to solve the aforementioned technical problems. It should be noted that the asynchronous model training method based on federated learning provided in this disclosure is applicable to large-scale longitudinal federated learning scenarios. Each participant can be a distributed machine learning architecture, such as the PS architecture shown in Figure 2, or other distributed architectures such as Spark or Ray; this disclosure does not impose any limitations on this.

[0055] The embodiments of this disclosure will be further explained below with reference to the accompanying drawings.

[0056] Figure 3 is a flowchart illustrating an asynchronous model training method based on federated learning according to an exemplary embodiment of this disclosure. The method is applied to a labeled first participant. Referring to Figure 3, the method includes:

[0057] S301: Obtain the first embedding vector based on the first bottom model and the first feature data, and obtain the second embedding vector from the version controller.

[0058] For example, the second embedding vector is obtained by the unlabeled second participant based on the second bottom model and the second feature data. The version controller is used to manage the version of the embedding vector and gradient information obtained in the federated learning process. The first feature data and the second feature data are feature data in the same sample space but with different feature dimensions.

[0059] For example, suppose the first participant is Company A and the second participant is Company B. Company A and Company B provide different services to users, and there is some overlap between the two companies' users. Therefore, the first feature data can be the business data of the overlapping users at Company A, and the second feature data can be the business data of the overlapping users at Company B.

[0060] It should be understood that in large-scale vertical federated learning scenarios, the first and second participants each deploy their own distributed machine learning architectures, where the first participant includes multiple first trainers and the second participant includes multiple second trainers.

[0061] In some possible approaches, the version controller includes an embedding vector version controller and a gradient information version controller. The embedding vector version controller is used to manage the version of the embedding vectors obtained during the federated learning process, and the gradient information version controller is used to manage the version of the gradient information obtained during the federated learning process.

[0062] For example, the version controller can be deployed on an electronic device capable of establishing communication connections with both the first and second participants. Version management of embedding vectors and gradient information can be performed on the same electronic device, for example, by setting up two storage lists, one for storing embedding vectors and the other for storing gradient information. Alternatively, two separate electronic devices can be used to manage the versions of embedding vectors and gradient information. The version controller can also be deployed within the participants; for example, the first participant may include an embedding vector version controller, and the second participant may include a gradient version controller. Both parties then obtain the required embedding vectors or gradient information based on the vertical federated interaction layer shown in Figure 1. The specific configuration can be tailored to requirements, and this disclosure does not impose any limitations on this.

[0063] Furthermore, this disclosure does not limit the storage format of the version controller; it can be stored in the form of an in-memory database, a relational database, a distributed database, etc., and the specific format can be configured according to requirements.

[0064] S302: Determine the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model.

[0065] It is worth noting that this disclosure does not limit the structure and function of the top model and the bottom models of each participant; they can be the same or different, and can be set according to requirements.

[0066] S303: Update the first bottom model based on the target gradient information and store the target gradient information in the version controller so that the second participant can obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

[0067] Furthermore, after the first participant updates the first bottom model, steps S301-S303 are repeated until the model converges. The model convergence condition can be set according to requirements, and this disclosure does not impose any restrictions on it.

[0068] Using the above method, the embedding vectors and gradient information obtained in the federated learning process are managed by a version controller. The first labeled participant obtains the required embedding vector from the version controller and stores the subsequently obtained target gradient information in the version controller for the unlabeled participants to update the model. That is, by introducing a caching mechanism for intermediate data, asynchronous model training in federated learning is realized, reducing idle waiting and communication overhead between trainers in the federated learning process, and improving the utilization of computing resources and model training efficiency.

[0069] In one possible manner, the version controller is used to store the second embedding vector obtained based on the latest version of the second bottom model; or, the version controller is used to store the second embedding vector obtained based on the latest version of the second bottom model and the second embedding vector obtained based on multiple historical versions of the second bottom model.

[0070] For example, the second embedding vector in the version controller can store only the embedding vector obtained based on the latest version of the second bottom model, or it can store the embedding vector obtained based on the latest version of the second bottom model and the second bottom models of multiple historical versions prior to the latest version, or it can store the embedding vector obtained based on the second bottom model of all versions. The specific settings can be configured according to requirements, and this disclosure does not impose any restrictions. Therefore, the number of second embedding vectors stored can be flexibly adjusted according to requirements or the storage resources of the version controller.

[0071] In one possible manner, obtaining the second embedding vector from the version controller may include: obtaining the second embedding vector from the version controller based on the latest version of the second bottom model.

[0072] For example, without considering the step size control of asynchronous training, each time the second embedding vector is obtained, the second embedding vector obtained based on the latest version of the second bottom model can be obtained.

[0073] In one possible approach, the second embedding vector in the version controller is associated with the model version of the corresponding second bottom model. Obtaining the second embedding vector from the version controller may include: when there is a target model version in the model versions stored in the version controller whose version difference with the top model is less than a preset threshold, obtaining the second embedding vector corresponding to the target model version.

[0074] For example, an asynchronous step size controller can be set in the version controller to control the utilization of the trainer and the model accuracy through a step size threshold (preset threshold). For instance, the second embedding vector obtained from the second bottom model based on versions V1-V5 in the version controller has a preset threshold of 3. If the top model version is V9, the second embedding vector is not returned. When the version controller stores the second embedding vector obtained from the second bottom model based on version V6, the second embedding vector obtained from the second bottom model based on version V6 is returned. If the top model version is V5, the second embedding vectors that meet the step size control include the second embedding vectors obtained from the second bottom models based on versions V2-V5. One of the second embedding vectors can be randomly selected, or the second embedding vector obtained from the latest version of the second bottom model can be selected. The specific settings can be configured according to requirements, and this disclosure does not impose any restrictions. This allows requesters with older versions to wait in a controlled manner for information that meets the step size threshold before continuing model training, effectively controlling the utilization of the trainer and the model accuracy.

[0075] In one possible approach, determining the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model may include: inputting the first embedding vector, the second embedding vector, the first feature data, and the label of the first feature data into the top model to obtain the target embedding vector; and determining the target gradient information based on the target embedding vector.

[0076] For example, referring to the federated training system shown in Figure 1, the Guest and Host sides each obtain their own forward embeddings based on their own bottom model and feature data. The Host side stores its forward embeddings in the version controller, and the Guest side obtains the Host side's forward embeddings from the version controller. Then, the Host side's forward embeddings and the Guest side's forward embeddings are merged and input into the top model along with the first feature data and its label to obtain the final forward embedding. Then, the Guest side calculates the target gradient information in reverse according to the chain rule.

[0077] In one possible approach, updating the first bottom model based on the target gradient information may include: splitting the target gradient information based on the first feature data to obtain the first gradient information for the first bottom model; and updating the first bottom model based on the first gradient information.

[0078] For example, since the final forward embedding includes gradient information for the bottom model of each participant, it needs to be decomposed. Each participant can decompose the target gradient information based on its own feature data. For instance, the Guest and Host parties can use a homomorphic encryption algorithm to exchange data through a federated vertical interaction layer and calculate their respective gradient information. For details, please refer to relevant technologies; this disclosure will not elaborate further. This allows them to obtain gradient information for their own bottom model and update the bottom model based on the gradient information, completing one round of model training.

[0079] Figure 4 is a flowchart illustrating an asynchronous model training method based on federated learning according to another exemplary embodiment of this disclosure. The method is applied to an unlabeled second participant, and as shown in Figure 4, includes:

[0080] S401: Obtain the second embedding vector based on the second bottom model and the second feature data, and store the second embedding vector in the version controller so that the first labeled participant can perform the following operations: obtain the second embedding vector from the version controller, and determine the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, and store the target gradient information in the version controller.

[0081] For example, a version controller is used to manage the version of the embedding vectors and gradient information obtained during federated learning. The first embedding vector is obtained by the first participant based on the first bottom model and the first feature data. The first feature data and the second feature data are feature data with the same sample space but different feature dimensions.

[0082] S402: Obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

[0083] Furthermore, after the second participant updates the second bottom model, steps S401-S402 are repeated until the model converges. The model convergence condition can be set according to requirements, and this disclosure does not impose any restrictions on it.

[0084] Using the above method, the embedding vectors and gradient information obtained in the federated learning process are managed by a version controller. The unlabeled second participant stores the embedding vectors in the version controller for the labeled participants to use in the subsequent federated learning process, and obtains the required gradient information from the version controller to update the model. That is, by introducing an intermediate data caching mechanism, asynchronous model training in federated learning is realized, reducing idle waiting and communication overhead between trainers in the federated learning process, and improving the utilization of computing resources and model training efficiency.

[0085] In one possible manner, the version controller is used to store the target gradient information obtained based on the latest version of the top model; or, the version controller is used to store the target gradient information obtained based on the latest version of the top model as well as the target gradient information obtained based on multiple historical versions of the top model.

[0086] For example, the target gradient information in the version controller can store only the gradient information obtained based on the latest version of the top model, or it can store the gradient information obtained based on the latest version of the top model and the top models of multiple historical versions prior to the latest version, or it can store the gradient information obtained based on the top models of all versions. The specific settings can be configured according to requirements, and this disclosure does not impose any restrictions. Therefore, the number of gradients stored can be flexibly adjusted according to requirements or the storage resources of the version controller.

[0087] In some possible ways, obtaining target gradient information from the version controller may include: obtaining target gradient information from the version controller based on the latest version of the top model.

[0088] For example, without considering the step size control of asynchronous training, each time the target gradient information is obtained, the target gradient information obtained based on the latest version of the top model can be obtained.

[0089] In one possible approach, the target gradient information in the version controller is associated with the model version of the corresponding top model. Obtaining the target gradient information from the version controller may include: when there is a target model version in the model versions stored in the version controller whose version difference with the second bottom model is less than a preset threshold, obtaining the target gradient information corresponding to the target model version.

[0090] For example, an asynchronous step size controller can be set in the version controller to control the utilization of the trainer and the model accuracy through a step size threshold (preset threshold). For instance, the version controller stores target gradient information obtained from the top models based on versions V1-V5, with a preset threshold of 3. If the second bottom model is version V9, the target gradient information is not returned. It is returned only when the version controller stores the target gradient information obtained from the top model based on version V6. If the second bottom model is version V5, the target gradient information satisfying the step size control includes the target gradient information obtained from the top models based on versions V2-V5. One of these target gradient information can be randomly selected, or the target gradient information obtained from the latest version of the bottom model can be selected. Specific settings can be made according to requirements, and this disclosure does not impose any restrictions. This allows requesters with older versions to wait in a controlled manner for information satisfying the step size threshold to arrive before continuing model training, effectively controlling the utilization of the trainer and the model accuracy.

[0091] In one possible approach, updating the second bottom model based on the target gradient information may include: splitting the target gradient information based on the second feature data to obtain the second gradient information for the second bottom model; and updating the second bottom model based on the second gradient information.

[0092] For example, since the final forward embedding includes gradient information for the bottom model of each participant, it needs to be decomposed. Each participant can decompose the target gradient information based on its own feature data. For details, refer to the process of the first participant decomposing the first gradient information; this disclosure will not elaborate further. This allows the acquisition of gradient information for the bottom model of the participant, and the bottom model is updated based on this gradient information, completing one round of model training.

[0093] Figure 5 illustrates a framework of an asynchronous model training system based on federated learning according to an exemplary embodiment of this disclosure. Referring to Figure 5, the asynchronous model training system 500 based on federated learning includes a labeled first participant 501, an unlabeled second participant 502, and a version controller 503. The version controller 503 is used to manage the versions of the embedding vectors and gradient information obtained during the federated learning process. The first participant 501 is used to execute the steps of the asynchronous model training method based on federated learning applied to the first participant, and the second participant 502 is used to execute the steps of the asynchronous model training method based on federated learning applied to the second participant.

[0094] It is worth noting that in large-scale vertical federated learning scenarios, the first and second participants can each deploy their own distributed machine learning architectures, where the first participant includes multiple first trainers and the second participant includes multiple second trainers.

[0095] Furthermore, the version controller can be deployed on an electronic device capable of establishing communication connections with both the first and second participants. Version management of the embedding vectors and gradient information can be performed using the same electronic device; for example, two storage lists can be set up, one for storing the embedding vectors and the other for storing the gradient information. Alternatively, two separate electronic devices can be used to manage the versioning of the embedding vectors and gradient information. The specific configuration can be tailored to requirements, and this disclosure does not impose any limitations on this.

[0096] The above system manages the embedding vectors and gradient information obtained during federated learning through a version controller. Each participant obtains the required embedding vectors or gradient information from the version controller. In other words, by introducing a caching mechanism for intermediate data, asynchronous model training in federated learning is achieved, reducing idle waiting and communication overhead between trainers during federated learning, and improving the utilization of computing resources and model training efficiency.

[0097] The following example illustrates the interaction process of the asynchronous model training method based on federated learning provided in this disclosure, using the example of two participants in the asynchronous model training of federated learning.

[0098] Figure 6 is a flowchart illustrating an asynchronous model training method based on federated learning according to yet another exemplary embodiment of this disclosure. Referring to Figure 6, the method includes:

[0099] S601: The first labeled participant obtains the first embedding vector based on the first bottom model and the first feature data.

[0100] S602: The unlabeled second participant obtains the second embedding vector based on the second bottom model and the second feature data, and stores the second embedding vector in the version controller.

[0101] The version controller is used to manage the version of the embedding vectors and gradient information obtained during federated learning. The first feature data and the second feature data are feature data with the same sample space but different feature dimensions.

[0102] S603: The first participant obtains the second embedding vector from the version controller, determines the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, updates the first bottom model based on the target gradient information, and stores the target gradient information in the version controller.

[0103] S604: The second participant obtains the target gradient information from the version controller and updates the second bottom model based on the target gradient information.

[0104] Using the above method, the embedding vectors and gradient information obtained in the federated learning process are managed by a version controller. Each participant obtains the required embedding vectors or gradient information from the version controller. In other words, by introducing a caching mechanism for intermediate data, asynchronous model training in federated learning is achieved, reducing idle waiting and communication overhead between trainers in the federated learning process, and improving the utilization of computing resources and model training efficiency.

[0105] It should be noted that, in the synchronous training process of vertical federated learning, taking the Guest and Host sides as shown in Figure 1 as an example, the training steps for one round are shown in Figure 7, including:

[0106] S701: Each party obtains its own forward embedding based on its own bottom model and feature data.

[0107] S702: After the Embedding is synchronously aggregated at the federated interaction layer, the Guest continues to obtain the final forward Embedding based on the aggregated Embedding and the data Label and Top model.

[0108] S703: The Guest side transmits gradient information back to the federated interaction layer in reverse according to the chain rule of differentiation.

[0109] S704: Then the federated interaction layer synchronously splits the reverse gradient into the bottom models of both sides.

[0110] S705: Both sides update their own bottom model based on the back gradient.

[0111] The above training steps are repeated in rounds until the model converges. The time series of the synchronous longitudinal federated training process is shown in Figure 8, where the numbers represent the time required for the corresponding stage. As shown in Figure 8, after the host side completes the forward embedding calculation, it needs to wait for the backward gradient information before continuing to the next round of training, resulting in the trainer being idle and unable to effectively utilize computing resources, thus leading to low overall training efficiency.

[0112] The asynchronous model training system based on federated learning provided in this disclosure, as shown in Figure 9, includes a labeled Guest side, an unlabeled Host side, and a version controller. The version controller can be divided into a forward Embedding version controller and a backward gradient controller, and can include an asynchronous step size controller. The distributed architecture of the Guest and Host sides, the structure of the version controller, and the storage method can be determined according to requirements, and this disclosure does not impose any restrictions on them.

[0113] The above system breaks down the synchronization barrier between the two Workers, as described in steps S702 and S704. After the Host receives the forward Embedding, it directly uses the Push Embedding operation provided by the forward Embedding version controller to record the current version of the Embedding and the corresponding sample ID in the forward Embedding version controller. Here, Em represents the Embedding and V represents the version. Symmetrically, after the Guest receives the backward gradient information, it records the current version of the gradient information and the corresponding sample ID in the backward gradient version controller. Here, Gr represents the gradient information and V represents the version. Thus, when the Guest needs to connect to the Host's forward Embedding data, it only needs to pull (pull) the forward Embedding data corresponding to the sample ID from the forward Embedding version controller. Conversely, when the Host needs to obtain the gradient information returned by the Guest, it only needs to pull the gradient information corresponding to the sample ID from the backward gradient version controller, thereby achieving high-performance asynchronous collaboration.

[0114] The time series of the asynchronous longitudinal federated training process is shown in Figure 10. The numbers in the figure represent the time required for the corresponding stage. As shown in Figure 10, by introducing a version controller, asynchronous model training in federated learning is achieved, reducing idle waiting and communication overhead between trainers during the federated learning process, and improving the utilization of computing resources and model training efficiency.

[0115] It's worth noting that the system can retrieve the forward embedding or gradient information of the latest version for the corresponding sample ID from the version controller. Alternatively, the step size controller can check the difference between the current version step count and the cached version step count during a pull operation. If the step count exceeds the step size threshold, no data is returned. This allows requesters with outdated versions to wait in a controlled manner for information meeting the step size threshold to arrive before continuing model training, effectively controlling the trainer's utilization and model accuracy.

[0116] It should be understood that this disclosure describes an example of asynchronous model training in federated learning by a first participant and a second participant. In practical applications, it can be applied to asynchronous model training in longitudinal federated learning with any number of parties.

[0117] Based on the same concept, this disclosure also provides an asynchronous model training device based on federated learning, applied to a labeled first participant, as shown in FIG11. The asynchronous model training device 11 based on federated learning includes:

[0118] The acquisition module 111 is used to obtain a first embedding vector based on the first bottom model and the first feature data, and to obtain a second embedding vector from the version controller. The second embedding vector is obtained by the unlabeled second participant based on the second bottom model and the second feature data. The version controller is used to manage the version of the embedding vector and gradient information obtained in the federated learning process. The first feature data and the second feature data are feature data with the same sample space but different feature dimensions.

[0119] The determination module 112 is used to determine the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model;

[0120] The first bottom model update module 113 is used to update the first bottom model based on the target gradient information and store the target gradient information in the version controller so that the second participant can obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

[0121] Optionally, the version controller is used to store the second embedding vector obtained based on the latest version of the second bottom model; or,

[0122] The version controller is used to store the second embedding vector obtained based on the latest version of the second bottom model and the second embedding vector obtained based on multiple historical versions of the second bottom model.

[0123] Optionally, the acquisition module 111 is used for:

[0124] Obtain the second embedding vector from the version controller, which is based on the latest version of the second bottom model.

[0125] Optionally, the second embedding vector in the version controller is associated with and stores the model version corresponding to the second bottom model. The acquisition module 111 is used for:

[0126] When there is a target model version in the model versions stored in the version controller whose version difference with the top model is less than a preset threshold, the second embedding vector corresponding to the target model version is obtained.

[0127] Optionally, the determining module 112 is used for:

[0128] Input the first embedding vector, the second embedding vector, the first feature data, and the label of the first feature data into the top model to obtain the target embedding vector;

[0129] The target gradient information is determined based on the target embedding vector.

[0130] Optionally, the first bottom model update module 113 is used for:

[0131] Based on the first feature data, the target gradient information is split to obtain the first gradient information for the first bottom model;

[0132] Update the first bottom model based on the first gradient information.

[0133] Optionally, the version controller includes an embedding vector version controller and a gradient information version controller. The embedding vector version controller is used to manage the version of the embedding vectors obtained during the federated learning process, and the gradient information version controller is used to manage the version of the gradient information obtained during the federated learning process.

[0134] Based on the same concept, another embodiment of this disclosure also provides an asynchronous model training device based on federated learning, applied to an unlabeled second participant. Referring to FIG12, the asynchronous model training device 12 based on federated learning includes:

[0135] Storage module 121 is used to obtain a second embedding vector based on the second bottom model and the second feature data, and store the second embedding vector in the version controller so that the labeled first participant can perform the following operations: obtain the second embedding vector from the version controller, and determine the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, and store the target gradient information in the version controller. The version controller is used to manage the version of the embedding vector and gradient information obtained in the federated learning process. The first embedding vector is obtained by the first participant based on the first bottom model and the first feature data. The first feature data and the second feature data are feature data with the same sample space but different feature dimensions.

[0136] The second bottom model update module 122 is used to obtain target gradient information from the version controller and update the second bottom model according to the target gradient information.

[0137] Optionally, the version controller is used to store the target gradient information obtained based on the latest version of the top model; or,

[0138] The version controller is used to store target gradient information obtained based on the latest version of the top model, as well as target gradient information obtained based on multiple historical versions of the top model.

[0139] Optionally, the storage module 121 is used for:

[0140] Obtain the target gradient information from the version controller, based on the latest version of the top model.

[0141] Optionally, the target gradient information in the version controller is associated with the model version of the corresponding top model, and the storage module 121 is used for:

[0142] When there is a target model version in the model versions stored in the version controller that has a version difference of less than a preset threshold with the second bottom model, the target gradient information corresponding to the target model version is obtained.

[0143] Optionally, the second bottom model update module 122 is used for:

[0144] The target gradient information is split based on the second feature data to obtain the second gradient information for the second bottom model;

[0145] The second bottom model is updated based on the second gradient information.

[0146] Based on the same concept, embodiments of this disclosure also provide a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the asynchronous model training method based on federated learning provided in any embodiment of this disclosure.

[0147] Based on the same concept, this disclosure also provides an electronic device that may include:

[0148] A storage device on which computer programs are stored;

[0149] A processing device is configured to execute the computer program stored in the storage device to implement the steps of the asynchronous model training method based on federated learning provided in any of the above embodiments.

[0150] Based on the same concept, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the asynchronous model training method based on federated learning provided in any of the above embodiments.

[0151] Referring now to FIG13, a schematic diagram of the structure of an electronic device 130 suitable for implementing embodiments of the present disclosure is shown. The terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG13 is merely an example and should not impose any limitation on the functionality and scope of use of embodiments of the present disclosure.

[0152] As shown in Figure 13, the electronic device 130 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 131, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 132 or a program loaded from a storage device 138 into a random access memory (RAM) 133. The RAM 133 also stores various programs and data required for the operation of the electronic device 130. The processing unit 131, ROM 132, and RAM 133 are interconnected via a bus 134. An input / output (I / O) interface 135 is also connected to the bus 134.

[0153] Typically, the following devices can be connected to I / O interface 135: input devices 136 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 137 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 138 including, for example, magnetic tapes, hard disks, etc.; and communication devices 139. Communication device 139 allows electronic device 130 to communicate wirelessly or wiredly with other devices to exchange data. Although FIG13 shows electronic device 130 with various devices, it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0154] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 139, or installed from a storage device 138, or installed from a ROM 132. When the computer program is executed by the processing device 131, it performs the functions defined in the methods of embodiments of this disclosure.

[0155] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0156] In some implementations, communication can be conducted using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can be interconnected with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0157] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0158] The aforementioned computer-readable medium carries one or more programs. When the aforementioned one or more programs are executed by the electronic device, the electronic device causes the following: to obtain a first embedding vector based on a first bottom model and first feature data, and to obtain a second embedding vector from a version controller, wherein the second embedding vector is obtained by an unlabeled second participant based on a second bottom model and second feature data; the version controller is used to manage the version of the embedding vectors and gradient information obtained during the federated learning process; the first feature data and the second feature data are feature data in the same sample space but with different feature dimensions; to determine target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model; to update the first bottom model based on the target gradient information, and to store the target gradient information in the version controller, so that the second participant can obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

[0159] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: obtain a second embedding vector based on a second bottom model and second feature data, and store the second embedding vector in a version controller, so that a labeled first participant performs the following operations: obtains the second embedding vector from the version controller, and determines target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model, and stores the target gradient information in the version controller, wherein the version controller is used to manage the version of the embedding vector and gradient information obtained during the federated learning process, the first embedding vector is obtained by the first participant based on the first bottom model and the first feature data, the first feature data and the second feature data are feature data in the same sample space but with different feature dimensions; obtains target gradient information from the version controller, and updates the second bottom model based on the target gradient information.

[0160] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: obtain a first embedding vector based on a first bottom model and first feature data; obtain a second embedding vector based on a second bottom model and second feature data, and store the second embedding vector in a version controller, the version controller being used to manage the version of the embedding vectors and gradient information obtained during the federated learning process, the first feature data and the second feature data being feature data in the same sample space but with different feature dimensions; the first participant obtains the second embedding vector from the version controller, determines the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model, updates the first bottom model based on the target gradient information, and stores the target gradient information in the version controller; the second participant obtains the target gradient information from the version controller and updates the second bottom model based on the target gradient information.

[0161] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0163] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the functionality of the module itself.

[0164] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0165] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0167] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0168] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. An asynchronous model training method based on federated learning, applied to a first participant with labels, the method comprising: obtaining a first embedding vector based on a first base model and first feature data, and obtaining a second embedding vector from a version controller, wherein the second embedding vector is obtained by a second participant without labels based on a second base model and second feature data, the version controller is used for version management of embedding vectors and gradient information obtained in a federated learning process, and the first feature data and the second feature data are feature data of the same sample space and different feature dimensions; determining target gradient information according to the first embedding vector, the second embedding vector, the first feature data, a label of the first feature data, and a top model; updating the first base model based on the target gradient information, and storing the target gradient information to the version controller, so that the second participant obtains the target gradient information from the version controller and updates the second base model based on the target gradient information.

2. The method of claim 1, wherein, The version controller is used for storing a second embedding vector obtained based on a latest version of the second base model; or The version controller is used for storing a second embedding vector obtained based on a latest version of the second base model and a second embedding vector obtained based on a plurality of historical versions of the second base model.

3. The method of claim 1 or 2, wherein, The obtaining of the second embedding vector from the version controller comprises: obtaining a second embedding vector obtained based on a latest version of the second base model from the version controller.

4. The method of claim 1 or 2, wherein, The second embedding vector in the version controller is associated with a model version of a corresponding second base model, and the obtaining of the second embedding vector from the version controller comprises: when there is a target model version with a version difference from the top model less than a preset threshold in the model versions stored in the version controller, obtaining a second embedding vector corresponding to the target model version.

5. The method according to any one of claims 1 to 4, wherein, The determining of the target gradient information according to the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model comprises: inputting the first embedding vector, the second embedding vector, the first feature data, and the label of the first feature data into the top model to obtain a target embedding vector; determining the target gradient information according to the target embedding vector.

6. The method according to any one of claims 1 to 5, wherein, The updating of the first base model based on the target gradient information comprises: splitting the target gradient information based on the first feature data to obtain first gradient information for the first base model; updating the first base model according to the first gradient information.

7. The method according to any one of claims 1 to 6, wherein, The version controller comprises an embedding vector version controller and a gradient information version controller, the embedding vector version controller is used for version management of embedding vectors obtained in a federated learning process, and the gradient information version controller is used for version management of gradient information obtained in the federated learning process.

8. An asynchronous model training method based on federated learning, applied to a second participant without labels, the method comprising: obtain a second embedding vector based on the second bottom model and the second feature data, and store the second embedding vector to a version controller, so that a first participant with a label performs the following operations: obtaining the second embedding vector from the version controller, and determining target gradient information according to the first embedding vector, the second embedding vector, first feature data, a label of the first feature data, and a top model, and storing the target gradient information to the version controller, wherein the version controller is configured to manage versions of embedding vectors and gradient information obtained in a federated learning process, the first embedding vector is obtained by the first participant based on a first bottom model and first feature data, and the first feature data and the second feature data are feature data of the same sample space and different feature dimensions. obtain the target gradient information from the version controller, and update the second bottom model according to the target gradient information.

9. The method of claim 8, wherein, The version controller is configured to store target gradient information obtained based on a latest version of the top model; or The version controller is configured to store target gradient information obtained based on a latest version of the top model and target gradient information obtained based on a plurality of historical versions of the top model.

10. The method of claim 8 or 9, wherein, The obtaining of the target gradient information from the version controller comprises: obtaining target gradient information obtained based on a latest version of the top model from the version controller.

11. The method of claim 8 or 9, wherein, The target gradient information in the version controller is associatedly stored with a model version of a corresponding top model, and the obtaining of the target gradient information from the version controller comprises: when there is a target model version in the model versions stored in the version controller, and a version difference between the target model version and the second bottom model is less than a preset threshold, obtaining target gradient information corresponding to the target model version.

12. The method according to any one of claims 8-11, wherein, The updating of the second bottom model according to the target gradient information comprises: splitting the target gradient information based on the second feature data to obtain second gradient information for the second bottom model; updating the second bottom model according to the second gradient information.

13. A federated learning based asynchronous model training system comprising a first participant with labels, a second participant without labels, and a version controller, wherein, The version controller is configured to manage versions of embedding vectors and gradient information obtained in a federated learning process, the first participant is configured to perform steps of the method in any one of claims 1-7, and the second participant is configured to perform steps of the method in any one of claims 8-12.

14. An asynchronous model training apparatus based on federated learning, applied to a first participant with a label, the apparatus comprising: an obtaining module configured to obtain a first embedding vector based on a first bottom model and first feature data, and obtain a second embedding vector from a version controller, wherein the second embedding vector is obtained by a second participant without a label based on a second bottom model and second feature data, the version controller is configured to manage versions of embedding vectors and gradient information obtained in a federated learning process, and the first feature data and the second feature data are feature data of the same sample space and different feature dimensions. determining a target gradient information according to the first embedding vector, the second embedding vector, the first feature data, a label of the first feature data, and a top model; a first bottom model updating module configured to update the first bottom model based on the target gradient information, and store the target gradient information to the version controller, so that the second participant obtains the target gradient information from the version controller, and updates the second bottom model based on the target gradient information.

15. An asynchronous model training apparatus based on federated learning, applied to a second participant without labels, the apparatus comprising: a storage module configured to obtain a second embedding vector based on a second bottom model and second feature data, and store the second embedding vector to a version controller, so that a first participant with labels performs the following operations: obtaining the second embedding vector from the version controller, and determining a target gradient information according to a first embedding vector, the second embedding vector, first feature data, a label of the first feature data, and a top model, and storing the target gradient information to the version controller, wherein the version controller is used for version management of embedding vectors and gradient information obtained in a federated learning process, the first embedding vector is obtained by the first participant based on a first bottom model and first feature data, and the first feature data and the second feature data are feature data of the same sample space and different feature dimensions; a second bottom model updating module configured to obtain the target gradient information from the version controller, and update the second bottom model according to the target gradient information.

16. A computer readable medium having stored thereon a computer program which, when executed by a processing apparatus, implements the method of any one of claims 1-12.

17. An electronic device comprising: a storage device storing a computer program; a processing apparatus configured to execute the computer program in the storage device to implement the method of any one of claims 1-12.

18. A computer program product comprising a computer program, wherein, The computer program is executed by the processor to implement the method of any one of claims 1-12.

Images