Asynchronous model training method, system and electronic device based on federated learning

Abstract

The present disclosure relates to an asynchronous model training method, system, and electronic device based on federated learning, which implements asynchronous model training of federated learning through a version controller, thereby improving the utilization of computing resources and model training efficiency. The method includes: obtaining a first embedding vector based on a first bottom model and first feature data, and obtaining a second embedding vector from a version controller, wherein the version controller is used to perform version management on the embedding vector and gradient information obtained during the federated learning process; determining target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model; updating the first bottom model based on the target gradient information, and storing the target gradient information in the version controller so that a second participant can obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

Description

Technical Field

[0001] The present disclosure relates to the technical field of federated learning, and in particular, to an asynchronous model training method, system, and electronic device based on federated learning. Background Art

[0002] Vertical federated learning is a method for collaborative model training among multiple data owners. Participants in vertical federated learning each launch a large number of trainers for collaborative training. However, due to factors such as load imbalance, hardware and software inconsistencies, and network instability, the trainers may run in different order and at different speeds, and may not complete the collaborative task at exactly the same time.

[0003] The synchronous training mechanism of vertical federated learning requires that the trainers of each participant complete one batch of training tasks before executing the next batch of training tasks, which causes the trainers to be idle and unable to effectively utilize computing resources, resulting in low overall training efficiency. Summary of the Invention

[0004] This summary is provided to briefly introduce concepts that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides an asynchronous model training method based on federated learning, applied to a first participant with a label, the method comprising:

[0006] Obtaining a first embedding vector based on the first underlying model and the first feature data, and obtaining a second embedding vector from a version controller, wherein the second embedding vector is obtained by an unlabeled second participant based on the second underlying model and the second feature data, the version controller being configured to perform version management on the embedding vector and gradient information obtained during the federated learning process, and the first feature data and the second feature data being feature data of the same sample space but different feature dimensions;

[0007] determining target gradient information based on the first embedding vector, the second embedding vector, the first feature data, a label of the first feature data, and a top model;

[0008] The first bottom model is updated based on the target gradient information, and the target gradient information is stored in the version controller, so that the second participant obtains the target gradient information from the version controller and updates the second bottom model based on the target gradient information.

[0009] In a second aspect, the present disclosure provides an asynchronous model training method based on federated learning, which is applied to an unlabeled second participant, the method comprising:

[0010] A second embedding vector is obtained based on the second bottom model and the second feature data, and the second embedding vector is stored in a version controller, so that the first participant with the label performs the following operations: obtaining the second embedding vector from the version controller, and determining target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, and storing the target gradient information in the version controller, wherein the version controller is used to perform version management on the embedding vector and gradient information obtained in the federated learning process, the first embedding vector is obtained by the first participant based on the first bottom model and the first feature data, and the first feature data and the second feature data are feature data of the same sample space but different feature dimensions;

[0011] The target gradient information is obtained from the version controller, and the second bottom model is updated according to the target gradient information.

[0012] In a third aspect, the present disclosure provides an asynchronous model training system based on federated learning, wherein the federated learning system includes a labeled first participant, an unlabeled second participant, and a version controller, wherein the version controller is used to perform version management on the embedding vector and gradient information obtained during the federated learning process, the first participant is used to execute the steps of any one of the methods described in the first aspect above, and the second participant is used to execute the steps of any one of the methods described in the second aspect above.

[0013] In a fourth aspect, the present disclosure provides an asynchronous model training device based on federated learning, which is applied to a first participant with a label, and the device includes:

[0014] an acquisition module, configured to obtain a first embedding vector based on the first underlying model and the first feature data, and to obtain a second embedding vector from a version controller, wherein the second embedding vector is obtained by an unlabeled second participant based on the second underlying model and the second feature data; the version controller is configured to perform version management on the embedding vector and gradient information obtained during the federated learning process; and the first feature data and the second feature data are feature data of the same sample space but different feature dimensions;

[0015] a determination module, configured to determine target gradient information based on the first embedding vector, the second embedding vector, the first feature data, a label of the first feature data, and a top model;

[0016] The first bottom model update module is used to update the first bottom model based on the target gradient information and store the target gradient information in the version controller so that the second participant can obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

[0017] In a fifth aspect, the present disclosure provides an asynchronous model training device based on federated learning, which is applied to an unlabeled second participant, and the device includes:

[0018] A storage module is configured to obtain a second embedding vector based on a second bottom model and second feature data, and store the second embedding vector in a version controller, so that the first participant with a label performs the following operations: obtaining the second embedding vector from the version controller, and determining target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model, and storing the target gradient information in the version controller, wherein the version controller is configured to perform version management on the embedding vector and gradient information obtained in the federated learning process, the first embedding vector is obtained by the first participant based on the first bottom model and the first feature data, and the first feature data and the second feature data are feature data of the same sample space but different feature dimensions;

[0019] The second bottom model updating module is configured to obtain the target gradient information from the version controller and update the second bottom model according to the target gradient information.

[0020] In a sixth aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in any one of the first or second aspects above.

[0021] In a seventh aspect, the present disclosure provides an electronic device, comprising:

[0022] a storage device having a computer program stored thereon;

[0023] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in any one of the first aspect or the second aspect above.

[0024] In an eighth aspect, the present disclosure provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method described in any one of the first or second aspects above.

[0025] Through the above technical solution, the first participant with a label obtains the first embedding vector based on the first bottom model and the first feature data, and obtains the second embedding vector obtained by the second participant based on the second bottom model and the second feature data from the version controller, and then determines the target gradient information according to the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, and finally updates the first bottom model based on the target gradient information, and stores the target gradient information in the version controller so that the second participant obtains the target gradient information from the version controller to update the second bottom model.

[0026] Using the above method, the embedding vectors and gradient information obtained in the federated learning process are version-managed through a version controller. The first participant with a label obtains the required embedding vector from the version controller and stores the subsequently obtained target gradient information in the version controller for the unlabeled participant to update the model. That is, by introducing a caching mechanism for intermediate data, asynchronous model training of federated learning is achieved, which reduces the idle waiting and communication overhead between trainers in the federated learning process, and improves the utilization of computing resources and model training efficiency.

[0027] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale. In the drawings:

[0029] Figure 1 1 is a schematic diagram of a process of vertical federated learning according to an exemplary embodiment of the present disclosure;

[0030] Figure 2 is a schematic diagram of asynchronous training of distributed machine learning according to an exemplary embodiment of the present disclosure;

[0031] Figure 3 is a flowchart of an asynchronous model training method based on federated learning according to an exemplary embodiment of the present disclosure;

[0032] Figure 4 is a flowchart of an asynchronous model training method based on federated learning according to an exemplary embodiment of the present disclosure;

[0033] Figure 5 is a schematic diagram of an asynchronous model training system based on federated learning according to an exemplary embodiment of the present disclosure;

[0034] Figure 6 is a flowchart of an asynchronous model training method based on federated learning according to an exemplary embodiment of the present disclosure;

[0035] Figure 7 is a synchronous training flow chart of vertical federated learning according to an exemplary embodiment of the present disclosure;

[0036] Figure 8 1 is a synchronous training sequence diagram of vertical federated learning according to an exemplary embodiment of the present disclosure;

[0037] Figure 9 is a schematic diagram of an asynchronous model training system based on federated learning according to an exemplary embodiment of the present disclosure;

[0038] Figure 10 1 is an asynchronous training sequence diagram of vertical federated learning according to an exemplary embodiment of the present disclosure;

[0039] Figure 11 1 is a structural block diagram of an asynchronous model training device based on federated learning according to an exemplary embodiment of the present disclosure;

[0040] Figure 12 1 is a structural block diagram of an asynchronous model training device based on federated learning according to an exemplary embodiment of the present disclosure;

[0041] Figure 13 The figure is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0043] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0044] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0045] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0046] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0047] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0048] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0049] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0050] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0051] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0052] At the same time, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0053] Vertical federated learning is a method of collaboratively training models between multiple data owners, completing secure joint machine learning while protecting the data privacy and security of each participant. Figure 1 As shown in Figure 2, in vertical federated learning, data is typically partitioned by features, with each participant possessing their own feature data. During training, these data are aligned using the same ID identifier. For example, in vertical federated learning involving two participants, the participant providing data labels is called the guest, and the participant without labels is called the host. Each participant trains its own bottom model. The guest, based on the vertical federated interaction layer, interactively connects the host's forward embeddings and backward gradient information to complete the training of the top model.

[0054] In large-scale vertical federated learning, participants each launch a large number of trainers (workers) for collaborative training. However, due to factors such as load imbalance, hardware and software inconsistencies, and network instability, the trainers may run in different order and at different speeds, and may not complete the collaborative task at exactly the same time. The synchronous training mechanism of vertical federated learning requires that each participant's trainer complete one batch of training tasks before executing the next. This results in idle trainers, inefficient use of computing resources, and low overall training efficiency.

[0055] It should be noted that asynchronous training of distributed machine learning is usually applied to unilateral machine learning scenarios, such as Figure 2 Taking the distributed PS (Parameter Server) architecture shown in the figure for distributed machine learning as an example, the PS architecture is divided into the PS side and the Worker side. The PS side stores and updates model parameters, and the Worker side is the model training node. Training data is typically distributed across multiple Worker nodes. Each Worker node can independently process local training data and update model parameters. It also updates gradients and shares model information through asynchronous communication with the PS side, enabling asynchronous model training. Large-scale vertical federated learning scenarios, however, require connecting the forward embedding and backward gradient information of each participant for cross-participant model training. Therefore, this approach is only suitable for single-party machine learning scenarios and cannot perform asynchronous training across participants, making it unsuitable for large-scale vertical federated learning scenarios.

[0056] In view of this, the present disclosure provides an asynchronous model training method, system and electronic device based on federated learning to solve the above technical problems. It should be noted that the asynchronous model training method based on federated learning provided by the present disclosure can be applied to large-scale vertical federated learning scenarios, where each participant can be a distributed machine learning architecture, such as Figure 2 The PS architecture shown, or other distributed architectures such as Spark and Ray, is not limited in this disclosure.

[0057] The following further explains the embodiments of the present disclosure with reference to the accompanying drawings.

[0058] Figure 3 This is a flowchart of an asynchronous model training method based on federated learning according to an exemplary embodiment of the present disclosure. The method is applied to the first participant with a label, referring to Figure 3 ,include:

[0059] S301: Obtain a first embedding vector based on the first bottom model and the first feature data, and obtain a second embedding vector from the version controller.

[0060] Among them, the second embedding vector is obtained by the unlabeled second participant based on the second bottom model and the second feature data. The version controller is used to manage the version of the embedding vector and gradient information obtained in the federated learning process. The first feature data and the second feature data are feature data with the same sample space and different feature dimensions.

[0061] For example, let's say the first participant is Company A and the second participant is Company B. Company A and Company B each provide different services to users, and there is some overlap between the two companies. Therefore, the first feature data can be the service data of the overlapping users at Company A, and the second feature data can be the service data of the overlapping users at Company B.

[0062] It should be understood that in a large-scale vertical federated learning scenario, the first participant and the second participant each deploy their own distributed machine learning architecture, where the first participant includes multiple first trainers and the second participant includes multiple second trainers.

[0063] In a possible manner, the version controller includes an embedding vector version controller and a gradient information version controller. The embedding vector version controller is used to version manage the embedding vector obtained in the federated learning process, and the gradient information version controller is used to version manage the gradient information obtained in the federated learning process.

[0064] For example, the version controller can be deployed in an electronic device that can establish a communication connection with the first participant and the second participant, and the embedding vector and gradient information can be version managed by the same electronic device, for example, two storage lists are set up, one for storing the embedding vector and the other for storing the gradient information. Or two electronic devices are used to manage the embedding vector and gradient information respectively. The version controller can also be deployed in the participants, for example, the first participant includes an embedding vector version controller, and the second participant includes a gradient version controller, then the two parties can manage the embedding vector and gradient information based on the above. Figure 1 The vertical federation interaction layer shown obtains the required embedding vector or gradient information, which can be set according to needs and is not limited in this disclosure.

[0065] In addition, the present disclosure does not limit the storage form of the version controller, which can be an in-memory database, a relational database, a distributed database, etc., and can be specifically set according to needs.

[0066] S302: Determine target gradient information according to the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model.

[0067] It is worth noting that the present disclosure does not limit the structures and functions of the top model and the bottom models of each participant, which can be the same or different and can be set according to specific needs.

[0068] S303: Update the first bottom model based on the target gradient information, and store the target gradient information in the version controller, so that the second participant obtains the target gradient information from the version controller and updates the second bottom model based on the target gradient information.

[0069] Furthermore, after the first participant updates the first bottom model, steps S301-S303 are repeated until the model converges. The model convergence condition can be set according to needs, and this disclosure does not impose any restrictions on this.

[0070] Using the above method, the embedding vectors and gradient information obtained in the federated learning process are version-managed through a version controller. The first participant with a label obtains the required embedding vector from the version controller and stores the subsequently obtained target gradient information in the version controller for the unlabeled participant to update the model. That is, by introducing a caching mechanism for intermediate data, asynchronous model training of federated learning is achieved, which reduces the idle waiting and communication overhead between trainers in the federated learning process, and improves the utilization of computing resources and model training efficiency.

[0071] In a possible manner, the version controller is used to store the second embedding vector obtained based on the latest version of the second bottom model; or, the version controller is used to store the second embedding vector obtained based on the latest version of the second bottom model and the second embedding vector obtained based on multiple historical versions of the second bottom model.

[0072] For example, the second embedding vectors in the version controller may store only embedding vectors based on the latest version of the second bottom model, embedding vectors based on the latest version of the second bottom model and multiple historical versions of the second bottom model prior to the latest version, or embedding vectors based on all versions of the second bottom model. The specific configuration can be determined based on actual needs and is not limited in this disclosure. Thus, the number of second embedding vectors stored can be flexibly adjusted based on demand or the storage resources of the version controller.

[0073] In a possible manner, obtaining the second embedding vector from the version controller may include: obtaining, from the version controller, a second embedding vector obtained based on the latest version of the second base model.

[0074] For example, without considering the step size control of asynchronous training, each time the second embedding vector is obtained, the second embedding vector obtained based on the latest version of the second bottom model can be obtained.

[0075] In a possible manner, the second embedding vector in the version controller is associated with and stores a model version corresponding to the second bottom model. Obtaining the second embedding vector from the version controller may include: when there is a target model version in the model version stored in the version controller whose version difference with the top model is less than a preset threshold, obtaining the second embedding vector corresponding to the target model version.

[0076] For example, an asynchronous step controller can be further set in the version controller to control the utilization rate and model accuracy of the trainer through the step threshold (preset threshold). For example, the second embedding vector obtained by the second bottom model based on the V1-V5 version in the version controller has a preset threshold of 3. If the version of the top model is V9, the second embedding vector is not returned. When the second embedding vector obtained by the second bottom model based on the V6 version is stored in the version controller, the second embedding vector obtained by the second bottom model based on the V6 version is returned. If the version of the top model is V5, the second embedding vector that meets the step control includes the second embedding vector obtained by the second bottom model based on the V2-V5 version. One of the second embedding vectors can be randomly selected, or the second embedding vector obtained by the second bottom model of the latest version can be selected. The specific setting can be based on demand, and the present disclosure does not impose any restrictions on this. This allows the requester whose version is too early to wait for the information that meets the step threshold to arrive in a controlled manner before continuing model training, thereby effectively controlling the utilization rate and model accuracy of the trainer.

[0077] In a possible manner, determining the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model can include: inputting the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data into the top model to obtain the target embedding vector; and determining the target gradient information based on the target embedding vector.

[0078] For example, refer to Figure 1 In the federated training system shown, the Guest and Host each obtain their own forward embeddings based on their own bottom models and feature data. The Host stores its forward embeddings in the version controller, and the Guest obtains the Host's forward embeddings from the version controller. The Host's forward embeddings and the Guest's forward embeddings are then aggregated and input into the top model together with the first feature data and the label of the first feature data to obtain the final forward embedding. The Guest then reversely calculates the target gradient information according to the chain rule.

[0079] In a possible manner, updating the first bottom model based on the target gradient information may include: splitting the target gradient information based on the first feature data to obtain first gradient information for the first bottom model; and updating the first bottom model according to the first gradient information.

[0080] For example, since the final forward embedding includes the gradient information of the bottom model for each participant, the final forward embedding needs to be disassembled, and each participant can split the target gradient information based on their own feature data. For example, the Guest and Host parties interact with each other through the federated vertical interaction layer based on the homomorphic encryption algorithm to calculate their respective gradient information. For details, please refer to the relevant technology, which will not be described in detail in this disclosure. In this way, the gradient information of the bottom model for the party is obtained, and the bottom model is updated based on the gradient information to complete a round of model training.

[0081] Figure 4 This is a flowchart of an asynchronous model training method based on federated learning according to an exemplary embodiment of the present disclosure. The method is applied to the unlabeled second participant, referring to Figure 4 ,include:

[0082] S401: Obtain a second embedding vector based on the second bottom model and the second feature data, and store the second embedding vector in the version controller so that the first participant with the label performs the following operations: obtain the second embedding vector from the version controller, and determine target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, and store the target gradient information in the version controller.

[0083] Among them, the version controller is used to manage the versions of the embedding vectors and gradient information obtained in the federated learning process. The first embedding vector is obtained by the first participant based on the first bottom model and the first feature data. The first feature data and the second feature data are feature data with the same sample space and different feature dimensions.

[0084] S402: Obtain target gradient information from the version controller, and update the second bottom model according to the target gradient information.

[0085] Furthermore, after the second participant updates the second bottom model, steps S401-S402 are repeated until the model converges. The model convergence condition can be set according to needs, and this disclosure does not impose any restrictions on this.

[0086] Using the above method, the embedding vectors and gradient information obtained in the federated learning process are version managed through the version controller. The unlabeled second participant stores the embedding vector in the version controller for the labeled participant to perform subsequent federated learning processes, and obtains the required gradient information from the version controller to update the model. That is, by introducing a caching mechanism for intermediate data, asynchronous model training of federated learning is realized, which reduces the idle waiting and communication overhead between trainers in the federated learning process, and improves the utilization of computing resources and model training efficiency.

[0087] In a possible manner, the version controller is used to store target gradient information obtained based on the latest version of the top model; or, the version controller is used to store target gradient information obtained based on the latest version of the top model and target gradient information obtained based on multiple historical versions of the top model.

[0088] For example, the target gradient information in the version controller can store only the gradient information obtained based on the latest version of the top model, or it can store the gradient information obtained based on the latest version of the top model and multiple historical versions of the top model before the latest version, or it can store the gradient information obtained based on all versions of the top model. The specific setting can be set according to needs, and this disclosure does not impose any restrictions on this. Therefore, the number of gradients stored can be flexibly adjusted according to needs or the storage resources of the version controller.

[0089] In a possible manner, obtaining the target gradient information from the version controller may include: obtaining the target gradient information based on the latest version of the top model from the version controller.

[0090] For example, without considering the step size control of asynchronous training, each time the target gradient information is obtained, the target gradient information obtained based on the latest version of the top model can be obtained.

[0091] In a possible manner, the target gradient information in the version controller is associated with and stores the model version of the corresponding top model. Obtaining the target gradient information from the version controller may include: when there is a target model version in the model version stored in the version controller whose version difference with the second bottom model is less than a preset threshold, obtaining the target gradient information corresponding to the target model version.

[0092] For example, an asynchronous step controller can also be set in the version controller to control the utilization rate and model accuracy of the trainer through the step threshold (preset threshold). For example, the target gradient information obtained based on the top model of version V1-V5 in the version controller has a preset threshold of 3. If the version of the second local model is V9, the target gradient information is not returned. When the target gradient information obtained based on the top model of version V6 is stored in the version controller, the target gradient information obtained based on the top model of version V6 is returned. If the version of the second bottom model is V5, the target gradient information that meets the step control includes the target gradient information obtained based on the top model of version V2-V5. One of the target gradient information can be randomly selected, or the target gradient information obtained from the bottom model of the latest version can be selected. It can be set specifically according to needs, and this disclosure does not limit this. This allows the requester whose version is too early to wait for the information that meets the step threshold to arrive in a controlled manner before continuing model training, thereby effectively controlling the utilization rate and model accuracy of the trainer.

[0093] In a possible manner, updating the second bottom model according to the target gradient information may include: splitting the target gradient information based on the second feature data to obtain second gradient information for the second bottom model; and updating the second bottom model according to the second gradient information.

[0094] For example, since the final forward embedding includes the gradient information of the bottom model for each participant, the final forward embedding needs to be disassembled. Each participant can split the target gradient information based on their own feature data. For details, please refer to the process of splitting the first gradient information by the first participant, which will not be described in detail in this disclosure. In this way, the gradient information of the bottom model for the participant is obtained, and the bottom model is updated based on the gradient information, completing a round of model training.

[0095] Figure 5This is a framework of an asynchronous model training system based on federated learning according to an exemplary embodiment of the present disclosure. Figure 5 The asynchronous model training system 500 based on federated learning includes a labeled first participant 501, an unlabeled second participant 502 and a version controller 503. The version controller 503 is used to perform version management on the embedding vector and gradient information obtained in the federated learning process. The first participant 501 is used to execute the steps of the above-mentioned asynchronous model training method based on federated learning applied to the first participant, and the second participant 502 is used to execute the steps of the above-mentioned asynchronous model training method based on federated learning applied to the second participant.

[0096] It is worth noting that in a large-scale vertical federated learning scenario, the first participant and the second participant can each deploy their own distributed machine learning architecture, where the first participant includes multiple first trainers and the second participant includes multiple second trainers.

[0097] Furthermore, the version controller can be deployed in an electronic device capable of establishing a communication connection with the first and second parties. Version management of both the embedding vector and the gradient information can be performed by the same electronic device, for example by setting up two storage lists, one for storing the embedding vector and one for storing the gradient information. Alternatively, two electronic devices can be used to perform version management of the embedding vector and gradient information, respectively. The specific configuration can be tailored to individual needs and is not limited in this disclosure.

[0098] Using the above system, the embedding vectors and gradient information obtained in the federated learning process are version-managed through a version controller. Each participant obtains the required embedding vectors or gradient information from the version controller. That is, by introducing a caching mechanism for intermediate data, asynchronous model training of federated learning is realized, reducing idle waiting and communication overhead between trainers in the federated learning process, and improving the utilization of computing resources and model training efficiency.

[0099] The following takes the asynchronous model training of two participants in federated learning as an example to illustrate the interactive process of the asynchronous model training method based on federated learning provided by the present disclosure.

[0100] Figure 6 1 is a flowchart of an asynchronous model training method based on federated learning according to an exemplary embodiment of the present disclosure. Figure 6 , the method comprising:

[0101] S601: The first participant with a label obtains a first embedding vector based on a first bottom model and first feature data.

[0102] S602: The unlabeled second participant obtains a second embedding vector based on the second bottom model and the second feature data, and stores the second embedding vector in the version controller.

[0103] Among them, the version controller is used to manage the versions of the embedding vectors and gradient information obtained in the federated learning process, and the first feature data and the second feature data are feature data with the same sample space and different feature dimensions.

[0104] S603: The first participant obtains the second embedding vector from the version controller, determines the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, updates the first bottom model based on the target gradient information, and stores the target gradient information in the version controller.

[0105] S604: The second participant obtains target gradient information from the version controller and updates the second bottom model according to the target gradient information.

[0106] Using the above method, the embedding vectors and gradient information obtained in the federated learning process are version-managed through a version controller. Each participant obtains the required embedding vectors or gradient information from the version controller. That is, by introducing a caching mechanism for intermediate data, asynchronous model training of federated learning is realized, which reduces the idle waiting and communication overhead between trainers in the federated learning process, and improves the utilization of computing resources and model training efficiency.

[0107] It should be noted that in the synchronous training process of vertical federated learning, Figure 1 Take the guest and host as an example to perform federated learning. The training steps are as follows: Figure 7 Shown, including:

[0108] S701: Each party obtains its own forward embedding based on its own bottom model and feature data.

[0109] S702: After the federated interaction layer synchronously aggregates the embeddings, the guest side continues to obtain the final forward embedding based on the aggregated embeddings and the data label and top model.

[0110] S703: The guest side transmits the gradient information back to the federation interaction layer according to the chain derivation rule.

[0111] S704: Then the federated interaction layer synchronously splits the reverse gradient into the bottom models of both parties.

[0112] S705: Both parties update their own bottom models according to the reverse gradient.

[0113] The above training steps are repeated repeatedly until the model converges. The time series of the synchronized longitudinal federated training process is as follows: Figure 8 As shown in the figure, the numbers in the figure represent the time required for the corresponding stage process. Figure 8 As shown in the figure, after the Host calculates the forward embedding, it needs to wait for the backward gradient information before continuing the next round of training, which causes the trainer to be idle and unable to effectively utilize computing resources, resulting in low overall training efficiency.

[0114] The asynchronous model training system based on federated learning provided by this disclosure is as follows Figure 9 As shown, the system includes a tagged guest, an untagged host, and a version controller. The version controller can be divided into a forward embedding version controller and a backward gradient controller, and the version controller can also include an asynchronous step controller. The distributed architecture of the guest and host, the structure of the version controller, and the storage method can be determined according to requirements and are not limited by this disclosure.

[0115] Through the above system, the synchronization barrier between the two workers is broken, as described in steps S702 and S704. After the host obtains the forward embedding, it directly uses the Push Embedding operation provided by the forward embedding version controller to record the current version of the embedding and the corresponding sample ID in the forward embedding version controller, where Em represents the embedding and V represents the version. Symmetrically, after the guest obtains the backward gradient information, it records the current version of the gradient information and the corresponding sample ID in the backward gradient version controller, where Gr represents the gradient information and V represents the version. This allows the guest to connect to the host's forward embedding data by simply pulling the forward embedding data corresponding to the sample ID from the forward embedding version controller. Meanwhile, when the host needs to obtain the gradient information returned by the guest, it simply pulls the gradient information corresponding to the sample ID from the backward gradient version controller, thus achieving high-performance asynchronous collaboration.

[0116] The time series of the asynchronous longitudinal federated training process is as follows Figure 10 As shown in the figure, the numbers in the figure represent the time required for the corresponding stage process. Figure 10 As shown in the figure, by introducing a version controller, asynchronous model training of federated learning is realized, which reduces the idle waiting and communication overhead between trainers in the federated learning process, and improves the utilization of computing resources and model training efficiency.

[0117] It's worth noting that you can pull the latest forward embedding or gradient information for the corresponding sample ID from the version controller. Alternatively, you can use the step size controller to check the difference between the current version step size and the cached version step size during a pull operation. If the step size exceeds the step size threshold, no data is returned. This allows requesters with versions that are too early to wait for information that meets the step size threshold before continuing model training, effectively controlling trainer utilization and model accuracy.

[0118] It should be understood that the present disclosure uses asynchronous model training of federated learning performed by a first participant and a second participant for illustration. In practical applications, the present disclosure can be applied to asynchronous model training of vertical federated learning performed by any number of parties.

[0119] Based on the same concept, the embodiment of the present disclosure also provides an asynchronous model training device based on federated learning, which is applied to a first participant with a label, such as Figure 11 As shown, the asynchronous model training device 11 based on federated learning includes:

[0120] An acquisition module 111 is configured to obtain a first embedding vector based on the first underlying model and the first feature data, and obtain a second embedding vector from a version controller, wherein the second embedding vector is obtained by an unlabeled second participant based on the second underlying model and the second feature data. The version controller is configured to perform version management on the embedding vector and gradient information obtained during the federated learning process. The first feature data and the second feature data are feature data of the same sample space but different feature dimensions.

[0121] a determination module 112, configured to determine target gradient information based on the first embedding vector, the second embedding vector, the first feature data, a label of the first feature data, and a top model;

[0122] The first bottom model update module 113 is used to update the first bottom model based on the target gradient information and store the target gradient information in the version controller so that the second participant can obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

[0123] Optionally, the version controller is used to store a second embedding vector obtained based on the latest version of the second bottom model; or,

[0124] The version controller is used to store a second embedding vector obtained based on the latest version of the second bottom model and a second embedding vector obtained based on multiple historical versions of the second bottom model.

[0125] Optionally, the acquisition module 111 is used to:

[0126] A second embedding vector obtained based on the latest version of the second bottom model is obtained from the version controller.

[0127] Optionally, the second embedding vector in the version controller is associated with and stored with a model version corresponding to the second bottom model, and the acquisition module 111 is configured to:

[0128] When there is a target model version in the model versions stored in the version controller whose version difference with the top model is less than a preset threshold, a second embedding vector corresponding to the target model version is obtained.

[0129] Optionally, the determining module 112 is configured to:

[0130] Inputting the first embedding vector, the second embedding vector, the first feature data, and the label of the first feature data into the top model to obtain a target embedding vector;

[0131] The target gradient information is determined according to the target embedding vector.

[0132] Optionally, the first bottom model updating module 113 is configured to:

[0133] Splitting the target gradient information based on the first feature data to obtain first gradient information for the first bottom model;

[0134] The first bottom model is updated according to the first gradient information.

[0135] Optionally, the version controller includes an embedding vector version controller and a gradient information version controller, the embedding vector version controller is used to perform version management on the embedding vector obtained in the federated learning process, and the gradient information version controller is used to perform version management on the gradient information obtained in the federated learning process.

[0136] Based on the same concept, the embodiment of the present disclosure also provides an asynchronous model training device based on federated learning, which is applied to the second participant without a label, referring to Figure 12 , the asynchronous model training device 12 based on federated learning includes:

[0137] A storage module 121 is configured to obtain a second embedding vector based on the second bottom model and the second feature data, and store the second embedding vector in a version controller, so that the first participant with the label performs the following operations: obtaining the second embedding vector from the version controller, and determining target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model, and storing the target gradient information in the version controller, wherein the version controller is configured to perform version management on the embedding vector and gradient information obtained in the federated learning process, the first embedding vector is obtained by the first participant based on the first bottom model and the first feature data, and the first feature data and the second feature data are feature data of the same sample space but different feature dimensions;

[0138] The second bottom model updating module 122 is configured to obtain the target gradient information from the version controller and update the second bottom model according to the target gradient information.

[0139] Optionally, the version controller is used to store target gradient information obtained based on the latest version of the top model; or,

[0140] The version controller is used to store target gradient information obtained based on the latest version of the top model and target gradient information obtained based on multiple historical versions of the top model.

[0141] Optionally, the storage module 121 is used to:

[0142] Obtain target gradient information based on the latest version of the top model from the version controller.

[0143] Optionally, the target gradient information in the version controller is associated with and stored with a model version of the corresponding top model, and the storage module 121 is configured to:

[0144] When there is a target model version in the model versions stored in the version controller, the target gradient information corresponding to the target model version is acquired, and the target model version has a version difference with the second base model that is less than a preset threshold.

[0145] Optionally, the second bottom model updating module 122 is configured to:

[0146] Splitting the target gradient information based on the second feature data to obtain second gradient information for the second bottom model;

[0147] The second bottom model is updated according to the second gradient information.

[0148] Based on the same concept, an embodiment of the present disclosure also provides a computer-readable medium on which a computer program is stored, which, when executed by a processing device, implements the steps of any asynchronous model training method based on federated learning.

[0149] Based on the same concept, an embodiment of the present disclosure further provides an electronic device, which may include:

[0150] a storage device having a computer program stored thereon;

[0151] A processing device is used to execute the computer program in the storage device to implement any step of the above-mentioned asynchronous model training method based on federated learning.

[0152] Based on the same concept, an embodiment of the present disclosure also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned asynchronous model training methods based on federated learning.

[0153] Reference below Figure 13 , which shows a schematic structural diagram of an electronic device 130 suitable for implementing the embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 13 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0154] like Figure 13 As shown, electronic device 130 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 131, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 132 or programs loaded from a storage device 138 into a random access memory (RAM) 133. RAM 133 also stores various programs and data required for the operation of electronic device 130. Processing device 131, ROM 132, and RAM 133 are connected to each other via a bus 134. An input / output (I / O) interface 135 is also connected to bus 134.

[0155] Typically, the following devices may be connected to the I / O interface 135: an input device 136 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 137 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 138 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 139. The communication device 139 may allow the electronic device 130 to communicate with other devices wirelessly or by wire to exchange data. Figure 13 The electronic device 130 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0156] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 139, or installed from the storage device 138, or installed from the ROM 132. When the computer program is executed by the processing device 131, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0157] It should be noted that the computer-readable medium described above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wire, optical cable, RF (radio frequency), or any suitable combination thereof.

[0158] In some embodiments, communications may be conducted using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0159] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0160] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a first embedding vector based on the first bottom model and the first feature data, and obtains a second embedding vector from the version controller, wherein the second embedding vector is obtained by the unlabeled second participant based on the second bottom model and the second feature data, and the version controller is used to perform version management on the embedding vector and gradient information obtained in the federated learning process, and the first feature data and the second feature data are feature data with the same sample space and different feature dimensions; determines the target gradient information according to the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model; updates the first bottom model based on the target gradient information, and stores the target gradient information to the version controller so that the second participant obtains the target gradient information from the version controller and updates the second bottom model based on the target gradient information.

[0161] Alternatively, the computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a second embedding vector based on the second bottom model and the second feature data, and stores the second embedding vector to the version controller, so that the first participant with a label performs the following operations: obtains the second embedding vector from the version controller, and determines the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, and stores the target gradient information to the version controller, wherein the version controller is used to perform version management on the embedding vector and gradient information obtained in the federated learning process, the first embedding vector is obtained by the first participant based on the first bottom model and the first feature data, and the first feature data and the second feature data are feature data with the same sample space and different feature dimensions; obtains the target gradient information from the version controller, and updates the second bottom model according to the target gradient information.

[0162] Alternatively, the computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: a first participant with a label obtains a first embedding vector based on the first bottom model and the first feature data; a second participant without a label obtains a second embedding vector based on the second bottom model and the second feature data, and stores the second embedding vector to a version controller, wherein the version controller is used to perform version management on the embedding vectors and gradient information obtained in the federated learning process, and the first feature data and the second feature data are feature data with the same sample space and different feature dimensions; the first participant obtains the second embedding vector from the version controller, determines the target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, updates the first bottom model based on the target gradient information, and stores the target gradient information to the version controller; the second participant obtains the target gradient information from the version controller, and updates the second bottom model based on the target gradient information.

[0163] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0164] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0165] The modules involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a module does not necessarily limit the module itself.

[0166] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0167] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0168] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the scope of the above disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0169] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0170] Although the subject matter has been described using language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims. Regarding the apparatus in the above-described embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method and will not be elaborated upon here.

Claims

1. An asynchronous model training method based on federated learning, characterized in that: Applied to a first party having a tag, the method comprises: Obtaining a first embedding vector based on the first underlying model and the first feature data, and obtaining a second embedding vector from a version controller, wherein the second embedding vector is obtained by an unlabeled second participant based on the second underlying model and the second feature data, the version controller being configured to perform version management on the embedding vector and gradient information obtained during the federated learning process, and the first feature data and the second feature data being feature data of the same sample space but different feature dimensions; determining target gradient information based on the first embedding vector, the second embedding vector, the first feature data, a label of the first feature data, and a top model; The first bottom model is updated based on the target gradient information, and the target gradient information is stored in the version controller, so that the second participant obtains the target gradient information from the version controller and updates the second bottom model based on the target gradient information.

2. The method according to claim 1, characterized in that The version controller is used to store the second embedding vector obtained based on the latest version of the second bottom model; or, The version controller is used to store a second embedding vector obtained based on the latest version of the second bottom model and a second embedding vector obtained based on multiple historical versions of the second bottom model.

3. The method according to claim 1, characterized in that The obtaining of the second embedding vector from the version controller includes: A second embedding vector obtained based on the latest version of the second bottom model is obtained from the version controller.

4. The method according to claim 1, wherein The second embedding vector in the version controller is associated with and stored in a model version corresponding to the second bottom model, and obtaining the second embedding vector from the version controller includes: When there is a target model version in the model versions stored in the version controller whose version difference with the top model is less than a preset threshold, a second embedding vector corresponding to the target model version is obtained.

5. The method according to any one of claims 1 to 4, characterized in that The determining target gradient information according to the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model includes: Inputting the first embedding vector, the second embedding vector, the first feature data, and the label of the first feature data into the top model to obtain a target embedding vector; The target gradient information is determined according to the target embedding vector.

6. The method according to any one of claims 1 to 4, characterized in that The updating of the first bottom model based on the target gradient information includes: Splitting the target gradient information based on the first feature data to obtain first gradient information for the first bottom model; The first bottom model is updated according to the first gradient information.

7. The method according to any one of claims 1 to 4, characterized in that The version controller includes an embedding vector version controller and a gradient information version controller. The embedding vector version controller is used to perform version management on the embedding vector obtained during the federated learning process, and the gradient information version controller is used to perform version management on the gradient information obtained during the federated learning process.

8. An asynchronous model training method based on federated learning, characterized in that: Applied to an untagged second party, the method comprises: A second embedding vector is obtained based on the second bottom model and the second feature data, and the second embedding vector is stored in a version controller, so that the first participant with the label performs the following operations: obtaining the second embedding vector from the version controller, and determining target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data and the top model, and storing the target gradient information in the version controller, wherein the version controller is used to perform version management on the embedding vector and gradient information obtained in the federated learning process, the first embedding vector is obtained by the first participant based on the first bottom model and the first feature data, and the first feature data and the second feature data are feature data of the same sample space but different feature dimensions; The target gradient information is obtained from the version controller, and the second bottom model is updated according to the target gradient information.

9. The method according to claim 8, characterized in that The version controller is used to store target gradient information obtained based on the latest version of the top model; or, The version controller is used to store target gradient information obtained based on the latest version of the top model and target gradient information obtained based on multiple historical versions of the top model.

10. The method according to claim 8, characterized in that The obtaining the target gradient information from the version controller includes: Obtain target gradient information based on the latest version of the top model from the version controller.

11. The method according to claim 8, characterized in that The target gradient information in the version controller is associated with and stored in a model version of the corresponding top model, and obtaining the target gradient information from the version controller includes: When there is a target model version in the model versions stored in the version controller, the target gradient information corresponding to the target model version is acquired, and the target model version has a version difference with the second base model that is less than a preset threshold.

12. The method according to any one of claims 8 to 11, characterized in that The updating of the second bottom model according to the target gradient information includes: Splitting the target gradient information based on the second feature data to obtain second gradient information for the second bottom model; The second bottom model is updated according to the second gradient information.

13. An asynchronous model training system based on federated learning, characterized in that: The asynchronous model training system includes a labeled first participant, an unlabeled second participant and a version controller, wherein the version controller is used to perform version management on the embedding vector and gradient information obtained during the federated learning process, the first participant is used to execute the steps of the method described in any one of claims 1 to 7 above, and the second participant is used to execute the steps of the method described in any one of claims 8 to 12 above.

14. An asynchronous model training device based on federated learning, characterized in that: Applied to a first participant having a tag, the apparatus comprises: an acquisition module, configured to obtain a first embedding vector based on the first underlying model and the first feature data, and to obtain a second embedding vector from a version controller, wherein the second embedding vector is obtained by an unlabeled second participant based on the second underlying model and the second feature data; the version controller is configured to perform version management on the embedding vector and gradient information obtained during the federated learning process; and the first feature data and the second feature data are feature data of the same sample space but different feature dimensions; a determination module, configured to determine target gradient information based on the first embedding vector, the second embedding vector, the first feature data, a label of the first feature data, and a top model; The first bottom model update module is used to update the first bottom model based on the target gradient information and store the target gradient information in the version controller so that the second participant can obtain the target gradient information from the version controller and update the second bottom model based on the target gradient information.

15. An asynchronous model training device based on federated learning, characterized in that: Applied to a second participant without a tag, the apparatus comprises: A storage module is configured to obtain a second embedding vector based on a second bottom model and second feature data, and store the second embedding vector in a version controller, so that the first participant with a label performs the following operations: obtaining the second embedding vector from the version controller, and determining target gradient information based on the first embedding vector, the second embedding vector, the first feature data, the label of the first feature data, and the top model, and storing the target gradient information in the version controller, wherein the version controller is configured to perform version management on the embedding vector and gradient information obtained in the federated learning process, the first embedding vector is obtained by the first participant based on the first bottom model and the first feature data, and the first feature data and the second feature data are feature data of the same sample space but different feature dimensions; The second bottom model updating module is configured to obtain the target gradient information from the version controller and update the second bottom model according to the target gradient information.

16. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 12 are implemented.

17. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 12.

18. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images