Trainer pairing method and apparatus in federated learning model training, and electronic device

Abstract

The present disclosure relates to a trainer pairing method and apparatus in federated learning model training, and an electronic device, achieving automatic pairing of trainers participating in federated learning model training, improving the pairing efficiency, and avoiding the problem of repeated pairing or missing pairing. The method comprises: when a first trainer is started, acquiring the serial number of the first trainer from a counter component; and querying the serial numbers of trainers of a second participant in the counter component, and when there is a target serial number, which is the same as the serial number of the first trainer, among the serial numbers of the trainers of the second participant, taking a second trainer corresponding to the target serial number as a paired trainer of the first trainer, wherein the first trainer and the second trainer are used for cooperatively performing model training in a federated learning process, and trainers started by a first participant and the second participant are numbered according to the same rule.

Description

Trainer pairing method, device and electronic equipment in federated learning model training

[0001] This application claims priority to Chinese Patent Application No. 202410704094.4, filed May 31, 2024, the disclosure of which is incorporated herein in its entirety as part of the present application. TECHNICAL FIELD

[0002] The present disclosure relates to a trainer pairing method, device and electronic equipment in federated learning model training. BACKGROUND

[0003] Longitudinal federated learning is a privacy-preserving machine learning paradigm that can combine data from multiple participants to perform secure machine learning training tasks. In a large-scale longitudinal federated learning scenario, for example with two participants, both parties will start a large number of trainers simultaneously and need to pair the trainers of both parties two by two.

[0004] In related technologies, a manual pairing method is usually used, i.e., a mapping pairing table of the trainers of both parties is manually created. However, manual pairing is inefficient and time-consuming, which reduces the timeliness of the model and can cause problems such as repeated pairing or missed pairing, so that the training data of the trainers cannot be aligned, resulting in abnormal training indicators. SUMMARY

[0005] This summary is provided to introduce a selection of concepts, which will be described in more detail below in the detailed description section. This summary is not intended to identify key or essential features of the claimed technology, nor is it intended to limit the scope of the claimed technology.

[0006] In a first aspect, the present disclosure provides a trainer pairing method in federated learning model training, applied to a first trainer of a first participant, the method comprising:

[0007] After the first trainer is started, the number of the first trainer itself is obtained from a counter component;

[0008] The number of the trainer of the second participant is queried in the counter component, and when there is a target number same as the number of the first trainer in the number of the trainer of the second participant, the second trainer corresponding to the target number is taken as the paired trainer of the first trainer;

[0009] The first trainer and the second trainer are used to cooperatively perform model training in a federated learning process, and the trainers started by the first participant and the second participant are numbered according to the same rule.

[0010] In a second aspect, the present disclosure provides a trainer pairing apparatus in a federated learning model training, applied to a first trainer of a first participant, the apparatus comprising:

[0011] an obtaining module configured to, after the first trainer is started, obtain a number of the first trainer itself from a counter component;

[0012] a pairing module configured to query a number of a trainer of a second participant in the counter component, and when there is a target number same as the number of the first trainer in the number of the trainer of the second participant, take a second trainer corresponding to the target number as a pairing trainer of the first trainer;

[0013] wherein the first trainer and the second trainer are configured to cooperatively perform model training in a federated learning process, and the trainers started by the first participant and the second participant are numbered according to the same rule.

[0014] In a third aspect, the present disclosure provides a computer readable medium having a computer program stored thereon, the program being executed by a processing apparatus to implement the steps of the method of any one of the first aspect.

[0015] In a fourth aspect, the present disclosure provides an electronic device comprising:

[0016] a storage device having a computer program stored thereon;

[0017] a processing apparatus configured to execute the computer program in the storage device to implement the steps of the method of any one of the first aspect.

[0018] In a fifth aspect, the present disclosure provides a computer program product comprising a computer program, the computer program being executed by a processor to implement the steps of the method of any one of the first aspect.

[0019] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other features and advantages of the embodiments of the present disclosure will become more apparent by describing in detail the embodiments thereof with reference to the attached drawings in which:

[0021] FIG. 1 is a process schematic diagram of data alignment in a vertical federated learning according to an exemplary embodiment of the present disclosure;

[0022] FIG. 2 is a flow chart illustrating a trainer pairing method in federated learning model training according to an example embodiment of the present disclosure;

[0023] FIG. 3 is a process diagram illustrating a trainer pairing according to an example embodiment of the present disclosure;

[0024] FIG. 4 is a diagram illustrating a cooperative control according to an example embodiment of the present disclosure;

[0025] FIG. 5 is a diagram illustrating a trainer pairing and cooperative control based on a distributed cooperative component according to an example embodiment of the present disclosure;

[0026] FIG. 6 is a timing diagram illustrating a trainer pairing and cooperative control based on a distributed cooperative component according to an example embodiment of the present disclosure;

[0027] FIG. 7 is a structural block diagram of a trainer pairing apparatus in federated learning model training according to an example embodiment of the present disclosure;

[0028] FIG. 8 is a structural diagram of an electronic device according to an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While several embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the present disclosure to those skilled in the art. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be understood that each of the steps recited in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] The term "comprising" and variations thereof as used herein are used inclusively, i.e., "comprising, but not limited to." The term "based on" is "based at least in part on." The term "one embodiment" means "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The term "some embodiments" means "at least some embodiments." Related definitions will be given in the description below.

[0032] It should be noted that the terms "first", "second", and the like used in the present disclosure are merely used to distinguish different devices, modules, or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules, or units.

[0032]

[0033] It should be noted that the modification of "one", "multiple" mentioned in the present disclosure is illustrative but not restrictive, and those skilled in the art should understand that unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0034] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0035] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in a proper manner according to relevant laws and regulations.

[0036] For example, in response to receiving the active request of the user, the prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using the personal information of the user. Thus, the user can voluntarily choose whether to provide the personal information to the software or hardware such as electronic device, application program, server or storage medium, etc. that performs the operation of the technical solution of the present disclosure according to the prompt information.

[0037] As an optional but not limited implementation manner, in response to receiving the active request of the user, the prompt information can be sent to the user in the form of a pop-up window, and the prompt information can be presented in the form of text in the pop-up window. In addition, the pop-up window can also carry selection controls for the user to select "agree" or "disagree" to provide the personal information to the electronic device.

[0038] It can be understood that the above notification and obtaining of user authorization process is only illustrative, and does not limit the implementation manner of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0039] At the same time, it can be understood that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the relevant laws and regulations and relevant provisions.

[0040] In longitudinal federated learning, the feature distribution of the training data of model training is in multiple participants, one of which holds the label (Label) at the same time. Taking two-party single-machine longitudinal federated learning of A party and B party as an example, as shown in FIG. 1, first, the training data of A party and B party are aligned, that is, the data of the same ID identifier are aligned. For example, A party is A company and B party is B company, A company and B company provide different services to users, and there are some overlapping users between the two companies. The ID identifier is the user ID, and the feature data of A company and B company are aligned, which is equivalent to obtaining the feature data of the overlapping users in A company and B company respectively. The aligned training data are input into the training device of both parties for model training. In the training process, the training devices of both parties interact through the federated connection layer (model forward and backward propagation). And during all the training processes, the training data of both parties need to be kept in alignment, otherwise the training will be disordered and the training accuracy will be abnormal.

[0041] In the two-party single-machine longitudinal federated learning scenario, data alignment is relatively simple. However, in actual application, we need to face super large-scale data and super long training process, and single-machine training cannot meet the actual performance and stability requirements, so distributed multi-machine parallel training is needed. At the same time, due to the high real-time requirement of the model in some scenarios, long-time online training of the model is required.

[0042] Therefore, in the large-scale longitudinal federated learning scenario, a large number of training devices will be started simultaneously by both parties, that is, two-party multi-machine longitudinal federated learning model training, so it is necessary to pair the training devices of both parties two by two. After pairing, training can continue in the original way. In related technologies, manual pairing is usually used, that is, a mapping pairing table of the training devices of both parties is manually created, and both parties obtain the pairing information from the mapping pairing table and communicate with each other according to the information.

[0043] However, two-by-two pairing first requires consistency, that is, it cannot be repeatedly paired (a training device cannot be paired with multiple training devices of the other party) and cannot be missed. If the pairing does not meet the consistency, the training data entering the training device cannot be aligned, resulting in abnormal training indicators. Secondly, manual pairing is low in efficiency and time-consuming, which reduces the timeliness of the model.

[0044] At the same time, in long-time training, it is inevitable to encounter unexpected exit of training devices, such as machine failure, migration, maintenance, etc. If the training device of one party exits unexpectedly, how to efficiently coordinate the exit of the paired training device also needs to be solved.

[0045] In view of this, the present disclosure provides a trainer pairing method, device and electronic equipment in federated learning model training to solve the above technical problems. It should be noted that the trainer pairing method in federated learning model training provided by the present disclosure can be applied to large-scale vertical federated learning scenarios. Each participant can be in a distributed structure, that is, each participant includes multiple trainers.

[0046] The embodiments of the present disclosure are further explained in conjunction with the accompanying drawings. In order to facilitate description, two-party vertical federated learning model training of A party and B party is used for embodiment description. In actual application, the model training can be applied to vertical federated learning of any number of parties.

[0047] FIG. 2 is a flowchart of a trainer pairing method in federated learning model training according to an exemplary embodiment of the present disclosure. Referring to FIG. 2, the method is applied to a first trainer of a first participant and includes:

[0048] S201: After the first trainer is started, the first trainer obtains its own number from a counter component.

[0049] S202: Query the number of the second participant's trainer in the counter component. When the number of the second participant's trainer contains a target number identical to the number of the first trainer, the second trainer corresponding to the target number is used as the paired trainer of the first trainer.

[0050] The first trainer and the second trainer are used to cooperatively perform model training in the federated learning process, and the trainers started by the first participant and the second participant are numbered according to the same rule.

[0051] Using the above method, first, the number of the trainer itself is obtained after the trainer is started. The trainers of the first participant and the second participant are numbered according to the same rule. Then, the trainers with the same number are paired as paired trainers. The paired trainers can cooperatively perform model training in the federated learning process. Thus, the automatic pairing of the trainers participating in the federated learning model training is realized, the pairing efficiency is improved, the pairing time consumption is reduced, the model timeliness is improved, and the problem of repeated pairing or missed pairing of the trainers is avoided, so that the training data of the trainers can be aligned, and the training indicators are ensured to be normal.

[0052] In a possible manner, the number of trainers started by the first participant and the second participant is equal.

[0053] In a possible manner, step S201 can include: after the first trainer is started, sending a registration request to the counter component, so that the counter component responds to the registration request and uses the registration serial number of the first trainer as the number of the first trainer; and obtaining the number of the first trainer itself from the counter component.

[0054] For example, as shown in FIG. 3, after the trainer is started, a counter component obtains a count value. The counter component can be a distributed counter, and can be determined according to requirements, as long as the counting function can be implemented, and the disclosure does not limit this. It should be understood that the starting values of the count values obtained by the two trainers of the participating parties in the vertical federated learning model training are the same, and the increment rules are the same, for example, the count values are all started from 0 and are incremented one by one, and the disclosure does not limit this. It is only necessary to ensure that the trainers of the two parties are numbered according to the same rule.

[0055] In a possible manner, the counter component stores a first number list including the trainer number of the first participating party and a second number list including the trainer number of the second participating party, the trainer number of the first participating party being determined by the counter component in response to a registration request of the trainer of the first participating party, and the trainer number of the second participating party being determined by the counter component in response to a registration request of the trainer of the second participating party.

[0056] For example, the counter component can store the trainer numbers of the multiple participating parties in response to the registration requests sent by the trainers of the multiple participating parties. Accordingly, when a certain trainer exits the model training, an exit request can be sent to the counter component, and the counter component deletes the number of the trainer in the number list. By managing the trainer numbers through the number list, it is convenient to maintain and update the trainer numbers of the multiple participating parties and subsequently pair and cooperatively control the trainers.

[0057] In a possible manner, querying the trainer number of the second participating party in the counter component can include: querying the second number list in the counter component through a number query interface to obtain the trainer number of the second participating party, and the number query interface is an interface provided by the counter component for querying the trainer number.

[0058] For example, the counter component can provide an interface for querying the trainer number, and the first trainer can query the second number list in the counter component through the number query interface to obtain the trainer number of the second participating party, and then determine the paired trainer according to the trainer number of the second participating party. Accordingly, the second trainer can also query the first number list in the counter component through the number query interface to obtain the trainer number of the first participating party, and then determine the paired trainer according to the trainer number of the first participating party.

[0059] Further, the trainers with the same count value constitute a pair of paired trainers, and the communication between the paired trainers can be based on the count value. In a possible manner, the method further includes: determining a communication channel based on the number of the first trainer, so as to send a message to the second trainer through the communication channel.

[0060] For example, taking the training task job_id_0 as an example, the training device 2 of the A party sends a message to the paired training device of the B party (i.e., the training device 1 of the B party), which can send a message to the Topic (topic, which can be understood as a communication channel) of the communication component “ / job_id_0 / B / 2”, and then the training device 1 of the B party receives the message from the Topic “ / job_id_0 / B / 2” of the communication component.

[0061] It is worth noting that the pairing process of the present disclosure does not require interaction between the two parties, but only simple interaction with the counter component. The counter component can be deployed in an electronic device capable of interacting with the first participant and the second participant, and the present disclosure does not limit this. Since the counter component can guarantee strong consistency, the pairing of the training device based on the counter component can also guarantee consistency, so that even in the case of thousands of training devices pairing at the same time, the situation of repeated pairing of training devices and missing pairing of training devices will not occur, thereby realizing efficient and stable pairing of training devices.

[0062] In a possible manner, the method further comprises: querying the training device number of the second participant in the counter component, and when there is no number same as the number of the first training device in the training device number of the second participant, and there is a number greater than the number of the first training device, determining to exit the model training.

[0063] For example, when the number obtained by the training device of the first participant is 4, if there is no 4 in the training device number of the second participant, and there is a number 5 greater than the number 4, it indicates that the second participant has a training device that has obtained the number 4 after starting, but before pairing, it may have exited the model training due to failure or other reasons, and deleted the number 4. Therefore, the training device with the number 4 of the first participant can exit the model training in coordination, so that after restarting to obtain a new number, a new paired training device is determined to participate in the model training, thereby realizing efficient and coordinated control of the training devices of the two parties.

[0064] In a possible manner, the method further comprises: querying the training device number of the second participant in the counter component, and when there is no number same as the number of the first training device in the training device number of the second participant, and there is no number greater than the number of the first training device, continuing to query the newly added number of the training device of the second participant; and determining whether to exit the model training according to the newly added number.

[0065] For example, when the number obtained by the training device of the first participant is 4, if there is no 4 in the training device number of the second participant, and there is no number greater than the number 4, the newly added number of the training device of the second participant can be continuously queried, and whether to exit the model training is determined according to the newly added number.

[0066] It should be understood that, in the case of authorization of the other party, the number addition, deletion, etc. of the training device of the other party can be monitored through a monitoring mechanism, and the number of the training device is irrelevant to the training data, which can ensure data privacy and security.

[0067] In a possible manner, determining whether to exit the model training according to the added number can include: when the added number is the same as the number of the first training device, taking the third training device corresponding to the added number as the paired training device of the first training device, and the first training device and the third training device are used to cooperatively perform model training in the federated learning process; and when the added numbers are all different from the number of the first training device, and the added numbers are greater than the number of the first training device, it is determined to exit the model training.

[0068] For example, the number obtained by the training device of the first participant is 4, and the numbers obtained by the training devices of the second participant include 0, 1, 2 and 3. If the newly started training device X of the second participant obtains the number 4, it indicates that the training device of the second participant has not started the training device X when the training device of the first participant with the number 4 obtains the number. The training devices with the number 4 of the two parties can be taken as a group of paired training devices.

[0069] Alternatively, if the newly started training device X of the second participant obtains the number 5, it indicates that the training device in the second participant has obtained the number 4, but before pairing, the training device may have exited the model training due to failure or other reasons, and deleted the number 4. The training device with the number 4 of the first participant can cooperatively exit the model training, so that after restarting to obtain a new number, a new paired training device is determined to participate in the model training, and efficient cooperative control of the training devices of the two parties is realized.

[0070] It should be understood that, if the number deletion event can be monitored before pairing, for example, the number obtained by the training device of the first participant is 4, and it is monitored that the training device with the number 4 of the second participant has exited the model training, the training device with the number 4 of the first participant can cooperatively exit the model training.

[0071] In a possible manner, the method further includes: after taking the second training device corresponding to the target number as the paired training device of the first training device, querying the operation events triggered by the second training device in the counter component through an event query interface, the event query interface being an interface provided by the counter component for querying the operation events of the training device; and when the number deletion event or the session loss event triggered by the second training device is queried, it is determined to exit the model training.

[0072] For example, after successful pairing, the number deletion event or session loss event of the paired trainer of the other party can be monitored based on the monitoring mechanism with the authorization of the other party. The number deletion event indicates that the trainer of the other party exits the model training, and the session loss event indicates that the trainer of the other party does not respond. If the number deletion event or the session loss event of the trainer of the other party is monitored, the trainer of the self-party can exit the model training cooperatively, thereby realizing efficient cooperative control of the trainers of the two parties.

[0073] For example, by using the node monitoring mechanism, the trainers can monitor each other to perceive the state of the trainer of the other party, thereby realizing cooperative training of the distributed trainers. As shown in FIG. 4, after successful pairing of the trainers, a self-party node is created, and the node identification path uses the count value of the self-party, for example, “ / {job_id} / {role} / {count_value}”. For example, the trainer of the self-party with the count value of 0 for the training task job_0 of the party A can be represented as / job_0 / A / 0.

[0074] Further, the node corresponding to the paired trainer of the other party is monitored. Since the trainers with the same count value are paired, the node path of the node of the other party can be determined based on the count value of the self-party. If the trainer corresponding to the node 0 of the self-party exits the model training, the node 0 created by the self-party is deleted. If the deletion event of the node 0 of the other party is monitored, the trainer corresponding to the node 0 of the self-party exits cooperatively. In this way, when the node of the self-party exits, the paired node of the other party can also perceive in a timely manner, thereby realizing fast and stable cooperative control.

[0075] By using the above method, the pairing of the trainers for the model training of the large-scale vertical federated learning can be realized based on the counter component. The two trainers respectively obtain the count value which is incremental and unique (unique in the self-party) from the counter component. The two trainers with the same count value automatically form the paired trainers. Based on the node monitoring mechanism, the cooperative control of the trainers for the model training of the large-scale vertical federated learning is realized, that is, each trainer creates a node representing the state of the self-party. The self-party trainer realizes cooperative control by monitoring the state of the paired node.

[0076] In a possible manner, the functions of the above counter component and monitoring mechanism can be realized by using a distributed application coordination service software which simultaneously has a distributed counter and a monitoring mechanism, which is hereinafter referred to as a distributed cooperative component. The distributed cooperative component can create a sequential node, the sequential node has an incremental and unique serial number, and can monitor the node creation, node deletion, and sub-node change events. Hereinafter, an example of the training task job_0 performed by the trainers of the party A and the party B is taken to illustrate the pairing and cooperative control of the trainers for the vertical federated learning based on the distributed cooperative component.

[0077] As shown in FIG. 5, after the self-side trainer is started, an ordered node is created to the distributed coordination component, and the order value of the node is obtained as a count value, that is, the number of the trainer. For the convenience of description, it is assumed that the count value corresponding to the trainer is 0, and the ordered node is represented as worker_0. The self-side trainer can query whether the worker_0 node exists in the ordered node created by the opposite side through an interface, and if the worker_0 node exists, the pairing is successful.

[0078] Alternatively, if the worker_0 node created by the opposite side does not exist, it is further queried whether the trainer corresponding to the worker_0 node of the opposite side has exited the model training, and if the trainer has exited, the trainer corresponding to the worker_0 node of the self side is also controlled to exit the model training, so as to realize the cooperative control before pairing.

[0079] For example, if the worker_0 node of the opposite side does not exist, and there is another node with a larger sequence number, it indicates that the trainer corresponding to the worker_0 node of the opposite side has exited the model training, and the trainer corresponding to the worker_0 node of the self side is also controlled to exit the model training. Alternatively, if the worker_0 node of the opposite side does not exist, and there is no other node with a larger sequence number, the change of the child node under / job_0 / B is listened to through the listening mechanism of the distributed coordination component, if a new node is added, and the sequence number of the new node is greater than that of the worker_0 node, it indicates that the trainer corresponding to the worker_0 node of the opposite side has exited the model training, and the trainer corresponding to the worker_0 node of the self side is also controlled to exit the model training. Alternatively, if the created event of the worker_0 node of the opposite side is listened to, the trainers corresponding to the worker_0 nodes of the two sides are regarded as a pair of paired trainers.

[0080] Further, after the pairing is successful, an additional thread is started in the background to listen to the node deletion event or the Session loss event of the opposite node, if the node deletion event or the Session loss event is listened to, the trainer corresponding to the node of the self side is controlled to exit the model training, so as to realize the cooperative control after pairing. The trainer pairing and cooperative control dynamic interaction process of the model training based on the distributed coordination component can be referred to FIG. 6.

[0081] It is worth noting that the implementation of the distributed coordination component in the embodiments of the present disclosure is not limited, and the distributed counter and the node listening mechanism can be realized through Zookeeper at the same time, or the distributed counter can be realized by redis, mysql, etc., and the node listening mechanism can be realized by etcd, consul, etc. The specific implementation can be determined according to the requirements.

[0082] Based on the same concept, the embodiments of the present disclosure also provide a trainer pairing device in federated learning model training, as shown in FIG. 7, the trainer pairing device 700 applied to a first trainer of a first participant in federated learning model training, comprising:

[0083] The acquisition module 701 is configured to acquire the number of the first trainer itself from the counter component after the first trainer is started;

[0084] The pairing module 702 is configured to query the number of a trainer of a second participant in the counter component, and when there is a target number same as the number of the first trainer in the number of the trainer of the second participant, take the second trainer corresponding to the target number as the paired trainer of the first trainer.

[0085] The first trainer and the second trainer are configured to cooperatively perform model training in a federated learning process, and the trainers started by the first participant and the second participant are numbered according to the same rule.

[0086] With the above device, the number of the trainer itself is acquired after the trainer is started, the trainers of the first participant and the second participant are numbered according to the same rule, and then the trainers with the same number are paired as the paired trainers, which can cooperatively perform model training in a federated learning process. Thus, the automatic pairing of the trainers participating in the federated learning model training is realized, the pairing efficiency is improved, the pairing time consumption is reduced, the model timeliness is improved, and the problem of repeated pairing or missed pairing of the trainers can be avoided, so that the training data of the trainers can be aligned, and the training index can be ensured to be normal.

[0087] Optionally, the trainer pairing device 700 in federated learning model training further comprises:

[0088] The first determination module is configured to query the number of the trainer of the second participant in the counter component, and when there is no number same as the number of the first trainer in the number of the trainer of the second participant, and there is no number greater than the number of the first trainer, continue to query the newly added number of the trainer in the second participant;

[0089] The first control module is configured to determine whether to exit the model training according to the newly added number.

[0090] Optionally, the first control module is configured to:

[0091] When the newly added number is the same as the number of the first trainer, take the third trainer corresponding to the newly added number as the paired trainer of the first trainer, and the first trainer and the third trainer are configured to cooperatively perform model training in a federated learning process.

[0092] When the added numbers are all different from the number of the first trainer, and the added numbers are greater than the number of the first trainer, it is determined to exit the model training.

[0093] Optionally, the trainer pairing apparatus 700 in the federated learning model training further comprises:

[0094] The second control module is configured to query the number of the trainer of the second participant in the counter component, and determine to exit the model training when there is no number same as the number of the first trainer in the number of the trainer of the second participant, and there is a number greater than the number of the first trainer.

[0095] Optionally, the trainer pairing apparatus 700 in the federated learning model training further comprises:

[0096] The second determination module is configured to query, after the second trainer corresponding to the target number is determined as the paired trainer of the first trainer, an operation event triggered by the second trainer in the counter component through an event query interface, the event query interface being an interface provided by the counter component for querying the operation event of the trainer.

[0097] The third control module is configured to determine to exit the model training when the number deletion event or the session loss event triggered by the second trainer is queried.

[0098] Optionally, the obtaining module 701 is configured to:

[0099] After the first trainer is started, the first trainer sends a registration request to the counter component, so that the counter component determines the registration serial number of the first trainer as the number of the first trainer in response to the registration request.

[0100] The first trainer obtains its own number from the counter component.

[0101] Optionally, the counter component stores a first number list comprising the number of the trainer of the first participant and a second number list comprising the number of the trainer of the second participant, the number of the trainer of the first participant being determined by the counter component in response to the registration request of the trainer of the first participant, and the number of the trainer of the second participant being determined by the counter component in response to the registration request of the trainer of the second participant.

[0102] Optionally, the pairing module 702 is configured to:

[0103] query a second number list in the counter component through a number query interface, to obtain the trainer number of the second participant, the number query interface being an interface provided by the counter component for querying a trainer number.

[0104] Optionally, the trainer pairing apparatus 700 in federated learning model training further includes:

[0105] The third determination module is configured to determine a communication channel based on the number of the first trainer, so as to send a message to the second trainer through the communication channel.

[0106] As to the apparatus in the above-described embodiments, the specific manners in which various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described here in detail.

[0107] Based on the same idea, the embodiments of the present disclosure further provide a computer readable medium having a computer program stored thereon, the program being executed by a processing apparatus to implement the steps of the trainer pairing method in federated learning model training.

[0108] Based on the same idea, the embodiments of the present disclosure further provide an electronic device, which can include:

[0109] a storage apparatus having a computer program stored thereon;

[0110] a processing apparatus configured to execute the computer program in the storage apparatus to implement the steps of the above-described trainer pairing method in federated learning model training.

[0111] Based on the same idea, the embodiments of the present disclosure further provide a computer program product including a computer program, the computer program being executed by a processor to implement the steps of the above-described trainer pairing method in federated learning model training.

[0112] Reference is made below to FIG. 8, which shows a structural schematic diagram of an electronic device 800 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle terminal (e.g., vehicle navigation terminal), and the like, as well as fixed terminal such as digital TV, desktop computers, and the like. The electronic device shown in FIG. 8 is merely an example, and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0113] As shown in FIG. 8, the electronic device 800 can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 802 or loaded into a random access memory (RAM) 803 from a storage device 808. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0114] In general, the following devices can be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication devices 809 can allow the electronic device 800 to communicate wirelessly or wired with other devices to exchange data. Although FIG. 8 shows the electronic device 800 with various devices, it should be understood that all of the shown devices are not required to be implemented or possessed. More or less devices can be alternatively implemented or possessed.

[0115] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 809, or installed from the storage devices 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-described functions defined in the methods of embodiments of the present disclosure are performed.

[0116] It should be noted that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the computer-readable program code is contained. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wire, cable, RF (radio frequency), etc., or any suitable combination of the above.

[0117] In some embodiments, communication can be conducted using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

[0118] The computer-readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device without being assembled into the electronic device.

[0119] The computer readable medium carries one or more programs when the one or more programs are executed by the electronic device, the electronic device is caused to: after the first trainer is started, obtain the number of the first trainer itself from a counter component; query the number of the trainer of a second participant in the counter component, when there is a target number same as the number of the first trainer in the number of the trainer of the second participant, take the second trainer corresponding to the target number as the paired trainer of the first trainer; wherein the first trainer and the second trainer are used to cooperatively perform model training in a federated learning process, and the trainers started by the first participant and the second participant are numbered according to the same rule.

[0120] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0121] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified functions. It should also be noted that, in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or in the opposite order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware-based systems and computer instructions.

[0122] The modules described in the embodiments of the present disclosure can be implemented in the form of software, or can be implemented in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0123] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, example types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), etc.

[0124] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0125] In a first aspect, one or more embodiments of the present disclosure provide a trainer pairing method in federated learning model training, applied to a first trainer of a first participant, the method comprising:

[0126] After the first trainer is started, the number of the first trainer itself is obtained from a counter component;

[0127] The number of the trainer of the second participant is queried in the counter component, and when the number of the trainer of the second participant contains a target number same as the number of the first trainer, the second trainer corresponding to the target number is taken as the paired trainer of the first trainer;

[0128] The first trainer and the second trainer are used to cooperatively perform model training in the federated learning process, and the trainers started by the first participant and the second participant are numbered according to the same rule.

[0129] Optionally, according to the trainer pairing method in federated learning model training of one or more embodiments of the present disclosure, the method further comprises:

[0130] query the trainer number of the second participant in the counter component, when there is no number same as the number of the first trainer and no number greater than the number of the first trainer in the trainer number of the second participant, continue to query the newly added number of the trainer in the second participant;

[0131] determine whether to exit the model training according to the newly added number.

[0132] Optionally, in the trainer pairing method in the federated learning model training according to one or more embodiments of the present disclosure, the determination of whether to exit the model training according to the newly added number comprises:

[0133] when the newly added number is same as the number of the first trainer, the third trainer corresponding to the newly added number is taken as the paired trainer of the first trainer, and the first trainer and the third trainer are used to cooperatively perform the model training in the federated learning process;

[0134] when the newly added numbers are all different from the number of the first trainer and the newly added numbers are all greater than the number of the first trainer, it is determined to exit the model training.

[0135] Optionally, in the trainer pairing method in the federated learning model training according to one or more embodiments of the present disclosure, the method further comprises:

[0136] query the trainer number of the second participant in the counter component, when there is no number same as the number of the first trainer and no number greater than the number of the first trainer in the trainer number of the second participant, continue to query the newly added number of the trainer in the second participant;

[0137] Optionally, in the trainer pairing method in the federated learning model training according to one or more embodiments of the present disclosure, the method further comprises:

[0138] after taking the second trainer corresponding to the target number as the paired trainer of the first trainer, query the operation event triggered by the second trainer in the counter component through an event query interface, the event query interface being an interface provided by the counter component for querying the operation event of the trainer;

[0139] when the number deletion event or the session loss event triggered by the second trainer is queried, it is determined to exit the model training.

[0140] Optionally, in the trainer pairing method in the federated learning model training according to one or more embodiments of the present disclosure, after the first trainer is started, the number of the first trainer itself is obtained from the counter component, comprising:

[0141] after the first trainer is started, a registration request is sent to the counter component, so that the counter component takes the registration serial number of the first trainer as the number of the first trainer in response to the registration request;

[0142] obtaining a number of the first trainer from a counter component.

[0143] Optionally, in the method for pairing trainers in federated learning model training according to one or more embodiments of the present disclosure, the counter component stores a first number list including the number of the trainer of the first participant and a second number list including the number of the trainer of the second participant, the number of the trainer of the first participant being determined by the counter component in response to a registration request of the trainer of the first participant, and the number of the trainer of the second participant being determined by the counter component in response to a registration request of the trainer of the second participant.

[0144] Optionally, in the method for pairing trainers in federated learning model training according to one or more embodiments of the present disclosure, querying the number of the trainer of the second participant in the counter component comprises:

[0145] querying the second number list in the counter component through a number query interface to obtain the number of the trainer of the second participant, the number query interface being an interface provided by the counter component for querying the number of the trainer.

[0146] Optionally, in the method for pairing trainers in federated learning model training according to one or more embodiments of the present disclosure, the method further comprises:

[0147] determining a communication channel based on the number of the first trainer, so as to send a message to the second trainer through the communication channel.

[0148] In a second aspect, one or more embodiments of the present disclosure further provide a device for pairing trainers in federated learning model training, applied to a first trainer of a first participant, comprising:

[0149] an obtaining module, configured to obtain a number of the first trainer from a counter component after the first trainer is started;

[0150] a pairing module, configured to query a number of a trainer of a second participant in the counter component, and when the number of the trainer of the second participant includes a target number same as the number of the first trainer, take the second trainer corresponding to the target number as a paired trainer of the first trainer.

[0151] The first trainer and the second trainer are used to cooperatively perform model training in a federated learning process, and the trainers started by the first participant and the second participant are numbered according to the same rule.

[0152] Optionally, in the device for pairing trainers in federated learning model training according to one or more embodiments of the present disclosure, the device for pairing trainers in federated learning model training further comprises:

[0153] The first determination module is configured to query the number of the trainer of the second participant in the counter component, and when there is no number same as the number of the first trainer and no number greater than the number of the first trainer in the number of the trainer of the second participant, continue to query the newly added number of the trainer of the second participant.

[0154] The first control module is configured to determine whether to exit the model training according to the newly added number.

[0155] Optionally, the trainer pairing apparatus in the federated learning model training according to one or more embodiments of the present disclosure, the first control module is configured to:

[0156] When the newly added number is same as the number of the first trainer, the third trainer corresponding to the newly added number is used as the paired trainer of the first trainer, and the first trainer and the third trainer are used to cooperatively perform the model training in the federated learning process.

[0157] When the newly added numbers are all different from the number of the first trainer and the newly added numbers are all greater than the number of the first trainer, it is determined to exit the model training.

[0158] Optionally, the trainer pairing apparatus in the federated learning model training according to one or more embodiments of the present disclosure, the trainer pairing apparatus in the federated learning model training further comprises:

[0159] The second control module is configured to query the number of the trainer of the second participant in the counter component, and when there is no number same as the number of the first trainer and there is a number greater than the number of the first trainer in the number of the trainer of the second participant, determine to exit the model training.

[0160] Optionally, the trainer pairing apparatus in the federated learning model training according to one or more embodiments of the present disclosure, the trainer pairing apparatus in the federated learning model training further comprises:

[0161] The second determination module is configured to, after taking the second trainer corresponding to the target number as the paired trainer of the first trainer, query an operation event triggered by the second trainer in the counter component through an event query interface, the event query interface being an interface provided by the counter component for querying the operation event of the trainer.

[0162] The third control module is configured to, when the number deletion event or the session loss event triggered by the second trainer is queried, determine to exit the model training.

[0163] Optionally, the trainer pairing apparatus in the federated learning model training according to one or more embodiments of the present disclosure, the acquisition module is configured to:

[0164] After the first trainer is started, a registration request is sent to the counter component, so that the counter component determines the registration number of the first trainer as the number of the first trainer in response to the registration request;

[0165] The number of the first trainer itself is obtained from the counter component.

[0166] Optionally, the trainer pairing apparatus in federated learning model training according to one or more embodiments of the present disclosure, the counter component stores a first number list including the number of the trainer of the first participant and a second number list including the number of the trainer of the second participant, the number of the trainer of the first participant is determined by the counter component in response to the registration request of the trainer of the first participant, and the number of the trainer of the second participant is determined by the counter component in response to the registration request of the trainer of the second participant.

[0167] Optionally, the trainer pairing apparatus in federated learning model training according to one or more embodiments of the present disclosure, the pairing module is configured to:

[0168] The second number list in the counter component is queried through a number query interface to obtain the number of the trainer of the second participant, and the number query interface is an interface provided by the counter component for querying the number of the trainer.

[0169] Optionally, the trainer pairing apparatus in federated learning model training according to one or more embodiments of the present disclosure, the trainer pairing apparatus in federated learning model training further comprises:

[0170] The third determination module is configured to determine a communication channel based on the number of the first trainer, so as to send a message to the second trainer through the communication channel.

[0171] As to the apparatuses in the above embodiments, the specific manners in which various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described here in detail.

[0172] In a third aspect, one or more embodiments of the present disclosure further provide a computer readable medium having a computer program stored thereon, the program being executed by a processing apparatus to implement the steps of the trainer pairing method in federated learning model training.

[0173] In a fourth aspect, one or more embodiments of the present disclosure further provide an electronic device, which can include:

[0174] A storage device having a computer program stored thereon;

[0175] A processing apparatus configured to execute the computer program in the storage device to implement the steps of the above-mentioned trainer pairing method in federated learning model training.

[0176] In a fifth aspect, one or more embodiments of the present disclosure further provide a computer program product comprising a computer program which, when executed by a processor, implements the steps of the training device pairing method in the federated learning model training.

[0177] The above description is merely that of the preferred embodiments of the present disclosure and the principles of the technology employed, and the scope of the disclosure disclosed herein is not limited to the specific combinations of technical features described above. Rather, the scope of the disclosure disclosed herein should be encompassed by any other technical solutions formed by the combinations of the technical features described above or their equivalents without departing from the concept of the present disclosure. For example, the technical solutions formed by the mutual replacement of the above-described features and the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0178] In addition, although each operation is described in a particular order, this should not be understood as requiring the operations to be performed in the specific order shown or in a sequential order. In certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be separated and implemented in multiple embodiments.

[0179] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely illustrative of the example forms of implementing the claims. As to the devices in the above-described embodiments, the specific manner in which the various modules perform operations has been described in detail in the embodiments related to the method, and will not be described here in detail.

Claims

1. A trainer pairing method in federated learning model training, characterized in that, The method, applied to a first trainer of the first participant, includes: After the first trainer is started, its own number is obtained from the counter component; Query the trainer number of the second participant in the counter component. When there is a target number in the trainer number of the second participant that is the same as the number of the first trainer, the second trainer corresponding to the target number is used as the paired trainer of the first trainer. The first trainer and the second trainer are used to collaboratively train the model during the federated learning process, and the trainers initiated by the first participant and the second participant are numbered according to the same rules.

2. The method of claim 1, wherein, The method further includes: Query the trainer number of the second participant in the counter component. If there is no trainer number in the second participant that is the same as the number of the first trainer and there is no trainer number that is larger than the number of the first trainer, continue to query the newly added trainer number in the second participant. Based on the newly added number, determine whether to exit the model training.

3. The method of claim 2, wherein, The step of determining whether to exit the model training based on the newly added number includes: When the newly added number is the same as the number of the first trainer, the third trainer corresponding to the newly added number is used as the paired trainer of the first trainer. The first trainer and the third trainer are used to train the model together in the federated learning process. When all newly added numbers are different from the numbers of the first trainer, and the newly added numbers are larger than the numbers of the first trainer, the model training is terminated.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The trainer ID of the second participant in the counter component is queried. If there is no trainer ID of the second participant that is the same as the trainer ID of the first trainer, and there is a trainer ID that is larger than the trainer ID of the first trainer, the model training is terminated.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: After the second trainer corresponding to the target number is used as the paired trainer of the first trainer, the operation event triggered by the second trainer in the counter component is queried through the event query interface. The event query interface is an interface provided by the counter component for querying trainer operation events. When a number deletion event or session loss event triggered by the second trainer is found, the model training is terminated.

6. The method according to any one of claims 1 to 5, characterized in that, After the first trainer starts, it obtains its own ID from the counter component, including: After the first trainer is started, it sends a registration request to the counter component, so that the counter component responds to the registration request and uses the registration number of the first trainer as the number of the first trainer. The number of the first trainer itself is obtained from the counter component.

7. The method according to any one of claims 1 to 6, characterized in that, The counter component stores a first number list including the first participant's trainer number and a second number list including the second participant's trainer number, the first participant's trainer number is determined by the counter component in response to the registration request of the first participant's trainer, and the second participant's trainer number is determined by the counter component in response to the registration request of the second participant's trainer.

8. The method of claim 7, wherein, The query of the second participant's trainer number in the counter component comprises: The second participant's trainer number is obtained by querying the second number list in the counter component through a number query interface, and the number query interface is an interface provided by the counter component for querying the trainer number.

9. The method according to any one of claims 1 to 8, characterized in that, The method further comprises: Determining a communication channel based on the number of the first trainer, so as to send a message to the second trainer through the communication channel.

10. An apparatus for trainer pairing in federated learning model training, comprising: The device applied to the first participant's first trainer comprises: An acquisition module, configured to acquire the number of the first trainer itself from a counter component after the first trainer is started; A pairing module, configured to query the second participant's trainer number in the counter component, and when the second participant's trainer number includes a target number same as the number of the first trainer, the second trainer corresponding to the target number is taken as the paired trainer of the first trainer; The first trainer and the second trainer are used to cooperatively perform model training in a federated learning process, and the trainers started by the first participant and the second participant are numbered according to the same rule.

11. A computer readable medium having stored thereon a computer program, characterized in that, The computer program is executed by the processing device to implement the steps of the method of any one of claims 1-9.

12. An electronic device, comprising: Comprise: A storage device having a computer program stored thereon; A processing device configured to execute the computer program in the storage device to implement the steps of the method of any one of claims 1-9.

13. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1-9. The computer program is executed by the processor to implement the steps of the method of any one of claims 1-9.

Images