A method, device and apparatus for selecting a trusted coordinator for federated learning

Abstract

The present invention provides a method, device and equipment for selecting a trusted coordinator for federated learning, belonging to the field of artificial intelligence technology. The method includes: when the federated learning process is started, an election is started for at least one federated node that meets the election conditions; it is determined whether there is a federated node that obtains the majority of votes, and the federated node that obtains the majority of votes wins the election and becomes the coordinator; the authority of the federated node is established through the coordinator and the generation of a new coordinator is prevented. When the federated learning task is started, at least one federated node that meets the election conditions is selected from multiple federated nodes, and then voting is started for these federated nodes to select a trusted third party as a new coordinator, and the new coordinator is used to process collaborative requests from participants.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device and equipment for selecting a trusted coordinator for federated learning. Background Art

[0002] Federated Learning is an emerging basic technology in the field of artificial intelligence. Its goal is to carry out efficient machine learning among multiple participants or multiple computing nodes while ensuring information security during big data exchange, protecting terminal data and personal data privacy, and ensuring legality and compliance.

[0003] Federated learning is a technical system that securely combines data from multiple parties for modeling. Based on the characteristics and sample spaces, federated learning is divided into horizontal federated learning, vertical federated learning, and federated transfer learning. Among them:

[0004] Horizontal federated learning is applied to scenarios where the data sets of each participant have the same feature space and different sample spaces. K participants with the same data structure collaboratively train a machine learning model with the help of a coordinator (aggregator node).

[0005] Vertical federated learning is applied to scenarios where participants have the same sample space and different feature spaces on the dataset. After the participants align their samples using encrypted entity alignment technology, they collaboratively train a machine learning model with the help of a third-party coordinator.

[0006] Transfer learning is used to build effective and accurate machine learning models in applications with only small amounts of data (fewer overlapping samples and features) and weak supervision (fewer labels).

[0007] The existence of the above-mentioned trusted third-party coordinator is a reasonable assumption. In fact, it is difficult to find a third party that can be trusted by all participants at the same time. Generally, this role is played by an authoritative agency, such as the government.

[0008] Secure multi-party computing, a distributed deep learning training mechanism, performs privacy-preserving calculations without a trusted third party, which to some extent solves the problem of privacy leakage of user original data in traditional centralized management and storage of data. However, the training parameters of each participant in the collaborative learning process need to be integrated by the aggregator (third party) and shared with other participants. Therefore, there is a risk of parameter leakage, which makes it vulnerable to attacks such as inference attacks.

[0009] A consortium chain is a blockchain jointly managed by multiple institutions. Each organization or institution manages one or more nodes, and its data is only allowed to be read, written and sent by different institutions in the system. With the access mechanism, transaction performance can be improved more easily and some problems caused by uneven participants can be avoided. Summary of the invention

[0010] The technical problem to be solved by the present invention is to provide a method, device and equipment for selecting a trusted coordinator for federated learning, so as to solve the potential risk of no trusted third party in the existing federated learning training mechanism. To solve the above technical problem, the embodiment of the present invention provides the following technical solutions:

[0011] An embodiment of the present invention provides a method for selecting a trusted coordinator for federated learning, the method comprising:

[0012] When the federated learning process starts, an election is started for at least one federated node that meets the election conditions;

[0013] Determine whether there is a federal node that has obtained the majority of votes. The federal node that has obtained the majority of votes wins the election and becomes the coordinator;

[0014] The authority of the federation node is established through the coordinator and the creation of new coordinators is prevented.

[0015] In a preferred embodiment, each federated node is in any of the participant, coordinator, candidate or follower states at different times, where:

[0016] Participants,both the initiator and the participants in the federated learning process are,participants;

[0017] Coordinator: A federated node in this state is a trusted third party that handles coordination requests from participants.

[0018] Candidate is the state of a federation node that is eligible to participate in the election of a new coordinator;

[0019] Followers do not send any requests and only respond to requests from coordinators or candidates.

[0020] In a preferred embodiment, when the federated learning process is initiated, before a request for data collaboration is made, the process further includes:

[0021] If the federated node as a follower does not receive any message for a period of time and determines that the election has timed out, the follower determines that there is no coordinator in the data federation and attempts to start the federated learning process.

[0022] In a preferred embodiment, starting an election for at least one federated node that meets the election conditions includes:

[0023] The participating federation nodes need to meet the election condition that the health HD is greater than the average value of all federation nodes in the data federation. The node health HD = f(ω1H+ω2S+ω3N+ω4L+ω5F). in:

[0024] H is the hardware configuration score of the federated node, which is related to one or more of the machine’s CPU, motherboard architecture, memory, and hard disk, H∈(0,100);

[0025] S is the software environment score of the federated node, which is related to one or more of the machine’s operating system type and version, platform software, and functional component versions, S∈(0,100);

[0026] N is the network performance score of the federated node, which is related to one or more of the network bandwidth, transmission delay and packet loss rate, N∈(0,100);

[0027] L is the current resource load score of the federated node, which is related to one or more of the CPU occupancy rate, memory occupancy rate, and network bandwidth occupancy rate, L∈(0,100);

[0028] F is the failure rate of the federated node in the most recent statistical period, which is related to the number of downtimes, F∈(0,100);

[0029] ω1, ω2, ω3, ω4, and ω5 are weight coefficients, which are determined based on experience and experiments.

[0030] In a preferred embodiment, determining whether there is a federal node that obtains the majority of votes, and the federal node that obtains the majority of votes wins the election and becomes the coordinator includes:

[0031] to start an election;

[0032] A federated node increases its current term number and transitions to candidate state;

[0033] The federated node sends a request-to-vote message RequestVote to other federated nodes in the data federation in parallel, wherein the RequestVote requests other federated nodes to vote for the candidate federated node;

[0034] The federated node remains in the candidate state until one of the three events occurs: Event 1, the follower itself wins the election and becomes the coordinator; Event 2, another federated node becomes the coordinator; Event 3, after a period of time, there is no winning federated node, ending the current election.

[0035] In a preferred embodiment, the method further comprises:

[0036] Each federal node that serves as a candidate will reset a random election timeout when starting an election, and wait for the voting results during the timeout. If the timeout expires, a new round of elections will begin.

[0037] In a preferred embodiment, the RequestVote requesting other federated nodes to vote for the candidate's federated node includes:

[0038] Each federated node follows the first voting criterion and only votes for nodes with a state no older than its own ledger entry; and,

[0039] Each federal node follows the second voting criteria and can only vote once in one term. Under the condition that the first voting criteria is met, the vote will be cast for the federal node corresponding to the RequestVote that is first received.

[0040] In a preferred embodiment, establishing the authority of the federation node and preventing the generation of a new coordinator includes: periodically sending heartbeat packets to all followers to maintain the authority of the federation node itself.

[0041] In a preferred embodiment, after the authority of the federation node is established and the generation of a new coordinator is prevented, the process further includes:

[0042] The coordinator coordinates federated learning tasks and synchronizes the federated ledger during the data collaborative modeling process.

[0043] In a preferred embodiment, the coordinator coordinates the federated learning tasks during the data collaborative modeling process, and synchronizing the federated ledger includes: the coordinator sends an append federated ledger entry message AppendEntries, which notifies other federated nodes to append federated ledger entries and coordinate the federated learning tasks.

[0044] In a preferred embodiment, coordinating federated learning tasks includes: horizontal federated learning, vertical federated learning;

[0045] The horizontal federated learning includes but is not limited to:

[0046] Process scheduling and state synchronization of each round of horizontal federated learning tasks;

[0047] Receive encrypted information from participants about each round of model training;

[0048] Securely aggregate the received encrypted information and perform corresponding aggregation operations according to the different types of encrypted information;

[0049] Send securely aggregated results back to participants;

[0050] Vertical federated learning includes but is not limited to:

[0051] Process scheduling and state synchronization of each round of vertical federated learning tasks;

[0052] Create key pairs for participants and send public keys to each participant;

[0053] Receive encrypted information from participants about each round of model training;

[0054] Decrypt and aggregate the received encrypted information;

[0055] Send results back to each participant;

[0056] Synchronization of the federated ledger includes but is not limited to:

[0057] Record the task status of each round of federated learning tasks;

[0058] Perform batch updates to the federated ledger;

[0059] Perform synchronization of the federated ledger.

[0060] In a preferred embodiment, in the federated ledger, each ledger entry stores a state machine instruction and a term number when the state machine instruction is received from the coordinator, and the term number is used to check whether there is any inconsistency; each ledger entry has an integer index value to indicate the position of the ledger entry in the federated ledger;

[0061] The coordinator sends an AppendEntries message to append federated ledger entries, and the AppendEntries message notifies other federated nodes to append federated ledger entries and coordinate federated learning tasks, including:

[0062] Each request received from a client contains a state machine instruction that is executed by the replicated state machine;

[0063] The coordinator appends the instruction as a new ledger entry to the federated ledger, and then sends the AppendEntries message to other federated nodes in parallel.

[0064] After the ledger entry has been securely replicated by other federated nodes that are followers, the coordinator will execute the ledger entry in the coordinator's replication state machine and return the execution result to the client.

[0065] When the coordinator replicates the created ledger entry to the majority of federated nodes, the ledger entry is committed; and before the coordinator commits the ledger entry, the entries created by other coordinators have also been committed;

[0066] Among them, the coordinator tracks the index of the largest ledger entry to be submitted, and the index value will be included in all future additional federated ledger entry messages, so that other federated nodes can know the submission position of the coordinator; if the follower knows that a ledger entry has been submitted, the follower will also apply this ledger entry to the local state machine in the order of the ledger entries.

[0067] In a preferred embodiment, after the ledger entry is securely replicated by other federated nodes that are followers, the coordinator applies the ledger entry to the coordinator's state machine, including:

[0068] The coordinator decides when it is safe to apply ledger entries to the state machines; ledger entries that have been applied to the state machines are called committed, and all committed ledger entries are persistent and will eventually be executed by all available state machines.

[0069] An embodiment of the present invention provides a device for selecting a trusted coordinator for federated learning, the device comprising:

[0070] The election module is started, which is used to start the election for at least one federation node that meets the election conditions when the federation learning process is started;

[0071] An election module is used to determine whether there is a federal node that obtains the majority of votes. The federal node that obtains the majority of votes wins the election and becomes the coordinator;

[0072] The coordinator generation module is used for the coordinator to establish the authority of the federation node and prevent the generation of new coordinators.

[0073] In a preferred embodiment, it also includes:

[0074] The node status module is used to record the status of each federated node at different times. Each federated node is in any of the participant, coordinator, candidate or follower states at different times, where:

[0075] Participant, is the state of the federation node before the election coordinator;

[0076] Coordinator: A federated node in this state is a trusted third party that handles coordination requests from participants.

[0077] Candidate is the state of a federation node that is eligible to participate in the election of a new coordinator;

[0078] Followers do not send any requests and only respond to requests from coordinators or candidates.

[0079] In a preferred embodiment, it also includes:

[0080] The startup judgment module is used to start the federated learning process. Before there is a request for data collaboration, if the federated node as a follower does not receive any message for a period of time and determines that the election has timed out, the follower determines that there is no coordinator in the data federation and attempts to start the federated learning process.

[0081] In a preferred embodiment, the election module includes:

[0082] The election condition determination unit is used to determine that the federated nodes participating in the election need to meet the election condition that the health HD is greater than the average value of all federated nodes in the data federation. The node health HD = f(ω1H+ω2S+ω3N+ω4L+ω5F). in:

[0083] H is the hardware configuration score of the federated node, which is related to the CPU, motherboard architecture, memory, and hard disk of the machine, H∈(0,100);

[0084] S is the software environment score of the federated node, which is related to the machine's operating system type and version, platform software, and functional component versions, S∈(0,100);

[0085] N is the network performance score of the federated node, which is related to network bandwidth, transmission delay and packet loss rate, N∈(0,100);

[0086] L is the current resource load score of the federated node, which is related to CPU occupancy, memory occupancy, and network bandwidth occupancy, L∈(0,100);

[0087] F is the failure rate of the federated node in the most recent statistical period, which is related to the number of downtimes, F∈(0,100);

[0088] ω1, ω2, ω3, ω4, and ω5 are weight coefficients, which are determined based on experience and experiments.

[0089] In a preferred embodiment, the election module includes:

[0090] The election execution unit is used to start an election;

[0091] A federated node increases its current term number and transitions to candidate state;

[0092] The federated node sends a request-to-vote message RequestVote to other federated nodes in the data federation in parallel, wherein the request-to-vote message RequestVote requests other federated nodes to vote for the candidate federated node;

[0093] The federated node remains in the candidate state until one of the three events occurs: Event 1, the follower itself wins the election and becomes the coordinator; Event 2, another federated node becomes the coordinator; Event 3, after a period of time, there is no winning federated node, ending the current election.

[0094] In a preferred embodiment, it also includes:

[0095] The synchronization module is used by the coordinator to coordinate federated learning tasks and synchronize the federated ledger during the data collaborative modeling process.

[0096] In a preferred embodiment, the coordinator generation module includes:

[0097] The heartbeat unit is used to periodically send heartbeat packets to all followers to maintain the authority of the federation node itself;

[0098] The synchronization module includes:

[0099] The message unit is used by the coordinator to send the federated ledger entry append message AppendEntries, which notifies other federated nodes to append federated ledger entries and coordinate federated learning tasks.

[0100] In a preferred embodiment, the message unit includes:

[0101] Coordinate federated learning task units to achieve horizontal federated learning and vertical federated learning;

[0102] The horizontal federated learning includes but is not limited to:

[0103] Process scheduling and state synchronization of each round of horizontal federated learning tasks;

[0104] Receive encrypted information from participants about each round of model training;

[0105] Securely aggregate the received encrypted information and perform corresponding aggregation operations according to the different types of encrypted information;

[0106] Send securely aggregated results back to participants;

[0107] Vertical federated learning includes but is not limited to:

[0108] Process scheduling and state synchronization of each round of vertical federated learning tasks;

[0109] Create key pairs for participants and send public keys to each participant;

[0110] Receive encrypted information from participants about each round of model training;

[0111] Decrypt and aggregate the received encrypted information;

[0112] Send results back to each participant;

[0113] Synchronization of the federated ledger includes but is not limited to:

[0114] Record the task status of each round of federated learning tasks;

[0115] Perform batch updates to the federated ledger;

[0116] Perform synchronization of the federated ledger.

[0117] In a preferred embodiment, the election execution unit includes:

[0118] The election timeout unit is used to set each federation node that is a candidate to reset a random election timeout when starting an election, and wait for the voting results within the timeout period.

[0119] In a preferred embodiment, the election execution unit includes:

[0120] A voting criteria unit, used to implement each federation node to follow the first voting criteria and only vote for nodes with a state no older than its own ledger entry; and,

[0121] Each federal node follows the second voting criterion and can only vote once in one term. Under the condition that the first voting criterion is met, the vote is cast for the federal node corresponding to the request vote message RequestVote that is first received.

[0122] In a preferred embodiment, the message unit includes:

[0123] A federated ledger update unit, where each request received from a client contains an instruction to be executed by the replicated state machine;

[0124] The coordinator appends the instruction as a new ledger entry to the federated ledger, and then sends the AppendEntries message to other federated nodes in parallel.

[0125] After the ledger entry has been securely replicated by other federated nodes that are followers, the coordinator applies the ledger entry to the coordinator's replication state machine and returns the execution result to the client.

[0126] When the coordinator replicates the created ledger entry to the majority of federated nodes, the ledger entry is committed; and before the coordinator commits the ledger entry, the entries created by other coordinators have also been committed;

[0127] The coordinator tracks the index of the largest ledger entry to be submitted, and the index value will be included in all future additional federated ledger entry messages, so that other federated nodes can know the submission position of the coordinator; if the follower knows that a ledger entry has been submitted, the follower will also apply this ledger entry to the local state machine in the order of the ledger entries;

[0128] In the federated ledger, each ledger entry stores a state machine instruction and the term number when the state machine instruction is received from the coordinator. The term number is used to check whether there are inconsistencies; each ledger entry has an integer index value to indicate the position of the ledger entry in the federated ledger.

[0129] In a preferred embodiment, the federated ledger updating unit includes:

[0130] The commit ledger entry unit is used by the coordinator to decide when it is safe to apply the ledger entry to the state machine; the ledger entry applied to the state machine is called committed, and all committed ledger entries are persistent and will eventually be executed by all available state machines.

[0131] An embodiment of the present invention also provides a device for selecting a trusted coordinator for federated learning, characterized in that it includes: a processor, a memory, and a program stored on the memory and executable on the processor, wherein when the program is executed by the processor, the steps in the method for selecting a trusted third party as a coordinator for federated learning are implemented.

[0132] An embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the steps in the method of selecting a trusted third party as a coordinator for federated learning are implemented.

[0133] The embodiments of the present invention have the following beneficial effects: when starting a federated learning task, at least one federated node that meets the election conditions is selected from multiple federated nodes, and then voting begins with these federated nodes to elect a trusted third party as a new coordinator, and the new coordinator is used to process collaboration requests from participants. BRIEF DESCRIPTION OF THE DRAWINGS

[0134] Figure 1 A flowchart of a method for selecting a trusted coordinator for federated learning according to an embodiment of the present invention;

[0135] Figure 2 A schematic diagram of a system architecture for defining a federated learning technology using a hierarchical structure according to an embodiment of the present invention;

[0136] Figure 3A schematic diagram of the deployment architecture of the federated learning technology based on the alliance chain according to an embodiment of the present invention;

[0137] Figure 4 Schematic diagram of state transition during the trusted third party (coordinator) election process in an embodiment of the present invention

[0138] Figure 5 A schematic diagram of how to determine the term of a coordinator according to an embodiment of the present invention;

[0139] Figure 6 A schematic diagram of a process of electing a trusted third party according to an embodiment of the present invention;

[0140] Figure 7 This is a schematic diagram of the account book structure of an embodiment of the present invention;

[0141] Figure 8 A first diagram is a schematic diagram of a device structure for selecting a trusted coordinator for federated learning according to an embodiment of the present invention;

[0142] Fig. 9 FIG2 is a second diagram showing the structure of a device for selecting a trusted coordinator for federated learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0143] In order to make the technical problems, technical solutions and advantages to be solved by the embodiments of the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0144] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0145] The term "comprise" and any variation thereof in the specification and claims of the present application are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. In addition, the use of "and / or" in the specification and claims to indicate at least one of the connected objects, such as A and / or B, means that A alone, B alone, and A and B exist in three cases.

[0146] In the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0147] For the mechanism and process of federated learning, the alliance chain with a certain access mechanism, through the member identity identification mechanism based on MSP, guarantees the credibility of the identities and roles of participants from different organizations in the blockchain network to a certain extent. And isolate participants from different organizations by defining channels. On this basis, combined with the inherent decentralized characteristics, consensus mechanism, and technical advantages of non-tampering of blockchain, a technical solution for selecting a trusted third party can be provided for federated learning to solve the potential risk of no trusted third party in the existing federated learning training mechanism.

[0148] In this application, based on the original federated learning mechanism, the alliance chain is used as the carrier of data federation, and a trusted third party (coordinator) election function is added. When starting a federated learning task, the trusted third party selection function is used to elect a trusted third party from the alliance members as a coordinator.

[0149] The embodiment of the present invention provides a method for selecting a trusted coordinator for federated learning, such as Figure 1 As shown, the method includes:

[0150] Step 101: When the federated learning process is initiated, an election is started for at least one federated node that meets the election conditions;

[0151] Step 102: Determine whether there is a federal node that obtains the majority of votes. The federal node that obtains the majority of votes wins the election and becomes the coordinator.

[0152] Step 103: Establish the authority of the federation node through the coordinator and prevent the generation of a new coordinator.

[0153] The technical solution provided in the application embodiment is used to select at least one federal node that meets the election conditions from multiple federal nodes when starting a federated learning task, and then start voting with these federal nodes to elect a trusted third party as a new coordinator. The new coordinator is used to process collaboration requests from participants.

[0154] To help those skilled in the art understand the technical solutions provided in this application, Figure 2 As shown in the figure, based on the alliance chain as the carrier of data federation, the system architecture of federated learning technology is defined using a hierarchical structure:

[0155] Resource layer, used to carry data and device resources.

[0156] The entity layer is used to carry each organization (also called organization member) in the data alliance. Each organization member in the data alliance can initiate a federated learning task. Organization members participating in the federated learning task are participants, and those who do not participate in the federated learning task are third parties. The data alliance includes multiple organizations, and each organization forms an alliance chain.

[0157] The platform layer defines the core platform components of the system architecture, including the alliance chain platform and the federated learning platform. By adding a trusted third-party selection component to the alliance chain platform, a trusted third-party selection function is provided for federated learning.

[0158] The application layer defines applications for users and provides views corresponding to various functions according to different user roles.

[0159] It can be seen that Figure 3 As shown in the figure, an actual deployment architecture of a federated learning system based on a consortium chain is described. In the deployment architecture of the federated learning technology based on the consortium chain, there are N organizations in the data alliance, and N organizations form a consortium chain. According to the specific data collaboration situation, a series of channels are defined, and each channel corresponds to a data federation. Each organization can join multiple channels, that is, join multiple data federations.

[0160] As shown in the figure, organizations 1, 2, and n all belong to data federation 1, and organizations 3 and n belong to data federation N. Nodes in the same data federation can participate in the common federated learning task.

[0161] In the consortium chain, there is a complete technical solution to elect which federation node is suitable to be a trusted third party. However, before the election, it is necessary to standardize and design the status of the federation node, the entry conditions for being elected, the functions and responsibilities of the coordinator, and the data structure.

[0162] In a preferred embodiment, each federated node is in any of the participant, coordinator, candidate or follower states at different times, where:

[0163] Participant, is the state of the federation node before the election coordinator;

[0164] Coordinator: A federated node in this state is a trusted third party that handles coordination requests from participants.

[0165] Candidate is the state of a federation node that is eligible to participate in the election of a new coordinator;

[0166] Followers do not send any requests, but only respond to requests from coordinators or candidates.

[0167] In an application scenario, a data federation (channel) contains several federation nodes, usually more than 3. Usually, there are at least 2 participants in the system, only 1 coordinator, and the other nodes are followers. The functions of each federation node are different:

[0168] Coordinator, a federated node in this state, handles coordination requests from participants;

[0169] Followers are passive. They do not send any requests but simply respond to requests from coordinators or candidates.

[0170] Candidates are used to describe federation nodes that are eligible to participate in the election process for a new coordinator.

[0171] In order to improve the efficiency of federated learning, such as improving communication efficiency and reducing the training time of each round of federated models, those federated nodes suitable for running for coordinators generally need to meet the corresponding access conditions, that is, only when these access conditions are met can they become candidates.

[0172] When a federated learning process starts, all federated nodes are followers. A federated node continues to remain in the follower state until it receives a vote request from the coordinator or candidate. The coordinator periodically sends heartbeat packets to all followers to maintain its authority.

[0173] In the case of two or more third parties (coordinators), the basic process of election, in a preferred embodiment, when the federated learning process is started and before there is a request for data collaboration, also includes: if the federated node as a follower does not receive any message for a period of time and determines that the election has timed out, the follower determines that there is no coordinator in the data federation and attempts to start the federated learning process. That is, when the election times out, the follower will believe that there is no available coordinator in the system and initiate an election to select a new coordinator.

[0174] In a preferred embodiment, the federated node has a node health HD; the entry condition that the federated node participating in the election needs to meet is: the health HD value is greater than the average value of all federated nodes in the data federation.

[0175] In the corresponding application scenario, in order to describe the admission conditions that the federated nodes that are running for the coordinator must meet, the node health HD = f(ω1H+ω2S+ω3N+ω4L+ω5F) is introduced.

[0176] in:

[0177] H (Hardware): The hardware configuration score of the federated node, which is related to one or more of the machine's CPU, motherboard architecture, memory, and hard disk. H∈(0,100).

[0178] S (Software): The software environment score of the federated node, which is related to one or more of the machine's operating system type and version, platform software, and functional component versions, S∈(0,100).

[0179] N (Network performance): The network performance score of the federated node, which is related to one or more of network bandwidth, transmission delay, and packet loss rate, N∈(0,100).

[0180] L(Load): The current resource load score of the federated node, which is related to one or more of the CPU occupancy, memory occupancy, and network bandwidth occupancy, L∈(0,100).

[0181] F (Fault rate): the failure rate of the federated node in the most recent statistical period, which is related to the number of downtimes, etc., F∈(0,100);

[0182] ω1, ω2, ω3, ω4, and ω5 are weight coefficients, which are determined based on experience and experiments.

[0183] Generally speaking, the health value of the federated nodes participating in the election of the coordinator needs to be greater than the average value of all federated nodes in the data federation. This health value can be adjusted according to the actual situation.

[0184] like Figure 5 As shown in Figure 1, after completing the design of the federated node's own status, access conditions, related functional responsibilities, and data structure, the trusted third party (coordinator) election is carried out. In the alliance chain, when a federated learning process is started, the trusted third party (coordinator) must be elected first, and the coordinator's term is initialized to the federated learning task cycle. The task cycle is marked with consecutive integers, and each term starts with an election, such as Figure 5 As shown in Figure 1, during the “election” phase at the beginning of term 1, one or more candidates attempt to become a coordinator. If a candidate wins the election and becomes a coordinator, it will perform the coordinator duties among the participants throughout the subsequent term. Figure 5 As shown in Figure 1, the "coordination" phase in term 1 lasts until the term expires or the server crashes. Sometimes elections fail, such as Figure 5 If the term is 2, then this term ends and a new round of elections begins.

[0185] If a candidate wins the election and becomes the coordinator, it will perform the coordinator duties among the participants throughout the subsequent term until the term expires or it crashes.

[0186] In a preferred embodiment, the election of a coordinator is implemented, and it is determined whether there is a federal node that obtains the majority of votes. The federal node that obtains the majority of votes wins the election and becomes the coordinator, which includes:

[0187] to start an election;

[0188] A federated node increases its current term number and transitions to candidate state;

[0189] The federated node sends a request-to-vote message RequestVote to other federated nodes in the data federation in parallel, wherein the request-to-vote message RequestVote requests other federated nodes to vote for the candidate federated node;

[0190] The federated node remains in the candidate state until one of the three events occurs: Event 1, the follower itself wins the election and becomes the coordinator; Event 2, another federated node becomes the coordinator; Event 3, after a period of time, there is no winning federated node, ending the current election.

[0191] In the corresponding application scenarios, such as Figure 4 As shown in the figure, to start an election process, the follower must first increase its current term number and switch to the candidate state. If a follower wants to switch to the candidate state, it must meet the entry conditions, such as the health HD value is greater than the average value of all federated nodes in the data federation. The candidate will send a message requesting a vote to other federated nodes in the data federation in parallel to vote for itself. When voting, other federated nodes should follow the corresponding voting rules. The candidate will continue to maintain the current state until one of the following three events occurs: Event 1, the follower itself wins the election and becomes the coordinator; Event 2, other federated nodes become coordinators; Event 3, no one wins after a period of time. End the current election.

[0192] Event 1: The follower wins the election and becomes the coordinator: When a candidate obtains votes for the same term number from the majority of federated nodes in the entire data federation, it wins the election and becomes the coordinator. According to the principle of first come first served, each node will cast at most one vote for one term number. The rule requiring a majority of votes ensures that at most only one candidate will win the election. Once a candidate wins the election, the candidate immediately becomes the coordinator. It then sends heartbeat messages to other nodes to establish its own authority and prevent the generation of new coordinators. Among them, the specific value of the majority is determined by the consensus algorithm of the blockchain through consensus. Consensus is a basic working mechanism of the blockchain. Different consensus algorithms can be selected as needed, such as BFT, PBFT, RAFT, LibraBFT, etc.

[0193] Event 2, other federated nodes become coordinators: While waiting for voting, the candidate may receive a message from another node stating that it is a coordinator. If the term number of the coordinator is not less than the candidate's current term number, the candidate will recognize the legitimacy of the coordinator and return to the follower state. If the term number in this message is smaller than its own, the candidate will reject this message and continue to maintain the candidate state, that is, the coordinator finds a node with an updated term and converts back to a follower.

[0194] Event 3, the candidate neither wins nor loses the election: If multiple followers become candidates at the same time, the votes may be evenly divided so that no candidate can win the majority. When this happens, each candidate will time out and a new election will start by increasing the current term number.

[0195] Sometimes elections fail, then the term ends and a new round of elections begins. In order to avoid split votes and resolve them quickly even if they occur, a variety of solutions can be used, such as setting a random election timeout, using node reputation rankings, etc. As a follower or candidate, if an election timeout occurs, a new election begins.

[0196] To prevent the votes from being evenly split initially, if a random election timeout is set, the election timeout is randomly chosen from a fixed interval. This spreads out the federation nodes so that in most cases only one node will time out; it then wins the election and sends heartbeats before the other nodes time out.

[0197] The same mechanism is also used in the case where the votes have been split. Each federation node that is a candidate will reset a random election timeout when starting an election and wait for the results of the vote within the timeout. This reduces the possibility of split votes in a new election.

[0198] In the alliance chain, the technical means in each embodiment are applied to various scenarios in the process of electing a trusted third party:

[0199] Scenario 1: There is no third party except the participants: If all nodes of a data federation (channel) come from participants, the coordinator can only be selected from the participants. At the beginning, all participants are followers and execute the process in "determining whether there is a federation node that obtains the majority of votes, and the federation node that obtains the majority of votes wins the election and becomes the coordinator".

[0200] Scenario 2: There is only one third party: If a data federation (channel) has only one third party, the coordinator selects this third party.

[0201] Scenario 3: If the selected trusted third party (coordinator) goes down: Under normal circumstances, the coordinator will continuously broadcast heartbeat information, and the follower will reset the timeout after receiving the coordinator's heartbeat information. When the coordinator crashes or goes offline abnormally, the follower nodes in the network cannot receive the heartbeat information, and the election process will be entered again after the timeout to elect a coordinator.

[0202] If the current coordinator node goes down, a new coordinator needs to be re-elected. At this time, the status of the follower nodes may be different. Some followers may have the same status as the coordinator that just went down, and the status is newer. Some followers may record a much smaller current index than the original coordinator node, and the status update is relatively lagging. At this time, from the perspective of system optimization, it is better to select the candidate with the latest status. From the perspective of correctness, the integrity of the coordinator must be ensured, that is, if a ledger entry is successfully submitted in a certain term, then this ledger entry must exist in the coordinator with a higher term. Conversely, if a candidate's status is older than the currently submitted status, it must not be selected as the coordinator.

[0203] When other federated nodes vote, they should follow the corresponding voting rules. The specific voting rules for the federated nodes should be specified. In one embodiment: 1) the federated node only votes for nodes with a ledger entry status no older than its own; 2) each federated node can only vote once during a term, and first come first served if the conditions in 1) are met.

[0204] In an application scenario, there are two or more third parties. Figure 6 As shown, the process includes:

[0205] Step 601: The data collaboration initiator requests data collaboration;

[0206] Step 602, the federated node updates the node health and node status;

[0207] Step 603: The federation nodes that meet the election conditions participate in the election and request a vote;

[0208] Step 604: The node that receives the voting request participates in the voting;

[0209] Step 605, determine whether a majority of votes are obtained, if so, go to step 606, if not, go to step 609; whether there is a tie in the votes, if so, go to step 607;

[0210] Step 606, the node that obtains the majority of votes claims to have won the election;

[0211] Step 607, elect arbitration when the votes are evenly divided; go to step 608;

[0212] Step 608: The federated nodes reach a consensus on the voting results and the process ends.

[0213] Step 609: No one gets a majority of votes, the election times out, and ends.

[0214] In the election process, in a preferred embodiment, the request vote message RequestVote requests other federation nodes to vote for the candidate's federation node to include:

[0215] Each federated node follows the first voting criterion and only votes for nodes with a state no older than its own ledger entry; and,

[0216] Each federated node follows the second voting criteria and can only vote once in a term. Under the condition that the first voting criteria are met, the vote is cast for the federated node corresponding to the first received request vote message (RequestVote). The basic coordinator election algorithm requires two types of messages: request vote messages, which are initiated by the participating federated nodes as candidates during the election period; and append federated ledger entry messages (AppendEntries) initiated by the coordinator to append federated ledger entries and provide a heartbeat mechanism.

[0217] After the federation node is generated through election, the new coordinator needs to complete some corresponding functions. In a preferred embodiment, establishing the authority of the federation node and preventing the generation of new coordinators includes: periodically sending heartbeat packets to all followers to maintain the authority of the federation node itself.

[0218] In the consortium chain, the functions and responsibilities of the trusted third party (coordinator) need to be clarified, that is, they need to be given corresponding specific functions and responsibilities. Once a coordinator is elected, it starts to provide services to the client.

[0219] Regarding the services provided by the coordinator, in a preferred embodiment, after establishing the authority of the federated node and preventing the generation of new coordinators, it also includes: the coordinator coordinates the federated learning tasks and synchronizes the federated ledger during the data collaborative modeling process.

[0220] In a preferred embodiment, the coordinator coordinates the federated learning tasks during the data collaborative modeling process, and synchronizing the federated ledger includes: the coordinator sends an append federated ledger entry message AppendEntries, which notifies other federated nodes to append federated ledger entries and coordinate the federated learning tasks.

[0221] In a preferred embodiment, coordinating federated learning tasks includes: horizontal federated learning, vertical federated learning;

[0222] The horizontal federated learning includes but is not limited to:

[0223] Process scheduling and state synchronization of each round of horizontal federated learning tasks;

[0224] Receive encrypted information from participants about each round of model training;

[0225] Securely aggregate the received encrypted information and perform corresponding aggregation operations according to the different types of encrypted information;

[0226] Send securely aggregated results back to participants;

[0227] Vertical federated learning includes but is not limited to:

[0228] Process scheduling and state synchronization of each round of vertical federated learning tasks;

[0229] Create key pairs for participants and send public keys to each participant;

[0230] Receive encrypted information from participants about each round of model training;

[0231] Decrypt and aggregate the received encrypted information;

[0232] Send results back to each participant;

[0233] The coordination of federated learning tasks varies according to different task types. In the corresponding application scenarios, for horizontal federated learning, the responsibilities of the trusted third party include but are not limited to:

[0234] 1. Process scheduling and state synchronization of each round of horizontal federated learning tasks;

[0235] 2. Receive encrypted information of each round of model training from participants, such as gradient information, model parameters, etc.;

[0236] 3. Safely aggregate the received information and perform corresponding aggregation operations according to the different types of information, such as gradient averaging, model averaging, etc.;

[0237] 4. Send the aggregated information back to the participants.

[0238] For vertical federated learning, the responsibilities of a trusted third party include but are not limited to:

[0239] 1. Process scheduling and state synchronization of each round of horizontal federated learning tasks;

[0240] 2. Create key pairs for participants and send public keys to each participant;

[0241] 3. Receive encrypted information of each round of model training from participants, such as gradient information, loss information, model parameters, etc.;

[0242] 4. Decrypt and aggregate the received information;

[0243] 5. Send the results back to each participant.

[0244] In a preferred embodiment, synchronization of the federated ledger includes but is not limited to:

[0245] Record the task status of each round of federated learning tasks;

[0246] Perform batch updates of the federated ledger; perform synchronization of the federated ledger.

[0247] In the alliance chain, in order to elect a trusted third party (coordinator) and to fulfill the assigned functional responsibilities after the trusted third party (coordinator) is elected, it is necessary to design the functions and data organization of the trusted third party (coordinator). In this application, the basic functions of the coordinator include but are not limited to the following data structures and methods:

[0248] 1. Data structure: federated node health

[0249] 2. Data structure: federated node status

[0250] 3. Method: Update health()

[0251] 4. Method: RequestVote()

[0252] 5. Method: Update the federated ledger (), which corresponds to the copy ledger instruction.

[0253] in

[0254] 1. Federation node health includes:

[0255] Hardware configuration score;

[0256] Software environment score;

[0257] Network performance score;

[0258] Resource load score;

[0259] Failure rate.

[0260] 2. In a preferred embodiment, the functions of the trusted third party (coordinator) include:

[0261] Federation node status FLStatus:

[0262] Node ID: the ID of the current node;

[0263] Node label: the type label of the current node, such as initiator, participant, coordinator, follower, etc.

[0264] Current term currentTerm: the last known term number of the node, initially 0, and continuously increasing;

[0265] Coordinator ID: The candidate ID that currently receives votes;

[0266] LedgerEntries: array of ledger entries;

[0267] Ledger entry lastIndex: The largest known submitted ledger entry index value.

[0268] 3. Update health: Update health()

[0269] Update the health of all nodes in the data federation.

[0270] 4. Campaign Request:

[0271] RequestVote(termNumber, candidateID, latest ledger entry, latest termNumber)

[0272] Return value:

[0273] Current term number: Current term number, used by candidates to update their own term number;

[0274] Elected: true / false, true if the candidate wins the current vote.

[0275] 5. Federated ledger replication instructions: called by the coordinator to replicate ledger entries and also to provide a heartbeat mechanism.

[0276] Update the federated ledger (term number, coordinator ID, latest ledger index, latest index term, ledger data);

[0277] In the parameters:

[0278] Term number: The term number of the coordinator

[0279] Coordinator ID: ID of the coordinator

[0280] Previous ledger index: the index of the previous ledger entry

[0281] Latest index term: the term corresponding to the previous ledger entry

[0282] Ledger data: entries to be written into the ledger. It is empty for heartbeat.

[0283] In specific technical details, in a preferred embodiment, in the federated ledger, each ledger entry stores a state machine instruction and a term number when the state machine instruction is received from the coordinator, and the term number is used to check whether there is any inconsistency; each ledger entry has an integer index value to indicate the position of the ledger entry in the federated ledger;

[0284] The coordinator sends an AppendEntries message to append federated ledger entries, and the AppendEntries message notifies other federated nodes to append federated ledger entries and coordinate federated learning tasks, including:

[0285] Each request received from a client contains an instruction to be executed by the replicated state machine;

[0286] The coordinator appends the instruction as a new ledger entry to the federated ledger, and then sends the AppendEntries message to other federated nodes in parallel.

[0287] After the ledger entry has been securely replicated by other federated nodes that are followers, the coordinator applies the ledger entry to the coordinator's replication state machine and returns the execution result to the client.

[0288] When the coordinator replicates the created ledger entry to the majority of federated nodes, the ledger entry is committed; and before the coordinator commits the ledger entry, the entries created by other coordinators have also been committed;

[0289] Among them, the coordinator tracks the index of the largest ledger entry to be submitted, and the index value will be included in all future additional federated ledger entry messages, so that other federated nodes can know the submission position of the coordinator; if the follower knows that a ledger entry has been submitted, the follower will also apply this ledger entry to the local state machine in the order of the ledger entries.

[0290] In the corresponding application scenario, each request from the client contains a state machine instruction executed by the replicated state machine. The coordinator appends this instruction as a new ledger entry to the federated ledger, and then sends an "append federated ledger entry" message to other federated nodes in parallel, asking them to replicate this ledger entry.

[0291] When the ledger entry has been securely replicated by other federated nodes as followers, the coordinator applies the ledger entry to its state machine and returns the result of the execution to the client.

[0292] If a follower crashes or loses packets, the coordinator will continue to retry the "append federated ledger entry" message until all followers eventually store all ledger entries.

[0293] In order to realize the services provided by the coordinator to the client, such as Figure 7 As shown in the figure, the federated ledger is organized. There are multiple blocks in the federated ledger, which are logically connected together. A block contains a block header, a data block, and metadata. The data block contains multiple ledger entries. Each ledger entry stores a state machine instruction and the term number when the instruction is received from the coordinator. The term number in the ledger entry is used to check for inconsistencies. Each ledger entry also has an integer index value to indicate its position in the federated ledger.

[0294] In a preferred embodiment, when the ledger entry is securely replicated by other federated nodes that are followers, the coordinator applies the ledger entry to the coordinator's state machine including:

[0295] The coordinator decides when it is safe to apply ledger entries to the state machines; ledger entries that have been applied to the state machines are called committed, and all committed ledger entries are persistent and will eventually be executed by all available state machines.

[0296] In the corresponding application scenario, under the premise of organizing the federated ledger, the coordinator decides when it is safe to apply the ledger entry to the state machine; such ledger entries are called committed, and the algorithm ensures that all committed ledger entries are persistent and will eventually be executed by all available state machines. The ledger entry is committed when the coordinator copies the created ledger entry to the majority of the federated nodes. In addition, all previous ledger entries in the coordinator's ledger entry will also be committed, that is, entries created by other coordinators will also be committed.

[0297] The coordinator keeps track of the index of the largest ledger entry that will be committed, and the index value will be included in all future additional federated ledger entry messages (including heartbeat packets), so that other federated nodes can eventually know the coordinator's commit position. Once a follower knows that a ledger entry has been committed, it will also apply the ledger entry to the local state machine, which will be in the order of the ledger entries.

[0298] The coordinator never overwrites or deletes its own ledger entries.

[0299] The embodiment of the present invention also provides a device for selecting a trusted coordinator for federated learning, such as Figure 8 As shown, the device comprises:

[0300] The starting election module 801 is used to start the election for at least one federation node that meets the election conditions when the federation learning process is started;

[0301] An election module 802 is used to determine whether there is a federal node that obtains the majority of votes. The federal node that obtains the majority of votes wins the election and becomes the coordinator;

[0302] The coordinator generation module 803 is used for the coordinator to establish the authority of the federation node and prevent the generation of a new coordinator.

[0303] The technical solution provided in the application embodiment is used to select at least one federal node that meets the election conditions from multiple federal nodes when starting a federated learning task, and then start voting with these federal nodes to elect a trusted third party as a coordinator. The new coordinator is used to process collaboration requests from participants.

[0304] In a preferred embodiment, if Fig. 9 As shown, the device also includes:

[0305] The node status module 804 is used to record the status of each federated node at different times. Each federated node is in any of the participant, coordinator, candidate or follower states at different times, where:

[0306] Participant, is the state of the federation node before the election coordinator;

[0307] Coordinator: A federated node in this state is a trusted third party that handles coordination requests from participants.

[0308] Candidate is the state of a federation node that is eligible to participate in the election of a new coordinator;

[0309] Followers do not send any requests, but only respond to requests from coordinators or candidates.

[0310] In a preferred embodiment, if Fig. 9 As shown, the device also includes:

[0311] The startup determination module 805 is used to, when the federated learning process is started and before there is a request for data collaboration, if the federated node as a follower does not receive any message for a period of time and determines that the election has timed out, the follower determines that there is no coordinator in the data federation and attempts to start the federated learning process.

[0312] In a preferred embodiment, the election module 802 includes:

[0313] The election condition determination unit is used to determine that the federated nodes participating in the election need to meet the election condition that the health HD is greater than the average value of all federated nodes in the data federation. The node health HD = f(ω1H+ω2S+ω3N+ω4L+ω5F). in:

[0314] H is the hardware configuration score of the federated node, which is related to the CPU, motherboard architecture, memory, and hard disk of the machine, H∈(0,100);

[0315] S is the software environment score of the federated node, which is related to the machine's operating system type and version, platform software, and functional component versions, S∈(0,100);

[0316] N is the network performance score of the federated node, which is related to network bandwidth, transmission delay and packet loss rate, N∈(0,100);

[0317] L is the current resource load score of the federated node, which is related to CPU occupancy, memory occupancy, and network bandwidth occupancy, L∈(0,100);

[0318] F is the failure rate of the federated node in the most recent statistical period, which is related to the number of downtimes, F∈(0,100);

[0319] ω1, ω2, ω3, ω4, and ω5 are weight coefficients, which are determined based on experience and experiments.

[0320] In a preferred embodiment, the election module 802 includes:

[0321] The election execution unit is used to start an election;

[0322] A federated node increases its current term number and transitions to candidate state;

[0323] The federated node sends a request-to-vote message RequestVote to other federated nodes in the data federation in parallel, wherein the request-to-vote message RequestVote requests other federated nodes to vote for the candidate federated node;

[0324] The federated node remains in the candidate state until one of the three events occurs: Event 1, the follower itself wins the election and becomes the coordinator; Event 2, another federated node becomes the coordinator; Event 3, after a period of time, there is no winning federated node, ending the current election.

[0325] In a preferred embodiment, if Fig. 9 As shown, the device also includes:

[0326] The synchronization module 806 is used by the coordinator to coordinate the federated learning tasks and synchronize the federated ledger during the data collaborative modeling process.

[0327] In a preferred embodiment, if Fig. 9 As shown, the coordinator generation module 803 includes:

[0328] The heartbeat unit is used to periodically send heartbeat packets to all followers to maintain the authority of the federation node itself;

[0329] The synchronization module 806 includes:

[0330] The message unit is used by the coordinator to send the federated ledger entry append message AppendEntries, which notifies other federated nodes to append federated ledger entries and coordinate federated learning tasks.

[0331] In a preferred embodiment, the message unit includes:

[0332] Coordinate federated learning task units to achieve horizontal federated learning and vertical federated learning;

[0333] The horizontal federated learning includes but is not limited to:

[0334] Process scheduling and state synchronization of each round of horizontal federated learning tasks;

[0335] Receive encrypted information from participants about each round of model training;

[0336] Securely aggregate the received encrypted information and perform corresponding aggregation operations according to the different types of encrypted information;

[0337] Send securely aggregated results back to participants;

[0338] Vertical federated learning includes but is not limited to:

[0339] Process scheduling and state synchronization of each round of vertical federated learning tasks;

[0340] Create key pairs for participants and send public keys to each participant;

[0341] Receive encrypted information from participants about each round of model training;

[0342] Decrypt and aggregate the received encrypted information;

[0343] Send results back to each participant;

[0344] Synchronization of the federated ledger includes but is not limited to:

[0345] Record the task status of each round of federated learning tasks;

[0346] Perform batch updates to the federated ledger;

[0347] Perform synchronization of the federated ledger.

[0348] In a preferred embodiment, the election execution unit includes:

[0349] The election timeout unit is used to set each federation node that is a candidate to reset a random election timeout when starting an election, and wait for the voting results within the timeout period.

[0350] In a preferred embodiment, the election execution unit includes:

[0351] A voting criteria unit, used to implement each federation node to follow the first voting criteria and only vote for nodes with a state no older than its own ledger entry; and,

[0352] Each federal node follows the second voting criterion and can only vote once in one term. Under the condition that the first voting criterion is met, the vote is cast for the federal node corresponding to the request vote message RequestVote that is first received.

[0353] In a preferred embodiment, the message unit includes:

[0354] A federated ledger update unit, where each request received from a client contains an instruction to be executed by the replicated state machine;

[0355] The coordinator appends the instruction as a new ledger entry to the federated ledger, and then sends the AppendEntries message to other federated nodes in parallel.

[0356] After the ledger entry has been securely replicated by other federated nodes that are followers, the coordinator applies the ledger entry to the coordinator's replication state machine and returns the execution result to the client.

[0357] When the coordinator replicates the created ledger entry to the majority of federated nodes, the ledger entry is committed; and before the coordinator commits the ledger entry, the entries created by other coordinators have also been committed;

[0358] The coordinator tracks the index of the largest ledger entry to be submitted, and the index value will be included in all future additional federated ledger entry messages, so that other federated nodes can know the submission position of the coordinator; if the follower knows that a ledger entry has been submitted, the follower will also apply this ledger entry to the local state machine in the order of the ledger entries;

[0359] In the federated ledger, each ledger entry stores a state machine instruction and the term number when the state machine instruction is received from the coordinator. The term number is used to check whether there are inconsistencies; each ledger entry has an integer index value to indicate the position of the ledger entry in the federated ledger.

[0360] In a preferred embodiment, the federated ledger updating unit includes:

[0361] The commit ledger entry unit is used by the coordinator to decide when it is safe to apply the ledger entry to the state machine; the ledger entry applied to the state machine is called committed, and all committed ledger entries are persistent and will eventually be executed by all available state machines.

[0362] An embodiment of the present invention also provides a device for selecting a trusted coordinator for federated learning, comprising: a processor, a memory, and a program stored on the memory and executable on the processor, wherein when the program is executed by the processor, the steps in the method for selecting a trusted third party as a coordinator for federated learning are implemented.

[0363] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method for selecting a trusted third party as a coordinator for federated learning as described above.

[0364] The technical solution provided in the application embodiment is used to select at least one federal node that meets the election conditions from multiple federal nodes when starting a federated learning task, and then start voting with these federal nodes to elect a trusted third party as a coordinator. The new coordinator is used to process collaboration requests from participants.

[0365] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage terminal devices to be detected or any other non-transmission media that can be used to store information that can be accessed by the computer terminal device to be detected. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carriers.

[0366] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for selecting a trusted coordinator for federated learning, characterized in that: The method comprises: When the federated learning process starts, an election is started for at least one federated node that meets the election conditions; Determine whether there is a federal node that has obtained the majority of votes. The federal node that has obtained the majority of votes wins the election and becomes the coordinator; Establish the authority of the federation node through the coordinator and prevent the emergence of new coordinators; Among them, starting an election for at least one federation node that meets the election conditions includes: The participating federation nodes need to meet the election condition that the health HD is greater than the average value of all federation nodes in the data federation. The node health HD = f(ω1H+ω2S+ω3N+ω4L+ω5F). in: H is the hardware configuration score of the federated node, which is related to one or more of the machine’s CPU, motherboard architecture, memory, and hard disk, H∈(0,100); S is the software environment score of the federated node, which is related to one or more of the machine’s operating system type and version, platform software, and functional component versions, S∈(0,100); N is the network performance score of the federated node, which is related to one or more of the network bandwidth, transmission delay and packet loss rate, N∈(0,100); L is the current resource load score of the federated node, which is related to one or more of the CPU occupancy rate, memory occupancy rate, and network bandwidth occupancy rate, L∈(0,100); F is the failure rate of the federated node in the most recent statistical period, which is related to the number of downtimes, F∈(0,100); ω1, ω2, ω3, ω4, and ω5 are weight coefficients, which are determined based on experience and experiments; The process of determining whether there is a federal node that has obtained the majority of votes and the federal node that has obtained the majority of votes wins the election and becomes the coordinator includes: to start an election; A federated node increases its current term number and transitions to candidate state; The federated node sends a request-to-vote message RequestVote to other federated nodes in the data federation in parallel, wherein the RequestVote requests other federated nodes to vote for the candidate federated node; The federation node remains in the candidate state until one of the three events occurs: Event 1, the follower wins the election and becomes the coordinator; Event 2, another federation node becomes the coordinator; Event 3, after a period of time, there is no winning federation node, ending the current election; The RequestVote requesting other federation nodes to vote for the candidate’s federation node includes: Each federated node follows the first voting criterion and only votes for nodes with a state no older than its own ledger entry; and, Each federal node follows the second voting criteria and can only vote once in one term. Under the condition that the first voting criteria is met, the vote will be cast for the federal node corresponding to the RequestVote that is first received.

2. The method according to claim 1, characterized in that Each federated node is in any of the participant, coordinator, candidate or follower states at different times, where: Participants,both the initiator and the participants in the federated learning process are,participants; Coordinator: A federated node in this state is a trusted third party that handles coordination requests from participants. Candidate is the state of a federation node that is eligible to participate in the election of a new coordinator; Followers do not send any requests and only respond to requests from coordinators or candidates.

3. The method according to claim 1, characterized in that When the federated learning process is initiated, before there is a request for data collaboration, it also includes: If the federated node as a follower does not receive any message for a period of time and determines that the election has timed out, the follower determines that there is no coordinator in the data federation and attempts to start the federated learning process.

4. The method according to claim 1, characterized in that: The method also includes: Each federal node that serves as a candidate will reset a random election timeout when starting an election, and wait for the voting results during the timeout. If the timeout expires, a new round of elections will begin.

5. The method according to claim 1, characterized in that Establishing the authority of the federation node and preventing the generation of a new coordinator includes periodically sending heartbeat packets to all followers to maintain the authority of the federation node itself.

6. The method according to claim 1, characterized in that After establishing the authority of the federation node and preventing the generation of new coordinators, it also includes: The coordinator coordinates federated learning tasks and synchronizes the federated ledger during the data collaborative modeling process.

7. The method according to claim 6, characterized in that The coordinator coordinates the federated learning tasks in the data collaborative modeling process. Synchronizing the federated ledger includes: the coordinator sends an AppendEntries message to append federated ledger entries, which notifies other federated nodes to append federated ledger entries and coordinate federated learning tasks.

8. A device for selecting a trusted coordinator for federated learning, characterized in that The device includes: The election module is started, which is used to start the election for at least one federation node that meets the election conditions when the federation learning process is started; An election module is used to determine whether there is a federal node that obtains the majority of votes. The federal node that obtains the majority of votes wins the election and becomes the coordinator; The coordinator generation module is used for the coordinator to establish the authority of the federation node and prevent the generation of new coordinators; Among them, starting an election for at least one federation node that meets the election conditions includes: The participating federation nodes need to meet the election condition that the health HD is greater than the average value of all federation nodes in the data federation. The node health HD = f(ω1H+ω2S+ω3N+ω4L+ω5F). in: H is the hardware configuration score of the federated node, which is related to one or more of the machine’s CPU, motherboard architecture, memory, and hard disk, H∈(0,100); S is the software environment score of the federated node, which is related to one or more of the machine’s operating system type and version, platform software, and functional component versions, S∈(0,100); N is the network performance score of the federated node, which is related to one or more of the network bandwidth, transmission delay and packet loss rate, N∈(0,100); L is the current resource load score of the federated node, which is related to one or more of the CPU occupancy rate, memory occupancy rate, and network bandwidth occupancy rate, L∈(0,100); F is the failure rate of the federated node in the most recent statistical period, which is related to the number of downtimes, F∈(0,100); ω1, ω2, ω3, ω4, and ω5 are weight coefficients, which are determined based on experience and experiments; The process of determining whether there is a federal node that has obtained the majority of votes and the federal node that has obtained the majority of votes wins the election and becomes the coordinator includes: to start an election; A federated node increases its current term number and transitions to candidate state; The federated node sends a request-to-vote message RequestVote to other federated nodes in the data federation in parallel, wherein the RequestVote requests other federated nodes to vote for the candidate federated node; The federation node remains in the candidate state until one of the three events occurs: Event 1, the follower wins the election and becomes the coordinator; Event 2, another federation node becomes the coordinator; Event 3, after a period of time, there is no winning federation node, ending the current election; The RequestVote requesting other federation nodes to vote for the candidate’s federation node includes: Each federated node follows the first voting criterion and only votes for nodes with a state no older than its own ledger entry; and, Each federal node follows the second voting criteria and can only vote once in one term. Under the condition that the first voting criteria is met, the vote will be cast for the federal node corresponding to the RequestVote that is first received.

9. The device according to claim 8, characterized in that Also includes: The node status module is used to record the status of each federated node at different times. Each federated node is in any of the participant, coordinator, candidate or follower states at different times, where: Participant, is the state of the federation node before the election coordinator; Coordinator: A federated node in this state is a trusted third party that handles coordination requests from participants. Candidate is the state of a federation node that is eligible to participate in the election of a new coordinator; Followers do not send any requests and only respond to requests from coordinators or candidates.

10. The device according to claim 8, characterized in that Also includes: The startup judgment module is used to start the federated learning process. Before there is a request for data collaboration, if the federated node as a follower does not receive any message for a period of time and determines that the election has timed out, the follower determines that there is no coordinator in the data federation and attempts to start the federated learning process.

11. A device for selecting a trusted coordinator for federated learning, characterized in that include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images