Federal learning node selection method, device and system and storage medium

Abstract

The invention relates to a federated learning node selection method, device and system and a storage medium. The federated learning node selection method comprises the following steps: receiving subscription information or request information sent by a central node; local data feature distribution information of candidate distributed nodes is determined, and the candidate distributed nodes are distributed nodes capable of participating in federated learning service; the data feature distribution information is sent to a center node, and the data feature distribution information is used for the center node to select a target distributed node from candidate distributed nodes. According to the invention, federal learning node selection can be carried out based on data feature distribution.

Description

Technical Field

[0001] The present disclosure relates to the field of wireless communications, and in particular to a method, device, system, and storage medium for selecting a federated learning node. Background Art

[0002] The federated learning system is essentially a distributed learning system, in which multiple distributed nodes can jointly provide services for the central node. Under the distributed training architecture based on federated learning, the distributed nodes on the edge side can use their own collected data to perform training tasks locally, and then upload the trained local model parameters to the central node for model aggregation. Compared with centralized learning that directly uploads the original training data, this architecture chooses to upload the model parameters after training, which can effectively reduce the data transmission cost. In addition, unlike traditional distributed learning, the distributed nodes (participants) of federated learning have complete autonomy over their local data, and can independently decide whether and when to join federated learning for federated learning modeling. It can solve the data island problem while meeting data privacy, security and regulatory requirements.

[0003] In each round of federated learning training, after each distributed node completes its own training, the trained model parameters are transmitted to the central node under an encryption mechanism. The central node then performs model fusion and sends it to the distributed nodes for the next round of training. Summary of the invention

[0004] The inventors found through research that due to differences in the quality of data sets, the directions of training model parameters are different, and not all model parameters uploaded by nodes are beneficial to the convergence direction of the global model. Therefore, it is necessary to select the nodes participating in the training.

[0005] In view of at least one of the above technical problems, the present disclosure provides a method, device and system, and storage medium for selecting a federated learning node, which can evaluate the reasoning analysis accuracy of network elements used to implement network data analysis functions, and can select federated learning nodes based on data feature distribution.

[0006] According to one aspect of the present disclosure, a method for selecting a federated learning node is provided, comprising:

[0007] Receive subscription information or request information sent by the central node;

[0008] Determine data feature distribution information of a local candidate distributed node, wherein the candidate distributed node is a distributed node that can participate in the federated learning service;

[0009] The data feature distribution information is sent to the central node, wherein the data feature distribution information is used for the central node to select a target distributed node from candidate distributed nodes.

[0010] In some embodiments of the present disclosure, the subscription information or request information includes at least one of a number of training rounds, a training cycle, a deviation threshold, and an expected waiting time.

[0011] In some embodiments of the present disclosure, determining the local data feature distribution information of the candidate distributed node includes: generating a mean vector and an offset value vector based on the local data feature distribution; determining whether the received subscription information or request information matches the candidate distributed node configuration information to determine whether to join federated learning; and if it is determined to join federated learning, executing the step of sending the data feature distribution information to the central node.

[0012] In some embodiments of the present disclosure, the determining whether the received subscription information or request information matches the distributed node configuration information includes at least one of the following steps, wherein:

[0013] Determine whether the mean of the offset value vector is greater than the deviation threshold;

[0014] Determine whether the training can be completed within the expected waiting time of the central node;

[0015] Determine whether the analysis type indicated by the central node is supported.

[0016] In some embodiments of the present disclosure, generating a mean vector and an offset value vector according to the local data feature distribution includes:

[0017] Generate a mean vector based on the local data feature distribution, where each element in the mean vector is the mean of the data feature distribution obtained by the specified number of rounds of federated learning training before the current round, where the specified number of rounds is determined by the number of training rounds preconfigured by the central node;

[0018] An offset value vector is determined according to the mean value vector, wherein each element in the offset value vector is a directional deviation between a data feature distribution of a specified number of rounds of federated learning training before the current round and an overall distribution.

[0019] In some embodiments of the present disclosure, sending the data feature distribution information to a central node includes:

[0020] Send response information or notification information to the central node, wherein the response information or notification information includes data feature distribution information, the data feature distribution information includes a mean vector and an offset value vector, and the response information or notification information also includes at least one of a candidate distributed node identifier, an estimated response time of the candidate distributed node, computing resources of the candidate distributed node, and a quality of a communication link from the candidate distributed node to the central node.

[0021] In some embodiments of the present disclosure, the federated learning node selection method further includes:

[0022] Register the node basic information in the information storage unit, wherein the node basic information is used for the central node to discover candidate distributed nodes through the information storage unit, and the node basic information includes at least one of node reachability, federated learning capability and analysis identifier, the node reachability is used to indicate whether the central node can establish a connection with the distributed node, the federated learning capability is used to indicate whether the node supports federated learning, and the analysis identifier is used to indicate the type of analysis service.

[0023] In some embodiments of the present disclosure, the federated learning node selection method further includes:

[0024] Receiving a federated learning service request sent by a central node, wherein the federated learning service request is a federated learning service request sent by the central node to a selected target distributed node;

[0025] Participate in completing model training and feedback training results or analysis results to the central node, where the training results or analysis results are used for the central node to aggregate the training results and analysis results, and the central node is used to send the aggregated results to the target distributed nodes for the next round of training.

[0026] In some embodiments of the present disclosure, in the federated learning process, the federated learning node selection method further includes at least one of the following steps, wherein:

[0027] Periodically trigger node updates through the central node;

[0028] Trigger node updates via target distributed nodes.

[0029] In some embodiments of the present disclosure, triggering node update through a target distributed node includes:

[0030] When at least one training condition is not met, feedback information is sent to the central node to trigger the node reselection mechanism, which triggers the central node to discover at least one candidate distributed node that can participate in the federated learning service through the information storage unit, and then execute the steps of receiving subscription information or request information sent by the central node, determining the local data feature distribution information of the candidate distributed node, and sending the data feature distribution information to the central node, wherein the training conditions include that the mean of the local data feature distribution offset value vector is not greater than the deviation threshold, and the target distributed node completes the training within the expected waiting time of the central node.

[0031] In some embodiments of the present disclosure, when at least one training condition is not met, sending feedback information to the central node includes:

[0032] If at least one training condition is not met, then determine whether the periodic trigger update condition is met;

[0033] When the periodic trigger update conditions are not met, information is fed back to the central node.

[0034] In some embodiments of the present disclosure, the periodic trigger update condition is that the current global training round number is equal to a positive integer multiple of the central node preconfigured training period round number plus 1.

[0035] According to another aspect of the present disclosure, a method for selecting a federated learning node is provided, comprising:

[0036] Send subscription information or request information to candidate distributed nodes;

[0037] Receiving data feature distribution information sent by the candidate distributed node, wherein the data feature distribution information is data feature distribution information of the candidate distributed node local determined by the candidate distributed node;

[0038] A target distributed node is selected from the candidate distributed nodes according to the data feature distribution information.

[0039] In some embodiments of the present disclosure, the subscription information or request information includes at least one of the number of training rounds, training cycle, deviation threshold and expected waiting time, wherein the number of training rounds is used to indicate the number of federated learning training rounds required by the central node for calculating the mean vector and offset value vector of the candidate distributed nodes, the training cycle is used to indicate the number of training rounds required for the federated learning training, the deviation threshold is used to indicate the central node's requirement for the stability of the data feature distribution in the candidate distributed nodes participating in this round of federated learning training over time, and the expected waiting time is used to indicate the maximum time the central node will wait for each candidate distributed node to report the training results or analysis results during a round of federated learning training.

[0040] In some embodiments of the present disclosure, the federated learning node selection method further includes:

[0041] The candidate distributed nodes are discovered through an information storage unit, wherein the information storage unit is used to store basic information of the registered nodes registered in the information storage unit, and the basic information of the nodes includes at least one of node reachability, federated learning capability and analysis identifier, the node reachability is used to indicate whether the central node and the distributed node can establish a connection, the federated learning capability is used to indicate whether the node supports federated learning, and the analysis identifier is used to indicate the type of analysis service.

[0042] In some embodiments of the present disclosure, the receiving of data feature distribution information sent by the candidate distributed node includes: receiving response information or notification information sent by the candidate distributed node, wherein the response information or notification information includes data feature distribution information, the data feature distribution information includes a mean vector and an offset value vector, and the response information or notification information also includes at least one of a distributed node identifier, an estimated response time of the distributed node, distributed node computing resources, and a communication link quality from the distributed node to the central node.

[0043] In some embodiments of the present disclosure, selecting a target distributed node from candidate distributed nodes based on the data feature distribution information includes: selecting a node as a target distributed node from candidate distributed nodes that have chosen to join federated learning training based on response information or notification information sent by the candidate distributed node.

[0044] In some embodiments of the present disclosure, selecting a node as a target distributed node from candidate distributed nodes selected to join the federated learning training according to response information or notification information sent by the candidate distributed node includes:

[0045] Node selection is performed based on the mean vector of each candidate distributed node, and the ratio of the global mean vector to the expected response time is used as the convergence rate;

[0046] A node subset with the largest convergence rate in the direction of the global mean is selected from the candidate distributed nodes, and the nodes in the node subset are used as target distributed nodes.

[0047] In some embodiments of the present disclosure, selecting a node as a target distributed node from the candidate distributed nodes selected to join the federated learning training according to the response information or notification information sent by the candidate distributed node also includes:

[0048] According to the quality of the communication link from the target distributed node to the central node, the communication rate under one communication resource block is estimated;

[0049] Based on the data size of the model parameters to be trained and the expected response time of the distributed nodes, the communication resources are optimally allocated to balance the time it takes for each target distributed node participating in the training to complete a federated learning training.

[0050] In some embodiments of the present disclosure, the federated learning node selection method further includes:

[0051] Send a federated learning service request to the selected target distributed node;

[0052] Receive training results or analysis results fed back by the target distributed node after completing model training;

[0053] The training results or analysis results are aggregated, and the aggregated results are sent to the target distributed node for the next round of training.

[0054] In some embodiments of the present disclosure, in the federated learning process, the federated learning node selection method further includes at least one of the following steps, wherein:

[0055] Periodically trigger node updates through the central node;

[0056] Trigger node updates via target distributed nodes.

[0057] In some embodiments of the present disclosure, periodically triggering node update through a central node includes:

[0058] The central node determines whether the periodic trigger update conditions are met;

[0059] When the periodic trigger update conditions are met, the node reselection mechanism is triggered, and the central node is triggered to discover at least one candidate distributed node that can participate in the federated learning service through the information storage unit, and then execute the steps of receiving subscription information or request information sent by the central node, determining the local data feature distribution information of the candidate distributed node, and sending the data feature distribution information to the central node.

[0060] In some embodiments of the present disclosure, the periodic trigger update condition is that the current global training round number is equal to a positive integer multiple of the central node preconfigured training period round number plus 1.

[0061] According to another aspect of the present disclosure, there is provided a distributed node, comprising:

[0062] An information receiving module, configured to receive subscription information or request information sent by a central node;

[0063] A data distribution determination module, configured to determine local data feature distribution information of a candidate distributed node;

[0064] The data distribution sending module is configured to send the data feature distribution information to the central node, wherein the data feature distribution information is used for the central node to select a target distributed node from the candidate distributed nodes.

[0065] According to another aspect of the present disclosure, there is provided a central node, comprising:

[0066] An information sending module, configured to send subscription information or request information to the candidate distributed nodes;

[0067] A data distribution receiving module is configured to receive data feature distribution information sent by a candidate distributed node, wherein the data feature distribution information is data feature distribution information of a local distributed node determined by the distributed node;

[0068] The node selection module is configured to select a target distributed node from the candidate distributed nodes according to the data feature distribution information.

[0069] According to another aspect of the present disclosure, a federated learning node selection device is provided, comprising:

[0070] a memory configured to store instructions;

[0071] The processor is configured to execute the instruction so that the federated learning node selection device performs operations to implement the federated learning node selection method described in any of the above embodiments.

[0072] According to another aspect of the present disclosure, a federated learning node selection system is provided, comprising a distributed node as described in any of the above embodiments and a central node as described in any of the above embodiments.

[0073] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0074] The present disclosure can select federated learning nodes based on data feature distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0076] Figure 1 Schematic diagram of some embodiments of the federated learning node selection method disclosed in the present invention.

[0077] Figure 2 Schematic diagrams of other embodiments of the federated learning node selection method disclosed in the present invention.

[0078] Figure 3 Schematic diagrams of some further embodiments of the federated learning node selection method disclosed in the present invention.

[0079] Figure 4 Schematic diagrams of other embodiments of the federated learning node selection method disclosed in the present invention.

[0080] Figure 5 Schematic diagrams of some further embodiments of the federated learning node selection method disclosed in the present invention.

[0081] Figure 6 Schematic diagrams of some further embodiments of the federated learning node selection method disclosed in the present invention.

[0082] Figure 7 Schematic diagram of some embodiments of the distributed nodes disclosed herein.

[0083] Figure 8 Schematic diagram of some embodiments of the central node of the present disclosure.

[0084] Fig. 9 A schematic diagram of the structure of some embodiments of the federated learning node selection device disclosed in the present invention. DETAILED DESCRIPTION

[0085] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0086] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0087] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0088] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0089] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0090] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0091] Through research, the inventors found that the related technology node selection method can improve the global model convergence rate of the selected nodes. However, in the process of federated learning, there is a situation where the node data set changes as the training progresses. The related technology node selection scheme does not consider the problem that the distribution of data characteristics collected by the nodes fluctuates over time, resulting in the deterioration of the data set quality.

[0092] Therefore, how to reasonably consider the volatility of the data set over time to select distributed nodes to participate in model aggregation in order to improve the efficiency and accuracy of federated learning has become an urgent problem to be solved.

[0093] In view of at least one of the above technical problems, the present disclosure provides a method, device and system for selecting a federated learning node, and a storage medium. The present disclosure is described below through specific embodiments.

[0094] Figure 1 Schematic diagram of some embodiments of the method for selecting a federated learning node of the present disclosure. Preferably, the present embodiment can be performed by the federated learning node selection system of the present disclosure or the distributed node of the present disclosure. The method includes at least one of steps 11 to 13, wherein:

[0095] Step 11: The candidate distributed node receives the subscription information or request information sent by the central node.

[0096] In some embodiments of the present disclosure, the subscription information or request information includes at least one of a number of training rounds, a training cycle, a deviation threshold, and an expected waiting time.

[0097] In some embodiments of the present disclosure, Figure 1 Before step 11 of the embodiment, the federated learning node selection method may further include: registering basic node information in an information storage unit, wherein the basic node information is used for the central node to discover candidate distributed nodes through the information storage unit, and the basic node information includes at least one of node reachability, federated learning capability and analysis identifier, the node reachability is used to indicate whether the central node can establish a connection with the distributed node, the federated learning capability is used to indicate whether the node supports federated learning, and the analysis identifier is used to indicate the type of analysis service.

[0098] Step 12: The candidate distributed node determines data feature distribution information local to the candidate distributed node, wherein the candidate distributed node is a distributed node that can participate in the federated learning service.

[0099] In some embodiments of the present disclosure, step 12 may include at least one of steps 121 to 123, wherein:

[0100] Step 121, generating a mean vector and an offset value vector according to the local data feature distribution.

[0101] In some embodiments of the present disclosure, step 121 may include: generating a mean vector based on the local data feature distribution, wherein each element in the mean vector is the mean of the data feature distribution obtained by the specified number of rounds of federated learning training before the current round, and the specified number of rounds is determined by the number of training rounds preconfigured by the central node; determining an offset value vector based on the mean vector, wherein each element in the offset value vector is the directional deviation between the data feature distribution of the specified number of rounds of federated learning training before the current round and the overall distribution.

[0102] Step 122, determining whether the received subscription information or request information matches the candidate distributed node configuration information to determine whether to join the federated learning.

[0103] In some embodiments of the present disclosure, in step 122, the step of determining whether the received subscription information or request information matches the candidate distributed node configuration information may include at least one of the following steps, wherein: determining whether the mean of the offset value vector is greater than a deviation threshold; determining whether the candidate distributed node itself can complete training within the expected waiting time of the central node; determining whether the candidate distributed node itself supports the analysis type indicated by the central node.

[0104] Step 123, when it is determined to join the federated learning, execute the step of sending the data feature distribution information to the central node.

[0105] Step 13: The candidate distributed nodes send the data feature distribution information to the central node, wherein the data feature distribution information is used by the central node to select a target distributed node from the candidate distributed nodes.

[0106] In some embodiments of the present disclosure, step 13 may include: the distributed node sends response information or notification information to the central node, wherein the response information or notification information includes data feature distribution information, the data feature distribution information includes a mean vector and an offset value vector, and the response information or notification information also includes at least one of a candidate distributed node identifier, an estimated response time of the candidate distributed node, computing resources of the candidate distributed node, and a communication link quality from the candidate distributed node to the central node.

[0107] In some embodiments of the present disclosure, Figure 1 After step 13 of the embodiment, the federated learning node selection method may further include: the target distributed node receives a federated learning service request sent by the central node, wherein the federated learning service request is a federated learning service request sent by the central node to the selected target distributed node; the target distributed node participates in completing the model training and feeds back the training results or analysis results to the central node, wherein the training results or analysis results are used for the central node to aggregate the training results and analysis results, and the central node is used to send the aggregated results to the target distributed node for the next round of training.

[0108] The above-mentioned embodiments of the present disclosure propose a method for selecting and updating federated learning nodes based on data feature distribution, thereby reasonably selecting distributed nodes to participate in model aggregation in the federated learning system, thereby improving learning efficiency and accuracy.

[0109] In some embodiments of the present disclosure, during the federated learning process, the federated learning node selection method may further include at least one of the following steps, wherein: node updates are periodically triggered by a central node; and node updates are triggered by a target distributed node.

[0110] In some embodiments of the present disclosure, the step of triggering node update through the target distributed node may include: when at least one training condition is not met, sending feedback information to the central node, triggering the node reselection mechanism, triggering the central node to discover at least one candidate distributed node that can participate in the federated learning service through the information storage unit, and then executing the steps of receiving subscription information or request information sent by the central node, determining the local data feature distribution information of the candidate distributed node, and sending the data feature distribution information to the central node, wherein the training conditions include that the mean of the local data feature distribution offset value vector is not greater than the deviation threshold, and the target distributed node completes the training within the expected waiting time of the central node; and then executing Figure 1 Step 11 to step 13 of the embodiment, that is, executing the steps of receiving subscription information or request information sent by the central node, determining the local data feature distribution information of the candidate distributed nodes, and sending the data feature distribution information to the central node.

[0111] The node update method selected in the above embodiment of the present disclosure further optimizes the node reselection trigger mechanism in the federated learning process based on the judgment criteria and the interaction mechanism.

[0112] The above-mentioned embodiments of the present disclosure propose a central node periodic triggering and distributed node triggering node update mechanism, that is, the central node triggers node reselection after every certain training rounds; and the distributed nodes trigger node reselection when the conditions are insufficient, thereby eliminating the need to transmit relevant information in each round of training.

[0113] In some embodiments of the present disclosure, the step of sending feedback information to the central node when at least one training condition is not met may include: when at least one training condition is not met, determining whether a periodic trigger update condition is met; and when the periodic trigger update condition is not met, feeding back information to the central node.

[0114] In some embodiments of the present disclosure, the periodic trigger update condition is that the current global training round number is equal to a positive integer multiple of the central node preconfigured training period round number plus 1.

[0115] In the distributed node triggered node update mechanism proposed in the above embodiments of the present disclosure, the distributed node can trigger node reselection only when the update condition is triggered non-periodically in the current training round, thereby avoiding the situation where both the central node and the distributed node trigger node reselection at the same time.

[0116] Figure 2 Schematic diagrams of other embodiments of the method for selecting a federated learning node of the present disclosure. Preferably, this embodiment can be performed by the federated learning node selection system of the present disclosure or the central node of the present disclosure. The method includes at least one of steps 21 to 23, wherein:

[0117] Step 21: The central node sends subscription information or request information to the candidate distributed nodes.

[0118] In some embodiments of the present disclosure, the subscription information or request information may include at least one of the number of training rounds, training cycle, deviation threshold and expected waiting time, wherein the number of training rounds is used to indicate the number of federated learning training rounds required by the central node for calculating the mean vector and offset value vector of the candidate distributed nodes, the training cycle is used to indicate the number of training rounds required for the federated learning training, the deviation threshold is used to indicate the central node's requirement for the stability of the data feature distribution in the candidate distributed nodes participating in this round of federated learning training over time, and the expected waiting time is used to indicate the maximum time the central node will wait for each candidate distributed node to report the training results or analysis results during a round of federated learning training.

[0119] In some embodiments of the present disclosure, Figure 2Before step 21 of the embodiment, the federated learning node selection method may also include: the central node discovers the candidate distributed node through the information storage unit, wherein the information storage unit is used to store the basic information of the registered node registered in the information storage unit, and the basic information of the node includes at least one of the node reachability, federated learning capability and analysis identifier, the node reachability is used to indicate whether the central node can establish a connection with the distributed node, the federated learning capability is used to indicate whether the node supports federated learning, and the analysis identifier is used to indicate the type of analysis service.

[0120] Step 22, the central node receives data feature distribution information sent by the candidate distributed node, wherein the data feature distribution information is the local data feature distribution information of the candidate distributed node determined by the candidate distributed node.

[0121] In some embodiments of the present disclosure, step 22 may include: the central node receives response information or notification information sent by the candidate distributed node, wherein the response information or notification information includes data feature distribution information, the data feature distribution information includes a mean vector and an offset value vector, and the response information or notification information also includes at least one of the candidate distributed node identifier, the candidate distributed node estimated response time, the candidate distributed node computing resources, and the communication link quality from the candidate distributed node to the central node.

[0122] Step 23: The central node selects a target distributed node from the candidate distributed nodes according to the data feature distribution information.

[0123] In some embodiments of the present disclosure, step 23 may include: the central node selects a node as a target distributed node from the candidate distributed nodes that choose to join the federated learning training according to the response information or notification information sent by the candidate distributed node.

[0124] In some embodiments of the present disclosure, the step of selecting a node as a target distributed node from distributed nodes selected to join the federated learning training according to the response information or notification information sent by the candidate distributed node may include at least one step from step 231 to step 232, wherein:

[0125] Step 231 , performing node selection based on the mean vector of each candidate distributed node, and taking the ratio of the global mean vector to the expected response time as the convergence rate.

[0126] Step 232: Select a node subset with the highest convergence rate in the direction of the global mean from the candidate distributed nodes, and use the nodes in the node subset as target distributed nodes.

[0127] In some embodiments of the present disclosure, the step of selecting a node as a target distributed node from distributed nodes selected to join the federated learning training according to the response information or notification information sent by the candidate distributed node may also include at least one step from step 233 to step 234, wherein:

[0128] Step 233, estimating the communication rate under one communication resource block according to the communication link quality from the target distributed node to the central node.

[0129] Step 234, based on the data size of the model parameters to be trained and the expected response time of the distributed nodes, optimize the allocation of communication resources (such as the number of communication resource blocks) to balance the time for each target distributed node participating in the training to complete a federated learning training.

[0130] In some embodiments of the present disclosure, Figure 2 After step 23 of the embodiment, the federated learning node selection method also includes: the central node sends a federated learning service request to the selected target distributed node; receives the training results or analysis results fed back by the target distributed node after completing the model training; aggregates the training results or analysis results, and sends the aggregated results to the target distributed node for the next round of training.

[0131] In response to the problem that the characteristic distribution of data collected by related technical nodes fluctuates over time, resulting in deterioration of data set quality, such as the technical problem that the characteristic distribution of the two rounds of training data sets of the nodes participating in federated learning training is quite different, the above-mentioned embodiments of the present disclosure propose that in the process of distributed nodes selecting to participate in training, the characteristic distribution results of data sets of multiple rounds of training are integrated, and the nodes whose characteristic distribution of data sets fluctuates greatly with the training are removed according to the set threshold, thereby solving the problem that the quality of data sets of the nodes participating in training deteriorates as the number of training rounds increases.

[0132] Furthermore, in the node selection process of the above-mentioned embodiment of the present disclosure, nodes are selected based on the degree of difference between the feature distribution of the data set of multiple rounds of training of the distributed node set and the feature distribution of the data set of global training, so as to avoid selecting nodes with small current differences but large long-term differences, so as to improve the long-term model aggregation efficiency.

[0133] In some embodiments of the present disclosure, during the federated learning process, the federated learning node selection method may further include at least one of the following steps, wherein: node updates are periodically triggered by a central node; and node updates are triggered by a target distributed node.

[0134] In some embodiments of the present disclosure, the periodic triggering of node update by the central node includes: the central node determines whether the periodic triggering update condition is met; if the periodic triggering update condition is met, the node reselection mechanism is triggered, and the central node is triggered to find at least one candidate distributed node that can participate in the federated learning service through the information storage unit; and then executes Figure 1 Step 11 to step 13 of the embodiment, that is, executing the steps of receiving subscription information or request information sent by the central node, determining the local data feature distribution information of the candidate distributed nodes, and sending the data feature distribution information to the central node.

[0135] In some embodiments of the present disclosure, the periodic trigger update condition is that the current global training round number is equal to a positive integer multiple of the central node preconfigured training period round number plus 1.

[0136] The above-mentioned embodiments of the present disclosure aim at the discovery / selection of distributed nodes, and propose a node selection and update scheme based on data feature distribution, thereby improving the model convergence rate.

[0137] Figure 3 Schematic diagram of some other embodiments of the method for selecting a federated learning node of the present disclosure. Preferably, this embodiment can be performed by the federated learning node selection system of the present disclosure. The method includes at least one of steps 1 to 7, wherein:

[0138] Step 1: The distributed node registers the basic information of the node in the information storage unit. The central node finds at least one candidate distributed node that can participate in the federated learning service through the information storage unit.

[0139] In some embodiments of the present disclosure, the basic node information may include node reachability, federated learning capability, analytics identifier (Analytics ID), etc.

[0140] In some embodiments of the present disclosure, node reachability is used to indicate whether a central node can establish a connection with a distributed node; federated learning capability is used to indicate whether the node supports federated learning; and an analysis identifier is used to indicate the type of analysis service.

[0141] Step 2: The central node sends subscription information or request information to the candidate distributed nodes.

[0142] In some embodiments of the present disclosure, the subscription information or request information may include the number of training rounds T0, training cycle, deviation threshold, expected waiting time, etc.

[0143] In some embodiments of the present disclosure, the number of training rounds is used to indicate the number of federated learning training rounds required in the process of the central node calculating the mean vector and the offset value vector for the candidate distributed nodes.

[0144] In some embodiments of the present disclosure, the training period is used to indicate the number of training rounds required for federated learning training.

[0145] In some embodiments of the present disclosure, the deviation threshold is used to indicate the central node's requirement on the stability of the distribution of data features in the candidate distributed nodes participating in this round of federated learning training over time.

[0146] In some embodiments of the present disclosure, the expected waiting time is used to indicate the maximum time that the central node will wait for each candidate distributed node to report the training results or analysis results within a certain round of federated learning training process, that is, each candidate distributed node needs to complete its own training / analysis within the expected waiting time and send the training / analysis results to the central node.

[0147] In step 3, the candidate distributed nodes generate mean vectors and offset value vectors based on the data feature distribution of a certain round of federated learning training of local data, and then determine whether the deviation threshold and expected waiting time conditions in the central node subscription information are met, so as to choose whether to join the training.

[0148] In some embodiments of the present disclosure, step 3 may include at least one of step 31 and step 32, wherein:

[0149] In step 31, based on the training round number T0 of the central node subscription / request information, the candidate distributed node generates a mean vector and an offset value vector for the local data feature distribution of the previous T0 rounds of federated learning training to characterize the statistical characteristics of this type of data features in a certain number of training rounds.

[0150] In some embodiments of the present disclosure, step 31 may include: in the calculation of generating the mean vector and the offset value vector by the candidate distributed node, let represents the various features and their number of the kth distributed node in the lth round of federated learning training. Where M represents the global M types of data features, It is represented by the number of distributions of the m-th feature in the k-th distributed node in the l-th round of federated learning training. When l>T0, let T=T0; otherwise, T=l-1. Then the mean vector of the k-th distributed node in the l-th round of federated learning training can be obtained by formula (1).

[0151]

[0152] The offset value vector can be obtained through formula (2).

[0153]

[0154] The elements in the offset value vector represent the directional deviation between the data feature distribution of the T rounds of federated learning training before this round and the overall distribution. Its value is [-1, 1]. The closer the value is to 1, the smaller the directional deviation.

[0155] Step 32: The candidate distributed node determines whether the received subscription / request information matches the candidate distributed node configuration information and decides whether to join the federated learning.

[0156] In some embodiments of the present disclosure, step 32 may include: the candidate distributed node determines whether the mean of the offset value vector is greater than the deviation threshold; the candidate distributed node determines whether it can complete training within the expected waiting time of the central node; the candidate distributed node determines whether it supports the analysis type indicated by the central node; the candidate distributed node determines whether it decides to join federated learning.

[0157] Step 4: The candidate distributed node sends notification information or response information to the central node.

[0158] In some embodiments of the present disclosure, the notification information or response information may include the candidate distributed node ID, the estimated response time of the candidate distributed node, the computing resources of the candidate distributed node, the mean vector, the offset value vector, the communication link quality from the candidate distributed node to the central node, and other information.

[0159] Step 5: The central node selects a target distributed node from the candidate distributed nodes according to the data feature distribution information.

[0160] In some embodiments of the present disclosure, step 5 may include: the central node selects a suitable node as the target distributed node from the candidate distributed nodes that choose to join the federated learning training based on the response / notification information fed back by the distributed nodes.

[0161] In some embodiments of the present disclosure, step 5 may include at least one of steps 5a to 5c, wherein:

[0162] Step 5a, node selection based on the mean vector.

[0163] In some embodiments of the present disclosure, step 5a may include: the central node performs node selection based on the collected mean vectors of each candidate distributed node.

[0164] In some embodiments of the present disclosure, step 5a may include: taking the ratio of the global mean vector to the expected response time as an evaluation index of the convergence rate; and selecting a subset with the largest convergence rate in the direction of the global mean from the candidate distributed nodes.

[0165] In some embodiments of the present disclosure, step 5a may include: the central node first selects candidate distributed nodes that are beneficial to the convergence of global model training under the expected waiting time constraint based on the mean vector of the candidate distributed nodes.

[0166] Figure 4 Schematic diagrams of other embodiments of the method for selecting a federated learning node of the present disclosure. Preferably, this embodiment can be executed by the federated learning node selection system of the present disclosure or the central node of the present disclosure. The method (e.g. Figure 3 Step 5a of the embodiment or Figure 2 Step 23 of the embodiment includes at least one of steps 51 to 59, wherein:

[0167] Since the results of model parameter training are related to the data feature distribution, the central node can obtain the mean vector and the mean vector of the global training data feature distribution of the previous T rounds according to the mean vector of each node. It represents the direction of the global model parameters to a certain extent, so the selected nodes should be those whose mean vectors are close to the direction of the global label vector.

[0168] Step 51, pre-initialize the candidate selection node set K = {1, ..., k, ..., K}, indicating that all nodes are selected to participate in federated learning; determine the convergence rate as the global convergence rate.

[0169] In some embodiments of the present disclosure, in step 51, the step of determining the convergence rate as the global convergence rate may include: the candidate distributed node estimated response time set The convergence rate can be expressed as the ratio of the projection of the selected node mean vector on the global mean vector to the maximum expected response time of the selected node. For example, if all nodes are initially selected to participate in federated learning, the global convergence rate can be expressed as v = |q l | / max(t comp,l ).

[0170] Step 52, sequentially obtain all subsets K_O from the node set K after removing one or two nodes.

[0171] Step 53: Obtain the maximum value t_K of the expected response time in all node subsets K_O.

[0172] Step 54, calculate the sum q_K of the respective label vectors in all node subsets K_O.

[0173] In some embodiments of the present disclosure, step 54 may include: firstly adding the mean vectors of each node in pairs, and forming a new mean vector set with the original mean vectors of each node. From the original mean vector and Eliminate the vectors in Q one by one and get the sum q_K of the label vectors in all node subsets K_O.

[0174] Step 55, calculate the projection |q_K0| of each node subset q_K on the global label vector.

[0175] Step 56, calculating the convergence rate of each node subset |q_K0| / t_K.

[0176] Step 57, determine whether the maximum convergence rate changes?

[0177] Step 58: Select the node subset K_O as the new node set K.

[0178] Step 59: Output the node set K with the maximum convergence rate as the target distributed node set.

[0179] In steps 56 to 59 of the above embodiment of the present disclosure, the convergence rates after removing one or two nodes from the selection node set K are calculated in turn, and the node set with the largest convergence rate is compared and selected as the new selection node set; the above steps are repeated until the convergence rate no longer increases, thereby outputting the final node selection set K.

[0180] The above-mentioned embodiment of the present disclosure proposes that in the distributed node discovery process, the feature distribution results of the data set of the previous T rounds of training are integrated, and the nodes whose feature distribution of the data set fluctuates greatly with the training are removed according to the set threshold, so as to solve the problem that the quality of the data set deteriorates with the training. Nodes are selected based on the difference between the feature distribution of the data set of the previous T rounds of training of the distributed node set and the feature distribution of the data set of the previous T rounds of training of the global node set, so as to avoid selecting nodes with small current differences but large long-term differences.

[0181] Step 5b, estimating the communication transmission rate.

[0182] In some embodiments of the present disclosure, step 5b may include: estimating a communication rate under a communication resource block according to the quality of the communication link from the target distributed node to the central node.

[0183] Step 5c: Optimize communication resource configuration.

[0184] In some embodiments of the present disclosure, step 5c may include: optimizing communication resource configuration to balance the time required for each node participating in the training to complete a federated learning training.

[0185] In some embodiments of the present disclosure, step 5c may include: estimating the communication rate under one communication resource block based on the quality of the communication link from the distributed node to the central node; finally, based on the amount of model parameter data to be trained and the expected response time of the distributed node, the number of allocated communication resource blocks can be optimized so that the time required for a round of federated learning training of the selected target distributed node is as balanced as possible.

[0186] In some embodiments of the present disclosure, step 5b and step 5c may include: the time required for the lth round of federated learning training of the kth distributed node is composed of the expected response time of the distributed node and the communication transmission delay, which can be expressed as represents the transmission delay, Z represents the amount of model training parameter data, It means that the central node estimates the communication rate under a communication resource block based on the quality of the communication link from the distributed nodes to the central node. represents the number of communication resource blocks allocated to the kth distributed node, which can be optimized The time required for one round of federated learning training of the selected target nodes is similar. The central node is based on max(t l ) Adjust the expected waiting time to avoid the distributed nodes being unable to complete training within the expected waiting time.

[0187] Step 6: The central node sends a federated learning service request to the selected target distributed node to start federated learning training.

[0188] In some embodiments of the present disclosure, step 6 may include: the central node sends a federated learning service request to the selected target distributed node; the distributed node participates in completing the model training and feeds back the training / analysis results; the central node aggregates the received training results or analysis results, and sends the aggregated results to the target distributed node for the next round of training.

[0189] The above-mentioned embodiment of the present disclosure discloses a node selection method based on data feature distribution. Based on the data feature distribution of a certain number of training rounds of distributed nodes, distributed nodes that are beneficial to the convergence of global model training are selected to participate in federated learning, so as to improve the learning efficiency and accuracy of model aggregation.

[0190] The above-mentioned embodiment of the present disclosure proposes that in the distributed node discovery process, the characteristic distribution results of the data set of the first T rounds of training are integrated, and the nodes whose characteristic distribution of the data set fluctuates greatly with the training are removed according to the set threshold, thereby solving the problem that the quality of the data set of the nodes participating in the training deteriorates with the increase of training rounds.

[0191] The above-mentioned embodiment of the present disclosure selects nodes based on the difference between the feature distribution of the data set of multiple rounds of training of the distributed node set and the feature distribution of the global data set, avoiding the selection of nodes with small current differences but large long-term differences, so as to improve the long-term model aggregation efficiency.

[0192] Step 7: Trigger node update periodically through the central node or the target distributed node, and perform the node update process.

[0193] In some embodiments of the present disclosure, the triggering mechanism of node reselection can be performed in two ways: periodic triggering of the central node and triggering of the target distributed node.

[0194] Figure 5 Schematic diagrams of some other embodiments of the method for selecting a federated learning node of the present disclosure. Preferably, this embodiment can be executed by the federated learning node selection system of the present disclosure. Figure 5 The embodiment provides a node update method of the present disclosure that is periodically triggered by a central node. The method (e.g. Figure 3 Step 7) of the embodiment may include at least one of steps 701 to 704, wherein:

[0195] Step 701: The central node determines whether the periodic trigger update condition is met, and if so, triggers the node reselection mechanism.

[0196] In some embodiments of the present disclosure, the periodic trigger update condition is that the current global training round number is equal to a positive integer multiple of the central node preconfigured training period round number plus 1.

[0197] Step 702: The central node discovers at least one candidate distributed node that can participate in the federated learning service through the information storage unit.

[0198] Step 703, according to the node selection process ( Figure 3 Step 2 to step 5c in the embodiment, Figure 1 Steps 11 to 13 of the embodiment, or Figure 2 Step 21-step 23) of the embodiment is used to select a node.

[0199] Step 704: Federated learning service request and model federated learning training.

[0200] In some embodiments of the present disclosure, step 704 may include Figure 3 The specific steps of step 6 in the embodiment.

[0201] In the above-mentioned embodiment of the present disclosure, the central node is periodically triggered by judging whether the current number of global training rounds is equal to a positive integer multiple of the number of pre-configured training cycle rounds of the central node plus 1, and if so, the node reselection mechanism is triggered.

[0202] Figure 6 Schematic diagrams of some other embodiments of the method for selecting a federated learning node of the present disclosure. Preferably, this embodiment can be executed by the federated learning node selection system of the present disclosure. Figure 6 The embodiment provides a node update method triggered by a target distributed node in the present disclosure. The method (eg Figure 3 Step 7) of the embodiment may include at least one of steps 71a to 74, wherein: the trigger mechanism

[0203] Step 71a: The target distributed node participating in the training determines that the mean of the local data feature distribution offset value vector of this round of training is not greater than the deviation threshold.

[0204] Step 71b: The target distributed node determines that it has completed training within the expected waiting time of the central node.

[0205] Step 71c, when one of the above conditions is not met (i.e., the mean of the local data feature distribution offset value vector of this round of training is greater than the deviation threshold, or the target distribution node determines that it cannot complete training within the expected waiting time of the central node), it is determined whether the periodic trigger update condition is met; if the periodic trigger update condition is not met, execute step 1d.

[0206] In some embodiments of the present disclosure, the periodic trigger update condition is that the current global training round number is equal to a positive integer multiple of the central node preconfigured training period round number plus 1.

[0207] Step 71d: The distributed node feeds back information to the central node and triggers a node reselection mechanism.

[0208] Step 72: The central node discovers at least one candidate distributed node that can participate in the federated learning service through the information storage unit.

[0209] Step 73, according to the node selection process ( Figure 3 Step 2 to step 5c in the embodiment, Figure 1 Steps 11 to 13 of the embodiment, or Figure 2 Step 21-step 23) of the embodiment is used to select a node.

[0210] Step 74: Federated learning service request and model federated learning training.

[0211] In some embodiments of the present disclosure, step 74 may include Figure 3 The specific steps of step 6 in the embodiment.

[0212] The distributed nodes in the above-mentioned embodiment of the present disclosure judge the offset value vector, estimate the response time, and determine whether the current round meets the deviation threshold required for the current central node training, the expected waiting time, and the periodic triggering update conditions. If not, the node reselection mechanism is triggered.

[0213] The node update method proposed in the above embodiment of the present disclosure includes a node update mechanism triggered by a central node and a node update mechanism triggered by a distributed node, which reduces the transmission of redundant information in each round of training; at the same time, the above embodiment of the present disclosure adds a distributed node current training round judgment mechanism to avoid the simultaneous triggering of the two node update mechanisms (central node triggering mechanism and distributed node triggering mechanism).

[0214] The node update method selected in the above embodiment of the present disclosure further optimizes the node reselection trigger mechanism in the federated learning process based on the judgment criteria and the interaction mechanism.

[0215] The above-mentioned embodiments of the present disclosure propose a node update mechanism triggered by a central node and a node update mechanism triggered by a distributed node, that is, the central node triggers node reselection after each certain training round; and the distributed node triggers node reselection when conditions are insufficient, thereby eliminating the need to transmit relevant information in each round of training.

[0216] The above-mentioned embodiments of the present disclosure propose a distributed node triggering node update mechanism, in which the distributed node is capable of triggering node reselection only when the current training round does not meet the periodic trigger update conditions, thereby avoiding the situation where both the central node and the distributed node trigger node reselection at the same time.

[0217] Figure 7 Schematic diagram of some embodiments of the distributed nodes disclosed in the present invention. Figure 7 As shown, the distributed node of the present disclosure may include an information receiving module 100, a data distribution determining module 200 and a data distribution sending module 300, wherein:

[0218] The information receiving module 100 is configured to receive subscription information or request information sent by the central node.

[0219] In some embodiments of the present disclosure, the subscription information or request information includes at least one of a number of training rounds, a training cycle, a deviation threshold, and an expected waiting time.

[0220] In some embodiments of the present disclosure, the distributed nodes of the present disclosure may also be configured to register node basic information in an information storage unit, wherein the node basic information is used for the central node to discover candidate distributed nodes through the information storage unit, and the node basic information includes at least one of node reachability, federated learning capability, and analysis identifier, the node reachability is used to indicate whether the central node can establish a connection with the distributed node, the federated learning capability is used to indicate whether the node supports federated learning, and the analysis identifier is used to indicate the type of analysis service.

[0221] The data distribution determination module 200 is configured to determine data feature distribution information local to a candidate distributed node, wherein the candidate distributed node is a distributed node that can participate in the federated learning service.

[0222] In some embodiments of the present disclosure, the data distribution determination module 200 can be configured to generate a mean vector and an offset value vector based on the local data feature distribution of the candidate distributed node; determine whether the received subscription information or request information matches the candidate distributed node configuration information to determine whether to join federated learning; and if it is determined to join federated learning, instruct the data distribution sending module 300 to execute the operation of sending the data feature distribution information to the central node.

[0223] In some embodiments of the present disclosure, the data distribution determination module 200, when generating a mean vector and an offset value vector according to the local data feature distribution of the candidate distributed node, can be configured to generate a mean vector according to the local data feature distribution, wherein each element in the mean vector is the mean of the data feature distribution obtained by the specified number of rounds of federated learning training before the current round, and the specified number of rounds is determined by the number of training rounds preconfigured by the central node; and determine the offset value vector according to the mean vector, wherein each element in the offset value vector is the directional deviation between the data feature distribution of the specified number of rounds of federated learning training before the current round and the overall distribution.

[0224] In some embodiments of the present disclosure, the data distribution determination module 200, when determining whether the received subscription information or request information matches the candidate distributed node configuration information to determine whether to join federated learning, can be configured to perform at least one of the following operations to determine whether to join federated learning, wherein: the operations include: determining whether the mean of the offset value vector is greater than the deviation threshold; determining whether the candidate distributed node itself can complete training within the expected waiting time of the central node; determining whether the candidate distributed node itself supports the analysis type indicated by the central node.

[0225] The data distribution sending module 300 is configured to send the data feature distribution information to the central node, wherein the data feature distribution information is used for the central node to select a target distributed node from candidate distributed nodes.

[0226] In some embodiments of the present disclosure, the data distribution sending module 300 can be configured to send response information or notification information to the central node, wherein the response information or notification information includes data feature distribution information, the data feature distribution information includes a mean vector and an offset value vector, and the response information or notification information also includes at least one of a candidate distributed node identifier, an estimated response time of the candidate distributed node, computing resources of the candidate distributed node, and a communication link quality from the candidate distributed node to the central node.

[0227] In some embodiments of the present disclosure, the distributed node may also be configured to receive a federated learning service request sent by the central node after sending a response message or a notification message to the central node, wherein the federated learning service request is a federated learning service request sent by the central node to a selected target distributed node; participate in completing the model training and feedback the training results or analysis results to the central node, wherein the training results or analysis results are used for the central node to aggregate the training results and analysis results, and the central node is used to send the aggregated results to the target distributed node for the next round of training.

[0228] The node selection scheme based on data feature distribution proposed in the above-mentioned embodiments of the present disclosure reports the local data feature distribution information by distributed nodes, without the need for node selection after training, thereby reducing communication and training costs.

[0229] In some embodiments of the present disclosure, node updates may be periodically triggered by a central node; and node updates may be triggered by a target distributed node.

[0230] In some embodiments of the present disclosure, the distributed node can also be configured to send feedback information to the central node when at least one training condition is not met, triggering the node reselection mechanism, triggering the central node to discover at least one candidate distributed node that can participate in the federated learning service through the information storage unit, and then executing the steps of receiving subscription information or request information sent by the central node, determining the local data feature distribution information of the candidate distributed node, and sending the data feature distribution information to the central node, wherein the training conditions include that the mean of the local data feature distribution offset value vector is not greater than the deviation threshold, and the target distributed node completes the training within the expected waiting time of the central node; and then executing the operations of receiving subscription information or request information sent by the central node, determining the local data feature distribution information of the candidate distributed node, and sending the data feature distribution information to the central node.

[0231] The node update scheme proposed in the above-mentioned embodiments of the present disclosure includes a node update mechanism triggered by a central node and a node update mechanism triggered by a distributed node, thereby reducing the transmission of redundant information in each round of training.

[0232] In some embodiments of the present disclosure, the operation of sending feedback information to the central node when at least one training condition is not met may include: when at least one training condition is not met, determining whether a periodic trigger update condition is met; and when the periodic trigger update condition is not met, feeding back information to the central node.

[0233] In some embodiments of the present disclosure, the periodic trigger update condition is that the current global training round number is equal to a positive integer multiple of the central node preconfigured training period round number plus 1.

[0234] The above-mentioned embodiment of the present disclosure adds a mechanism for determining the current training round of distributed nodes to avoid the simultaneous triggering of the two node update mechanisms.

[0235] Figure 8 Schematic diagram of some embodiments of the central node of the present disclosure. Figure 8 As shown, the central node of the present disclosure may include an information sending module 400, a data distribution receiving module 500 and a node selection module 600, wherein:

[0236] The information sending module 400 is configured to send subscription information or request information to the candidate distributed nodes.

[0237] In some embodiments of the present disclosure, the subscription information or request information may include at least one of the number of training rounds, training cycle, deviation threshold and expected waiting time, wherein the number of training rounds is used to indicate the number of federated learning training rounds required by the central node for calculating the mean vector and offset value vector of the candidate distributed nodes, the training cycle is used to indicate the number of training rounds required for the federated learning training, the deviation threshold is used to indicate the central node's requirement for the stability of the data feature distribution in the candidate distributed nodes participating in this round of federated learning training over time, and the expected waiting time is used to indicate the maximum time the central node will wait for each candidate distributed node to report the training results or analysis results during a round of federated learning training.

[0238] In some embodiments of the present disclosure, the central node may also be configured to discover candidate distributed nodes through an information storage unit, wherein the information storage unit is used to store basic information of registered nodes registered in the information storage unit, and the basic information of the nodes includes at least one of node reachability, federated learning capability and analysis identifier, the node reachability is used to indicate whether the central node can establish a connection with the distributed node, the federated learning capability is used to indicate whether the node supports federated learning, and the analysis identifier is used to indicate the type of analysis service.

[0239] The data distribution receiving module 500 is configured to receive data feature distribution information sent by the candidate distributed node, wherein the data feature distribution information is the local data feature distribution information of the candidate distributed node determined by the candidate distributed node.

[0240] In some embodiments of the present disclosure, the data distribution receiving module 500 can be configured to receive response information or notification information sent by a candidate distributed node, wherein the response information or notification information includes data feature distribution information, the data feature distribution information includes a mean vector and an offset value vector, and the response information or notification information also includes at least one of the candidate distributed node identifier, the candidate distributed node estimated response time, the candidate distributed node computing resources, and the communication link quality from the candidate distributed node to the central node.

[0241] The node selection module 600 is configured to select a target distributed node from the candidate distributed nodes according to the data feature distribution information.

[0242] In some embodiments of the present disclosure, the node selection module 600 may be configured to select a node as a target distributed node from the candidate distributed nodes that choose to join the federated learning training based on the response information or notification information sent by the candidate distributed nodes.

[0243] In some embodiments of the present disclosure, the node selection module 600 can be configured to perform node selection based on the mean vector of each candidate distributed node, and use the ratio of the global mean vector to the expected response time as the convergence rate; select a node subset with the largest convergence rate in the direction of the global mean from the candidate distributed nodes, and use the nodes in the node subset as the target distributed nodes.

[0244] In some embodiments of the present disclosure, the node selection module 600 can also be configured to estimate the communication rate under a communication resource block based on the quality of the communication link from the target distributed node to the central node; optimize the allocation of communication resources (such as the number of communication resource blocks) based on the data size of the model parameters to be trained and the expected response time of the distributed nodes, and balance the time for each target distributed node participating in the training to complete a federated learning training.

[0245] In some embodiments of the present disclosure, the central node may also be configured to send a federated learning service request to a selected target distributed node; receive training results or analysis results fed back by the target distributed node after completing model training; aggregate the training results or analysis results, and send the aggregated results to the target distributed node for the next round of training.

[0246] In some embodiments of the present disclosure, node updates are periodically triggered by a central node; and node updates are triggered by target distributed nodes.

[0247] In some embodiments of the present disclosure, the central node may also be configured to determine whether a periodic trigger update condition is met; if the periodic trigger update condition is met, trigger the node reselection mechanism, triggering the central node to discover at least one candidate distributed node that can participate in the federated learning service through the information storage unit; and then execute the operation of receiving subscription information or request information sent by the central node, determining the local data feature distribution information of the candidate distributed node, and sending the data feature distribution information to the central node.

[0248] In some embodiments of the present disclosure, the periodic trigger update condition is that the current global training round number is equal to a positive integer multiple of the central node preconfigured training period round number plus 1.

[0249] Fig. 9 The following is a schematic diagram of the structure of some embodiments of the federated learning node selection device disclosed in the present invention. Fig. 9 As shown, the federated learning node selection device includes a memory 91 and a processor 92.

[0250] The memory 91 is used to store instructions. The processor 92 is coupled to the memory 91. The processor 92 is configured to execute the above-mentioned embodiments (for example, Figure 1-Figure 6 The method for selecting a federated learning node as described in any embodiment.

[0251] like Fig. 9 As shown, the federated learning node selection device also includes a communication interface 93 for information exchange with other devices. At the same time, the federated learning node selection device also includes a bus 94, through which the processor 92, the communication interface 93, and the memory 91 communicate with each other.

[0252] The memory 91 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory. The memory 91 may also be a memory array. The memory 91 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules.

[0253] In addition, the processor 92 may be a central processing unit (CPU), or may be an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.

[0254] Figure 3 , Figure 5 and Figure 6 The embodiment also provides schematic diagrams of some embodiments of the federated learning node selection system disclosed in the present invention. Figure 3 , Figure 5 or Figure 6 As shown in the embodiment, the federated learning node selection system disclosed in the present invention may include a distributed node 1 and a central node 2, wherein:

[0255] The distributed node 1 may be any of the above embodiments (for example Figure 7 The distributed node described in embodiment).

[0256] The central node 2 may be any of the above embodiments (for example Figure 8 The central node described in embodiment).

[0257] like Figure 3 , Figure 5 or Figure 6 As shown in the embodiment, the federated learning node selection system disclosed in the present invention may include an information storage unit 3, wherein:

[0258] The information storage unit 3 can be configured to store the basic node information of the distributed node 1 to realize the registration of the distributed node 1, wherein the basic node information includes at least one of the node reachability, federated learning capability and analysis identifier, the node reachability is used to indicate whether the central node and the distributed node can establish a connection, the federated learning capability is used to indicate whether the node supports federated learning, and the analysis identifier is used to indicate the type of analysis service.

[0259] The federated learning node selection method and system disclosed in the present invention are described below through some specific embodiments.

[0260] Some embodiments of the first type

[0261] NWDAF (Network Data Analytics Function) is a network function in 5GC (5G core net), which supports collecting data from NFs, AFs and OAM (Operations, Administration and Management), and provides analysis information to NFs, AFs and OAM, where NF stands for Network Function and AF stands for Application Function. In multiple NWDAF deployment scenarios, when the analysis service requested by the NWDAF service consumer (the service is identified by the analysis ID) needs to be provided by multiple NWDAFs, multiple NWDAFs can jointly provide analysis services to consumers.

[0262] In the above scenario, the NWDAF service consumer entity serves as the central node, and multiple NWDAFs that provide services to the consumer entity serve as distributed nodes. During the node discovery process, the distributed node NWDAF forms a mean vector and an offset value vector based on the subscription / request information of the central node and its own data feature distribution, and chooses whether to join the federated learning according to the requirements of the subscription / request information, and responds / notifies the central node NWDAF; the central node NWDAF selects the distributed node NWDAF based on the selection algorithm, and reselects the node based on the node reselection trigger mechanism during the federated learning process to ensure the validity of the selected node.

[0263] For multiple NWDAF deployment scenarios, the federated learning node selection method disclosed herein may include at least one of steps 81 to 88, wherein:

[0264] Step 81, the distributed node NWDAF registers basic information (node ​​reachability, federated learning capability, analysis ID) in NRF (Network Repository Function). The NWDAF service consumer (central node NWDAF) discovers candidate distributed node NWDAF through NRF. In this scenario, NRF is an information storage unit.

[0265] Step 82: The central node NWDAF sends subscription / request information to the candidate distributed nodes NWDAF, including the preconfigured round number, deviation threshold, and expected waiting time.

[0266] Step 83, the candidate distributed node NWDAF generates a mean vector and an offset vector based on the feature distribution of various local data sets; and chooses whether to join the candidate node according to the expected waiting time and its own capabilities.

[0267] Step 84, the candidate distributed node NWDAF sends a response / notification to the NWDAF service consumer, where the notification includes the distributed node ID, the estimated response time of the distributed node, the computing resources of the distributed node, the mean vector, and the quality of the communication link from the distributed node to the central node.

[0268] Step 85, the NWDAF service consumer selects a node that is beneficial to the global model training convergence rate from the candidate distributed nodes NWDAF based on the response / notification information and the selection algorithm;

[0269] Step 86: The NWDAF service consumer sends a federated learning service request message to the selected target distributed node NWDAF and performs model training.

[0270] Step 87, during the federated learning process, when the target distributed node NWDAF finds that it cannot meet the demand, it triggers a reselection mechanism; when the central node NWDAF finds that the current training round has reached the pre-configured periodic round, it triggers a reselection mechanism.

[0271] In step 88, the NWDAF service consumer obtains the selected node based on the selection algorithm, and performs model training in the next round of federated learning training and reselects the distributed node NWDAF.

[0272] Some embodiments of the second type

[0273] MDAS (Management Data Analysis Service) provides a management domain data analysis service.

[0274] Similar to some embodiments of the first type, in a scenario where multiple MDASs are deployed, the MDAS service consumer is used as a central node, and multiple MDASs that provide services to the consumer are used as distributed nodes.

[0275] Some examples of the third type

[0276] MDAS can provide analysis results for multiple gNBs (next generation NodeBase station). MDAS is used as the central node and multiple gNBs are used as distributed nodes.

[0277] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, any of the above embodiments (for example Figure 1-Figure 6 The method for selecting a federated learning node as described in any embodiment.

[0278] In some embodiments of the present disclosure, the computer-readable storage medium may be a non-transitory computer-readable storage medium.

[0279] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, devices, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0280] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0281] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0282] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0283] The federated learning node selection device, distributed node, information receiving module, data distribution determination module, data distribution sending module, central node, information sending module, data distribution receiving module and node selection module described above can be implemented as a general-purpose processor, programmable logic controller (PLC), digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component or any appropriate combination thereof for performing the functions described in this application.

[0284] So far, the present disclosure has been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.

[0285] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing the relevant hardware through a program, and the program may be stored in a non-transitory computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0286] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A method for selecting nodes in federated learning. include: Receive subscription information or request information sent by the central node; Determine data feature distribution information of a local candidate distributed node, wherein the candidate distributed node is a distributed node that can participate in the federated learning service; The data feature distribution information is sent to the central node, wherein the data feature distribution information is used for the central node to select a target distributed node from candidate distributed nodes.

2. The method for selecting a federated learning node according to claim 1, in, The subscription information or request information includes at least one of a number of training rounds, a training cycle, a deviation threshold, and an expected waiting time; The method of determining the local data feature distribution information of the candidate distributed node includes: generating a mean vector and an offset value vector based on the local data feature distribution; judging whether the received subscription information or request information matches the candidate distributed node configuration information to determine whether to join the federated learning; and if it is determined to join the federated learning, executing the step of sending the data feature distribution information to the central node.

3. The method for selecting a federated learning node according to claim 2, in, The determining whether the received subscription information or request information matches the distributed node configuration information includes at least one of the following steps, wherein: Determine whether the mean of the offset value vector is greater than the deviation threshold; Determine whether the training can be completed within the expected waiting time of the central node; Determine whether the analysis type indicated by the central node is supported.

4. The method for selecting a federated learning node according to claim 2, in, Generating a mean vector and an offset value vector according to the local data feature distribution includes: Generate a mean vector based on the local data feature distribution, where each element in the mean vector is the mean of the data feature distribution obtained by the specified number of rounds of federated learning training before the current round. The specified number of rounds is determined by the number of training rounds preconfigured by the central node. An offset value vector is determined according to the mean value vector, wherein each element in the offset value vector is a directional deviation between a data feature distribution of a specified number of rounds of federated learning training before the current round and an overall distribution.

5. The method for selecting a federated learning node according to any one of claims 1 to 4, in, The sending the data feature distribution information to the central node comprises: Send response information or notification information to the central node, wherein the response information or notification information includes data feature distribution information, the data feature distribution information includes a mean vector and an offset value vector, and the response information or notification information also includes at least one of a candidate distributed node identifier, an estimated response time of the candidate distributed node, computing resources of the candidate distributed node, and a quality of a communication link from the candidate distributed node to the central node.

6. The method for selecting a federated learning node according to any one of claims 1 to 4, further comprising: include: Register the node basic information in the information storage unit, wherein the node basic information is used for the central node to discover candidate distributed nodes through the information storage unit, and the node basic information includes at least one of node reachability, federated learning capability and analysis identifier, the node reachability is used to indicate whether the central node can establish a connection with the distributed node, the federated learning capability is used to indicate whether the node supports federated learning, and the analysis identifier is used to indicate the type of analysis service.

7. The method for selecting a federated learning node according to any one of claims 1 to 4, further comprising: include: Receiving a federated learning service request sent by a central node, wherein the federated learning service request is a federated learning service request sent by the central node to a selected target distributed node; Participate in completing model training and feed back training results or analysis results to the central node, where the training results or analysis results are used for the central node to aggregate the training results and analysis results, and the central node is used to send the aggregated results to the target distributed nodes for the next round of training.

8. The method for selecting a federated learning node according to any one of claims 1 to 4, in, In the federated learning process, the federated learning node selection method further includes at least one of the following steps, wherein: Periodically trigger node updates through the central node; Trigger node updates via target distributed nodes.

9. The method for selecting a federated learning node according to claim 8, in, The triggering of node update by the target distributed node includes: When at least one training condition is not met, feedback information is sent to the central node to trigger the node reselection mechanism, which triggers the central node to discover at least one candidate distributed node that can participate in the federated learning service through the information storage unit, and then execute the steps of receiving subscription information or request information sent by the central node, determining the local data feature distribution information of the candidate distributed node, and sending the data feature distribution information to the central node, wherein the training conditions include that the mean of the local data feature distribution offset value vector is not greater than the deviation threshold, and the target distributed node completes the training within the expected waiting time of the central node.

10. The method for selecting a federated learning node according to claim 9, in, When at least one training condition is not met, sending feedback information to the central node includes: If at least one training condition is not met, then determine whether the periodic trigger update condition is met; When the periodic trigger update conditions are not met, information is fed back to the central node.

11. The method for selecting a federated learning node according to claim 10, in, The periodic trigger update condition is that the current global training round number is equal to a positive integer multiple of the central node pre-configured training period round number plus 1.

12. A method for selecting nodes in federated learning. include: Send subscription information or request information to candidate distributed nodes; Receiving data feature distribution information sent by the candidate distributed node, wherein the data feature distribution information is data feature distribution information of the candidate distributed node local determined by the candidate distributed node; A target distributed node is selected from the candidate distributed nodes according to the data feature distribution information.

13. The method for selecting a federated learning node according to claim 12, in, The subscription information or request information includes at least one of the number of training rounds, training cycle, deviation threshold and expected waiting time, wherein the number of training rounds is used to indicate the number of federated learning training rounds required by the central node for calculating the mean vector and the offset value vector for the candidate distributed nodes, the training cycle is used to indicate the number of training rounds required for the federated learning training, the deviation threshold is used to indicate the central node's requirement for the stability of the data feature distribution in the candidate distributed nodes participating in this round of federated learning training over time, and the expected waiting time is used to indicate the maximum time the central node will wait for each candidate distributed node to report the training results or analysis results during a round of federated learning training.

14. The method for selecting a federated learning node according to claim 12 or 13, further comprising: include: The candidate distributed nodes are discovered through an information storage unit, wherein the information storage unit is used to store basic information of the registered nodes registered in the information storage unit, and the basic information of the nodes includes at least one of node reachability, federated learning capability and analysis identifier, the node reachability is used to indicate whether the central node and the distributed node can establish a connection, the federated learning capability is used to indicate whether the node supports federated learning, and the analysis identifier is used to indicate the type of analysis service.

15. The method for selecting a federated learning node according to claim 12 or 13, in: The receiving of data feature distribution information sent by the candidate distributed node includes: receiving response information or notification information sent by the candidate distributed node, wherein the response information or notification information includes data feature distribution information, the data feature distribution information includes a mean vector and an offset value vector, and the response information or notification information also includes at least one of a distributed node identifier, an estimated response time of the distributed node, computing resources of the distributed node, and a communication link quality from the distributed node to the central node; The selecting a target distributed node from candidate distributed nodes according to the data feature distribution information includes: selecting a node as a target distributed node from candidate distributed nodes selected to join the federated learning training according to response information or notification information sent by the candidate distributed node.

16. The method for selecting a federated learning node according to claim 15, in, The step of selecting a node as a target distributed node from the candidate distributed nodes selected to join the federated learning training according to the response information or notification information sent by the candidate distributed node includes: Node selection is performed based on the mean vector of each candidate distributed node, and the ratio of the global mean vector to the expected response time is used as the convergence rate; A node subset with the largest convergence rate in the direction of the global mean is selected from the candidate distributed nodes, and the nodes in the node subset are used as target distributed nodes.

17. The method for selecting a federated learning node according to claim 16, in, The selecting a node as a target distributed node from the candidate distributed nodes selected to join the federated learning training according to the response information or notification information sent by the candidate distributed node also includes: According to the quality of the communication link from the target distributed node to the central node, the communication rate under one communication resource block is estimated; Based on the data size of the model parameters to be trained and the expected response time of the distributed nodes, the communication resources are optimally allocated to balance the time it takes for each target distributed node participating in the training to complete a federated learning training.

18. The method for selecting a federated learning node according to claim 12 or 13, further comprising: include: Send a federated learning service request to the selected target distributed node; Receive training results or analysis results fed back by the target distributed node after completing model training; The training results or analysis results are aggregated, and the aggregated results are sent to the target distributed node for the next round of training.

19. The method for selecting a federated learning node according to claim 12 or 13, in, In the federated learning process, the federated learning node selection method further includes at least one of the following steps, wherein: Periodically trigger node updates through the central node; Trigger node updates via target distributed nodes.

20. The method for selecting a federated learning node according to claim 19, in, The periodic triggering of node update by the central node includes: The central node determines whether the periodic trigger update conditions are met; When the periodic trigger update conditions are met, the node reselection mechanism is triggered, and the central node is triggered to discover at least one candidate distributed node that can participate in the federated learning service through the information storage unit, and then execute the steps of receiving subscription information or request information sent by the central node, determining the local data feature distribution information of the candidate distributed node, and sending the data feature distribution information to the central node.

21. The method for selecting a federated learning node according to claim 20, in, The periodic trigger update condition is that the current global training round number is equal to a positive integer multiple of the central node pre-configured training period round number plus 1.

22. A distributed node, include: An information receiving module, configured to receive subscription information or request information sent by a central node; A data distribution determination module, configured to determine local data feature distribution information of a candidate distributed node; The data distribution sending module is configured to send the data feature distribution information to the central node, wherein the data feature distribution information is used for the central node to select a target distributed node from the candidate distributed nodes.

23. A central node, include: An information sending module, configured to send subscription information or request information to the candidate distributed nodes; A data distribution receiving module is configured to receive data feature distribution information sent by a candidate distributed node, wherein the data feature distribution information is data feature distribution information of a local distributed node determined by the distributed node; The node selection module is configured to select a target distributed node from the candidate distributed nodes according to the data feature distribution information.

24. A federated learning node selection device, include: a memory configured to store instructions; The processor is configured to execute the instruction so that the federated learning node selection device performs the operation of implementing the federated learning node selection method as described in any one of claims 1-21.

25. A federated learning node selection system, comprising the distributed node as claimed in claim 22 and the central node as claimed in claim 23.

26. A computer-readable storage medium, in, The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the federated learning node selection method according to any one of claims 1 to 21 is implemented.

Images