A conflict-aware multi-server federated learning client scheduling method and device

Abstract

The application provides a conflict-aware multi-server federated learning client scheduling method and device, relates to the technical field of client scheduling, and constructs a decentralized reinforcement learning framework, each server serving as an independent intelligent agent, and dynamically selecting clients based on local observation information; the history sequence of the selected clients is completed by introducing a hidden Markov model, the conflict risk probability of each client in the current round is estimated in real time, and the probability is embedded in the intelligent agent state space to guide the intelligent agent to preferentially select clients with low communication delay and small conflict risk. In order to further alleviate the unfair situation caused by the over-frequent selection of high-performance clients, a fairness-aware reward mechanism is designed, the communication delay, conflict penalty and client participation balance are weighted and fused, and the server is encouraged to improve the system throughput while ensuring long-term training fairness.

Description

Technical Field

[0001] This invention relates to the field of client scheduling technology, and specifically to a conflict-aware multi-server federated learning client scheduling method and apparatus. Background Technology

[0002] Against the backdrop of the rapid development of federated learning technology, distributed model training has become an important path to resolve the contradiction between data privacy and computational efficiency. Traditional federated learning architectures typically adopt a single-server model, where all clients interact with only one central server. After local training is completed, the model parameters are uploaded, and the central server performs global aggregation. This model exposes structural defects such as severe communication bottlenecks at the central node, high risk of single point of failure, and frequent network congestion when the number of clients increases or their geographical distribution becomes widespread.

[0003] To alleviate these issues, the industry has successively proposed hierarchical federated learning and clustered federated learning. The former incorporates a multi-layered architecture of edge servers or cloud servers, forming a three-tiered "client-edge-cloud" topology, reducing direct traffic to the cloud through regional aggregation. Specifically, the client first sends model updates to the edge server, which aggregates the client models it covers to generate regional models. These regional models are then sent to the cloud server for final aggregation to generate the global model. The latter dynamically divides clients into clusters based on data distribution or network location. Each cluster is trained independently before submitting its cluster model to the central server. Specifically, clients are divided into different clusters, each training a model independently. The cluster center aggregates models from clients within its cluster to generate a cluster model, which is then sent to the central server for final aggregation to generate the global model.

[0004] While these solutions reduce cross-network transmission volume to some extent, they introduce new sources of latency: the secondary convergence link between the edge and cloud in the layered architecture still faces bandwidth constraints, and the increased layers lead to a sharp increase in system deployment and fault tolerance complexity; the clustering architecture requires periodic recalculation of cluster partitioning to adapt to non-independent identically distributed (Non-IID) data drift, and the re-clustering process itself introduces additional communication rounds. Simultaneously, independent training between clusters prevents the global model from fully utilizing cross-cluster information, limiting generalization performance. More critically, when the same geographical area is overlapped by multiple edge or cloud servers, each server may simultaneously select the same client in each iteration, forcing that client to download multiple initial models in parallel and execute multiple training tasks locally. This results in resource conflicts such as computational resource contention, memory overflow, training timeouts, and upload failures. Such conflicts not only directly reduce the model update success rate and lengthen the overall convergence time, but also cause some clients to be unable to receive any server calls for extended periods, resulting in continuously idle data and further weakening model accuracy and fairness.

[0005] In view of the above, this application is hereby submitted. Summary of the Invention

[0006] This invention provides a conflict-aware multi-server federated learning client scheduling method and apparatus, which can at least partially improve the above-mentioned problems.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A conflict-aware multi-server federated learning client scheduling method, comprising:

[0009] For each server, a corresponding intelligent agent is generated, and the communication time is obtained according to the surrounding environment. The state of each intelligent agent is composed of the communication time and the probability value of the conflict risk of each client.

[0010] The missing historical working sequences of the client are filled in to obtain the complete historical working sequences. The complete historical working sequences are then used to make predictions and obtain the probability values ​​of conflict risks.

[0011] The state of the agent is determined based on the communication time and the probability value of the conflict risk, and a client with low communication latency and low conflict risk is selected.

[0012] The present invention also provides a conflict-aware multi-server federated learning client scheduling device, comprising:

[0013] The agent generation unit is used to generate a corresponding agent for each server and obtain the communication time according to the surrounding environment. The state of each agent is composed of the communication time and the probability value of the conflict risk of each client.

[0014] The probability value calculation unit is used to complete the missing historical working sequences of the client to obtain the complete historical working sequences. The conflict prediction network is then used to predict the complete historical working sequences to obtain the conflict risk probability value.

[0015] The client selection unit is used to determine the state of the agent based on the communication time and the probability value of the conflict risk, and to select a client with low communication latency and low conflict risk.

[0016] In simple terms, this method is used for client selection in multi-server federated learning. Each server employs conflict risk prediction, utilizing a classification Hidden Markov Model to complete outdated historical sequences of clients and estimating their potential conflict risk based on their selection sequences. Subsequently, each server independently selects clients using deep reinforcement learning, relying only on local observations. A novel fairness-aware reward mechanism is also designed to encourage continued client participation, thereby improving overall training performance.

[0017] In summary, this invention addresses the core pain points of multi-server federated learning scenarios, such as computational conflicts, increased communication latency, and lack of training fairness caused by repeated client selection. It proposes a complete client scheduling framework based on "conflict risk prediction—decentralized reinforcement decision-making—fairness-aware reward." By deploying a conflict probability estimation module based on a Hidden Markov Model locally on each server, it real-time completes and predicts the probability of client preemption. The quantified conflict risk value and communication latency together constitute the reinforcement learning state space, enabling each server to independently make low-conflict, low-latency client selections without central coordination. Simultaneously, it innovatively... The reward function incorporates a fairness factor representing the balance of participation, dynamically balancing system efficiency with the long-term balance of client scheduling. This effectively suppresses the unfairness of oversampling high-resource nodes and significantly improves the model's utilization and generalization ability for long-tail data. The scheme is completely decentralized, possessing good horizontal scalability and topological robustness. In edge intelligence environments with limited bandwidth, heterogeneous node capabilities, and overlapping coverage areas, it simultaneously achieves multiple benefits such as reduced communication overhead, lower conflict failure rate, faster convergence speed, and improved model accuracy. This provides a general and easily deployable solution for building efficient, fair, and scalable multi-server federated learning systems. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the conflict-aware multi-server federated learning client scheduling method provided in the first embodiment of the present invention.

[0019] Figure 2 This is a scenario diagram of the multi-server federated learning system provided in an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of a conflict-aware multi-server federated learning client scheduling device provided in the second embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0022] In multi-server federated learning, communication time between clients and servers, frequent conflicts between clients, uneven client selection, and limited communication resources are the main bottlenecks in current applications. In multi-server federated learning, when the same client within a simultaneously selected region is involved, a client may receive models from different servers at the same time, requiring simultaneous training of multiple models. This leads to computational timeouts, inability to complete the required model training, conflicts, and failure to train data from some clients, resulting in training failures, limited model accuracy, and increased communication latency. Furthermore, some clients may remain unselected for extended periods, impacting model performance. This invention aims to design an efficient and universal decentralized multi-server federated learning client selection framework. Through multi-level optimization, it significantly reduces communication time while maximizing model accuracy, ensuring the performance of some client training sessions and fairness in client selection across all clients.

[0023] refer to Figure 1 , Figure 2 As shown, the first embodiment of the present invention discloses a conflict-aware multi-server federated learning client scheduling method, which can be executed by a conflict-aware multi-server federated learning client scheduling device (hereinafter referred to as the scheduling device), specifically, by one or more processors within the scheduling device, to implement the following method:

[0024] S1. For each server, a corresponding intelligent agent is generated, and the communication time is obtained according to the surrounding environment. The state of each intelligent agent is composed of the communication time and the probability value of the conflict risk of each client.

[0025] Preferably, the communication time is determined by the communication distance and channel conditions.

[0026] Specifically, in this embodiment, a multi-server federated learning client selection framework based on a decentralized architecture is designed, applying reinforcement learning methods to the client selection task. First, an independent agent is instantiated for each server. This agent's state is a vector composed of "communication time" and "conflict risk probability values ​​of each client." The communication time is obtained first, where it is determined by communication distance and channel conditions; while the conflict risk is estimated by a dedicated conflict prediction network. Therefore, the agent can select clients with lower communication latency and lower conflict risk based on its own state information, thereby effectively optimizing the overall communication efficiency of the system.

[0027] S2, complete the missing historical working sequences for the client to obtain the complete historical working sequences, and use the conflict prediction network to predict the complete historical working sequences to obtain the conflict risk probability value;

[0028] Specifically, step S2 further includes:

[0029] Obtain the client's historical work sequence and determine if it is missing. In the historical work sequence, 0 represents that it was not selected by any agent, and 1 represents that it was selected to participate in training.

[0030] If not, the input to the seat conflict prediction network for the historical work sequence that is not missing is processed to obtain the conflict risk probability value. ;

[0031] If so, a Hidden Markov Model is used to complete the missing historical work sequences, resulting in a complete historical work sequence. Then, a conflict prediction network is used to predict the probability of conflict occurring in the current round for the client. ;

[0032] The formula for the Hidden Markov Model is as follows: , Let be the probability that client i observes a value of 1 at time 1. Let be the probability that the observation result of client i at time t is 1. Let be the observation value at time t. Let d be the observation sequence of client i from 0 to td, where d is the delay step during prediction. Let A be the hidden state distribution vector at time t-1, and let A be the hidden state transition probability matrix. Let be the emission probability sequence where the observation is 1 in each hidden state.

[0033] Specifically, in this embodiment, the input information of the conflict prediction network comes from partially observable data. The relevant information of a client can only be obtained when the server actually selects it. This includes its historical working sequence, where 1 represents not being selected by any agent and 1 represents being selected for training. Since some clients are not selected in the current round, the observation data is incomplete, resulting in missing historical working sequences. If a missing historical working sequence is found, it is fed into a Hidden Markov Model (HMM) for completion. Specifically, the hidden state distribution is estimated using a forward algorithm, and then the most likely symbol at the missing position is recursively calculated using the transition matrix and emission probability vector, thus piecing together the broken 0 / 1 string into a continuous and complete historical working sequence (including the actual observations and the HMM-completed part). The completed sequence retains the original "idle-selected" alternation pattern, enabling the conflict prediction network to accurately identify clients that have just ended a long idle period or are currently in a continuous selection period during subsequent inference, thereby outputting a more reliable conflict risk probability value.

[0034] The completed historical work sequence is input into the conflict prediction network, which outputs the probability of each client clashing in the current round. This predicted probability is incorporated into the agent's state representation to help it avoid clients with a high risk of conflict during client selection. Alternatively, if no historical work sequence is missing, it can be directly input into the conflict prediction network, which outputs the probability of each client clashing in the current round.

[0035] S3, determine the state of the agent based on the communication time and the probability value of the conflict risk, and select a client with low communication latency and low conflict risk.

[0036] Preferably, the state of the intelligent agent It is the input to the agent's policy network, and its formula is: , For communication time, This represents the probability value of conflict risk.

[0037] Specifically, in this embodiment, after collecting communication time and conflict risk probability values, the agent state is immediately constructed locally. Based on this state, the policy network prioritizes clients with low communication latency and low conflict risk. Since the state consists of only these two items, the policy network can complete inference locally without additional features, and each server makes independent decisions without relying on a central node. By continuously favoring "fast and safe" clients, the system effectively reduces the situation where multiple servers simultaneously compete for the same terminal, lowers the probability of training failure, shortens air interface waiting time, and enables the overall scheduling to achieve a smoother convergence experience while maintaining decentralized characteristics.

[0038] Preferably, the method further includes: judging the quality of the selected client based on a fairness-aware reward mechanism, calculating a reward value according to a reward function, and adjusting and updating the agent's policy network based on the reward value to obtain a new policy output. , For client 1, For client 2, For client s;

[0039] In a fairness-aware reward mechanism, a reward function is used to evaluate the agent's policy network to determine the quality of the agent's current client selection. The reward function... Communication time Conflict punishment And the fairness factor f is composed of the following: , , For client communication latency, These are weighting coefficients used to adjust the fairness factor. This represents the average level of client engagement. The standard deviation of client engagement. To prevent small constants with a denominator of 0.

[0040] The communication time is used to characterize the latency overhead of the agent when completing the task. The longer the communication time, the lower the reward. The conflict penalty is an additional penalty introduced when multiple agents simultaneously select the same client or resource. The fairness factor is used to characterize the balance of client participation by calculating the ratio of the mean to the standard deviation of the number of times all clients are selected. The more fair the client participation, the larger the value of this factor.

[0041] The agent's policy network is adjusted and updated based on the reward value, specifically by: judging whether the reward value is lower than a preset threshold.

[0042] If so, the agent's policy network is adjusted and updated. When the reward value is lower than a preset threshold, it means that the selected client is poor and will be excluded in subsequent selections.

[0043] If not, the agent's policy network will not be adjusted or updated.

[0044] In this embodiment, in multi-server client selection, simply minimizing communication latency often leads to high-performance or high-resource clients being frequently selected, resulting in unfairness. Therefore, a fairness-aware reward mechanism is introduced. Specifically, after client selection, a fairness-aware reward is immediately calculated to verify the validity of the selection: the measured communication latency (i.e., communication time), conflict penalty, and fairness factor are substituted into the reward function; the longer the communication time, the lower the reward. If multiple servers compete for the same client, additional points are deducted from the conflict penalty to suppress performance degradation caused by resource competition. The fairness factor characterizes the balance of client participation by calculating the ratio of the mean to the standard deviation of the number of times all clients are selected; the more fair the client participation, the larger the factor value, thus providing additional positive incentives. By adjusting the weighting coefficients, a dynamic balance can be achieved between system efficiency (low latency, low conflict) and fairness assurance, thereby improving the long-term performance and stability of the overall system.

[0045] The process involves using reward values ​​to evaluate the agent's current choice and updating the agent's policy accordingly. If the reward is poor, the agent's policy will be adjusted. When the reward value is very poor, it indicates that the selected client is unsuitable, and the agent's policy is adjusted. Based on the adjusted policy, the agent will not choose this client again in the next selection.

[0046] In summary, this invention proposes a client selection method for multi-server federated learning by combining reinforcement learning, conflict prediction, and fairness-aware rewards. This method effectively reduces communication overhead during federated learning while maintaining model training accuracy. It exhibits good adaptability and versatility, making it particularly suitable for resource-constrained applications such as those with low bandwidth and high latency.

[0047] Please see Figure 3 A second embodiment of the present invention provides a conflict-aware multi-server federated learning client scheduling device, comprising:

[0048] The agent generation unit 101 is used to generate a corresponding agent for each server and obtain the communication time according to the surrounding environment. The state of each agent is composed of the communication time and the probability value of the conflict risk of each client.

[0049] The probability value calculation unit 102 is used to complete the missing historical working sequence of the client to obtain the complete historical working sequence. The conflict prediction network is used to predict the complete historical working sequence to obtain the conflict risk probability value.

[0050] The client selection unit 103 is used to determine the state of the agent based on the communication time and the probability value of the conflict risk, and to select a client with low communication latency and low conflict risk.

[0051] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A conflict-aware multi-server federated learning client scheduling method, characterized in that, include: For each server, a corresponding intelligent agent is generated, and the communication time is obtained according to the surrounding environment. The state of each intelligent agent is composed of the communication time and the probability value of the conflict risk of each client. The missing historical working sequences on the client side are completed to obtain the complete historical working sequences. A conflict prediction network is then used to predict the conflict risk probability value of the complete historical working sequences. Specifically: Obtain the client's historical work sequence and determine if it is missing. In the historical work sequence, 0 represents that it was not selected by any agent, and 1 represents that it was selected to participate in training. If not, the historical work sequences that are not missing are used as input to the conflict prediction network for prediction processing to obtain conflict risk probability values. ; If so, a Hidden Markov Model is used to complete the missing historical work sequences, resulting in a complete historical work sequence. Then, a conflict prediction network is used to predict the probability of conflict occurring in the current round for the client. ; The formula for the Hidden Markov Model is as follows: , Let be the probability that client i observes a value of 1 at time 1. Let be the probability that the observation result of client i at time t is 1. Let be the observation value at time t. Let d be the observation sequence of client i from 0 to td, where d is the delay step during prediction. Let A be the hidden state distribution vector at time t-1, and let A be the hidden state transition probability matrix. Let be the emission probability sequence where the observation is 1 in each hidden state; The state of the agent is determined based on the communication time and the probability value of the conflict risk, and a client with low communication latency and low conflict risk is selected.

2. The conflict-aware multi-server federated learning client scheduling method according to claim 1, characterized in that, The communication time is determined by the communication distance and channel conditions.

3. The conflict-aware multi-server federated learning client scheduling method according to claim 1, characterized in that, The state of the intelligent agent It is the input to the agent's policy network, and its formula is: , For communication time, This represents the probability value of conflict risk.

4. The conflict-aware multi-server federated learning client scheduling method according to claim 1, characterized in that, Also includes: The agent uses a fairness-aware reward mechanism to judge the quality of selected clients, calculates reward values ​​based on a reward function, and adjusts and updates the agent's policy network based on these reward values ​​to obtain a new policy output. , For client 1, For client 2, For client s; In a fairness-aware reward mechanism, a reward function is used to evaluate the agent's policy network to determine the quality of the agent's current client selection. The reward function... Communication time Conflict punishment And the fairness factor f is composed of the following: , , For communication time, These are weighting coefficients used to adjust the fairness factor. This represents the average level of client engagement. The standard deviation of client engagement. To prevent small constants with a denominator of 0.

5. The conflict-aware multi-server federated learning client scheduling method according to claim 4, characterized in that, The communication time is used to characterize the latency overhead of the agent when completing the task. The longer the communication time, the lower the reward. The conflict penalty is an additional penalty introduced when multiple agents simultaneously select the same client or resource. The fairness factor is used to characterize the balance of client participation by calculating the ratio of the mean to the standard deviation of the number of times all clients are selected. The more fair the client participation, the larger the value of this factor.

6. The conflict-aware multi-server federated learning client scheduling method according to claim 4, characterized in that, The agent's policy network is adjusted and updated based on the reward value, specifically as follows: The reward value is evaluated to determine whether it is lower than a preset threshold. If so, the agent's policy network is adjusted and updated. When the reward value is lower than a preset threshold, it means that the selected client is poor and will be excluded in subsequent selections. If not, the agent's policy network will not be adjusted or updated.

7. A conflict-aware multi-server federated learning client scheduling device, characterized in that, include: The agent generation unit is used to generate a corresponding agent for each server and obtain the communication time according to the surrounding environment. The state of each agent is composed of the communication time and the probability value of the conflict risk of each client. The probability value calculation unit is used to complete the missing historical working sequences for the client, obtaining a complete historical working sequence. A conflict prediction network is then used to predict the complete historical working sequence to obtain the conflict risk probability value, specifically: Obtain the client's historical work sequence and determine if it is missing. In the historical work sequence, 0 represents that it was not selected by any agent, and 1 represents that it was selected to participate in training. If not, the historical work sequences that are not missing are used as input to the conflict prediction network for prediction processing to obtain conflict risk probability values. ; If so, a Hidden Markov Model is used to complete the missing historical work sequences, resulting in a complete historical work sequence. Then, a conflict prediction network is used to predict the probability of conflict occurring in the current round for the client. ; The formula for the Hidden Markov Model is as follows: , Let be the probability that client i observes a value of 1 at time 1. Let be the probability that the observation result of client i at time t is 1. Let be the observation value at time t. Let d be the observation sequence of client i from 0 to td, where d is the delay step during prediction. Let A be the hidden state distribution vector at time t-1, and let A be the hidden state transition probability matrix. Let be the emission probability sequence where the observation is 1 in each hidden state; The client selection unit is used to determine the state of the agent based on the communication time and the probability value of the conflict risk, and to select a client with low communication latency and low conflict risk.

Images