A multi-element control decision method based on a federal split learning framework

Abstract

A multi-element control decision method based on a federal split learning framework is provided to improve model training accuracy while maximizing resource utilization utility. First, initial information of nodes is dynamically constructed in a network environment, including four types of routers, federal learning servers, clients and computing nodes (including a limited number of training servers). The federal learning server strategically selects a group of clients from the client set for training tasks. A dedicated controller in the federal learning server acts as a task initiator to decide client admission, model splitting, computing node selection, routing and bandwidth allocation. The training servers of the clients and their corresponding computing nodes perform model training tasks according to the decision. After training, the trained models are uploaded to the federal learning server for global model aggregation. The present application enables training as much data as possible within a given time, improving model accuracy while maximizing resource utilization utility.

Description

Technical Field

[0001] This invention belongs to the field of distributed machine learning, and more specifically, it relates to designing a multivariate control decision-making method based on a federated split learning framework in a heterogeneous network environment on the client side. Background Technology

[0002] Combining federated learning with split learning, federated split learning, as a promising distributed learning paradigm, offers many advantages such as data privacy, parallel training, and lightweight on-device computational requirements. However, a key challenge that existing federated split frameworks fail to consider is the heterogeneity of clients, such as varying computational and communication capabilities. This leads to the "bucket effect," resulting in excessively long training times and wasted computational and communication resources.

[0003] With the deep integration of networks and computing, computing power-aware networks have become a hot topic. They can comprehensively monitor computing and network resources, enabling mutual awareness and high-level collaboration between the two. Furthermore, ubiquitous connectivity allows for interconnection of pervasive computing, achieving efficient collaboration between cloud, edge, and network, improving the utilization efficiency of network and computing resources, and ultimately meeting business needs such as user experience quality and dynamic service deployment. It is evident that the characteristics of computing power-aware networks are particularly suitable for federated decomposition learning frameworks with heterogeneous client environments. However, there is currently almost no application or work on intelligent scheduling and efficient allocation methods and approaches for computing power resources and services.

[0004] Therefore, in a computing power-aware network environment, it is of great significance to design a multivariate control decision-making method based on a federated split learning framework for heterogeneous clients. Summary of the Invention

[0005] In response to the aforementioned research status and existing problems, this invention provides a multivariate control decision-making method based on a federated split learning framework to improve model training accuracy while maximizing resource utilization efficiency.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A multivariate control decision-making method based on a federated split learning framework includes the following steps:

[0008] (1) The dedicated controller in the federated learning server selects the deep learning model training task to be trained. At the beginning of each time period, it obtains the amount of data that each client can participate in training, the average number of times it has been admitted, the current link status information, the unit cost on the computing node, and the unit bandwidth cost on the link.

[0009] (2) Based on the amount of data that each client can participate in training and the average number of times it has been admitted before, strategically select a set of clients from all client sets to perform the model training task;

[0010] (3) For clients participating in training, construct and solve a multivariate control problem based on federated split learning, and further decide on admission status, computing node allocation, model splitting point, routing and bandwidth allocation;

[0011] (4) For clients with an admission status and the training servers on their corresponding computing nodes, the training tasks are executed according to the decision. After training, the trained models are uploaded to the federated learning server for global model aggregation.

[0012] Preferably, when a task training needs to be initiated, the federated learning server uses the Lyapunov virtual queue and related properties of queuing theory to decide whether the client should participate in the model training task in this round, based on the amount of data that the client can participate in training and the average number of times it has been admitted in the past, and calculates the model selected by the client.

[0013] Preferably, when the federated learning server constructs a multivariate control problem based on the federated split learning framework in step (3), it calculates the system overhead model based on the current link state information, the unit overhead on the computing node and the unit bandwidth overhead on the link, and defines the ratio of the client selection model to the system overhead model as the multivariate control problem based on the federated split learning framework.

[0014] Preferably, in step (3), when solving the multivariate control problem based on the federated decomposition learning framework, the Tinkelbach transform is used to transform the multivariate control problem based on the federated decomposition learning framework into a simplified linearized parameter objective function. A heuristic greedy algorithm is designed by relaxing integers to decimals, rounding, and verifying, and the solution is started. The specific steps are as follows:

[0015] (3-1) Initialize all admissible client sets and non-admissible client sets to empty, and the undecided client set initially contains all clients;

[0016] (3-2) If the set of undecided clients is not empty, proceed to step (3-3); otherwise, skip to step (3-6).

[0017] (3-3) For each client i in the set of undecided clients, the fractional solution of the decision of the computation node j and the specific path l that the client can choose is obtained by solving the multivariate control problem of the constructed federated split learning framework. Substituting the values ​​of the decimal solutions into the multivariate control problem within the federated splitting learning framework yields a descending sort of the results. ordered list

[0018] (3-4) For lists Each element in Perform the following operations according to the list order:

[0019] elements Set to 1, and the rest of the elements Set the value to 0 and use the solver to verify if a feasible solution exists; if a feasible solution exists, then set the element to 0. Add to the set of solutions In the process, add client i to the set of admissible clients, remove client i from the set of clients that have not yet made a decision, and execute steps (3-5); otherwise, undo the operations before this step and reset the elements. From list Remove from list If not empty, continue executing steps (3-4) in a loop;

[0020] (3-5) For each client i in the set of undecided clients and Not here If the client i is not allowed, add the client i to the set of clients that are not allowed to enter, and remove the client i from the set of clients that have not made a decision; then return to step (3-2);

[0021] The final solution to problem (3-6) is a set If the result is within the given tolerance range, the solution ends, and the decision result is obtained, namely the admitted client, the computing node connected to the admitted client, the model split point trained on the training server of the admitted client and its corresponding computing node, and the end-to-end bandwidth and routing from the admitted client to the training server of its corresponding computing node; otherwise, return to step (3-1) to solve again.

[0022] Preferably, a dedicated controller is deployed in the federated learning server to maintain information on clients and computing nodes within the network, as well as network status information. Based on this information, a multivariate control problem based on federated split learning is constructed, and the decision result is obtained by solving it, thereby improving model accuracy while maximizing resource utilization.

[0023] Based on the decision results obtained by the dedicated controller in the federated learning server, namely client admission, model splitting, server selection, routing, and bandwidth allocation results, the admitted clients and their corresponding training servers on the computing nodes execute model training tasks according to the decision results. After training, they upload the trained models to the federated learning server for global model aggregation.

[0024] The advantages and beneficial effects of this invention are as follows:

[0025] Because model training requires a limited training time, this invention considers the heterogeneity of clients, the diversity of model splitting points, and the number of computing nodes when constructing a multivariate control problem based on a federated splitting learning framework. Therefore, this invention can improve model training accuracy and maximize resource utilization. Attached Figure Description

[0026] Figure 1 The attached figure is a flowchart of the decision-making method of the present invention.

[0027] Figure 2 The attached figure is a flowchart of the problem-solving process of this invention.

[0028] Figure 3 The attached figure is a network architecture diagram of the present invention.

[0029] Figure 4 The attached figure is a comparison of experimental results of the method of this embodiment and other methods on the model training task.

[0030] Figure 5 The attached figure is a comparison of experimental results regarding resource utilization efficiency between the method of the present invention and other methods, (a) DenseNet model task, (b) MobileNet model task. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Please see the appendix Figure 1 , Figure 2 This invention provides a multivariate control decision-making method based on a federated split learning framework, comprising the following steps:

[0033] (1) The dedicated controller in the federated learning server selects the deep learning model training task to be trained. At the beginning of each time period, it obtains the amount of data that each client can participate in training, the average number of times it has been admitted, the current link status information, the unit cost on the computing node, and the unit bandwidth cost on the link.

[0034] (2) Based on the amount of data that each client can participate in training and the average number of times it has been admitted before, strategically select a set of clients from all client sets to perform the model training task;

[0035] (3) For clients participating in training, construct and solve a multivariate control problem based on federated split learning, and further decide on admission status, computing node allocation, model splitting point, routing and bandwidth allocation;

[0036] (4) For clients with an admission status and the training servers on their corresponding computing nodes, the training tasks are executed according to the decision. After training, the trained models are uploaded to the federated learning server for global model aggregation.

[0037] In this embodiment, for the network scenario, we consider a computing power-aware network (see appendix of the specification). Figure 3 Let it be defined as a directed graph G = {V, E}, where V = {V C V R V S}, set V C V represents the set of clients in the network. R V represents the set of routers in the network. S This represents the set of computational nodes in the network. A dedicated network slice can be provided in a computation-aware network for a federated split learning framework. Under this network slice, extended sub-slices can be provided for each specific training task. It is assumed that for a specific training task, a set of... Clients and their corresponding collections in Trusted computing nodes (such as microclouds and data centers deployed or leased by the task initiator). One client and Each computing node consists of Connections, multiple paths between each pair of client i and compute node j are used This indicates that each path consists of multiple physical network links.

[0038] In this embodiment, for each training task, similar to the working principle of many federated learning and federated split learning frameworks, each task initiator deploys a federated learning server (i.e., a parameter server) on a computing node to be responsible for global model aggregation, aiming to derive the optimal training model w. * (i.e., the model parameter vector), which is to minimize the loss function in T global rounds. Each client has its own dedicated dataset D locally. i This determines the client's weight.

[0039] In this embodiment, for each training task, unlike federated learning which has only one computing node for aggregation and federated split learning frameworks where the model has only one splitting point, in our problem, for each client... It has its own dedicated computing nodes. In each global iteration t∈{1,2,…,T}, the two modules of the split model are trained respectively. where k∈{1,2,…,K} w} represents the split point of the model, K w This represents the number of layers that model w can be split into during the training task. This refers to the part of the model trained on the client side. This represents the part of the model trained on the training server of its corresponding computing node. When K = K w This indicates that the client trains the entire model locally. Considering the privacy of the client's data, k=0 is not allowed. In other words, the data samples are uploaded to the computing node and the model is trained only on the computing node.

[0040] In this embodiment, for each global round t, the client i participating in training is represented by c. it This represents the computing power of client i. For each global round t, the computing power of client i remains constant, but may change for different global rounds. For each compute node j, multiple virtualized compute servers, such as containers, can be provided, denoted by ω. j The computing power of its training server is represented by Ω. j This indicates the number of computing servers that can be enabled. For each training task involving a training model, given a split point k, use... and These represent the computing power required for the client-side and server-side training processes, denoted by s. k This represents the amount of data transfer between the client and server for each training batch size. For each training task, the number of training iterations per global round t is ε, and the batch size is H. For each network link e∈E, B is used. e This indicates the bandwidth capacity of each link.

[0041] In this embodiment, when the dedicated controller in the federated learning server issues a task or needs to send the decision results to the client and the training server, the dedicated controller in the federated learning server constructs a multivariate control problem based on the federated split learning framework.

[0042] In this embodiment, the dedicated controller in the federated learning server maintains all node and network state information, including the global number of training rounds, the training time for each round, the client's computing power, the client's location, the amount of data possessed by the client, the computing power of the computing nodes, the location of the computing nodes, the number of dedicated training servers that the computing nodes can open, the link bandwidth, the computing power required for the training model task, the split points of the training model task, and the amount of data to be transmitted at different split points. Based on the maintained global state information, the controller solves the constructed multivariate control problem based on the federated split learning framework and sends the solution to the client and server for execution.

[0043] In this embodiment, the multivariate control decision-making method based on the federated split learning framework involves client admission, model splitting, server selection, routing, and bandwidth allocation. For admitted clients, corresponding training server resources and bandwidth are allocated; for denied clients, no action is taken.

[0044] If a portion of the clients are deemed inadmissible based on the solution results, then those clients will not be trained on the model task.

[0045] If a portion of the clients are deemed eligible for admission based on the solution results, these clients and their corresponding training servers will receive the model and related decision information multicast from the federated learning server for task training and will be routed according to the bandwidth determined by the decision.

[0046] In this embodiment, since the multivariate control decision-making process is executed only once per round and the model size is finite, during model training, for each admitted client i given the decision result, computation and communication operations will be performed in each global round. This factor dominates the overall training latency. The latency for each client-training server pair during the split model training process is defined as: The first term represents the total computation time of client i for each batch training size; the second term represents the total computation time on the training server side of the computing node j connected to client i for each batch training size; and the third term represents the total data transmission time between client i and computing node j for each batch training size. The latency of the entire task is approximated using the latency of the split model training process.

[0047] In this embodiment, the multivariate control problem based on the federated split learning framework is solved by first deciding whether a client should participate in training, and then deciding on client admission, its corresponding training server, end-to-end bandwidth and routing, in order to improve the model training accuracy within a given time while maximizing resource utilization.

[0048] In this embodiment, the multivariate control problem and decision-making algorithm based on the federated split learning framework are defined as follows:

[0049] The variables to be solved include: whether the client is admitted, represented by the 0-1 variable z; the association between the client and the compute node, represented by the 0-1 variable x; the end-to-end bandwidth of the client, represented by the positive real number variable y; and the routing path between the client and the compute node, represented by... express;

[0050] Based on the client's data volume and the average number of times they have been admitted previously, a decision is made as to whether the client participates in the model training task in this round. Define the client's model selection:

[0051]

[0052] Among them, Q i (t+1)=Q i (t)-z it +p i Q i (0) = 0, where ∑ i p i z it The larger the amount of data the client possesses, the larger pi becomes; ∑ i Q i (t)z it This indicates that the average number of times each client is selected is consistent with the given sampling probability throughout the training process; λ is a parameter used to balance the above two terms.

[0053] According to Lyapunov's virtual queue and queuing theory, when the virtual queue Q... i (t) When stable, the average service rate Not less than the average arrival rate pi. If client i has not been selected in the most recent rounds, then its corresponding Q... i (t) will continue to accumulate, so it is more likely to be selected in the next round.

[0054] Based on the task's time constraints, the computational power required by the model at different split points, and the amount of data transmitted, the system determines the training server, end-to-end bandwidth, and routing on the computing nodes connected to the client. For a given model training task, each client should utilize computing and network resources on demand to avoid significant resource waste and high rental fees. Therefore, the system overhead model is defined as follows:

[0055]

[0056] in, This indicates the number of training servers used on compute node j. Indicates the link bandwidth used. α jt β represents the unit cost on compute node j. et This represents the unit bandwidth cost on link e.

[0057] The multivariate control problem based on the federated decomposition learning framework is defined as the problem of maximizing resource utilization utility, i.e. This represents the average training utility achieved per unit of system overhead per round. In other words, the multivariate control problem is transformed into maximizing RUE, considering client admission, model splitting, server selection, end-to-end bandwidth, and routing.

[0058] The mathematical formula for this problem is as follows:

[0059] max RUE

[0060]

[0061]

[0062]

[0063] τ it ≤Δ

[0064] var z it ∈{0,1},

[0065] Wherein, constraint (1a) indicates that the training server in the compute node should be assigned to each admitted client, and constraint (1b) indicates that the maximum number of clients that the available training servers in compute node j can serve is Ω. j Constraint (1c) represents the link bandwidth capacity constraint, where This is a binary indicator function, representing whether network link e is a path between client i and compute node j. Above. Constraint (1d) represents the training latency constraint, where Δ represents the upper limit of the time required to execute the model training task in each global round initially specified by the task initiator;

[0066] In this embodiment, to simplify the problem form, considering that the multivariate control problem is independent in different global rounds, the round number t is omitted. For constraint (1a), we use... Replace z it Therefore, the problem becomes:

[0067]

[0068] stC1:

[0069] C2:

[0070] C3:

[0071] C4:

[0072] var.

[0073] In this embodiment, algorithm P0 presents two challenges: first, P0 belongs to mixed integer-decimal programming, which is difficult to solve; second, constraints C3 and C4 are non-convex constraints. Please refer to the appendix for the algorithm's solution process. Figure 2 The specific steps are as follows:

[0074] ① Linearize the fractional function in the original problem using the Tinkelbach transform. Assume the original problem is... The problem can then be transformed into Where ρ is the parameterized parameter (initialized to 0). This is achieved by using a parameterized objective function. Solving the problem iteratively Each iteration uses the constraints of the original problem and updates When the calculated parameterized objective function When the value is within the given tolerance range (set to 0.001 in this embodiment), the optimal solution {x} is obtained. * ,y * This is the optimal solution to the original problem. Using the method described above, a parameterized objective function can be used. Solve the problem The optimal solution (i.e., linearizing the fractional function).

[0075] ② Since constraints C3 and C4 involve the product of control variables, they need to be linearized. For constraint C4, assuming that client i and computation node j can complete the model training task at split point k within a given time interval, then the control variables... The minimum value is Since the end-to-end bandwidth requirement is as small as possible, minimum bandwidth is allocated. This will not affect the optimal solution to the problem. Therefore, constraints C3 and C4 can be transformed into:

[0076] C′3:

[0077] Through the above operations, the non-convex constraints are linearized, and the original problem is finally transformed into a simplified linearized parametric objective function.

[0078] Lemma 1: Given a pair of client i and compute node j, when At that time, for each separable point k, the optimal separable point k * satisfy

[0079] According to Lemma 1, when the client i and computing node j under a given task are known, the optimal split point k of the model can be obtained according to constraint C′3. * .definition Therefore, the original problem can be further simplified to:

[0080]

[0081] st

[0082]

[0083]

[0084] var

[0085] However, the problem This is a variant of the multi-item indivisible flow problem, which cannot be solved in polynomial time. Therefore, a heuristic greedy algorithm is designed by relaxing integers to decimals, rounding, and verifying the result. The specific steps are as follows:

[0086] (1) Initialize all admissible client sets and non-admissible client sets to empty. The initial state of the undecided client set includes all clients, i.e., the undecided client set A. cur A is a set of all clients that can be admitted. acc A set of clients that are not allowed access rej It is an empty set;

[0087] (2) When set A cur If the value is not empty, proceed to step (3); otherwise, proceed to step (6).

[0088] (3) For set A cur For each client i, obtain the fractional solution of the decision for the computation node j and the specific path l that it can choose. make Will according to The values ​​are sorted in descending order, thus obtaining a... ordered list

[0089] (4) For lists elements in Perform the following operations according to the list order:

[0090] Will Set to 1, the rest Set to 0, and use the SMT solver to verify the effect on three constraints and A. accDoes a feasible solution exist under ∪{i}? If a feasible solution exists, then make a decision. Add to the set of solutions Client i * Add to set A acc In the process, client i will be removed from set A. cur Remove (④) from the list and proceed to step (5); otherwise, undo the operations prior to this step. from Remove from list If not empty, continue with step (4);

[0091] (5) For set A cur Each client i in Not here When the time comes, add client i to set A. rej In the process, client i will be removed from set A. cur Remove from the middle. Then return to step (2);

[0092] (6) The final solution to the problem is a set If the result of the objective function is within the given tolerance range, the solution ends, and the decision result can be obtained, namely, the admitted clients, the computing nodes connected to the admitted clients, the split points of the models trained on the admitted clients and their training servers, the end-to-end bandwidth and routing from the admitted clients to their training servers. Otherwise, update ρ and solve again, returning to step (1).

[0093] See the instruction manual appendix Figure 3 In this embodiment, under the computing power awareness network, it is assumed that the dedicated controller deployed in the federated learning server can have the client and computing node information for each global round, as well as the link status information of the entire network.

[0094] The following section provides a detailed explanation of the multivariate control decision-making method based on a federated split learning framework within a computing power-aware network environment:

[0095] (1) Determine the algorithm input parameters maintained by the dedicated controller in the federated learning server.

[0096] The parameters in the algorithm include:

[0097] Client set V in the network C

[0098] The set of routers V in the network R

[0099] The set of computing nodes V in the network S

[0100] The set of links E in the network

[0101] Link bandwidth capacity B in the network

[0102] Link unit cost β in the network e

[0103] Connection path between clients and computing nodes in the network

[0104] Given the client admission balancing parameter λ in the model training task

[0105] Given a virtual queue Q in the model training task i (t)

[0106] Given the parameterized parameters ρ in the model training task

[0107] Given the model parameters w in the model training task

[0108] The set of clients participating in training under a given model training task

[0109] The set of clients participating in training under a given model training task

[0110] Given the global number of rounds T for the model training task

[0111] Given the model training task, the upper limit Δ of the task execution time in each global round t is 1.

[0112] The number of training iterations ε in each global round t given the model training task.

[0113] Given a model training task, the next batch training size H

[0114] Given the number K of model split points under the model training task w

[0115] Given a model training task with k split points, the computational power required by the client model for each batch training size.

[0116] Given a model training task with k split points, the required computing power of the server model per batch size.

[0117] Given a model training task with k split points, the amount of data transfer s between the client and server per batch training size. k

[0118] The dataset D that client i owns locally i

[0119] The computational power c of client i in global round tit

[0120] The number of compute servers Ω that compute node j can start. j

[0121] The computing power ω of the computing server on computing node j j

[0122] The unit cost α of the compute server on compute node j j

[0123] (2) Decision-making client whether to participate in model training task:

[0124] The federated learning server obtains current network status information, as well as information about clients and computing nodes, and strategically selects a set of clients from all client sets for model training tasks.

[0125] (3) Construct and solve a multivariate control problem based on a federated decomposition learning framework:

[0126] The dedicated controller in the federated learning server acts as the task initiator to construct a multivariate control problem based on the federated split learning framework. For the clients participating in the training, it further solves the decision-making regarding their admission status, computing node allocation, model splitting points, routing, and bandwidth allocation.

[0127] (4) Model training process:

[0128] The admitted clients and their corresponding training servers on the computing nodes execute model training tasks according to the decision. After training is completed, they upload the trained models to the federated learning server for global model aggregation.

[0129] This invention uses computer simulation to conduct experiments.

[0130] This invention simulates a computing power-aware network in a Mininet environment, creating federated learning servers, clients, and computing nodes, and initializing the nodes. In each global round, a multivariate control decision-making method based on a federated split learning framework is used to make decisions on client admission, model splitting, server selection, routing, and bandwidth allocation to complete model training. Experimental results show that our proposed method improves model accuracy while maximizing resource utilization within a limited timeframe.

[0131] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not limited to the present invention.

[0132] In this embodiment, the two deep learning models used for training are DenseNet and MobileNet, and the dataset used is ImageNet2012. The number of epochs is set to 1. For MobileNet, the batch size is set to 4, and for DenseNet, it is set to 8. The global round duration is set to 5 for MobileNet and 140 for DenseNet. Two real-world network topologies are used to represent network states of different sizes: NSFNET represents a small topology with 14 nodes and 21 links, and USNET represents a denser topology with 24 nodes and 43 links. Four network scenarios are considered for different client distributions:

[0133] • NS1: Randomly select 6 nodes as computing nodes, and connect the remaining 8 nodes to 6 clients in NSFNET respectively;

[0134] • NS2: Randomly select 6 nodes as computing nodes, and each of the 16 nodes is connected to one client in USNET;

[0135] NS3: Randomly select 6 nodes as computing nodes, and connect the 16 nodes to 3 clients in USNET respectively;

[0136] NS4: Randomly select 6 nodes as computing nodes, and 3 nodes are connected to 16 clients in USNET respectively.

[0137] The number of training servers per compute node in "NS1", "NS3", and "NS4" is set to 8, while in "NS2" it is set to 3. The unit bandwidth cost of the network link is randomized from 0.1 to 1 for MobileNet and from 1 to 10 for DenseNet. The bandwidth capacity of the network link is randomized between 3000 and 5000. Each client has 4000 to 20000 random data samples. An auxiliary parameter p′ = 10000 is introduced to balance the values ​​of training utility and system cost.

[0138] For different deep learning training algorithm frameworks, using the default solution strategy, 10 experiments were conducted in each network scenario (i.e., NS1-NS4). Please refer to [link / reference]. Figure 4Tables I and II compare experimental results of the present invention with existing methods for model training tasks, with evaluation metrics being the average training amount and normalized accuracy. The present invention's algorithmic frameworks include CAN-FedSL, FedAvg (a framework for local training federated averaging within a specified time), SplitFed(Unlimited) (a framework for federated splitting without limitations on link bandwidth and the number of training servers), SplitFed(Limited) (a framework for federated splitting considering capacity constraints), and CAN-FedSL(NQ) (a framework for client admission without considering fairness awareness). Overall, for each training task, our performance (in bold) significantly outperforms FedAvg in every network scenario. Compared to SplitFed(Unlimited), our performance achieves the closest possible performance (i.e., normalized accuracy) and improves normalized accuracy by 11.4% (8.2%) compared to SplitFed(Limited) in MobileNet (DenseNet). Although CAN-FedSL requires slightly less training resources than CAN-FedSL(NQ), it achieves slightly higher accuracy. This result underscores the effectiveness of fairness-aware client admission. In conclusion, we can show that CAN-FedSL outperforms standard and state-of-the-art learning frameworks across various settings.

[0139] Based on our deep learning training algorithm framework, we conducted 10 experiments in each network scenario (NS1-NS4) using different solution strategies. Please see [link / reference]. Figure 5 a and Figure 5Figure b shows a comparison of experimental results regarding resource utilization efficiency using the present invention and existing methods. The evaluation index is the comprehensive evaluation index RUE. The solution method of the present invention is Ours; the solution method that randomly selects each customer based on customer weight p is RCA; the solution method that splits the model at the same split point is RMP; and the solution method that selects the shortest path after the server selection decision is completed is RPS. Compared with RCA, the algorithm of the present invention improves the RUE of NS1, NS2, NS3, and NS4 by an average of 1.84 times, 1.19 times, 1.89 times, and 1.62 times, respectively. These results indicate that although RCA can also achieve fairness-aware client admission, it cannot fully utilize available computing resources (i.e., some clients are not selected due to randomness). Compared with RMP, the algorithm of the present invention improves the RUE of NS1, NS2, NS3, and NS4 by an average of 3.04 times, 1.73 times, 2.68 times, and 2.26 times, respectively. These results emphasize that flexible model splitting is more meaningful than random splitting. In fact, for a single split point, some clients and their corresponding training servers on the compute nodes may lead to more data exchange (e.g., not the optimal split point), thus increasing system costs. Compared to RPS, the algorithm of this invention improves the average RUE of NS1, NS2, NS3, and NS4 by 1.14 times, 1.07 times, 1.50 times, and 1.74 times, respectively. Because the algorithm of this invention considers server selection and routing, its RUE is always greater than 1. Furthermore, better performance can be obtained when clients are evenly distributed in the network (i.e., NS3). In summary, each stage of the multivariate control in the algorithm of this invention has a significant impact on RUE performance.

[0140] The embodiments in this specification are described in a progressive manner. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple. For relevant parts, please refer to the description in the method section.

[0141] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multivariate control decision-making method based on a federated decomposition learning framework, characterized in that, Includes the following steps: (1) The dedicated controller in the federated learning server selects the deep learning model training task to be trained. At the beginning of each time period, it obtains the amount of data that each client can participate in training, the average number of times it has been admitted, the current link status information, the unit cost on the computing node, and the unit bandwidth cost on the link. (2) Based on the amount of data that each client can participate in training and the average number of times it has been admitted before, strategically select a group of clients from all client sets to perform model training tasks; (3) For clients participating in training, a multivariate control problem based on federated split learning is constructed and solved to further determine client admission status, computing node allocation, model splitting points, routing, and bandwidth allocation; wherein, the multivariate control problem based on the federated split learning framework is defined as the problem of maximizing resource utilization utility, i.e. , representing the average training utility achieved per unit system overhead in each round; (4) For clients with an admission status and the training servers on their corresponding computing nodes, the training tasks are executed according to the decision. After training, the trained models are uploaded to the federated learning server for global model aggregation.

2. The multivariate control decision-making method based on a federated split learning framework according to claim 1, characterized in that, When strategically selecting a set of clients from all client sets as described in step (2), the federated learning server, based on the amount of data that the current client can participate in training and the average number of times it has been admitted previously, uses the properties of Lyapunov virtual queues and queuing theory to decide whether the client should participate in the model training task in this round, and calculates the client's model selection: ; in, , ,in The larger the amount of data the client possesses, the larger pi becomes. This indicates that the average number of times each client is selected is consistent with the sampling probability given throughout the training process; The parameter is used to balance the two items mentioned above.

3. The multivariate control decision-making method based on a federated splitting learning framework according to claim 1, characterized in that, When constructing the multivariate control problem based on the federated split learning framework in step (3), the system overhead model is calculated based on the current link state information, the unit overhead on the computing node, and the unit bandwidth overhead on the link: ; in, This indicates the number of training servers used on compute node j. Indicates the amount of link bandwidth used. This represents the unit cost on compute node j. This represents the unit bandwidth cost on link e; The ratio of the client selection model to the system overhead model is defined as a multivariate control problem based on a federated split learning framework.

4. The multivariate control decision-making method based on a federated split learning framework according to claim 1, characterized in that, In step (3), when solving the multivariate control problem based on the federated decomposition learning framework, the Tinkelbach transform is used to transform the multivariate control problem based on the federated decomposition learning framework into a simplified linearized parameter objective function. A heuristic greedy algorithm is designed by relaxing integers to decimals, rounding, and verifying, and the solution is started. The specific steps are as follows: (3-1) Initialize all admissible client sets and non-admissible client sets to empty, and the undecided client set contains all clients; (3-2) If the set of undecided clients is not empty, proceed to step (3-3); otherwise, skip to step (3-6). (3-3) For each client i in the set of undecided clients, the computational node j and specific path that the client can choose are obtained through the multivariate control problem of the constructed federated split learning framework. Decimal solution of decision The multivariate control problem of the federated decomposition learning framework is simplified to: ； make Solve the decimal according to Arrange the values ​​in descending order to obtain ordered list ; (3-4) For lists Each element in Perform the following operations according to the list order: elements Set to 1, and the rest of the elements Set it to 0, and use the solver to verify whether a feasible solution exists; If a feasible solution exists, then Add to the set of solutions In the process, add client i to the set of admissible clients, remove client i from the set of clients that have not yet been decided, and execute steps (3-5); otherwise, undo the operations before this step and reset the elements. From list Remove from list If not empty, continue to repeat steps (3-4). (3-5) For each client i in the set of undecided clients and Not here If the client i is not allowed, add the client i to the set of clients that are not allowed to enter, and remove the client i from the set of clients that have not made a decision; then return to step (3-2). The final solution to problem (3-6) is a set. If the result is within the given tolerance range, the solution ends and the decision result is obtained, namely the admitted client, the computing node connected to the admitted client, the model split point trained on the training server of the admitted client and its corresponding computing node, and the end-to-end bandwidth and routing from the admitted client to the training server of its corresponding computing node. Otherwise, return to step (3-1) and solve again.

Images