A cost-optimization approach for split federated learning

By computing and optimizing the client's data calculation and communication delay and energy consumption in split federated learning system, the problem of high time and energy consumption in split federated learning is solved, and the cost of model training is realized Optimization and data privacy protection.

CN117521778BActive Publication Date: 2024-11-05GUANGDONG UNIV OF TECH
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202311398490.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-25
Publication Date
2024-11-05
Estimated Expiration
2043-10-25

AI Technical Summary

Technical Problem

In splitting federated learning, IoT devices require continuous computing and data transmission, resulting in high time and energy consumption costs, which poses practical challenges.

Method used

By building a split federated learning system model, calculate the client's data calculation and communication delay and energy consumption, use the model split layer and bandwidth as decision variables, establish and solve cost optimization problems, obtain the model splitting and bandwidth allocation strategy, and Optimize the training process of the client.

Benefits of technology

It effectively reduces the training cost and communication delay of the client, optimizes the model training process, and protects the privacy of client data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117521778B_ABST
    Figure CN117521778B_ABST
Patent Text Reader

Abstract

The present invention discloses a cost optimization method for split federated learning, including: S1: constructing a split federated learning system model, the model includes a server and several clients, each client has its own local data set; S2: calculating the delay and energy consumption of data calculation and communication of all clients; S3: using the delay and energy consumption of client data calculation and communication, taking the model splitting layer and bandwidth as decision variables, analyzing the objective function and related constraints, and establishing a split federated learning system cost optimization problem; S4: solving the optimization problem, obtaining the model splitting and bandwidth allocation strategy; S5: organizing all clients to cooperate with the server to train a given complete model based on the obtained model splitting and bandwidth allocation strategy, and obtaining the trained complete model. The present invention calculates the optimal model splitting and bandwidth allocation strategy, so that the weighted sum of the total time and total energy consumption of all clients in the split federated learning process is minimized, thereby reducing the cost of time and energy consumption.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and more specifically, to a cost optimization method and system for split federated learning. Background Art

[0002] Federated Learning (FL) and Split Learning (SL) are distributed model training technologies in deep learning that do not share user data. In federated learning, the server first initializes a global model. In each round of global model training, the server sends the global model to all clients. Each client trains the model based on local data on the local side, and then uploads the updated model parameters to the server. The server aggregates and updates the global model. The above process continues to iterate until the accuracy of the global model converges or the global model training round reaches the preset value. In split learning between the server and a client, the complete model is first split into two parts at the specified position (i.e., the split layer), and the front part sub-model is assigned to each client, and the back part sub-model is distributed to the server. In each round of model training, the client performs forward propagation to the split layer based on local data, obtains the crushed data and uploads it to the server. The server completes the forward propagation of the remaining layers. The server also performs back propagation to the split layer, obtains the gradient information of the above crushed data and transmits it back to the client, and the client then completes the back propagation of the remaining layers. The above two processes of forward and backward propagation are iterated continuously until the client completes all local data calculations, and then the updated client sub-model parameters are sent to the next client participating in model training, and the above steps are repeated until the model converges. The advantage of federated learning is that it allows multiple clients to train models in parallel while protecting data privacy. Split learning allows the client to train only sub-models separately. In order to combine the advantages of both, the existing literature [Thapa, Chandra, et al. "Splitfed: When federated learning meets split learning." Proceedings of the AAAIConference on Artificial Intelligence. Vol. 36. No. 8. 2022.] proposed split federated learning (Splitfed Learning, SFL), in which all clients and servers use split learning to complete model training, and the server uses federated learning to update the unified client sub-model. However, considering the participation of energy-constrained IoT devices in split federated learning, although split federated learning distributes most of the computational load involved in model training to the server, each client still needs to continuously calculate, send and receive data during the training process, which brings huge time and communication overhead, which puts a lot of pressure on IoT devices. Therefore, in order to ensure the practicality of split federated learning, it is crucial to optimize the cost management of all clients participating in split federated learning, including time and energy consumption. Summary of the invention

[0003] In order to overcome the defects of the above-mentioned prior art in that the model training time and energy consumption are relatively high, the present invention provides a cost optimization method for split federated learning.

[0004] The primary purpose of the present invention is to solve the above technical problems. The technical solution of the present invention is as follows:

[0005] A first aspect of the present invention provides a cost optimization method for splitting federated learning, comprising the following steps:

[0006] S1: Build a split federated learning system model, which includes a server and several clients, each of which has its own local dataset;

[0007] S2: Calculates the latency and energy consumption of all client data computation and communication;

[0008] S3: Using the latency and energy consumption of client data computing and communication, taking the model splitting layer and bandwidth as decision variables, analyzing the objective function and related constraints, and establishing the cost optimization problem of the split federated learning system;

[0009] S4: Solve the optimization problem and obtain the model splitting and bandwidth allocation strategy;

[0010] S5: In the split federated learning system, all clients are organized to cooperate with the server to train the given complete model based on the obtained model splitting and bandwidth allocation strategy to obtain the trained complete model.

[0011] Furthermore, step S2 calculates the latency and energy consumption of all client data computation and communication, firstly calculating the client local computation energy consumption and computation latency, and then calculating the split federated learning system client energy cost and computation communication latency.

[0012] Furthermore, the specific process of calculating the energy consumption and the calculation delay of the local calculation client is as follows:

[0013] The local computing load of client i for unit data client sub-model training is:

[0014]

[0015] Among them, γ(·) and ξ(·) represent the number of operations required for the forward and reverse calculations of the client sub-model and server sub-model samples, respectively. The unit is FLOPs, where C represents the number of layers in the client submodel; similarly, it corresponds to the computing load that requires server collaboration.

[0016] Client local computing energy consumption Calculating latency They are

[0017]

[0018]

[0019] Among them, the calculation frequency of client i is f i , unit is cycle / s, k i is the computing intensity of client i, in FLOPs / cycle, and the computing speed of the client is defined as φ i =f i ×k i , in FLOPS; the client computing energy consumption coefficient is ε i , the unit is J / FLOPs; D i is the local data set owned by client i; the computing speed of the server cooperating with client i is ψ.

[0020] Furthermore, the calculation splits the energy cost of the federated learning system client and the calculation communication delay. The specific process is as follows:

[0021] When transmitting the shredded data and updated client submodel parameters, the uplink data transmission rate r of client i i up for:

[0022]

[0023] Among them, P i up is the transmission power when client i uploads data, b i is the bandwidth allocated by the server to client i, σ 2 is the Gaussian channel noise, is the channel gain between the server and the client, where α0 represents the channel gain at a distance of d = 1 m, and d i Represents the Euclidean distance between the server and the client;

[0024] The downlink data transmission rate r of the server transmitting the crushing data gradient information to the client i i down for:

[0025]

[0026] Among them, B D is the fixed bandwidth used by the server to broadcast to each user, P B is the average transmission power; client upload delay Download delay They are

[0027]

[0028]

[0029] Among them, the size of a single sample crushed data is D(C), the size of the gradient is G(C), the size of the data label is β, and the size of the client sub-model parameter is

[0030] Communication delay T i comm for:

[0031]

[0032] Client communication energy consumption for

[0033]

[0034] Among them, P i down is the client receiving power, P i up Average transmit power of client i;

[0035] Client i time cost T i total Energy consumption They are:

[0036] T i total =T i comm +T i comp

[0037]

[0038] In a specified M global model training rounds, the split federated learning system client energy cost E and computational communication delay T are:

[0039]

[0040]

[0041] Furthermore, in step S3, the system cost optimization problem established is as follows:

[0042]

[0043] subject to:

[0044] C1:C min ≤C≤C max

[0045] C2: b min ≤ b i ≤ b max

[0046] C3:

[0047] Among them, B total is the available uplink bandwidth for all clients; C1 specifies the range of model split layers, constraining the number of client sub-model layers; C2 specifies the range of bandwidth values ​​that can be allocated to each client, which is pre-set; C3 ensures that the total bandwidth resources allocated to the client does not exceed B total ; represents the weight between total energy consumption and total time delay,

[0048] Further, in step S4, solving the optimization problem specifically includes: discretizing the bandwidth variable and setting the discrete interval to Δ, then Among them, b i is the bandwidth allocated by the server to client i, b min and b max Respectively represent the minimum and maximum bandwidth values ​​that can be allocated to the client.

[0049] Furthermore, the optimal solution to the integer programming problem is solved using the Scipy library tool in Python or an existing planning problem solver.

[0050] Furthermore, in step S5, all clients are organized to cooperate with the server to train a given complete model based on the obtained model splitting and bandwidth allocation strategy. The specific process is as follows:

[0051] The given complete model is split into a global client sub-model and a global server sub-model. In each round of global model training, the global client sub-model and the global server sub-model are trained, which includes two aspects: first, each client receives the global client sub-model sent by the server, trains this sub-model by jointly training the server, and then uploads the updated client sub-model parameters to the server to aggregate and update the global client sub-model; in addition, the server internally assigns an associated server sub-model to each client, and updates this associated server sub-model when coordinating each client to train its client sub-model. Finally, the global server sub-model is updated by aggregating all associated server sub-models.

[0052] Furthermore, the specific process of step S5 is as follows:

[0053] S51: The server selects a model splitting strategy to determine the number of layers C contained in the client sub-model, and splits the given complete model into the client sub-model W client Send to each client;

[0054] S52: Each client receives W client , this sub-model is updated based on local data in cooperation with the server, and the server is responsible for updating the server sub-model;

[0055] S53: Any client i has a local dataset D i , the training batch size is H, and the local dataset is divided into batches;

[0056] S54: Any client i uses the current batch data to perform forward propagation of the client sub-model, and sends the crushed data and the data label corresponding to the current batch to the server, where the crushed data is the output of the splitting layer;

[0057] S55: The server uses the received crushed data to perform forward propagation of the server sub-model associated with client i, and calculates the loss function according to the received data label, performs back propagation calculation on the associated server sub-model to obtain the gradient of the crushed data, and updates the associated server sub-model based on the crushed data gradient; the above steps for all clients are executed in parallel inside the server;

[0058] S56: The server sends the gradient information of all crushed data to the corresponding clients respectively;

[0059] S57: Each client receives the gradient information of the crushed data, performs back propagation in the client sub-model and updates its client sub-model;

[0060] S58: Repeat S54 to S57 until all local data is used up;

[0061] S59: Each client uploads the updated local client sub-model parameters to the server. The server uses weighted averaging to aggregate and update the global client sub-model, and then re-sends the global client sub-model to all clients for a new round of local client sub-model training. The server aggregates all associated server sub-models through weighted averaging to update the global server sub-model.

[0062] A second aspect of the present invention provides a cost optimization system for split federated learning, the system comprising: a memory, a processor, the memory comprising a cost optimization method program for split federated learning, the cost optimization method program for split federated learning being executed by the processor to implement the following steps:

[0063] S1: Build a split federated learning system model, which includes a server and several clients, each of which has its own local dataset;

[0064] S2: Calculates the latency and energy consumption of all client data computation and communication;

[0065] S3: Using the latency and energy consumption of client data computing and communication, taking the model splitting layer and bandwidth as decision variables, analyzing the objective function and related constraints, and establishing the cost optimization problem of the split federated learning system;

[0066] S4: Solve the optimization problem and obtain the model splitting and bandwidth allocation strategy;

[0067] S5: In the split federated learning system, all clients are organized to cooperate with the server to train the given complete model based on the obtained model splitting and bandwidth allocation strategy to obtain the trained complete model.

[0068] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0069] The present invention first constructs a split federated learning system model, which includes a server and several clients, each of which has its own local data set; secondly, the delay and energy consumption of data calculation and communication of all clients are calculated; then, the delay and energy consumption of data calculation and communication of the clients are used, the model splitting layer and bandwidth are used as decision variables, the objective function and related constraints are analyzed, and the cost optimization problem of the split federated learning system is established; then, the optimization problem is solved to obtain the model splitting and bandwidth allocation strategy, and the optimal model splitting and bandwidth allocation strategy is obtained; finally, in the split federated learning system, based on the obtained model splitting and bandwidth allocation strategy, all clients are organized to cooperate with the server to train a given complete model, and the trained complete model is obtained, so as to protect the data privacy of the client and enable the client to complete the model training task in a training cost optimization manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A flow chart of a cost optimization method for split federated learning provided in an embodiment of the present invention.

[0071] Figure 2 A model diagram of a split federated learning system provided in an embodiment of the present invention.

[0072] Figure 3 A workflow diagram of a split federated learning system provided in an embodiment of the present invention.

[0073] Figure 4 A flow chart of model splitting and bandwidth allocation strategy calculation provided by an embodiment of the present invention.

[0074] Figure 5A schematic diagram of the ResNet18 model splitting provided in an embodiment of the present invention.

[0075] Figure 6 A simulation result diagram is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0077] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0078] Example 1

[0079] like Figure 1 As shown, the first aspect of the present invention provides a cost optimization method for splitting federated learning, comprising the following steps:

[0080] S1: Build a split federated learning system model, which includes a server and I clients. Each client has its own local dataset, such as Figure 2 Shown is a model diagram of a split federated learning system.

[0081] More specifically, we build a split federated learning system model, such as Figure 2 As shown, it includes a server and I clients. The server performs server sub-model training, aggregates and updates the client sub-model shared by all clients, and allocates different communication bandwidths to different clients. Each client has its own local data set to train its own client sub-model and communicates data with the server through a wireless channel.

[0082] S2: Calculate the latency and energy consumption of all client data calculations and communications.

[0083] More specifically, the specific process of calculating the client's local computing energy consumption and computing latency is as follows:

[0084] The local computing load of client i for unit data client sub-model training is γ(·) and ξ(·) respectively calculate the number of operations required for the forward and reverse calculations of a single sample of the network model. The unit is FLOPs, where C represents the number of layers in the client submodel; similarly, it corresponds to the computing load that requires server collaboration. Client i calculates frequency fi , unit is cycle / s, k i is the computing intensity of client i, in FLOPs / cycle, and the computing speed of the client is defined as φ i =f i ×k i , in FLOPS; the client computing energy consumption coefficient is ε i , the unit is J / FLOPs; D i is the local data set owned by client i; the computing speed of the server and client i is ψ, then the local computing energy consumption of the client is Calculate the delay T i comp They are

[0085]

[0086]

[0087] The calculation splitting federated learning system client energy cost and calculation communication delay are as follows:

[0088] P i up is the transmission power when client i uploads data, b i is the bandwidth allocated by the server to client i, σ 2 is the Gaussian channel noise, is the channel gain between the server and the client, where α0 represents the channel gain at a distance of d = 1m, and d i represents the Euclidean distance between the server and the client; when transmitting the shredded data and the updated client submodel parameters, the uplink data transmission rate r of client i i up :

[0089]

[0090] Similarly, the downlink data transmission rate r of the server transmitting the crushing data gradient information to the client i is i down :

[0091]

[0092] Among them, B D is the fixed bandwidth used by the server to broadcast to each user, P B is the average transmission power;

[0093] The size of a single sample crushed data is D(C), the size of the gradient is G(C), the size of the data label is β, and the size of the client sub-model parameter is The client upload delay Download delay They are

[0094]

[0095]

[0096] Communication delay T i comm for:

[0097]

[0098] P i down is the client receiving power, P i up The average transmission power of client i is for

[0099]

[0100] Client i time overhead Energy consumption They are:

[0101] T i total =T i comm +T i comp

[0102]

[0103] In a specified M global model training rounds, the split federated learning system client energy cost E and computational communication delay T are:

[0104]

[0105]

[0106] S3: Using the latency and energy consumption of client data computing and communication, taking the model splitting layer and bandwidth as decision variables, analyzing the objective function and related constraints, and establishing the cost optimization problem of the split federated learning system.

[0107] More specifically, in step S3, the system cost optimization problem established is as follows:

[0108]

[0109] subject to:

[0110] C1:Cmin ≤C≤C max

[0111] C2: b min ≤ b i ≤ b max

[0112] C3:

[0113] Among them, B total is the available uplink bandwidth for all clients; C1 specifies the range of model split layers, constraining the number of client sub-model layers; C2 specifies the range of bandwidth values ​​that can be allocated to each client, which is pre-set; C3 ensures that the total bandwidth resources allocated to the client does not exceed B total ; represents the weight between total energy consumption and total time delay,

[0114] S4: Solve the optimization problem and obtain the model splitting and bandwidth allocation strategy.

[0115] More specifically, in order to simplify the problem solving in practical applications, consider discretizing the bandwidth variable and setting the discrete interval to Δ, then At this point, the original problem will be transformed into a traditional integer programming problem, and then the optimal solution to the integer programming problem will be solved using the Scipy library tool in Python or an existing planning problem solver (such as Lingo).

[0116] S5: In the split federated learning system, all clients are organized to cooperate with the server to train the given complete model based on the obtained model splitting and bandwidth allocation strategy to obtain the trained complete model.

[0117] More specifically, the given complete model is split into a global client sub-model and a global server sub-model. In each round of global model training, the global client sub-model and the global server sub-model are trained, which includes two aspects: first, each client receives the global client sub-model sent by the server, trains this sub-model by jointly training the server, and then uploads the updated client sub-model parameters to the server to aggregate and update the global client sub-model; in addition, the server internally assigns an associated server sub-model to each client, and updates this associated server sub-model when coordinating the training of each client's client sub-model. Finally, the global server sub-model is updated by aggregating all associated server sub-models.

[0118] More specifically, Figure 3 To split the workflow diagram of the federated learning system, the specific process of step S5 is as follows:

[0119] S51: The server selects a model splitting strategy to determine the number of layers C contained in the client sub-model, and splits the given complete model into the client sub-model W client Send to each client;

[0120] S52: Each client receives W client , this sub-model is updated based on local data in cooperation with the server, and the server is responsible for updating the server sub-model;

[0121] S53: Any client i has a local dataset D i , the training batch size is H, and the local dataset is divided into batches;

[0122] S54: Any client i uses the current batch data to perform forward propagation of the client sub-model, and sends the crushed data and the data label corresponding to the current batch to the server, where the crushed data is the output of the splitting layer;

[0123] S55: The server uses the received crushed data to perform forward propagation of the server sub-model associated with client i, and calculates the loss function according to the received data label, performs back propagation calculation on the associated server sub-model to obtain the gradient of the crushed data, and updates the associated server sub-model based on the crushed data gradient; the above steps for all clients are executed in parallel inside the server;

[0124] S56: The server sends the gradient information of all crushed data to the corresponding clients respectively;

[0125] S57: Each client receives the gradient information of the crushed data, performs back propagation in the client sub-model and updates its client sub-model;

[0126] S58: Repeat S54 to S57 until all local data is used up;

[0127] S59: Each client uploads the updated local client sub-model parameters to the server. The server uses weighted averaging to aggregate and update the global client sub-model, and then re-sends the global client sub-model to all clients for a new round of local client sub-model training. The server aggregates all associated server sub-models through weighted averaging to update the global server sub-model.

[0128] A second aspect of the present invention provides a cost optimization system for split federated learning, the system comprising: a memory, a processor, the memory comprising a cost optimization method program for split federated learning, the cost optimization method program for split federated learning being executed by the processor to implement the following steps:

[0129] S1: Build a split federated learning system model, which includes a server and several clients, each of which has its own local dataset;

[0130] S2: Calculates the latency and energy consumption of all client data computation and communication;

[0131] S3: Using the latency and energy consumption of client data computing and communication, taking the model splitting layer and bandwidth as decision variables, analyzing the objective function and related constraints, and establishing the cost optimization problem of the split federated learning system;

[0132] S4: Solve the optimization problem and obtain the model splitting and bandwidth allocation strategy;

[0133] S5: In the split federated learning system, all clients are organized to cooperate with the server to train the given complete model based on the obtained model splitting and bandwidth allocation strategy to obtain the trained complete model.

[0134] Example 2

[0135] The flowchart for implementing split federated learning cost optimization is as follows: Figure 4 As shown. The following is a specific experimental simulation to demonstrate the effect of the method. Consider a split federated learning system with 5 clients. The server is located in the center and the clients are evenly distributed 100 to 200 meters around the server. The training task is the classic CIFAR10 dataset classification task, and the global iteration number M = 100. The CIFAR10 dataset consists of 60,000 32×32 color images in 10 categories, each category has 6,000 images, and 50,000 training images and 10,000 test images are selected. The model selects the convolutional neural network model ResNet-18, defines the number of model layers as 10, and the model split candidate positions are as follows Figure 5 As shown. Considering heterogeneity, the client local data is evenly distributed in [8000, 10000], and the calculation speed φ i Satisfy [0.01,0.03] TFLOPS uniform distribution, client energy consumption coefficient ε i Satisfy [0.5,0.7] J / TFLOPs uniform distribution, transmission power P i up Satisfies [0.1,0.2]W uniform distribution. For simplicity, let P i up =P i down The computing speed of the server and each client is ψ2TFLOPS, and the total bandwidth of the system is B total The channel gain α0 is set to -50dB, the server transmission power is 1W, and the bandwidth used for broadcasting is B D The weight coefficient in the objective function is 15MHz.

[0136] The bandwidth discrete interval is Δ = 0.1MHz, and the minimum bandwidth b allocated to the user min The maximum bandwidth is 0.5MHz. max =3.5MHz, then b i ∈{0.5+k×Δ,|k=0,1,2,…,30}, the simulation results are as follows Figure 6 shown.

[0137] When C=10, the original federated learning solution is executed. As can be seen from the table, this patent obtains the optimal solution at the split layer C=6, at which the bandwidth allocation is (1.3, 2.6, 1.8, 1.1, 1.2), and the objective function Objective function that is better than the original federated learning solution

[0138] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A cost optimization method for split federated learning, characterized in that: The following steps are involved: S1: Build a split federated learning system model, which includes a server and several clients, each of which has its own local dataset; S2: Calculates the computing delay, computing energy consumption, communication delay, and communication energy consumption of all client data; S3: using the computing delay, computing energy consumption, communication delay and communication energy consumption of the client data, taking the model splitting layer and bandwidth as decision variables, and establishing an optimization goal of minimizing the cost of splitting the federated learning system according to the objective function and constraints; S4: Solve the optimization problem and obtain the model splitting and bandwidth allocation strategy; S5: In the split federated learning system, all clients are organized to cooperate with the server to train the given complete model based on the obtained model splitting and bandwidth allocation strategy to obtain the trained complete model; In step S2, the computing delay, computing energy consumption, communication delay and communication energy consumption of all client data are calculated. First, the computing energy consumption and computing delay of the client are calculated locally, and then the communication energy consumption and communication delay of the client of the split federated learning system are calculated. The specific process of calculating the local computing energy consumption and computing delay of the client is as follows: The local computing load of client i for unit data client sub-model training is: Among them, γ client (C) represents the number of operations required for the model to perform forward calculations on a single client sample at the split point C; ξ client (C) represents the number of operations required for the model to perform forward calculations on a single client sample at the split point C; The unit is FLOPs, where C represents the number of layers in the client submodel; similarly, it corresponds to the computing load that requires server collaboration. Client local computing energy consumption Calculate the delay T i comp They are Among them, the calculation frequency of client i is f i , unit is cycle / s, k i is the computing intensity of client i, in FLOPs / cycle, and the computing speed of the client is defined as φ i =f i ×k i , in FLOPS; the client computing energy consumption coefficient is ε i , the unit is J / FLOPs; D i is the local data set owned by client i; the computing speed of the server cooperating with client i is ψ.

2. A cost optimization method for split federated learning according to claim 1, characterized in that: The calculation splits the communication energy consumption and communication delay of the federated learning system client. The specific process is as follows: When transmitting the shredded data and updated client submodel parameters, the uplink data transmission rate r of client i i up for: Among them, Pi up is the transmission power when client i uploads data, bi is the bandwidth allocated by the server to client i, σ 2 is the Gaussian channel noise, is the channel gain between the server and the client, where α0 represents the channel gain at a distance of d = 1m, and di represents the Euclidean distance between the server and the client; The downlink data transmission rate r of the server transmitting the crushing data gradient information to the client i i down for: Among them, B D is the fixed bandwidth used by the server to broadcast to each user, P B is the average transmission power; Client upload delay Download delay They are Among them, the size of a single sample crushed data is D(C), the size of the gradient is G(C), the size of the data label is β, and the size of the client sub-model parameter is Communication delay for: Client communication energy consumption for Among them, Pi down For client receiving power, Pi up Average transmit power of client i; Client i time cost T i total Energy consumption They are: T i total =T i comm +T i comp In the specified M global model training rounds, the total time cost T and total energy cost E of all clients in the split federated learning system are: T=Mmax(T1 total ,T2 total ,...,T I total ) 3. A cost optimization method for split federated learning according to claim 2, characterized in that: In step S3, based on the objective function and constraints, the optimization goal of minimizing the cost of splitting the federated learning system is established as follows: The objective function is: The constraints are: subject to: C1:C min ≤C≤C max C2:b min ≤b i ≤b max C3: Among them, B total is the available uplink bandwidth for all clients; C1 is the range of model split layers, which constrains the number of client sub-model layers; C2 is the constraint on the bandwidth value range that can be allocated to each client; C3 is the constraint on the total bandwidth resources allocated to the client; represents the weight between the total energy consumption overhead and the total delay overhead; b i is the bandwidth allocated by the server to client i; C is the number of layers contained in the client sub-model.

4. The cost optimization method for split federated learning according to claim 1, characterized in that: In step S4, solving the optimization problem specifically includes: discretizing the bandwidth variable and setting the discrete interval to Δ, then Among them, b i is the bandwidth allocated by the server to client i, b min and b max They represent the minimum and maximum bandwidth values ​​that can be allocated to the client respectively; k is the kth discrete interval.

5. A cost optimization method for split federated learning according to claim 4, characterized in that: Use the Scipy library tools in Python or existing planning problem solvers to find the optimal solution to the integer programming problem.

6. The cost optimization method for split federated learning according to claim 1, characterized in that: Step S5 organizes all clients to cooperate with the server to train a given complete model based on the obtained model splitting and bandwidth allocation strategy. The specific process is as follows: The given complete model is split into a global client sub-model and a global server sub-model. In each round of global model training, the global client sub-model and the global server sub-model are trained, which includes two aspects: first, each client receives the global client sub-model sent by the server, trains this sub-model by jointly training the server, and then uploads the updated client sub-model parameters to the server to aggregate and update the global client sub-model; in addition, the server internally assigns an associated server sub-model to each client, and updates this associated server sub-model when coordinating each client to train its client sub-model. Finally, the global server sub-model is updated by aggregating all associated server sub-models.

7. A cost optimization method for split federated learning according to claim 6, characterized in that: The specific process of step S5 is as follows: S51: The server selects a model splitting strategy to determine the number of layers C contained in the client sub-model, and splits the given complete model into the global client sub-model W client Send to each client; S52: Each client receives W client , this sub-model is updated based on local data in cooperation with the server, and the server is responsible for updating the server sub-model; S53: Any client i has a local dataset D i , the training batch size is H, and the local dataset is divided into batches; S54: Any client i uses the current batch data to perform forward propagation of the client sub-model, and sends the crushed data and the data label corresponding to the current batch to the server, where the crushed data is the output of the splitting layer; S55: The server uses the received crushed data to perform forward propagation of the server sub-model associated with client i, and calculates the loss function according to the received data label, performs back propagation calculation on the associated server sub-model to obtain the gradient of the crushed data, and updates the associated server sub-model based on the crushed data gradient; the above steps for all clients are executed in parallel inside the server; S56: The server sends the gradient information of all crushed data to the corresponding clients respectively; S57: Each client receives the gradient information of the crushed data, performs back propagation in the client sub-model and updates its client sub-model; S58: Repeat S54 to S57 until all local data is used up; S59: Each client uploads the updated local client sub-model parameters to the server. The server uses weighted averaging to aggregate and update the global client sub-model, and then re-sends the global client sub-model to all clients for a new round of local client sub-model training. The server aggregates all associated server sub-models through weighted averaging to update the global server sub-model.

8. A cost optimization system for split federated learning, characterized in that: The system includes: a memory and a processor, wherein the memory includes a cost optimization method program for splitting federated learning, and when the cost optimization method program for splitting federated learning is executed by the processor, the following steps are implemented: S1: Build a split federated learning system model, which includes a server and several clients, each of which has its own local dataset; S2: Calculates the computing delay, computing energy consumption, communication delay, and communication energy consumption of all client data; S3: using the computing delay, computing energy consumption, communication delay and communication energy consumption of the client data, taking the model splitting layer and bandwidth as decision variables, and establishing an optimization goal of minimizing the cost of splitting the federated learning system according to the objective function and constraints; S4: Solve the optimization problem and obtain the model splitting and bandwidth allocation strategy; S5: In the split federated learning system, all clients are organized to cooperate with the server to train the given complete model based on the obtained model splitting and bandwidth allocation strategy to obtain the trained complete model; In step S2, the computing delay, computing energy consumption, communication delay and communication energy consumption of all client data are calculated. First, the computing energy consumption and computing delay of the client are calculated locally, and then the communication energy consumption and communication delay of the client of the split federated learning system are calculated. The specific process of calculating the local computing energy consumption and computing delay of the client is as follows: The local computing load of client i for unit data client sub-model training is: Among them, γ client (C) represents the number of operations required for the model to perform forward calculations on a single client sample at the split point C; ξ client (C) represents the number of operations required for the model to perform forward calculations on a single client sample at the split point C; The unit is FLOPs, where C represents the number of layers in the client submodel; similarly, it corresponds to the computing load that requires server collaboration. Client local computing energy consumption Calculate the delay T i comp They are Among them, the calculation frequency of client i is f i , unit is cycle / s, k i is the computing intensity of client i, in FLOPs / cycle, and the computing speed of the client is defined as φ i =f i ×k i , in FLOPS; the client computing energy consumption coefficient is ε i , the unit is J / FLOPs; D i is the local data set owned by client i; the computing speed of the server cooperating with client i is ψ.

Citation Information

Patent Citations

  • Data processing method and device, storage medium and electronic equipment

    CN115130548A

  • Multi-element control decision-making method based on federal splitting learning framework

    CN115907038A

  • Internet of Things heterogeneous device abnormal traffic detection method based on federal splitting learning

    CN116418589A

  • Parameter optimization and resource allocation method for low-energy-consumption wireless federated learning system

    CN116418687A

  • Model training method and communication apparatus

    WO2023083312A1