Federal learning training method and system based on model segmentation and resource allocation

Abstract

The invention discloses a federal learning training method and system based on model segmentation and resource allocation, and relates to the technical field of digital information transmission, and the method comprises the steps: building a target function; solving the objective function to obtain a client model segmentation scheme; classifying the clients; segmenting the client model of the second type of clients to obtain a top layer model and a bottom layer model; allocating computing resources to each second-class client; the first class of clients train a client model, and the second class of clients train a corresponding bottom layer model firstly, and then train a top layer model by using an intermediate result output by the bottom layer model; and after the training is finished, aggregating the parameters of the client model and the parameters of the bottom layer model and the top layer model to obtain a global model. According to the method, for the federal learning task of the resource-limited client in the edge environment, the model is segmented according to the computing power of the client, and the computing resources of the edge server are reasonably allocated, so that the training efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of digital information transmission technology, and in particular to a federated learning training method and system based on model segmentation and resource allocation. Background Art

[0002] Federated Learning (FL) is a privacy-preserving distributed machine learning framework. In federated learning, all participating devices perform the model training process in parallel using local data, and aggregate the model parameters by uploading them to the edge server to update the parameters. In this method, devices do not need to upload local raw data to the edge server, which not only meets the increasingly important privacy protection issue, but also can use the massive data generated by the edge server to train a highly generalized model. In practical applications, the performance of the federated learning framework may be affected by multiple factors, such as data heterogeneity, device systematicity, aggregation efficiency, etc. The goal of federated learning is to provide support for intelligent applications by accessing a large amount of private data, thereby greatly improving usability. This has significant application potential for fields with high privacy requirements, such as finance and healthcare.

[0003] Currently, the commonly used federated learning training methods are as follows.

[0004] Training method based on model compression: Model compression is a method that adaptively reduces the model depth and the number of hidden layer neurons according to the client's computing resources. In essence, it solves the device heterogeneity problem faced in federated learning by reducing the number of model parameters, such as HeteroFL and ScaleFL. However, model compression along the width (reducing the number of neurons) will cause the sub-model to become narrow and deep, and the narrow network (limiting input channels) loses the ability to recognize basic features to a certain extent. Deep networks require more computing resources and time, and more data to avoid overfitting. For the above reasons, the quality of the global model obtained after federated learning will drop sharply, and devices with limited resources will suffer serious accuracy loss when the model is deployed.

[0005] Training method based on model splitting: Model splitting is a method proposed by Gupta, Vepakomm and others, which alleviates the problem of client resource constraints by offloading part of the local computing tasks to the edge server. On this basis, Thapa et al. proposed Split Federated Learning (SFL), which combines the advantages of parallel training of FL and model splitting of SL (Split Learning), assigning a unified part of the network structure to the client, and the remaining part is calculated by the edge server. The model split point is highly related to the size of the data transmitted during training (the data size is related to the dimensions of the channels and feature maps of the network in the model). Smaller split layers will produce larger intermediate results, but larger split layers bring more computational load. This allocation method is manually predefined and completely ignores the heterogeneity between client devices.

[0006] The existing split federated learning methods have the following main shortcomings: Device heterogeneity is an important factor affecting the federated learning training task. Many methods assume that the computing power, bandwidth, etc. of the devices are the same, but the computing power of the devices in actual scenarios is diverse. This heterogeneity seriously affects the efficiency of training. The existing split federated learning methods mainly focus on using different technologies to improve the performance of the model, while ignoring the optimization of the federated learning training process from the system level, thereby limiting the efficiency of model training. Existing methods usually assume that all clients have the same split layer and the server has the same resource configuration scheme, but this does not conform to the device conditions in the actual environment. The existing aggregation method relies on all clients to upload model parameters after completing training. However, this process is limited by the slowest client, resulting in devices with stronger computing resources having to wait for devices with limited resources, thereby reducing the overall training efficiency. Summary of the invention

[0007] The embodiments of the present application provide a federated learning training method and system based on model segmentation and resource allocation, so as to solve the problem of low efficiency of the segmented federated learning method in the prior art.

[0008] On the one hand, the embodiment of the present application provides a federated learning training method based on model segmentation and resource allocation, including:

[0009] Get local information of each client;

[0010] Determine the training time of the client based on local information, establish an objective function with the goal of minimizing the training time;

[0011] Solve the objective function and obtain the corresponding client model segmentation solution;

[0012] All clients are classified according to the client model segmentation scheme. Clients that can complete training under the limitation of their own computing resources are classified as first-class clients, and clients that cannot complete training under the limitation of their own computing resources are classified as second-class clients.

[0013] The client model of the second type of client is segmented according to the client model segmentation scheme to obtain a top-level model and a bottom-level model, and the top-level model is sent to the edge server;

[0014] Allocate computing resources to each second-category client on the principle that the total allocated resources do not exceed the computing resources of the edge server;

[0015] The first type of client uses its own local data to train the corresponding client model. The second type of client first uses its own local data to train the corresponding underlying model, and then sends the intermediate results output by the underlying model to the edge server. The edge server uses the computing resources allocated to the second type of client to train the top-level model using the intermediate results.

[0016] After each round of training, the parameters of the client model in the first type of clients and the parameters of the bottom model and the top model corresponding to the second type of clients are aggregated to obtain the current aggregated parameters;

[0017] The current aggregation parameters are used as the parameters of the client model in the first type of clients and the parameters of the bottom model and the top model corresponding to the second type of clients, and the next round of training is carried out until the training is completed to obtain the trained global model.

[0018] On the other hand, the embodiment of the present application also provides a federated learning training system based on model segmentation and resource allocation, including:

[0019] The client information acquisition module is used to obtain the local information of each client;

[0020] An objective function establishment module is used to determine the training time of the client according to local information, and to establish an objective function with the minimum training time as the goal;

[0021] Function solving module, used to solve the target function and obtain the corresponding client model segmentation solution;

[0022] The client classification module is used to classify all clients according to the client model segmentation scheme, and classify clients that can complete training under the limitation of their own computing resources as first-class clients, and clients that cannot complete training under the limitation of their own computing resources as second-class clients;

[0023] A model segmentation module is used to segment the client model of the second type of client according to the client model segmentation scheme to obtain a top model and a bottom model, and send the top model to the edge server;

[0024] A resource allocation module, configured to allocate computing resources to each second-category client on the principle that the total allocated resources do not exceed the computing resources of the edge server;

[0025] The first model training module is used for the first type of client to train the corresponding client model using its own local data, and the second type of client first trains the corresponding underlying model using its own local data, and then sends the intermediate results output by the underlying model to the edge server, and the edge server uses the computing resources allocated to the second type of client to train the top model using the intermediate results;

[0026] A parameter aggregation module, used to aggregate the parameters of the client model in the first type of clients and the parameters of the bottom model and the top model corresponding to the second type of clients after each round of training to obtain the current aggregated parameters;

[0027] The second model training module is used to use the current aggregation parameters as the parameters of the client model in the first type of client and the parameters of the bottom model and the top model corresponding to the second type of client to perform the next round of training until the training is completed to obtain the trained global model.

[0028] The federated learning training method and system based on model segmentation and resource allocation in this application have the following advantages:

[0029] 1. For the federated learning tasks of resource-constrained clients in edge environments, the model segmentation method and resource allocation strategy are further improved to improve training efficiency and performance.

[0030] 2. In view of the fact that existing methods fail to fully consider the differences in client computing power and the lack of computing resources on edge servers, this application proposes an optimization solution that combines model segmentation and resource allocation. This solution divides the deep neural network model according to the computing power of the client and reasonably allocates the computing resources of the edge server to accelerate the training process of federated learning.

[0031] 3. This application designs a personalized model segmentation layer and resource configuration method, so that each device can dynamically adjust the client model, batch size and server resource configuration according to its own computing power, thereby accelerating the training process more effectively.

[0032] 4. The method proposed in this application has strong executability and can help researchers and practitioners apply it in large-scale federated learning tasks, significantly improve efficiency and reduce resource consumption, and has important practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 A flowchart of a federated learning training method based on model segmentation and resource allocation provided in an embodiment of the present application.

[0035] Figure 2 This is a comparison chart after a model segmentation ablation experiment was performed on the federated learning training method based on model segmentation and resource allocation provided in an embodiment of the present application.

[0036] Figure 3 This is a comparison chart after a resource configuration ablation experiment was conducted on the federated learning training method based on model segmentation and resource configuration provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] Figure 1 A flowchart of a federated learning training method based on model segmentation and resource allocation provided in an embodiment of the present application. An embodiment of the present application provides a federated learning training method based on model segmentation and resource allocation, including:

[0039] S100, obtaining local information of each client.

[0040] Exemplarily, the local information includes the client's computing resources and local data size d i ,When a client uploads its local information to the edge server, it indicates that the client is willing to participate in the ,federated learning task. After that, the edge server needs to initialize the ,global model based on the local information of all clients. The initialized global model can be used as the ,client model in subsequent use.

[0041] In an embodiment of the present application, the global model, or client model, may adopt a Resnet-18 model, and the global model may be initialized using the Kaiming initialization method.

[0042] S110, determining the training time of the client according to the local information, and establishing an objective function with the minimum training time as the goal.

[0043] For example, in order to simplify the problem and facilitate the solution, the time required for the objective function to converge is converted into a problem of minimizing the single-round training time. Because the global training time is the accumulation of all single-round training times, the global training time can be effectively reduced by minimizing the single-round training time. When establishing the objective function, the training time required for each client in a round of training is determined based on local information, and the maximum value of multiple training times is selected. The objective function is established with the goal of minimizing the maximum value.

[0044] The objective function is expressed as:

[0045]

[0046] Among them, min means taking the minimum value, H means the total number of training rounds, h means the number of training rounds, T h Indicates the training time of the i-th client in the h-th round of training The maximum training time of all clients including .

[0047] Training time Calculated by the following formula:

[0048]

[0049] in, It indicates the time required for the i-th client to complete the h-th round of training. Indicates the parameter aggregation time required by the i-th client. The expression is:

[0050]

[0051] Among them, τ i represents the number of iterations of the i-th client, b i represents the batch size of local data in the i-th client, l i represents the number of network layers of the underlying model in the i-th client, μ i (l i ) represents the time required for the underlying model in the i-th client to process a single sample, ν i (l i ) represents the time required for the i-th top model in the edge server to process a single sample, β i (l i ) represents the time required for the i-th client to transmit the intermediate result to the edge server, and They represent the time required for the i-th client to upload the client model to the edge server and the time required for the i-th client to download the client model from the edge server, respectively.

[0052]

[0053] Among them, e c (l i ) represents the number of floating-point calculations when the client model in the i-th client processes a single sample, represents the computing resources of the ith client, e s (l i ) represents the number of floating point calculations when the i-th top model in the edge server processes a single sample, represents the computing resources of the i-th top-level model in the edge server, g(l i ) represents the size of the intermediate result output by the underlying model in the i-th client, Represents the client model, Indicates the size of the client model, and They represent the upload bandwidth and download bandwidth of the i-th client respectively.

[0054] After establishing the objective function, there are the following constraints:

[0055] f(w H )-f(w * )≤ε

[0056]

[0057] Among them, w H represents the global model of the Hth round, w * represents the optimal global model, f(·) represents the model performance evaluation function, ε represents the convergence threshold, W h represents the average waiting time of the client in round h, N represents the number of clients, ξ represents the predefined training time threshold, C i represents the total computing resources of the i-th client, C represents the total computing resources of the edge server, and R D and R U They represent the download bandwidth and upload bandwidth of the edge server respectively. The first constraint condition indicates that after H rounds of training, the performance of the global model reaches the convergence requirement. The second constraint condition indicates that the average waiting time of the client cannot be too long. The third and fourth constraints represent the resource constraints of the client and edge server respectively. The fifth constraint condition represents the bandwidth constraint.

[0058] S120, solving the objective function to obtain a corresponding client model segmentation solution.

[0059] For example, when solving the objective function, the problem of solving the objective function is converted into model segmentation and resource allocation sub-problems, which are solved separately, and an alternating optimization method is used to find the optimal solution of the problem, which includes the optimal number of segmentation layers l i , batch size b i and resource configuration in edge servers The first two data come from the client model segmentation scheme, and the third data comes from the result of allocating computing resources on the edge server.

[0060] Due to the limitation of computing, bandwidth and other resources, edge servers can usually only serve some clients. Client model segmentation is to specify appropriate model segmentation and resource allocation schemes under the constraints of edge server performance and the goal of minimizing training time.

[0061] The time of the i-th client in a training round can be approximated as:

[0062]

[0063] After determining the structure of the client model, e c (l i )、e s (l i ) and g(l i ) can be obtained by calculation. For example, when the client model adopts the Resnet-18 model, the number of floating-point calculations increases linearly with the number of network layers in the model. Therefore, e c (l i ) and e s (l i ) can be approximately regarded as a function of l i The linear equation is expressed as and in and Respectively represent e c (l i ) and e s (l i ) is the result after approximation processing, where α1, α2 and α3 are the coefficients of each term respectively.

[0064] The size of the intermediate result is expressed as:

[0065]

[0066] Among them, S is the data type size of the parameter in the client model. For example, when the data type is single-precision floating point number FP32, the value of S is 4. It is the first i The number of channels of the layer, is the dimension of the feature map. From this expression, we can see that g(l i ) is a piecewise constant function. Therefore, the above formula can be rewritten as:

[0067]

[0068] For the model segmentation sub-problem, the greedy algorithm can be used to solve the objective function and obtain the optimal client segmentation solution. If the edge server does not allocate resources to a client, it means that the client does not participate in the segmentation training process and needs to train the complete client model locally. Otherwise, the number of model segmentation layers with the theoretical minimum training time is calculated according to the resource allocation plan and the above formula. Regardless of whether the client model participates in the segmentation training process, the bottom model and the top model are adjusted by adjusting the batch size b i Reduce training time.

[0069] For the resource allocation problem, Where i,j∈client set, but When the total computing resources of the edge server is C, λ satisfies:

[0070]

[0071] According to the client's computing resources, the current optimal client model segmentation scheme and other information, the shortest time λ that satisfies the server's maximum resource constraint is calculated using the above formula using the bisection method.

[0072] For one of the N clients, first calculate the size of the resource configuration in the edge server. If the size of the resource configuration exceeds the client's own computing resources, but the client still wants to participate in the model segmentation training, then its resource configuration size can be set to 0. This process needs to be performed on all clients, but only one client is removed at a time to ensure that the solution obtained is the optimal solution for the current situation.

[0073] S130, classify all clients according to the client model segmentation scheme, classify clients that can complete training under the limitation of their own computing resources into the first category of clients, and classify clients that cannot complete training under the limitation of their own computing resources into the second category of clients.

[0074] Exemplarily, the edge server performs a model segmentation operation based on the local information uploaded by the client to obtain the model segmentation results of each client. The client model has multiple layers l i ∈[1,2,…,L], where L represents the total number of layers in the client model. i=L, it means that the client model in the current client is exactly the same as the initialized global model. At this time, the client has sufficient computing resources to train the complete client model, so there is no need to perform model segmentation and resource configuration.

[0075] S140, segmenting the client model of the second type of client according to the client model segmentation scheme to obtain a top-level model and a bottom-level model.

[0076] For example, if the number of network layers l in the client model is less than L, it means that the client model in the current client is only a part of the initialized global model. At this time, the client does not have the ability to train a complete client model, so model segmentation and resource allocation are required. Therefore, the edge server will segment the complete client model into a bottom model and a top model, and send the bottom model to the corresponding client.

[0077] S150, allocating computing resources to each second-category client on the principle that the total allocated resources do not exceed the computing resources of the edge server.

[0078] For example, for clients with sufficient resources, the edge server will not allocate computing resources to them. Therefore, clients participating in model training can be divided into two categories: one is clients that train complete client models, and the other is clients that cannot train complete client models. The former follows the training process of federated learning, and the latter follows the training process of split federated learning. represents the computing resources configured by the i-th client on the edge server. The resource configuration constraints of the edge server can be expressed as:

[0079]

[0080] After the model segmentation and resource configuration phases are completed, the edge server distributes the client model, including the complete client model and the segmented underlying model, to each client.

[0081] S160, the first type of client uses its own local data to train the corresponding client model, and the second type of client first uses its own local data to train the corresponding underlying model, and then sends the intermediate results output by the underlying model to the edge server. The edge server uses the computing resources allocated to the second type of client to train the top-level model using the intermediate results.

[0082] For example, after starting training, all clients, including the first and second types of clients, will perform the forward propagation process in parallel, and the i-th client randomly selects a batch size of b i The local data and its sample labels can be expressed as X i and Yi The current batch of local data is propagated from the first layer of the network in the client model or the underlying model to the model segmentation layer, which is the last layer, to generate intermediate results. The size of g(l i ).

[0083] For the second type of client, the intermediate results output by the underlying model and the corresponding sample label Y i The edge server sends the intermediate results to the edge server. Input the top model and get the corresponding prediction label The predicted label and sample label Y i The difference between is used as the loss function. During the back propagation process, the parameters of the top model are adjusted according to the loss function to obtain the updated top model:

[0084]

[0085] in, represents the i-th top model updated after the h-th round of k-th iteration training, represents the i-th top model after the h-th round of training (k+1), η represents the learning rate, Represents the gradient calculated by the i-th top model.

[0086] After the parameters of the top-level model are adjusted, the edge server transmits the gradient of each layer during the parameter adjustment process of the top-level model to the bottom-level model. The bottom-level model calculates the gradient of each layer by back propagation and adjusts the parameters of the bottom-level model according to the calculated gradient.

[0087] Specifically, when the client receives the gradient transmitted by the edge server, it will perform a back-propagation process to calculate the gradient and update the parameters in the client model or the underlying model. For the i-th client, the updated client model or the underlying model is expressed as:

[0088]

[0089] in, Represents the gradient calculated by the i-th client model or the underlying model.

[0090] S170, after each round of training, the parameters of the client model in the first type of clients and the parameters of the bottom model and the top model corresponding to the second type of clients are aggregated to obtain current aggregated parameters.

[0091] Exemplarily, a federated average aggregation algorithm is used to aggregate the parameters of the client model and the parameters of the bottom model and the top model.

[0092] Specifically, after each training round, parameter aggregation is performed to generate a new global model by merging the parameters of all client models, bottom-level models, and top-level models. First, all clients upload the parameters of their local client models and bottom-level models to the edge server to participate in model aggregation. The edge server uses the federated average aggregation algorithm to generate the global model for the next training round. h represents the global model of the hth round. After multiple iterations, the global model is updated to w h+1 Therefore, the aggregation process of the edge server in the h+1th round can be expressed as:

[0093]

[0094] in, After parameter aggregation is completed, the edge server distributes the new round of global model to all clients for the next round of model training.

[0095] S180, using the current aggregation parameters as the parameters of the client model in the first type of clients and the parameters of the bottom model and the top model corresponding to the second type of clients, and performing the next round of training until the training is completed to obtain a trained global model.

[0096] The embodiment of the present application also provides a federated learning training system based on model segmentation and resource allocation, the system comprising:

[0097] The client information acquisition module is used to obtain the local information of each client;

[0098] An objective function establishment module is used to determine the training time of the client according to local information, and to establish an objective function with the minimum training time as the goal;

[0099] Function solving module, used to solve the target function and obtain the corresponding client model segmentation solution;

[0100] The client classification module is used to classify all clients according to the client model segmentation scheme, and classify clients that can complete training under the limitation of their own computing resources as first-class clients, and clients that cannot complete training under the limitation of their own computing resources as second-class clients;

[0101] A model segmentation module is used to segment the client model of the second type of client according to the client model segmentation scheme to obtain a top model and a bottom model, and send the top model to the edge server;

[0102] A resource allocation module, configured to allocate computing resources to each second-category client on the principle that the total allocated resources do not exceed the computing resources of the edge server;

[0103] The first model training module is used for the first type of client to train the corresponding client model using its own local data, and the second type of client first trains the corresponding underlying model using its own local data, and then sends the intermediate results output by the underlying model to the edge server, and the edge server uses the computing resources allocated to the second type of client to train the top model using the intermediate results;

[0104] A parameter aggregation module, used to aggregate the parameters of the client model in the first type of clients and the parameters of the bottom model and the top model corresponding to the second type of clients after each round of training to obtain the current aggregated parameters;

[0105] The second model training module is used to use the current aggregation parameters as the parameters of the client model in the first type of client and the parameters of the bottom model and the top model corresponding to the second type of client to perform the next round of training until the training is completed to obtain the trained global model.

[0106] In order to more intuitively illustrate the effect of the method in this application, the Resnet-18 deep neural network is used as the global model, and experiments are conducted on the settings of different heterogeneous scenarios and the public datasets Cifar-10 and Cifar-100. In the experiment, only the calculation time of forward propagation and backpropagation is recorded, and the time of reading data from the disk is not included. The number of client samples in the non-independent and identically distributed is set to [10, 15, 20, 25, 30]. For the Cifar-10 and Cifar-100 datasets, the convergence thresholds of the global model are set to 89% and 67% respectively.

[0107] The rest of the experimental settings are as follows:

[0108] Table 1 Other experimental parameters

[0109]

[0110]

[0111] Table 2. Training time required for global model convergence under different settings

[0112]

[0113] In the table, IID stands for independent and identically distributed, Non-IID stands for non-independent and identically distributed, AC stands for sufficient resource scenario, LC stands for limited resource scenario, MIX stands for mixed resource scenario, and ESFL stands for efficient segmentation federated learning. The experimental results show that compared with the existing methods, the method proposed in this application shows a faster convergence speed when conducting federated learning training.

[0114] In order to analyze the impact of model segmentation and resource allocation strategies on model training separately, Figure 2 and Figure 3 These are ablation experiments for model segmentation and resource allocation strategies. SFL and the method proposed in this application are compared. As shown in the figure, under the given experimental settings, the model segmentation strategy shows a stronger acceleration effect than the resource allocation strategy.

[0115] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0116] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A federated learning training method based on model segmentation and resource allocation, characterized in that: include: Get local information of each client; Determine the training time of the client according to the local information, and establish an objective function with the minimum training time as the goal; Solving the objective function to obtain a corresponding client model segmentation solution; Classifying all the clients according to the client model segmentation scheme, classifying the clients that can complete the training under the limitation of their own computing resources into the first category of clients, and classifying the clients that cannot complete the training under the limitation of their own computing resources into the second category of clients; Segmenting the client model of the second type of client according to the client model segmentation scheme to obtain a top-level model and a bottom-level model; Allocate computing resources to each of the second-category clients in accordance with the principle that the total allocated resources do not exceed the computing resources of the edge server; The first type of client uses its own local data to train the corresponding client model, the second type of client first uses its own local data to train the corresponding underlying model, and then sends the intermediate result output by the underlying model to the edge server, and the edge server uses the computing resources allocated to the second type of client to train the top model using the intermediate result; After each round of training, the parameters of the client model in the first type of clients and the parameters of the bottom model and the top model corresponding to the second type of clients are aggregated to obtain current aggregated parameters; The current aggregation parameters are used as parameters of the client model in the first type of clients and parameters of the bottom model and the top model corresponding to the second type of clients, and the next round of training is performed until the training is completed to obtain a trained global model.

2. The federated learning training method based on model segmentation and resource allocation according to claim 1, characterized in that: When establishing the objective function, the training time required for each client in a round of training is determined according to the local information, and the maximum value of multiple training times is selected, and the objective function is established with the goal of minimizing the maximum value.

3. The federated learning training method based on model segmentation and resource allocation according to claim 1, characterized in that: The intermediate results and corresponding sample labels output by the bottom model are sent to the edge server, and the edge server inputs the intermediate results into the top model to obtain corresponding predicted labels, uses the difference between the predicted labels and the sample labels as a loss function, and adjusts the parameters of the top model according to the loss function.

4. The federated learning training method based on model segmentation and resource allocation according to claim 3 is characterized in that: After the parameters of the top-level model are adjusted, the edge server transmits the gradient of each layer during the parameter adjustment process of the top-level model to the bottom-level model. The bottom-level model calculates the gradient of each layer by back propagation, and adjusts the parameters of the bottom-level model according to the calculated gradient.

5. The federated learning training method based on model segmentation and resource allocation according to claim 1, characterized in that: The parameters of the client model and the parameters of the bottom model and the top model are aggregated using a federated average aggregation algorithm.

6. The federated learning training method based on model segmentation and resource allocation according to claim 1, characterized in that: The objective function is expressed as: Among them, min means taking the minimum value, H means the total number of training rounds, h means the number of training rounds, T h Represents the maximum value of the training time of all the clients.

7. The federated learning training method based on model segmentation and resource allocation according to claim 1, characterized in that: The objective function is solved using a greedy algorithm to obtain the client model segmentation solution.

8. The federated learning training method based on model segmentation and resource allocation according to claim 1, characterized in that: The computing resources are allocated to each of the second-category clients using a binary search method.

9. A system using the federated learning training method based on model segmentation and resource allocation as described in any one of claims 1 to 8, characterized in that: include: The client information acquisition module is used to obtain the local information of each client; An objective function establishment module, used to determine the training time of the client according to the local information, and establish an objective function with the minimum training time as the goal; A function solving module, used to solve the objective function and obtain a corresponding client model segmentation solution; A client classification module, used to classify all the clients according to the client model segmentation scheme, classify the clients that can complete the training under the limitation of their own computing resources into the first category of clients, and classify the clients that cannot complete the training under the limitation of their own computing resources into the second category of clients; A model segmentation module, used to segment the client model of the second type of client according to the client model segmentation scheme to obtain a top model and a bottom model; a resource allocation module, configured to allocate computing resources to each of the second-category clients on the principle that the total allocated resources do not exceed the computing resources of the edge server; A first model training module is used for the first type of client to train the corresponding client model using its own local data, and the second type of client first trains the corresponding underlying model using its own local data, and then sends the intermediate result output by the underlying model to the edge server, and the edge server uses the computing resources allocated to the second type of client to train the top model using the intermediate result; A parameter aggregation module, used for aggregating the parameters of the client model in the first type of clients and the parameters of the bottom model and the top model corresponding to the second type of clients after each round of training to obtain current aggregated parameters; The second model training module is used to use the current aggregation parameters as the parameters of the client model in the first type of client and the parameters of the bottom model and the top model corresponding to the second type of client, to perform the next round of training until the training is completed to obtain a trained global model.

Images