Model optimization method based on quantitative splitting federated learning

Abstract

The invention discloses a model optimization method and system based on quantitative splitting federated learning, and relates to the technical field of customer edge computing, and the method comprises the following steps: S1, constructing an initial global model which comprises a server model and a client model; s2, adaptively selecting a split layer according to the resources of the client, and determining the time delay and energy consumption of the split layer under the initial global model; s3, constructing a global model optimization problem based on the initial global model; s4, obtaining a corresponding client transmission power strategy, a client quantification strategy and a client bandwidth allocation rate strategy; and S5, training the initial global model by using the customer data sample to obtain a trained global model. Under the condition that the model training time delay is reduced, energy consumption is reduced, client computing resources are saved, the aggregation precision of the global model is improved, and optimization of the global model is achieved.

Description

Technical Field

[0001] This invention relates to the field of customer edge computing technology, and more specifically to a model optimization method based on quantized split federated learning. Background Technology

[0002] With the rapid development of artificial intelligence technology, large-scale AI models (LAMs) have demonstrated powerful perception and reasoning capabilities in fields such as natural language processing, image recognition, and autonomous driving. However, since these models typically contain hundreds of millions of parameters, their deployment and training heavily rely on high computing power, high memory, and high-speed communication resources, posing significant challenges to their application on resource-constrained devices such as mobile terminals and in-vehicle systems.

[0003] To fully utilize users' local private data while reducing communication overhead, Federated Learning (FL), as a decentralized distributed collaborative training paradigm, has been widely studied. It optimizes the global model by deploying the model locally on the terminal and uploading only model parameters or gradients to the server for aggregation, thus protecting privacy. However, traditional federated learning typically requires the terminal device to deploy the entire large model, which is difficult to meet the computing power and storage requirements in resource-constrained scenarios.

[0004] To alleviate the aforementioned problems, Split Learning (SL) was proposed. This method splits a deep model into two parts: a device-side model and a server-side model. The front-end model runs locally on the device, while the back-end model is handled by the server, effectively reducing the computational and memory burden on the device. Building on this, the Split Federated Learning (SFL) architecture, formed by combining FL and SL, further improves training efficiency and privacy protection capabilities, and is considered a more suitable framework for collaborative training of large models in edge intelligence scenarios.

[0005] However, existing SFL methods still suffer from the following technical bottlenecks: First, the frequent transmission of intermediate activation features between the terminal and the server generates a large communication load, limiting the scalability of the system in bandwidth-constrained or mobile networks; second, the lack of fine-grained awareness of heterogeneous terminal resources (such as computing frequency, communication bandwidth, and transmission power) during model training easily leads to high system energy consumption and large latency, making it difficult to meet the system requirements of real-time performance and low power consumption; third, most existing works adopt a fixed splitting strategy, failing to fully utilize the model structure characteristics and terminal device resource status for joint optimization.

[0006] Therefore, designing a collaborative training method for large models within the SFL framework that can both efficiently compress intermediate communication data and jointly optimize resource allocation has become a key technical challenge for realizing the application of large models in edge intelligence scenarios. Summary of the Invention

[0007] To address the above problems, this invention proposes a model optimization method based on quantized decomposition federated learning.

[0008] The technical solution of this invention is: a model optimization method based on quantized decomposition federated learning, comprising the following steps:

[0009] S1. Construct the initial global model, which includes the server-side model and the client-side model;

[0010] S2. Based on the client's resources, adaptively select the splitting layer and determine the latency and energy consumption of the splitting layer under the initial global model;

[0011] S3. Based on the initial global model, construct a global model optimization problem;

[0012] S4. The global model optimization problem is broken down into terminal transmission power optimization sub-problem, quantization bit width management sub-problem and bandwidth resource allocation sub-problem, and solved. Based on the latency and energy consumption of the initial global model, the corresponding customer transmission power strategy, customer quantization strategy and customer bandwidth allocation rate strategy are obtained.

[0013] S5. Based on the customer transmission power strategy, customer quantization strategy and customer bandwidth allocation rate strategy, configure the parameters of the initial global model, and train the initial global model using customer data samples to obtain the trained global model.

[0014] Furthermore, in S1, the server-side model is deployed on the edge cloud server, and the client-side model is deployed on the client, with one client-side model deployed on one client.

[0015] Loss function of the initial global model The expression is:

[0016]

[0017]

[0018] In the formula, L n (ω) represents the loss function of the client-side model, L(ω) represents the global loss function, N represents the number of clients participating in training, and ρ n This represents the ratio of the data volume of the nth customer to the total data volume of all customers, and ω represents the global model. Represents the customer dataset. Let l represent the i-th data sample of the n-th customer, and l(·) represent the loss function for a data sample of the customer.

[0019] Furthermore, S2 includes the following sub-steps:

[0020] S21. Based on the client's resources, adaptively select the splitting layer and determine the modules involved in the client's workflow in the initial global model, specifically including the model unloading module, client model training module, edge communication module, server model training module, and model upload and aggregation module.

[0021] S22. Based on several split layers, calculate the energy consumption of each module according to the latency of each module;

[0022] S23. Determine the latency and energy consumption of the initial global model based on the latency and energy consumption of each module.

[0023] The latency of the model unloading module is negligible; due to sufficient downlink bandwidth, the latency and energy consumption of the client model uploading and aggregation modules are negligible.

[0024] Furthermore, the latency of the client-side model training module The calculation formula is:

[0025]

[0026] In the formula, γ d (·) represents the computation required by the client model to process a data sample at the split layer, b represents the number of mini-batch samples in each training round, and f n This indicates the clock speed of the GPU installed in the device, C. n D indicates the number of cores in the GPU installed in the device. n This represents the number of floating-point operations that each core can complete per cycle, ∈ n Indicates the split point;

[0027] Energy consumption of the client-side model training module The calculation formula is:

[0028]

[0029] In the formula, κ1 represents the power consumption coefficient;

[0030] latency of edge communication module The calculation formula is:

[0031]

[0032] In the formula, s n,k This represents the amount of data transmitted by the nth customer in the kth round. Indicates the uplink communication speed of the edge cloud server;

[0033] Energy consumption of edge communication module The expression is:

[0034]

[0035] In the formula, This represents the uplink transmission power of the nth customer;

[0036] Server-side model training module latency The expression is:

[0037]

[0038] In the formula, γ s (·) represents the computation required by the server-side model at the split layer when processing a data sample, f s Indicates the clock speed of the GPU used in the server, C s D represents the number of GPU cores used by the server. s This indicates the number of floating-point operations supported per core per cycle.

[0039] In S23, the client's latency T in the initial global model n，k The expression is:

[0040]

[0041] In S23, the customer's energy consumption E in the initial global model n，k The calculation formula is:

[0042]

[0043] Furthermore, in S3, the expression for the global model optimization problem is:

[0044]

[0045] In the formula, The question name is represented by z, the communication resource block variable is represented by p, the transmit power variable is represented by q, and the quantization bit depth variable is represented by q. T represents system efficiency. n，k T represents the latency of the client in the initial global model. max Indicates the upper bound of the time threshold. Represents a customer set. Indicates whether a resource block is allocated to a device; M represents the number of resource blocks. p represents the set of resource block numbers. min q represents the minimum transmit power. min p represents the minimum number of quantization bits.n，k q represents the device's upload and transmit power in the k-th round. n，k p represents the activation quantization bit width used by the device in the k-th round. max q represents the maximum transmit power. max Indicates the maximum number of quantization bits, a n，k The indicator variable representing whether the device participates in the k-th round of training.

[0046] Furthermore, in S4, the expression for the customer transmission power strategy is:

[0047]

[0048] In the formula, Let p represent a non-convex optimization problem, where p represents the transmit power variable. The value represents the energy consumption of the client-side model training module, B represents the subcarrier bandwidth, log2(·) represents the logarithmic function to the base 2, and s n Indicates the size of the uploaded data, p n Indicates the device's upload power, h o d represents the channel gain. n，k N represents the distance between the device and the server, γ represents the path loss exponent, and N represents the distance between the device and the server. o T represents thermal noise power. max p represents the upper bound of the time threshold. min p represents the minimum transmit power variable. max This represents the maximum transmit power variable;

[0049] In S4, the expression for the customer quantification strategy is:

[0050]

[0051] In the formula, This represents a quasi-convex optimization problem, where q represents the quantization bit number variable. Indicates system efficiency. Denotes a customer set. n，k The indicator variable representing whether device n participates in the k-th round of training, log(·) represents the logarithmic function, and q n，k This indicates the activation quantization bit width used by device n in the k-th round. This indicates the power consumption of the edge communication module;

[0052] In S4, the expression for the customer bandwidth allocation rate strategy is:

[0053]

[0054] In the formula, Let z represent the minimum weight perfect bipartite graph matching problem. k This represents the subcarrier allocation variable for the k-th round. Indicates whether a resource block is allocated to a device, q n，k E represents the activation quantization bit width representation used by the device in the k-th round. n，k This represents the energy consumption of a client in the initial global model, M represents the number of resource blocks, and N represents the number of clients participating in training.

[0055] Furthermore, in S4, the continuous convex approximation algorithm is used to solve the terminal transmission power optimization subproblem; the greedy search algorithm is used to select the optimal bit width for each device to solve the quantization bit width management subproblem; the bandwidth resource allocation subproblem is transformed into a minimum weight bipartite graph matching problem, and the linear programming algorithm or the Hungarian algorithm is used to solve the bandwidth resource allocation subproblem.

[0056] The beneficial effects of this invention are as follows: Based on decomposed federated learning, this invention constructs a global model optimization problem on the basis of global model decomposition. The global model optimization problem is decomposed into a terminal transmission power optimization sub-problem, a quantization bit width management sub-problem, and a bandwidth resource allocation sub-problem, which are solved to obtain the customer transmission power strategy, customer quantization strategy, and customer bandwidth allocation rate strategy. According to the customer transmission power strategy, customer quantization strategy, and customer bandwidth allocation rate strategy, the parameters of the global model are configured. Then, according to the customer selection strategy, customer samples are screened. The screened customer data is used as samples to import into the constructed initial global model for model training, resulting in a trained global model. This solves the shortcomings of traditional serial decomposed learning, realizes parallel training of the global model, reduces energy consumption and saves customer computing resources while reducing model training latency, improves the aggregation accuracy of the global model, and achieves optimization of the global model.

[0057] Based on the above methods, this invention also proposes a model optimization system based on quantized decomposition federated learning, including an initial global model construction module, a latency and energy consumption calculation module, an optimization problem construction module, an optimization problem solving module, and a model training module;

[0058] The initial global model building module is used to build the initial global model, which includes a server-side model and a client-side model.

[0059] The latency and energy consumption calculation module is used to adaptively select the splitting layer based on the client's resources and determine the latency and energy consumption of the splitting layer under the initial global model;

[0060] The optimization problem building module is used to construct a global model optimization problem based on the initial global model;

[0061] The optimization problem-solving module is used to decompose the global model optimization problem into terminal transmission power optimization sub-problems, quantization bit width management sub-problems, and bandwidth resource allocation sub-problems, solve them, and obtain the corresponding customer transmission power strategy, customer quantization strategy, and customer bandwidth allocation rate strategy based on the latency and energy consumption of the initial global model.

[0062] The model training module is used to configure the parameters of the initial global model based on the customer transmission power strategy, customer quantization strategy, and customer bandwidth allocation rate strategy, and to train the initial global model using customer data samples to obtain the trained global model.

[0063] The beneficial effects of this invention are: the system of this invention reduces energy consumption and saves customer computing resources while reducing model training latency, improves the aggregation accuracy of the global model, and achieves optimization of the global model. Attached Figure Description

[0064] Figure 1 This is a flowchart of a model optimization method based on quantized decomposition federated learning.

[0065] Figure 2 This is a schematic diagram of the structure of a model optimization system based on quantized split federated learning. Detailed Implementation

[0066] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0067] like Figure 1 As shown, this invention provides a model optimization method based on quantized decomposition federated learning, comprising the following steps:

[0068] S1. Construct the initial global model, which includes the server-side model and the client-side model;

[0069] S2. Based on the client's resources, adaptively select the splitting layer and determine the latency and energy consumption of the splitting layer under the initial global model;

[0070] S3. Based on the initial global model, construct a global model optimization problem;

[0071] S4. The global model optimization problem is broken down into terminal transmission power optimization sub-problem, quantization bit width management sub-problem and bandwidth resource allocation sub-problem, and solved. Based on the latency and energy consumption of the initial global model, the corresponding customer transmission power strategy, customer quantization strategy and customer bandwidth allocation rate strategy are obtained.

[0072] S5. Based on the customer transmission power strategy, customer quantization strategy and customer bandwidth allocation rate strategy, configure the parameters of the initial global model, and train the initial global model using customer data samples to obtain the trained global model.

[0073] In this embodiment of the invention, in S1, the server-side model is deployed on an edge cloud server, and the client-side model is deployed on a client, with one client-side model deployed on one client.

[0074] Loss function of the initial global model The expression is:

[0075]

[0076]

[0077] In the formula, L n (ω) represents the loss function of the client-side model, L(ω) represents the global loss function, N represents the number of clients participating in training, and ρ n This represents the ratio of the data volume of the nth customer to the total data volume of all customers, and ω represents the global model. Represents the customer dataset. Let l represent the i-th data sample of the n-th customer, and l(·) represent the loss function for a data sample of the customer.

[0078] In this embodiment of the invention, S2 includes the following sub-steps:

[0079] S21. Based on the client's resources, adaptively select the splitting layer and determine the modules involved in the client's workflow in the initial global model, specifically including the model unloading module, client model training module, edge communication module, server model training module, and model upload and aggregation module.

[0080] S22. Based on several split layers, calculate the energy consumption of each module according to the latency of each module;

[0081] S23. Determine the latency and energy consumption of the initial global model based on the latency and energy consumption of each module.

[0082] After constructing the initial global model, the edge cloud server obtains hardware capability information for each client, including but not limited to the device's video memory capacity, GPU clock speed, number of cores, current load status, and remaining available memory. Based on the above resource status, the cloud server calculates the resource consumption of each candidate model splitting layer on different clients, evaluates its video memory consumption, forward computation latency, and communication load, and selects the optimal splitting layer strategy that meets constraints (such as video memory threshold and maximum tolerable computation latency) so that the client does not exceed local resource limits when running the front-end model. The upper bound combination of all client's optional splitting layers is E = {1, ..., ∈ n}

[0083] In this embodiment of the invention, the latency of the client-side model training module... The calculation formula is:

[0084]

[0085] In the formula, γ d (·) represents the computation required by the client model to process a data sample at the split layer, b represents the number of mini-batch samples in each training round, and f n This indicates the clock speed of the GPU installed in the device, C. n D indicates the number of cores in the GPU installed in the device. n This represents the number of floating-point operations that each core can complete per cycle, ∈ n Indicates the split point; the total computational cost of each round of local training on the device is bγ. d .

[0086] Energy consumption of the client-side model training module The calculation formula is:

[0087]

[0088] In the formula, κ1 represents the power consumption coefficient; the power consumption coefficient reflects the energy efficiency characteristics of the device architecture;

[0089] latency of edge communication module The calculation formula is:

[0090]

[0091] In the formula, s n，k This represents the amount of data transmitted by the nth customer in the kth round. Indicates the uplink communication speed of the edge cloud server;

[0092] Energy consumption of edge communication module The expression is:

[0093]

[0094] In the formula, This represents the uplink transmission power of the nth customer;

[0095] Server-side model training module latency The expression is:

[0096]

[0097] In the formula, γ s (·) represents the computation required by the server-side model at the split layer when processing a data sample, f s Indicates the clock speed of the GPU used in the server, C sD represents the number of GPU cores used by the server. s This represents the number of floating-point operations supported per core per cycle; the total computational cost of the server in each training round is bγ. s .

[0098] In S23, the client's latency T in the initial global model n，k The expression is:

[0099]

[0100] In S23, the customer's energy consumption E in the initial global model n，k The calculation formula is:

[0101]

[0102] In this embodiment of the invention, in S3, the expression for the global model optimization problem is:

[0103]

[0104] In the formula, The question name is represented by z, the communication resource block variable is represented by p, the transmit power variable is represented by q, and the quantization bit depth variable is represented by q. T represents system efficiency. n，k T represents the latency of the client in the initial global model. max Indicates the upper bound of the time threshold. Represents a customer set. Indicates whether a resource block is allocated to a device; M represents the number of resource blocks. p represents the set of resource block numbers. min q represents the minimum transmit power. min p represents the minimum number of quantization bits. n，k q represents the device's upload and transmit power in the k-th round. n，k p represents the activation quantization bit width used by the device in the k-th round. max q represents the maximum transmit power. max Indicates the maximum number of quantization bits, a n，k The indicator variable representing whether the device participates in the k-th round of training.

[0105] In this embodiment of the invention, in S4, given the resource block allocation strategy z, it can be found that the transmission power selection strategies of different devices do not affect each other and each contributes independently to the system objective function. Therefore, the problem of minimizing the total system energy consumption can be equivalently transformed into the problem of minimizing the energy consumption of each device separately. The expression for the customer transmission power strategy is:

[0106]

[0107] In the formula, Let p represent a non-convex optimization problem, where p represents the transmit power variable. The value represents the energy consumption of the client-side model training module, B represents the subcarrier bandwidth, log2(·) represents the logarithmic function to the base 2, and s n Indicates the size of the uploaded data, p n Indicates the device's upload power, h o d represents the channel gain. n，k N represents the distance between the device and the server, γ represents the path loss exponent, and N represents the distance between the device and the server. o T represents thermal noise power. max p represents the upper bound of the time threshold. min p represents the minimum transmit power variable. max This represents the maximum transmit power variable;

[0108] In S4, for each device in each training round, given the transmission power and resource block allocation strategy, this invention proposes to independently optimize the quantization bit width to improve system efficiency and meet latency constraints. The goal is to find the optimal number of quantization bits for each device, thereby minimizing system cost while ensuring model performance. In this problem, the objective function is a fractional function, with the numerator relating to q. n，k Monotonically increasing, the communication energy consumption term in the denominator is related to q. n，k The problem exhibits a monotonically decreasing trend, thus displaying a quasi-convex structure and easily solvable through exhaustive search or greedy algorithm. The expression for the customer quantification strategy is:

[0109]

[0110] In the formula, This represents a quasi-convex optimization problem, where q represents the number of quantization bits. Indicates system efficiency. Denotes a customer set. n，k The indicator variable representing whether the device participates in the k-th round of training, log(·) represents the logarithmic function, and q n，k This indicates the activation quantization bit width used by the device in the k-th round. This indicates the power consumption of the edge communication module;

[0111] In S4, to achieve optimal matching between devices and uplink resource blocks under limited resource block conditions, this invention designs a bandwidth resource allocation subproblem to maximize system efficiency while satisfying system constraints. Its objective is to optimize the resource block allocation strategy to maximize the total model performance per unit cost. This optimization problem is a typical nonlinear integer programming problem with high solution complexity. Therefore, this invention equivalently transforms it into a minimum-weight perfect bipartite graph matching problem and finds the optimal solution in polynomial time. The expression for the customer bandwidth allocation rate strategy is:

[0112]

[0113] In the formula, Let z represent the minimum weight perfect bipartite graph matching problem. k This represents the subcarrier allocation variable for the k-th round. Indicates whether a resource block is allocated to a device, q n，k E represents the activation quantization bit width representation used by the device in the k-th round. n,k This represents the energy consumption of a client in the initial global model, M represents the number of resource blocks, and N represents the number of clients participating in training.

[0114] In this embodiment of the invention, in S4, the continuous convex approximation algorithm is used to solve the terminal transmission power optimization subproblem; the greedy search is used to select the optimal bit width for each device to solve the quantization bit width management subproblem; the bandwidth resource allocation subproblem is transformed into a minimum weight bipartite graph matching problem, and the linear programming algorithm or the Hungarian algorithm is used to solve the bandwidth resource allocation subproblem.

[0115] In this embodiment of the invention, the optimal scheduling scheme for system resources is obtained by matching the minimum cost between devices and resource blocks, thus ensuring improved system efficiency and reduced energy consumption.

[0116] In embodiments of the present invention, based on the above methods, such as Figure 2 As shown, this invention also proposes a model optimization system based on quantized decomposition federated learning, including an initial global model construction module, a latency and energy consumption calculation module, an optimization problem construction module, an optimization problem solving module, and a model training module;

[0117] The initial global model building module is used to build the initial global model, which includes a server-side model and a client-side model.

[0118] The latency and energy consumption calculation module is used to adaptively select the splitting layer based on the client's resources and determine the latency and energy consumption of the splitting layer under the initial global model;

[0119] The optimization problem building module is used to construct a global model optimization problem based on the initial global model;

[0120] The optimization problem-solving module is used to decompose the global model optimization problem into terminal transmission power optimization sub-problems, quantization bit width management sub-problems, and bandwidth resource allocation sub-problems, solve them, and obtain the corresponding customer transmission power strategy, customer quantization strategy, and customer bandwidth allocation rate strategy based on the latency and energy consumption of the initial global model.

[0121] The model training module is used to configure the parameters of the initial global model based on the customer transmission power strategy, customer quantization strategy, and customer bandwidth allocation rate strategy, and to train the initial global model using customer data samples to obtain the trained global model.

[0122] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A model optimization method based on quantized decomposition federated learning, characterized in that, Includes the following steps: S1. Construct an initial global model, which includes a server-side model and a client-side model; S2. Based on the client's resources, adaptively select the splitting layer and determine the latency and energy consumption of the splitting layer under the initial global model; S3. Based on the initial global model, construct a global model optimization problem; S4. The global model optimization problem is broken down into terminal transmission power optimization sub-problem, quantization bit width management sub-problem and bandwidth resource allocation sub-problem, and solved. Based on the latency and energy consumption of the initial global model, the corresponding customer transmission power strategy, customer quantization strategy and customer bandwidth allocation rate strategy are obtained. S5. Based on the customer transmission power strategy, customer quantization strategy and customer bandwidth allocation rate strategy, configure the parameters of the initial global model, and train the initial global model using customer data samples to obtain the trained global model.

2. The model optimization method based on quantized decomposition federated learning according to claim 1, characterized in that, In S1, the server-side model is deployed on an edge cloud server, and the client-side model is deployed on a client, with one client-side model deployed on one client. The loss function of the initial global model The expression is: In the formula, L n (ω) represents the loss function of the client-side model, L(ω) represents the global loss function, N represents the number of clients participating in training, and ρ n This represents the ratio of the data volume of the nth customer to the total data volume of all customers, and ω represents the global model. Represents the customer dataset. Let l represent the i-th data sample of the n-th customer, and l(·) represent the loss function for a data sample of the customer.

3. The model optimization method based on quantized decomposition federated learning according to claim 1, characterized in that, S2 includes the following sub-steps: S21. Based on the client's resources, adaptively select the splitting layer and determine the modules involved in the client's workflow in the initial global model, specifically including the model unloading module, client model training module, edge communication module, server model training module, and model upload and aggregation module. S22. Based on several split layers, calculate the energy consumption of each module according to the latency of each module; S23. Determine the latency and energy consumption of the initial global model based on the latency and energy consumption of each module.

4. The model optimization method based on quantized decomposition federated learning according to claim 3, characterized in that, The latency of the client-side model training module The calculation formula is: In the formula, γ d (·) represents the computation required by the client model to process a data sample at the split layer, b represents the number of mini-batch samples in each training round, and f n This indicates the clock speed of the GPU installed in the device, C. n D indicates the number of cores in the GPU installed in the device. n This represents the number of floating-point operations that each core can complete per cycle, ∈ n Indicates the split point; Energy consumption of the client-side model training module The calculation formula is: In the formula, κ1 represents the power consumption coefficient; The latency of the edge communication module The calculation formula is: In the formula, s n，k This represents the amount of data transmitted by the nth customer in the kth round. Indicates the uplink communication speed of the edge cloud server; The power consumption of the edge communication module The expression is: In the formula, This represents the uplink transmission power of the nth customer; The latency of the server-side model training module The expression is: In the formula, γ s (·) represents the computation required by the server-side model at the split layer when processing a data sample, f s Indicates the clock speed of the GPU used in the server, C s D represents the number of GPU cores used by the server. s This indicates the number of floating-point operations supported per core per cycle. In S23, the client's latency T in the initial global model n，k The expression is: In S23, the customer's energy consumption E in the initial global model n，k The calculation formula is:

5. The model optimization method based on quantized decomposition federated learning according to claim 1, characterized in that, In S3, the expression for the global model optimization problem is: In the formula, The question name is represented by z, the communication resource block variable is represented by p, the transmit power variable is represented by q, and the quantization bit depth variable is represented by q. T represents system efficiency. n，k T represents the latency of the client in the initial global model. max Indicates the upper bound of the time threshold. Represents a customer set. Indicates whether a resource block is allocated to a device; M represents the number of resource blocks. p represents the set of resource block numbers. min q represents the minimum transmit power. min p represents the minimum number of quantization bits. n，k q represents the device's upload and transmit power in the k-th round. n，k p represents the activation quantization bit width used by the device in the k-th round. max q represents the maximum transmit power. max Indicates the maximum number of quantization bits, a n，k The indicator variable representing whether the device participates in the k-th round of training.

6. The model optimization method based on quantized decomposition federated learning according to claim 1, characterized in that, In step S4, the expression for the customer transmission power strategy is: p min ≤p n ≤p max . In the formula, Let p represent a non-convex optimization problem, where p represents the transmit power variable. The value represents the energy consumption of the client-side model training module, B represents the subcarrier bandwidth, log2(·) represents the logarithmic function to the base 2, and s n Indicates the size of the uploaded data, p n Indicates the device's upload power, h o d represents the channel gain. n，k N represents the distance between the device and the server, γ represents the path loss exponent, and N represents the distance between the device and the server. o T represents thermal noise power. max p represents the upper bound of the time threshold. min p represents the minimum transmit power variable. max This represents the maximum transmit power variable; In S4, the expression for the customer quantification strategy is: In the formula, This represents a quasi-convex optimization problem, where q represents the number of quantization bits. Indicates system efficiency. Denotes a customer set. n，k The indicator variable representing whether the device participates in the k-th round of training, log(·) represents the logarithmic function, and q n，k This indicates the activation quantization bit width used by the device in the k-th round. This indicates the power consumption of the edge communication module; In S4, the expression for the customer bandwidth allocation rate strategy is: In the formula, Let z represent the minimum weight perfect bipartite graph matching problem. k This represents the subcarrier allocation variable for the k-th round. Indicates whether a resource block is allocated to a device, q n，k E represents the activation quantization bit width representation used by the device in the k-th round. n,k This represents the energy consumption of a client in the initial global model, M represents the number of resource blocks, and N represents the number of clients participating in training.

7. The model optimization method based on quantized decomposition federated learning according to claim 1, characterized in that, In step S4, the continuous convex approximation algorithm is used to solve the terminal transmission power optimization subproblem; and the greedy search algorithm is used to select the optimal bit width for each device to solve the quantization bit width management subproblem. The bandwidth resource allocation subproblem is transformed into a minimum weighted bipartite graph matching problem, and then solved using a linear programming algorithm or the Hungarian algorithm.

8. A model optimization system based on quantized decomposition federated learning, characterized in that, It includes an initial global model building module, a latency and energy consumption calculation module, an optimization problem building module, an optimization problem solving module, and a model training module; The initial global model construction module is used to construct an initial global model, which includes a server-side model and a client-side model. The latency and energy consumption calculation module is used to adaptively select the splitting layer based on the client's resources and determine the latency and energy consumption of the splitting layer under the initial global model. The optimization problem construction module is used to construct a global model optimization problem based on the initial global model; The optimization problem solving module is used to decompose the global model optimization problem into terminal transmission power optimization sub-problems, quantization bit width management sub-problems, and bandwidth resource allocation sub-problems, solve them, and obtain the corresponding customer transmission power strategy, customer quantization strategy, and customer bandwidth allocation rate strategy based on the latency and energy consumption of the initial global model. The model training module is used to configure the parameters of the initial global model based on the customer transmission power strategy, customer quantization strategy and customer bandwidth allocation rate strategy, and to train the initial global model using customer data samples to obtain the trained global model.

Images