Calculation power self-adaptive training model distribution method and system and electronic equipment
By optimizing the split points and resource allocation within the segmented federated learning framework, and using the DDQN algorithm to optimize the selection of model split points, the problems of high computational resource requirements and large communication overhead in edge networks are solved, achieving efficient model convergence and low latency.
Patent Information
- Application Number
- CN202511059885.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-30
- Publication Date
- 2025-11-25
AI Technical Summary
Traditional federated learning in edge networks has high computational resource requirements and large communication overhead, and the transmission of model parameters leads to exponential growth, affecting model convergence and efficiency.
We adopt a computationally adaptive training model allocation method. By constructing a convergence rate and latency optimization problem under the segmented federated learning framework, we use the DDQN algorithm to optimize the selection of split points and resource allocation, and combine deep reinforcement learning to optimize model split points and resource allocation.
It effectively improves the model's convergence performance and computational communication efficiency, reduces latency, and ensures the model's convergence and resource utilization efficiency.
Smart Images
Figure CN121009302A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of communication, resource optimization, and partitioned federated learning, and more specifically, to a method, system, and electronic device for adaptive training model allocation based on computing power. Background Technology
[0002] With the widespread adoption of emerging artificial intelligence services in next-generation wireless edge networks, Federated Learning (FL) has emerged as a distributed machine learning paradigm. By training models on local devices and sharing only model parameters rather than raw data, it effectively addresses data privacy and compliance issues. However, traditional FL suffers from two significant drawbacks: (1) the edge device must perform the entire model training, placing high demands on computing resources; and (2) the communication overhead from model parameter transmission increases exponentially with model complexity. To overcome these limitations, Split Federated Learning (SFL), which divides the learning model into server-side and client-side models, has become a highly attractive technology that can address the heavy computational burden on network edge clients.
[0003] In the SFL architecture, the choice of cut layer directly affects the system's computational load distribution, communication efficiency, and privacy. Some papers have theoretically analyzed the impact of cut layer selection on convergence rate, revealing that model segmentation with a smaller client model size leads to better convergence performance. Meanwhile, in edge systems, the client's wireless channel conditions and heterogeneous computing capabilities can significantly affect SFL latency.
[0004] Therefore, to ensure the convergence of the model and the efficiency of computation and communication, this invention, while considering resource and privacy constraints, formulates a convergence rate and delay optimization problem, and develops an optimization strategy for joint split points, communication and computational resources based on the DDQN algorithm. Summary of the Invention
[0005] This invention addresses the technical problems existing in the prior art by providing a computationally adaptive training model allocation method, system, and electronic device. By jointly considering the selection of split points and the allocation of computational and communication resources, the convergence rate and latency of the model are optimized, effectively improving the convergence performance and computational and communication efficiency of the model.
[0006] According to a first aspect of the present invention, a computationally adaptive training model allocation method is provided, comprising: An optimization problem is constructed within the framework of segmented federated learning, taking into account both convergence rate and model training latency. By introducing auxiliary variables, the optimization problem is decomposed into a resource allocation subproblem and a split point selection subproblem. The split point selection subproblem is modeled as a Markov decision process, and a deep reinforcement learning algorithm is used to solve for the optimal policy, thereby obtaining the optimal parameter vector to realize the optimal split point selection policy.
[0007] Based on the above technical solution, the present invention can also be improved as follows.
[0008] Optionally, the segmentation federated learning framework is a segmentation federated learning method based on aggregated gradient broadcasting, which includes: a server and N client devices jointly training a global model, and the dataset is owned locally by the client devices and is not shared.
[0009] Optionally, the optimization problem that comprehensively considers the convergence rate and model training latency is a problem of minimizing the weighted sum of convergence and latency indices, thereby achieving a trade-off between convergence and latency. This includes: analyzing the convergence of segmentation federated learning based on aggregated gradient broadcasting, describing the upper bound of the difference between segmentation federated learning based on aggregated gradient broadcasting and the traditional segmentation federated learning architecture, and constructing a weighted problem P1 for convergence and latency indices.
[0010] Optionally, the introduced auxiliary variables are the upper bound of the computational delay and the upper bound of the communication delay during training. After their introduction, the delay problem is expressed as minimizing the upper bound of the delay, and a delay constraint is given.
[0011] Optionally, the resource allocation subproblem is a convex optimization problem, which is solved using the CVX method. By optimizing the allocation of communication resources, the introduction of auxiliary variables is minimized.
[0012] Optionally, the subproblem of selecting the split point is an integer programming problem, and optimal convergence performance is achieved by optimizing the model split point.
[0013] Optionally, modeling the split point selection subproblem as a Markov decision process and using a deep reinforcement learning algorithm to solve for the optimal policy includes: The DDQN algorithm is used to transform the integer programming problem into a Markov decision process problem, which includes a state space S, an action space A, state transition probabilities P, and a reward R. Here, the state space S represents the channel gain observed at the beginning of the current communication round, the action space A represents all possible choices of the model split point, the state transition probability P represents the probability of state transition under a certain action, and the reward R represents the quality of the learning policy under the current state and action, and is represented by the sum of the convergence index and the delay auxiliary variable. Define the state action function of the DDQN algorithm. , used to represent the strategy The expected long-term rewards, among which, Indicates state, Indicates discrete actions. It is the parameter vector of the Q network; Define a loss function, and optimize the parameters by minimizing the loss function. ; By obtaining the optimal parameter vector To achieve the optimal split point selection strategy.
[0014] Optionally, the strategy for obtaining the optimal parameter vector to achieve the optimal split point selection includes: Initialize the parameter vector, experience buffer, and maximum number of iterations for the Q-network; Perform the following operations in each iteration: Reset the initial state; Perform T rounds of communication to update the Q network parameters, including, for the t-th communication round: The DDQN agent determines the state based on the observed state. Select discrete actions ; Solve the resource allocation subproblem to obtain the optimal communication and computing resource allocation strategy; Calculate the reward based on the optimal resource allocation strategy for this round; Observe the next state ; intermediate experience Add to the experience buffer; Sample a small batch from the experience buffer; Update the DQN network using gradient descent. .
[0015] According to a second aspect of the present invention, a computationally adaptive training model allocation system is provided, comprising: The optimization problem construction module is used to construct optimization problems that take into account both convergence rate and model training latency within the segmented federated learning framework. The optimization problem decomposition module is used to introduce auxiliary variables to decompose the optimization problem into resource allocation sub-problems and split point selection sub-problems. The optimal policy solution module is used to model the split point selection subproblem as a Markov decision process and use a deep reinforcement learning algorithm to solve for the optimal policy, thereby obtaining the optimal parameter vector to realize the optimal split point selection policy.
[0016] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the processor is configured to implement a computationally adaptive training model allocation method when executing a computer program stored in the memory.
[0017] The technical effects and advantages of this invention are as follows: This invention provides a computationally adaptive training model allocation method, system, and electronic device. By considering the wireless network environment, heterogeneous computing capabilities, and privacy constraints, it adaptively and dynamically selects model split points and allocates training resources. By jointly considering split point selection and computational communication resource allocation, it optimizes the model's convergence rate and latency, effectively ensuring model convergence and reducing latency.
[0018] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description
[0019] Figure 1 A flowchart of a computational power adaptive training model allocation method provided in an embodiment of the present invention; Figure 2 A diagram illustrating the solution process of the DDQN algorithm provided in this embodiment of the invention; Figure 3 This is a schematic diagram showing the accuracy and convergence rate results of different node segmentation schemes provided in the embodiments of the present invention; Figure 4 This diagram illustrates the accuracy and latency under different resource allocation strategies provided in embodiments of the present invention. Detailed Implementation
[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0021] Understandably, based on the deficiencies in the background technology, this embodiment of the invention proposes a computationally adaptive training model allocation method, specifically as follows: Figure 1 As shown, it includes the following steps: Step S1: Construct an optimization problem that comprehensively considers convergence rate and model training latency within the segmented federated learning framework; For details on the partitioned federated learning framework, please refer to [link / reference]. Figure 2 As shown, the segmentation federated learning framework is a segmentation federated learning method based on aggregated gradient broadcasting. Specifically, a server and N client devices jointly train a global model. The dataset is owned locally by the client devices and is not shared.
[0022] In this embodiment, model training includes the following steps: S1.1. Initialize the global model, and divide the initialized global model into a server-side model and a client-side model based on the limitations of the local device. A global model is a machine learning model used to handle a specific task. Size is ,Include layer; The splitting is performed at the v-th layer of the global model, where ; The size of the client model after segmentation is The model parameters are ; The server-side model size is The model parameters are .
[0023] S1.2. Distribute the client model to all local devices; The distribution process takes place over the network between the local device and the server; The client model is distributed to all local devices to obtain the initial client model. .
[0024] S1.3. Based on the client-side model, multiple local devices simultaneously perform forward propagation to calculate the shredded data; a small batch data sample set for local device n. From its local training dataset Randomly selected from; The formula for calculating the crushing data is:
[0025] In the formula, For local devices No. The shredded data obtained through forward propagation after rounds of training. This is a mapping function on the local device side from input data to shredded data. For the first The client-side model parameters after the first round of training, This is the data sample set.
[0026] S1.4. Transmit the shredded data and training data tags from the local device to the edge server; S1.5. Perform forward propagation in parallel based on the received shredding data from different local devices, calculate the loss of each local device by combining the corresponding training data labels, and backpropagate to obtain the loss function and the gradient of the shredding data. The loss function for each client is:
[0027] In the formula, For the local device during the t-th round of training The loss function value, For the local training dataset of local device n, For loss function, For data sample set The first in One sample, It is the input training data. These are the training data labels. For the first The client-side model parameters after the first round of training, For the data sample set, For local devices In the The parameters of the server-side model after training.
[0028] The formula for calculating the gradient of the loss function is: (2) In the formula, This represents the gradient of the loss function of the server-side model on local device n during the t-th training round. This indicates the parameters of the server-side model. Find the gradient. It is a loss function, whose inputs include server-side model parameters. Client model parameters and local data samples .
[0029] The gradient calculation formula for pulverized data is: (3) In the formula, For local devices No. The gradient of the shattered data calculated through forward propagation after each training round. For local devices No. The shredded data obtained by forward propagation calculation after round of training.
[0030] S1.6. Aggregate the gradients of the shredding data to obtain the aggregated gradients, and broadcast the aggregated gradients of the shredding data to all local devices; the formula for calculating the aggregated gradient is: (4) In the formula, To aggregate the gradient of the data after the t-th training round, For local training dataset, For the number of local devices, Weighting coefficients: (5) S1.7. The server and local device update the server-side model and client-side model respectively based on the loss function gradient and the aggregation gradient, and then aggregate the server-side model; The update formulas for the server-side model and client-side model for device n in the t-th communication round are: (6) In the formula, For local device n, the complete model parameters are after training in round t. Let n be the server-side model parameters after the t-th round of training on local device n. To allow clients to share model parameters after the t-th round of training, It's the learning rate. It is the number of local epochs. Let represent the gradient of the loss function of the server-side model on local device n during the i-th iteration in the t-th training round. For the t-th round of training Client gradient for the next iteration.
[0031] The server-side model aggregation formula is: (7) In the formula, These are the parameters of the server-side aggregated model after the t-th round of training. Let n be the server-side model parameters after the t-th round of training on the local device n.
[0032] S1.8. Repeat steps S1.3 to S1.7 until the global model converges or the maximum number of communication rounds is reached, and obtain the global model obtained by segmented federated learning based on aggregated gradient broadcasting.
[0033] The global model obtained by segmented federated learning based on aggregated gradient broadcasting is: (8) In the formula, These are the parameters of the server-side aggregated model after the t-th round of training. To allow clients to share model parameters after the t-th round of training.
[0034] The optimization problem that comprehensively considers convergence rate and model training latency is a problem of minimizing the weighted sum of convergence and latency metrics. This aims to achieve a trade-off between convergence and latency, specifically including: The convergence of segmented federated learning based on aggregated gradient broadcasting is analyzed, given a function. The upper bound used to describe the difference between the segmentation federated learning architecture based on aggregated gradient broadcasting and the traditional segmentation federated learning architecture is as follows: yes It is a monotonically non-decreasing function, where k is a positive constant; The communication computation delay model for the t-th communication round is as follows: (9) In the formula, Let n be the latency for local device n to transmit shredded data to the server during the t-th round of training. The computational latency for local device n during the t-th round of training, derived from the forward propagation of the client model, is given. The computational delay for the server to continue performing forward and backward propagation based on the server-side model on local device n during the t-th round of training is given. Let $T$ be the delay at which the server transmits the aggregated gradient to the local device $n$ during the $t$-th training round. Let n be the delay calculated for backpropagation on local device n during the t-th round of training.
[0035] Considering convergence rate and delay, we construct a weighted problem P1 of convergence and delay indices, expressed as:
[0036] In the formula, w is a weighting factor balancing the convergence rate and delay; the optimization problem P1 also needs to satisfy the following constraints:
[0037] Where (10a) is the layer constraint of the ML model, and (10b) and (10c) are... and These are the central processing unit resources and transmit power of local device n during the t-th round of training. and These are the maximum computing resources and maximum transmit power for each client, respectively, in (10d) This represents the central processing unit (CPU) computing resources allocated by the server to client n in the server-side model. This represents the maximum computational resource constraint for server-side model updates; (10e) is the privacy protection constraint, and (10f) contains... B is the bandwidth allocated to client n, and B is the total available uplink bandwidth.
[0038] Step S2: Introduce auxiliary variables to decompose the optimization problem into a resource allocation subproblem and a split point selection subproblem; It should be noted that the optimization problem P1 is a min-maximum mixed integer nonlinear programming problem, which is a coupled problem of tangent point selection and resource allocation. By introducing auxiliary variables, the optimization problem P1 is re-formulated, and the coupled problem is decomposed. The introduced auxiliary variables are the upper bounds of the computation delay and the communication delay during training.
[0039] In one specific embodiment, an auxiliary variable is introduced. and , and Let be the upper bounds of computation and communication latency during training, respectively; then optimization problem P1 can be transformed into P2:
[0040] The optimization problem P2 must satisfy the following constraints:
[0041]
[0042] The optimization problem P2 above can be decomposed into two subproblems: the resource allocation subproblem and the split point selection subproblem; where, The resource allocation subproblem is a convex optimization problem, solved using the CVX method. It minimizes the introduced auxiliary variables by optimizing the allocation of communication resources.
[0043] P2.1 must satisfy the following constraints:
[0044]
[0045]
[0046] The subproblem of selecting the split point is an integer programming problem. By optimizing the model split point, the optimal convergence performance can be achieved.
[0047] P2.2 must satisfy the following constraints:
[0048]
[0049] .
[0050] Step S3: Model the split point selection subproblem as a Markov decision process, and use a deep reinforcement learning algorithm to solve for the optimal policy and obtain the optimal parameter vector to realize the optimal split point selection policy.
[0051] The optimization problem P2.2, namely the split-point selection subproblem, is an integer programming problem. It is solved using the Double Deep Q-Network (DDQN) algorithm, first modeling the optimization problem as a Markov decision process. Specifically, the modeling of the split-point selection subproblem as a Markov decision process is as follows: First, the Markov decision process includes a state space S, an action space A, state transition probabilities P, and a reward R. The state space S is defined as the channel gain observed at the beginning of this communication round; Action space A is defined as all possible choices of model split points; The state transition probability P is defined as the probability of a state transition under a certain action; The reward R is defined as the quality of the learning policy under the current state and action, and is expressed as the sum of the convergence index and the delay auxiliary variable.
[0052] Define the state space, action space, state transition probability, and reward for the t-th communication round: The state space is: ; The motion space is: ; The state transition probability is: ; The reward is:
[0053] In the formula, C is a sufficiently large number used as a penalty.
[0054] Specifically, solving for the optimal policy using a deep reinforcement learning algorithm, i.e., solving the optimization problem P2.2 using the DDQN algorithm, includes the following steps: First, define the state action function of the DDQN algorithm. , used to represent the strategy The expected long-term rewards, among which, Indicates state, Indicates discrete actions. It is the parameter vector of the Q network; Calculate the cumulative discount reward for round t:
[0055] In the formula, This refers to the weighted discount of rewards at future time points, defining state and action functions based on cumulative discounted rewards. .
[0056] Define a loss function, and optimize the parameters by minimizing the loss function. The loss function is expressed as: .
[0057] Finally, the optimal parameter vector is obtained to implement the optimal split point selection strategy.
[0058] Understandable. Figure 3 This is a schematic diagram illustrating the accuracy and convergence rate results of different node segmentation schemes provided in this embodiment of the invention. The step of obtaining the optimal parameter vector to achieve the optimal segmentation point selection strategy includes the following steps: Initialize the parameter vector of the Q network Initialize the experience buffer and the maximum number of iterations. ; Execute in each iteration: Reset initial state ; Perform T rounds of communication to update the Q network parameters. Specifically, for the t-th communication round: The DDQN agent determines the state based on the observed state. Select discrete actions ; Solve the resource allocation subproblem P2.1 to obtain the optimal communication and computing resource allocation strategy. and ; Based on the optimal resource allocation strategy in this round and Calculate rewards ; Observe the next state ; intermediate experience Add to the experience buffer; Sample a small batch from the experience buffer; Update the DQN network using gradient descent. .
[0059] By obtaining the optimal parameter vector To obtain the optimal split point selection strategy.
[0060] The following section will analyze the experimental results to demonstrate the high communication efficiency of this method.
[0061] Training Setup: Experiments were conducted in an environment with a server and N = 10 devices. The learning task was to train convolutional neural network (CNN) models for different classification tasks. To evaluate our proposed scheme, we conducted experiments on three different datasets: MNIST, FMNIST, and CIFAR-10. The model architecture consisted of a CNN with two 5x5 convolutional layers (the first with 32 channels and the second with 64 channels, each followed by a 2x2 max pooling), a fully connected layer with 512 units and ReLU activations, and a final softmax output layer. The maximum CPU cycle frequency per client was 0.1 GHz, and the total CPU cycle frequency of the server was 100 GHz. We assumed the computational workload per client and server to be 5.6 MFlops and 86.01 MFlops, respectively.
[0062] Communication Network Setup: We assume the path loss of the wireless channel between the device and the edge server is given by 128.1 + 37.6 log10(d) (in dB), where d represents the distance in kilometers (km). We assume a thermal noise spectral density of -174 dBm. The maximum transmit power budgets for each client and server are 25 dBm and 33 dBm, respectively. The total bandwidth is 20 MHz.
[0063] Figure 4 This diagram illustrates the accuracy and latency under various resource allocation strategies. Figure 4 Figure (a) shows the experimental results for the MNIST dataset. Figure 4 Figure (b) shows the experimental results for the FMNIST dataset. Figure 4 (c) in the figure shows the experimental results for the CIFAR-10 dataset. Figure 4 In this paper, we consider fixed-slice layer and random-slice layer strategies. Each strategy is evaluated using optimal and fixed computational and communication resource allocation. Under the same computational and communication resource allocation, the computational power adaptive training model allocation method proposed in this embodiment of the invention greatly reduces latency compared with the random strategy across different datasets.
[0064] In summary, the computationally adaptive training model allocation method provided by this invention optimizes the model's convergence rate and latency by jointly considering the selection of split points and the allocation of computational and communication resources, effectively improving the model's convergence performance and computational and communication efficiency. In addition, by dynamically selecting split points and optimizing communication and computational resources, the model's convergence can be effectively guaranteed and latency reduced.
[0065] According to a second aspect of the present invention, a computationally adaptive training model allocation system is provided, comprising: The optimization problem construction module is used to construct optimization problems that take into account both convergence rate and model training latency within the segmented federated learning framework. The optimization problem decomposition module is used to introduce auxiliary variables to decompose the optimization problem into resource allocation sub-problems and split point selection sub-problems. The optimal policy solution module is used to model the split point selection subproblem as a Markov decision process and use a deep reinforcement learning algorithm to solve for the optimal policy, thereby obtaining the optimal parameter vector to realize the optimal split point selection policy.
[0066] It is understood that the computing power adaptive training model allocation system provided by the present invention corresponds to the computing power adaptive training model allocation method provided in the foregoing embodiments. The relevant technical features of the computing power adaptive training model allocation system can be referred to the relevant technical features of the computing power adaptive training model allocation method, and will not be repeated here.
[0067] Additionally, embodiments of the present invention also provide an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The processor can invoke logical instructions in the memory to execute the implementation steps of the above-described adaptive computing power training model allocation method.
[0068] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0069] It should be understood that any parts not described in detail in this specification belong to the prior art.
[0070] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0071] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
[0072] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for allocating training models based on computing power, characterized in that, Includes the following steps: An optimization problem is constructed within the framework of segmented federated learning, taking into account both convergence rate and model training latency. By introducing auxiliary variables, the optimization problem is decomposed into a resource allocation subproblem and a split point selection subproblem. The split point selection subproblem is modeled as a Markov decision process, and a deep reinforcement learning algorithm is used to solve for the optimal policy, thereby obtaining the optimal parameter vector to realize the optimal split point selection policy.
2. The computational power adaptive training model allocation method according to claim 1, characterized in that, The segmentation federated learning framework is a segmentation federated learning method based on aggregated gradient broadcasting, which includes: a server and N client devices jointly training a global model, and the model dataset is owned locally by the client devices and is not shared.
3. The computational power adaptive training model allocation method according to claim 2, characterized in that, The optimization problem that comprehensively considers convergence rate and model training latency is a problem of minimizing the weighted sum of convergence and latency indices, realizing a trade-off between convergence and latency. This includes: analyzing the convergence of segmentation federated learning based on aggregated gradient broadcasting, describing the upper bound of the difference between segmentation federated learning based on aggregated gradient broadcasting and the traditional segmentation federated learning architecture, and constructing a weighted problem P1 for convergence and latency indices.
4. The computational power adaptive training model allocation method according to claim 1, characterized in that, The introduced auxiliary variables are the upper bound of the computational delay and the upper bound of the communication delay during training. After their introduction, the delay problem is expressed as minimizing the upper bound of the delay, and delay constraints are given.
5. The computational power adaptive training model allocation method according to claim 1, characterized in that, The resource allocation subproblem is a convex optimization problem, which is solved by using the CVX method to optimize the allocation of computational communication resources and minimize the introduced auxiliary variables.
6. The computational power adaptive training model allocation method according to claim 1, characterized in that, The subproblem of selecting the split point is an integer programming problem. By optimizing the model split point, the optimal convergence performance can be achieved.
7. The computational power adaptive training model allocation method according to claim 1, characterized in that, The step of modeling the segmentation point selection subproblem as a Markov decision process and using a deep reinforcement learning algorithm to solve for the optimal strategy includes: The DDQN algorithm is used to transform the integer programming problem into a Markov decision process problem, which includes a state space S, an action space A, state transition probabilities P, and a reward R. Here, the state space S represents the channel gain observed at the beginning of the current communication round, the action space A represents all possible choices of the model split point, the state transition probability P represents the probability of state transition under a certain action, and the reward R represents the quality of the learning policy under the current state and action, and is represented by the sum of the convergence index and the delay auxiliary variable. Define the state action function of the DDQN algorithm. , used to represent the strategy The expected long-term rewards, among which, Indicates state, Indicates discrete actions. It is the parameter vector of the Q network; Define a loss function, and optimize the parameters by minimizing the loss function. ; By obtaining the optimal parameter vector To achieve the optimal split point selection strategy.
8. The computational power adaptive training model allocation method according to claim 7, characterized in that, The strategy for obtaining the optimal parameter vector to achieve optimal split point selection includes: Initialize the parameter vector, experience buffer, and maximum number of iterations for the Q-network; Perform the following operations in each iteration: Reset the initial state; Perform T rounds of communication to update the Q network parameters, including for the t-th communication round; The DDQN agent determines the state based on the observed state. Select discrete actions ; Solve the resource allocation subproblem to obtain the optimal communication and computing resource allocation strategy; Calculate the reward based on the optimal resource allocation strategy for this round; Observe the next state ; intermediate experience Add to the experience buffer; Sample a small batch from the experience buffer; Update the DQN network using gradient descent. .
9. A computationally adaptive training model allocation system, characterized in that, include: The optimization problem construction module is used to construct optimization problems that take into account both convergence rate and model training latency within the segmented federated learning framework. The optimization problem decomposition module is used to introduce auxiliary variables to decompose the optimization problem into resource allocation sub-problems and split point selection sub-problems. The optimal policy solution module is used to model the split point selection subproblem as a Markov decision process and use a deep reinforcement learning algorithm to solve for the optimal policy, thereby obtaining the optimal parameter vector to realize the optimal split point selection policy.
10. An electronic device, characterized in that, It includes a memory and a processor, wherein the processor is used to implement a computing power adaptive training model allocation method as described in any one of claims 1 to 8 when executing a computer program stored in the memory.