Adaptive decomposition and unified scheduling method of mixed tasks in computing power network

Abstract

This invention discloses an adaptive decomposition and unified scheduling method for hybrid tasks in computing power networks, belonging to the field of computing power network task scheduling technology. The method first establishes a discrete time-slot model and obtains the task set and network state; then, it estimates the optimal decomposition scheme and determines the benefit through a pre-optimization method; next, it establishes a unified entity representation and scheduling optimization model; finally, it uses the ADUS-TD3 deep reinforcement learning algorithm to output the schedule. This method includes determining the profitability and adaptively decomposing divisible tasks based on the attribute characteristics of the tasks and the real-time resource state of the computing power network; and then modeling and co-scheduling the decomposed subtasks and indivisible tasks in a unified manner, thereby achieving online optimization that minimizes task completion latency and maximizes resource utilization. This invention can adaptively generate the optimal decomposition scheme based on the real-time resource state of the computing power network, significantly improving the rationality of decomposition decisions and the efficiency of pre-decision decisions.

Description

Technical Field

[0001] This invention belongs to the field of computing network task scheduling technology, specifically relating to an adaptive decomposition and unified scheduling method for hybrid tasks in a computing network. Background Technology

[0002] With the rapid development of technologies such as big data, artificial intelligence, and the Internet of Things, the number and scale of computationally intensive tasks have exploded. While traditional cloud computing provides powerful centralized computing capabilities, it faces challenges such as high network latency, excessive load on central nodes, and difficulty in meeting real-time requirements. Edge computing, as an emerging computing paradigm, effectively reduces task completion latency and alleviates pressure on central nodes by pushing computing resources to the network edge. However, edge nodes have limited computing power, and a single edge node often struggles to independently handle large-scale computing tasks. Computing Power Network (CPN), as a novel network architecture, achieves deep integration of computing and networking by sensing and coordinating distributed computing resources, providing a new approach to solving these problems.

[0003] In a computing network environment, task scheduling faces numerous challenges. First, tasks are heterogeneous, with significant differences in data volume, computational load, latency constraints, and other attributes. Some tasks are divisible, capable of being broken down into multiple subtasks for parallel execution, while others are indivisible. Determining whether a task should be decomposed and making a reasonable decomposition decision is a key issue in improving scheduling efficiency. Second, the resource status of the computing network is dynamic; parameters such as the remaining computing power of nodes, queue length, and available bandwidth of links change continuously over time, requiring scheduling decisions to adapt to this dynamism. Third, there is a coupling relationship between task decomposition and scheduling. The decomposition scheme affects the scheduling effect, and the scheduling result, in turn, verifies the rationality of the decomposition scheme. This coupling increases the complexity of the problem. Existing task scheduling methods either consider task decomposition without integrating it with the computing network or consider computing network-based scheduling optimization without considering task decomposition, lacking a unified optimization framework and making it difficult to achieve global optimization. Therefore, there is an urgent need for a computing network-based method that comprehensively considers task decomposability judgment, adaptive decomposition, and unified scheduling to effectively solve the scheduling optimization problem of mixed tasks in a computing network environment. Summary of the Invention

[0004] Purpose of the invention: This invention aims to provide an adaptive decomposition and unified scheduling method for hybrid tasks in a computing power network. Through the collaborative mechanism of pre-optimization judgment and ADUS-TD3 deep reinforcement learning, it achieves unified optimization of task decomposition and scheduling, minimizing task completion latency and improving resource utilization while ensuring decision feasibility.

[0005] Technical Solution: An adaptive decomposition and unified scheduling method for hybrid tasks in a computing power network, wherein the hybrid tasks include both divisible and indivisible computational tasks, and includes the following implementation steps: S1. Obtain the current state of the computing network, including the remaining computing power of each computing node, queue workload, and available bandwidth of each communication link. Then, divide the continuous time into discrete time slots with fixed duration. In each time slot, obtain the set of newly arrived tasks and their attributes, including the data volume, computational volume, latency constraints, maximum number of decomposable parts, parallel efficiency parameters, decomposition overhead parameters, and prior labels of each task. S2. For each decomposable task in the task set, estimate the optimal decomposition scheme through heuristic pre-optimization, including the optimal number of decompositions, the optimal decomposition ratio vector, and the optimal target node set. The decomposition benefit score is calculated based on the optimal decomposition scheme. A hybrid decision formula is used to verify the benefit by combining the prior task labels with the real-time resource status of the system, and the decomposition decision is obtained. S3. Represent indivisible tasks and undecomposed tasks as a single entity to be scheduled, and represent multiple decomposed subtasks as multiple entities to be scheduled. Establish a unified completion delay model and resource utilization model that includes transmission delay, queuing delay, computation delay, and result aggregation waiting delay. This includes constructing a scheduling optimization problem and clarifying node computing power constraints, link bandwidth constraints, and allocation uniqueness constraints. S4. Transform the scheduling optimization problem into a Markov decision process, and construct a state space containing network state and task characteristics, and an action space containing decomposition, scheduling, and resource allocation decisions. The ADUS-TD3 deep reinforcement learning algorithm is used to output continuous action parameters, which are then transformed into discrete decisions including the number of decompositions, the scale vector, the target node, and the transmission path through action mapping. S5. The decomposition and scheduling decisions are sent to the computing network controller for execution, the node queue workload and link bandwidth status are updated, and the process enters the next scheduling time slot for cyclic execution, thereby minimizing task completion latency and improving resource utilization.

[0006] Beneficial effects: Compared with the prior art, the substantive features and significant effects of the present invention include: (1) By establishing a discrete time slot scheduling model and a profitable decomposable pre-optimization judgment mechanism, this invention suppresses the latency degradation caused by excessive task decomposition from the source. It can adaptively generate the optimal decomposition scheme according to the real-time resource status of the computing network, which greatly improves the rationality of decomposition decision and the efficiency of pre-decision decision.

[0007] (2) This invention ensures the consistency of scheduling constraints while taking into account the global resource utilization rate, effectively simplifies the scheduling framework and reduces the complexity of system implementation.

[0008] (3) This invention is adapted to the online scheduling requirements in a dynamic computing network environment and has high practicality. Attached Figure Description

[0009] Figure 1 This is a flowchart of the adaptive decomposition and unified scheduling method of the present invention; Figure 2 This is a schematic diagram of the computing power network task scheduling system architecture of the present invention. Detailed Implementation

[0010] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0011] Combination Figure 1 The present invention provides an adaptive decomposition and unified scheduling method for hybrid tasks under a computing power network. The hybrid tasks include both divisible and indivisible tasks. The implementation process includes the following steps: Step S1: To achieve periodic analysis of the continuous scheduling process, this invention first divides continuous time into time segments of fixed duration, called time slots. Therefore, the set of time slots is defined as follows:

[0012] in, Represents a set of time slots. This represents the total number of time slots within a scheduling cycle. Let... This indicates the duration of each time slot. This represents the time slot index. Within each time slot, the system state is considered quasi-static, meaning that system parameters remain relatively stable within a single time slot but can dynamically change with different time slots. The reason for adopting a discrete time slot model is twofold: firstly, task arrival and resource state changes in CPN have time-varying characteristics, and discretization facilitates periodic scheduling decisions; secondly, the discrete time slot model matches the decision time steps in reinforcement learning, making it easier to subsequently use deep reinforcement learning methods to solve the optimization problem.

[0013] In scheduling time slots The scheduling controller receives the newly arrived set of tasks. Time slots are defined. The arrived task set is as follows:

[0014] in, Indicates time slot The total number of tasks that arrived, Indicates time slot The first One task. Each task Tuples with the following attributes:

[0015] The meanings of each attribute are as follows: Indicates task Data volume, in units This represents the scale of data required for task transmission; Indicates task The computational load, in units of The cycle represents the computing resources required to complete a task; Indicates task The time delay constraint is in seconds, and the representation task must be completed within this time limit; Indicates task The maximum number of decomposable parts represents the maximum number of subtasks that a task can be divided into. In particular, if This indicates that the task cannot be decomposed; Indicates task The parallel efficiency factor, with a range of values. This reflects the parallel execution efficiency after task decomposition; Indicates task The breakdown cost, in units of Period represents the additional computational overhead generated by the task decomposition operation itself; Indicates task The communication overhead coefficient represents the amount of additional data generated by communication between subtasks after task decomposition. Indicates task The prior label, with a value of , Or empty, representing prior information on whether the task provided by the business side has been decomposed.

[0016] Simultaneously, the state vector of the computing power network is obtained. The computing power network considered in this invention can be abstracted as a directed connected graph. ,in Represents the set of CPN nodes. The total number of nodes; Represents a set of network links. This represents the total number of links. This represents the set of paths between nodes. Each CPN node It has heterogeneous computing capabilities, and uses Represents a node The computing power resources, measured in CPU cycles per second or Each link With maximum bandwidth capacity The unit is , and These represent two different CPN nodes in the computing power network.

[0017] Network status includes: the remaining computing power of each computing node. This represents the computing resources currently available to the node; queue workload. The unit is CPU cycles, representing the total amount of computation currently queued for a node; the current node's task queue set. This includes historically arrived but not yet completed tasks; and the available bandwidth of each link. This indicates the currently available transmission bandwidth for the link.

[0018] Specifically, define nodes In the time slot The queue workload is as follows:

[0019] In the above formula, Indicates historical time slots The first One task; Indicates task The computational load; Indicates the historical time slot index (earlier than the current time slot) (time slot).

[0020] Queue workload It has an explicit update equation. At the end of each time slot, the node queue workload is updated according to the following rules:

[0021] in, This indicates that the task-node assignment indicator variable is used if the task Assigned to a node The value is then taken as Otherwise ; Represents a node In the time slot The amount of computation that can be processed internally. Indicates the task index. Indicates time slot The set of tasks that have arrived Indicates task The computational load, This indicates that the task-node assignment indicator variable is used if the task Assigned to a node The value is then taken as Otherwise ; Represents a node In the time slot The computational workload that the system can handle. This update equation reflects the dynamic changes in the queue workload: the computational workload of newly arriving tasks increases the queue workload, while node processing decreases it. (Take the maximum value.) This is to ensure that the workload of the queue does not become negative.

[0022] It is important to note that the task attributes and network status extracted in this step form the basis for all subsequent decisions. Task attributes characterize the inherent features of the task, including its computational scale, data scale, latency requirements, and decomposability-related information; network status reflects the current resource availability of the system, including the remaining computing resources and the bandwidth of communication resources. Accurate acquisition of these two types of information is a prerequisite for ensuring the correctness of subsequent judgments and decisions.

[0023] Step S2: Determining the decomposability of the heuristic pre-optimization scheme: The core of this step is the task set. Each decomposable task in ( The optimal decomposition scheme is estimated using a heuristic pre-optimization method, and then a benefit determination is made based on this scheme. It is important to note that this step only involves estimation and determination; no actual subtask entities are created. Specifically, it includes the following steps: Step S201, calculate the overall estimated execution latency: To determine whether decomposition brings time-saving benefits, we first need to calculate the estimated completion time of the task without decomposition as a baseline. candidate node set Each node The estimated completion time at this node is the computational delay when the task is not broken down. The overall estimated execution time consists of three parts: transmission delay, queuing delay, and computation delay. Transmission delay reflects the time required for data to be transmitted from the task entry point to the target node; queuing delay reflects the time the task waits in the queue at the target node; and computation delay reflects the time required for the task to perform computations on the target node.

[0024] Therefore, define the task At the node The overall expected execution latency is calculated using the following formula:

[0025] in, Indicates task At the node Estimated completion time without decomposition; Indicates task Transmit to node The expected transmission delay; Indicates task At the node Expected queuing delay; Indicates task At the node The formula calculates the expected computational latency. It decomposes the completion latency into three orthogonal components, facilitating the analysis of the contribution and bottlenecks of each stage. Transmission latency consists of path propagation latency and data transmission latency; queuing latency is determined by queue workload and service rate; and computation latency is determined by task computation load and node computing power.

[0026] Furthermore, the formula for calculating transmission delay is:

[0027] in, Representing a path The propagation delay is the sum of the propagation delays of all links along the path. Indicates link The propagation delay; Indicates task The amount of data; Representing a path In the time slot The available bandwidth is taken as the minimum of the available bandwidth of all links on the path, which is the bottleneck bandwidth. This formula decouples the transmission delay into two parts: a fixed propagation delay independent of the data volume. And data transmission latency, which is directly proportional to the amount of data and inversely proportional to the bottleneck bandwidth. The bottleneck bandwidth model is used because in end-to-end transmission, the transmission capacity of the path is limited by the narrowest link. This approximation can conservatively estimate the transmission capacity and avoid scheduling decision errors caused by overly optimistic bandwidth estimation.

[0028] The formula for calculating queuing delay is:

[0029] in, Represents a node In the time slot The workload of queuing; Represents a node The formula uses a workload approximation model, assuming that tasks in the queue are executed sequentially according to the first-in-first-out (FIFO) principle, and that queuing latency is determined by the ratio of the current queue workload to the node service rate.

[0030] The formula for calculating latency is:

[0031] Take the minimum value among all candidate nodes as the overall estimated execution latency:

[0032] This estimated latency will serve as the benchmark for subsequent decomposition benefit calculations. It is used to determine whether decomposition can result in latency savings. By selecting the latency corresponding to the optimal node as the benchmark, the fairness of the comparison is ensured—if the latency after decomposition is lower than the execution latency of the optimal single node, then the decomposition is meaningful.

[0033] Step S202, Heuristic pre-optimization decomposition scheme: This sub-step estimates the optimal decomposition scheme, including the optimal number of decomposition parts, using a heuristic method. Optimal proportional vector and the optimal target node set It is important to emphasize that this sub-step only performs an estimation and does not actually create the subtask entity. First, the effective service capacity weight of the candidate nodes needs to be calculated. This weight comprehensively considers the node's remaining computing power and queue length, reflecting the node's current actual processing capacity. Higher remaining computing power indicates more abundant available computing resources; a shorter queue length indicates fewer tasks waiting to be executed on the node, resulting in higher processing efficiency. Therefore, nodes with higher effective service capacity weights respond faster to new tasks, and prioritizing the allocation of such nodes to subtasks can minimize queuing time and computation latency.

[0034] Therefore, the formula for calculating the effective service capability weight is defined as follows:

[0035] in, Represents a node In the time slot Effective service capability weight; Represents a node In the time slot The remaining available computing power; Represents a node In the time slot The weighting formula takes into account both the node's remaining computing power and the queue length. Higher remaining computing power and a shorter queue result in a stronger effective service capability for the node, leading to a higher weight. Adding 1 to the denominator avoids division by zero and ensures that when the queue is empty, the weight is determined solely by the remaining computing power. This design prioritizes assigning the most powerful node to subtasks after task decomposition, thereby minimizing queuing time and computation latency for subtasks.

[0036] Then, take The nodes form a candidate node set, and the decomposition ratio is allocated according to the effective service capacity weight. The initial value of the ratio vector is calculated using the following formula:

[0037] in, Indicates the first The unconstrained decomposition ratio of individual subtasks; Indicates assignment to the first Nodes of each subtask Effective service capability weighting Indicates the candidate node index. Indicates the number of candidate nodes. Indicates the first The effective service capacity weights of each candidate node are assigned proportionally. This allocation method ensures that the computational load matches the node's processing capacity, with more capable nodes undertaking more computational load, thereby reducing the overall completion time of the subtask. This weighted allocation design embodies the idea of ​​load balancing, making the completion time of each subtask more consistent and preventing some subtasks from becoming bottlenecks due to excessive computational load.

[0038] However, the unconstrained proportional allocation described above may result in excessively small subtasks, leading to excessive scheduling overhead. To prevent this, a minimum proportional lower bound constraint is introduced. Then, perform simplex projection to ensure the scale vector satisfies the constraints. The formula for applying the minimum scale constraint and performing simplex projection is as follows:

[0039] in, This is the lower bound of the minimum proportion; It is a set of simplexes with a lower bound; This represents the projection operation onto the set. The projection operation ensures that the scaling vector satisfies the lower bound constraint while maintaining the sum of its components at 1. This constraint is introduced to avoid generating excessively small subtasks. Although excessively small subtasks have low computational cost, their scheduling overhead (such as task splitting, state management, and result collection) accounts for a high proportion of the computational cost, which may lead to a decrease in overall efficiency.

[0040] Next, the latency of this decomposition scheme needs to be estimated. The formula for estimating the expected computational latency of the subtasks is as follows:

[0041] in, Subtasks At the node The estimated calculation delay; Indicates the first The breakdown ratio of individual sub-tasks; Indicates the original task The computational load; This indicates the cost coefficient for decomposition management calculation; Indicates the number of parts; This represents the parallel efficiency parameter; Represents a node The formula reflects the two components of subtask computation latency: the proportionally allocated computational load of the original task, and the amortization of computational overhead through decomposition and management. The first term... The second item represents the proportionally allocated amount of computation from the original task. To decompose the allocation of management calculation costs, the denominator contains This reflects the loss of parallel efficiency. When When, it represents an ideal linear speedup; when When the value is 0, it indicates that there is a loss of parallel efficiency and the actual speedup is lower than the ideal value.

[0042] The estimated execution latency consists of two parts. The first part is the maximum completion latency of each subtask (i.e., the completion time of the slowest subtask, reflecting the bottleneck effect in parallel execution); the second part is the result aggregation latency. The formula for calculating the estimated execution latency is as follows:

[0043] in, Indicates the number of parts to be divided. The proportional vector is The estimated completion time; for A set of target nodes; Subtasks To the node The expected transmission delay; Represents node n j Expected queuing delay; Subtasks At the node The estimated computation time is given by the estimated computation time estimation formula for the above subtasks; This represents the aggregate latency. Aggregate latency can be modeled as a combination of fixed overhead and scale-dependent overhead:

[0044] in, This indicates fixed aggregation overhead, such as result collection initialization; This represents the aggregation overhead coefficient that increases with the number of subtasks, such as the complexity of merging results as the number of subtasks increases.

[0045] Finally, from arrive Perform a one-dimensional search within the range and calculate each Corresponding target value:

[0046] in, This is an over-decomposition penalty coefficient used to suppress unnecessary excessive decomposition. Excessive decomposition increases scheduling complexity and the risk of resource fragmentation. Furthermore, the benefits of decomposition exhibit diminishing marginal returns, thus requiring a trade-off between latency gains and decomposition costs. The optimal number of decompositions, 's', is chosen to minimize the objective value. :

[0047] The optimal decomposition scheme is obtained through the above heuristic pre-optimization process. ,in and These are tasks The optimal decomposition scale vector and A set of target nodes, This represents the optimal target node for the j-th subtask. This scheme, based on the current resource state, uses heuristic methods to estimate the optimal decomposition parameters, providing a basis for subsequent profit determination and serving as a reference and fallback guarantee for TD3 decision-making.

[0048] Step S203, Profit Comparison Determination: Based on the optimal decomposition scheme obtained from pre-optimization, calculate the decomposition reward score:

[0049] in, This represents the decomposition benefit score; a positive value indicates that the decomposition is beneficial, while a negative value or zero indicates that the decomposition is neither beneficial nor harmful. This is the estimated completion time without decomposition. The estimated completion time under the heuristic optimal decomposition scheme is as follows: Indicates task The optimal number of decomposition parts, Indicates task The optimal decomposition ratio vector. The physical meaning of this formula is: if the speedup benefit from parallel execution of subtasks after decomposition is greater than the decomposition cost, then... Decomposition is beneficial; conversely, if the cost of decomposition exceeds the benefits of parallelism, then... It should not be decomposed.

[0050] Then, a hybrid decision-making formula is used for the final determination. This hybrid formula combines prior labels of the task structure with system benefit verification, utilizing both prior knowledge from the business side and maintaining the system's adaptability. The hybrid decision-making formula is as follows:

[0051] in, This indicates the pre-optimization judgment result; This indicates an indicator function; the value inside the parentheses is true if true. Otherwise ; Indicates prior weights; This represents the numerical representation of the prior labels. but , but If no label is provided, it will cause and ; express Smoothing function; This indicates the profit threshold; a reference value can be taken as... ; Indicates the smoothing temperature; This represents the threshold value; the default value can be 0.5.

[0052] It should be noted that the design principle of the hybrid decision-making mechanism is: when the prior label is reliable ( (Larger), the judgment result mainly depends on the prior label; when the prior label is unreliable or not provided ( Smaller or The judgment result mainly depends on Verification of the smoothed system revenue. The function's purpose is to map the payout score to... Range, and simultaneously through temperature parameters Control the smoothness of the mapping. Revenue threshold. The introduction of this feature is to avoid frequent decomposition due to excessively small returns; decomposition only occurs when the returns exceed a certain threshold (such as a reference value). This means that the revenue should at least exceed that of the undecomposed form. It is only when this is considered worthwhile that decomposition is considered.

[0053] Step S3, Establishing the unified entity representation and scheduling optimization problem:

[0054] To uniformly schedule all divisible tasks, indivisible tasks, and subtasks, these tasks need to be uniformly defined. Based on the calculation of non-decomposed tasks and the calculation of subtasks after task decomposition described in step S2, these tasks can be uniformly defined and represented as an optimization problem.

[0055] Step S301, Unified Entity Representation:

[0056] This sub-step establishes a unified entity abstraction and a unified scheduling optimization model. Unified entity set It includes three types of entities: indivisible task entities, undecomposed decomposable task entities, and decomposed subtask entities. The definition of the unified entity set is as follows:

[0057] in, Indicates time slot An indivisible set of tasks Indicates time slot A set of decomposable tasks, and having ; Indicates task Decomposition decision; This indicates the number of parts to be decomposed. This definition unifies three cases into a single entity to be scheduled: an indivisible task as a single entity; a decomposable but not yet decomposed task as a single entity; and a decomposable and already decomposed task as multiple subtask entities. From this point onward, the unified scheduling model only considers entities. Modeling can be done without distinguishing between task types.

[0058] Node utilization is defined as the sum of the proportions of the computing power allocated to each node:

[0059] in, Represents a node In the time slot Utilization rate; Indicates time slot Assigned to nodes Total computational load, Represents a node In the time slot Total processing capacity within the node. This definition reflects the node's resource usage. When When, it indicates that the node is completely idle; when When the node's computing power is fully allocated, it indicates that the node's computing power has been fully allocated.

[0060] Global average utilization is defined as the average of the utilization of all nodes:

[0061] The unified latency model decomposes entity completion latency into three parts: transmission latency, queuing latency, and computation latency. Entity completion latency is the sum of these three:

[0062] in Indicates the task completion time. This indicates the time when the task will arrive, while They are entities Transmission delay, entity Queuing delay, physical The calculation of the delay is basically the same as that in step S201, and will not be repeated here.

[0063] Step S302, Establishing the unified scheduling optimization problem: Based on the modeling in the previous sub-step, the objective function of the unified scheduling optimization problem can be defined as follows:

[0064] in, Representing entities The weights are used to reflect the latency sensitivity of different tasks; Representing entities Completion delay (the time from when the task arrives to when it is completed); The secondary target weight for utilization rate is much smaller than that of the target. Prioritize latency. Indicates time slot The global average utilization rate.

[0065] The constraints are as follows: Node computing power constraints:

[0066] in, This indicates that the task-node assignment indicator variable is used when the task... Assigned to CPN node Time value Otherwise This constraint ensures that each node is in the time slot. The computational load allocated within the system shall not exceed its processing capacity.

[0067] Link bandwidth constraints:

[0068] in, This indicates a task-link routing indicator variable, when the entity Data passes through the link The value is 1 if the condition is met, and 0 otherwise. This constraint ensures that the amount of data transmitted by each link within time slot t does not exceed its bandwidth capacity.

[0069] Assignment uniqueness constraint:

[0070] This constraint ensures that each scheduling entity must be assigned to one and only one node.

[0071] Path uniqueness constraint:

[0072] in, This indicates the task-path selection indicator variable, when the scheduling entity... Select path Time value Otherwise . This represents the set of candidate paths from the task entry point to node n. This constraint ensures that an entity assigned to a node must choose a path to that node.

[0073] Step S4, MDP transformation and ADUS-TD3 solution: This step transforms the unified scheduling optimization problem into a Markov Decision Process (MDP) problem and solves it using the ADUS-TD3 algorithm. An MDP is defined as a quintuple. .

[0074] Step S401, State space definition: The state space S contains the remaining computing power of the nodes. Node queue workload Available bandwidth of the link The current task characteristics and historical situation. The state vector is defined as follows:

[0075] The current task feature vector includes task attributes and the output of the decision phase (profit score). and judgment results This enables TD3 to utilize decision information:

[0076] The historical situation vector is defined as follows:

[0077] in, This indicates the recent average task arrival rate; Indicates near Average utilization rate of each time slot This represents the average completion delay over the last L time slots. The historical data vector is used to enhance observability, helping TD3 understand the long-term trends of the system and thus make better scheduling decisions.

[0078] Step S402, Action Space Definition: Action space A is defined as continuous parameterized actions, encompassing the raw outputs of decomposition decisions, scheduling decisions, and resource allocation decisions:

[0079] in, Indicates a tendency for continuous decomposition; Indicates the intensity of continuous decomposition; This represents the original output of the scaling vector; Indicates the node preference score; This represents the path preference score. This continuously parameterized action design allows ADUS-TD3 to output continuous values, which are then transformed into discrete feasible decisions through an action mapping mechanism.

[0080] Step S403, Action Mapping Mechanism: The action mapping function maps continuous actions to executable discrete actions. Whether to decompose the mapping is determined by a threshold.

[0081] Simultaneously apply decision constraints: If If the pre-optimization decision deems it inappropriate to decompose, then TD3 will not force decomposition. This ensures that the decision made during the decision phase is respected; TD3 will not force decomposition if the pre-optimization decision deems it inappropriate to decompose.

[0082] The decomposition mapping uses a floor function:

[0083] Scale vector normalization adopts Functions and simplex projections with lower bounds:

[0084] Node selection mapping adopts choose: {

[0085] Path selection mapping uses the maximum preference score among candidate paths:

[0086] The aforementioned action mapping also plays a crucial role in ensuring the uniqueness of paths and allocations in the scheduling optimization problem, so that each entity is assigned to a unique node through a unique path.

[0087] Step S404, Reward Function Design and MDP Problem Representation: When considering the design of the reward function, in addition to satisfying the latency and utilization in the scheduling optimization problem, in order to ensure that the decision meets the constraint requirements, the constraints can be added as penalty terms to the reward function. Since certain constraints have already been guaranteed in the action mapping, the reward function focuses on considering latency constraints, link bandwidth constraints, and node computation constraints.

[0088] Based on the above analysis, the reward function is defined as the negative of the system cost per time slot:

[0089] The formula for calculating the system cost function is as follows:

[0090] in, Indicates time slot A complete set of entities; This indicates the secondary target weight for utilization rate; This represents the timeout penalty coefficient; This represents the correction cost coefficient and the correction indicator function. When the correction is executed Otherwise ; This represents the penalty coefficient for consistent judgment. This represents the penalty coefficient for violating the constraint, and the penalty for violating the constraint... Defined as and The two parts correspond to the node computing power constraint and the node bandwidth constraint in the scheduling problem, respectively.

[0091] Through the above series of steps, we have implicitly handled each constraint in the action mapping and reward function. Finally, the entire scheduling optimization problem can be described as the following MDP problem: In the formula This represents the calculation of mathematical expectation. Discount factor of Power of 1 Indicates the state Execute action The instant reward obtained afterward Indicates all strategies Find the maximum value.

[0092] Step S405, ADUS-TD3 algorithm described in detail: This invention employs the Adaptive Decomposition and Unified Scheduling based TD3 (ADUS-TD3) algorithm. This algorithm is an improved version of TD3, incorporating adaptive task decomposition. Through a special action mapping mechanism and reward function design, it ensures that decisions satisfy various constraints as much as possible, enabling effective task decomposition and scheduling, and possessing feasibility correction and backoff mechanisms. Simultaneously, the algorithm retains the basic architecture of the classic TD3, effectively addressing Q-value overestimation and training instability issues through three mechanisms: a dual-critic network, delayed policy update, and target policy smoothing. The ADUS-TD3 network architecture includes: a... network Two are responsible for independently generating actions. network Evaluate the value of actions and their corresponding target networks. Used to calculate the target Q value.

[0093] The decision-making process of the ADUS-TD3 algorithm in each time slot is described in detail below: (1) Action generation and exploration noise: During the training phase, the Actor network generates actions based on the current state s and adds exploration noise to encourage exploration:

[0094] in, Indicates the actual action performed; This indicates that the Actor network depends on the state. Deterministic actions output; Indicates exploratory noise, which follows a mean of 1 / 2. variance is The Gaussian distribution is used. The role of exploration noise is to enable the agent to explore underexplored regions in the state-action space, avoiding premature convergence to local optima. It should be noted that the variance of the exploration noise... This is an important hyperparameter; too large a value can lead to policy oscillations, while too small a value results in insufficient exploration. Preferably, this invention employs a gradually decaying noise strategy, using larger noise levels in the early stages of training to encourage exploration, and gradually reducing the noise level as training progresses to stabilize the policy.

[0095] (2) Target policy smoothing: In calculating the target When the value is reached, for the target Adding noise to the network output actions to achieve smoothness of the target strategy:

[0096] in, This indicates the smoothed target action; Indicates the next state; Indicate target Actions output via the network; Indicates target policy noise; This indicates that the noise is clipped to... Within the scope. The design principle of target policy smoothing is: by adding a small amount of noise to the target action and pruning it, similar state-action pairs are smoothed out. Value estimation is smoother and reduces Overfitting of the value function in local regions can improve the target value. Robustness of value estimation. This mechanism can effectively suppress... It is worth overestimating, because overestimation often stems from... Value functions are overly sensitive to certain actions.

[0097] (3) Double Target Value calculation: One of the core design features of the TD3 is the use of dual... Network computing objectives The value is the minimum of the two values.

[0098] in, Indicate target value; Indicates an immediate reward; Indicates the discount factor; and Representing two objectives respectively Network for next state-action pairs of Value estimation. The design principle of taking the minimum value is that, due to the function approximation error, Networks often generate The value was overestimated, while two independently trained [systems]... The network has a low probability of simultaneously overestimating the probability of the same state-action pair. By taking the minimum value, this can be effectively suppressed. The value was overestimated, causing the target to be... The values ​​are more conservative and accurate, thereby improving the stability of training.

[0099] (4) Network update: two The network is updated by minimizing the mean squared error (MSE) loss:

[0100] in, Indicates the first indivual The network's loss function; Indicates the first indivual Network for current state-action pairs of Value estimation. By minimizing the MSE loss, Networks learn to predict the true value of state-action pairs, for The network's policy optimization provides accurate gradient signals.

[0101] (5) Network delay update: The network employs a delayed update strategy, updating every [time period]. Update once per step:

[0102] in, express The objective function of the network is to maximize the first The network estimates the Q-value of actions under the current policy. The design principle of delayed updates is that, in the early stages of training, The network estimates are not yet accurate, if Frequent network updates may lead to policy optimization based on incorrect Q-value estimates, resulting in policy divergence. Delaying updates can address this. The Internet provides The network spends more time converging to a more accurate estimate, thereby improving the stability of policy optimization.

[0103] (6) Target network soft update: The target network employs a soft update strategy, slowly tracking the parameters of the main network.

[0104] in, This represents the soft update coefficient. The design principle of soft update is that the target network is used to calculate the target... If the target network parameters change too rapidly, it will cause the target... Unstable values ​​affect training convergence. Through soft updates, the target network parameters slowly track the main network, thus maintaining the target network's performance. The values ​​are relatively stable, while allowing the target network to gradually incorporate the knowledge learned by the main network.

[0105] Step S406, Feasibility Correction and Rollback Mechanism: Because the actions output by ADUS-TD3 may violate resource constraints, feasibility checks and corrections are required before execution. If the decision is not feasible, the following correction strategy should be adopted for gradual adjustment: Node capacity check; if a node... If the predicted increase in workload exceeds the threshold, the subtask is reassigned to the next candidate node; link bandwidth is checked, and if link congestion results in insufficient available bandwidth on the path, the path is switched to an alternative path; decomposition and rollback are performed, and if reassignment is still not feasible, decomposition and rollback are executed: Recalculate And reselect Node; if Still not feasible, let If all the above corrections fail, the pre-optimized solution will be rolled back to the pre-optimized solution output during the pre-optimization stage. .

[0106] Correct the trigger indicator function The definition is as follows: = { 1, if the feasibility correction step is executed; 0, otherwise} Step S5: This step executes the scheduling decision and updates the system status. First, the scheduling decision is sent to the computing network controller for execution. If the decomposition decision output by ADUS-TD3... Then, according to the number of decompositions output by ADUS-TD3 Scale vector and target node set Execute task decomposition and generate Each subtask entity is assigned to a corresponding node; if If so, the task is scheduled to the target node as a single entity.

[0107] After executing the scheduling, update the node queue workload status:

[0108] In addition to the link bandwidth status, the status of the next time slot can be obtained. Then calculate the reward. ,in This indicates the environmental state of the computing power network system in the current scheduling time slot. This indicates that the Actor neural network is based on the current state. Directly output unmapped, continuously parameterized actions. This represents the feedback reward value obtained by the system after performing an action. This indicates that after the scheduling decision (i.e., issuing the decomposition and scheduling decision and updating resources) is executed in the current time slot, the system enters a new state in the next scheduling time slot, and finally the network parameters are updated according to the ADUS-TD3 deep reinforcement learning algorithm update rules.

[0109] Next, the experience tuples Store the data in the experience replay buffer. Finally, update the network parameters according to the ADUS-TD3 update rules.

[0110] During online inference, the following steps are performed in each time slot: read the state. ; For task sets Each task in the process undergoes pre-optimization assessment, and the output is... and ;pass Network inference yields Discrete decisions are obtained by mapping the execution actions. Perform constraint verification and feasibility correction; issue scheduling decisions for execution; observe the next state. And calculate the reward Storage experience Enter the experience playback buffer; sample and update network parameters from the experience playback according to the ADUS-TD3 update rules.

[0111] Through the above process, this invention can achieve the synergy between pre-optimization judgment and deep reinforcement learning. While ensuring the feasibility of scheduling decisions, it makes full use of the long-term optimization capabilities of deep reinforcement learning, effectively solving the adaptive scheduling problem of mixed divisible and indivisible tasks under computing power networks.

[0112] This invention is applicable to real-world deployment environments with mixed tasks (divisible and indivisible), dynamic resource changes, and low latency and high utilization requirements, such as computing power networks, edge computing, cloud computing centers, industrial internet, AI inference clusters, vehicle-road cooperation, and 5G / 6G computing power private networks.

[0113] Example verification: The computing network used in this example consists of a layered architecture of core nodes and edge nodes, and the node parameters are shown in Table 1.

[0114] Table 1. Parameter Table of Computing Network Nodes

[0115] Based on Table 1 above, core nodes account for 30% of the network, undertaking the aggregation and processing functions of large-scale computing tasks; edge nodes account for 70%, responsible for accessing user tasks nearby and performing preliminary processing. The computing capabilities of the nodes are heterogeneously distributed, with a difference of up to 3.3 times, providing a resource diversity basis for parallel scheduling after task decomposition.

[0116] The network links employ a hierarchical bandwidth configuration strategy, with the link bandwidth between core nodes and edge nodes being significantly higher than the link bandwidth between edge nodes. Table 2 provides overall parameter statistics for the network links.

[0117] Table 2 Computing Power Network Link Parameter Table

[0118] To further analyze network topology characteristics, Table 3 presents hierarchical statistical results by link type.

[0119] Table 3. Link Type Hierarchical Parameter Table

[0120] As shown in Table 3, the average bandwidth of the edge-core link is 8.2 times that of the edge-edge link, reflecting the high-throughput design concept of the core node; the edge-edge link has lower bandwidth but similar propagation latency, making it suitable for collaborative task transmission between edge nodes. This hierarchical architecture provides the network foundation for multi-path parallel transmission after task decomposition.

[0121] In computing power networks, tasks are divided into two categories: indivisible tasks and decomposable tasks, and their attribute parameters differ significantly. Table 4 shows a comparison of the key parameters of the two types of tasks.

[0122] Table 4. Comparison of parameters between indivisible and decomposable tasks

[0123] Table 4 shows that the data volume of decomposable tasks is significantly higher than that of indivisible tasks, ranging from approximately 20 times that of indivisible tasks, indicating that decomposable tasks are typically large-scale data processing tasks. The computational costs of the two types of tasks are similar, but the latency constraints of decomposable tasks are more relaxed, providing a time margin for their decomposition and parallel processing. The maximum number of decomposable tasks is 2–6, with parallel efficiency ranging from 0.65 to 1.00, and a decomposition cost of 2.0 × 10⁻⁶. 7 ~2.2×10 8 cycles, these parameters together determine the feasibility and potential benefits of task decomposition.

[0124] To verify the effectiveness of the proposed method, a comparative experiment was conducted between the proposed ADUS-TD3 algorithm and the Greedy strategy under the same network environment and task load conditions. The experiment covered 3000 scheduling slots, totaling 9243 tasks, of which 59.4% were decomposable. Experimental results show that the proposed method achieves significant improvements in several key performance indicators, as detailed below: The average task completion time decreased by approximately 49.8%; The average computing power utilization rate of nodes has increased by approximately 38.5%; The task timeout rate decreased by approximately 21.8%; Overall scheduling performance (weighted latency and utilization) improved by approximately 39.1%; The link bandwidth utilization has been significantly improved, effectively alleviating network congestion problems; The divisible task adaptive decomposition decision is accurate and effective, ensuring the stable operation of the system under high load scenarios.

[0125] The aforementioned performance improvements demonstrate that the method of this invention, through adaptive task decomposition and unified scheduling mechanisms, can effectively reduce task completion latency, improve resource utilization efficiency, and significantly enhance the system's quality of service assurance capabilities. This method is particularly suitable for complex scheduling scenarios in computing networks where heterogeneous tasks arrive in combination and resources dynamically change.

Claims

1. An adaptive decomposition and unified scheduling method for hybrid tasks in a computing power network, wherein the hybrid tasks include divisible and indivisible computational tasks, characterized in that, Includes the following steps: S1. Obtain the current state of the computing network, including the remaining computing power of each computing node, queue workload, and available bandwidth of each communication link. Then, divide the continuous time into discrete time slots with fixed duration. In each time slot, obtain the set of newly arrived tasks and their attributes, including the data volume, computational volume, latency constraints, maximum number of decomposable parts, parallel efficiency parameters, decomposition overhead parameters, and prior labels of each task. S2. For each decomposable task in the task set, estimate the optimal decomposition scheme through heuristic pre-optimization, including the optimal number of decompositions, the optimal decomposition ratio vector, and the optimal target node set. The decomposition benefit score is calculated based on the optimal decomposition scheme. A hybrid decision formula is used to verify the benefit by combining the prior task labels with the real-time resource status of the system, and the decomposition decision is obtained. S3. Represent indivisible tasks and undecomposed tasks as a single entity to be scheduled, and represent multiple decomposed subtasks as multiple entities to be scheduled. Establish a unified completion delay model and resource utilization model that includes transmission delay, queuing delay, computation delay, and result aggregation waiting delay. This includes constructing a scheduling optimization problem and clarifying node computing power constraints, link bandwidth constraints, and allocation uniqueness constraints. S4. Transform the scheduling optimization problem into a Markov decision process, and construct a state space containing network state and task characteristics, and an action space containing decomposition, scheduling, and resource allocation decisions. The ADUS-TD3 deep reinforcement learning algorithm is used to output continuous action parameters, which are then transformed into discrete decisions including the number of decompositions, the scale vector, the target node, and the transmission path through action mapping. S5. The decomposition and scheduling decisions are sent to the computing network controller for execution, the node queue workload and link bandwidth status are updated, and the process enters the next scheduling time slot for cyclic execution, thereby minimizing task completion latency and improving resource utilization.

2. The method of claim 1, wherein, Step S1 includes the following specific steps: Represent the time slot as ,in, Represents a set of time slots. Represents the total number of time slots within a scheduling period, let... This indicates the duration of each time slot. Indicates the time slot index; In scheduling time slots The set of arriving tasks is represented as ,in, Indicates time slot The total number of tasks that arrived, Indicates time slot The first One task; The computing power network can be abstracted as a directed connected graph. ,in Represents the set of CPN nodes; Represents a set of network links; This represents the set of paths between nodes, and the network state includes the remaining computing power of each computing node. and queue workload The current node's task queue set Includes historically arrived but not yet completed tasks and available bandwidth for each link. , and These represent two different CPN nodes in the computing power network.

3. The method of claim 2, wherein, In step S1, the node is defined In time slot Queue workloads Are as follows: Queue workload It has an explicit update equation. It is the current node's task queue set. Indicates historical time slots The first One task; Indicates task The computational load; This represents a historical time slot index, that is, a time slot earlier than the current time slot. The time slot; At the end of each time slot, the node queue workload is updated according to the following rules: in, Indicates the task index. Indicates time slot The set of tasks that have arrived Indicates task The computational load, This indicates that the task-node assignment indicator variable is used if the task Assigned to a node The value is then taken as Otherwise ; Represents a node In the time slot The amount of computation that can be processed internally.

4. The method of claim 1, wherein, Step S2 includes the following specific steps: S201. When the computation task is not decomposed, the minimum value of the overall expected execution delay at each candidate node is taken as the baseline delay. S202. Select the optimal node based on the effective service capability weight of the node, allocate the decomposition ratio according to the weight, introduce the minimum ratio constraint and perform simplex projection, determine the optimal number of decomposition parts through one-dimensional search, and obtain the optimal decomposition scheme. S203. Calculate the decomposed profit score, combine the prior label and profit verification to complete the mixed judgment, and output the decision of whether to decompose.

5. The method of claim 4, wherein, In step S201, the task In the node The overall execution predicted delay calculation formula is: in, Indicates task At the node Estimated completion time without decomposition; Indicates task Transmit to node The expected transmission delay; Indicates task At the node Expected queuing delay; Indicates task At the node The estimated computational delay; Step S202 uses the following formula to calculate the effective service capability weight of a node: in, Represents a node In the time slot Effective service capability weight; Represents a node In the time slot The remaining available computing power; Represents a node In the time slot The length of the queue; Take The node constitutes a candidate node set, and the initial value of the proportion vector is calculated according to the following formula: ； in, Indicates the first The unconstrained decomposition ratio of individual subtasks; Indicates assignment to the first Nodes of each subtask Effective service capability weighting Indicates the candidate node index. Indicates the number of candidate nodes. Indicates the first The effective service capability weight of each candidate node; Introducing minimum ratio lower bound constraints and performing simplex projection to make the ratio vector satisfy the constraints, the formula of minimum ratio constraint and simplex projection is as follows: in, This is the lower bound of the minimum proportion; It is a set of simplexes with a lower bound; This represents a projection operation onto the set. The projection operation ensures that the scaling vector satisfies the lower bound constraint while keeping the sum of its components equal to 1. In step 203, based on the optimal decomposition scheme obtained through pre-optimization, the decomposition profit score is calculated as follows: in, Indicates the first The decomposition benefit score for each task; a positive value indicates that the decomposition is beneficial, while a negative value or zero indicates that the decomposition is neither beneficial nor harmful. This is the estimated completion time without decomposition. The estimated completion time under the heuristic optimal decomposition scheme is as follows: Indicates task The optimal number of decomposition parts, Indicates task The optimal decomposition scaling vector; The hybrid decision-making process combines prior labels of task structure with system benefit verification, and uses a hybrid decision-making formula for the final determination: in, This indicates the pre-optimization judgment result; This indicates an indicator function; the value inside the parentheses is true if true. Otherwise ; Indicates prior weights; This represents the numerical representation of the prior labels. but , but If no label is provided, it will cause and ; express Smoothing function; Indicates the profit threshold. Indicates the smoothing temperature; This indicates the threshold for judgment.

6. The method of claim 5, wherein, Step S202 includes estimating the latency of the decomposition scheme. The estimation formula for the expected computational latency of the subtask is as follows: in, Subtasks At the node The estimated calculation delay; Indicates the first The breakdown ratio of individual sub-tasks; Indicates the original task The computational load; This indicates the cost coefficient for decomposition management calculation; Indicates the number of parts; This represents the parallel efficiency parameter; Represents a node The computing power capacity; The estimated execution latency consists of two parts: the first part is the maximum value of the completion latency of each subtask; the second part is the result aggregation latency. The calculation formula for the estimated execution latency is as follows: in, Indicates the number of parts to be divided. The proportional vector is The estimated completion time; for A set of target nodes; Subtasks To the node The expected transmission delay; Represents a node Expected queuing delay; Subtasks At the node The estimated computation time is given by the formula for estimating the estimated computation time of the subtasks mentioned above. Indicates aggregation delay; The aggregated latency Modeling as a combination of fixed overhead and scale-dependent overhead: in, This indicates fixed aggregation overhead, such as result collection initialization. The aggregation overhead coefficient represents the increase in the number of subtasks. from arrive Perform a one-dimensional search within the range and calculate each Corresponding target value: ,in, This is an over-decomposition penalty coefficient, used to suppress unnecessary excessive decomposition; Choose the option that minimizes the objective value. As the optimal number of parts : The optimal decomposition scheme is obtained through the above heuristic pre-optimization process. ,in and These are tasks The optimal decomposition scale vector and A set of target nodes, This represents the optimal target node for the j-th subtask assignment.

7. The method of claim 1, wherein, Step S3 specifically includes the following steps: S301. Unified entity representation, including the establishment of a unified entity abstraction and a unified scheduling optimization model; Unified Entity Set The unified entity set, comprising indivisible task entities, undivisible divisible task entities, and decomposed subtask entities, is defined as follows: in, Indicates time slot An indivisible set of tasks Indicates time slot A set of decomposable tasks, and having , Indicates task Decomposition decision, Indicates the number of parts; Node utilization is defined as the sum of the proportions of the computing power allocated to each node: in, Represents a node In the time slot Utilization rate; Indicates time slot Assigned to nodes Total computational load, Represents a node In the time slot Total processing capacity within the system; Global average utilization is defined as the average of the utilization of all nodes: The unified latency model decomposes entity completion latency into three parts: transmission latency, queuing latency, and computation latency. The entity completion latency is the sum of these three. ,in Indicates the task completion time. This indicates the time when the task will arrive, while They are entities Transmission delay, entity Queuing delay and physical The computational delay; S302, Establishment of a unified scheduling optimization problem; Based on the modeling in step S301, the objective function of the unified scheduling optimization problem is defined as: in, Representing entities The weights are used to reflect the latency sensitivity of different tasks; Representing entities The completion delay is the time from when the task arrives until it is completed. The secondary target weight for utilization rate is much smaller than that of the target. Prioritize latency. Indicates time slot Global average utilization rate; The constraints are as follows: 1) Node computing power constraint, used to ensure that the total computing power allocated to each computing node in a time slot does not exceed the maximum computing power it can handle; 2) Link bandwidth constraints are used to ensure that the total amount of data transmitted on each communication link does not exceed the maximum transmission capacity of that link; 3) Assign uniqueness constraints to ensure that each task / subtask must be assigned to one node for execution; 4) Path uniqueness constraint, which ensures that once a task / subtask is assigned to a node, it has one and only one path for transmission.

8. The method of claim 1, wherein, Step S4 includes constructing a reward function that includes a latency term, utilization term, timeout penalty, constraint violation penalty, decision consistency penalty, and correction cost, and defining the reward as the negative value of the system cost; The ADUS-TD3 deep reinforcement learning algorithm includes a feasibility correction and rollback mechanism: first, node capacity and link bandwidth are checked, and non-compliant subtasks are reallocated; if it is still not feasible, the number of decompositions is reduced step by step. If decomposition is still not feasible with only 1 part, cancel the decomposition; if all corrections fail, revert to the heuristic pre-optimization scheme.

9. The adaptive decomposition and unified scheduling method of hybrid tasks in a computing power network according to claim 1 or 8, characterized in that, Step S4 involves the following steps: S401、State space definition: the state space S contains the node residual computing power , node queue workload , link available bandwidth , current task characteristics and historical conditions; S402, Action Space Definition: Action space A is defined as continuous parameterized actions, covering the original outputs of decomposition decisions, scheduling decisions, and resource allocation decisions; S403, Action Mapping Mechanism: The action mapping function maps continuous actions into executable discrete actions, including using a threshold to determine whether to decompose the mapping; S404, Reward Function Design and MDP Problems: In the formula This represents the calculation of mathematical expectation. Discount factor of Power of 1 Indicates the state Execute action The instant reward obtained afterward Indicates all strategies Find the maximum value; S405, solved using the ADUS-TD3 deep reinforcement learning algorithm. The network architecture of this algorithm includes... network Two are responsible for independently generating actions. network Evaluate the value of the action and the corresponding target network. Used to calculate the target Q value; S406. Based on the ADUS-TD3 deep reinforcement learning algorithm optimization feasibility correction and rollback mechanism, the strategy adjustment process is as follows: Node capacity check; if a node exists... If the predicted increase in workload exceeds the threshold, the subtask will be reassigned to the next candidate node. Link bandwidth check: If link congestion results in insufficient available bandwidth on the path, switch the path to an alternative path; Decompose and rollback: If reallocation is still not feasible, execute decompose and rollback: Recalculate And reselect Node; if Still not feasible, let ; Pre-optimization scheme rollback, if the above modifications all fail, rollback to the pre-optimization scheme output by the pre-optimization stage .

10. The method of claim 1, wherein, Step S5 specifically includes: S501, The scheduling decision is sent to the computing network controller for execution. If the decomposition decision output by the ADUS-TD3 deep reinforcement learning algorithm is... Then, according to the number of decompositions output by the ADUS-TD3 deep reinforcement learning algorithm , proportional vector and target node set Execute task decomposition and generate Each subtask entity is assigned to a corresponding node; if If so, the task is scheduled to the target node as a single entity; S502. After executing the scheduling, update the node queue workload status: and link bandwidth status, resulting in the state of the next time slot ; S503, Calculate Rewards , experience tuple The data is stored in the experience replay buffer, and finally the network parameters are updated according to the ADUS-TD3 deep reinforcement learning algorithm update rules; whereby... This indicates the environmental state of the computing power network system in the current scheduling time slot. This indicates that the Actor neural network is based on the current state. Directly output unmapped, continuously parameterized actions. This represents the feedback reward value obtained by the system after performing an action. This indicates the new state of the system when it enters the next scheduling slot after the scheduling decision has been executed in the current time slot, that is, after the decomposition and scheduling decision has been issued and the resources have been updated. During online inference, the following steps are performed in each time slot: read state ; For task sets Each task in the process undergoes pre-optimization assessment, and the output is... and ; By Network inference obtains ; Performing action mapping results in discrete decisions ; Perform decision constraint verification and feasibility correction; Issue and execute scheduling decisions; observe next state and compute reward ; storing experiences to the experience replay buffer; Update network parameters by sampling from experience replay according to the ADUS-TD3 update rules.

Images