Cloud edge end collaborative computing power network hierarchical scheduling optimization method and system and medium

Abstract

The application discloses a kind of based on cloud edge end coordination's computing power network layered scheduling optimization method, system and medium, the method steps include: step S01. The information of node in computing power network is acquired, and computing power network uses by cloud edge end three-layer computing architecture;Step S02. According to the dependence between task node establishes DAG;Step S03. MDP model is constructed, state space is defined as the combination of DAG information and task unloading plan, action space is defined as unloading task to any layer, reward function is defined as with the maximum service quality of scheme as target, and service quality is calculated according to time delay and energy consumption;Step S04. Task unloading decision scheme is generated using deep reinforcement learning model, and all tasks are executed in cooperation according to task unloading decision scheme.The application can improve the execution efficiency and resource utilization of computing task by three-layer computing architecture, and significantly improve the overall performance and flexibility of system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computing power network, and in particular to a computing power network hierarchical scheduling optimization method and system based on cloud edge-end cooperation and a medium. BACKGROUND

[0002] With the rapid development of cloud computing, edge computing and 5G / 6G communication technology, computing power network has gradually become a new type of infrastructure supporting intelligent society. In the computing power network, resource allocation and task scheduling are one of the key problems, and the core goal is to dynamically schedule distributed heterogeneous resources (such as computing, storage, network bandwidth) to meet the needs of diversified businesses for low latency, high reliability and energy efficiency optimization.

[0003] For task scheduling in computing power network, the existing technology usually adopts scheduling methods based on rules or static optimization models. The rule-based scheduling method is to assign priorities according to the urgency and importance of tasks, and to process high-priority tasks first, or to assign tasks to nodes according to the resource requirements (such as CPU, memory, storage, etc.) of the tasks and the resource state of the nodes. The scheduling method based on static optimization model is to select the corresponding nodes for task scheduling according to the cost of resources, the load of resource nodes or energy consumption, etc. However, the above scheduling methods based on rules or static optimization models are difficult to cope with the coupling problems of dynamic heterogeneous resource pool, complex business demand and multi-objective constraints. Especially in real-time sensitive scenarios such as Internet of Things and industrial Internet, the response speed, global optimization ability and environmental adaptability of traditional scheduling methods based on rules or static optimization models have significant bottlenecks.

[0004] Although various methods have been proposed by practitioners to optimize the allocation of edge computing resources and task scheduling, such as heuristic algorithms, game theory, deep learning and deep reinforcement learning, etc., they are usually designed for single cloud computing task allocation or edge computing load balancing, i.e. the allocation of computing resources and task scheduling is carried out in the cloud or edge computing end. However, the computing power of single cloud or edge computing end is limited, which is difficult to meet the complex business demand and complex heterogeneous computing environment. SUMMARY

[0005] The technical problem to be solved by the present application is that, in view of the technical problems existing in the prior art, the present application provides a computing power network hierarchical scheduling optimization method, system and medium based on cloud edge-end cooperation, which has the advantages of simple implementation method, low cost, high execution efficiency and resource utilization rate, and strong flexibility, can improve the execution efficiency and resource utilization rate of computing tasks by integrating three-layer computing architecture, and significantly improve the overall performance and flexibility of the system,

[0006] To solve the above technical problems, the technical solution provided by the present application is:

[0007] A hierarchical scheduling optimization method for computing power networks based on cloud-edge-device collaboration includes the following steps:

[0008] Step S01. Obtain information about task nodes and dependent target task nodes in the computing power network, wherein the computing power network adopts a three-layer computing architecture consisting of cloud, edge computing layer and local user terminal;

[0009] Step S02. Use the obtained task nodes as nodes and build a DAG (Directed Acyclic Graph) based on the dependency relationships between task nodes to represent the dependency relationships between tasks.

[0010] Step S03. Model the task scheduling problem as a Markov decision process and construct an MDP (Markov Decision Process) model;

[0011] In the constructed MDP model, the state space is defined as a combination of DAG information and task offloading plan. The DAG information includes DAG topology information and task embedding information. The action space is defined as offloading tasks to computing nodes in any layer of the three-layer computing architecture. The reward function is defined as maximizing the service quality of the task scheduling scheme. The service quality of the task scheduling scheme is calculated based on the latency and energy consumption of each task in the scheduling scheme. The latency of each task is obtained based on the maximum value among the local user terminal completion time, the edge computing layer completion time, and the cloud completion time. The energy consumption of each task is obtained based on the energy consumption generated by executing the task on the local user terminal, the energy consumption generated by offloading the task to the edge computing layer, and the energy consumption generated by offloading the task to the cloud.

[0012] Step S04. Based on the constructed MDP model, a deep reinforcement learning model is established using deep reinforcement learning methods. The current task to be executed is embedded into the pre-trained deep reinforcement learning model to generate a corresponding task offloading decision scheme. The local user terminal, edge computing layer, and cloud collaboratively complete the execution of all tasks according to the generated task offloading decision scheme.

[0013] Furthermore, the reward function is defined as follows:

[0014] ,

[0015] ,

[0016] ,

[0017] in, , These represent the average latency and energy consumption of tasks in the DAG, respectively. Indicates task The amount of latency contributed by the individual. Indicates task Energy consumption contributed independently and These represent the first task to the next task. Cumulative energy consumption, from the first task to the next task Cumulative energy consumption Indicates the transition from the first task to the next task. The cumulative delay, Indicates the transition from the first task to the next task. The cumulative delay, and These represent the total latency and total energy consumption metrics under the local execution baseline, respectively. It is the weighting coefficient for delay. It is the weighting coefficient for energy consumption. , They represent the first i The first task, the first i- 1 task This indicates that the task is unloaded and executed at a different level. This indicates that the task is executed locally. This indicates that the task is executed at the edge. This indicates that the task is executed in the cloud. Indicates selecting a task action The state at that time.

[0018] Furthermore, the time delay of the task scheduling scheme is calculated according to the following formula:

[0019] ,

[0020] ,

[0021] ,

[0022] ,

[0023] in, This represents the total latency of the task scheduling scheme. Indicates the first i One task, This represents the set of exit tasks in the DAG graph. This indicates that the task is executed on the local user's device. Completion time, This indicates that the task is executed at the edge computing layer. Completion time, Indicates performing tasks in the cloud. Time to complete the processing phase; Indicates task Available time on the local user's device. Indicates task The set of precursor missions, Indicates task The direct precursor mission, This indicates that the task is executed on the local user's device. Completion time, This indicates that the edge computing layer has received the task. The completion time of the result, This indicates that the task has been received in the cloud. The completion time of the result, This indicates executing a local task on the local user's device. The time delay; Indicates task Available time in the edge computing layer Indicates task The completion time of uplink transmission at the edge. This indicates that the task is executed at the edge computing layer. Completion time, This indicates that the task is executed at the edge computing layer. Execution latency; Indicates task Available time in the cloud Indicates that the task is executed in the cloud. The upload phase completion time, Indicates performing tasks in the cloud. The completion time of the processing phase, Indicates performing tasks in the cloud. The execution delay.

[0024] Furthermore, if the task is offloaded to the cloud, The latency includes: latency during the cloud transmission phase. Cloud execution phase latency and latency during cloud reception ,in, , These represent cloud upload tasks. The amount of data and the cloud upload and transmission rate, Indicates the execution of a task Number of CPU cycles required This indicates the CPU clock speed of the virtual machine in the cloud. , These respectively indicate that the task has been received in the cloud. The amount of data and the downlink propagation rate in the cloud;

[0025] If the task is offloaded to the edge computing layer, The latency includes: latency during the edge computing layer transmission phase. Latency during the execution phase of the edge computing layer and the latency of the edge computing layer receiving stage ,in, Indicates the uplink propagation rate at the edge computing layer. This indicates the CPU clock speed of the edge computing layer. , These represent the tasks received by the edge computing layer. The amount of data and the downlink propagation rate of the edge computing layer;

[0026] If executed on a local user's device, the local latency is... , This indicates the CPU clock speed of the local user terminal;

[0027] Task Completion time of uplink transmission at the edge The calculation expression is:

[0028] ,

[0029] in, Indicates task Available time during the upload phase when executing in the cloud Indicates task Cloud transmission latency;

[0030] Cloud-based task execution Time to complete during the upload phase The calculation expression is:

[0031] ,

[0032] ,

[0033] in, Indicates task Time required to complete the cloud download phase. Indicates task The availability time of the downlink radio channel, Indicates performing tasks in the cloud. The completion time of the processing phase, Indicates task Latency during the receiving phase when unloading to the cloud. Indicates task Transmission latency in the cloud.

[0034] Furthermore, the energy consumption of the task scheduling scheme is calculated according to the following formula:

[0035] ,

[0036] ,

[0037] ,

[0038] in, This represents the total energy consumption of the task scheduling scheme. Indicates the first i One task, Represents a set of tasks. This indicates that the task is executed on the local user's device. The energy consumption generated , It is the power factor. It is a constant. Indicates the CPU frequency of the local device Power of 1 This indicates the latency of executing local tasks on the local user's device. Indicates task Energy consumption generated by offloading to the edge computing layer and These are the transmit power and receive power, respectively. Indicates task Energy consumption generated from offloading to the cloud; This indicates that the task is unloaded and executed at a different level. This indicates that the task is executed locally. This indicates that the task is executed at the edge. This indicates that the task is executed in the cloud. Indicates task The transmission phase latency during offloading to the edge computing layer Indicates task Reception stage latency when offloading to the edge computing layer Indicates task Delay during the sending phase when unloading to the cloud. Indicates task Latency during the receiving phase when unloading to the cloud.

[0039] Furthermore, the policy in the deep reinforcement learning model Indicates the state The following is the task Select Action The probability, This indicates that the task is unloaded and executed at a different level. This indicates that the task is executed locally. This indicates that the task is executed at the edge. This indicates that the task is executed in the cloud. This represents the graph structure obtained after encoding the DAG. Indicates from the task To the mission The task unloading plan for the task sequence is obtained by using an S2S neural network as the policy network in the deep reinforcement learning model to obtain the unloading decision for each task.

[0040] Furthermore, in the deep reinforcement learning model, the policy network and value network share the parameters of the encoder and decoder networks to extract general features from the DAG. The policy network also includes a fully connected layer on the decoder output, and uses a softmax function to convert the output into a distribution of actions. Similarly, the value network includes another fully connected layer on the decoder output to output state values. .

[0041] Furthermore, the step of using an S2S neural network as the policy network to obtain the unloading decision for each task includes:

[0042] The tasks in the DAG are sorted by calculating their priority values ​​using the following formula:

[0043]

[0044]

[0045]

[0046]

[0047] in, Indicates task priority value, Indicates task The set of direct successor tasks, Indicates task The direct successor task, This represents the set of export tasks. Indicates task Delay during the sending phase when unloading to the cloud. Indicates task Delay during the execution phase when unloading to the cloud. Indicates task Latency during the receiving phase when unloading to the cloud. Indicates task The transmission phase latency during offloading to the edge computing layer Indicates task The execution phase latency when offloading to the edge computing layer Indicates task Reception stage latency when offloading to the edge computing layer Indicates task Total latency when offloading to the edge computing layer. Indicates task Total latency when uninstalling to the cloud;

[0048] Each task is encoded into a series of embedding vectors according to the calculated priority value, and used as input to the S2S neural network. The S2S neural network then generates the task unloading decision output.

[0049] A cloud-edge-device collaborative computing network hierarchical scheduling optimization system includes a processor and a memory. The memory is used to store computer programs, and the processor is used to execute the computer programs to perform the methods described above.

[0050] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0051] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0052] 1. This invention adopts a three-layer computing architecture (cloud, edge, and device) and uses a deep reinforcement learning model for task scheduling decisions across these three layers. The model models the communication and computation processes of tasks, determines the latency and energy consumption of tasks based on the latency and energy consumption at each endpoint, evaluates the service quality of the scheme based on the latency and energy consumption of the tasks, and then uses deep reinforcement learning to generate a scheduling scheme with the goal of maximizing service quality. This results in the lowest possible latency and energy consumption during the execution of the scheduling scheme, thereby improving the execution efficiency and resource utilization of computing tasks.

[0053] 2. This invention uses a directed acyclic graph (DAG) to represent the dependencies between tasks during the scheduling process, and combines deep reinforcement learning methods to optimize the scheduling order of tasks. By considering the dependencies between tasks during the scheduling process, it can ensure that the order of tasks and resource allocation can meet the computational dependency constraints in a multi-task environment, reduce resource contention and waiting time during task execution, and improve the overall execution efficiency of computational tasks.

[0054] 3. By adopting a three-layer computing architecture, this invention can improve the overall task processing capability of the system. At the same time, through global optimization and dynamic allocation of computing resources, it can significantly improve the overall performance of the system, such as response speed, processing capability and resource utilization. The three-layer computing architecture can also enhance the system's flexibility, reduce the system's operating costs, and avoid unnecessary waste of computing resources and duplication of construction. Attached Figure Description

[0055] Figure 1 This is a schematic diagram illustrating the implementation process of the cloud-edge-device collaborative computing power network hierarchical scheduling optimization method in this embodiment.

[0056] Figure 2 This is a schematic diagram illustrating the principle of the three-layer computing architecture constructed in this embodiment.

[0057] Figure 3 This is a schematic diagram illustrating the principle of the specific three-layer computing architecture constructed in a particular application embodiment of the present invention.

[0058] Figure 4 This is a flowchart illustrating the process of executing tasks using a three-tier computing architecture. Detailed Implementation

[0059] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0060] This invention employs a three-layer computing architecture encompassing cloud computing, edge computing, and terminal devices. It combines deep reinforcement learning methods to achieve layered scheduling of computing power networks in a cloud-edge-device collaborative manner. This adapts to the complexity of heterogeneous computing environments, enabling intelligent allocation and dynamic scheduling of computing resources to meet complex business needs. Furthermore, during task scheduling, the optimization objective is to maximize the service quality while considering both task latency and energy consumption. Task latency and energy consumption are determined by considering the impact of the cloud on these factors, adapting to the three-layer cloud-edge-device computing architecture. This comprehensive consideration of cloud-based task latency and energy consumption improves the accuracy of service quality assessment in the scheduling scheme. By combining a deep reinforcement learning model with the goal of maximizing service quality, the latency and energy consumption of task scheduling are minimized, effectively improving the execution efficiency and resource utilization of computing tasks.

[0061] like Figure 1 As shown, the steps of the cloud-edge-device collaborative computing power network hierarchical scheduling optimization method in this embodiment include:

[0062] Step S01. Obtain information about computing nodes and dependent target task nodes in the computing power network. The computing power network adopts a three-layer computing architecture consisting of cloud, edge computing layer and local user terminal.

[0063] In this embodiment, the information obtained about task nodes and dependent target task nodes in the computing power network includes task identifier (ID), task running time, task dependency relationship (i.e., predecessor and successor in the directed acyclic graph), task dependency depth, amount of data contained in the task, number of CPUs required by the task, etc.

[0064] In this embodiment, based on the characteristics and transmission features between dependent tasks in the computing power network, a three-layer computing architecture is established considering the cloud, edge, and local terminals. Compared to the traditional two-layer architecture that only includes the terminal device layer and the edge computing layer, this three-layer computing architecture can form a more powerful and flexible computing architecture. Figure 2 As shown, the cloud computing layer includes at least one cloud server, the edge computing layer includes multiple edge computing machines (MECs) and a training offloading and task graph pool, wherein the task graph pool stores information of the DAG graph constructed by the local computing layer, the training offloading is used to train an offloading model based on a deep reinforcement learning model to implement task offloading decisions, the local computing layer includes a local processor (UE), on which a DAG graph is built according to the dependencies between tasks, and the edge computers, local processors and cloud servers collaboratively execute all tasks according to the offloading plan.

[0065] In specific application embodiments, such as Figure 3 As shown, in this embodiment's three-layer computing network architecture, the cloud computing layer, edge computing layer, and local computing layer work together to schedule and execute application tasks. The cloud computing layer provides centralized computing and storage resources for processing tasks with high complexity or low latency sensitivity. The edge computing layer consists of multiple edge computing machines (MECs), which are interconnected through the core network. They can not only independently execute assigned tasks but also collaboratively handle cross-node task data transmission and computation. The local computing layer includes end-user equipment (UE), which performs some lightweight tasks locally.

[0066] In this three-layer computing network architecture, application tasks are represented as a Directed Acyclic Graph (DAG). Different task nodes and their dependencies are represented as nodes and edges in the graph. The DAG is stored in a task graph pool, and the execution position of each task node is determined based on the deep reinforcement learning offloading model trained by the offloading module. When tasks are assigned to different computing nodes, the core network provides communication guarantees for data transmission between tasks. Finally, the cloud computing layer, edge computing layer, and local computing layer jointly complete the execution of all tasks in the DAG according to the offloading plan, thereby achieving efficient, low-latency computing power collaboration and optimized resource utilization.

[0067] Step S02. Use the obtained task nodes as nodes and build a DAG based on the dependencies between task nodes to represent the dependencies between tasks.

[0068] In this embodiment, based on the dependencies between tasks, task nodes are simplified into a directed acyclic graph (DAG) composed of nodes and directed edges. Specifically, each task node is used as a node (vertices) in the DAG, and a corresponding node set V is created. The dependencies between task nodes are abstracted as edges (directed edges) between nodes in the DAG. Each edge represents a dependency between nodes, and each edge can be represented by an ordered pair (u,v) from node u to node v. Therefore, a DAG can be used to represent the dependencies between tasks. By constructing a DAG, it can be ensured that in a multi-task environment, the order of tasks and resource allocation satisfy computational dependency constraints. Subsequent task scheduling decisions are based on this DAG, taking into account the dependencies between tasks during scheduling, thereby optimizing the scheduling order and improving the overall execution efficiency of computational tasks.

[0069] Step S03. Model the task scheduling problem as a Markov decision process and construct an MDP model.

[0070] To enable Deep Reinforcement Learning (DRL) to adapt to solving task scheduling problems, this embodiment first models the task scheduling objective problem of the computing power network as a Markov decision process, using a 6-tuple. Define the computing power network scheduling problem, where and Let R represent the state space, action space, state transition probability matrix, and initial state distribution, respectively, and let R represent the reward function. This represents the discount factor. Assume that in a network scenario, each device on the network can initiate at most one computational task simultaneously. The state space, action space, and reward function of the MDP are defined as follows: The state space is... Action space , Indicates that the task starts from the node. Scheduled to service node Process it; if this action does not exist, then... Define the task set as Where n represents the number of tasks, and the reward function is... For time Instant rewards. As a strategy, specify the action to be taken in a given state s. a The probability of.

[0071] To adapt to the three-tier computing architecture of cloud, edge computing layer, and local user terminal, this embodiment further considers the latency and energy consumption of task execution in the cloud, and the latency and energy consumption during the task unloading process. The detailed steps are as follows:

[0072] This embodiment is based on a three-layer computing architecture. Tasks can be offloaded to the cloud, to the MEC host (a node in the edge computing layer), or to the UE (user equipment) on the local user end for execution. During the calculation of latency, the following is first defined... and These represent the CPU clock speeds used for virtual machines in the UE and MEC hosts, respectively. Indicates the execution of a task Number of CPU cycles required Indicates the first i One task, For the task The transmission stage latency when offloading to the edge computing layer, i.e., the task... Transmission latency at the edge computing layer For the task The execution phase latency when offloading to the edge computing layer, i.e., task latency. In the execution latency of MEC, For the task The latency of the receiving phase when offloading to the edge computing layer, i.e., the task In the MEC's ​​reception delay, Indicates task The latency during the sending phase when unloading to the cloud, i.e., the task... Cloud transmission latency Indicates task The execution phase latency when unloading to the cloud, i.e., task latency. Execution latency in the cloud Indicates task The latency during the receiving phase when unloading to the cloud, i.e., the task... Reception latency in the cloud. Task The available time for the uplink wireless channel, edge computing layer (MEC host), downlink wireless channel, local user terminal, and cloud are respectively , , , , , This indicates that the task is executed on the local user's device. Completion time, This indicates that the edge computing layer has received the task. The completion time of the result, To indicate the task The direct precursor mission.

[0073] Specifically, computational tasks The latency, if the task is offloaded to the cloud, includes three stages:

[0074] (1) Cloud transmission stage latency: The UE transmits the task via the wireless channel. Send to the cloud, then the task Delay during the sending phase when unloading to the cloud ,in , These represent cloud upload tasks. The amount of data and the cloud upload and transmission rate.

[0075] (2) Cloud execution phase latency: The cloud executes the received tasks. Then the task Execution phase latency when unloading to the cloud , Indicates the execution of a task Number of CPU cycles required This indicates the CPU clock speed of the virtual machine in the cloud.

[0076] (3) Cloud reception stage latency: The cloud returns the result to the user UE, then the task... Receiving stage latency when unloading to the cloud ,in , These respectively indicate that the task has been received in the cloud. The amount of data and the downlink propagation rate in the cloud.

[0077] If offloading to the edge computing layer (MEC) also involves three stages:

[0078] (1) Edge computing layer transmission stage delay: The UE transmits the task through the wireless channel The task is sent to the MEC host. Transmission phase latency when offloading to the edge computing layer ,in , These represent the amount of data uploaded and the upload propagation rate of the edge computing layer, respectively.

[0079] (2) Edge computing layer execution phase latency: The MEC host executes the received task Then the task Execution phase latency when offloading to the edge computing layer , Indicates the execution of a task Number of CPU cycles required This indicates the CPU clock speed of the edge computing layer.

[0080] (3) Edge computing layer reception stage delay: When the MEC host returns the result to the user UE, the task... Reception phase latency when offloading to the edge computing layer , , These represent the tasks received by the edge computing layer. The amount of data and the downlink propagation rate of the edge computing layer.

[0081] If the local execution time is ,but , This indicates the CPU clock speed of the local user terminal.

[0082] Calculate the task to be executed on the local client Completion time At this time, a task can only begin after all direct predecessors have completed their tasks, and these direct predecessor tasks may be located on the MEC or locally. It is a task All of the precursor missions, It is a task direct front-wheel drive, For the edge computing layer to receive tasks The completion time of the result, Execute tasks on the local user client. The completion time will be determined by obtaining the task from the cloud and executing it on the local user's device. Completion time The calculation expression is:

[0083] (1)

[0084] in, Indicates task Available time on the local user's device. This indicates that the task is executed on the local user's device. Completion time, This indicates that the task has been received in the cloud. The completion time of the result.

[0085] Then the task The available time on the local user terminal (local processor) is updated as follows:

[0086]

[0087] in, Indicates task Available time on the local user's device. This indicates that the task is executed on the local user's device. The completion time.

[0088] The latency calculation for task unloading is a continuous summation process. When each task arrives, this embodiment selects one of the three endpoints (cloud, edge, and client) to perform task unloading. Since each endpoint may have tasks running, and there may be dependencies between tasks, it is necessary to consider the completion time of the previous task. The next task can only begin execution after the previous task is completed. This embodiment calculates the time of the currently being processed for the cloud, edge computing layer, and local user endpoint respectively, and comprehensively considers the completion time of the local user endpoint, the edge computing layer, and the cloud endpoint to obtain an accurate completion time for the task executed by the local user endpoint. This can accurately model the local completion time in a three-layer architecture.

[0089] Furthermore, the completion time of tasks executed by the MEC (Mean Edge Computing) layer includes:

[0090] When the task When scheduled for execution at the edge computing layer (MEC), it needs to wait. All precursor missions Complete; if task If executed locally, calculations will be performed after the local execution is complete; if it is a predecessor task... If the task is uploaded to the MEC (Medium-Range Computing) layer for execution, it must wait for the MEC layer to complete its execution; if the preceding task... If the task is uploaded to the cloud for execution, you will need to wait for the cloud execution to complete. Send the task to the edge computing layer (MEC) before sending it. .

[0091] This embodiment considers the impact of the cloud on latency when calculating the completion time of the task transmission phase in the edge computing layer (MEC). Completion time for uploading to the edge computing layer The calculation expression is:

[0092] (2)

[0093] in, Indicates task Available time during the upload phase when executing in the cloud Indicates task The set of precursor missions, For the edge computing layer to receive tasks The completion time of the result, i.e., the completion time of the downlink channel during the download phase of the edge computing layer, is calculated as follows: , Indicates task The direct precursor mission. Indicates task Transmission latency in the cloud.

[0094] Then the task Available time of uplink radio channel Updated to:

[0095] (3)

[0096] in, Indicates task Available time of uplink wireless channel in the cloud. Indicates task The completion time of uplink channel transmission.

[0097] Execute the task The execution requires: First, the task The sending process should be completed. Secondly, Precursor mission The execution should be completed when the edge computing layer (MEC) host is not occupied by other tasks, that is, when the edge computing layer (MEC) host is available. For the task The available time during the execution phase at the edge computing layer, and therefore the completion time:

[0098] (4)

[0099] in, Indicates the execution of a task Time required to complete at the edge computing layer This indicates that the task is performed at the edge computing layer. The execution latency of the task. The available time of the uplink channel is ,in Indicates task During the available time of the uplink channel, Indicates the execution of a task Time is completed at the edge computing layer.

[0100] With the task The available time of the uplink channel is Similarly, during the reception phase, the downlink channel completion time... The available time for updating the downlink is: .

[0101] Therefore, the completion time for cloud execution includes the upload phase completion time, the cloud processing phase completion time, and the download phase completion time. Thus, this embodiment considers the sum of the end time of the previous edge task and the upload time of the current task as the upload phase completion time, i.e., the cloud execution task... Time to complete during the upload phase The calculation expression is:

[0102] (5)

[0103] Then the task The available time for the upload phase when executing in the cloud is:

[0104] (6)

[0105] Performing tasks in the cloud The completion time for the processing phase is:

[0106] (7)

[0107] As shown in equations (5) to (7), the maximum value between the end time of the upload task and the completion time of the previously sorted tasks and the current available time in the cloud is taken as the start time of the cloud processing task. Then, the start time is added to the execution time in the cloud to obtain the time when the cloud processing stage is completed.

[0108] This embodiment adds the ability to receive tasks from the cloud. Completion time of results Furthermore, when considering upload time, the end time of the previous edge task is added to the upload time of the current task to obtain the upload phase completion time. This can be adapted to the cloud-edge-device three-layer computing architecture to accurately model the upload phase completion time of tasks executed in the cloud.

[0109] Furthermore, the task The available time during the cloud processing phase can be expressed as:

[0110] (8)

[0111] This allows for task execution in the cloud. The download phase completion time is:

[0112] (9)

[0113] Task The available time during the cloud download phase is:

[0114] (10)

[0115] Energy consumption includes both computation and transmission. Local computation generates a certain amount of energy, and tasks are executed on the local user end. Energy consumption is recorded as Defined as , It is a power factor. Indicates the CPU frequency of the local device Power of 1. This represents the operating frequency of the local CPU. A higher frequency means a greater amount of computation can be processed per unit time. This can depict the effect of a sharp increase in power consumption at higher frequencies. It can take values ​​close to 3, representing a power-law relationship between energy consumption and frequency. For example, if The energy consumption is linearly related to the frequency. Considering that power consumption increases more rapidly when the CPU frequency increases, a suitable value can be selected. . It depicts the effect that the higher the frequency, the more power consumption increases.

[0116] When the task is unloaded, the power consumption mainly comes from wireless transmission. The energy consumption of offloading to the edge computing layer is denoted as (Transmission energy consumption is directly proportional to power and transmission time.) and These are the transmit power and receive power, respectively. Then the task... Energy consumption generated by offloading to the edge computing layer The calculation expression is:

[0117] (11)

[0118] Similarly, the task The energy consumption generated by offloading to the cloud is:

[0119] (12)

[0120] Task dependencies are represented by a directed acyclic graph (DAG). Based on the above definition, when a DAG task scheduling scheme is given, the overall time delay of the task scheduling scheme can be calculated:

[0121] (13)

[0122] in, This represents the total latency of the task scheduling scheme. This represents the set of exit tasks in a DAG (Directed Acyclic Graph). Exit tasks are those in the DAG that have no successor tasks, representing the "end point" of the entire process. This indicates that the task is executed on the local user's device. Completion time, This indicates that the task is executed at the edge computing layer. Completion time, Indicates performing tasks in the cloud. Time to complete the processing phase.

[0123] This embodiment calculates the final completion time of tasks on each terminal by considering the impact of cloud latency, and finally determines the total latency of the task by combining the final completion time of tasks on each terminal. This allows for a more accurate determination of the overall latency of the entire scheduling scheme, enabling more precise energy consumption modeling for a three-layer computing architecture.

[0124] Accordingly, considering the impact of the cloud on task energy consumption, the total energy consumption of the task is determined by comprehensively considering the energy consumption when executing the task on each terminal, and thus the overall energy consumption of the task scheduling scheme can be determined:

[0125] (14)

[0126] in, This represents the total energy consumption of the task scheduling scheme. Indicates task Energy consumption generated by offloading to the edge computing layer Indicates task Energy consumption generated by offloading to the cloud This indicates that the task is executed on the local user's device. The energy consumption generated This indicates that different sets of tasks are represented, if This indicates that the task is executed locally. This indicates that the task is executed at the edge. This indicates that the task is executed in the cloud. This represents the set of all tasks.

[0127] Thus, a computational model is constructed to calculate the latency and energy consumption of each task in the computational task scheduling scheme. The latency of each task is obtained based on the maximum value among the local user terminal completion time, the edge computing layer completion time, and the cloud completion time, as shown in Equation (13). The energy consumption of each task is obtained by combining the energy consumption generated at the local user terminal, the energy consumption generated when the task is unloaded to the edge computing layer, and the energy consumption generated when the task is unloaded to the cloud, as shown in Equation (14).

[0128] This embodiment further calculates QoS (Quality of Service) based on the latency and energy consumption of each task execution in the task scheduling scheme to measure the merits of the task scheduling scheme, that is, comprehensively considering latency and energy consumption to evaluate the performance of the scheduling scheme. In this embodiment, QoS is defined as the weighted sum of the normalized differences in latency and energy consumption between the task scheduling scheme (offload scheme) and local execution:

[0129] (15)

[0130] in, and These are the latency and energy consumption of all tasks executed locally on the local user terminal (UE).

[0131] This embodiment further models the task scheduling problem as a Markov decision process. The state space, action space, and reward function of the Markov decision process are defined as follows: The state space is defined as the state space in the task scheduling process. At that time, the state of the edge computing layer depends on the scheduled tasks. The scheduling result of the predecessor task (i.e., the partial offloading plan) is used to define the state space as a combination of DAG information and task offloading plan. The DAG information includes DAG topology information and task embedding information (task overview). The action space is defined as offloading the task to any computing node in the three-layer computing architecture. The reward function is defined as maximizing the service quality of the task scheduling scheme. The service quality of the task scheduling scheme is calculated based on the latency and energy consumption of each task in the scheduling scheme. The latency of each task is obtained based on the maximum value of the local user terminal completion time, the edge computing layer completion time, and the cloud completion time. The energy consumption of each task is obtained by combining the energy consumption generated at the local user terminal, the energy consumption generated when the task is offloaded to the edge computing layer, and the energy consumption generated when the task is offloaded to the cloud.

[0132] Specifically, set This represents the encoded DAG. Indicates starting from the first task To the i Task The unloading plan for the task sequence, therefore the state space is defined as G consists of a series of task embeddings, each of which can contain the following three elements: 1) a vector containing the task The index and estimated task cost, local execution latency Delay during the sending phase when unloading to the cloud Execution phase latency when unloading to the edge computing layer Latency during the receiving phase when unloading to the cloud Delay during the transmission phase when offloading to the edge computing layer Execution phase latency when unloading to the edge computing layer Reception stage latency when offloading to the edge computing layer ; 2) An index vector for an immediate predecessor task; 3) An index vector for an immediate successor task. The length of the predecessor / successor task index vectors is limited to [value missing]. (For example If the number of preceding / following tasks is less than If the vector is not filled, then -1 is used to fill it. In the subsequent execution of step S04, this embedding vector is input into the S2S neural network of the deep learning model to obtain the unloading plan.

[0133] When defining the action space, a task in a computing network can be offloaded to the cloud, an MEC host, or run locally on the UE. This indicates that the task is offloaded to the cloud for computation. This indicates that the data will be unloaded to the MEC host. Indicates task Executed locally, therefore, the action space is defined as follows: .

[0134] In this embodiment, the reward function is defined with the goal of maximizing the Quality of Service (QoS). To achieve this goal, the reward function is defined as the estimated increment of QoS at each step:

[0135] (16)

[0136] (17)

[0137] (18)

[0138] in, Indicates selecting a task action The state at that time, , These represent the average latency and energy consumption of tasks in the DAG, respectively. Specifically, they can be categorized as follows: , Calculations show that Indicates task The amount of latency contributed by the individual. Indicates task Energy consumption contributed independently and These are the first task to the next task. Cumulative energy consumption, from the first task to the next task Cumulative energy consumption , They represent the first i The first task, the first i- 1 task Indicates the transition from the first task to the next task. The cumulative latency (all executed under the baseline strategy). Indicates the transition from the first task to the next task. The cumulative delay, and These are the total latency and total energy consumption metrics under the local execution benchmark. This is a latency weighting factor used to reflect the importance of latency in Quality of Service (QoS). A larger value indicates that the system has higher latency requirements. This is a weighting factor for energy consumption, used to reflect the importance of energy consumption in Quality of Service (QoS). If the value is larger, the proportion of energy consumption will be greater.

[0139] As shown in equation (16), for the delayed portion, Indicates the first The improvement in incremental latency for each task compared to the baseline average latency, if Less than This corresponds to a reduction in latency, which is positive, resulting in a positive reward. In other words, actions that reduce latency are rewarded to minimize delay and energy consumption. Similarly.

[0140] Step S04. Based on the constructed MDP model, a deep reinforcement learning model is established using deep reinforcement learning methods. The current task to be executed is embedded into the pre-trained deep reinforcement learning model to generate a corresponding offloading decision scheme. The local user terminal, edge computing layer, and cloud collaboratively complete the execution of all tasks according to the generated offloading decision scheme.

[0141] This embodiment combines deep reinforcement learning methods with a cloud-edge-device architecture, which can easily realize intelligent allocation and dynamic scheduling of computing resources based on task priority, time requirements, and the load of nodes at each layer. It can quickly and intelligently generate task scheduling schemes with the lowest energy consumption and latency, and flexibly adapt to the complexity of heterogeneous computing environments.

[0142] This embodiment, based on the definition of the MDP model, transforms the offloading problem into a neural network prediction problem, where the input to the neural network is a sequence of task embeddings, and the output is a scheduling plan for these tasks. Specifically, the strategy in the deep reinforcement learning model... Indicates the state The following is the task Select Action The probability, This represents the graph structure obtained after encoding the DAG. Indicates from the task arrive Task sequence, task unloading plan, strategy Indicates the state The following is the task Select Action The probability of. In the picture Given An uninstallation plan is obtained in the case of a task. The probability can be obtained by applying the chain rule of probability:

[0143] (19)

[0144] To effectively approximate the policy defined above, this embodiment uses an S2S (Sequence-to-Sequence) neural network as the policy network to output the offloading decision for each task in the deep reinforcement learning model. The S2S neural network is combined with an attention mechanism to approximate the policy and value function of DRLTO (Deep Reinforcement Learning Dynamic Optimization), including:

[0145] For task embedding sequences , n This represents the number of tasks. The hidden states of the encoder network in the S2S neural network are: , It is the encoding step The hidden state, These are the parameters of the encoder network. Steps Task embedding at the time of input, A function for the encoder network;

[0146] The decoder network of the S2S neural network is input to the hidden states of the previous layer in the decoding stage. The predicted input from the previous stage Given the context vector C(j) of the current encoding stage corresponding to the current prediction, the new state of the current decoding stage is... for:

[0147] (20)

[0148] (twenty one)

[0149] (twenty two)

[0150] in, These are the parameters of the decoder network. It is the context vector of the attention mechanism. It is each hidden state of the encoder network The weight, The scoring function is used to measure the encoder network's performance at a given location. The input and decoder network at location The degree of matching between the outputs.

[0151] In this embodiment, the policy network and value network in the deep reinforcement learning model share the parameters of the encoder network and decoder network to extract general features from the DAG. Furthermore, for the policy neural network, the output of the decoder... A fully connected layer is also added, and the softmax function is used to convert the output into a distribution of actions. For value neural networks, at the output of the decoder Another fully connected layer is also set up on top, and the output is used to represent the state value. .

[0152] This embodiment combines PPO (Proximal Policy Optimization) and S2S neural networks for task scheduling decisions. It can intelligently determine at which level a task should be processed based on task priority, time requirements, and the load of nodes at each layer, thereby effectively avoiding the waste of computing resources and the occurrence of bottlenecks.

[0153] In this embodiment, the steps for obtaining the unloading decision for each task using an S2S neural network as the policy network include:

[0154] Step S401. Sort the tasks in the DAG by calculating their priority values ​​using the following formula through topological sorting:

[0155] (twenty three)

[0156] (twenty four)

[0157] (25)

[0158] (26)

[0159] in, Indicates task priority value, Indicates task The set of direct successor tasks, Indicates task The direct successor task, This represents the set of export tasks. Indicates task Delay during the sending phase when unloading to the cloud. Indicates task Delay during the execution phase when unloading to the cloud. Indicates task Latency during the receiving phase when unloading to the cloud. Indicates task The transmission phase latency during offloading to the edge computing layer Indicates task The execution phase latency when offloading to the edge computing layer Indicates task Reception stage latency when offloading to the edge computing layer Indicates task Total latency when offloading to the edge computing layer. Indicates task Total latency when uninstalling to the cloud;

[0160] Step S402. Encode each task into a series of embedding vectors according to the calculated priority values, and use them as input to the S2S neural network. The S2S neural network then generates the task unloading decision output. For example, input the embedding vectors encoded by each task according to the calculated priority values ​​into the encoder of the S2S neural network. The encoder calculates the context vector, and in the first step... Each decoding step is as follows The uninstallation decision is generated.

[0161] like Figure 4 As shown, when using the three-layer architecture constructed in this embodiment for task scheduling, the task controller at the local user end (local computing layer) prioritizes tasks based on their priority values ​​to schedule high-priority tasks first. At the edge computing layer, based on the computing power information provided by the edge computing point and the task information provided by the task controller, the edge controller obtains a scheduling decision by using a dynamic task scheduling algorithm. Based on the scheduling decision, the edge controller controls the execution of tasks at the local user end, edge computing point, or cloud (cloud computing layer) until the task scheduling is completed.

[0162] In summary, this invention schedules the computing power network by simultaneously considering computing nodes at three layers: cloud, edge, and terminal. By treating service nodes in the computing power resources at each layer as independent action decision points, it can achieve refined action decision-making at the node level.

[0163] By constructing a Directed Acyclic Graph (DAG) to represent the dependencies between tasks, the scheduling process takes into account the dependencies between tasks. At the same time, a deep reinforcement learning model is used for task scheduling decisions across three layers of nodes: cloud, edge, and terminal. The model models the communication and computation processes of tasks, determines the latency and energy consumption of tasks based on the latency and energy consumption of tasks at each terminal, and evaluates the service quality of the scheme based on the latency and energy consumption of the tasks. Then, by combining deep reinforcement learning methods, a scheduling scheme is generated with the goal of maximizing service quality, which can minimize the latency and energy consumption of the scheduling scheme, thereby improving the execution efficiency and resource utilization of computing tasks.

[0164] This invention uses a directed acyclic graph (DAG) to represent the dependencies between tasks during the scheduling process, and combines deep reinforcement learning methods to optimize the scheduling order of tasks. By considering the dependencies between tasks during the scheduling process, it can ensure that the order of tasks and resource allocation can meet the computational dependency constraints in a multi-task environment, reduce resource contention and waiting time during task execution, and improve the overall execution efficiency of computational tasks.

[0165] This invention employs a three-layer computing architecture. The cloud layer provides robust support, allowing the edge computing layer to focus on real-time tasks, while complex tasks are handled by the cloud layer. This enhances the overall task processing capability of the system. Furthermore, through global optimization and dynamic allocation of computing resources, it significantly improves overall system performance, including response speed, processing power, and resource utilization. The three-layer architecture also enables the system to flexibly handle tasks ranging from simple to complex, from high real-time requirements to computationally intensive tasks. It can also adapt to changing task demands in different scenarios, enhancing system flexibility. Whether dealing with high real-time requirements or computationally intensive tasks, the system can intelligently schedule and find the optimal processing path. Optimizing the allocation and utilization of computing resources also helps reduce system operating costs and avoids unnecessary waste and redundant construction.

[0166] This embodiment further provides a cloud-edge-device collaborative computing network hierarchical scheduling optimization system, including a processor and a memory. The memory is used to store computer programs, and the processor is used to execute the computer programs to perform the methods described above.

[0167] It is understood that the method described in this embodiment can be executed by a single device, such as a computer or server, or it can be applied to a distributed scenario where multiple devices cooperate to complete the task. In a distributed scenario, one of the multiple devices may execute only one or more steps of the method described in this embodiment, and the multiple devices interact to complete the method. The processor can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to implement the method described in this embodiment. The memory can be implemented using read-only memory (ROM), random access memory (RAM), static storage devices, and dynamic storage devices. The memory can store the operating system and other applications. When the method described in this embodiment is implemented through software or firmware, the relevant program code is stored in the memory and called and executed by the processor.

[0168] This embodiment further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0169] Those skilled in the art will understand that the above embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce implementations of the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0170] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A cloud edge-end collaborative based computing power network hierarchical scheduling optimization method, characterized by the steps of Comprise: Step S01. Obtain information of task nodes and dependent target task nodes in a computing power network, the computing power network adopts a three-layer computing architecture composed of a cloud, an edge computing layer, and a local user end, wherein the cloud provides centralized computing and storage resources; Step S02. Establish a DAG according to the dependency relationship between the task nodes using the obtained task nodes as nodes; Step S03. Model the task scheduling problem as a Markov decision process and construct an MDP model; In the constructed MDP model, the state space is defined as the combination of DAG information and task offloading plans, the DAG information includes DAG topology information and task embedding information, the action space is defined as offloading tasks to computing power nodes in any layer of the three-layer computing architecture, the reward function is defined as maximizing the service quality of the task scheduling scheme, the service quality of the task scheduling scheme is calculated according to the time delay and energy consumption generated by executing each task in the scheduling scheme, the time delay of each task is obtained according to the maximum value of the local user end completion time, the edge computing layer completion time, and the cloud completion time, the energy consumption of each task is obtained according to the energy consumption generated by executing the task on the local user end, the energy consumption generated by offloading the task to the edge computing layer, and the energy consumption generated by offloading the task to the cloud; The time delay of the task scheduling scheme is calculated according to the following formula: ， ， ， ， in, This represents the total latency of the task scheduling scheme. Indicates the first i One task, This represents the set of exit tasks in the DAG graph. This indicates that the task is executed on the local user's device. Completion time, This indicates that the task is executed at the edge computing layer. Completion time, Indicates performing tasks in the cloud. Time to complete the processing phase; Indicates task Available time on the local user's device. Indicates task The set of precursor missions, Indicates task The direct precursor mission, This indicates that the task is executed on the local user's device. Completion time, This indicates that the edge computing layer has received the task. The completion time of the result, This indicates that the task has been received in the cloud. The completion time of the result, This indicates executing a local task on the local user's device. The time delay; Indicates task Available time in the edge computing layer Indicates task The completion time of uplink transmission at the edge. This indicates that the task is executed at the edge computing layer. Completion time, This indicates that the task is executed at the edge computing layer. Execution latency; Indicates task Available time in the cloud Indicates that the task is executed in the cloud. The upload phase completion time, Indicates performing tasks in the cloud. The completion time of the processing phase, Indicates performing tasks in the cloud. Execution latency; Task Completion time of the edge uplink channel transmission The calculation expression is: ， wherein, representing a task available time for upload phase when executed in the cloud, representing a task delay in transmission in the cloud; Executing tasks in the cloud In the upload phase completion time The calculation expression is: ， ， wherein, denotes a task the completion time of the download phase in the cloud, denotes a task the available time of the downlink wireless channel, denotes the execution of a task the completion time of the processing phase, denotes a task the reception phase delay when offloaded to the cloud, denotes a task the transmission delay in the cloud; Step S04. Based on the constructed MDP model, a deep reinforcement learning model is established using a deep reinforcement learning method, the current to-be-executed task embedding sequence is input into the pre-trained deep reinforcement learning model, and a corresponding task offloading decision scheme is generated, the local user end, the edge computing layer, and the cloud cooperatively complete the execution of all tasks according to the generated task offloading decision scheme, in the process of generating the task offloading decision scheme, the priority value of the task is calculated according to the following formula to sort the tasks of the DAG: wherein, represents a priority value of a task , represents a direct successor task set of a task , represents a direct successor task of a task , represents a set of exit tasks, represents a transmission phase latency of a task when offloaded to the cloud, represents an execution phase latency of a task when offloaded to the cloud, represents a transmission phase latency of a task when offloaded to the edge computing layer, represents an execution phase latency of a task when offloaded to the edge computing layer, represents a reception phase latency of a task when offloaded to the edge computing layer, represents a total latency of a task when offloaded to the edge computing layer, represents a total latency of a task when offloaded to the cloud.

2. The cloud edge-end collaboration based computing power network layered scheduling optimization method of claim 1, wherein, The defined reward function is: ， ， ， in, , These represent the average latency and energy consumption of tasks in the DAG, respectively. Indicates task The amount of latency contributed by the individual. Indicates task Energy consumption contributed independently and These represent the first task to the next task. Cumulative energy consumption, from the first task to the next task Cumulative energy consumption Indicates the transition from the first task to the next task. The cumulative delay, Indicates the transition from the first task to the next task. The cumulative delay, and These represent the total latency and total energy consumption metrics under the local execution baseline, respectively. It is the weighting coefficient for delay. It is the weighting coefficient for energy consumption. , They represent the first i The first task, the first i- 1 task This indicates that the task is unloaded and executed at a different level. This indicates that the task is executed locally. This indicates that the task is executed at the edge. This indicates that the task is executed in the cloud. Indicates selecting a task action The state at that time.

3. The cloud edge-end collaboration based computing power network layered scheduling optimization method of claim 1, wherein, If offloaded to the cloud, the latency of a task includes the cloud sending phase latency , the cloud execution phase latency , and the cloud receiving phase latency , where , denote the data size of the task uploaded to the cloud and the upload propagation rate of the cloud, respectively, denotes the number of CPU cycles required to execute the task , denotes the CPU clock speed of the cloud virtual machine, , denote the data size of the task received by the cloud and the download propagation rate of the cloud, respectively. If offloaded to the edge computing layer, the time delay of the task includes: edge computing layer sending phase time delay , edge computing layer execution phase time delay and edge computing layer receiving phase time delay , wherein, denotes the upload propagation rate of the edge computing layer, denotes the CPU clock speed of the edge computing layer, , denote the data size of the task received by the edge computing layer and the downlink propagation rate of the edge computing layer, respectively; If executed at the local user end, the local latency is , denotes the CPU clock speed of the local user end.

4. The cloud edge-end collaboration based computing power network layered scheduling optimization method of claim 1, wherein, The energy consumption of the task scheduling scheme is calculated according to the following formula: ， ， ， in, This represents the total energy consumption of the task scheduling scheme. Indicates the first i One task, Represents a set of tasks. This indicates that the task is executed on the local user's device. The energy consumption generated , It is the power coefficient. It is a constant. Indicates the CPU frequency of the local device Power of 1 This indicates the latency of executing local tasks on the local user's device. Indicates task Energy consumption generated by offloading to the edge computing layer and These are the transmit power and receive power, respectively. Indicates task Energy consumption generated from offloading to the cloud; This indicates that the task is unloaded and executed at a different level. This indicates that the task is executed locally. This indicates that the task is executed at the edge. This indicates that the task is executed in the cloud. Indicates task The transmission phase latency during offloading to the edge computing layer Indicates task Reception stage latency when offloading to the edge computing layer Indicates task Delay during the sending phase when unloading to the cloud. Indicates task Latency during the receiving phase when unloading to the cloud.

5. The cloud edge-end collaborative computing power network hierarchical scheduling optimization method according to any one of claims 1-4, characterized in that, policy in the deep reinforcement learning model representing a state under a task selecting an action with a probability, representing task offloading to different layers for execution, if representing task execution locally, representing task execution at an edge end, representing task execution in the cloud, representing a graph structure obtained after encoding a DAG, representing a task offloading plan for a task sequence from a task to a task , the deep reinforcement learning model adopts a S2S neural network as a policy network to obtain an offloading decision for each task.

6. The cloud edge-end collaboration based computing power network layered scheduling optimization method of claim 5, wherein, The policy network and the value network in the deep reinforcement learning model share parameters of an encoder network and a decoder network to extract general features in the DAG, and a fully connected layer is further arranged on the output of the decoder in the policy network, and a softmax function is used to convert the output into a distribution of actions, and another fully connected layer is further arranged on the output of the decoder in the value network to output a state value .

7. The cloud edge-end collaboration based computing power network layered scheduling optimization method of claim 5, wherein, In the task offloading decision obtained by using the S2S neural network as the policy network, after the priority value of the task is calculated according to the following formula to sort the tasks of the DAG, each task is encoded into a series of embedding vectors in the order of the calculated priority value, and is input into the S2S neural network, and the S2S neural network generates a task offloading decision output.

8. A computing power network layered scheduling optimization system based on cloud edge-end cooperation, comprising a processor and a memory, the memory being used to store a computer program, characterized in that, The processor is configured to execute the computer program to perform the method of any one of claims 1-7.

9. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-7.

Images