Multi-layer computing network task unloading method

Abstract

The invention provides a multi-layer computing network task unloading method, which comprises the following steps that a multi-layer computing network model is constructed, and an edge node obtains real-time network state information and current computing resource information of all nodes based on task characteristics and task demand indexes submitted by a user in an unloading request; the edge node calculates time delay and energy consumption generated when tasks are unloaded to different nodes according to the real-time network information; and establishing a task unloading objective function according to the business demand indexes, and obtaining an unloading decision by using a deep reinforcement learning method. According to the method, computing resources of different layers can be efficiently integrated and utilized, personalized customization services are provided for users, and the satisfaction rate of the users is increased.

Description

Technical Field

[0001] The present application relates to the field of cloud-edge collaborative computing, and specifically to a multi-layer computing network task offloading method. Background Art

[0002] Mobile edge computing and fog computing deploy computing power closer to users to help limited mobile devices offload tasks, thereby reducing task delays and costs and improving user experience.

[0003] Task offloading is an important feature of cloud-edge collaborative computing. The research direction of existing technologies is focused on edge computing or edge-cloud combined networks, and is committed to studying how to utilize edge nodes and cloud nodes rather than how to utilize the extensive computing power of the two.

[0004] In actual situations, the network is usually a multi-layer computing network containing a large number of heterogeneous resources, and the nodes in the network are described by computing resources and topological locations. Different multi-layer computing network nodes can compensate for the shortcomings of cloud-edge collaborative computing through a collaborative service architecture. However, the resources in a multi-layer computing network are widely distributed and heterogeneous, and the network conditions continue to change dynamically, which makes the task offloading work complicated. In addition, existing task offloading research mostly hopes to reduce the latency and energy consumption during task offloading, but ignores the characteristics of the task itself, and does not consider the personalized needs of users in the task. That is, how to use the design resources of different layers to reduce the overall system overhead, how to provide customized services from the user dimension, and improve user satisfaction, the existing technology has failed to provide corresponding technical solutions. Summary of the invention

[0005] The present application provides a multi-layer computing network task offloading method, which can efficiently integrate and utilize computing resources at different layers, provide users with personalized customized services, and improve user satisfaction.

[0006] The technical solution of this application is as follows: A multi-layer computing network task offloading method comprises the following steps: S1) constructing a multi-layer computing network model based on heterogeneous computing resources in the network, wherein the node types of the multi-layer computing network model include: cloud nodes, fog nodes and edge nodes; S2) The edge node obtains real-time network status information and current computing resource information of all nodes based on the task characteristics and task demand indicators submitted by the user in the offloading request; S3) The latency and energy consumption generated when edge nodes offload computing tasks to different nodes based on real-time network information; S4) establishing a task offloading objective function according to the business demand indicator, and obtaining an offloading decision using a deep reinforcement learning method, wherein the offloading decision is used to indicate an offloading node of the task.

[0007] Further, in step S1), the six-tuple represents a computing node; wherein m represents a computing node index; Indicates computing power in floating-point operations per second; Indicates the computing power of a single chip; Indicates the number of chips that can be called by the computing node; Indicates the number of layers of computing nodes in the network; Indicates the maximum storage space of the computing node; Indicates the chip type, including CPU chip and GPU chip.

[0008] Further, in step S2), represents a task, where n represents the index of the task; Indicates the task size in bits; Indicates the computing requirements of the task, in floating-point operations; Indicates parallel ratio; , , together as a task characteristic; Indicates the delay limit of the service; Indicates the energy consumption limit of the service; Indicates the storage resource requirements of the business; Indicates the computing chip type requirements of the business; , , , , together as business demand indicators.

[0009] Furthermore, in step S3), the total energy consumption of the task when unloading the computing node is calculated as follows: ; In the formula, represents the total energy consumption; represents the transmission energy consumption of the task; Indicates the computational energy consumption of the task; The total delay of the task when it is unloaded on the computing node is calculated as follows: ; In the formula, Indicates the total delay; Indicates the transmission delay of the task; Indicates the waiting delay incurred by a task while waiting in the queue; Indicates the computational latency of the task.

[0010] Furthermore, the transmission delay of the task is calculated as follows: ; In the formula, Indicates the number of layers of computing nodes in the network; Indicates the network transmission rate of the i-th layer; The transmission energy consumption of the task is calculated as follows: ; In the formula, Represents the average transmission power between each layer of computing network; The computational delay of the task is calculated as follows: ; The computing energy consumption of the task is calculated as follows: ; The waiting delay of the task is calculated as follows: ; In the formula, Indicates that the computing node is at time The queue delay, time The moment when the offload request is made to the network for the task.

[0011] Furthermore, the queue delay of a computing node after executing all tasks in its own queue is calculated as follows: ; In the formula, Indicates the task The moment the uninstall request is made, Indicates the task Uninstallation decision; Compute nodes in The remaining storage resources at the moment are calculated as follows: ; In the formula, Indicates that the computing node is The remaining storage resources at any time; represents the maximum storage space of the computing node, k represents the index of all tasks before task n, represents the total delay of task k, represents the storage resource requirement of task k, represents the offloading decision of task k; F represents the indicator function, Indicates * is a fact, otherwise 0.

[0012] Further, in step S4), the objective function is the user satisfaction rate under the current uninstallation decision, and the user satisfaction rate is calculated as follows: ; In the formula, Represents the user satisfaction rate. When the constraint is met, its value is 1, otherwise it is 0; The constraints of user satisfaction rate and business demand indicators are as follows: ; ; ; .

[0013] Furthermore, in step S4), the deep reinforcement learning method takes maximum satisfaction as the optimization goal and takes the unloading decision as the optimization variable.

[0014] Furthermore, the optimal offloading decision is solved based on the D3QN algorithm. The specific steps are as follows: SA1) Define state space, action space and reward function; The state space includes task characteristics, business demand indicators, node computing performance and computing resources; the node computing performance includes total energy consumption and total latency; the computing resources include remaining storage resources and computing chip types; The action space represents offloading decisions; The reward function is: ; In the formula, represents the reward function, represents the user satisfaction penalty, which is calculated as follows: ; Indicates the delay penalty caused by offloading, Indicates the delay penalty coefficient. The delay penalty is calculated according to the following formula: ; represents the energy penalty caused by unloading, Represents the energy consumption penalty coefficient. The energy consumption penalty is calculated according to the following formula: ; represents the storage penalty caused by uninstallation, Represents the storage penalty coefficient. The storage penalty is calculated according to the following formula: ; SA2) Initialize the network parameters of the evaluation network and target network in the D3QN algorithm; SA3) Get the state space of the task , input into the evaluation network, and get the action ; SA4) Perform actions and observe rewards and the next state , the sample Put it into the experience pool; represents the reward at time t; SA5) In the evaluation network, a batch of samples is randomly selected from the experience pool for training, and the parameters of the evaluation network are iteratively updated by minimizing the loss function; SA6) Repeat steps SA3) to SA6) at fixed update step lengths, copying the network parameters of the evaluation network to the target network until the training is completed.

[0015] Due to the adoption of the above technical solution, the beneficial effects of this application are as follows: 1. This application integrates heterogeneous resources in the network, fully utilizes computing resources at different layers, reduces the average waiting delay of tasks in nodes, and reduces the overall overhead of the system.

[0016] 2. Starting from the user satisfaction rate, this application fully considers the task characteristics and the personalized needs of users, and can be applied to a variety of application scenarios. The technical solution of this application realizes task offloading in a multi-layer computing network based on deep reinforcement learning, which helps to achieve on-demand matching of multi-layer heterogeneous resources with user tasks, and effectively improves user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0018] Figure 1 A flowchart of a multi-layer computing network task offloading method provided by this application; Figure 2 This is a model architecture diagram of a multi-layer computing network system in this application; Figure 3 Flowchart of the D3QN algorithm in this application; Figure 4 This is the neural network architecture diagram in the D3QN algorithm. DETAILED DESCRIPTION

[0019] Based on the background technology, cloud computing provides powerful computing capabilities, but its deployment location is generally far away from users, resulting in high latency and high costs. Mobile edge computing and fog computing deploy computing capabilities near users, which can help mobile devices with limited resources to offload tasks, thereby reducing task latency and costs and improving user experience. Figure 1 and attached Figure 2The present application provides a multi-layer computing network task offloading method, comprising the following steps: S1) constructing a multi-layer computing network model based on heterogeneous computing resources in the network, wherein the node types of the multi-layer computing network model include: cloud nodes, fog nodes and edge nodes.

[0020] Cloud nodes have sufficient computing resources and are server clusters deployed at locations far from users. Edge computing nodes have basic computing power and are located in the first layer closest to users, providing fast computing services; while fog nodes have computing power and location between edge nodes and cloud nodes. Users can submit offload requests through wireless channels, and tasks can be transmitted in multiple hops between different layers.

[0021] In step S1), assume that there are M nodes in total, in the form of six-tuples represents a computing node; wherein m represents a computing node index; Indicates computing power in floating-point operations per second; Indicates the computing power of a single chip; Indicates the number of chips that can be called by the computing node; Indicates the number of layers of computing nodes in the network; Indicates the maximum storage space of the computing node; Indicates the type of chip, including CPU chip and GPU chip. , where 0 represents the CPU chip, which is used to execute CPU type tasks; 1 represents the GPU chip, which is used to execute GPU type tasks.

[0022] S2) The edge node obtains real-time network status information and current computing resource information of all nodes based on the task characteristics and task demand indicators submitted by the user in the offloading request.

[0023] In step S2), N tasks send unloading requests to the edge layer in order, and the arrival time follows the Poisson distribution. Each task selects a computing node for task unloading. The task unloading at a certain node is x, and Indicates: If , indicating that task n selects node m for unloading; Indicates that task n did not select node m for offloading, and the offloading decision satisfies .

[0024] In this embodiment, the task is a divisible heterogeneous task, which can be divided into serial segment subtasks and parallel segment subtasks. According to the task type, the parallel segment can use multiple CPU / GPU computing to achieve parallel acceleration. represents a task, where n represents the index of the task; Indicates the task size in bits; Indicates the computing requirements of the task, in floating-point operations; Indicates parallel ratio; , , together as a task characteristic; Indicates the delay limit of the service; Indicates the energy consumption limit of the service; Indicates the storage resource requirements of the business; Indicates the computing chip type requirements of the business; , , , , together as business demand indicators. It should be noted that, in order to facilitate distinction, this application adds an index mark of the task under each parameter.

[0025] In this embodiment, the multi-layer computing network adopts a multi-hop transmission communication model. When an offload request is made to the network at any time, there must be a communication path between any two layers of service nodes, and the network transmission rate of the i-th layer is At this time, the queue delay of the computing node in the network after executing all the tasks in its own queue is , the idle storage resources are , Indicates free storage resources, and the corresponding time is shown in brackets.

[0026] S3) The edge node calculates the delay and energy consumption when the task is unloaded to different nodes based on real-time network information. Tasks unloaded to different computing nodes will generate different delays and energy consumption. The specific process is: the task generates transmission delay and transmission energy consumption during the transmission process, the task generates waiting delay during the queue waiting process of the computing node, and the task generates computing delay and computing energy consumption during the computing process of the computing node.

[0027] In step S3), the total energy consumption of the task when unloading on the computing node is calculated as follows: ; In the formula, represents the total energy consumption; represents the transmission energy consumption of the task; Indicates the computational energy consumption of the task; The total delay of the task when it is unloaded on the computing node is calculated as follows: ; In the formula, Indicates the total delay; Indicates the transmission delay of the task; Indicates the waiting delay incurred by a task while waiting in the queue; Indicates the computational latency of the task.

[0028] Specifically, task transmission requires multiple layers of communication in a multi-layer computing network, and the transmission delay is Determined by the task size and the transmission rate between layers, the task transmission delay is calculated as follows: ; In the formula, Indicates the number of layers of computing nodes in the network; Indicates the network transmission rate of the i-th layer; The transmission energy consumption generated during the transmission process is determined by the transmission delay and transmission power. The transmission energy consumption of the task is calculated as follows: ; In the formula, Represents the average transmission power between each layer of computing network; The computational delay generated during the computation is determined by the computational requirements of the task, the parallel ratio, and the number of chips in the computing node. The computational delay of the task is calculated as follows: ; The computing energy consumption generated during the calculation process is determined by the computing latency, the number of chips in the computing node, and the computing power. The computing energy consumption of the task is calculated as follows: ; After the task transmission arrives at the node, a first-come-first-served strategy is adopted. Therefore, the waiting delay generated during the waiting process can be calculated by subtracting the transmission delay from the queue delay. The waiting delay of the task is calculated as follows: ; In the formula, Indicates that the computing node is at time The queue delay, time The moment when the offload request is made to the network for the task.

[0029] When a task sends an offload request, the node needs to broadcast its own queue delay, and the node's queue delay is determined by the total delay of all tasks in the queue. The queue delay of the computing node after executing all tasks in its own queue is calculated as follows: ; In the formula, Indicates the task The moment the uninstall request is made, Indicates the task Uninstallation decision; When a task is unloaded, the node needs to reserve the storage resources required for the task and remove the occupied storage space after the calculation is completed. The remaining storage resources at the moment are calculated as follows: ; In the formula, Indicates that the computing node is The remaining storage resources at any time; represents the maximum storage space of the computing node, k represents the index of all tasks before task n, represents the total delay of task k, represents the storage resource requirement of task k, represents the offloading decision of task k. F represents the indicator function, Indicates * is a fact, otherwise 0.

[0030] S4) See Appendix Figure 3 and attached Figure 4 , establish the task offloading objective function according to the business demand indicators, and use the deep reinforcement learning method to obtain the offloading decision, which is used to indicate the offloading node of the task, namely: ; The constraints are: ; In step S4), the objective function is the user satisfaction rate under the current uninstallation decision, and the user satisfaction rate is calculated as follows: ; In the formula, Represents the user satisfaction rate. When the constraint is met, its value is 1, otherwise it is 0; The constraints of user satisfaction rate and business demand indicators are as follows: ; ; ; .

[0031] In step S4), the deep reinforcement learning method takes maximum satisfaction as the optimization goal and takes the unloading decision as the optimization variable.

[0032] In this embodiment, the optimal offloading decision is solved based on the D3QN algorithm, and the specific steps are as follows: SA1) Define state space, action space and reward function; The state space includes task characteristics, business demand indicators, node computing performance and computing resources; the node computing performance includes total energy consumption and total latency; the computing resources include remaining storage resources and computing chip types; This embodiment uses Represents the task characteristics, ,use Indicates business demand indicators, ,use Indicates the node computing performance, use Represents computing resources .

[0033] The action space represents the offloading decision, and each action represents "offloading a task to a computing node". The formula for action selection is as follows: ; In the formula, Indicates the current state. represents all actions in the action space, represents the evaluation network parameters, represents the neural network function, Indicates taking action.

[0034] The reward function is used to evaluate the uninstallation decision, and the reward function is: In the formula, represents the user satisfaction penalty, which is calculated as follows: Indicates the delay penalty caused by offloading, Indicates the delay penalty coefficient. The delay penalty is calculated according to the following formula: represents the energy penalty caused by unloading, Represents the energy consumption penalty coefficient. The energy consumption penalty is calculated according to the following formula: represents the storage penalty caused by offloading, Represents the storage penalty coefficient. The storage penalty is calculated according to the following formula: SA2) Initialize the network parameters of the evaluation network and target network in the D3QN algorithm; SA3) Get the state space of the task , input into the evaluation network, and get the action ; SA4) Perform actions and observe rewards and the next state , the sample Put it into the experience pool; the next state is the state observed from the environment at the next moment.

[0035] SA5) In the evaluation network, a batch of samples is randomly selected from the experience pool for training, and the parameters of the evaluation network are iteratively updated by minimizing the loss function; the loss function is: ; In the formula, Indicates the current state. Indicates the execution of an action. Indicates reward, Indicates the next state, represents all actions in the action space, represents the evaluation network parameters, represents the target network parameters, Represents a neural network function.

[0036] In this embodiment, a batch of samples is extracted during the training process to update the network parameters, such as 64 samples as a batch.

[0037] SA6) Repeat steps SA3) to SA6) at fixed update steps, copying the network parameters of the evaluation network to the target network until the training is completed. In this embodiment, the update step is 10, which means that the parameters of the evaluation network are copied to the target network once every 10 evaluation network trainings.

[0038] After the model is trained, the evaluation network outputs the optimal action, that is, the optimal unloading decision.

[0039] Anything not described in this application can be achieved by adopting or drawing on existing technologies. The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A multi-layer computing network task offloading method, characterized in that: The following steps are involved: S1) constructing a multi-layer computing network model based on heterogeneous computing resources in the network, wherein the node types of the multi-layer computing network model include: cloud nodes, fog nodes and edge nodes; S2) The edge node obtains real-time network status information and current computing resource information of all nodes based on the task characteristics and task demand indicators submitted by the user in the offloading request; S3) The latency and energy consumption generated when edge nodes offload computing tasks to different nodes based on real-time network information; S4) establishing a task offloading objective function according to the business demand indicator, and obtaining an offloading decision using a deep reinforcement learning method, wherein the offloading decision is used to indicate an offloading node of the task.

2. A multi-layer computing network task offloading method according to claim 1, characterized in that: In step S1), the six-tuple represents a computing node; wherein m represents a computing node index; Indicates computing power in floating-point operations per second; Indicates the computing power of a single chip; Indicates the number of chips that can be called by the computing node; Indicates the number of layers of computing nodes in the network; Indicates the maximum storage space of the computing node; Indicates the chip type, including CPU chip and GPU chip.

3. A multi-layer computing network task offloading method according to claim 2, characterized in that: In step S2), represents a task, where n represents the index of the task; Indicates the task size in bits; Indicates the computing requirements of the task, in floating-point operations; Indicates parallel ratio; , , together as a task characteristic; Indicates the delay limit of the service; Indicates the energy consumption limit of the service; Indicates the storage resource requirements of the business; Indicates the computing chip type requirements of the business; , , , , together as business demand indicators.

4. A multi-layer computing network task offloading method according to claim 3, characterized in that: In step S3), the total energy consumption of the task when unloading on the computing node is calculated as follows: ； In the formula, represents the total energy consumption; represents the transmission energy consumption of the task; Indicates the computational energy consumption of the task; The total delay of the task when it is unloaded on the computing node is calculated as follows: ； In the formula, represents the total delay; Indicates the transmission delay of the task; Indicates the waiting delay incurred by a task while waiting in the queue; Indicates the computational latency of the task.

5. A multi-layer computing network task offloading method according to claim 4, characterized in that: The transmission delay of the task is calculated as follows: ； In the formula, Indicates the number of layers of computing nodes in the network; Indicates the network transmission rate of the i-th layer; The transmission energy consumption of the task is calculated as follows: ； In the formula, Represents the average transmission power between each layer of computing network; The computational delay of the task is calculated as follows: ； The computing energy consumption of the task is calculated as follows: ； The waiting delay of the task is calculated as follows: ； In the formula, Indicates that the computing node is at time The queue delay, time The moment when the offload request is made to the network for the task.

6. A multi-layer computing network task offloading method according to claim 5, characterized in that: The queue delay of a computing node after executing all tasks in its own queue is calculated as follows: ； In the formula, Indicates the task The moment the uninstall request is made, Indicates the task Uninstallation decision; Compute nodes in The remaining storage resources at the moment are calculated as follows: ； In the formula, Indicates that the computing node is The remaining storage resources at any time; represents the maximum storage space of the computing node, k represents the index of all tasks before task n, represents the total delay of task k, represents the storage resource requirement of task k, represents the offloading decision of task k; F represents the indicator function, Indicates * is a fact, otherwise 0.

7. A multi-layer computing network task offloading method according to claim 6, characterized in that: In step S4), the objective function is the user satisfaction rate under the current uninstallation decision, and the user satisfaction rate is calculated as follows: ； In the formula, Represents the user satisfaction rate. When the constraint is met, its value is 1, otherwise it is 0; The constraints of user satisfaction rate and business demand indicators are as follows: ； ； ； 。 8. A multi-layer computing network task offloading method according to claim 7, characterized in that: In step S4), the deep reinforcement learning method takes maximum satisfaction as the optimization goal and takes the unloading decision as the optimization variable.

9. A multi-layer computing network task offloading method according to claim 8, characterized in that: The optimal unloading decision is solved based on the D3QN algorithm. The specific steps are as follows: SA1) Define state space, action space and reward function; The state space includes task characteristics, business demand indicators, node computing performance and computing resources; the node computing performance includes total energy consumption and total latency; the computing resources include remaining storage resources and computing chip types; The action space represents offloading decisions; The reward function is: ； In the formula, represents the reward function, represents the user satisfaction penalty, which is calculated as follows: ； Indicates the delay penalty caused by offloading, Indicates the delay penalty coefficient. The delay penalty is calculated according to the following formula: ； represents the energy penalty caused by unloading, Represents the energy consumption penalty coefficient. The energy consumption penalty is calculated according to the following formula: ； represents the storage penalty caused by uninstallation, Represents the storage penalty coefficient. The storage penalty is calculated according to the following formula: ； SA2) Initialize the network parameters of the evaluation network and target network in the D3QN algorithm; SA3) Get the state space of the task , input into the evaluation network, and get the action ; SA4) Perform actions and observe rewards and the next state , the sample Put it into the experience pool; represents the reward at time t; SA5) In the evaluation network, a batch of samples is randomly selected from the experience pool for training, and the parameters of the evaluation network are iteratively updated by minimizing the loss function; SA6) Repeat steps SA3) to SA6) at fixed update step lengths, copying the network parameters of the evaluation network to the target network until the training is completed.

Images