Multi-granularity and multi-level computing power scheduling method in end-side computing power network

Abstract

The application discloses a multi-granularity and multi-level computing power scheduling method in an end-side computing power network, which comprises the following steps: deploying an online deep reinforcement learning agent and a system state monitoring module on a base station side; detecting and collecting channel states and computing resources and sending them to the deep reinforcement learning agent; outputting a task segmentation strategy of each user and a scheduling decision of each subtask; a task-type device (TD) segments tasks according to the instruction and schedules the tasks to target computing nodes according to the instruction for parallel processing; the task-type device (TD) receives the computing results of all the subtasks scheduled out in parallel; and the scheduling method segments the tasks of each user in multiple granularities according to current system state information, including channel gain, idle resources provided by a resource-type device (RD) and resources allocated to each user by an edge computing server, and schedules the segmented subtasks to suitable computing nodes in parallel, so that the execution time delay of an application program is further shortened by taking advantage of parallel execution.

Description

Technical Field

[0001] This invention belongs to the field of communication and computing technology, and specifically relates to a multi-granularity, multi-level computing power scheduling method in an edge computing power network. Background Technology

[0002] With the development of the Internet of Things (IoT) and artificial intelligence (AI), the intelligence level of terminal devices is constantly increasing, and their quantity and variety are enormous. However, due to strict device size limitations and production cost considerations, terminal devices usually have batteries with limited capacity and low-performance processors. Therefore, the limited device lifespan and low computing power cannot support the increasing number of new applications that require sustainable and high-performance computing, such as autonomous driving and augmented reality applications. Therefore, how to solve these two basic performance limitations is a key issue in the research and development of modern IoT technology.

[0003] Mobile edge computing (MEC) has become a key technology to help mobile devices perform tasks with limited battery capacity and computing resources. Computationally intensive and latency-sensitive tasks generated by mobile devices can be offloaded to nearby MEC servers (ES), which is an effective way to reduce latency costs. MEC has great potential advantages, but it also faces many challenges. First, the computing power of a single ES is usually limited compared to cloud servers. Second, the computing power and task requirements of ES are unevenly distributed in time and space, which leads to low utilization efficiency of computing resources and unbalanced workload. Third, task requirements and the edge computing network environment change dynamically, which requires real-time decision-making for task scheduling.

[0004] Furthermore, many computationally intensive tasks are divisible, meaning a single task can be divided into multiple subtasks, which can be processed collaboratively by multiple Elastic Compute Service (ES) nodes. However, current research on task partitioning models tasks as being able to be divided into two or more parallel subtasks in any proportion, which does not actually reflect the structure of real-world applications. In edge computing networks, the partitioned parallel subtasks can be distributed across multiple terminal computing nodes, overcoming the bottleneck of limited resource capabilities of a single device, enabling flexible service deployment and efficient resource utilization, while also achieving parallel processing of tasks and significantly reducing task processing latency. However, due to the significant differences in idle computing resources among different terminal devices and the time-varying nature of channel quality, it is necessary to perform multi-granularity partitioning of the task based on the multi-level computing resources distributed around the user and the real-time channel quality, and then schedule these multi-granularity subtasks to computing nodes with corresponding levels of computing power to minimize the total processing latency of the task.

[0005] In situations where user terminal resources are limited but large-scale computationally intensive tasks need to be processed, and when edge computing requires the transmission of large amounts of data, and the resources provided by the edge server are also limited, existing technologies cannot solve the technical problems.

[0006] In view of the above-mentioned technical problems in the existing technology, the present invention provides a multi-granularity, multi-level computing power scheduling method in edge computing power network. Summary of the Invention

[0007] This invention proposes a multi-granularity, multi-level computing power scheduling method in edge computing power networks.

[0008] The present invention adopts the following technical solution:

[0009] A multi-granularity, multi-level computing power scheduling method in an edge computing power network includes:

[0010] Step 1: Deploy an online deep reinforcement learning agent and a system status monitoring module on the base station side. The system status monitoring module is responsible for collecting system status information, and the online deep reinforcement learning agent is used to make decisions and continuously update the strategy.

[0011] Step 2: At the beginning of each time frame, the system status monitoring module detects and collects the channel status between each task-type device (TD), each resource-type device (RD), and the base station, as well as the idle resources of each resource-type device (RD) and the computing resources allocated by the mobile edge computing server (ES) to each task-type device (TD) in the current time frame, and sends them to the deep reinforcement learning agent.

[0012] Step 3: The deep reinforcement learning agent outputs the task partitioning strategy and the scheduling decision of each subtask for each user based on the system status information provided by the system status detection module, and sends the task partitioning and subtask scheduling decisions to the corresponding task-type device TD.

[0013] Step 4: Upon receiving the task partitioning and subtask scheduling scheme, the task-type device (TD) will partition its own tasks according to the instructions and schedule them in parallel to the target computing node for parallel processing.

[0014] Step 5: The task-oriented device TD receives the calculation results of all the subtasks that have been scheduled out in parallel and completes the task.

[0015] Step 6: Every certain number of time frames, the deep reinforcement learning agent updates the parameters of its deep neural network based on historical experience.

[0016] Furthermore, step 3 includes online deep reinforcement learning steps for joint task segmentation and parallel subtask scheduling.

[0017] Furthermore, step 3 includes:

[0018] Step 3.1: Input the system state information into the deep neural network of the online deep reinforcement learning agent;

[0019] Step 3.2: The activation function of the output layer of the deep neural network is sigmoid. The output of the DNN is a relaxed task segmentation action, in which the value of each element is relaxed to between 0 and 1.

[0020] Step 3.3: Quantize the relaxed task segmentation action into Q task segmentation actions using the sliding threshold quantization algorithm, where Q is a user-defined hyperparameter.

[0021] Step 3.4: Filter the generated Q task segmentation actions, remove duplicate and infeasible task segmentation actions, leaving Q' feasible task segmentation actions.

[0022] Step 3.5: For each of the remaining Q' feasible task partitioning schemes, use the parallel task scheduling algorithm (KM algorithm) to obtain the target processing position and the average processing latency of each subtask set.

[0023] Step 3.6: Select the task partitioning scheme with the smallest average processing latency from the remaining Q' feasible task partitioning schemes and the corresponding task scheduling strategy. Then, distribute the task partitioning scheme and the corresponding task scheduling strategy to each task-type device (TD). After normalizing the selected task partitioning action, store it together with the current system state label and the system state in the experience playback cache. If the cache is full, replace the oldest data.

[0024] Furthermore, step 3.3 includes:

[0025] Step 3.31: Determine the threshold based on the number of results that each user's task needs to output. When the task needs to output N results, the thresholds are 1 / N, 2 / N, ... (N-1) / N.

[0026] Step 3.32: Determine the sliding step size of the segmentation threshold for each user task based on the number of actions to be generated;

[0027] Step 3.33: Classify the task results based on a threshold. The result classified into a category indicates that it is the output result of a subtask, and the position of the result is quantized as the minimum value of the sequence number in the result, which represents the sequence number of the subtask. The number of categories corresponds to the number of subtasks.

[0028] Step 3.34: Generate the remaining Q-1 task segmentation actions by sliding the threshold Q-1 times.

[0029] Furthermore, in step 3.32, when Q actions need to be generated, the threshold sliding step size for the task of calculating N results is 1 / NQ.

[0030] Furthermore, in step 1, the parameters are updated to update the strategy by randomly sampling a batch of sample data from the experience replay cache at regular intervals.

[0031] Compared with the prior art, the superior effects of the present invention are as follows:

[0032] The multi-granularity, multi-level computing power scheduling method in the edge computing power network described in this invention divides each user's task into multiple granularities based on the current system state information, including channel gain, idle resources provided by RD, and resources allocated to each user by ES. The divided subtasks are then scheduled to appropriate computing nodes in parallel, thereby leveraging the advantages of parallel execution to further shorten the execution latency of the application. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of parallelizable tasks in an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of a system including N task-type terminals (TDs) in an embodiment of the present invention;

[0035] Figure 3 This is a flowchart illustrating the multi-granularity, multi-level computing power scheduling method in the edge computing power network in an embodiment of the present invention.

[0036] Figure 4 This is a schematic diagram of the online deep reinforcement learning process for joint task segmentation and parallel subtask scheduling in an embodiment of the present invention.

[0037] Figure 5 This is a schematic diagram of the sliding threshold quantization process in an embodiment of the present invention;

[0038] Figure 6 This is a schematic diagram of a system including two task-type terminals (TDs) performing task segmentation and action quantization in an embodiment of the present invention.

[0039] Figure 7 This is a schematic diagram illustrating the intelligent agent agent's output task segmentation action, parallel scheduling strategy, and intelligent agent strategy update process in an embodiment of the present invention. Detailed Implementation

[0040] To better understand the above-mentioned objectives, features and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0041] The system running the multi-granularity, multi-level computing power scheduling method in the edge computing network includes N terminal devices, called task devices (TDs), that execute parallelizable, computationally intensive tasks. The parallelizable tasks include... Figure 1 As shown, M terminal devices with abundant computing resources that perform latency-insensitive tasks are called resource-based devices (RDs), one base station, one edge computing server connected to the base station via optical fiber, and one cloud computing center. In each time frame, in addition to processing its own tasks, the RDs also have idle computing resources provided to the TDs to assist in processing their tasks. Since resource allocation and computational offloading have different time sensitivities, resource allocation is often completed before computational offloading, meaning resource allocation and computational offloading have different time scales. This invention only considers the computational offloading part, assuming that the resource allocation status can be known before computational offloading, and then a computational offloading decision is made. Therefore, in each time frame... In each time frame, the edge server allocates certain resources to each TD (Task Controller). This resource allocation is determined before task partitioning and parallel scheduling. Since the cloud computing center has abundant resources, the latency of offloading tasks to the cloud is considered to be mainly determined by transmission latency. At the beginning of each time frame, each TD needs to decide how many subtasks it will divide its task into, and the granularity of each subtask (i.e., the required calculation result). Then, it needs to decide which computing node each subtask should be scheduled to for processing. It can execute locally, be scheduled to an RD (Run-Rank Controller) via a D2D link, be scheduled to an edge server, or be scheduled to the cloud center. A schematic diagram of the system is shown below. Figure 2 As shown.

[0042] The multi-granularity, multi-level computing power scheduling method in the edge computing power network, where multi-granularity means dividing the user's application into sub-tasks with different granularities that can be executed in parallel, and multi-level corresponds to the differentiated computing power resources provided by resource-based terminal devices in each time slot, the computing power resources allocated by edge servers, and the computing power resources provided by the cloud center, such as... Figure 2As shown, a device-assisted edge computing network system is described. The method proposed in this embodiment runs on an intelligent learning agent deployed at a base station. The agent is responsible for generating decisions and continuously learning and updating its own strategies. The generated decisions are distributed to task-oriented terminal devices to guide them in taking corresponding actions. The device-assisted edge computing network system includes N task-oriented terminals (TDs), which need to perform computationally intensive tasks but have limited computing power; M resource-oriented terminals, which perform latency-insensitive tasks and have relatively sufficient computing resources, thus having a certain amount of idle computing resources in each time frame. These computing resources can help the TDs perform tasks; an edge cloud server; and a cloud center. Since resource allocation and computational offloading have different time sensitivities, making these decisions simultaneously will cause the offloading process to wait for resource allocation, thereby reducing real-time requirements. Alternatively, this could lead to frequent resource coordination, resulting in high operating costs and system instability. Therefore, resources are often already configured before offloading decisions are made, and these decisions have different time scales. Therefore, this technical solution performs multi-granularity segmentation and parallel scheduling of tasks for different users based on the system channel state and resource allocation.

[0043] Example

[0044] Specifically, such as Figure 3 As shown, the multi-granularity, multi-level computing power scheduling method in the edge computing power network includes:

[0045] Step 1: Network pre-configuration. Deploy a deep reinforcement learning agent and a system status monitoring module on the base station side. The system status monitoring module is responsible for collecting system status information, while the deep reinforcement learning agent is responsible for making task partitioning strategies and parallel scheduling strategies.

[0046] Step 2, Status Update: At the beginning of each time frame, the system status monitoring module detects and collects the channel status between each TD and each RD, as well as the base station (BS), the idle resources of each RD, and the computing resources allocated by the edge server to each TD in the current time frame, and sends them to the deep reinforcement learning agent.

[0047] Step 3: Decision generation. The deep reinforcement learning agent outputs the task splitting strategy for each user and the scheduling decision for each subtask based on the current system state information, and sends these decisions to the corresponding TD.

[0048] like Figure 4 As shown, the specific steps of deep reinforcement learning in generating decisions include:

[0049] Step 3.1: Input the channel quality information and resource supply information into the deep neural network (DNN) of the deep reinforcement learning agent, such as... Figure 7 As shown;

[0050] Step 3.2: The output layer of the DNN outputs a relaxed task segmentation action after passing through the sigmoid activation function, where each element is scaled to between 0 and 1;

[0051] Step 3.3: Feed the relaxed task segmentation action into the sliding threshold quantization algorithm to generate Q candidate task segmentation actions, where Q is a user-defined hyperparameter. At this time, each element of each action is quantized into an integer, and the integer value represents the subtask number to which the result belongs.

[0052] Among them, such as Figure 5 As shown, the specific steps of the sliding threshold quantization algorithm are as follows:

[0053] Step 3.31: Determine the initial threshold based on the number of results that each user's task needs to output. When the number of results that a user's task needs to output is N, the thresholds for quantizing the relaxed task segmentation action of that user are: 1 / N, 2 / N, ..., (N-1) / N.

[0054] Step 3.32: Determine the sliding step size of the quantization threshold for each user based on the number of actions Q to be generated. According to the definition, the sliding step size can be determined as 1 / (NQ).

[0055] Step 3.33: Based on the current threshold, classify the results to be calculated for each task. Results classified into the same category are obtained by the same subtask. The number of result categories for each task represents the number of subtasks after the task is divided.

[0056] Step 3.34: Through Q-1 threshold sliding steps, a candidate task segmentation action is generated after each sliding step, thereby generating the remaining Q-1 candidate task segmentation actions;

[0057] in, Figure 6This demonstrates that when there are two Task Quantization (TD) operations in the system, TD1 needs to calculate 4 results, and TD2 needs to calculate 3 results. The DNN output is [0.2, 0.4, 0.7, 0.9, 0.3, 0.7, 0.9]. The first 4 components correspond to the task segmentation scheme of TD1 (relaxed), and the last 3 components correspond to the task segmentation scheme of TD2. The quantization process of the sliding threshold quantization algorithm and the 4 generated task segmentation schemes are shown when Q is set to 4. Two of the quantized actions are repeated. Taking the second quantization action as an example, the task of TD1 is divided into two subtasks with indices 1 and 3. The indices are determined by the minimum value of the indices of the output results. The subtask with indices 1 needs to calculate the results with indices 1 and 2, and the subtask with indices 3 needs to calculate the results with indices 3 and 4. The task of TD2 is also divided into two subtasks with indices 1 and 2. The subtask with indices 1 needs to calculate the result with indices 1, and the subtask with indices 2 needs to calculate the results with indices 2 and 3.

[0058] Step 3.4: Filter the generated Q task splitting actions, remove duplicate and infeasible task splitting actions, leaving Q' feasible task splitting actions.

[0059] Step 3.5: For the set of subtasks generated by each action of the remaining Q' task splitting actions, use the task allocation algorithm (KM algorithm) to obtain the target computing node for each subtask and the average processing latency of all tasks;

[0060] Step 3.6: Compare the average processing latency that each task partitioning scheme can achieve, select the task partitioning scheme with the smallest average latency and the corresponding scheduling strategy, and distribute these decisions to each TD;

[0061] Step 4: Execute the decision. Upon receiving the task partitioning and subtask scheduling scheme, the TD will partition its own tasks according to the instructions and schedule them in parallel to the target computing node for parallel processing.

[0062] Step 5: Complete the task. The task is completed once TD receives the calculation results of all subtasks.

[0063] Step 6: Policy Update. Every certain number of time frames, the deep reinforcement learning agent will randomly select a batch of data samples from the experience replay cache to train its own deep neural network and update the parameters.

[0064] This invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the invention as claimed. The scope of protection of this invention is defined by the appended claims.

Claims

1. A multi-granularity, multi-level computing power scheduling method in an edge computing power network, characterized in that, include: Step 1: Deploy an online deep reinforcement learning agent and a system status monitoring module on the base station side. The system status monitoring module is responsible for collecting system status information, and the online deep reinforcement learning agent is used to make decisions and continuously update the strategy. Step 2: At the beginning of each time frame, the system status monitoring module detects and collects the channel status between each task-type device (TD), each resource-type device (RD), and the base station, as well as the idle resources of each resource-type device (RD) and the computing resources allocated by the mobile edge computing server (ES) to each task-type device (TD) in the current time frame, and sends them to the deep reinforcement learning agent. Step 3: The deep reinforcement learning agent outputs the task partitioning strategy and the scheduling decision of each subtask for each user based on the system status information provided by the system status detection module, and sends the task partitioning and subtask scheduling decisions to the corresponding task-type device TD. Step 4: Upon receiving the task partitioning and subtask scheduling scheme, the task-type device (TD) will partition its own tasks according to the instructions and schedule them in parallel to the target computing node for parallel processing. Step 5: The task-oriented device TD receives the calculation results of all the subtasks that have been scheduled out in parallel and completes the task. Step 6: Every certain number of time frames, the deep reinforcement learning agent updates the parameters of its deep neural network based on historical experience.

2. The multi-granularity, multi-level computing power scheduling method in the edge computing power network according to claim 1, characterized in that, Step 3 includes: Step 3.1: Input the system state information into the deep neural network of the online deep reinforcement learning agent; Step 3.2: The activation function of the output layer of the deep neural network is sigmoid. The output of the DNN is a relaxed task segmentation action, in which the value of each element is relaxed to between 0 and 1. Step 3.3: Quantize the relaxed task segmentation action into Q task segmentation actions using the sliding threshold quantization algorithm, where Q is a user-defined hyperparameter. Step 3.4: Filter the generated Q task segmentation actions, remove duplicate and infeasible task segmentation actions, leaving Q' feasible task segmentation actions. Step 3.5: For each of the remaining Q' feasible task partitioning schemes, a parallel task scheduling algorithm is used to obtain the target processing location and the average processing latency of each subtask set. Step 3.6: Select the task partitioning scheme with the smallest average processing latency from the remaining Q' feasible task partitioning schemes and the corresponding task scheduling strategy. Then, distribute the task partitioning scheme and the corresponding task scheduling strategy to each task-type device (TD). After normalizing the selected task partitioning action, store it together with the current system state label in the experience playback cache. If the cache is full, replace the oldest data.

3. The multi-granularity, multi-level computing power scheduling method in the edge computing power network according to claim 2, characterized in that, Step 3.3 includes: Step 3.31: Determine the threshold based on the number of results that each user's task needs to output. When the task needs to output N results, the thresholds are 1 / N, 2 / N, ... (N-1) / N. Step 3.32: Determine the sliding step size of the segmentation threshold for each user task based on the number of actions to be generated; Step 3.33: Classify the task results based on the threshold. The result classified into a category indicates that it is the output result of a subtask. The position of the result is quantized as the minimum value of the sequence number in the result, which represents the sequence number of the subtask. The number of categories corresponds to the number of subtasks. Step 3.34: Generate the remaining Q-1 task segmentation actions by sliding the threshold Q-1 times.

4. The multi-granularity, multi-level computing power scheduling method in the edge computing power network according to claim 3, characterized in that, In step 3.32, when Q actions need to be generated, the threshold sliding step size for the task of calculating N results is 1 / NQ.

5. The multi-granularity, multi-level computing power scheduling method in an edge computing power network according to claim 1 or 3, characterized in that, In step 1, the parameters are updated to update the strategy by randomly sampling a batch of sample data from the experience replay cache at regular intervals.

Images