A method for reinforcement learning task scheduling based on size resource tree

Abstract

The application relates to the field of edge computing networks, and discloses a method for task scheduling based on a size resource tree reinforcement learning, which comprises the following steps: first, calculating the computing resources in a MEC network; second, constructing a size resource tree model; third, using the size resource tree model to perform task decomposition on a task; and fourth, scheduling the task after decomposition to different node calculations. The method for task scheduling based on the size resource tree reinforcement learning can quickly decide to schedule the task to a node with computing capability, further improves the execution speed of the task, and reduces the execution time of the task.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of edge computing networks, in particular to a method for task scheduling based on reinforcement learning of size resource tree. BACKGROUND

[0002] In the edge computing network, the terminal device accelerates the calculation of the task by offloading the generated task to other nodes. The disadvantages of the traditional terminal device transitionally relying on the cloud computing mode have gradually emerged. The dependence on cloud computing cannot meet the demand of newly born high-bandwidth and low-delay application, and the edge computing mode is proposed to solve this bottleneck problem. The edge computing mode advocates offloading the computing task to the edge node to reduce the computing delay,

[0003] The existing edge computing is optimized for solving special problems in a certain scene, and the applicable conditions are relatively harsh, which is not suitable for general scenes, has low applicability, and leads to the need to develop suitable task scheduling strategies for different scenes.

[0004] The potential computing capacity of the MEC network is not fully utilized, and the demand for computing capacity still excessively relies on the cloud server, resulting in a very small optimization effect of the delay of executing the computing task. There are a large number of terminal nodes in the MEC network that can execute the computing task, and if the computing capacity of these nodes can be fully tapped, not only the computing capacity of the entire MEC network can be improved, but also the computing task can be offloaded nearby, and the computing delay of the task can be further reduced.

[0005] There is no real global optimization idea, and most of the work is to optimize the scheduling of local area tasks. Some related researches strive to achieve global optimization by achieving local optimization, but this is not real global optimization and cannot achieve global optimization. The information exchange between the nodes of the MEC network in the domain is not considered, so that only the potential computing capacity of the nodes in the domain can be tapped.

[0006] Therefore, we propose a method for task scheduling based on reinforcement learning of size resource tree. SUMMARY

[0007] (I) Technical problems solved

[0008] In view of the shortcomings of the prior art, the present application provides a method for task scheduling based on reinforcement learning of size resource tree, which solves the above problems.

[0009] (II) Technical solutions

[0010] In order to achieve the above purpose, the present application provides the following technical solutions: a method for task scheduling based on reinforcement learning of size resource tree, comprising the following steps:

[0011] Step 1: Calculate the computing resources in the MEC network;

[0012] Second step: build a size resource tree model;

[0013] Third step: use the size resource tree to decompose the task;

[0014] Fourth step: schedule the decomposed task to the nodes in the MEC network for calculation.

[0015] Preferably, the computing resources in the first step include inter-node network bandwidth resources, node CPU frequency resources, node memory resources, node hard disk storage resources, and node GPU resources.

[0016] Preferably, the size resource tree model is used to manage the computing resources in the MEC network. The size resource tree model adopts a domain mechanism. All resource nodes in the large resource tree management domain can be more detailedly divided into subareas, and a small resource tree is constructed to manage the resource nodes in the subareas.

[0017] Preferably, the task decomposition includes two parts of function decomposition and data decomposition. The function decomposition is to parallelize the application function after decomposing the application function at the application layer to reduce the task calculation delay. The data decomposition is to pass the input data of the task to other nodes for calculation after decomposing the input data.

[0018] Preferably, the creation of the size resource tree in the second step includes the following steps:

[0019] First step: read the resource table from the storage medium, judge whether the resource table exists, if the resource table does not exist, obtain the resource table from the server in the domain, judge again whether the resource table exists, if not, obtain the resource table from the cloud server, if the resource table does not exist in the cloud node, it means that the resource table has not been created;

[0020] The resource tree has not been created, and the size resource tree is constructed:

[0021] 1. Collect the information of the nodes reported within a time period by the master nodes in each area;

[0022] 2. Wait for and notify other nodes in the area to report their current state information;

[0023] 3. Report the node information of each area master node to the edge layer master node (set the timeout time as t timeout );

[0024] 4. The edge layer master node pushes the collected information into the queue in batches, and then dequeues accordingly to construct a B+ resource tree;

[0025] 5. Broadcast the constructed B+ resource tree to all nodes, and re-feed the first step to read the resource table.

[0026] Second step: when the resource table is created, the MEC needs to be broadcast to all nodes, and the resource table needs to be maintained;

[0027] Maintenance of resource tree: regular update: update all node information at a certain time;

[0028] Event update: the number of new nodes exceeds the threshold node and forces update.

[0029] (Three) beneficial effects

[0030] Compared with the prior art, the present application provides a method for task scheduling based on size resource tree reinforcement learning, which has the following beneficial effects:

[0031] 1. The method for task scheduling based on size resource tree reinforcement learning breaks through the bottleneck encountered by the traditional solution that needs to be scheduled by each layer of master node, avoids the problem of secondary task unloading, and can fully utilize all computing resources in the entire system to complete the task calculation in the shortest time.

[0032] 2. The method for task scheduling based on size resource tree reinforcement learning reduces the computing delay of tasks from two key dimensions based on the granularity difference task decomposition algorithm based on computing amount perception and the task scheduling algorithm based on size resource tree and reinforcement learning, and can fully utilize the potential computing amount of edge computing network.

[0033] 3. The method for task scheduling based on size resource tree reinforcement learning can quickly decide to schedule tasks to nodes with computing ability, further improve the execution speed of tasks and reduce the execution time of tasks. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 Formula 4 for network modeling in the present application;

[0035] Figure 2 Different stages and resource information table schematic diagram;

[0036] Figure 3 Size resource tree model symbol meaning contrast representation intention;

[0037] Figure 4 Small resource tree diagram divided according to slices in the domain;

[0038] Figure 5 Resource tree construction process schematic diagram;

[0039] Figure 6 Computing task submission to computing completion whole cycle schematic diagram;

[0040] Figure 7 Fig. 1 is a schematic diagram of a DQN algorithm model;

[0041] Figure 8 Fig. 4 is a schematic diagram of parallel scheduling of decomposed subtask sets;

[0042] Figure 9 Fig. 7 is a schematic diagram of the construction and updating process of the size resource tree;

[0043] Figure 10 Fig. 8 is a schematic diagram of the implementation algorithm of the construction and updating process of the size resource tree;

[0044] Figure 11 Fig. 9 is a schematic diagram of the implementation of the granularity difference task decomposition algorithm based on the computation amount perception;

[0045] Figure 12 Fig. 10 is a schematic diagram of the deep reinforcement learning task scheduling algorithm of the size resource tree; DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0047] Please refer to Figures 1-12 .

[0048] A method for reinforcement learning task scheduling based on a size resource tree comprises the following steps:

[0049] Step 1: Network model

[0050] The MEC network node is composed of cloud, edge and end nodes. Assuming that the total number of nodes in the entire system is n, then:

[0051]

[0052] respectively represent the number of cloud, edge and end nodes, and denotes the communication delay between node i and node j of type ψ, then the n-order delay matrix of node ψ i to other nodes is ψ∈S γ , S γ ={C, E, D}, S γThis represents the set of cloud, edge, and endpoint node types. To ensure that each node in the system can perceive the communication latency with other nodes, each node in the MEC network needs to dynamically maintain a latency matrix between itself and other nodes. This matrix is ​​used to select the most suitable node to run the offloaded subtask. Assuming that the upper triangular part of the latency matrix represents the uplink communication latency and the lower triangular part represents the downlink communication latency, and using the average round-trip time (RTT) to replace the sum of the uplink and downlink communication latency, the latency set of node i with other nodes can be represented as:

[0053]

[0054] Assuming node j has the minimum communication latency with the current node, then j satisfies the following condition:

[0055]

[0056] For a size of The size of the returned result is The calculation time is And looking forward to Task Q to be completed within the time limit i In other words, task Q i By machine ψ α Uninstall to ψ β The transmission delay, computation time, and communication delay throughout the entire process should meet the following conditions:

[0057]

[0058] in This represents the current node ψ α Uplink and downlink bandwidth, This represents the program's execution time. The target node ψ that executes the subtask β The remaining computational load and task Q i The amount of computation required is determined by both ω(ψ) and ω(ψ). β This represents the average number of cycles required to process each bit. Indicates device ψ β frequency, This represents the round-trip delay between nodes.

[0059] Let the terminal device ψ α To the target device ψ β The minimum bandwidth is Due to the calculation results The size is usually very small, and its transmission delay is negligible. Therefore, the bandwidth required for task offloading must meet the following requirements:

[0060]

[0061] The same kind of computing resources of the nodes are structured into a table, as shown in Figure 1 The computing delay of a task is:

[0062] t cost = t part + t trans1 + t disk + t mem + t cpu + t gpu + t trans2 + t merge (7)

[0063] Where t part represents the time consumed by task decomposition, t trans1 represents the propagation delay of the task in the unloading process, t disk represents the time consumed by task writing to the hard disk, t mem represents the time consumed by task loading into memory from the hard disk after preparation for running, t cpu , t gpu represent the time consumed by task execution in CPU and GPU respectively, t trans2 represents the propagation delay of task execution result feedback, and t merge represents the time consumed by merging the execution results of subtasks.

[0064] The calculation method of each part of formula (8) is as follows:

[0065]

[0066] As shown in Figure 2 , the figure is the process of unloading a task to the computing of three nodes, where the PART table represents the task decomposition process, the NET table represents the task propagation process in the MEC network, the DISK table represents the task writing to the hard disk process, the MEM table represents the task loading into memory from the hard disk process, the CPU table and the GPU table represent the task execution process, and the MERGE table represents the merging process of the task calculation results.

[0067] Second step: size resource tree model

[0068] The resource types in the MEC network include CPU, GPU, memory, hard disk, and bandwidth, which are important computing-related resources. How to organize and manage these computing resources and make the task complete in the shortest possible time is the key problem to be solved in this research.

[0069] A large and small resource tree model is constructed to manage computing resources in the MEC network. Nodes submitting tasks in the MEC network are terminal nodes, and nodes participating in computation include terminal nodes, edge nodes, and cloud nodes. To manage the massive number of nodes in the entire MEC network, a domain-based mechanism is adopted. Within a domain, large and small resource trees are used to manage nodes, with the large resource tree managing all resource nodes within the domain. Due to the large number of nodes, the update speed is reduced to minimize additional computational overhead. The domain can be further subdivided into regions, such as... Figure 2 As shown, a small resource tree is constructed to manage resource nodes within the area. Due to the small number of nodes, the update frequency can be increased, providing faster status information updates for nodes within the area.

[0070] Different types of tasks have different resource requirements. This study constructs resource trees containing five common resource requirements: CPU, GPU, memory, hard disk, and bandwidth. For node j, these can be represented by symbols... The notation indicates that Ω represents the resource tree type, which can be a large resource tree within a domain or a small resource tree in a region. For any type of resource tree, it contains resource information such as global ID, local ID, node type, resource type, maximum value, unit, utilization rate, and available size. The resource type corresponding to node i in the resource tree is represented by the symbol... and To express.

[0071] Since only terminal nodes can submit tasks, the small resource tree can only be constructed by terminal nodes. The radius of the circle representing the region of the small resource tree is determined by the maximum task latency that the terminal node submitting the task can accept and the maximum number of nodes that the small resource tree can accommodate. Together, it is determined that when node i and terminal node j satisfy the following conditions:

[0072]

[0073] C2:

[0074] C1 indicates that the delay between the two nodes is less than the maximum delay that the terminal node j can accept. Node i can be added to the resource tree. and the set of resource trees added to the new node i. Add a new value Otherwise, delay the list of terminal nodes in node i. Add a delay record between node i and node j. Establish a reverse association, skip the current node, and continue searching for the next node that meets the requirements. This is done through calculation. length The number of small resource trees that the current node i joins can be determined. The larger the value of N, the more tasks the node can receive from more terminal nodes. If N = 0, the node is in a free state and has not joined the small resource tree. The terminal node cannot schedule tasks to the node. In order to avoid the waste of resources caused by the node being free, the node whose length satisfies the following condition is found:

[0075]

[0076] The value of the number of resource trees that node j needs to join at least. The value is related to the number of nodes in the domain and is controlled by percentage. Delay list It can be used to find the terminal node with smaller delay from the current node. After the node modifies the delay radius, it can be used to apply to rejoin the resource tree of the node.

[0077] In a single domain k, it is assumed that the total number of nodes contained is The number of terminal nodes e is The more nodes the resource tree of the terminal node contains, the more abundant the schedulable resources the terminal node has. Conversely, the node may face the problem that tasks cannot be decomposed and scheduled to other nodes for running, thereby causing task execution timeout. In order to more fully and reasonably utilize the resources in the domain, the node utilization rate in the domain is defined as and the end node occupancy rate as an evaluation index to optimize the construction of the resource tree, which is defined as follows:

[0078]

[0079] The node utilization rate in the domain reflects the number of resource trees that the nodes in the domain are joined (for resource nodes). The larger the value, the greater the probability of using the node. However, this value is not the larger the better. When all terminal nodes join the node to their own small resource tree is the maximum, but this will cause the resource update speed to slow down and cause more serious problems. The end node occupancy rate reflects the ratio of the number of nodes in the resource tree of the terminal node j in the domain to the total number of nodes in the domain (for device nodes). The value reflects how many resource trees the terminal node can schedule tasks to. Similarly, the value is not the larger the better.

[0080] When a new terminal node e joins the MEC network, it will automatically find other nodes in the domain and join the small resource tree of the node according to formula (15) and formula (16) until there is no node in the domain that satisfies the condition or has reached the maximum value, then stop updating.

[0081] For a task Q i , the most basic resource requirements for completing the task need to be considered, including the clock speed, memory, bandwidth, and latency of the computer, and key indicators, and a clock speed B+ tree is constructed for the entire system Memory B+ tree Bandwidth B+ tree Since the delay between nodes is not the same, a delay B+ tree needs to be constructed for each node in the region The B+ tree is maintained by the master node in the region and shared with other nodes in the region. Before building the B+ resource tree, the information of each node needs to be pushed into the queue in the order of return, and then dequeued and built into the B+ resource tree. Because the edge layer is the middle layer, it can quickly communicate with the cloud and terminal, and has relatively strong computing capacity among the three types of nodes, so the B+ tree construction work is given to the edge layer to complete.

[0082] In the process of constructing the B+ resource tree, the most time-consuming part is the collection of node information in the system, so multiple master nodes are set in each region or each layer to collect the node resource information in their adjacent regions, and then the master nodes in each layer transmit the node information in the region to the master node in the edge layer. Push into the queue Q info , and then dequeue in turn to construct the B+ resource tree. After the B+ resource tree is constructed, it is distributed to the master nodes of each layer by the master node of the edge layer By the master nodes of each layer Distribute to all resource nodes in their sub-regions, thus completing a resource tree update.

[0083] Step 3: Task decomposition model

[0084] Task decomposition includes functional decomposition and data decomposition, which consists of two parts. Functional decomposition is to parallelize the application function after decomposing the application function at the application level to reduce task computing delay, which needs to be decoupled by application designers and developers, and has poor flexibility. Data decomposition is to pass the input data of the task to other nodes for calculation after decomposing the input data, which reduces the task computing burden of a single node and reduces the overall computing time. Common tasks that can be data decomposed include image, audio, video and other types of data classification, recognition and calculation, etc. These types of tasks can directly achieve task calculation acceleration and significantly reduce terminal device waiting delay through data decomposition.

[0085] First, we can conclude that the finer the granularity of task decomposition, the better the task decomposition is not too small, too small granularity of task decomposition will bring waste of local computing resources and longer decomposition time; of course, the granularity of task decomposition can not be too large, otherwise it may lead to the decomposition of the task still no node can be directly calculated, and longer task transmission waiting time. Therefore, we need to determine a reasonable task decomposition granularity π i Since the remaining computing resources of the target offload nodes are inconsistent and dynamically changing, we propose a granularity difference task decomposition strategy based on computing capacity perception. This strategy dynamically determines the size of the task decomposition granularity cal and the number of subtasks m according to the remaining resources SR

[0086] Assume that a terminal node submits a new task The demand for computing resources of this task can be composed of basic resources such as CPU, GPU, memory, hard disk, and bandwidth, which are represented by symbols Computing capacity perception includes two aspects, namely the remaining computing capacity of each node in the MEC network and the computing capacity required by the currently submitted task The perception of task computing capacity is achieved by comprehensive calculation of the demand for the above five types of resources. Different types of tasks have different resource requirements. If the volume of a task is large, since the edge node has the characteristics of large bandwidth and low delay, it is more inclined to offload most of the data of the task to the edge node for calculation; the cloud node is more suitable for tasks that do not consume a large amount of computing resources; the terminal node is suitable for executing tasks that are small in size and small in computing capacity. Use α, β, γ three weight factors to represent the degree of inclination of the task to be offloaded to the end, edge, and cloud nodes, where:

[0087] α+β+γ=1 (18)

[0088] By calculating the total demand of the task for various resources, the computing capacity required by the task is obtained, and the relationship between the computing capacity required by the executed task is:

[0089]

[0090] k1+k2=1 (20)

[0091] where k1 and k2 are proportional coefficients, the values of which are related to the type of task.

[0092] Assume that the remaining computing resources in the small resource tree are​​ Submit task The computing resource left in the node itself is In order to process the task results completed by other nodes in time, we do not allow the resource occupancy rate of the node submitting the task to reach 100%, so we set a local resource maximum occupancy factor Then the task The computing amount to be decomposed is:

[0093]

[0094] Among them denotes the total computing resource (AllLocal Resource, ALR) possessed by the local node i, so denotes the local remaining allocable computing resource. At the same time, the local remaining computing amount must satisfy:

[0095]

[0096] Now according to the remaining resource SR cal in the small resource tree, the computing amount to be decomposed of the task is further decomposed. In order to avoid too small and too large task decomposition granularity, the minimum and maximum decomposition granularity π min and π max are set respectively. If the following conditions are met, one step of task decomposition is carried out, and the decomposed subtask is added to the subtask set

[0097]

[0098] Among them is the subtask executed by the local node, which is also the first subtask decomposed by the local node i, and denote the number of subtasks in the subtask set and the number of resource nodes in the small resource tree respectively.

[0099] Fourth step: task scheduling model

[0100] In the task model of hybrid computing architecture, a large number of nodes are involved in resource scheduling and computing. If the number of nodes in the entire MEC network is small, the performance of the task scheduling algorithm cannot be reflected. Once the number of resource nodes becomes very large, if the task scheduling algorithm is not optimized, it will cause node overload or starvation phenomenon, and the task calculation cannot be completed within the expected time. In order to further reduce the delay of task calculation and improve the efficiency of task scheduling, a multi-class resource management model based on B+ tree is proposed. Since different types of tasks have different demands for various resources of the computer system, in order to provide optimal system resource allocation for different types of tasks, a size resource tree is constructed for different types of resources, and a DQN task scheduling algorithm based on the size resource tree is used to schedule the decomposed tasks to different nodes for calculation, so as to speed up the scheduling of sub-tasks to the target nodes with sufficient computing power and meeting the delay requirement. The block diagram of task submission and calculation completion is shown in Figure 6 .

[0101] In the task scheduling model based on reinforcement learning, the three most important components of reinforcement learning are the state of the environment (state), the action space of the agent (action), and the reward after executing the action (reward). Due to the large number of nodes in the MEC network, the state space and action space are too large, so a neural network is used to fit.

[0102] The state space is:

[0103]

[0104] where represents the remaining resources of each type of node i at time t, represents the task set waiting in node i at time t, represents the state of the resource tree in node i at time t.

[0105] The action space is:

[0106]

[0107] a i =[a1(t),a2(t),…,a n (t)],a n (t)∈{0,1} (28)

[0108] Q i =[q1(t),q2(t),…,q n (t)],0≤q n (t)≤Q i (29)

[0109] where ai Let Q represent the unloading decision vector of node i. i This represents the task assignment vector for node i.

[0110] The reward function is:

[0111]

[0112] Task speedup A higher task speedup ratio indicates a better off-target decision-making performance for the agent, and vice versa. The upper limit of the task speedup ratio depends on the hardware performance in the MEC network.

[0113]

[0114] The loss function is calculated as shown in formula (32).

[0115] In the DQN algorithm, the agent obtains the state s from the environment. t According to state s t Action a is performed with probability ε. t The environment will generate a reward r accordingly. t and obtain a new environment. t+1 , will (s t ,a t ,r t ,s t+1 The data is stored in the experience replay pool for later use. Every N steps, the Q-network updates the parameter information to the target-Q network. The target-Q network and the Q-network have identical structures, differing only in parameters. Updating the Q-target network every N steps makes the algorithm converge more stably. The Q-network adapts to the input environment s... t The operation yields Q(s) t ,a t This is called Q estimation, where the target network outputs the next environmental state s (target-Q). t+1 The value obtained after the operation is called Q-true. The loss value is obtained by calculating the mean square error of Q-true and Q-estimate, and the Q-network parameters are updated.

[0116] In the task scheduling and distribution process, the DQN task scheduling algorithm based on large and small resource trees innovatively adopts a fully parallel task scheduling scheme, such as... Figure 8 The node that submits the task will decompose the task and schedule it to other nodes in the MEC network through a task decomposition algorithm and a task scheduling algorithm. This scheduling strategy breaks through the bottleneck encountered by traditional solutions that require scheduling through master nodes at each layer, avoids problems such as secondary task unloading, and can give full play to all the computing resources in the entire system to complete the task computation in the shortest time.

[0117] Algorithms for creating and maintaining large and small resource trees

[0118] The general steps for creating a resource tree are as follows: First, it attempts to read the resource table from the storage medium to determine if the resource table exists. If the resource table does not exist, it attempts to retrieve the resource table from the domain server, and again checks if the resource table exists. If it does not exist, it attempts to retrieve the resource table from the cloud server. If the resource table also does not exist on the cloud node, it means that the resource table has not been created, and the following steps are performed:

[0119] 1. The master nodes in each region collect t max Information of nodes to be reported within a specified time;

[0120] 2. Wait for and notify other nodes in the area to report their current status information;

[0121] 3. Each regional master node reports its node information to the edge layer master node (with a timeout set to t). timeout );

[0122] 4. The edge layer master node pushes the collected information into the queue in batches, and then dequeues it in the same way to construct the B+ resource tree;

[0123] 5. Broadcast the completed B+ resource tree to all nodes;

[0124] Once the resource tree table is created, it needs to be broadcast to all nodes via MEC. Simultaneously, the resource tree needs to be maintained. The following two update strategies can be used for resource tree maintenance:

[0125] Scheduled updates: All node information is updated at regular intervals;

[0126] Event Update: When the number of newly added nodes exceeds the Threshold, the node is forced to update.

[0127] Computationally Aware Granularity Difference Task Decomposition Algorithm

[0128] Based on the decomposition and calculation of task resource requirements, the computational load of completing a task is perceived, and then the task is analyzed according to the size of the resource tree. When performing granularity-based decomposition, it's important to note that because resources in an MEC network are constantly changing, the same task decomposed at different times will yield subtasks of different granularities. This is done based on the resource tree size. The task Decomposed into a set of subtasks like Figure 11 .

[0129] Deep reinforcement learning task scheduling algorithm based on large and small resource trees

[0130] State space: the state space of the MEC network is composed of the remaining computing resources c i (t) of each node in the entire MEC network at the current time the number of newly generated tasks of the terminal node the computing amount required by each new task and the highest delay requirement of each task Let s(t) represent the state of the agent at time t, and s(t) is defined as follows:

[0131]

[0132] wherein represent the cloud, edge and terminal nodes,

[0133] Action space: the action space of the agent is composed of whether to schedule the task decision and task scheduling node a i (t) selection. Among them 0 and 1 represent whether to schedule the task, Sq i (t) = [q 1(t) , q 2(t) , …, q nodemax (t)], q i(t) represents the computing amount of task Q allocated to the node.

[0134] Reward function: each task scheduling will perform the corresponding action, and after the action is executed, we need to score the action. Since the purpose of task scheduling is to face low latency, we define the "task speedup ratio" parameter as the reward function of the agent. The larger the task speedup ratio The larger the task speedup ratio, the better the effect of the agent's offloading decision, and vice versa, wherein:

[0135]

[0136] Although embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for task scheduling based on size resource tree reinforcement learning, comprising the following steps: Step 1: Calculate the computing resources in the MEC network; Step 2: Build a size resource tree model; Step 3: Use the size resource tree to decompose the task; Step 4: Schedule the decomposed task to the nodes in the MEC network for calculation; The construction of the size resource tree model in the second step includes the following steps: Step 1: Read the resource table from the storage medium, judge whether the resource table exists, if the resource table does not exist, get the resource table from the domain server, judge again whether the resource table exists, if not, get the resource table from the cloud server, if the cloud node also does not exist the resource table, it means that the resource table has not been created, re-construct the size resource tree; Step 2: When the resource tree is created, it needs to be broadcast to all nodes by MEC, and the resource tree needs to be maintained; The size resource tree model is used to manage the computing resources in the MEC network, and the size resource tree model adopts a domain mechanism, and the large resource tree manages all resource nodes in the domain, which can be further divided into subareas to construct a small resource tree to manage the resource nodes in the subarea; The construction of the size resource tree includes the following contents: (1) Collect the information of the nodes reported within a period of time by the master nodes in each area; (2) Wait and notify other nodes in the area to report their current state information; (3) Report the node information of each area master node to the edge layer master node; (4) The edge layer master node pushes the collected information into the queue in batches, and then dequeues to construct a B+ resource tree; (5) Broadcast the completed B+ resource tree to all nodes.

2. The method of claim 1, wherein the method is based on a size resource tree. The calculation of resources in the first step includes the network bandwidth resources between nodes, node CPU frequency resources, node memory resources, node hard disk storage resources, and node GPU resources.

3. The method of claim 1, wherein: The task decomposition includes two parts: function decomposition and data decomposition. Function decomposition is to decompose the functions of an application at the application level and parallelize the processing to reduce task computation delay. Data decomposition is to pass the input data of the task to other nodes for calculation to reduce task computation delay.

4. The method of claim 3, wherein: The maintenance of the size resource tree adopts the following two update strategies: Resource tree maintenance: Timely update: update all node information at a certain time; Event update: If the number of new nodes exceeds Threshold node, force update.

Images