Method and system for joint task offloading and resource allocation of multi-access edge computing

Through deep Q network and reinforcement learning technology, task execution and resource allocation in multi-access edge computing are optimized, multi-task dependency and resource allocation problems are solved, task execution with low latency and low energy consumption is achieved, and computing-intensive applications are met demand.

CN113747504BActive Publication Date: 2024-12-03SHANDONG NORMAL UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202110863448.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-29
Publication Date
2024-12-03
Estimated Expiration
2041-07-29

AI Technical Summary

Technical Problem

The existing research on multi-access edge computing mainly focuses on independent tasks, making it difficult to effectively deal with multiple interdependent tasks, resulting in difficult task execution priority and resource allocation, and cannot meet the low latency and low energy consumption requirements of computing-intensive applications. .

Method used

Deep Q network (DQN) combined with reinforcement learning, optimize task execution and resource allocation through state, action and reward functions, determine task priority and execution strategy, and use ε greedy algorithm and experience playback mechanism to optimize IoT devices and edge servers. and the resource configuration of remote servers to achieve optimal uninstallation and allocation of tasks.

Benefits of technology

It effectively reduces the long-term weighted sum of task execution delay and energy consumption, meets the QoS needs of IoT devices, and improves the efficiency and applicability of task execution.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113747504B_ABST
    Figure CN113747504B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of mobile communication technology, and provides a method and system for joint task offloading and resource allocation for multi-access edge computing. The method includes determining the priority of task execution according to the specified latest completion time of the tasks constituting each application and the latest allowed completion time during the actual execution process; determining the three elements of reinforcement learning for the current application environment: state, action and reward; based on the resource status of the Internet of Things device, edge server and remote server, the status of the task to be executed and the priority queue status, using the DQN network, combining the resource status of the Internet of Things device, edge server and remote server and the real-time information of the application generated by the Internet of Things device, to obtain various actions in the current state and the rewards corresponding to each action, until the DQN network converges, and the optimal task execution and resource allocation strategy is obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of mobile communications, and in particular, relates to a method and system for multi-access edge computing joint task offloading and resource allocation. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rise of 5G networks, services such as Industry 4.0, virtual / augmented reality, and autonomous driving are developing rapidly, and mobile data traffic is experiencing explosive growth. These computing-intensive applications are usually characterized by low latency and high energy consumption. Due to the storage and computing characteristics of IoT devices, it is difficult to meet the requirements of these applications or simply cannot run complex programs. Therefore, IoT devices must upload application data to remote servers for further storage and processing. Although the computing resources of remote servers are unlimited, reaching the cloud through the public Internet may cause network congestion and it is difficult to meet the low latency requirements of computing-intensive applications. Therefore, neither IoT devices nor remote cloud servers can meet the needs of development.

[0004] Multi-access edge computing (MEC) enables efficient computational offloading and provides an effective way to cope with the growing computational demands. MEC provides cloud computing capabilities by deploying edge servers near IoT devices. Typically, edge servers are located at the aggregation point of the core network or wireless access point (RAP) to ensure the requirements of low-latency applications. In this way, a large number of latency-sensitive applications generated by IoT devices (such as smart cities, e-health, and driverless driving) can offload a large amount of perception data to edge servers for further processing and storage. Compared with cloud computing, MEC can not only reduce latency, but also provide a better IoT device experience.

[0005] Most of the existing research on multi-access edge computing focuses on independent tasks. In real scenarios, an application often consists of multiple interdependent tasks. The task execution priority and execution location will affect the performance of task execution, and thus affect the execution of the entire application. Summary of the invention

[0006] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a method and system for multi-access edge computing joint task offloading and resource allocation, which optimizes the target by studying the offloading of multi-dependent tasks of multiple IoT devices in the MEC system, and minimizes the long-term weighted sum of the execution delay and energy consumption of the task while satisfying the preferences of different IoT devices.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A first aspect of the present invention provides a method for multi-access edge computing joint task offloading and resource allocation.

[0009] The method of multi-access edge computing joint task offloading and resource allocation includes:

[0010] Determine the priority of task execution according to the specified latest completion time of the tasks constituting each application and the latest allowed completion time during the actual execution process;

[0011] Determine the three elements of reinforcement learning for the current application environment: state, action, and reward;

[0012] Based on the resource status of IoT devices, edge servers and remote servers, the status of tasks to be executed and the status of priority queues, the DQN network is used. Combined with the resource status of IoT devices, edge servers and remote servers and the real-time information of applications generated by IoT devices, various actions in the current state and the rewards corresponding to each action are obtained until the DQN network converges to obtain the optimal task execution and resource allocation strategy.

[0013] Furthermore, suppose task X tnq The initial required completion time is SCT tnq , as its predecessor tasks are executed, task X tnq The latest acceptable completion time for the ACT tnq and start time AST tnq ; Define STS tnq It is Task X tnq The set of successor tasks, ACT tni For Task X tni ∈STS tnq The latest acceptable completion time, ETx tni For Task X tni Execution time on device x; if X tnw For application X tn The last task of ACT tnw Equal to SCT tnw , complete the time constraint;

[0014] For Task X tnq From its next mission X tni Reversely calculate its latest acceptable completion time, that is,

[0015]

[0016] Therefore, Task X tnq The latest acceptable start time for

[0017]

[0018] According to formulas (1) and (2), the latest acceptable start time of all tasks is obtained, and based on the latest acceptable start time of all tasks, the priority of task execution is obtained.

[0019] Furthermore, the state elements include: state space, task status information, device status information and queue status information; the state space includes: information on three parts: task, device and queue; the task status information includes: the size of the task before preprocessing, input data size, computational complexity, output data size, maximum tolerable delay and latest acceptable completion time; the device status information includes: computing resource allocation of local devices, edge servers and remote servers; the queue status information includes: priority queue information.

[0020] Furthermore, the action elements include:

[0021] AC1: For the selection of k tasks in the decision period, the following rules are followed: First, select the task with the highest priority from the unexecuted tasks, and determine which IoT device and application the task belongs to; second, determine whether the task has an unexecuted predecessor task; if so, skip the task and determine the next priority task in the same way until an executable task is found or all tasks are checked; otherwise, execute the task and send SA k qs The element corresponding to the task is set to 0, SA k qs It is a row vector of N*Q*T*1, where N is the number of IoT devices, Q is the number of tasks generated by each IoT application in each time slot, and T is time.

[0022] AC2: includes an action, when AC1 selects task X to be performed tnq When task X is determined tnq On which device to execute; another action, specifying how much computing resources the device will allocate to the task; combining the possible values ​​of execution decision and computing resource allocation to obtain AC2.

[0023] Furthermore, the reward elements include: the agent is in state s∈SA k After executing action a∈AC, you will get a reward r(s,a) and enter the next state s'. The reward function is consistent with the optimization objective.

[0024] Furthermore, the optimization goal is to make the best offloading decision by configuring and adjusting the CPU frequency of the IoT device, and the allocation of computing resources on the edge server and the remote server, so as to minimize the total cost of the execution delay and energy consumption of the IoT device.

[0025] Furthermore, the DQN network includes: an input layer, two hidden layers and an output layer, the input layer has (6*nt*m×n c ) neurons, where nt represents the total number of tasks; the first hidden layer size is 128 neurons, the second hidden layer size is 64 neurons, and the output layer is a fully connected layer with neurons of the same size as the total action space.

[0026] Specifically, at each decision, the agent uses the ε greedy algorithm to make a decision. That is, the probability of selecting a random action and a greedy action is ε and 1-ε respectively, where ε min <ε<1. And the experience replay mechanism is adopted in DQN, that is, a replay memory with a capacity of NM is used to store the experienced data (s, a, r(s, a), s'). Every time the parameters are updated, the minibath data is extracted from the replay memory for updating, so as to break the association between the data. In the present invention, in order to maintain the dependency between the same application tasks, minibath IoT devices are randomly extracted from the replay memory for updating. Since each part of the state s has a different value range, such as the computing resource is regarded as a single-channel image, for each computing unit cr iτ,ic,k The value range of is (10,40), and the computational complexity of the task ranges from (1,3). Therefore, before the state s is input, the normalization method for each part is as follows:

[0027]

[0028] Among them, Max and min are the maximum and minimum values ​​of the sample data respectively. X is the current value that needs to be changed, and X* is the value after the change. After normalization, all values ​​of state s are between (0,1). All parts of state s are rearranged into a one-dimensional column vector by row. Then it is sent to the input layer of DQN, which has (6*nt*m×n c ) neurons, where nt represents the total number of tasks. Then it passes through 2 hidden layers. The first hidden layer size is 128 neurons, and the second hidden layer size is 64 neurons. Finally, it passes through a fully connected layer with neurons of the same size as the total action space |AC| as the output layer, and the output of each neuron corresponds to the action value function a∈AC.

[0029] A second aspect of the present invention provides a system for multi-access edge computing joint task offloading and resource allocation.

[0030] A system for multi-access edge computing joint task offloading and resource allocation, including:

[0031] A priority arrangement module, which is configured to: determine the priority of task execution according to the specified latest completion time of the tasks constituting each application and the allowed latest completion time during the actual execution process;

[0032] An element determination module is configured to: determine three elements of reinforcement learning for the current application environment: state, action and reward;

[0033] The optimal decision acquisition module is configured as follows: based on the resource status of IoT devices, edge servers and remote servers, the status of tasks to be executed and the status of priority queues, the DQN network is used, combined with the resource status of IoT devices, edge servers and remote servers, and the real-time information of applications generated by IoT devices, to obtain various actions in the current state and the rewards corresponding to each action, until the DQN network converges to obtain the optimal task execution and resource allocation strategy.

[0034] A third aspect of the present invention provides a computer-readable storage medium.

[0035] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method for multi-access edge computing joint task offloading and resource allocation as described in the first aspect above.

[0036] A fourth aspect of the present invention provides a computer device.

[0037] A computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for multi-access edge computing joint task offloading and resource allocation as described in the first aspect above are implemented.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] In the past, most studies on task offloading in MEC scenarios only considered independent tasks with a single goal, which was difficult to meet the actual task offloading situation and the obtained effect was not obvious. The present invention studies the offloading of multiple IoT devices and multiple applications in MEC, while minimizing the long-term weighted sum of task execution delay and energy consumption, and designs an effect function that minimizes the long-term weighted sum of task execution delay and energy consumption of IoT devices. An emerging deep Q network technology is used to solve this problem, which well balances the relationship between task execution delay and energy consumption, meets the QoS requirements of IoT devices, and has low complexity and good applicability.

[0040] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0042] Figure 1 is a flow chart of a method for multi-access edge computing joint task offloading and resource allocation of the present invention;

[0043] Figure 2 It is a schematic diagram of a multi-access edge computing scenario of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0045] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0048] Embodiment 1

[0049] like Figure 1 As shown, this embodiment provides a method for multi-access edge computing joint task offloading and resource allocation. This embodiment uses the method applied to a server as an example for illustration. It can be understood that the method can also be applied to a terminal, and can also be applied to a terminal, a server, and a system, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0050] Determine the priority of task execution according to the specified latest completion time of the tasks constituting each application and the latest allowed completion time during the actual execution process;

[0051] Determine the three elements of reinforcement learning for the current application environment: state, action, and reward;

[0052] Based on the resource status of IoT devices, edge servers and remote servers, the status of tasks to be executed and the status of priority queues, the DQN network is used. Combined with the resource status of IoT devices, edge servers and remote servers and the real-time information of applications generated by IoT devices, various actions in the current state and the rewards corresponding to each action are obtained until the DQN network converges to obtain the optimal task execution and resource allocation strategy.

[0053] Specifically, Figure 2 As shown: The multi-access edge computing system considered in the present invention includes several IoT devices, several edge servers and a remote cloud server. The IoT device can choose to execute the task locally on the IoT device or upload the task to the edge server or remote server for execution. Note: The service range of each server is limited. Each task can only select an edge server within its range as an access point, and execute on the access point or upload the task to the remote server for execution.

[0054] Figure 1 The flowchart of the method of the present invention comprises the following steps:

[0055] Step 1: Initialize the system and obtain the resource status of IoT devices, edge servers, and remote servers, as well as the real-time information of applications generated by IoT devices, as follows:

[0056] The computing and storage resources of IoT devices and edge servers are limited, and IoT devices can only offload tasks to optional edge servers. We define the set of optional edge servers for IoT device n as M(n). The computing and storage resources of remote servers far exceed those of edge servers. We divide time T into multiple time slots t∈T={1, 2, …, T}, and the duration of each time slot is equal to τ. Each IoT device continuously generates a variety of applications. Application generation is considered to be λ={0,1}, where λ=1 means that an application is generated, otherwise, no task is generated in this time slot. Tasks of the same IoT device have dependencies, that is, the task can only be executed after the predecessor task is completed. Tasks of different IoT devices are independent of each other. In each IoT device, some tasks can be executed on the local device, while other tasks can be transmitted to the edge server via radio or uploaded by the edge server to the remote server for execution during the same period. Regarding the real-time information about the applications generated by IoT devices, the qth task of the application generated by IoT device n in time slot t can be expressed as X tnq ={L tnq p , L tnq in , B tnq , L tnq out , D tnq Here, L tnq p is the size of the task before preprocessing, L tnq in Represents task X tnq The input data size of task X tnq The workload that needs to be transferred from IoT device n to edge server when offloading computing includes input parameters, program code, etc. tnq Represents task X tnq The computational complexity of executing task X tnq The required computing resources. Assuming that the task is executed locally on the IoT device or offloaded to an edge server or remote server, B tnq Remain unchanged. tnq out Represents task X tnq The output data size of task X tnq After the execution is completed from the edge server or remote server, the workload is returned to the IoT device n from the edge server or remote server. tnq Represents task X tnq The maximum tolerable delay of task X tnq The execution delay cannot exceed D tnqThis will be one of the important constraints of the research problem in this embodiment. This embodiment assumes that the tasks are indivisible, that is, each task can only be executed on one device.

[0057] Step 2: Determine the priority of task execution based on the specified latest completion time of the tasks that make up each application and the latest allowed completion time during the actual execution process. The specific process is as follows:

[0058] We assume that task X tnq The initial required completion time is SCT tnq , with the execution of its predecessor task, we can get task X tnq The latest acceptable completion time for the ACT tnq and start time AST tnq The smaller the latest acceptable start time of a task, the more urgent the task is and the earlier it needs to be executed, and the higher its priority. tnq It is Task X tnq The set of successor tasks, ACT tni For Task X tni ∈STS tnq The latest acceptable completion time, ETx tni For Task X tni The execution time on device x. If X tnw For application X tn The last task of (application generated by IoT device n in time slot t) is ACT tnw Equal to SCT tnw , that is, the completion time constraint. For task X tnq We can start from its next task X tni Reversely calculate its latest acceptable completion time, that is,

[0059]

[0060] Therefore, Task X tnq The latest acceptable start time for

[0061]

[0062] According to formulas (1) and (2), we can get the latest acceptable start time of all tasks, and then place all tasks in the priority queue in order of task priority.

[0063] Step 3: Determine the three elements of reinforcement learning: state, action, and reward function. The state elements of reinforcement learning in this embodiment are defined as follows:

[0064] In our work, the definition of state space includes information about three parts: tasks, devices, and queues. The state information of tasks consists of the size of the task before preprocessing, the size of input data, the computational complexity, the size of output data, the maximum tolerable delay, and the latest acceptable completion time. The state information of devices consists of the computing resource allocation of local devices, edge servers, and remote servers. The state information of queues is the information of priority queues. The state space SA at decision point k k It can be expressed as:

[0065]

[0066] Where m represents the number of devices, eq k m Indicates the computing resources of the mth device at the kth decision period. k es Indicates the computing resource allocation status of all devices, SA k ts is the task status information to be processed, SA k qs It is the priority queue information. k Each part is described in detail below.

[0067] Computing Resource Allocation SA k es is an m×n c The matrix, n c is the maximum number of resources among all devices. k es Each row of SA represents the computing resources of a device. k es Each element in the definition represents a computing unit in a decision period and is the smallest unit of resource allocation. iτ,ic,k , represents the allocation state of the icth computing resource of the ithτth device at the decision period k, where iτ∈{1, 2, …, m}, ic∈{1, 2, …, n c}.

[0068] Computational unit cr iτ,ic,k The possible value of is one of {-1, 0, 1}, representing three different states, as shown below:

[0069] Unavailable (cr iτ,ic,k =-1): For each device, not all computing resources are used to process tasks, and a portion needs to be reserved for its own use. iτ,ic,k =-1 means that the computing resource is a reserved resource and cannot be allocated to any task.

[0070] Available (criτ,ic,k =0): indicates that the computing unit is a computing resource allocated to task execution and is not allocated to any task.

[0071] ·Allocation (cr iτ,ic,k =1): indicates that the computing unit is a computing resource allocated to task execution, but has been allocated to the task and is unavailable at decision time k.

[0072] Initially, the computing resources allocated by the device to the task are all in an available state. When a task starts to be executed, a device allocates computing resources to the task as needed. After the allocation, the status of the allocated computing unit will change from 0 to 1.

[0073] SA k ts is the task status information to be processed at decision time k, including the size of the task before preprocessing, input data size, computational complexity, output data size, maximum tolerable delay, and the latest acceptable completion time. k qs is the queue information of decision period k. Its size is equal to the priority queue size after sorting in claim 3, so SA k qs is a row vector of N*Q*T*1. N is the number of IoT devices, Q is the number of tasks generated by each IoT application in each time slot, and T is the time. k qs Each element in corresponds to each task in the priority queue. k qs Each element in is set to 1. When there is a task assigned in the priority queue, SA k qs The element corresponding to this task is set to 0.

[0074] The action elements of reinforcement learning in this embodiment are defined as follows:

[0075] Action AC consists of the following two parts:

[0076] AC1: specifies how to schedule tasks in the priority queue. For the selection of k tasks in the decision period, we follow the following rules: First, select the task with the highest priority from the unexecuted tasks, and we can know which IoT device and application the task belongs to. Second, we determine whether the task has an unexecuted predecessor task. If so, we will skip the task and use the same method to determine the next priority task until we find an executable task or finish checking all tasks; otherwise, execute the task and send SA k qs The element corresponding to the task is set to 0.

[0077] AC2: AC2 consists of two parts. One is when AC1 selects task X to be executed. tnq When task X is determined tnq On which device to execute, that is, the execution decision {1,2,…,m}. Another action is to specify how much computing resources the device will allocate to the task, that is, the computing resource allocation {1,2,…,n c Combining some possible values ​​of execution decision and computing resource allocation, we can get AC2 = {1,2,…,m,1,2,…,n c}.

[0078] Action AC2 can be further written as:

[0079]

[0080] Among them, for action a∈AC2, we use a tuple to represent it. (iτ,ic,it) means that the icth computing resource of the iτth device is allocated to task X from the current moment for it time. tnq In other words, (iτ,ic,it) is also in SA k es Find available computing resources to place task X tnq However, due to the limitations of the maximum tolerable delay of the task, not all actions in AC2 are valid. Therefore, for task X tnq , only some actions are valid. To reduce the size of the action space, we filter out invalid actions before learning. tnq , if T tnq p +T tnq l >D tnq , then task X tnq Execution on the local device is invalid. And the allocated computing resources should be greater than

[0081] The reward elements for reinforcement learning in this embodiment are defined as follows:

[0082] For each step, the agent is in state s∈SA kAfter executing action a∈AC, a reward r(s,a) will be obtained and the next state s' will be entered. In our work, the reward function is designed to be consistent with the optimization goal. The optimization goal of the embodiment is to find an optimal task execution strategy by jointly optimizing the computational offloading decision of each IoT device n, the CPU frequency of the IoT device, and the computing resources of the edge server to minimize the total cost of long-term execution delay and energy consumption. By decoupling the objective function into small pieces in each decision period, the reward r(s,a) is defined as:

[0083]

[0084] Since the greater the reward for executing action a in state s in the short term, the better the decision is, that is, the objective function of DQN is to obtain the maximum reward, the reward value should be negatively correlated with the objective function.

[0085] According to formula (5), we can further obtain the reward. The reward is defined as the cumulative reward obtained in each decision period in the long term. The reward R(k) of decision period k is defined as:

[0086]

[0087] Among them, 0<γ≤1 is the discount factor. When γ approaches 0, it means that the current moment's reward value is the main factor and is less affected by the future. When γ approaches 1, it means that the influence of the future moment's reward value becomes greater and is equally important as the current moment's reward value. s(k) and a(k) represent the state and action at the kth decision period, respectively. Therefore, maximizing the expected return E[(R(k)] is equivalent to optimizing the objective function.

[0088] The optimization objectives of this embodiment are as follows:

[0089] In this embodiment, we formulate the multi-dependent task offloading model into a mixed integer nonlinear programming problem. By jointly optimizing the computation offloading decision x of each IoT device n, tnq z , the CPU frequency of IoT devices, the computing resources of edge servers and remote servers, our optimization goal is to find an optimal task execution strategy to minimize the total cost of execution latency and energy consumption of IoT devices. The optimization problem is stated as:

[0090]

[0091] Among them, the binary variable x tnq z represents the computation offloading decision of the application task q generated by IoT device n at time slot t. tnq zThe value of z is one of the set {1, 2, 3}, z = 1 means that the task is executed on the local device, z = 2 means that the task is offloaded to the edge server for execution, and z = 3 means that the task is offloaded from the edge server to the remote server for execution. Constraint C1 ensures that at any time slot t, the total bandwidth allocated to all tasks offloaded to edge server j cannot exceed the total bandwidth W of edge cloud j jmax ; Constraint C2 ensures that no matter which computing method is chosen, the execution delay of the task does not exceed the maximum tolerable delay of the task; Constraints C3 and C4 are the CPU frequency of the IoT device and the computing resource constraints of the edge server, respectively.

[0092] This embodiment considers two parts, task preprocessing and subsequent task execution, when proposing the optimization target. The details of task preprocessing are as follows:

[0093] In the real world, in order to reduce storage, transmission, computing and other resources, preprocessing (such as compression) is usually required before task execution. However, task preprocessing consumes the information processing resources of IoT devices. Therefore, the time cost and energy consumption cost of task preprocessing mainly depend on the CPU frequency of IoT devices. tnq The delay and energy consumption of preprocessing are T tnq p and E tnq p The computing power of IoT devices is f tn ∈F tn , (ie, CPU cycles per second) and F tn ={f tn 1 ,...,f tn max}, therefore, task X tnq The preprocessing delay can be defined as:

[0094]

[0095] The power consumption of IoT device n during preprocessing in time slot t can be divided into two parts, P tnq p =P leak +P tnq dynamic Among them, P leak is independent and has nothing to do with the CPU frequency. We define it as a constant ε; P tnq dynamic With V tn 2 f tn Proportional, which means that at f tn The power consumed by logic gate switching, where V tnis the CPU supply voltage of IoT device n in time slot t. Since DVFS is for energy saving purpose, higher f tn The larger V tn Provided, and V tn ∝(α1f tn +α2); α1, α2 ≥ 0. Therefore, task X tnq The energy consumption of preprocessing can be defined as:

[0096]

[0097] According to (8) and (9), the cost of task preprocessing can be obtained as:

[0098]

[0099] Among them, I tnq 1 and I tnq 2 Represents task X respectively tnq The weight coefficient of execution delay and energy consumption satisfies 0≤I tnq 1 ≤1,0≤I tnq 2 ≤1, and the weight coefficient of each task may be different.

[0100] The specific process of subsequent task execution in this embodiment is as follows:

[0101] After task preprocessing, the task can continue to be executed on the IoT device, or it can be offloaded to the edge server or remote server for execution. Task execution latency and IoT device energy consumption are two key issues to consider for IoT devices in edge computing networks, which depend on the adopted task offloading decision and computing resource allocation. Therefore, the following introduces the task execution latency and IoT device energy consumption under three execution strategies.

[0102] 1) Local computing model

[0103] If task X tnq If you choose to execute locally on the IoT device, only Task X is included. tnq The execution delay T tnq l , defined as:

[0104]

[0105] Since Task X tnq It can be executed only after all its predecessor tasks have completed, i.e., task X tnq The ready time on the local device is:

[0106]

[0107] Among them, preX tnq Represents task X tnq The set of predecessor tasks; FT tni l , FT tni e and FT tni c Represents task X respectively tni The local computation completion time, edge server execution completion time and remote server execution completion time are calculated. Note that when a task is executed on the edge or remote server, a task is considered completed only when the task execution result is returned to the local device. If task X tni If executed locally, RT tnq l ≥FT tni l , because task X tnq Only when task X tni It can only be executed after the task is completed. tnq If the computation is offloaded to an edge server or a remote server, then RT tnq l ≥FT tni e or FT tni c , because task X tnq Must wait until task X tni The execution results are available locally.

[0108] Therefore, Task X tnq The local computation takes time to complete.

[0109]

[0110] Task X tnq The energy consumption of local computation is

[0111]

[0112] where v tn represents the energy consumption per CPU cycle. We design v tn =10 -27 (f tn ) 2 .

[0113] According to (11), (12), (13), and (14), the total cost of local computation of the task is:

[0114]

[0115] 2) Edge computing model

[0116] If task X tnq If you choose to offload to the edge server, then Task X tnq The time T for transmission to the edge cloud server through the wireless channel tnq eu , Task X tnq Execution time T on the edge cloud server tnq e , Task X tnq The execution result is returned to the IoT device at time T tnq ed .

[0117] Only when task X tnq Task X can only be executed when the previous task has been completed and the result has been transmitted back to the local device. tnq offloaded to the edge server. Therefore, task X tnq The ready time is:

[0118]

[0119] When the channel from local device n to edge server is available, task X can be sent to the local device n. tnq Send to the edge server, then task X tnq The available sending time is:

[0120]

[0121] Among them, AV nj represents the channel availability time from IoT device n to edge server j, and j∈M(n).

[0122] In this embodiment, we use TCP as the transport layer protocol for task transmission, and the transmission delay is related to the CPU frequency of the IoT device. This is because TCP requires a lot of CPU capacity to avoid retransmission requests, transmission congestion, etc. If the CPU frequency is too low, the CPU capacity may not be enough to process TCP tasks, thereby reducing TCP throughput. In addition, task X tnq The wireless transmission rate R tnq Or Task X? tnq The bandwidth W obtained from the edge server j it accesses tnqj Therefore, R tnq It is W tnqj and IoT device CPU frequency f tn The function of R tnq (W tnqj ,f tn ). In the embodiment, we define

[0123]

[0124] Among them, r tn max (W tnqj ) is the network throughput, which is only related to bandwidth; r tn * (f tn ) shows the impact of the CPU frequency of IoT devices on TCP transmission.

[0125] Then task X tnq The time from the local device to the edge server is:

[0126]

[0127] The transmission energy consumption is:

[0128]

[0129] Among them, P tnq u It is Task X tnq The average transmission power.

[0130] Define f tnq j (j≠0) is the computing resources (i.e., CPU cycles per second) allocated by edge server j for task q, then task X tnq The execution time on edge cloud server j is:

[0131]

[0132] Note: Definition F j is the total resources allocated to task processing by edge cloud server j. We assume that in each time slot, the resources reserved for task processing by each edge server are the same. Then if task X tnq In time slot t, the total computing resources of edge server j for executing tasks in time slot t must not exceed F j ,Right now:

[0133]

[0134] When the task is waiting to be sent on the local device and the task is executed on the edge server, we assume that the IoT device remains idle and define the power of the IoT device during idle time as P tnq bs , the corresponding energy consumption is:

[0135]

[0136] Task X tnqAfter the edge server completes the execution, the transmission time back to the IoT device can be expressed as:

[0137]

[0138] Among them, R tnq d is the data download rate of the local IoT device n. Since the result size of the task is much smaller than the task size and the download data rate is very high, the energy consumption of this part of IoT devices is ignored in this embodiment.

[0139] Therefore, Task X tnq The time it takes to complete execution on the edge server and return to the local device is:

[0140]

[0141] The corresponding energy consumption is:

[0142]

[0143] Based on the above analysis, the cost of the IoT device edge computing model is:

[0144]

[0145] 2) Remote server computing model

[0146] If task X tnq If you choose to offload to a remote server, then task X tnq The local device first offloads to a nearby edge server, and then offloads to the remote server. Task X tnq After execution on the remote server, the execution result is transmitted back to the local device via the edge server. Task X tnq Done. We assume that V e,c is the transmission rate between the edge server and the remote server, f tnq c is the computing power of the remote server. So the process consists of task X tnq Time T for offloading from local devices to edge servers tnq cu =L in tnq / R tnq (W tnqj ,f tn ), Task X tnq The time T for transmitting from the edge server to the remote server via the cable tnq cu’ =L in tnq / V e,c, Task X tnq Execution time T on the remote server tnq c =B tnq / f tnq c . Task X tnq The time T for the execution result to be transmitted from the remote server to the edge cloud server tnq cd =L tnq out / V e,c , and then the time T that is transmitted back to the local device by the edge cloud server tnq cd’ =L tnq out / R tnq d .

[0147] Among them, task X tnq The earliest ready time of the local device, task X tnq The earliest time that the IoT device can send to the edge server is the same as the second computing model, that is, RT tnq cl =RT tnq el RT tnq cs =RT tnq es .

[0148] Task X tnq The earliest time to reach the edge server is:

[0149]

[0150] Task X tnq The earliest possible time for the edge server to send data to the remote server C is:

[0151]

[0152] Among them, AV ec Indicates the earliest available time of the channel from the edge server to the remote server C.

[0153] Therefore, Task X tnq The time it takes to complete execution on the remote server and return to the local device is:

[0154]

[0155] The energy consumption includes the transmission energy consumption of the local device during task transmission and the energy consumption of other local devices when they are idle. Similarly, since the result size of the task is much smaller than the task size and the download data rate is very high, the energy consumption of the IoT device in the last part is also ignored. Therefore, the energy consumption can be defined as:

[0156]

[0157] Based on the above analysis, the cost of the remote computing model of IoT devices is:

[0158]

[0159] Step 4: Take the resource status of IoT devices, edge servers and remote servers, the status of the tasks to be executed and the priority queue status as the input of DQN, select the next task to be executed through the DQN network and obtain the corresponding reward. The details are as follows:

[0160] At each decision, the agent uses the ε greedy algorithm to make decisions. That is, the probability of selecting a random action and a greedy action is ε and 1-ε respectively, where ε min <ε<1. And the experience replay mechanism is adopted in DQN, that is, a replay memory with a capacity of NM is used to store the experienced data (s, a, r(s, a), s'). Each time the parameters are updated, the minibath of data is extracted from the replay memory for updating, so as to break the association between the data. In the embodiment, in order to maintain the dependency between the same application tasks, we randomly extract minibath IoT devices from the replay memory for updating. Since each part of the state s has a different value range, such as the computing resource is regarded as a single-channel image, for each computing unit cr iτ,ic,k The value range of is (10,40), and the computational complexity of the task ranges from (1,3). Therefore, before the state s is input, the normalization method for each part is as follows:

[0161]

[0162] Among them, Max and min are the maximum and minimum values ​​of the sample data respectively. X is the current value that needs to be changed, and X* is the value after the change. After normalization, all values ​​of state s are between (0,1). All parts of state s are rearranged into a one-dimensional column vector by row. Then it is sent to the input layer of DQN, which has (6*nt*m×n c) neurons, where nt represents the total number of tasks. Then it passes through 2 hidden layers. The first hidden layer size is 128 neurons, and the second hidden layer size is 64 neurons. Finally, it passes through a fully connected layer with neurons of the same size as the total action space |AC| as the output layer, and the output of each neuron corresponds to the action value function a∈AC.

[0163] Embodiment 2

[0164] This embodiment provides a system for multi-access edge computing joint task offloading and resource allocation.

[0165] A system for multi-access edge computing joint task offloading and resource allocation, including:

[0166] A priority arrangement module, which is configured to: determine the priority of task execution according to the specified latest completion time of the tasks constituting each application and the allowed latest completion time during the actual execution process;

[0167] An element determination module is configured to: determine three elements of reinforcement learning for the current application environment: state, action and reward;

[0168] The optimal decision acquisition module is configured as follows: based on the resource status of IoT devices, edge servers and remote servers, the status of tasks to be executed and the status of priority queues, the DQN network is used, combined with the resource status of IoT devices, edge servers and remote servers, and the real-time information of applications generated by IoT devices, to obtain various actions in the current state and the rewards corresponding to each action, until the DQN network converges to obtain the optimal task execution and resource allocation strategy.

[0169] It should be noted that the above-mentioned priority ranking module, factor determination module and optimal decision acquisition module are the same as the examples and application scenarios implemented by the steps described in Embodiment 1, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the above-mentioned modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0170] Embodiment 3

[0171] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the method for multi-access edge computing joint task offloading and resource allocation as described in the above embodiment 1 are implemented.

[0172] Embodiment 4

[0173] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for multi-access edge computing joint task offloading and resource allocation as described in the first embodiment are implemented.

[0174] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0175] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0178] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0179] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for joint task offloading and resource allocation for multi-access edge computing, characterized in that: include: Determine the priority of task execution according to the specified latest completion time of the tasks constituting each application and the latest allowed completion time during the actual execution process; Determine the three elements of reinforcement learning for the current application environment: state, action, and reward; Based on the resource status of IoT devices, edge servers and remote servers, the status of tasks to be executed and the status of priority queues, the DQN network is used to combine the resource status of IoT devices, edge servers and remote servers and the real-time information of applications generated by IoT devices to obtain various actions and rewards corresponding to each action in the current state until the DQN network converges to obtain the optimal task execution and resource allocation strategy; Assume that task X tnq The initial required completion time is SCT tnq , as its predecessor tasks are executed, task X tnq The latest acceptable completion time for the ACT tnq and start time AST tnq ; Define STS tnq It is Task X tnq The set of successor tasks, ACT tni For Task X tni ∈STS tnq The latest acceptable completion time, ETx tni For Task X tni Execution time on device x; If X tnw For application X tn The last task of ACT tnw Equal to SCT tnw , complete the time constraint; For Task X tnq From its next mission X tni Reversely calculate its latest acceptable completion time, that is, Therefore, Task X tnq The latest acceptable start time for According to formulas (1) and (2), the latest acceptable start time of all tasks is obtained, and based on the latest acceptable start time of all tasks, the priority of task execution is obtained; By jointly optimizing the computation offloading decision x of each IoT device n tnq z , the CPU frequency of IoT devices, the computing resources of edge servers and remote servers, our optimization goal is to find an optimal task execution strategy to minimize the total cost of execution latency and energy consumption of IoT devices; the optimization problem is expressed as: Where N is the number of IoT devices, Q is the number of tasks generated by each IoT application in each time slot, T is the time, and W is the tnqj It is Task X tnq The bandwidth T obtained from the edge server j it accesses tnq p It is Task X tnq Preprocessing delay, f tn is the CPU frequency of the IoT device, f tnq j (j≠0) is the computing resource allocated by edge server j for task q, F j is the total resources allocated to task processing by edge cloud server j, For Task X tnq The execution cost of preprocessing on the local device, C tnq l For Task X tnq The total cost of choosing to execute on the local device, C tnq e For Task X tnq Choose the total cost of executing on the edge cloud server, C tnq c For Task X tnq The total cost of selecting the execution on the remote cloud server, D tnq For Task X tnq The maximum tolerable delay of the binary variable x tnq z represents the computation offloading decision of the application task q generated by IoT device n at time slot t; in the decision variable x tnq z The value of z is one of the set {1, 2, 3}, z = 1 means that the task is executed on the local device, z = 2 means that the task is offloaded to the edge server for execution, and z = 3 means that the task is offloaded from the edge server to the remote server for execution; Constraint C1 ensures that at any time slot t, the total bandwidth allocated to all tasks offloaded to edge server j cannot exceed the total bandwidth W of edge cloud j jmax ; Constraint C2 ensures that no matter which computing method is chosen, the execution delay of the task does not exceed the maximum tolerable delay of the task; Constraints C3 and C4 are the CPU frequency of the IoT device and the computing resource constraints of the edge server, respectively.

2. The method for multi-access edge computing joint task offloading and resource allocation according to claim 1, characterized in that: The state elements include: state space, task state information, device state information and queue state information; the state space includes: information of three parts: task, device and queue; the task state information includes: the size of the task before preprocessing, input data size, computational complexity, output data size, maximum tolerable delay and latest acceptable completion time; the device state information includes: computing resource allocation of local devices, edge servers and remote servers; the queue state information includes: priority queue information.

3. The method for multi-access edge computing joint task offloading and resource allocation according to claim 1, characterized in that: The action elements include: AC1: For the selection of k tasks in the decision period, the following rules are followed: First, select the task with the highest priority from the unexecuted tasks, and determine which IoT device and application the task belongs to; second, determine whether the task has an unexecuted predecessor task; if so, skip the task and determine the next priority task in the same way until an executable task is found or all tasks are checked; otherwise, execute the task and send SA k qs The element corresponding to the task is set to 0, SA k qs is a row vector of N*Q*T*1, where N is the number of IoT devices, Q is the number of tasks generated by each IoT application in each time slot, and T is time; The SA k qs It is the priority queue information; AC2: includes an action, when AC1 selects task X to be performed tnq When task X is determined tnq On which device to execute; another action, specifying how much computing resources the device will allocate to the task; combining the possible values ​​of execution decision and computing resource allocation to obtain AC2.

4. The method for multi-access edge computing joint task offloading and resource allocation according to claim 1, characterized in that: The reward elements include: the agent is in state s∈SA k After executing action a∈AC, you will get reward r(s,a) and enter the next state s'. The reward function is consistent with the optimization goal; Among them, SA k is the state space at decision point k.

5. The method for multi-access edge computing joint task offloading and resource allocation according to claim 4, characterized in that: The optimization goal is to make the best offloading decision by configuring and adjusting the CPU frequency of the IoT device, the computing resource allocation on the edge server and the remote server, so as to minimize the total cost of the execution delay and energy consumption of the IoT device.

6. The method for multi-access edge computing joint task offloading and resource allocation according to claim 4, characterized in that: The DQN network includes: an input layer, two hidden layers and an output layer. The input layer has (6*nt*m×n c ) neurons, where nt represents the total number of tasks; the first hidden layer size is 128 neurons, the second hidden layer size is 64 neurons, and the output layer is a fully connected layer of neurons with the same size as the total action space; Among them, n c is the maximum number of resources among all devices, and m represents the number of devices.

7. A system for multi-access edge computing joint task offloading and resource allocation, based on the method for multi-access edge computing joint task offloading and resource allocation as described in any one of claims 1 to 6, characterized in that: include: A priority arrangement module, which is configured to: determine the priority of task execution according to the specified latest completion time of the tasks constituting each application and the allowed latest completion time during the actual execution process; An element determination module is configured to: determine three elements of reinforcement learning for the current application environment: state, action and reward; The optimal decision acquisition module is configured as follows: based on the resource status of IoT devices, edge servers and remote servers, the status of tasks to be executed and the status of priority queues, the DQN network is used to combine the resource status of IoT devices, edge servers and remote servers and the real-time information of applications generated by IoT devices to obtain various actions and rewards corresponding to each action in the current state until the DQN network converges to obtain the optimal task execution and resource allocation strategy; Assume that task X tnq The initial required completion time is SCT tnq , as its predecessor tasks are executed, task X tnq The latest acceptable completion time for the ACT tnq and start time AST tnq ; Define STS tnq It is Task X tnq The set of successor tasks, ACT tni For Task X tni ∈STS tnq The latest acceptable completion time, ETx tni For Task X tni Execution time on device x; If X tnw For application X tn The last task of ACT tnw Equal to SCT tnw , complete the time constraint; For Task X tnq From its next mission X tni Reversely calculate its latest acceptable completion time, that is, Therefore, Task X tnq The latest acceptable start time for According to formulas (1) and (2), the latest acceptable start time of all tasks is obtained, and based on the latest acceptable start time of all tasks, the priority of task execution is obtained; By jointly optimizing the computation offloading decision x of each IoT device n tnq z , the CPU frequency of IoT devices, the computing resources of edge servers and remote servers, the optimization goal is to find an optimal task execution strategy to minimize the total cost of execution delay and energy consumption of IoT devices; the optimization problem is expressed as: Where N is the number of IoT devices, Q is the number of tasks generated by each IoT application in each time slot, T is the time, and W is the tnqj It is Task X tnq The bandwidth T obtained from the edge server j it accesses tnq p It is Task X tnq Preprocessing delay, f tn is the CPU frequency of the IoT device, f tnq j (j≠0) is the computing resource allocated by edge server j for task q, F j is the total resources allocated to task processing by edge cloud server j, C tnq p For Task X tnq The execution cost of preprocessing on the local device, C tnq l For Task X tnq The total cost of choosing to execute on the local device, C tnq e For Task X tnq Choose the total cost of executing on the edge cloud server, C tnq c For Task X tnq The total cost of selecting the execution on the remote cloud server, D tnq For Task X tnq The maximum tolerable delay of the binary variable x tnq z represents the computation offloading decision of the application task q generated by IoT device n at time slot t; in the decision variable x tnq z The value of z is one of the set {1, 2, 3}, z = 1 means that the task is executed on the local device, z = 2 means that the task is offloaded to the edge server for execution, and z = 3 means that the task is offloaded from the edge server to the remote server for execution; Constraint C1 ensures that at any time slot t, the total bandwidth allocated to all tasks offloaded to edge server j cannot exceed the total bandwidth W of edge cloud j jmax ; Constraint C2 ensures that no matter which computing method is chosen, the execution delay of the task does not exceed the maximum tolerable delay of the task; Constraints C3 and C4 are the CPU frequency of the IoT device and the computing resource constraints of the edge server, respectively.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the method for multi-access edge computing joint task offloading and resource allocation as described in any one of claims 1 to 6 are implemented.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the method for multi-access edge computing joint task offloading and resource allocation as described in any one of claims 1-6 are implemented.

Citation Information

Patent Citations

  • Content caching decision and resource allocation combined optimization method based on mobile edge computing in Internet of Vehicles

    CN110312231A

  • Computing resource allocation and task unloading method for edge computing of super-dense network

    CN110798849A

  • Vehicle-mounted edge network task distribution and unloading method based on deep Q network

    CN111556461A

  • Mobile edge computing task unloading method based on independent learning

    CN112616152A

  • Method and system for unloading dependent tasks in multi-access edge computing

    CN112860337A