Method for unloading terminal equipment tasks in ultra-dense mobile edge computing environment

Abstract

The invention provides a method for unloading tasks of terminal equipment in an ultra-dense mobile edge computing environment. The method comprises the following steps: S1, constructing an ultra-dense mobile edge computing network system model; s2, determining a communication mode, an energy consumption calculation mode and a time delay calculation mode based on the ultra-dense mobile edge computing network system model, and calculating the total time delay and the total energy consumption of the terminal equipment for completing the computing task; s3, determining a target optimization problem according to the total time delay and the total energy consumption of the terminal device for completing the calculation task; and S4, solving the target optimization problem by adopting a deep reinforcement learning algorithm based on DDPG to obtain a task unloading strategy of the terminal equipment. According to the invention, the cooperation between the related base stations is considered, the load of the related base stations is balanced, the calculation task of the terminal equipment can be solved in effective time, and the communication delay of the terminal equipment can be reduced by adopting the NOMA technology when the terminal equipment transmits the calculation task to the related base stations.

Description

Technical Field

[0001] The present invention relates to a task offloading method, specifically a method for offloading tasks of terminal devices in an ultra-dense mobile edge computing environment. Background Art

[0002] With the rapid development of 5G networks and Internet of Things technologies, various application programs emerge in an endless stream, such as augmented reality (AR), virtual reality (VR), and intelligent video acceleration. These application programs are computationally intensive and latency-sensitive. They have high requirements for data transmission rate and computing power. However, the limited battery capacity and computing power of mobile devices cannot provide corresponding services for these application programs.

[0003] Mobile edge computing has achieved great development as a complementary computing paradigm to overcome the above challenges. Mobile Edge Computing (MEC) sinks computing resources to the network edge, and mobile devices can distribute their computing tasks to servers located at the "edge" of the radio access network. This can not only relieve network congestion in the cloud but also achieve lower computing latency and energy consumption, meeting the quality of service requirements of mobile devices. However, MEC under traditional cellular network deployment is difficult to meet the requirements of large-scale device access and communication quality. Therefore, the combination of mobile edge computing and ultra-dense networks has received extensive attention as a complementary computing paradigm to overcome the above challenges.

[0004] Although the combination of mobile edge computing and ultra-dense networks can expand the service coverage range and meet the needs of more mobile device computing tasks. However, there are still some problems when mobile devices offload tasks. For example, when multiple terminal devices transmit computing tasks and the OFDMA communication method is used, only one terminal device is allowed to transmit computing tasks in a time period, and other mobile devices need to wait at this time; when the base station associated with the terminal device does not have enough computing resources to provide services for it, the task needs to be uploaded to the macro base station, which will increase the corresponding transmission latency and cause network congestion in the cloud, thus affecting the quality of service for terminal devices. Summary of the Invention

[0005] The purpose of the present invention is to provide a terminal device task offloading method based on the DDPG algorithm in an ultra-dense mobile edge computing environment. This method can reduce the transmission latency generated when mobile devices transmit tasks and offload tasks to appropriate edge servers according to the offloading decisions of mobile devices, thereby minimizing the cost of the system.

[0006] The present invention is implemented as follows:

[0007] A method for offloading tasks of terminal devices in an ultra-dense mobile edge computing environment, comprising the following steps:

[0008] S1. Build a hyper-dense mobile edge computing network system model;

[0009] S2. Determine the communication mode, task offloading mode, energy consumption calculation mode, and delay calculation mode based on the hyper-dense mobile edge computing network system model. Calculate the total delay for the terminal device to complete the computing task based on the communication mode, task offloading mode, and delay calculation mode. Calculate the total energy consumption for the terminal device to complete the computing task based on the task offloading mode, delay calculation mode, and energy consumption calculation mode;

[0010] S3. Determine the target optimization problem according to the total delay and total energy consumption of the terminal device to complete the computing task;

[0011] S4. Use the deep reinforcement learning algorithm based on DDPG to solve the target optimization problem and obtain the task offloading strategy of the terminal device.

[0012] Further, the mobile edge computing network system model includes:

[0013] S micro base stations, and the set of micro base stations is represented by S = {1, 2,..., S}, and each micro base station is equipped with a server;

[0014] NS terminal devices, and the set of terminal devices covered by each micro base station is represented by N s = {n 1,s , n 2,s ,..., n N,s};

[0015] K sub-channels, represented by the set K = {1, 2,..., K}; and

[0016] A macro base station;

[0017] In the micro base station, a terminal device generates a computing request ψ once n,s represented by the quadruple <D n,s , T n,s , C n,s >, where D n,s represents the size of the task data requested by the terminal device for computing, T n,s represents the maximum tolerable delay to complete the task, and C n,s represents the number of CPU cycles required to complete the request of the terminal device.

[0018] Further, the communication mode includes:

[0019] The terminal device transmits the computing task of the terminal device to the associated base station through NOMA technology;

[0020] The relevant base station transmits the computing tasks of the terminal device to the neighboring base station adjacent to the relevant base station through the X2 link; and

[0021] The relevant base station transmits the computing tasks of the terminal device to the macro base station through the wired backhaul link.

[0022] Furthermore, the task offloading methods include local computing, offloading to the relevant base station for computing, neighboring base station computing, and offloading to the macro base station for computing;

[0023] Use binary variables to represent the task offloading methods. The binary variable indicates whether the computing task of the terminal device is offloaded. indicates that the terminal device offloads the computing task to the relevant base station, otherwise processes it locally; the binary variable indicates whether the computing task of the terminal device is computed in the relevant base station. indicates that the computing task of the terminal device is computed in the relevant base station, otherwise offloads it to the neighboring base station for computing; the binary variable indicates whether the computing task of the terminal device is transmitted to the neighboring base station for computing. indicates that the computing task of the mobile terminal device is transmitted to the neighboring base station for computing through the relevant base station, otherwise processes it in the relevant base station; the binary variable indicates whether the computing task of the terminal device is offloaded by the relevant base station to the macro base station for computing. indicates that the computing task of the terminal device is transmitted to the macro base station for computing through the relevant base station, otherwise computes it in the relevant base station.

[0024] Furthermore, the delay calculation methods include the total delay of local computing, the total delay of offloading to the relevant base station for computing, the total delay of offloading to the neighboring base station for computing, and the total delay of offloading to the macro base station for computing;

[0025] The total delay of offloading to the relevant base station for computing includes: the transmission delay of the computing task from the terminal device to the relevant base station and the computing delay of the relevant base station; the total delay of offloading to the neighboring base station for computing includes: the transmission delay of the computing task from the terminal device to the relevant base station, the transmission delay of the computing task from the relevant base station to the neighboring base station, and the computing delay of the neighboring base station. The total delay of offloading to the macro base station for computing includes: the transmission delay of the computing task from the terminal device to the relevant base station, the transmission delay of the computing task from the relevant base station to the macro base station, and the computing delay of the macro base station;

[0026] The energy consumption calculation method includes the energy consumption of local computing and the energy consumption generated by offloading to the relevant base station for computing.

[0027] Further, determine the weighted sum of the total delay and energy consumption for the terminal device to complete the computing task according to different task offloading methods, and minimize the weighted sum as the final objective function;

[0028] Determine the constraint conditions based on the offloading decision of the terminal device, the computing resource constraints of the relevant base station, the device energy consumption constraints, the number of terminal devices connected to the sub-channel constraints, and the power constraints;

[0029] Establish an objective optimization problem based on the objective function and the constraint conditions.

[0030] Further, the total delay for the terminal device to complete the computing task is:

[0031]

[0032] Among them, is the delay for local processing of the terminal device's computing task, is the delay for the relevant base station associated with the terminal device to process the terminal device's task, is the delay for the neighboring base station to process the terminal device's task, is the delay for the macro base station to process the terminal device's task;

[0033] The total energy consumption for the terminal device to complete the computing task is:

[0034]

[0035] Among them, is the energy consumption for local processing of the terminal device's computing task, is the energy consumption for the terminal device to transmit the computing task to the relevant base station.

[0036] Further, the objective optimization problem is:

[0037]

[0038]

[0039]

[0040]

[0041]

[0042] Among them, λ is the weight of the computing delay and energy consumption, represents the computing delay under different processing methods of the terminal device's computing task, represents the device energy consumption under different processing methods of the terminal device's computing task, T n,s represents the maximum tolerable delay of the terminal device's computing task, E maxRepresents the maximum energy consumption of the terminal device, Represents the maximum transmission power of the user equipment, Indicates whether the micro base station s occupies the sub-channel k. M represents the maximum number of micro base stations connected to the sub-channel k, S represents the number of micro base stations, and K represents the number of sub-channels.

[0043] Furthermore, the transmission time for the computing task of the terminal device to be transmitted from the terminal device to the relevant base station is:

[0044]

[0045]

[0046]

[0047] Wherein, Is the signal-to-interference-plus-noise ratio of the terminal device n in the micro base station s on the sub-channel k, Is the transmission power of the terminal device n in the micro base station s on the sub-channel k, Is the channel gain of the terminal device n in the micro base station s on the sub-channel k, Is the interference between terminal devices in the micro base station, Is the interference between micro base stations, σ 2 Is the power of Gaussian white noise, and B is the bandwidth of the uplink system, Represents the transmission time for the computing task of the terminal device to be transmitted from the terminal device to the relevant base station, D n,s Represents the size of the task data requested by the user for computing.

[0048] Furthermore, the energy consumption of local computing is:

[0049]

[0050] Wherein, the value of k is determined by the chip structure of the mobile device. Generally, κ = 1×10 -27 , Is the CPU frequency of the terminal device n in the micro base station s, Is the total delay of local computing;

[0051] The energy consumption generated by offloading to the relevant base station for computing is:

[0052]

[0053] Wherein, Is the transmission power of the terminal device n in the micro base station s on the sub-channel k, Represents the transmission time for the computing task of the terminal device to be transmitted from the terminal device to the relevant base station.

[0054] The present invention determines the communication method, offloading method, delay calculation method, and energy consumption calculation method through a ultra-dense mobile edge computing network system model, and calculates the total delay and total energy consumption of the terminal device to complete the computing task according to the communication method, offloading method, delay calculation method, and energy consumption calculation method. The optimal offloading decision is calculated through the DDPG algorithm, and the computing task of the terminal device is offloaded to the appropriate relevant base station for calculation, which can ensure that the final decision is optimal or close to optimal. When the terminal device transmits the computing task to the relevant base station, a new communication method, namely NOMA, is introduced. When multiple terminal devices transmit the computing task to the relevant base station, they can transmit simultaneously without waiting, greatly reducing the transmission delay of the terminal device, thereby reducing the overall computing delay. At the same time, the cooperation between relevant base stations is considered to balance the load of relevant base stations and avoid waste of computing resources. Brief Description of the Drawings

[0055] Figure 1 is the flowchart of the method of the present invention.

[0056] Figure 2 is the diagram of the ultra-dense mobile edge computing network system model. Detailed Embodiment

[0057] The following further describes the present invention in detail with reference to the drawings and embodiments.

[0058] The method for offloading tasks of a terminal device in an ultra-dense mobile edge computing environment provided by the present invention specifically includes the following steps:

[0059] As Figure 1 shown, the method for offloading tasks of a terminal device in an ultra-dense mobile edge computing environment provided by the present invention includes the following steps: First, construct an ultra-dense mobile edge computing network system model, then determine the communication method, task computing method, and energy consumption calculation method based on the ultra-dense mobile edge computing network, and then determine the optimization objective and constraint conditions according to the communication method, task computing method, and energy consumption calculation method.

[0060] S1. Construct an ultra-dense mobile edge computing network system model.

[0061] As Figure 2 shown, the ultra-dense mobile edge computing network system model includes S micro base stations, that is, edge nodes, and a macro base station. The set of micro base stations is represented by S = {1, 2,..., S}, and each micro base station is equipped with an edge server, enabling the micro base station to have computing capabilities, and all MEC servers can communicate with each other through wired channels; the set of terminal devices covered by each micro base station is represented by is represented; there are K sub-channels in the network, represented by the set K = {1, 2,... K}; in the small cell s, the terminal device n generates a computing request ψ at a time n,s , represented by the triple <D n,s , T n,s , C n,s >.

[0062] Among them, D n,s represents the size of the task data volume requested by the terminal device for computing, T n,s represents the maximum tolerable delay to complete the computing task of the terminal device, C n,s represents the number of CPU cycles required to complete the request of the terminal device, M represents the maximum number of small cells connected to the sub-channel k, S represents the number of small cells, and K represents the number of sub-channels.

[0063] S2. Determine the communication mode, task offloading mode, energy consumption calculation mode, and delay calculation mode based on the ultra-dense mobile edge computing network system model. Calculate the total delay for the terminal device to complete the computing task based on the communication mode, task offloading mode, and delay calculation mode. Calculate the total energy consumption for the terminal device to complete the computing task based on the task offloading mode, delay calculation mode, and energy consumption calculation mode.

[0064] The communication modes include: the computing task on the mobile terminal device is transmitted to the associated relevant base station through the NOMA communication technology and processed by the MEC server equipped with the base station; when the associated relevant base station does not have enough computing resources, it is transmitted to other associated relevant base stations through the X2 link; and the computing task is transmitted to the macro base station (i.e., the cloud center) through the wired backhaul link and processed by the MEC server equipped with the macro base station. Since the size of the output data is much smaller than the input data, the present invention ignores the transmission delay of sending the computing result back to the mobile terminal device.

[0065] The offloading modes include: the present invention uses binary offloading decision variables to represent whether the computing task of the terminal device is offloaded, represents that the terminal device offloads the computing task to the relevant base station s, otherwise processes it locally; the binary variable represents whether the computing task of the terminal device is computed in its associated relevant base station s, represents that the computing task of the terminal device is computed in its associated relevant base station s, otherwise offloads it to the neighboring base station s' for computing; the binary variable represents whether the computing task of the terminal device is transmitted to the neighboring base station s' for computing, represents that the computing task of the mobile terminal device is transmitted through the relevant base station s to the neighboring base station s' for computing, otherwise processes it in the relevant base station s; the binary variable ​Indicates whether the computing task of the terminal device is offloaded by the relevant base station s to the macro base station C for computing. Indicates that the computing task of the terminal device is transmitted by the relevant base station s to the macro base station C for computing, otherwise it is computed at the relevant base station s.

[0066] Different offloading methods and different communication methods correspond to different delay calculation methods and energy consumption calculation methods. Based on the communication method, task offloading method, and delay calculation method, calculate the total delay for the terminal device to complete the computing task. Based on the task offloading method, delay calculation method, and energy consumption calculation method, calculate the total energy consumption for the terminal device to complete the computing task.

[0067] A terminal device with sufficient power can process the computing task on the local device, and the local computing delay of the computing task and local energy consumption can be expressed as:

[0068]

[0069]

[0070] Where is the CPU frequency of the terminal device n in the micro base station s. The value of κ is determined by the chip structure of the mobile device. Generally, κ = 1×10 -27 .

[0071] When the computing task of the terminal device is transmitted from the terminal device to the relevant base station, on sub-channel k, for the set of terminal devices N served by the micro base station s s , assuming that its channel gain follows in order. Therefore, the formula for calculating the signal-to-interference-plus-noise ratio (SINR) of the terminal device n in the micro base station s on sub-channel k is as follows:

[0072]

[0073]

[0074]

[0075] Where is the interference between terminal devices in the micro base station, indicates whether the i-th terminal device in the micro base station s performs offloading, indicates that the terminal device offloads the computing task to the relevant base station S, otherwise it is processed locally, is the transmit power of the terminal device n in the micro base station s on sub-channel k, is the channel gain of the terminal device n in the micro base station s on sub-channel k, It is the interference between micro base stations. r is the micro base station occupying sub-channel k, and the binary variable indicates whether the micro base station r occupies sub-channel k. It indicates that the micro base station r occupies sub-channel k, otherwise it indicates that the micro base station r does not occupy sub-channel k. N r is the total number of terminal devices within r micro base stations. indicates whether the i-th terminal device in the micro base station r performs offloading. It indicates that the terminal device offloads the computing task to the relevant base station r, otherwise it processes locally. is the transmit power of the terminal device n in the micro base station r on sub-channel k. is the signal-to-interference-plus-noise ratio of the terminal device n in the micro base station s on sub-channel k. σ 2 is the power of Gaussian white noise.

[0076] According to Shannon's formula, the data transmission rate of the terminal device n on sub-channel k is calculated as:

[0077]

[0078] where B is the bandwidth of the uplink system.

[0079] Therefore, for the mobile terminal device task ψ n,s the transmission time for offloading to the relevant base station s on sub-channel k is:

[0080]

[0081] where represents the transmission delay for offloading the computing task from the terminal device n to the relevant base station s.

[0082] When the relevant base station s associated with the terminal device has sufficient computing resources, the relevant base station must execute the computing task ψ of the terminal device n,s , so the formula for calculating the total delay of processing the terminal device task at the relevant base station s is as follows:

[0083]

[0084]

[0085] where is the computing delay calculated by the relevant base station, is the computing resources allocated by the relevant base station s, is the total delay for offloading the computing task to the relevant base station for calculation.

[0086] When the associated base station s of the terminal device does not have sufficient computing resources to meet the computing requirements of the terminal device, the computing task of the terminal device is transmitted from the associated base station s to another neighboring base station s' that can meet the requirements of the terminal device through the X2 link. The associated base station s forwards the computing task of the terminal device to another associated base station (i.e., the neighboring base station s') for processing, and its data transmission time is:

[0087]

[0088] Among them, is the transmission delay for the computing task to be transmitted from the associated base station s to the neighboring base station s', is the link capacity between the associated base station s and the neighboring base station s'.

[0089] Therefore, the formula for the total delay of processing the computing task at the neighboring base station s' is as follows:

[0090]

[0091]

[0092] Among them, is the computing delay of the neighboring base station s', is the computing resources allocated by the neighboring base station s', is the total delay for the computing task to be offloaded to the neighboring base station s' for computing.

[0093] When the associated base station cannot provide computing services for the terminal device because it does not have sufficient computing resources to meet the computing requirements of the terminal device, the computing task of the terminal device is transmitted from the associated base station S to the macro base station C. The associated base station s transmits the computing task of the terminal device to the macro base station through the wired backhaul link. The formula for the total delay of processing the computing task at the macro base station C is as follows:

[0094]

[0095]

[0096] Among them, is the transmission delay for the computing task to be transmitted from the associated base station s to the macro base station C, is the link capacity between the associated base station S and the macro base station C, is the total delay for the computing task to be offloaded to the macro base station C for computing, is the computing resources allocated by the macro base station C.

[0097] In the ways of offloading the computing tasks of the terminal device to the relevant base station for computing, offloading to the neighboring base station of the associated relevant base station for computing, and offloading to the macro base station for computing, the energy consumption of the terminal device only occurs when the terminal device transmits the computing task to the relevant base station s. Therefore, the transmission energy consumption of the terminal device can be expressed as:

[0098]

[0099] Therefore, the total offloading delay and total energy consumption for the terminal device to complete the computing task are:

[0100]

[0101] Therefore, according to different task offloading methods, the weighted sum of the total time delay and energy consumption for the terminal device to complete the computing task is determined, and the total cost of processing the computing task of the mobile terminal device is:

[0102]

[0103] Among them, λ and 1 - λ represent the preference parameters (i.e., weights) of the terminal device for computing time delay and energy consumption when completing its computing task, and the total cost of the system is the weighted sum of the computing time delay and energy consumption.

[0104] S3. Determine the target optimization problem according to the total time delay and total energy consumption for the terminal device to complete the computing task.

[0105] Among them, in order to ensure that the computing task ψ of the terminal device n,s is only executed at one location, that is, computing is performed on the mobile terminal device, the MEC server, or the macro base station. In the present invention, the following constraints are imposed:

[0106]

[0107]

[0108] Construct the constraint conditions of the target problem: the offloading decision of the terminal device, the computing resource constraint of the relevant base station, the device energy consumption constraint, the number of terminal devices connected to the sub-channel constraint, the power constraint, and the computing task processing time delay constraint of the terminal device. Establish the computing task offloading optimization problem of the mobile terminal device according to minimizing the system cost and the constraint conditions.

[0109] The main objective of the present invention is to make full use of the computing resources of the relevant base station, design a cooperative offloading scheme, and minimize the total cost of the system on the premise of meeting the maximum tolerable time delay of the mobile terminal device. The optimization problem of the present invention can be described as:

[0110]

[0111]

[0112]

[0113]

[0114]

[0115]

[0116]

[0117]

[0118] Among them, λ is the weight for calculating the delay and energy consumption, represents the computing delay under different processing methods of the computing tasks of the terminal device, represents the device energy consumption under different processing methods of the computing tasks of the terminal device, T n,s represents the maximum tolerable delay of the computing tasks of the terminal device, E max represents the maximum energy consumption of the terminal device, represents the maximum transmit power of the terminal device n in the micro base station s on the subchannel k, represents whether the micro base station s occupies the subchannel k, M represents the maximum number of micro base stations connected to the subchannel k, S represents the number of micro base stations, and K represents the number of subchannels.

[0119] Among them, the CORS objective function is to minimize the total system cost (i.e., the minimization of the weighted sum of the user task computing delay and device energy consumption), C1 - C8 are the constraint conditions. The C1 constraint ensures that the delay for completing the computing tasks of the mobile terminal device does not exceed its maximum tolerable delay. The C2 constraint ensures that the total energy consumption of the mobile terminal device for data transmission and local computing does not exceed the maximum energy limit of the device. The C3 constraint ensures that the power of the mobile terminal device does not exceed its maximum power limit. The C4 - C6 constraints ensure that the subchannel allocation follows the binary decision variable, and each subchannel can only be reused by M related base stations. The C7 - C8 constraints ensure that the computing tasks of the mobile terminal device can only be computed at the terminal device, related base stations, neighboring base stations, or macro base stations.

[0120] S4. Use a deep reinforcement learning algorithm based on the Deep Deterministic Policy Gradient (DDPG) algorithm to solve the objective optimization problem and obtain the task offloading strategy of the terminal device.

[0121] Since the joint optimization problem is non-convex and the optimal solution cannot be obtained in polynomial time, to address large-scale Internet of Things devices, the present invention proposes a computational offloading method based on deep reinforcement learning to solve the target problem. In the present invention, a Markov Decision Process (MDP) model will be constructed in combination with the actual mobile edge computing scenario. The state of the relevant base station in the current time slot is only related to the state (computing resources of the relevant base station) and action at the previous moment (whether the computing task of the terminal device is offloaded to the relevant base station). The present invention is based on the DDPG algorithm to solve the computing task offloading problem of mobile terminal devices.

[0122] To comprehensively consider the characteristics between the computing tasks of mobile terminal devices and the relevant base station servers in the ultra-dense mobile edge computing network, the state space can be expressed as: S t =(D n,s , T n,s , C n,s , φ, U 1 , U 2 ,..., U 2+s ), where φ represents the remaining available computing resources of the edge server, φ=(φ 1 , φ 2 ,…, φ i ,…, φ s ), 2 + S are 2 + S computing devices, namely: S micro base stations, one terminal device, and one macro base station, and U i represents the CPU utilization rate of the i-th computing device in time slot t.

[0123] To offload the computing tasks of mobile devices to appropriate computing devices, in deep reinforcement learning, the action space is set to correspond to the set of available computing devices, and (0 / 1) i j is used to represent whether the computing task of the i-th terminal device is offloaded to the j-th device (micro base station or macro base station).

[0124] The optimization goal of the present invention is to minimize the cost of the system. In each step, after the agent executes an action A, it will obtain a corresponding reward value R in the state S corresponding to this action. Generally speaking, the reward function is related to the objective function. The goal of deep learning is to obtain the maximum reward return. Therefore, the reward value is negatively correlated with the system cost, that is

[0125] The Critic network in DDPG obtains the offloading strategy according to the current network state, and updates the corresponding reward by executing the action according to the obtained offloading strategy, and enters the next state; the current state s t 、next state st+1 、Action a t and reward r t Store it in the experience pool D, and randomly sample a portion (s t ,a t ,r t ,s t+1 ) as training data. Based on the current state, next state, action and reward, the current Q value and target Q value are calculated through the Critic network to minimize the loss function. The Critic current network uses the policy gradient to update the policy of the current network, and then updates the parameters of the target network to output the optimal unloading policy.

[0126] The contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0127] The above is a description of the specific implementation of the present invention in combination with the accompanying drawings and technical solutions. The above embodiments are only preferred embodiments for fully illustrating the present invention. The protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by technicians in the technical field on the basis of the present invention are all within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for offloading tasks of terminal devices in an ultra-dense mobile edge computing environment, characterized in that, it includes the following steps: S1. Construct an ultra-dense mobile edge computing network system model; S2. Determine the communication mode, task offloading mode, energy consumption calculation mode, and delay calculation mode based on the ultra-dense mobile edge computing network system model. Calculate the total delay for the terminal device to complete the computing task based on the communication mode, task offloading mode, and delay calculation mode, and calculate the total energy consumption for the terminal device to complete the computing task based on the task offloading mode, delay calculation mode, and energy consumption calculation mode; S3. Determine the target optimization problem according to the total delay and total energy consumption of the terminal device to complete the computing task; S4. Use the deep reinforcement learning algorithm based on DDPG to solve the target optimization problem to obtain the task offloading strategy for the terminal device.

2. The method for offloading tasks of terminal devices in the ultra-dense mobile edge computing environment according to claim 1, characterized in that, the mobile edge computing network system model includes: S micro base stations, and the set of micro base stations is represented by S = {1, 2,..., S}, and each micro base station is equipped with a server; NS terminal devices, and the set of terminal devices covered by each micro base station is represented by Ns = {n 1,s , n 2,s ,..., n N,s}; K sub-channels, represented by the set K = {1, 2,..., K}; and one macro base station; In a small cell base station, a terminal device generates a computing request ψ one by one. n,s It is represented by a quadruple <D n,s , T n,s , C n,s >, where D n,s represents the size of the task data requested by the terminal device for computing, T n,s represents the maximum tolerable delay for completing the task, and C n,s represents the number of CPU cycles required to complete the request of the terminal device.

3. The method for offloading tasks of terminal devices in the ultra-dense mobile edge computing environment according to claim 2, characterized in that, the communication mode includes: The terminal device transmits the computing task of the terminal device to the associated relevant base station through NOMA technology; The relevant base station transmits the computing task of the terminal device to the adjacent neighboring base station through the X2 link; and The relevant base station transmits the computing task of the terminal device to the macro base station through the wired backhaul link.

4. The method for offloading tasks of terminal devices in the ultra-dense mobile edge computing environment according to claim 3, characterized in that, The task offloading mode includes local computing, offloading to the relevant base station for computing, neighboring base station computing, and offloading to the macro base station for computing; Use binary variables to represent the task offloading method. The binary variable indicates whether the computing task of the terminal device is offloaded. It means that the terminal device offloads the computing task to the relevant base station, otherwise processes it locally. Binary variable Indicates whether the computing task of the terminal device is computed in the relevant base station, Indicates that the computing task of the terminal device is computed in the relevant base station, otherwise it is offloaded to a neighboring base station for computing; Binary variable Indicates whether the computing task of the terminal device is transmitted to a neighboring base station for computing, Indicates that the computing task of the mobile terminal device is transmitted to a neighboring base station for computing through the relevant base station, otherwise it is processed in the relevant base station; Binary variable Indicates whether the computing task of the terminal device is offloaded to the macro base station for computing by the relevant base station Indicates that the computing task of the terminal device is transmitted to the macro base station for computing through the relevant base station, otherwise it is computed at the relevant base station 5. The method for offloading tasks of terminal devices in the ultra-dense mobile edge computing environment according to claim 4, characterized in that, The delay calculation mode includes the total delay of local computing, the total delay of offloading to the relevant base station for computing, the total delay of offloading to the neighboring base station for computing, and the total delay of offloading to the macro base station for computing; The total delay of offloading to the relevant base station for computing includes: the transmission delay of the computing task from the terminal device to the relevant base station and the computing delay of the relevant base station; the total delay of offloading to the neighboring base station for computing includes: the transmission delay of the computing task from the terminal device to the relevant base station, the transmission delay of the computing task from the relevant base station to the neighboring base station, and the computing delay of the neighboring base station. The total delay of offloading to the macro base station for computing includes: the transmission delay of the computing task from the terminal device to the relevant base station, the transmission delay of the computing task from the relevant base station to the macro base station, and the computing delay of the macro base station; The energy consumption calculation mode includes the energy consumption of local computing and the energy consumption generated by offloading to the relevant base station for computing.

6. The method for offloading tasks of terminal devices in the ultra-dense mobile edge computing environment according to claim 5, characterized in that, Determine the weighted sum of the total delay and energy consumption of the terminal device to complete the computing task according to different task offloading methods, and minimize the weighted sum as the final objective function; Determine the constraint conditions based on the offloading decision of the terminal device, the computing resource constraints of the relevant base station, the device energy consumption constraints, the number of terminal devices connected to the sub-channel constraints, and the power constraints; Establish an objective optimization problem based on the objective function and the constraint conditions.

7. The method for offloading the task of the terminal device in the ultra-dense mobile edge computing environment according to claim 6, characterized in that, The total delay for the terminal device to complete the computing task is: wherein, is the latency of the local processing terminal device for computing tasks, is the latency of the associated base station related to the terminal device for processing the terminal device tasks, is the latency of the neighboring base station for processing the terminal device tasks, is the latency of the macro base station for processing the terminal device tasks; The total energy consumption for the terminal device to complete the computing task is: Among them, is the energy consumption of the local processing terminal device for computing tasks, is the energy consumption of the terminal device for transmitting the computing task to the relevant base station.

8. The method for offloading the task of the terminal device in the ultra-dense mobile edge computing environment according to claim 7, characterized in that, The objective optimization problem is: where λ is the weight for calculating the latency and energy consumption, represents the computing latency of the terminal device under different processing methods of the computing task, represents the device energy consumption of the terminal device under different processing methods of the computing task, T n,s represents the maximum tolerable latency of the terminal device's computing task, E max represents the maximum energy consumption of the terminal device, represents the maximum transmission power of the user equipment, indicates whether the micro base station s occupies the subchannel k. M represents the maximum number of micro base stations connected to the subchannel k, S represents the number of micro base stations, and K represents the number of subchannels.

9. The method for offloading the task of the terminal device in the ultra-dense mobile edge computing environment according to claim 5, characterized in that, The transmission time for the computing task of the terminal device to be transmitted from the terminal device to the relevant base station is: Among them, is the signal-to-interference-plus-noise ratio of the terminal device n on the sub-channel k in the small cell s, is the transmit power of the terminal device n on the sub-channel k in the small cell s, is the channel gain of the terminal device n on the sub-channel k in the small cell s, is the interference between terminal devices in the small cell, is the interference between small cells, σ 2 is the power of the additive white Gaussian noise, B is the bandwidth of the uplink system, represents the transmission time for the computing task of the terminal device to be transmitted from the terminal device to the relevant base station, D n,s represents the size of the task data requested by the user for computing.

10. The method for offloading the task of the terminal device in the ultra-dense mobile edge computing environment according to claim 5, characterized in that, The energy consumption of local computing is: Among them, the value of k is determined by the chip structure of the mobile device. Generally, κ = 1×10 -27 , is the CPU frequency of the terminal device n in the micro base station s, and is the total delay of local computing; The energy consumption generated by offloading to the relevant base station for computing is: Among them, is the transmission power of the terminal device n in the micro base station s on the sub-channel k, represents the transmission time for the computing task of the terminal device to be transmitted from the terminal device to the relevant base station.

Images