A fine-grained task offloading and resource allocation method for MEC networks

Abstract

The present invention relates to a fine-grained task offloading and resource allocation method for MEC network, including a model training phase and a decision reasoning phase, including: (1) creating an Internet of Things system; (2) deploying a fine-grained task offloading and resource allocation algorithm model based on deep reinforcement learning on an SDN controller; (3) collecting training data; (4) training the fine-grained task offloading and resource allocation algorithm model; (5) redeploying the trained fine-grained task offloading and resource allocation algorithm model to the SDN controller for implementing decision reasoning; (7) the trained SDN controller makes a computing offloading decision, a MEC computing resource allocation decision and a bandwidth allocation decision, and sends them to an intelligent Internet of Things device and an MEC server in the form of a control instruction. The present invention takes into account the dynamically changing network environment and random system environment in the Internet of Things scenario, and the present invention effectively reduces the long-term task processing delay of the system.

Description

Technical Field

[0001] The present invention relates to a fine-grained task offloading and resource allocation method for an MEC network, and belongs to the technical field of 5G and artificial intelligence development. Background Art

[0002] The development of 5G and artificial intelligence has promoted the promotion and application of the Internet of Things in various scenarios, such as the Industrial Internet of Things and the Internet of Vehicles. Along with these new scenarios, a large number of intelligent applications have also emerged, such as intelligent manufacturing and intelligent resource scheduling. These computationally intensive applications usually require a large amount of computing resources, but due to the limited computing power of IoT devices, it is difficult to implement these calculations. To address this problem, offloading computing tasks to remote cloud centers and using their powerful computing resources for computing processing is a good solution. However, since the cloud server is far away from the user, this method of offloading computing to the cloud increases the end-to-end processing latency of the task, which is difficult to meet the low latency requirements of intelligent applications.

[0003] Multi-Access Edge Computing (MEC) is an effective computing model that provides users with nearby services by deploying servers at the edge of the network close to IoT devices. Compared with cloud computing, it can greatly reduce the communication delay in task processing delay. Although edge servers can provide computing services for IoT devices, a large amount of data exchange will also bring communication overhead to the system, and the transmission delay of computing tasks is affected by dynamic network communication conditions. In addition, the computing resources of edge servers are limited, and the increase in the number of IoT devices makes resource competition on the edge server side inevitable. Therefore, in order to minimize the long-term task processing delay of the system, efficiently utilize edge servers and local computing resources, and accurately adapt to the dynamically changing network environment, designing effective dynamic offloading and resource allocation strategies is an important way to achieve reasonable task scheduling.

[0004] In actual IoT scenarios, channel conditions are subject to long-term dynamic changes. In order to cope with the dynamic characteristics of the environment and achieve long-term task scheduling process optimization, deep reinforcement learning (DRL) is a good solution. Deep reinforcement learning combines deep learning with reinforcement learning to solve huge state and action spaces and achieve strategy learning.

[0005] Some of the existing task offloading and resource allocation methods for the Internet of Things use static scheduling solutions to implement one-time task scheduling for the entire system, which cannot guarantee the optimal long-term benefits of dynamic systems. Some solutions that use dynamic scheduling can achieve long-term task scheduling process optimization, but most of them perform overall scheduling on a computing task. However, a complex computing task with a large amount of data cannot be calculated and completed within the coherent time. The technical solution of overall task scheduling is difficult to achieve the optimal long-term benefits of the system. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a fine-grained task offloading and resource allocation method based on deep reinforcement learning for MEC networks. The method is based on a multi-access edge computing architecture and is used to design intelligent computing task offloading and resource allocation strategies for the Internet of Things system to improve the long-term overall benefits of the system. The method uses an SDN centralized controller to train and reason about the decision model, and performs multi-time slot fine-grained scheduling of intelligent computing tasks with large amounts of data by collecting dynamic system status information, such as channel conditions and random task arrival conditions. This method can make the task offloading and resource allocation decisions of the Internet of Things system better adapt to the dynamically changing environment, and effectively reduce the long-term task processing delay of the system.

[0007] Terminology explanation:

[0008] 1. Multi-Access Edge Computing (MEC): An evolution of cloud computing, which forms an edge cloud by sinking computing resources to the edge of the network and providing computing services to multiple users, which can effectively reduce service latency.

[0009] 2. Deep Deterministic Policy Gradient (DDPG) algorithm: A deep reinforcement learning method that uses an Actor-Critic dual network structure to solve continuous action space problems.

[0010] 3. Software Defined Networking (SDN): A network management method that supports dynamic network configuration and improves network performance and management efficiency. SDN controllers can collect dynamic system information and manage and control network resources.

[0011] The technical solution of the present invention is:

[0012] A fine-grained task offloading and resource allocation method for MEC networks includes a model training phase and a decision reasoning phase. The specific steps include:

[0013] A. Model training phase, including:

[0014] (1) Create an IoT system: The IoT system includes an MEC server, an SDN controller, and smart IoT devices. The MEC server provides computing services for smart IoT devices, and the SDN controller is responsible for the training and decision reasoning of task scheduling and resource allocation algorithm models.

[0015] (2) Deploy a fine-grained task offloading and resource allocation algorithm model based on deep reinforcement learning on the SDN controller and initialize the fine-grained task offloading and resource allocation algorithm model;

[0016] (3) The SDN controller acts as an intelligent agent and interacts with the IoT system environment to collect training data: decision generation and control command issuance are performed in a time slot manner, where the time slot length is the coherent time length. The interaction information between the intelligent agent and the system environment in each time slot is defined as a data tuple and stored in the experience pool;

[0017] (4) The stochastic gradient descent algorithm is used to train the fine-grained task offloading and resource allocation algorithm model. The training goal is to minimize the total task execution delay of the system.

[0018] (5) Repeat steps (3) and (4) until the fine-grained task offloading and resource allocation algorithm model converges, the training process of the fine-grained task offloading and resource allocation algorithm model is completed, and the decision reasoning stage is entered;

[0019] B. Decision-making reasoning stage, including:

[0020] (6) Redeploy the trained fine-grained task offloading and resource allocation algorithm model to the SDN controller to implement decision reasoning;

[0021] (7) At the beginning of the time slot, the SDN controller collects the IoT system status information, and the trained SDN controller makes computing offloading decisions, MEC computing resource allocation decisions, and bandwidth allocation decisions, and sends them to the smart IoT devices and MEC servers in the form of control instructions;

[0022] (8) During the remaining time of this time slot, the smart IoT devices and MEC servers execute the received control instructions and complete the offload calculation or local calculation of the smart computing task;

[0023] (9) Repeat steps (7) and (8) until the entire process is completed.

[0024] Preferably, according to the present invention, the wireless communication process of the Internet of Things system is:

[0025] The MEC server and smart IoT devices use orthogonal frequency division to transmit data, and each smart IoT device uses a different frequency band;

[0026] Define the bandwidth ratio allocated by the IoT system to all smart IoT devices as b(t)=(b1(t),…,b n (t),…,b N (t)), where b n (t) is the channel bandwidth ratio allocated by the IoT system to the smart IoT device n in time slot t; according to the Shannon formula, the maximum transmission rate of data from the smart IoT device n to the MEC server is R n (t) is defined as shown in formula (I):

[0027]

[0028] In formula (I), B is the total communication bandwidth of the IoT system, p n is the transmission power of smart IoT device n, g n (t) is the wireless channel gain of smart IoT device n in time slot t, and N0 is the Gaussian noise power spectral density.

[0029] Preferably, according to the present invention, the fine-grained task offloading and resource allocation algorithm model is a dual-network model, including an action network Actor network and an evaluation network Critic network;

[0030] Both the Actor network and the Critic network adopt a fully connected neural network structure, including an input layer, multiple hidden layers and an output layer; define s(t) as the state information of the IoT system at time slot t, a(t) as the decision action executed at time slot t, the input of the Actor network is the state information s(t) of the IoT system, and the output is the decision action a(t) to be executed. The neurons in the output layer of the Actor network are divided into two parts, one part uses the softmax activation function to output continuous normalized resource allocation decisions, and the other part uses the sigmoid activation function to output the probability of offloading decisions;

[0031] Among them, the Actor network follows the strategy a(t)=π(s(t)|θ μ ) outputs a deterministic action, and the Critic network outputs an estimated Q value Q(s(t),a(t)|θ Q ) to evaluate the quality of this deterministic action, where θ μ and θ Q They are the network parameters of the Actor network and the Critic network respectively.

[0032] According to the preferred embodiment of the present invention, the process of the intelligent agent interacting with the IoT system environment in step (3) and the generated data tuple (s meu (t),s su(t),s l (t), g(t)): The specific implementation process of step (3) is:

[0033] Step 3-1: At the beginning of each time slot, the SDN controller, as an intelligent agent, collects the state information s(t) of the IoT system in time slot t and makes decisions based on s(t). The state space S is described as formula (II):

[0034] S={s(t)|s(t)=(s meu (t),s su (t),s l (t),g(t))} (II)

[0035] Among them, the MEC server includes an execution unit, which is responsible for receiving intelligent tasks for computational offloading and performing computational processing; Represents the amount of data in the execution unit of the MEC server. The smart IoT device includes a scheduling unit, which is responsible for storing the data of the computing tasks being executed; represents the amount of data in the dispatch unit of the smart IoT device, represents the task queue length of the smart IoT device, g(t)=(g1(t),…,g n (t),…,g N (t)) represents the wireless channel gain of the smart IoT device, and the dimension of the state space of the IoT system is 4N;

[0036] Step 3-2: After collecting the state information s(t) of the IoT system at time slot t, the SDN controller will make a decision action a(t) and send a control signal to the MEC server and smart IoT devices; the action space A is described as formula (III):

[0037]

[0038] Among them, x(t), φ(t) and b(t) are the offloading decision, MEC computing resource allocation decision and system bandwidth resource allocation decision respectively. The dimension of the action space of the IoT system is 3N.

[0039] Step 3-3: After selecting and executing the decision action a(t), the IoT system transfers from s(t) to s(t+1), where the state transition probability is expressed as P(s(t+1)|s(t),a(t));

[0040] Step 3-4: After selecting and executing the decision action a(t), the IoT system generates an immediate reward r(t) for this decision action a(t); the opposite of the IoT system optimization target value per time slot is used as the reward function of the IoT system; the reward function r(t) is defined as formula (IV):

[0041]

[0042] Among them, ξ is the penalty parameter for task overflow, α n,m (t) is the task overflow indicator parameter, d n,m (t) represents the delay of the mth task of smart IoT device n in time slot t, that is, if the task is waiting in the queue or performing calculation, then there is d n,m (t) = T s , otherwise d n,m (t) = 0. T s is the time slot length.

[0043] Preferably, in step (4), the optimization goal of the Internet of Things system is to minimize the total delay of task execution of the Internet of Things system; taking minimizing the overall task completion delay of the Internet of Things system as the objective function, a joint computing offloading and resource allocation decision algorithm is designed, and the established optimization problem is as follows:

[0044]

[0045] st

[0046] C1:x=(x1,…,x N )

[0047] C2:

[0048] C3:b=(b1,…,b N )

[0049] C4:

[0050] C5:

[0051] C6:

[0052] C7:

[0053] C8:

[0054] Among them, d n,m (t) represents the delay of the mth task of smart IoT device n in time slot t, that is, if the task is waiting in the queue or performing calculation, then there is d n,m (t) = T s , otherwise d n,m (t) = 0; α n,m is the task overflow indication parameter. When the mth task of smart IoT device n overflows, α n,m =1, otherwise α n,m=0; ξ is the time penalty parameter; C1, C2 and C3 are the offloading decision, MEC server computing resource allocation decision and bandwidth resource allocation decision respectively; constraint C4 means that each smart IoT device can only choose one of the local computing mode or computing offloading mode to process its task in a time slot; constraint C5 means that the proportion of computing resources allocated by the MEC server to smart IoT devices does not exceed 1, constraint C6 means that the proportion of bandwidth resources allocated by the system to smart IoT devices does not exceed 1, constraint C7 means that the computing resources allocated by the MEC server to all smart IoT devices cannot exceed the total computing resources of the MEC server, and constraint C8 means that the bandwidth resources allocated by the system to all smart IoT devices cannot exceed the total bandwidth resources of the system.

[0055] According to the preferred embodiment of the present invention, according to the Bellman equation, the value function Q(s(t), a(t)) is the expected cumulative return of taking action a(t) in state s(t), expressed as formula (VI):

[0056]

[0057] Among them, γ∈[0,1] is the discount factor; therefore, the training objective of the fine-grained task offloading and resource allocation algorithm model is formula (VII):

[0058] Q(s(t),a(t))≈Q(s(t),a(t)|θ Q ) (VII)

[0059] The Critic network uses the stochastic gradient descent method to update parameters, and its loss function L(θ Q ) is represented by formula (VIII):

[0060]

[0061] Among them, Q′(s(t+1),a′(t+1)) is the Q value output by the target Critic network;

[0062] The Actor network is updated using the stochastic gradient descent algorithm, and the gradient of the objective function It is expressed as formula (IX):

[0063]

[0064] Preferably, in step (4), Ornstein-Uhlenbeck noise is added to the deterministic action, and the noise is set to gradually decrease as the training time increases. A soft replacement strategy is used to update the parameters of the target fine-grained task offloading and resource allocation algorithm model. The parameter update process in each training cycle is formula (X):

[0065] θ' t+1 =(1-ε)·θ' t +ε·θ t (X)

[0066] Among them, θ' t is the target fine-grained task offloading and resource allocation algorithm model parameter, θ t Model parameters for estimating fine-grained task offloading and resource allocation algorithms.

[0067] Preferably, according to the present invention, in step (4), the training process of the fine-grained task offloading and resource allocation algorithm model is:

[0068] Initialize the parameters of the fine-grained task offloading and resource allocation algorithm model and other fixed parameters. Before training, collect some data tuples through interaction with the environment and store them in the experience pool. For each training round, extract a small batch of samples from the experience pool, and update the parameters of the fine-grained task offloading and resource allocation algorithm model according to formulas (VIII), (IX) and (X). At the same time, the data in the experience pool is updated as the training process proceeds. Perform K rounds of training on the target fine-grained task offloading and resource allocation algorithm model until the target fine-grained task offloading and resource allocation algorithm model converges to obtain the final target fine-grained task offloading and resource allocation algorithm model.

[0069] Preferably, according to the present invention, in step (7), the fine-grained task scheduling process of the Internet of Things system is:

[0070] First, at the beginning of each time slot, the SDN controller collects the status information of the IoT system. The trained SDN controller makes computing offloading decisions and MEC computing resource allocation decisions and bandwidth resource allocation decisions, and sends control instructions to smart IoT devices and MEC servers.

[0071] The IoT system will then execute the command in the remaining time to complete the offloaded or local computation;

[0072] The tasks generated by smart IoT devices are serial dependent task flows, and the tasks arrive randomly. Different smart IoT devices have different workloads. All unprocessed tasks are waiting in the local task queue, and the first task generated has a higher priority.

[0073] The internal structure of the smart IoT device includes a scheduling unit, a local execution unit, and a data transmission unit. The scheduling unit is responsible for storing the data of the computing task being executed, the local execution unit is responsible for executing the computing task locally, and the data transmission unit is responsible for transmitting the offloaded task data to the MEC server for calculation; the MEC server includes an execution unit, which is responsible for receiving the offloaded intelligent task and performing calculation processing;

[0074] When a task is about to be executed, it is placed in the scheduling unit; thereafter, in each time slot, the task is either forwarded to the local execution unit for processing or offloaded to the MEC server through the data transmission unit; the execution unit of the MEC server receives this data and performs calculation processing in the next time slot;

[0075] Finally, the calculation results after processing are sent back to the smart IoT devices; the set of smart IoT devices is represented as The set of tasks generated by the nth smart IoT device is expressed as M n The number of tasks generated for the entire process.

[0076] Preferably, according to the present invention, the task computing mode of the Internet of Things system includes a local computing mode and an offload computing mode:

[0077] The local computing modes are:

[0078] Define the processor processing frequency of smart IoT device n as The time slot length is T s , the offloading decision of the smart IoT device in time slot t is defined as x(t)=(x1(t),…,x n (t),…,x N (t)), where x n (t) is the offloading decision of smart IoT device n, x n (t) = 0 means the task is executed locally, x n (t) = 1 means that the task is offloaded to the MEC server for execution. In the t time slot, the amount of data calculated locally is It is expressed as formula (XI):

[0079]

[0080] In formula (XI), k n The number of CPU revolutions required for 1 bit of data in the task currently being processed in the smart IoT device n;

[0081] The offload calculation mode is:

[0082] Define the processor processing frequency of the MEC server as f c, define the computing resource allocation ratio of MEC server in time slot t as in represents the proportion of computing resources allocated by the MEC server to smart IoT device n; therefore, the amount of data computed by smart IoT device n at the edge in time slot t It is expressed as formula (XII):

[0083]

[0084] In formula (XII), k n It is the number of CPU revolutions required for 1 bit of data in the task currently being processed in the smart IoT device n.

[0085] Preferably, according to the present invention, in step (7), the SDN controller collects the status information s(t) of the Internet of Things system, including: meu (t),s su (t),s l (t),g(t)): Indicates the amount of data in the execution unit of the MEC server, Indicates the amount of data in the local scheduling unit, represents the task queue length of IoT devices, g(t)=(g1(t),…,g n (t),…,g N (t)) represents the wireless channel gain of the smart IoT device.

[0086] Preferably, according to the present invention, in step (7), the decision information a(t) of the SDN controller includes: x(t)=(x1(t),…,x n (t),…,x N (t)) represents the offloading decision of the smart IoT device, represents the MEC server computing resource allocation decision, b(t)=(b1(t),…,b n (t),…,b N (t)) represents the bandwidth resource allocation decision of the IoT system.

[0087] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a fine-grained task offloading and resource allocation method for an MEC network when executing the computer program.

[0088] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for fine-grained task offloading and resource allocation for an MEC network.

[0089] The beneficial effects of the present invention are:

[0090] 1. The present invention takes into account the dynamically changing network environment and random system environment in the Internet of Things scenario, and designs a fine-grained task offloading and resource allocation method based on a multi-access edge computing structure, which effectively reduces the long-term task processing delay of the system.

[0091] 2. The present invention adopts a fine-grained task scheduling method. For tasks that can be partially offloaded, offloading and resource allocation decisions can be made based on the current system environment status. Compared with the overall scheduling of tasks, the present invention can obtain a more optimized scheduling solution and achieve lower system long-term task processing delay, which is suitable for delay-sensitive applications in the Internet of Things.

[0092] 3. The present invention adopts deep reinforcement learning method to solve the problem. Starting from reality, the optimization problem is modeled as a Markov decision process, and the DDPG model structure is used to effectively solve the huge state and mixed action space, which solves the difficulty of traditional convex optimization methods in solving mixed integer nonlinear programming problems and reduces the complexity of problem solving. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 A schematic diagram of the basic workflow of the fine-grained task offloading and resource allocation method for MEC networks provided by the present invention;

[0094] Figure 2 A schematic diagram of the structure of the Internet of Things system provided by the present invention;

[0095] Figure 3 This is a schematic diagram of the algorithm model structure of the fine-grained task offloading and resource allocation method for MEC networks;

[0096] Figure 4 Schematic diagram of algorithm convergence under different learning rates;

[0097] Figure 5 This is a schematic diagram comparing the total system delay solutions of the IoT system with different numbers of IoT devices;

[0098] Figure 6 This is a schematic diagram comparing the average task processing delay solutions of the IoT system with different numbers of IoT devices. DETAILED DESCRIPTION

[0099] The present invention is described in detail below with reference to the accompanying drawings and examples, but is not limited thereto.

[0100] Example 1

[0101] A fine-grained task offloading and resource allocation method for MEC networks, such as Figure 1As shown, it includes the model training stage and the decision reasoning stage. The specific steps include:

[0102] A. Model training phase, including:

[0103] (1) Create an IoT system: Figure 2 As shown in the figure, it can be applied to industrial IoT scenarios. The IoT system includes MEC servers, SDN controllers, and smart IoT devices. According to the specific application of the IoT system, select IoT devices and MEC servers with reasonable specifications, build wireless data transmission links, and ensure the normal operation of each component. Specifically, the IoT system consists of a MEC server and N smart IoT devices. The set of IoT devices is represented by N IoT devices have limited computing power. Assume that the entire system works in a time slot manner, where the length of a time slot is T. s . MEC servers can receive intelligent computing tasks offloaded by devices and provide edge computing services for them. For MEC servers, NVIDIA A100 with intelligent computing capabilities can be used, and IoT devices can use Raspberry Pi, intelligent detection equipment, etc. as service terminals. The SDN controller is responsible for the training and decision reasoning of task scheduling and resource allocation algorithm models; MEC servers and intelligent IoT devices have limited computing resources, and information communication and data transmission are carried out between various system components through wireless links.

[0104] (2) Deploy a fine-grained task offloading and resource allocation algorithm model based on deep reinforcement learning on the SDN controller and initialize the fine-grained task offloading and resource allocation algorithm model; minimize the long-term task execution delay of the system by jointly optimizing computing offloading, MEC computing resource allocation, and system bandwidth resource allocation.

[0105] (3) The SDN controller acts as an intelligent agent and interacts with the IoT system environment to collect training data: decisions are generated and control instructions are issued in the form of time slots, where the time slot length is the coherence time length. The interaction information between the intelligent agent and the system environment in each time slot is defined as a data tuple and stored in the experience pool. The time slot length uses the coherence time length, and fine-grained task scheduling is performed in units of time slots because the channel state of the system can be considered unchanged within the coherence time. This scheduling method can make scheduling decisions more in line with environmental changes, obtain better system benefits, and reduce task processing delays.

[0106] (4) The stochastic gradient descent algorithm is used to train the fine-grained task offloading and resource allocation algorithm model. The training goal is to minimize the total task execution delay of the system.

[0107] (5) Repeat steps (3) and (4) until the fine-grained task offloading and resource allocation algorithm model converges, the training process of the fine-grained task offloading and resource allocation algorithm model is completed, and the decision reasoning stage is entered;

[0108] B. Decision-making reasoning stage, including:

[0109] (6) Redeploy the trained fine-grained task offloading and resource allocation algorithm model to the SDN controller to implement decision reasoning;

[0110] (7) At the beginning of the time slot, the SDN controller collects the IoT system status information, and the trained SDN controller makes computing offloading decisions, MEC computing resource allocation decisions, and bandwidth allocation decisions, and sends them to the smart IoT devices and MEC servers in the form of control instructions;

[0111] (8) During the remaining time of this time slot, the smart IoT devices and MEC servers execute the received control instructions and complete the offload calculation or local calculation of the smart computing task;

[0112] (9) Repeat steps (7) and (8) until the entire process is completed.

[0113] Example 2

[0114] The fine-grained task offloading and resource allocation method for MEC networks described in Example 1 is different in that:

[0115] The wireless communication process of the IoT system is:

[0116] The MEC server and smart IoT devices use orthogonal frequency division to transmit data, and each smart IoT device uses a different frequency band;

[0117] Define the bandwidth ratio allocated by the IoT system to all smart IoT devices as b(t)=(b1(t),…,b n (t),…,b N (t)), where b n (t) is the channel bandwidth ratio allocated by the IoT system to the smart IoT device n in time slot t; according to the Shannon formula, the maximum transmission rate of data from the smart IoT device n to the MEC server is R n (t) is defined as shown in formula (I):

[0118]

[0119] In formula (I), B is the total communication bandwidth of the IoT system, p n is the transmission power of smart IoT device n, g n(t) is the wireless channel gain of smart IoT device n in time slot t, and N0 is the Gaussian noise power spectral density.

[0120] At time slot t, the offloading decision of IoT devices is expressed as x(t) = (x1(t),…,x n (t),…,x N (t)) represents. For device n, x n (t) = 0 means the task is processed locally, x n (t) = 1 means that the task is offloaded to the MEC server for execution. Therefore, the amount of data that device n transmits during the time slot t is:

[0121] The process of interaction between the agent and the IoT system environment in step (3) and the generated data tuple (s meu (t),s su (t),s l (t), g(t)): The specific implementation process of step (3) is:

[0122] The optimization problem of the Internet of Things system is a mixed integer nonlinear optimization problem with multiple random factors, which is very difficult to solve using traditional optimization methods. Therefore, the problem is modeled as a Markov decision process and solved using deep reinforcement learning methods.

[0123] The five key factors that define MDP are Where S is the state space, A is the action space, P represents the state transition probability, r is the reward function, and γ∈[0,1] is the discount factor.

[0124] Step 3-1: At the beginning of each time slot, the SDN controller, as an intelligent agent, collects the state information s(t) of the IoT system in time slot t and makes decisions based on s(t). The state space S is described as formula (II):

[0125] S={s(t)|s(t)=(s meu (t),s su (t),s i (t),g(t))} (II)

[0126] Among them, the MEC server includes an execution unit, which is responsible for receiving intelligent tasks for computational offloading and performing computational processing; Represents the amount of data in the execution unit of the MEC server. The smart IoT device includes a scheduling unit, which is responsible for storing the data of the computing tasks being executed; represents the amount of data in the dispatch unit of the smart IoT device, represents the task queue length of the smart IoT device, g(t)=(g1(t),…,g n(t),…,g N (t)) represents the wireless channel gain of the smart IoT device, and the dimension of the state space of the IoT system is 4N;

[0127] Step 3-2: After collecting the state information s(t) of the IoT system at time slot t, the SDN controller will make a decision action a(t) and send a control signal to the MEC server and smart IoT devices; the action space A is described as formula (III):

[0128]

[0129] Among them, x(t), φ(t) and b(t) are the offloading decision, MEC computing resource allocation decision and system bandwidth resource allocation decision respectively. The dimension of the action space of the IoT system is 3N.

[0130] Step 3-3: After selecting and executing the decision action a(t), the IoT system transfers from s(t) to s(t+1), where the state transition probability is expressed as P(s(t+1)|s(t),a(t)); the state transition probability is determined by the environment. Due to the randomness and dynamics of the environment, the state transition probability is difficult to obtain and needs to be explored through algorithms;

[0131] Step 3-4: After selecting and executing the decision action a(t), the IoT system generates an immediate reward r(t) for this decision action a(t); the setting of the reward function is related to the optimization goal of the system. Considering that the optimization goal of this system is to minimize the long-term task completion delay of the system, the opposite of the optimization target value of the IoT system in each time slot is used as the reward function of the IoT system; the reward function r(t) is defined as formula (IV):

[0132]

[0133] Among them, ξ is the penalty parameter for task overflow, α n,m (t) is the task overflow indicator parameter, d n,m (t) represents the delay of the mth task of smart IoT device n in time slot t, that is, if the task is waiting in the queue or performing calculation, then there is d n,m (t) = T s , otherwise d n,m (t) = 0. T s is the time slot length.

[0134] In step (4), the optimization goal of the IoT system is to minimize the total delay of IoT system task execution. Taking minimizing the overall task completion delay of the IoT system as the objective function, the joint computing offloading and resource allocation decision are designed, and the optimization problem established is as follows:

[0135]

[0136]

[0137] C1:x=(x1,…,x N )

[0138] C2:

[0139] C3:b=(b1,…,b N )

[0140] C4:

[0141] C5:

[0142] C6:

[0143] C7:

[0144] C8:

[0145] Among them, d n,m (t) represents the delay of the mth task of smart IoT device n in time slot t, that is, if the task is waiting in the queue or performing calculation, then there is d n,m (t) = T s , otherwise d n,m (t) = 0; α n,m is the task overflow indication parameter. When the mth task of smart IoT device n overflows, α n,m =1, otherwise α n,m =0; Considering the limited storage space of each device, once the number of tasks waiting in the task queue exceeds the queue capacity, the tasks that arrive later will overflow. In order to ensure the reliability of the system and prevent tasks from overflowing from the waiting queue, this paper adds a large time penalty parameter ξ to the overflowing tasks. ξ is the time penalty parameter; C1, C2 and C3 (x(t), φ(t) and b(t)) are the offloading decision, MEC server computing resource allocation decision and bandwidth resource allocation decision respectively; Constraint C4 means that each smart IoT device can only choose one of the local computing mode or computing offloading mode to process its tasks in a time slot; Constraint C5 means that the proportion of computing resources allocated by the MEC server to smart IoT devices does not exceed 1, Constraint C6 means that the proportion of bandwidth resources allocated by the system to smart IoT devices does not exceed 1, Constraint C7 means that the computing resources allocated by the MEC server to all smart IoT devices cannot exceed the total computing resources of the MEC server, and Constraint C8 means that the bandwidth resources allocated by the system to all smart IoT devices cannot exceed the total bandwidth resources of the system.

[0146] In the proposed MDP problem, since the action space consists of discrete offloading decisions and continuous resource allocation decisions, it is impossible to solve it using traditional value-based reinforcement learning methods. To this end, the present invention proposes a joint computation offloading and dynamic resource allocation method based on the DDPG framework. Figure 3 As shown, the fine-grained task offloading and resource allocation algorithm model is a dual-network model, including an action network Actor network and an evaluation network Critic network; the decision action in the present invention includes discrete offloading decisions and continuous resource allocation decisions. The present invention solves the mixed action space problem by modifying the output layer structure of the action network and effectively reduces the complexity of the algorithm solution.

[0147] The neural network model is built using the TensorFlow framework. Both the action network and the evaluation network adopt a fully connected neural network structure (Dense layer), including an input layer, three hidden layers and an output layer. s(t) is defined as the state information of the IoT system at time slot t, and a(t) is the decision action executed at time slot t. The input of the Actor network is the state information s(t) of the IoT system, and the output is the decision action a(t) to be executed. Because the policy action a(t) output by the Actor network contains continuous resource allocation actions and discrete unloading actions, the neurons in the output layer of the Actor network are divided into two parts. One part uses the softmax activation function to output continuous normalized resource allocation decisions, and the other part uses the sigmoid activation function to output the probability of unloading decisions. In addition, the target network is used in the algorithm to assist training, making the learning process more stable and converging faster.

[0148] Among them, the Actor network follows the strategy a(t)=π(s(t)|θ μ ) outputs a deterministic action, and the Critic network outputs an estimated Q value Q(s(t),a(t)|θ Q ) to evaluate the quality of this deterministic action, where θ μ and θ Q They are the network parameters of the Actor network and the Critic network respectively.

[0149] In step (4), the stochastic gradient descent method is used to train the Actor and Critic neural network models:

[0150] According to the Bellman equation, the value function Q(s(t), a(t)) is the expected cumulative return of taking action a(t) in state s(t), expressed as formula (VI):

[0151]

[0152] Among them, γ∈[0,1] is the discount factor; therefore, the training objective of the fine-grained task offloading and resource allocation algorithm model is formula (VII):

[0153] Q(s(t),a(t))≈Q(s(t),a(t)|θ Q )(VII)

[0154] The Critic network uses the stochastic gradient descent method to update parameters, and its loss function L(θ Q ) is represented by formula (VIII):

[0155]

[0156] Among them, Q′(s(t+1),a′(t+1)) is the Q value output by the target Critic network;

[0157] The Actor network is updated using the stochastic gradient descent algorithm, and the gradient of the objective function It is expressed as formula (IX):

[0158]

[0159] The present invention uses an experience pool to store historical data tuples (s meu (t),s su (t),s l (t),g(t)), during training, small batches of data will be randomly selected for training, eliminating the correlation between the data.

[0160] In order to fully explore the environment and obtain a better strategy, in step (4), Ornstein-Uhlenbeck (OU) noise is added to the deterministic action, and the noise is set to gradually decrease with the increase of training time. In order to improve the stability of the learning process, a soft replacement strategy is used to update the parameters of the target fine-grained task offloading and resource allocation algorithm model. The parameter update process in each training cycle is formula (X):

[0161] θ' t+1 =(1-ε)·θ' t +ε·θ t (X)

[0162] Among them, θ' t is the target fine-grained task offloading and resource allocation algorithm model parameter, θ t Model parameters for estimating fine-grained task offloading and resource allocation algorithms.

[0163] In step (4), the training process of the fine-grained task offloading and resource allocation algorithm model is as follows:

[0164] Initialize the parameters of the fine-grained task offloading and resource allocation algorithm model and other fixed parameters. Before training, collect some data tuples through interaction with the environment and store them in the experience pool. For each training round, extract a small batch of samples from the experience pool, and update the parameters of the fine-grained task offloading and resource allocation algorithm model according to formulas (VIII), (IX) and (X). At the same time, the data in the experience pool is updated as the training process proceeds. Perform K rounds of training on the target fine-grained task offloading and resource allocation algorithm model until the target fine-grained task offloading and resource allocation algorithm model converges to obtain the final target fine-grained task offloading and resource allocation algorithm model.

[0165] In step (7), the fine-grained task scheduling process of the IoT system is:

[0166] First, at the beginning of each time slot, the SDN controller collects the status information of the IoT system. The trained SDN controller makes computing offloading decisions and MEC computing resource allocation decisions and bandwidth resource allocation decisions, and sends control instructions to smart IoT devices and MEC servers.

[0167] The IoT system will then execute the command in the remaining time to complete the offloaded or local computation;

[0168] For a smart IoT device, the intelligent computing tasks it generates, such as target detection, AR, and VR application tasks, have the characteristics of large resource requirements and high latency requirements. The tasks generated by smart IoT devices are serial dependent task flows, and the tasks arrive randomly. There are differences in workloads between different smart IoT devices. All unprocessed tasks are waiting in the local task queue, and the first tasks generated have higher priority.

[0169] The internal structure of the smart IoT device includes a scheduling unit, a local execution unit, and a data transmission unit. The scheduling unit is responsible for storing the data of the computing task being executed, the local execution unit is responsible for executing the computing task locally, and the data transmission unit is responsible for transmitting the offloaded task data to the MEC server for calculation; the MEC server includes an execution unit, which is responsible for receiving the offloaded intelligent task and performing calculation processing;

[0170] When a task is about to be executed, it is placed in the scheduling unit; thereafter, in each time slot, the task is either forwarded to the local execution unit for processing or offloaded to the MEC server through the data transmission unit; the execution unit of the MEC server receives this data and performs calculation processing in the next time slot;

[0171] Finally, the calculation results after processing are sent back to the smart IoT devices; the set of smart IoT devices is represented as The set of tasks generated by the nth smart IoT device is expressed as M n The number of tasks generated for the entire process.

[0172] The task computing modes of the IoT system include local computing mode and offload computing mode:

[0173] The local computing modes are:

[0174] Define the processor processing frequency of smart IoT device n as The time slot length is T s , the offloading decision of the smart IoT device in time slot t is defined as x(t)=(x1(t),…,x n (t),…,x N (t)), where x n (t) is the offloading decision of smart IoT device n, x n (t) = 0 means the task is executed locally, x n (t) = 1 means that the task is offloaded to the MEC server for execution. In the t time slot, the amount of data calculated locally is It is expressed as formula (XI):

[0175]

[0176] In formula (XI), k n The number of CPU revolutions required for 1 bit of data in the task currently being processed in the smart IoT device n;

[0177] The offload calculation mode is:

[0178] Define the processor processing frequency of the MEC server as f c , define the computing resource allocation ratio of MEC server in time slot t as in represents the proportion of computing resources allocated by the MEC server to smart IoT device n; therefore, the amount of data computed by smart IoT device n at the edge in time slot t It is expressed as formula (XII):

[0179]

[0180] In formula (XII), k n It is the number of CPU revolutions required for 1 bit of data in the task currently being processed in the smart IoT device n.

[0181] In step (7), the SDN controller collects the status information s(t) of the IoT system, including: meu (t),s su (t),s l(t),g(t)): Indicates the amount of data in the execution unit of the MEC server, Indicates the amount of data in the local scheduling unit, represents the task queue length of IoT devices, g(t)=(g1(t),…,g n (t),…,g N (t)) represents the wireless channel gain of the smart IoT device.

[0182] In step (7), the decision information a(t) of the SDN controller includes: x(t)=(x1(t),…,x n (t),…,x N (t)) represents the offloading decision of the smart IoT device, represents the MEC server computing resource allocation decision, b(t)=(b1(t),…,b n (t),…,b N (t)) represents the bandwidth resource allocation decision of the IoT system.

[0183] Figure 4 The convergence change of the cumulative reward value of the IoT system during training under different learning rates is shown. Figure 4 It can be seen that when the learning rate is set to 8×10 -5 When the learning rate is set to 1.6×10 -4 When the learning rate is set to 3×10 -5 When the learning rate is set too high, the algorithm converges at about 2000 training rounds. When the learning rate is set too high, the algorithm converges faster and fluctuates more, and it is easy to jump out of the global optimal solution. When the learning rate is set too low, the algorithm converges slowly and is easy to fall into the local optimum. Therefore, setting the learning rate too high or too low cannot achieve the best convergence result. The learning rate finally selected by this invention is 8×10 -5 .

[0184] The proposed method is compared with three benchmark solutions, and the total task execution delay and average task delay of the IoT system under different numbers of devices are compared. The selection of the benchmark solution fully considers the feasibility and advancement of the solution in the IoT system proposed by the present invention. The three selected benchmark solutions are as follows:

[0185] Benchmark solution 1 is to randomly offload tasks and evenly distribute computing resources, in which all devices' tasks are randomly offloaded to the MEC server, and the computing resources of the MEC server are evenly distributed to each device; Benchmark solution 2 is to offload all tasks and evenly distribute resources, in which all devices' tasks are offloaded to the MEC server, and the computing resources of the MEC server are evenly distributed to each device; Benchmark solution 3 implements overall task scheduling and resource allocation based on tasks based on the DDPG algorithm, in which the task scheduling decision is made only once when the task starts to execute, so that all scheduled tasks can be offloaded or all local calculations can be achieved.

[0186] Figure 5 The total delay of system task execution of four schemes in the case of different numbers of devices in the IoT system is given. Figure 5 It can be found that the algorithm proposed in the present invention has the shortest total task execution delay in all cases. On the whole, the average resource allocation delay of all tasks unloading is the largest, followed by the average resource allocation of random unloading, and then the overall task scheduling scheme based on DDPG, the total delay of the algorithm proposed in the present invention is the smallest. The scheme of all tasks unloading has the largest total task execution delay because it does not utilize local computing resources. Compared with the overall task scheduling scheme based on DDPG in units of tasks, the proposed scheme better adapts to the dynamic environment by fine-grained scheduling of tasks, and effectively reduces the total task execution delay of the system. Therefore, the fine-grained task unloading and resource allocation method based on deep reinforcement learning proposed in the present invention has great performance advantages.

[0187] As can be seen from Figure 5, the total delay of the IoT system increases with the number of users. When the number of devices is 10, the total system delay of the proposed scheme is reduced by 29%, 50% and 11% respectively compared with the other three schemes.

[0188] Figure 6 The average task execution delay of four schemes under different numbers of devices in the IoT system is given. Figure 6 It can be seen that the algorithm proposed in the present invention also has the smallest average task execution delay in all cases. When the number of devices is 10, compared with the other three solutions, the average task delay of the proposed solution is reduced by 20%, 45% and 5% respectively.

[0189] Based on the scheduling architecture of multi-access edge computing in the Internet of Things, the present invention aims at the fine-grained scheduling problem of intelligent computing tasks with large resource requirements, and designs a joint optimization scheme for task offloading computing resource allocation to minimize the long-term task execution delay of the system. Since the state space and action space of the Internet of Things system are huge, and continuous and discrete variables coexist, the present invention models the optimization problem as an MDP solution, performs joint offloading and resource allocation based on the DDPG framework, and realizes the separation of mixed actions by improving the output layer of the action network. The final results show that the method proposed in the present invention has good convergence characteristics and has a smaller system task delay than other benchmark algorithms.

[0190] Example 3

[0191] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the fine-grained task offloading and resource allocation method for MEC network described in embodiment 1 or 2 are implemented.

[0192] Example 4

[0193] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for fine-grained task offloading and resource allocation for MEC networks described in Embodiment 1 or 2 are implemented.

Claims

1. A fine-grained task offloading and resource allocation method for MEC networks, characterized in that: It includes the model training phase and the decision reasoning phase. The specific steps include: A. Model training phase, including: (1) Create an IoT system: The IoT system includes an MEC server, an SDN controller, and smart IoT devices. The MEC server provides computing services for smart IoT devices, and the SDN controller is responsible for the training and decision reasoning of task scheduling and resource allocation algorithm models. (2) Deploy a fine-grained task offloading and resource allocation algorithm model based on deep reinforcement learning on the SDN controller and initialize the fine-grained task offloading and resource allocation algorithm model; (3) The SDN controller acts as an intelligent agent and interacts with the IoT system environment to collect training data: decision generation and control command issuance are performed in a time slot manner, where the time slot length is the coherent time length. The interaction information between the intelligent agent and the system environment in each time slot is defined as a data tuple and stored in the experience pool; (4) The stochastic gradient descent algorithm is used to train the fine-grained task offloading and resource allocation algorithm model. The training goal is to minimize the total task execution delay of the system. (5) Repeat steps (3) and (4) until the fine-grained task offloading and resource allocation algorithm model converges, the training process of the fine-grained task offloading and resource allocation algorithm model is completed, and the decision reasoning stage is entered; B. Decision-making reasoning stage, including: (6) Redeploy the trained fine-grained task offloading and resource allocation algorithm model to the SDN controller to implement decision reasoning; (7) At the beginning of the time slot, the SDN controller collects the IoT system status information, and the trained SDN controller makes computing offloading decisions, MEC computing resource allocation decisions, and bandwidth allocation decisions, and sends them to the smart IoT devices and MEC servers in the form of control instructions; (8) During the remaining time of this time slot, the smart IoT devices and MEC servers execute the received control instructions and complete the offload calculation or local calculation of the smart computing task; (9) Repeat steps (7) and (8) until the entire process is completed; The fine-grained task offloading and resource allocation algorithm model is a dual-network model, including an action network Actor network and an evaluation network Critic network; Both the Actor network and the Critic network adopt a fully connected neural network structure, including an input layer, multiple hidden layers and an output layer; define s(t) as the state information of the IoT system at time slot t, a(t) as the decision action executed at time slot t, the input of the Actor network is the state information s(t) of the IoT system, and the output is the decision action a(t) to be executed. The neurons in the output layer of the Actor network are divided into two parts, one part uses the softmax activation function to output continuous normalized resource allocation decisions, and the other part uses the sigmoid activation function to output the probability of offloading decisions; Among them, the Actor network follows the strategy a(t)=π(s(t)|θ μ ) outputs a deterministic action, and the Critic network outputs an estimated Q value Q(s(t),a(t)|θ Q ) to evaluate the quality of this deterministic action, where θ μ and θ Q They are the network parameters of the Actor network and the Critic network respectively; The process of interaction between the agent and the IoT system environment in step (3) and the generated data tuple (s meu (t),s su (t),s l (t), g(t)): The specific implementation process of step (3) is: Step 3-1: At the beginning of each time slot, the SDN controller, as an intelligent agent, collects the state information s(t) of the IoT system in time slot t and makes decisions based on s(t). The state space S is described as formula (II): S={s(t)|s(t)=(s meu (t),s su (t),s l (t),g(t))} (II) Among them, the MEC server includes an execution unit, which is responsible for receiving intelligent tasks for computational offloading and performing computational processing; Represents the amount of data in the execution unit of the MEC server. The smart IoT device includes a scheduling unit, which is responsible for storing the data of the computing tasks being executed; represents the amount of data in the dispatch unit of the smart IoT device, represents the task queue length of the smart IoT device, g(t)=(g1(t),…,g n (t),…,g N (t)) represents the wireless channel gain of the smart IoT device, and the dimension of the state space of the IoT system is 4N; Step 3-2: After collecting the state information s(t) of the IoT system at time slot t, the SDN controller will make a decision action a(t) and send a control signal to the MEC server and smart IoT devices; the action space A is described as formula (III): Among them, x(t), φ(t) and b(t) are the offloading decision, MEC computing resource allocation decision and system bandwidth resource allocation decision respectively. The dimension of the action space of the IoT system is 3N. Step 3-3: After selecting and executing the decision action a(t), the IoT system transfers from s(t) to s(t+1), where the state transition probability is expressed as P(s(t+1)|s(t),a(t)); Step 3-4: After selecting and executing the decision action a(t), the IoT system generates an immediate reward r(t) for this decision action a(t); the opposite of the IoT system optimization target value per time slot is used as the reward function of the IoT system; the reward function r(t) is defined as formula (IV): Among them, ξ is the penalty parameter for task overflow, α n,m (t) is the task overflow indicator parameter, d n,m (t) represents the delay of the mth task of smart IoT device n in time slot t. If the task is waiting in the queue or is executing calculation, then d n,m (t) = T s , otherwise d n,m (t) = 0, M n The number of tasks generated for the entire process.

2. According to claim 1, a fine-grained task offloading and resource allocation method for MEC network is characterized in that: The wireless communication process of the IoT system is: The MEC server and smart IoT devices use orthogonal frequency division to transmit data, and each smart IoT device uses a different frequency band; Define the bandwidth ratio allocated by the IoT system to all smart IoT devices as b(t)=(b1(t),…,b n (t),…,b N (t)), where b n (t) is the channel bandwidth ratio allocated by the IoT system to the smart IoT device n in time slot t; according to the Shannon formula, the maximum transmission rate of data from the smart IoT device n to the MEC server is R n (t) is defined as shown in formula (I): In formula (I), B is the total communication bandwidth of the IoT system, p n is the transmission power of smart IoT device n, g n (t) is the wireless channel gain of smart IoT device n in time slot t, and N0 is the Gaussian noise power spectral density.

3. According to claim 1, a fine-grained task offloading and resource allocation method for MEC network is characterized in that: In step (4), the optimization goal of the IoT system is to minimize the total delay of IoT system task execution. Taking minimizing the overall task completion delay of the IoT system as the objective function, a joint computing offloading and resource allocation decision algorithm is designed, and the established optimization problem is as follows: st C1:x=(x1,…,x N ) C2: C3:b=(b1,…,b N ) C4: C5: C6: C7: C8: Among them, d n,m (t) represents the delay of the mth task of smart IoT device n in time slot t. If the task is waiting in the queue or is executing calculation, then d n,m (t) = T s , otherwise d n,m (t) = 0; α n,m is the task overflow indication parameter. When the mth task of smart IoT device n overflows, α n,m =1, otherwise α n,m =0; ξ is the time penalty parameter; C1, C2 and C3 are the offloading decision, MEC server computing resource allocation decision and bandwidth resource allocation decision respectively; constraint C4 means that each smart IoT device can only choose one of the local computing mode or computing offloading mode to process its task in a time slot; constraint C5 means that the proportion of computing resources allocated by the MEC server to smart IoT devices does not exceed 1, constraint C6 means that the proportion of bandwidth resources allocated by the system to smart IoT devices does not exceed 1, constraint C7 means that the computing resources allocated by the MEC server to all smart IoT devices cannot exceed the total computing resources of the MEC server, and constraint C8 means that the bandwidth resources allocated by the system to all smart IoT devices cannot exceed the total bandwidth resources of the system.

4. According to claim 1, a fine-grained task offloading and resource allocation method for MEC network is characterized in that: According to the Bellman equation, the value function Q(s(t), a(t)) is the expected cumulative return of taking action a(t) in state s(t), expressed as formula (VI): Among them, γ∈[0,1] is the discount factor; therefore, the training objective of the fine-grained task offloading and resource allocation algorithm model is formula (VII): Q(s(t),a(t))≈Q(s(t),a(t)|θ Q )(VII) The Critic network uses the stochastic gradient descent method to update parameters, and its loss function L(θ Q ) is represented by formula (VIII): Among them, Q′(s(t+1),a′(t+1)) is the Q value output by the target Critic network; The Actor network is updated using the stochastic gradient descent algorithm, and the gradient of the objective function It is expressed as formula (IX):

5. According to claim 1, a fine-grained task offloading and resource allocation method for MEC network is characterized in that: In step (4), Ornstein-Uhlenbeck noise is added to the deterministic action, and the noise is set to gradually decrease as the training time increases. A soft replacement strategy is used to update the parameters of the target fine-grained task offloading and resource allocation algorithm model. The parameter update process in each training cycle is formula (X): I will t+1 =(1-e)·θ' t +e·i t (X) Among them, θ' t is the target fine-grained task offloading and resource allocation algorithm model parameter, θ t To estimate the model parameters of fine-grained task offloading and resource allocation algorithms; In step (4), the training process of the fine-grained task offloading and resource allocation algorithm model is as follows: Initialize the parameters of the fine-grained task offloading and resource allocation algorithm model and other fixed parameters. Before training, collect some data tuples through interaction with the environment and store them in the experience pool. For each training round, extract a small batch of samples from the experience pool, and update the parameters of the fine-grained task offloading and resource allocation algorithm model according to formulas (VIII), (IX) and (X). At the same time, the data in the experience pool is updated as the training process proceeds. Perform K rounds of training on the target fine-grained task offloading and resource allocation algorithm model until the target fine-grained task offloading and resource allocation algorithm model converges to obtain the final target fine-grained task offloading and resource allocation algorithm model.

6. A fine-grained task offloading and resource allocation method for MEC networks according to any one of claims 1 to 5, characterized in that: In step (7), the fine-grained task scheduling process of the IoT system is: First, at the beginning of each time slot, the SDN controller collects the status information of the IoT system. The trained SDN controller makes computing offloading decisions and resource allocation decisions, and sends control instructions to smart IoT devices and MEC servers. The IoT system will then execute the command in the remaining time to complete the offloaded or local computation; The tasks generated by smart IoT devices are serial dependent task flows, and the tasks arrive randomly. Different smart IoT devices have different workloads. All unprocessed tasks are waiting in the local task queue, and the first task generated has a higher priority. The internal structure of the smart IoT device includes a scheduling unit, a local execution unit, and a data transmission unit. The scheduling unit is responsible for storing the data of the computing task being executed, the local execution unit is responsible for executing the computing task locally, and the data transmission unit is responsible for transmitting the offloaded task data to the MEC server for calculation; the MEC server includes an execution unit, which is responsible for receiving the offloaded intelligent task and performing calculation processing; When a task is about to be executed, it is placed in the scheduling unit; thereafter, in each time slot, the task is either forwarded to the local execution unit for processing or offloaded to the MEC server through the data transmission unit; the execution unit of the MEC server receives this data and performs calculation processing in the next time slot; Finally, the calculation results after processing are sent back to the smart IoT devices; the set of smart IoT devices is represented as The set of tasks generated by the nth smart IoT device is expressed as M n The number of tasks generated for the entire process.

7. According to claim 6, a fine-grained task offloading and resource allocation method for MEC network is characterized in that: The task computing modes of the IoT system include local computing mode and offload computing mode: The local computing modes are: Define the processor processing frequency of smart IoT device n as The time slot length is T s , the offloading decision of the smart IoT device in time slot t is defined as x(t)=(x1(t),…,x n (t),…,x N (t)), where x n (t) is the offloading decision of smart IoT device n, x n (t) = 0 means the task is executed locally, x n (t) = 1 means that the task is offloaded to the MEC server for execution. In the t time slot, the amount of data calculated locally is It is expressed as formula (XI): In formula (XI), k n The number of CPU revolutions required for 1 bit of data in the task currently being processed in the smart IoT device n; The offload calculation mode is: Define the processor frequency of the MEC server as f c , define the computing resource allocation ratio of MEC server in time slot t as in represents the proportion of computing resources allocated by the MEC server to smart IoT device n; therefore, the amount of data computed by smart IoT device n at the edge in time slot t It is expressed as formula (XII): In formula (XII), k n It is the number of CPU revolutions required for 1 bit of data in the task currently being processed in the smart IoT device n.

8. According to claim 6, a fine-grained task offloading and resource allocation method for MEC network is characterized in that: In step (7), the SDN controller collects the status information s(t) of the IoT system, including: meu (t),s su (t),s l (t),g(t)): Indicates the amount of data in the execution unit of the MEC server, Indicates the amount of data in the local scheduling unit, represents the task queue length of IoT devices, g(t)=(g1(t),…,g n (t),…,g N (t)) represents the wireless channel gain of the smart IoT device.

9. According to claim 6, a fine-grained task offloading and resource allocation method for MEC network is characterized in that: In step (7), the decision information a(t) of the SDN controller includes: x(t)=(x1(t),…,x n (t),…,x N (t)) represents the offloading decision of the smart IoT device, represents the MEC server computing resource allocation decision, b(t)=(b1(t),…,b n (t),…,b N (t)) represents the bandwidth resource allocation decision of the IoT system.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the fine-grained task offloading and resource allocation method for MEC networks described in any one of claims 1-9 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the fine-grained task offloading and resource allocation method for MEC networks described in any one of claims 1 to 9 are implemented.

Images