MEC-based energy internet multi-task traffic unloading method

Abstract

The invention provides an energy internet multi-task traffic unloading method based on MEC, which belongs to the technical field of communication resource allocation, and comprises the following steps: establishing a single-cell multi-user MEC network model comprising a task model, a queue model, a local calculation model and an unloading calculation model, and obtaining the network state of the MEC network model; constructing an MEC system migration scheduling decision and resource allocation problem model according to the MEC network model; inputting the collected network state parameters into a reinforcement learning model, and solving the problem model by using the reinforcement learning model to obtain an unloading decision and resource allocation and bandwidth allocation of a to-be-unloaded task; and performing unloading scheduling on the task according to the unloading strategy, the resource allocation and the bandwidth allocation. The method has the advantages that a dynamic priority mechanism based on service delay tolerance and reliability requirements is introduced, so that the problem that differentiated requirements of power diversified services are difficult to consider is solved; the mechanism can automatically generate a task processing queue according to business criticality.

Description

Technical Field

[0001] The present invention belongs to the technical field of communication resource allocation, and in particular relates to a multi-task traffic offloading method for energy Internet based on MEC. Background Art

[0002] With the rapid development of the Energy Internet, power services are becoming more diverse, intelligent, and real-time, placing higher demands on communication network performance. Ultra-reliable, low-latency services such as distribution automation and millisecond-level load control, ultra-high-bandwidth video services such as intelligent inspection, and massively connected services such as electricity consumption data collection and load monitoring urgently require lower-latency, higher-reliability computing and communication support. Traditional cloud computing models struggle to meet the stringent latency requirements of these services due to the remote locations of data centers. Mobile Edge Computing (MEC) technology, by deploying computing and storage resources at the network edge close to power terminal devices (such as substations, distribution rooms, and smart meters), particularly when combined with high-performance, low-latency 5G private networks for power, provides an ideal support platform for the Energy Internet. In a typical 5G MEC deployment scenario for power, a single MEC node, via a base station, covers a single cell and serves multiple user devices (such as various monitoring terminals, sensors, and controllers). Compute-intensive tasks generated by user devices can be executed locally or offloaded to edge servers via 5G private network wireless channels, enabling rapid task processing and response.

[0003] However, existing MEC-based task offloading and resource allocation strategies still have significant shortcomings when addressing the diverse needs of power services. Power services such as distribution control, inspection video analysis, and load monitoring vary significantly in latency tolerance, reliability requirements (e.g., transmission error rate), task data volume, and computational complexity. Existing strategies often lack a refined distinction between service-critical QoS, making it difficult to dynamically optimize resource allocation based on service priorities. Secondly, in typical scenarios where a single MEC node serves multiple user devices, limited edge computing resources and wireless channel bandwidth become bottlenecks. When multiple user devices simultaneously choose to offload tasks, resource competition is intense. Existing static or simple priority allocation mechanisms struggle to ensure strict QoS for high-priority services, such as extremely low latency and high reliability, while efficiently utilizing resources and minimizing overall system energy consumption. Summary of the Invention

[0004] In view of this, the present invention addresses the deficiencies in the existing technology and provides a multi-task traffic offloading method for the energy Internet based on MEC. To solve the above technical problems, the technical solution adopted by the present invention is: establishing a single-cell multi-user MEC network model including a task model, a queue model, a local computing model and an offloading computing model, and obtaining the network status of the MEC network model; constructing a problem model for MEC system migration scheduling decision and resource allocation based on the MEC network model; inputting the collected network status parameters into a reinforcement learning model, and using the reinforcement learning model to solve the problem model to obtain the offloading decision and the resource allocation and bandwidth allocation of the task to be offloaded; and scheduling the offloading of tasks according to the offloading strategy, resource allocation and bandwidth allocation.

[0005] Furthermore, the method for establishing a single-cell multi-user MEC network model includes:

[0006] The edge server connected to the base station serves N user devices; assuming τ is a time slot, the time set is The user set is When a mobile device is within the coverage of a base station, each user device will generate a computationally intensive task to be completed within the time slot τ. The task generated by the i-th user device within the time slot τ is A i (t), the tasks generated by each user device are independent of each other, where: τ is the length of a single time slot; T is the set of discrete time slots; t is the current time slot index; N is the total number of user devices; i is the user device index;

[0007] Use θ i (t)∈(0,1) is used as a task migration indicator, indicating that the task A generated by the i-th user equipment in the time slot τ i (t) Whether to migrate, the computation offloading strategy of all user devices in the time slot τ is used Indicates; θ i =1 indicates task A i (t) needs to be migrated to the edge server for execution, θ i =0 means task A i (t) Executed in the local CPU.

[0008] Furthermore, the method for constructing the task model includes:

[0009] Task A generated by the i-th user equipment in time slot τ i (t) consists of three parts:

[0010]

[0011] Among them, b i (t) represents the task A that needs to be performed i (t) is the data size in bits; di (t) indicates the completion of task A i (t) The amount of computing resources required, expressed in terms of required CPU cycles; Represents Task A i (t) delay time threshold, that is, task A i The total time taken for (t) cannot exceed The three parameters of the task are related to the business characteristics and are estimated through the task profile;

[0012] Using multi-priority task offloading based on QoS constraints, considering the two attributes of reliability and latency, the task priority calculation function is:

[0013] Priority=α×Time+β×Reliability

[0014] Where: Time is the delay attribute, which is the maximum tolerance value of the task processing event generated by the user device, expressed as t max , Reliability is the reliability attribute, and is the task transmission error rate ε i (t); α, β are priority weight coefficients, and α+β=1; Task A i The priority value of (t) is expressed as P r,i (t).

[0015] Furthermore, the method for constructing the queue model includes:

[0016] The MEC edge server sets up a task cache for each mobile user device to store tasks that have been offloaded but not yet processed; there is a queue to store tasks waiting to be executed;

[0017] Define the task B that offloads to the edge server i (t) = θ i (t)·A i (t), Q i (t) is the queue length of the i-th user device in the edge server in the time slot τ, C i (t) is the task scheduling decision of the MEC edge server in the time slot τ, that is, the number of tasks being processed by the i-th user device in the edge server in the time slot τ; then the queue length Q of the next time slot τ is i (t+1) is:

[0018] Q i (t+1)=max{Q i (t)-Q i (t),0}+B i (t),t∈{0,1,2,…}

[0019] The task buffer is initially empty, i.e. Q i (0) = 0, i∈N; the MEC computing resources allocated to the task being processed by the i-th user device in the edge server within the time slot τ are:

[0020]

[0021] Where: f s,i (t) is the computing resource of the edge server; F ser is the total computing capacity of MEC;

[0022] And satisfy the resource capacity constraint: ∑ i∈N C i (t) b i (t)·≤F ser τ, indicating completion The number of CPU cycles required for scheduling decisions does not exceed the total available computing cycles of the server;

[0023] Where: C(t) is the scheduling decision vector;

[0024] And the queue stability constraint is satisfied: Ensure that all arriving computational tasks are completed within a bounded delay.

[0025] Furthermore, the method for constructing the local computing model includes:

[0026] The local CPU calculation cycle frequency of the i-th user device is Cannot exceed its maximum value Use collection represents the local CPU computing cycle frequency; the local execution power consumption of the i-th mobile device in, Indicates the energy consumption of each CPU cycle to complete the task; the local execution latency is:

[0027] Furthermore, the method for constructing the offloading calculation model includes:

[0028] Task A i (t) Transmission rate v offloaded to edge server i (t) = γ i (t)·BW·(log2(1+δ i (t))-ψ); where γ i (t) is task A i (t) The proportion of wireless bandwidth allocated during offloading to the edge server,

[0029] Only the unloaded tasks are selected to participate in the allocation, that is, θ i(t)=1,0≤γ i (t)≤1 and∑ i∈N γ i (t) = 1;

[0030] To uninstall task A i (t) Signal-to-noise ratio to edge server where p tx is the transmission power, g0 is the channel gain, and N0 is the additive Gaussian channel noise power;

[0031] Reliability Factor Where: V τ is the channel dispersion, which can be approximated to 1; Q -1 (·) represents the inverse of the Gaussian Q function, ε i (t) represents task A i (t) Transmission error rate during offloading to edge servers;

[0032] Average waiting delay in queues

[0033] Assuming the computation delay of the edge server

[0034] Task A i (t) The latency of the entire offloading process to the edge server Energy consumption for:

[0035] Among them, κ ser is the CPU capacitance coefficient of the MEC edge server; t up is the uplink transmission delay; f s,i (t) the real-time computing resource rate allocated to the edge server;

[0036] The overall cost of a single-cell multi-user MEC system is obtained based on the local execution energy consumption and offload execution energy consumption: Where: E all is the total energy consumption of the system.

[0037] Furthermore, the method of constructing a problem model for MEC system migration scheduling decision and resource allocation based on the MEC network model includes:

[0038] The problem model is constructed with the goal of minimizing total energy consumption as follows:

[0039] The constraints defining the problem model are:

[0040]

[0041]

[0042] Furthermore, the reinforcement learning model is implemented using the A3C algorithm, which consists of a central neural network, the global, and n independent agents. The global and agents each maintain a network with exactly the same structure, with both an actor network and a critic network.

[0043] The agent obtains the experience gained in recent rounds through the actor-critic network and uploads it to the global network. The global network aggregates the experience of all agents and distributes it to each agent.

[0044] The network state parameters s = (te, aw, af) are input into the Actor network, where te represents the overall execution delay of the current task, aw represents the available wireless bandwidth resources, and af represents the available edge computing resources. The discrete action space action = {0, 1, 2, ..., 7} is output, and each action corresponds to a pair (ra, rc) of bandwidth and computing resource allocation ratios. ra represents the bandwidth allocation ratio γ i (t), rc represents the computing resource allocation ratio f s,i (t);

[0045] Unloading decision θ i (t) is implicitly determined by the bandwidth allocation ratio, when γ i (t)>0θ i (t)=1, otherwise θ i (t) = 0;

[0046] Allocated wireless bandwidth ratio γ i (t) and the computing resource allocation ratio f s,i (t);

[0047] The total energy consumption of the system E all As a reward signal; update the Actor-Critic network parameters through TD-error, and optimize the offloading decision and resource allocation end-to-end. The temporal difference error is expressed as:

[0048] TD_error=Q(s,a)-V(s)=r+γV(s′)-V(s)

[0049] The policy network Actor input is state state, and the output is the probability of all discrete actions act_prob. The loss function is -log(at_prob)*td_error. The value network Critic input is state state, and the output is state value v. The loss function is (td_error) 2; Where: TD_error is the temporal difference error; Q(s,a) is the state-action value function; V(s) is the value estimate of the new state; r is the immediate reward, and E is the total energy consumption of the system all ; γ is the discount factor.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. At the business support level, by introducing a dynamic priority mechanism based on business latency tolerance and reliability requirements, the problem of difficulty in balancing the differentiated needs of diversified power services (such as distribution control, intelligent inspection, and load monitoring) is solved. This mechanism can automatically generate task processing queues based on business criticality, so that high-real-time and high-reliability services can obtain more sufficient resource guarantees, thereby more effectively meeting the stringent service quality requirements of millisecond-level control services such as distribution automation.

[0052] 2. At the resource optimization level, resource utilization efficiency is improved by jointly optimizing three key dimensions: task offloading decisions, wireless bandwidth allocation ratio, and edge server computing resource allocation. Especially in multi-user concurrent offloading scenarios, the priority-based dynamic proportional allocation mechanism can alleviate the competition between computing resources and wireless channels, giving priority to key services under limited resource conditions while reducing the overall energy consumption of the system. Compared with traditional methods, this mechanism shows better adaptability when facing an increase in the number of users or limited resources.

[0053] The distributed collaborative learning framework employed in this solution enhances the algorithm's ability to solve complex dynamic environments. By sharing experience and conducting collaborative training among agents, this framework can more efficiently respond to fluctuations in task loads and changes in channel states, improving the convergence speed and stability of policy solutions. This helps maintain a stable task queue, reduce task backlogs, and maintain overall system performance reliability and scalability during peak traffic or scale expansion, meeting the robust computing resource scheduling requirements of large-scale power IoT deployments. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The present invention will be described in further detail below with reference to the accompanying drawings.

[0055] Figure 1 : Schematic diagram of the single-cell multi-user MEC network structure in the present invention;

[0056] Figure 2 : Schematic diagram of task queue and resource allocation in the MEC edge server of the present invention;

[0057] Figure 3 : Schematic diagram of the Actor network and Critic network in the present invention;

[0058] Figure 4 : Schematic diagram of Push and Pull modules in the present invention;

[0059] Figure 5 : Schematic diagram of the action space in the present invention;

[0060] Figure 6 : Schematic diagram of the relationship between the total energy consumption and the number of users of the present invention;

[0061] Figure 7 : Schematic diagram of the relationship between total energy consumption and MEC computing resource capacity in the present invention. DETAILED DESCRIPTION

[0062] In order to better understand the present invention, the content of the present invention is further clearly described below in conjunction with the examples and drawings, but the protection content of the present invention is not limited to the following examples. In the following description, a large number of specific details are given in order to provide a more thorough understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be implemented without one or more of these details.

[0063] like Figure 1 Figure 1 shows a system model for computing offloading and resource allocation for multiple users in a single MEC node scenario. Consider a scenario where a single MEC node serves multiple users simultaneously. Each node has a base station that is connected to a high-performance MEC edge server to perform offloading tasks.

[0064] Example 1: See Figure 2-7 ,In this embodiment, a multi-task traffic offloading method of the energy Internet based on MEC is established,a single-cell multi-user MEC network model is established, where the edge server service connected to the base station is connected to N user devices, and τ is a time slot, and the time set is The user set is When a mobile device is within the coverage of a base station, each user device will generate a computationally intensive task to be completed in the time slot τ. The network status of the MEC network model is obtained, including: in each time slot t, constructing a computationally intensive task including user device i, defining the amount of data b i (t), calculation amount d i (t) and delay threshold The task triplet and business priority parameter P r,i (t); build a queue model containing the state parameters Qi(t) of each user task queue in the edge server; build a queue model containing the local computing delay parameters and energy consumption Calculation model of the transmission delay t up , queue waiting Edge computing latency t exe Offloading model; obtain available wireless bandwidth BW and total MEC computing resources F ser parameter;

[0065] The task generated by the i-th user equipment in the time slot τ is A i (t), the tasks generated by each user device are independent of each other. Task A i (t) can be offloaded to the edge server through the wireless private network channel or can be executed locally. It is assumed here that the task cannot be split and can only be offloaded to the edge server or executed on the local device. i (t)∈(0,1) is used as a task migration indicator, indicating that the task A generated by the i-th user equipment in the time slot τ i (t) Whether to migrate, the computation offloading strategy of all user devices in the time slot τ is used Indicates. θ i =1 means A i (t) needs to be migrated to the edge server for execution, θ i =0 means task A i (t) is executed in the local CPU. Where: τ is the length of a single time slot; T is the set of discrete time slots; t is the current time slot index; N is the total number of user devices; i is the user device index; A i (t) is the user-task triplet; θ i (t) is the task offloading decision indicator; Λ(t) is the offloading decision vector.

[0066] (1) Task Model

[0067] Define the amount of data b i (t), calculation amount d i (t) and delay threshold The task triplet and business priority parameter P r,i (t) Task model;

[0068] This application assumes that the bandwidth of the wireless channel is BW (MHz). Due to the single-cell multi-user system, the impact of inter-cell interference is ignored.

[0069] In the integrated 5G power network, MEC can be used in ultra-reliable and low-latency control services such as distribution automation and millisecond-level load control, ultra-large bandwidth video services such as intelligent inspection, and massive connection collection services such as low-voltage power consumption information collection and non-intrusive load monitoring.

[0070] The tasks generated by these services on user devices have different requirements in terms of delay threshold, reliability, and processing volume. Therefore, this application will introduce a service priority mechanism, design a priority calculation function Priority based on QoS attribute constraints, calculate the respective priority values ​​of each task, and define an ordered queue of priority values ​​Priority before task processing. In this application, task offloading and resource allocation will adopt the idea of ​​multi-priority task offloading based on QoS constraints. When the task is offloaded, the task with the highest priority value Priority will obtain the most unlimited resource bandwidth. When the task is offloaded to the edge server for processing, the task with the highest priority value Priority will obtain the most computing resources. The service quality QoS attributes of mobile edge computing mainly include reliability and delay. The design is based on the two attributes of reliability and delay. The calculation formula of the task priority calculation function Priority is:

[0071] Priority=α×Time+β×Reliability

[0072] In the priority calculation function Priority based on QoS constraints, Time is the maximum tolerance value of the task processing event generated by the user equipment, which can be expressed as t max , Reliability is the reliability requirement of the task generated by the user equipment, which can be expressed as the task transmission error rate ε i (t). α and β are coefficient values, and α+β=1, representing the proportion of emphasis the MEC computing system places on these two attributes. The values ​​of Time and Reliability will be selected by the user device based on the QoS attribute preference of the task. The task A generated by the i-th user device in the time slot τ i The priority value of (t) can be expressed as P r,i (t). Where: Time is the delay attribute; Reliability is the reliability attribute.

[0073] Task A generated by the i-th user equipment in time slot τ i (t) consists of three parts:

[0074]

[0075] Among them, b i (t) represents the task A that needs to be performed i (t) is the data size in bits. i (t) indicates the completion of task A i (t) The amount of computing resources required, here expressed as the required CPU computing cycles. Due to the difference in computing power between the edge server and the local server, the time required to run the same task is also different. Represents Task A i (t) delay time threshold, that is, task A i The total time taken for (t) cannot exceed All three parameters are related to the characteristics of the business and can be estimated through the task profile, so they may vary greatly between different types of businesses.

[0076] (2) Queue model

[0077] See Figure 2 , construct a queue model that includes the state parameters Qi(t) of each user task queue in the edge server. Assume that there are only three priority values. MEC edge servers have limited computing resources, so MEC edge servers set up a task cache for each mobile user device to store offloaded but unprocessed tasks. There is a queue to store tasks waiting to be executed.

[0078] For each time slot τ, the task A generated by the i-th user equipment i (t), the task that arrives at the edge server after being offloaded, assuming that the task is B i (t), B i (t) = θ i (t)·A i (t). Assume Q i (t) The queue length of the i-th user device in the edge server in time slot τ, which is assumed to have a large enough capacity, similar to the capacity of the mobile terminal. C i (r) represents the task scheduling decision of the MEC edge server in the time slot τ, that is, the number of tasks being processed in the edge server for the i-th user device in the time slot τ. i (t), unloading arrived task B i (t) and task scheduling decision C i (t), the queue length Q of the next time slot τ can be derived i (t+1), its expression is:

[0079] Q i (t+1)=max{Q i (t)-C i (t),0}+B i (t),t∈{0,1,2,…}

[0080] Assume that the task buffer is initially empty, that is, Q i (0) = 0, i∈N. The computing resources of the edge server can be allocated to the computing tasks offloaded from different mobile user devices, so the MEC edge server scheduling decision should meet the following conditions:

[0081] ∑ i∈N C i (t) b i (t)·≤F ser ·τ

[0082] This means completion The number of CPU cycles required should not be greater than the total number of cycles available on all CPUs in the MEC edge server. i (t) is the unloading task; Q i (t) is the queue length; C i (t) is the task scheduling decision; F ser is the total computing capacity of MEC; C(t) is the scheduling decision vector.

[0083] When the task set of the i-th user device is being processed in the edge server within the time slot τ, the MEC computing resources allocated to it will also be allocated proportionally according to the task priority value. The MEC computing resources allocated to the task of the same priority value being processed by the i-th user device in the edge server within the time slot τ is f s,i (t), the expression is as follows:

[0084]

[0085] (3) Local computing model

[0086] Construct a local calculation delay parameter and energy consumption The calculation model of the i-th user device is: Cannot exceed its maximum value Use collection Indicates the local CPU calculation cycle frequency. Assuming that the edge server's calculation speed is much faster than the local CPU's maximum calculation speed, the time delay for the task to be completed on the local CPU is the task's calculation time. Represents Task A i (t) The local delay is expressed as follows:

[0087]

[0088] The local execution power consumption of the i-th mobile device is,

[0089]

[0090] in, Indicates the energy consumption per CPU cycle to complete the task; is the local CPU frequency; is the maximum frequency of the local CPU; fl is the local computing resource vector; The local execution delay.

[0091] (4) Unloading calculation model

[0092] Construct a system that includes the fusion transmission delay t up , queue waiting Edge computing latency t exe The offloading model is used to obtain the available wireless bandwidth BW and the total amount of MEC computing resources F ser Parameters; If multiple user devices choose to offload tasks to the edge server within the time slot τ, the wireless bandwidth will be allocated proportionally to the offloading users for uploading data according to the task priority value. The expression of the transmission rate must also take into account the extreme delay and reliability requirements. This application considers the impact of the transmission error rate, which is different from the classic Shannon formula. Task A generated by the i-th user device in the time slot τ i (t) The transmission rate offloaded to the edge server can be expressed as:

[0093] v i (t) = γ i (t)·BW·(log2(1+δ i (t))-ψ)

[0094] Among them, γ i (t) is the task A generated by the i-th user equipment in time slot τ i (t) The proportion of wireless bandwidth allocated during offloading to the edge server, γ i The value of (t) is affected by the priority and can be expressed as

[0095]

[0096] Otherwise 0≤γ i (t)≤1 and∑ i∈N γ i (t)=1. where p tx is the i-th user equipment in time slot τ used to offload task A i (t) is the transmission power to the edge server, g0 is the power of the i-th user equipment used to offload task A in time slot τ i (t) is the channel gain to the edge server process, N0 is the number of user devices i used to offload task A in time slot τ i (t) is the additive Gaussian channel noise power from the process to the edge server. ψ is the reliability factor, which can be expressed as This application considers that the maximum transmission rate is related to the transmission error rate, V τ is the channel dispersion, which can be approximated to 1, indicating the random variation of the channel relative to a deterministic channel of the same capacity.-1 (·) represents the inverse of the Gaussian Q function, ε i (t) represents the task A generated by the i-th user equipment in the time slot τ i (t) Transmission error rate during offloading to edge servers; γ i (t) is the bandwidth allocation ratio; δ i (t) is the signal-to-noise ratio.

[0097] The i-th user device uninstalls task B i (t) reaches the edge server, the whole offloading process will be divided into three steps. First, the i-th user device needs to upload task B i The input data of (t) is forwarded to the base station through the wireless private network, and the base station forwards the data to the MEC edge server. Then, task B i (t) enters the waiting queue and waits for the MEC edge server to allocate computing resources to perform the computing task. Finally, the MEC edge server returns the execution result to the i-th user device. For the last step of the offload calculation, the time required is the return delay of the processing result, which is expressed as t down The download data rate is generally very high, and the data size of the task processing result is much smaller than the input data, so this application ignores the delay of this step.

[0098] According to the above steps, in the first step, task B i (t) Transmission delay t generated from offloading from user equipment to edge server up , when the MEC edge server performs a task, assuming that it occupies all computing resources, the expression is as follows:

[0099]

[0100] Where: b i (t) is the amount of uploaded data for the offloading task; v i (t) is the transmission rate;

[0101] For the second step of offloading computation, the computing resources of the MEC edge server are limited, and all offloaded tasks cannot be processed at the same time. The tasks that arrive but are not processed enter the waiting queue. Assume that the time required in the edge server is the computation delay t of the MEC edge server. exe and the average waiting time of tasks in the queue The computation delay t of the MEC edge server exe Expressed as

[0102]

[0103] Where: d i (t) is the total amount of computation required at time t; fs,i (t) the real-time computing resource rate allocated to the edge server;

[0104] According to Little's law, each user device has task B i The average waiting time experienced by (t) is proportional to the average queue length of its task buffer in the MEC edge server. Therefore, the average queue length of the task buffer of each user device is used as one of the metrics of execution delay and can be expressed as

[0105]

[0106] This application uses Represents task A of the i-th user device i (t) The delay of the entire offloading computation process to the edge server can be expressed as

[0107]

[0108] This application uses represents the task A of the i-th user equipment in time slot τ i (t) The energy consumption generated during the entire offload computation process of migrating to the edge server mainly occurs during data transmission and task processing, and is expressed as follows:

[0109]

[0110] Among them, κ ser represents the effective switching capacitance of the CPU core on the MEC edge server, which is related to the chip architecture; up is the uplink transmission delay; t exe Calculate latency for MEC; is the average waiting time of the queue; is the total offloading delay; is the total energy consumption of unloading;.

[0111] In summary, the total energy consumption of a single-cell multi-user MEC system is:

[0112]

[0113] Where: E all is the total energy consumption of the system.

[0114] (5) Problem Model

[0115] The migration scheduling decision and resource allocation of the MEC system is regarded as an optimization problem, with the goal of minimizing the total energy consumption. In the MEC system, the task migration indicator Λ(t), the local CPU computing resource allocation f l(t) and the task scheduling decision C(t), task offloading and resource allocation schemes are formulated considering task priorities. The problem is stated as follows:

[0116]

[0117] Among them, constraint 1 indicates that the task can only be executed locally or offloaded to the edge server for execution, constraint 2 is that the total delay of the task must be less than the task delay threshold, constraint 3 is the constraint on the bandwidth allocation ratio, constraint 4 is the upper limit of the local CPU calculation cycle frequency, constraint 5 indicates that the number of CPU calculation cycles required for C(t) should not be greater than the total calculation cycles available for all CPUs in the MEC edge server, and constraint 6 indicates that the queue of the task buffer is stable, ensuring that all arriving computing tasks can be completed within a limited delay.

[0118] See Figure 3 This application uses the A3C algorithm to solve the problem model. The A3C algorithm has a central neural network global and n independent agents (the value of n is generally taken as the number of CPUs, and there are as many agents as there are CPUs). The global and the agents each maintain a network with exactly the same structure, and both have an Actor network and a Critic network. The agent obtains the experience gained in recent rounds through the actor-critic network and uploads it to the global network. The global aggregates the experience of all agents and sends it to each agent. The experience is the parameters of the actor and critic networks, but what is uploaded is the gradient obtained by differentiating the loss function with respect to the neural network parameters. The global obtains this gradient from the agent and updates its own network parameters; the downloaded experience is that the agent directly accepts all the neural network parameters in the global and overwrites its current actor and critic neural networks.

[0119] See Figure 4 The network structure of each agent is the same. Before each round, the operating parameters are obtained from the global network through the pull function. After the operation is completed, the parameters are returned to the global network through the push function to update the parameters of the global network.

[0120] In this module, we can obtain the state space and action space action. This part is combined with the model proposed in Chapter 3, s = (te, aw, af), action = (0, 1, 2, 3, 4, 5, 6, 7). Here, action corresponds to the action pair (ra, rc), where ra represents the bandwidth allocation ratio γ i (t), rc represents the computing resource allocation ratio f s,i(t), specifically: action 0 corresponds to (0.0, 0.0), action 1 corresponds to (0.0, 0.5), action 2 corresponds to (0.0, 1.0), action 3 corresponds to (0.5, 0.0), action 4 corresponds to (0.5, 0.5), action 5 corresponds to (0.5, 1.0), action 6 corresponds to (1.0, 0.0), action 7 corresponds to (1.0, 1.0); the corresponding relationship is as follows Figure 5-6 shown.

[0121] Allocated wireless bandwidth ratio γ i (t) and the computing resource allocation ratio f s,i (t);

[0122] The total energy consumption of the system E all As a reward signal; TD-error is used to update the parameters of the Actor-Critic network, achieving end-to-end joint optimization of offloading decisions and resource allocation. The Actor network and the Critic network are connected by TD-error, that is, the action value function of the strategy is subtracted from the value function of the strategy, which represents the evaluation of action a in state s. If behavior a is better than the average, it is positive, otherwise it is negative.

[0123] TD_error=Q(s,a)-V(s)=r+γV(s′)-V(s)

[0124] The input of the Actor network (policy network) is state state, and the output is the probability of all discrete actions act_prob. The loss function is -log(act_prob)*td_error. The input of the Critic network (value network) is state state, and the output is the state value v. The loss function is (td_error) 2 Where: TD_error is the temporal difference error; Q(s,a) is the state-action value function; V(s) is the state value function; r is the immediate reward, and E is the total energy consumption of the system. all ; γ is the discount factor.

[0125] Here, the MEC computing resources are set to 5 GHz / sec, the wireless bandwidth is 10 MHz, and other different system parameters are set. This application compares the DQN, WoLF-PHC, and A3C algorithms with the basic method of full local execution. The number of users (UEs) is a variable input, so the full local execution performs poorly. The initial curves of the three methods are almost the same before num_ue = 5. However, as the number of num_ue increases, the total overhead increases rapidly, and the three methods also show different performance, with A3C performing the best. Analysis shows that when the number of num_ue increases and offloading is selected, the MEC edge server with limited computing resources cannot provide sufficient computing resources for each user, but instead drags down overall performance. This shows that we should make trade-offs for users and reasonably choose to offload computing to ensure overall performance.

[0126] like Figure 7 As shown, W=10, num_ue=5, Figure 7 The horizontal axis represents the computing resource capacity of the MEC edge server, that is, the maximum computing power F, and the vertical axis represents the total system energy consumption. In this set of experiments, the number of user devices, num_ue, was fixed at 5. This set of experiments discussed the impact of the computing capacity of the MEC edge server on total energy consumption. The A3C method curve consistently remained at the bottom, demonstrating the best performance. WoLF-phc performed slightly better than DQN when F < 6, but then matched its performance. The stable curve for fully local computation indicates that local offloading is unaffected by MEC computing resources. This demonstrates that fully offloading computations is not feasible when computing resources are limited. However, when computing resources overflow, simple baseline methods may achieve good performance. Therefore, we must flexibly select algorithms to solve problems based on actual conditions.

[0127] Beneficial effects:

[0128] 1. At the business support level, by introducing a dynamic priority mechanism based on business latency tolerance and reliability requirements, the problem of difficulty in balancing the differentiated needs of diversified power services (such as distribution control, intelligent inspection, and load monitoring) is solved. This mechanism can automatically generate task processing queues based on business criticality, so that high-real-time and high-reliability services can obtain more sufficient resource guarantees, thereby more effectively meeting the stringent service quality requirements of millisecond-level control services such as distribution automation.

[0129] 2. At the resource optimization level, resource utilization efficiency is improved by jointly optimizing three key dimensions: task offloading decisions, wireless bandwidth allocation ratio, and edge server computing resource allocation. Especially in multi-user concurrent offloading scenarios, the priority-based dynamic proportional allocation mechanism can alleviate the competition between computing resources and wireless channels, giving priority to key services under limited resource conditions while reducing the overall energy consumption of the system. Compared with traditional methods, this mechanism shows better adaptability when facing an increase in the number of users or limited resources.

[0130] The distributed collaborative learning framework employed in this solution enhances the algorithm's ability to solve complex dynamic environments. By sharing experience and conducting collaborative training among agents, this framework can more efficiently respond to fluctuations in task loads and changes in channel states, improving the convergence speed and stability of policy solutions. This helps maintain a stable task queue, reduce task backlogs, and maintain overall system performance reliability and scalability during peak traffic or scale expansion, meeting the robust computing resource scheduling requirements of large-scale power IoT deployments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limiting. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in this field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.

Claims

1. A multi-task traffic offloading method for energy internet based on MEC, characterized in that: include: Establish a single-cell multi-user MEC network model including a task model, a queue model, a local computing model, and an offload computing model, and obtain the network status of the MEC network model; Construct a problem model for MEC system migration scheduling decision and resource allocation based on the MEC network model; Inputting the collected network status parameters into a reinforcement learning model, and using the reinforcement learning model to solve the problem model to obtain the offloading decision and the resource allocation and bandwidth allocation of the task to be offloaded; The offloading tasks are scheduled according to the offloading strategy, resource allocation and bandwidth allocation.

2. The method for offloading multi-task traffic of the energy internet based on MEC according to claim 1, characterized in that: Methods for establishing a single-cell multi-user MEC network model include: The edge server connected to the base station serves N user devices; assuming τ is a time slot, the time set is The user set is When a mobile device is within the coverage of a base station, each user device will generate a computationally intensive task to be completed within the time slot τ. The task generated by the i-th user device within the time slot τ is A i (t), the tasks generated by each user device are independent of each other, where: τ is the length of a single time slot; T is the set of discrete time slots; t is the current time slot index; N is the total number of user devices; i is the user device index; Use θ i (t)∈(0,1) is used as a task migration indicator, indicating that the task A generated by the i-th user equipment in the time slot τ i (t) Whether to migrate, the computation offloading strategy of all user devices in the time slot τ is used Indicates; θ i =1 indicates task A i (t) needs to be migrated to the edge server for execution, θ i =0 means task A i (t) Executed in the local CPU.

3. The method for offloading multi-task traffic of the energy internet based on MEC according to claim 2, characterized in that: The method for constructing the task model includes: Task A generated by the i-th user equipment in time slot τ i (t) consists of three parts: Among them, b i (t) represents the task A that needs to be performed i (t) is the data size in bits; d i (t) indicates the completion of task A i (t) The amount of computing resources required, expressed in terms of required CPU computing cycles; Represents Task A i (t) delay time threshold, that is, task A i The total time taken for (t) cannot exceed The three parameters of the task are related to the business characteristics and are estimated through the task profile; Using multi-priority task offloading based on QoS constraints, considering the two attributes of reliability and latency, the task priority calculation function is: Priority=α×Time+β×Reliability Where: Time is the delay attribute, which is the maximum tolerance value of the task processing event generated by the user device, expressed as t max , Reliability is the reliability attribute, and is the task transmission error rate ε i (t); α, β are priority weight coefficients, and α+β=1; Task A i The priority value of (r) is expressed as P r,i (t).

4. The method for offloading multi-task traffic of the energy internet based on MEC according to claim 3 is characterized in that: The method for constructing the queue model includes: The MEC edge server sets up a task cache for each mobile user device to store tasks that have been offloaded but not yet processed; there is a queue to store tasks waiting to be executed; Define the task B that offloads to the edge server i (t) = θ i (t)·A i (t), Q i (t) is the queue length of the i-th user device in the edge server in the time slot τ, C i (t) is the task scheduling decision of the MEC edge server in the time slot τ, that is, the number of tasks being processed by the i-th user device in the edge server in the time slot τ; then the queue length Q of the next time slot τ is i (y+1) is: Q i (t+1)=max{Q i (t)-C i (t),0}+B i (t),t∈{0,1,2,…} The task buffer is initially empty, i.e. Q i (0) = 0, i∈N; the MEC computing resources allocated to the task being processed by the i-th user device in the edge server in time slot t are: Where: f s,i (t) is the computing resource of the edge server; F ser is the total computing capacity of MEC; And satisfy the resource capacity constraint: ∑ i∈N C i (t) b i (t)·≤F ser τ, indicating completion The number of CPU cycles required for scheduling decisions does not exceed the total available computing cycles of the server; Where: C(r) is the scheduling decision vector; And the queue stability constraint is satisfied: Ensure that all arriving computational tasks are completed within a bounded delay.

5. The method for offloading multi-task traffic of the energy internet based on MEC according to claim 4 is characterized in that: The method for constructing the local computing model includes: The local CPU computation cycle frequency of the i-th user device is f i l , cannot exceed its maximum value Use collection represents the local CPU computing cycle frequency; the local execution power consumption of the i-th mobile device in, Indicates the energy consumption of each CPU cycle to complete the task; the local execution latency is:

6. The method for offloading multi-task traffic of the energy internet based on MEC according to claim 5, characterized in that: The method for constructing the offloading calculation model includes: Task A i (t) Transmission rate v offloaded to edge server i (t) = γ i (t)·BW·(log2(1+δ i (t))-ψ); where γ i (t) is task A i (t) The proportion of wireless bandwidth allocated during offloading to the edge server, Only the unloaded tasks are selected to participate in the allocation, that is, θ i (t)=1,0≤γ i (t)≤1 and∑ i∈N γ i (t) = 1; To uninstall task A i (t) Signal-to-noise ratio to edge server where p tx is the transmission power, g0 is the channel gain, and N0 is the additive Gaussian channel noise power; Reliability Factor Where: V τ is the channel dispersion, which can be approximated to 1; Q -1 (·) represents the inverse of the Gaussian Q function, ε i (t) represents task A i (t) Transmission error rate during offloading to edge servers; Average waiting delay in queues Assuming the computation delay of the edge server Task A i (t) The latency of the entire offloading process to the edge server Energy consumption for: Among them, κ ser is the CPU capacitance coefficient of the MEC edge server; t up is the uplink transmission delay; f s,i (t) the real-time computing resource rate allocated to the edge server; The overall cost of a single-cell multi-user MEC system is obtained based on the local execution energy consumption and offload execution energy consumption: Where: E all is the total energy consumption of the system.

7. The method for offloading multi-task traffic of the energy internet based on MEC according to claim 6, characterized in that: Methods for constructing a problem model for MEC system migration scheduling decision and resource allocation based on the MEC network model include: The problem model is constructed with the goal of minimizing total energy consumption as follows: The constraints defining the problem model are:

8. The method for offloading multi-task traffic of the energy internet based on MEC according to claim 7 is characterized in that: The reinforcement learning model is implemented using the A3C algorithm, which consists of a central neural network, the global, and n independent agents. The global and agents each maintain a network with exactly the same structure, consisting of an actor network and a critic network. The agent obtains the experience gained in recent rounds through the actor-critic network and uploads it to the global network. The global network aggregates the experience of all agents and distributes it to each agent. The network state parameters s = (te, aw, af) are input into the Actor network, where te represents the overall execution delay of the current task, aw represents the available wireless bandwidth resources, and af represents the available edge computing resources. The discrete action space action = {0, 1, 2, ..., 7} is output, and each action corresponds to a pair (ra, rc) of bandwidth and computing resource allocation ratios. ra represents the bandwidth allocation ratio γ i (t), rc represents the computing resource allocation ratio f s,i (t); Unloading decision θ i (t) is implicitly determined by the bandwidth allocation ratio, when γ i (t)>0θ i (t)=1, otherwise θ i (t) = 0; Allocated wireless bandwidth ratio γ i (t) and the computing resource allocation ratio f s,i (t); The total energy consumption of the system E all As a reward signal; update the Actor-Critic network parameters through TD-error, and optimize the offloading decision and resource allocation end-to-end. The temporal difference error is expressed as: TD_error=Q(s,a)-V(s)=r+γV(s′)-V(s) The policy network Actor input is state state, and the output is the probability of all discrete actions act_prob. The loss function is -log(act_prob)*td_error. The value network Critic input is state state, and the output is state value v. The loss function is (td_error) 2 ; Where: TD_error is the temporal difference error; Q(s,a) is the state-action value function; V(s) is the value estimate of the new state; r is the immediate reward, and E is the total energy consumption of the system all ; γ is the discount factor.

Images