Electric power Internet of Things edge computing task unloading and resource management method and system

Abstract

The invention relates to the field of artificial intelligence, in particular to an electric power Internet of Things edge computing task unloading and resource management method and system, and the method comprises the steps: constructing a signal model for describing the transmission characteristics when a task is unloaded to an edge server and a base station; constructing a cost model; based on the signal model and the cost model, a joint optimization objective function and constraint conditions are constructed, and the objective of the joint optimization objective function is to minimize the overall load of the first user equipment; and based on a deep reinforcement learning method, solving the joint optimization objective function to obtain an optimal unloading resource allocation strategy. According to the method, a deep learning method is introduced into a task scheduling scene, the optimal strategy can be learned through continuous interaction of the intelligent agent and the environment, calculation unloading and intelligent spectrum resource regulation and control under multi-device cooperation are achieved, and on the premise that the service quality is guaranteed, the system load is remarkably reduced, and the resource utilization efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a power internet of things edge computing task offloading and resource management method and system. BACKGROUND

[0002] With the intelligent development of energy systems and the continuous advancement of urbanization, the number of terminal devices connected in the power internet of things (PIoT) system grows exponentially. A large number of intelligent devices generate high-frequency data in power transmission, power transformation, power distribution, and power consumption, and require efficient and real-time computing processing capabilities. Although the traditional cloud computing architecture has strong processing capabilities, it is prone to data congestion, response delay, and other problems when facing time delay sensitive tasks in power grid operation and bandwidth limited scenarios, thereby reducing the overall system operation efficiency and stability.

[0003] To alleviate the cloud load pressure and improve the real-time performance of task processing, edge computing (Edge Computing) is widely introduced into the power internet of things system. Edge servers can be deployed at nodes close to terminals such as substations and power distribution rooms to provide intermediate layer computing support. However, due to the limited computing power and device resources of edge computing nodes, it is difficult to meet the low delay and high accuracy task processing requirements of critical devices (such as intelligent protection devices and fault monitoring modules) with a single offloading strategy and static resource allocation mechanism in a complex and variable network environment.

[0004] In addition, traditional task offloading and resource scheduling methods are mostly based on centralized optimization or heuristic algorithms, which are prone to local optimization, high training complexity, poor adaptability, and other problems when facing multi-user collaborative competition, dynamic changes in wireless interference, and other non-ideal factors. Especially in the case of sudden increase in task quantity or unstable network conditions, it is easy to cause the overall system computing load to rise, the communication resource allocation to be unbalanced, and thus affect the task execution efficiency and quality of service (QoS). SUMMARY

[0005] (I) Invention purpose The purpose of the present application is to provide a power internet of things edge computing task offloading and resource management method and system, which introduces a deep learning method into the task scheduling scene, can learn the optimal strategy through the continuous interaction of agents and the environment, realize the intelligent regulation and control of computing offloading and spectrum resources under the cooperation of multiple devices, and significantly reduce the system load and improve the resource utilization efficiency under the premise of ensuring the quality of service.

[0006] (II) Technical solutions To address the above problems, this invention provides a method for offloading and managing edge computing tasks and resources in the power Internet of Things (IoT), comprising: Construct a signal model to describe the transmission characteristics when tasks are offloaded to edge servers and base stations; A cost model is constructed, which includes a network operation sub-cost model for local computing, a network operation sub-cost model for task offloading to edge servers, and a network operation cost model for task offloading to base stations; Based on the signal model and cost model, a joint optimization objective function and constraints are constructed. The objective of the joint optimization objective function is to minimize the overall load of the first user equipment. Based on deep reinforcement learning, the joint optimization objective function is solved to obtain the optimal unloading resource allocation strategy.

[0007] In another aspect of the present invention, preferably, the signal model is represented by the following formula: Based on the aforementioned signal model, the received signal-to-interference-plus-noise ratio (SIR) of the base station and the edge server is expressed using the following formula: in Indicates the transmission power. Indicates the communication channel. Represents the uploaded task. The variance of additive white Gaussian noise is represented by m, where m represents the user ID and e represents the edge server. This indicates the signal received by the edge server. This represents the signal power received by the edge server from the m-th user. This represents the channel from user m to the edge server. This indicates the signal uploaded by user m to the edge server. Indicates user The signal power uploaded to the edge server Indicates user Channel to the edge server Indicates user The signal uploaded to the edge server, where 'c' represents the base station. This indicates the signal received by the base station. This represents the signal power received by the base station from the m-th user. This represents the channel from user m to the base station. This represents the signal uploaded by user m to the base station. Indicates user The signal power uploaded to the base station, Indicates user Channel to base station Indicates user The signal is uploaded to the base station; The received signal-to-interference-plus-noise ratio (SIR) of the edge server. This represents the received signal-to-interference-plus-noise ratio (SIR) of the base station.

[0008] In another aspect of the present invention, preferably, the locally computed network operation sub-cost model is expressed using the following formula: in, This represents the network operating cost of local computing. Indicates the effective switching capacitor of the CPU. Indicates power Internet of Things (IoT) devices Time slot The periodic frequency of time, Indicates the first The number of CPU cycles required for an application in a power IoT device to execute 1 bit of data. This indicates the local queue to be executed. This indicates the cost of local computation.

[0009] In another aspect of the present invention, preferably, the network operation sub-cost model for task offloading to the edge server includes a first link transmission cost and an edge server usage cost, wherein the first link transmission cost is expressed using the following formula: in, Indicates the transmission cost of the first link. Indicates in time slot Internal users Select the Unload the edge servers. For users signal transmission power, This represents the total number of tasks offloaded to the edge server. Indicates the transmission rate; The cost of using the edge server is expressed using the following formula: in, This indicates the cost of using edge servers. Indicates the time slot length. Indicates time slot No. An edge IoT device is assigned to the user. Computing resources.

[0010] In another aspect of the present invention, preferably, the network operation sub-cost model for task offloading to the base station includes the second link transmission cost and the base station's computing cost; The transmission cost of the second link is expressed using the following formula: in, Indicates the transmission cost of the second link. for Fixed transmission rate to BS, The amount of work to be unloaded to BS; in, This represents the computing cost of the base station. Assigned to BS computing resources Indicates the time slot length.

[0011] In another aspect of the present invention, preferably, the joint optimization objective function and constraints are expressed using the following formula: Where T represents the total transmission time, M represents the total number of users, and C1 and C2 are the power upload constraints of the power IoT terminals. This represents the signal power received by the edge server from the m-th user. This represents the maximum signal power received by the edge server from the m-th user. This represents the signal power received by the base station from the m-th user. This represents the maximum signal power received by the base station from the m-th user; C3 and C4 are computing power constraints. Indicates that the edge server is assigned to Computing resources This indicates that edge servers can be assigned to The maximum computing power; Assigned to BS Computing resources This indicates that BS can be assigned to The maximum computing power; C5 and C6 represent upload latency constraints, respectively; This indicates the total number of uninstalled tasks. This indicates the transmission rate of the edge server. This indicates the maximum receive latency of the edge server. This indicates the amount of tasks unloaded to the BS (Browser / Server) area. Indicates the transmission rate of the BS. C7 and C8 indicate the maximum reception delay of the BS; C7 and C8 indicate that there will be no interruption during signal transmission. This is expressed as the received signal-to-interference-plus-noise ratio (SIR / NDR) of the edge server. This is expressed as the minimum received signal-to-interference-plus-noise ratio (SIN / NDR) for the edge server. This is expressed as the received signal-to-interference-plus-noise ratio (SIR) of the BS. C9 represents the minimum received signal-to-interference-plus-noise ratio (SIN / NDR) for the BS; C9 is the computational task constraint. These represent the network operation cost model for local computing, the network operation cost model for task offloading to edge servers, and the network operation cost model and cost model for task offloading to base stations, respectively.

[0012] In another aspect of the present invention, preferably, the deep reinforcement learning method is based on a multi-agent deep Q-network; Based on deep reinforcement learning, the joint optimization objective function is solved to obtain the optimal offloading resource allocation strategy, including: Set the state space, action space, and reward function of the multi-agent deep Q-network; The input information of the state space includes task data volume, computing power information, channel gain, interference information, and QoS indicators. The action space includes several actions, including unloading mode, spectrum sub-band, and transmit power; The reward function is constructed based on the objective of minimizing the overall load and satisfying QoS. Based on the reward function, calculate the Q value corresponding to each time slot input information and several actions, and determine the action with the largest Q value as the optimal unloading resource allocation strategy.

[0013] In another aspect of the present invention, preferably, the reward function is expressed using the following formula: in, Represents the reward function, This represents the network operating cost of power Internet of Things (IoT) device m. This represents the network operating cost of local computing. This indicates the power penalty for edge servers. Indicates the BS power penalty term. This represents the signal-to-interference-plus-noise ratio (SIR) penalty term for edge servers. This indicates the BS signal-to-interference-to-noise ratio penalty term.

[0014] In another aspect of the present invention, preferably, the multi-agent deep Q-network is trained using a preset training network and a target network, wherein the preset training network and the target network have the same deep neural network structure; The training updates the parameters of the training network using the stochastic gradient descent method and optimizes the training network based on the least squares loss function, so that the output value of the training network approximates the target network value.

[0015] In another aspect of the present invention, a preferred power Internet of Things (IoT) edge computing task offloading and resource management system includes: First building module: Building a signal model, which is used to describe the transmission characteristics when tasks are offloaded to edge servers and base stations; The second building module: building a cost model, which includes a network operation sub-cost model for local computing, a network operation sub-cost model for task offloading to edge servers, and a network operation sub-cost model for task offloading to base stations; The third construction module: Based on the signal model and cost model, a joint optimization objective function and constraints are constructed. The objective of the joint optimization objective function is to minimize the overall load of the first user equipment. Solution module: Based on deep reinforcement learning, the joint optimization objective function is solved to obtain the optimal unloading resource allocation strategy.

[0016] (III) Beneficial Effects The above-described technical solution of the present invention has the following beneficial technical effects: This invention, by constructing signal and cost models, comprehensively characterizes the transmission characteristics and resource consumption of tasks under different offloading modes, ensuring the accuracy and completeness of the modeling. Through intelligent decision-making, it rationally allocates computing and communication resources, effectively reducing the overall computing and communication burden on critical user equipment. While ensuring QoS requirements, it improves the success rate and response speed of task offloading, and shortens task completion latency. It maintains good performance under dynamic channel environments, varying task loads, and interference conditions, fully leveraging the computing power advantages of local devices, edge servers, and base stations to achieve efficient resource scheduling and utilization. Attached Figure Description

[0017] Figure 1 This is an overall flowchart of one embodiment of the present invention; Figure 2 This is a model diagram of a power Internet of Things (IoT) task offloading and resource allocation system according to an embodiment of the present invention; Figure 3 This is a training process and reward value convergence curve according to an embodiment of the present invention; Figure 4 This is a graph showing the relationship between the overall system computing load and the number of critical devices in one embodiment of the present invention; Figure 5 This is a trend graph showing the change in the unloading mode selection ratio with the number of CMTCDs according to an embodiment of the present invention; Figure 6 These are the system energy efficiency change curves for different algorithms under the condition of increasing number of critical equipment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0019] The accompanying drawings illustrate a layer structure according to an embodiment of the present invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0020] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0021] In the description of this invention, it should be noted that the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0022] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0023] The invention will now be described in more detail with reference to the accompanying drawings. In the various drawings, the same elements are indicated by similar reference numerals. For clarity, the various parts in the drawings are not drawn to scale.

[0024] Example 1 A method for offloading and managing edge computing tasks and resources in the power Internet of Things (IoT) Figure 1 A schematic diagram of the overall process of one embodiment of the present invention is shown. Figure 2 A model diagram of a power Internet of Things (IoT) task offloading and resource allocation system according to an embodiment of the present invention is shown, as follows: Figure 1 and Figure 2 As shown, it includes: A signal model is constructed to describe the transmission characteristics when tasks are offloaded to edge servers and base stations. By modeling the wireless transmission process between user equipment and edge servers and base stations in the power Internet of Things, the communication quality and link status under different offloading methods can be accurately described for subsequent optimization calculations.

[0025] When a terminal needs to offload a task to an edge server, the terminals in the cluster upload the task to the edge server within the same physical resource block; similarly, when offloading a task to a base station (BS), the terminals in the cluster upload the task to a different physical resource block. For ease of processing, it is assumed that the edge server and the base station (BS) can receive information from other users. At this point, the [missing information - likely a specific step or condition] when uploading a task to the edge server... The signal of a user can be represented as: Similarly, when uploading the task to the base station (BS), the first... The signal of a user can be represented as: in Indicates the transmission power. Indicates the communication channel. Represents the uploaded task. The variance of additive white Gaussian noise is represented by m, where m represents the user ID and e represents the edge server. This indicates the signal received by the edge server. This represents the signal power received by the edge server from the m-th user. This represents the channel from user m to the edge server. This indicates the signal uploaded by user m to the edge server. Indicates user The signal power uploaded to the edge server Indicates user Channel to the edge server Indicates user The signal uploaded to the edge server, where 'c' represents the base station. This indicates the signal received by the base station. This represents the signal power received by the base station from the m-th user. This represents the channel from user m to the base station. This represents the signal uploaded by user m to the base station. Indicates user The signal power uploaded to the base station, Indicates user Channel to base station Indicates user The signal is uploaded to the base station. At this point, the received signal-to-interference-plus-noise ratio (SIR) of the base station and the edge server can be written as: in The received signal-to-interference-plus-noise ratio (SIR) of the edge server. This represents the received signal-to-interference-plus-noise ratio (SIR) of the base station.

[0026] A cost model is constructed, comprising a network operation sub-cost model for local computation, a network operation sub-cost model for task offloading to edge servers, and a network operation sub-cost model for task offloading to base stations. The ultimate goal of offloading and optimizing resources for tasks generated by power IoT devices is to minimize the long-term network operation cost. The network operation cost is defined as three parts: local network computation cost, i.e., the computing energy consumption, latency overhead, and storage usage consumed when user equipment completes tasks locally; network operation cost for task offloading to edge servers, i.e., the transmission energy consumption, transmission latency, computing resource usage, and scheduling overhead incurred when tasks are transmitted to edge servers for execution via wireless links; and network operation cost for task offloading to base stations, i.e., the communication overhead, computing latency, energy consumption, and potential queue waiting latency when tasks are computed via base stations. The cost model can calculate comparable quantitative indicators between various transmission paths. In this embodiment, the network operation sub-cost model for local computation is expressed using the following formula: in, This represents the network operating cost of local computing. Indicates the effective switching capacitor of the CPU. Indicates power Internet of Things (IoT) devices Time slot The periodic frequency of time, Indicates the first The number of CPU cycles required for an application in a power IoT device to execute 1 bit of data. This indicates the local queue to be executed. This indicates the cost of local computation.

[0027] The network operation cost model for offloading tasks to edge servers includes the first link transmission cost and the edge server usage cost. The first link transmission cost is expressed using the following formula: in, Indicates the transmission cost of the first link. Indicates in time slot Internal users Select the Unload the edge servers. For users signal transmission power, This represents the total number of tasks offloaded to the edge server. Indicates the transmission rate; The cost of using the edge server is expressed using the following formula: in, This indicates the cost of using edge servers. Indicates the time slot length. Indicates time slot No. An edge IoT device is assigned to the user. Computing resources.

[0028] The network operation sub-cost model for offloading tasks to the base station includes the second link transmission cost and the base station's computing cost; The transmission cost of the second link is expressed using the following formula: in, Indicates the transmission cost of the second link. for Fixed transmission rate to BS, The amount of work to be unloaded to BS; in, This represents the computing cost of the base station. Assigned to BS computing resources Indicates the time slot length.

[0029] In the time slot At that time, power Internet of Things devices The network operating cost is: in, This represents the network operating cost of power Internet of Things (IoT) device m. This represents the network operating cost of local computing. This represents the network operating cost of offloading tasks to edge servers. This represents the network operating cost of offloading tasks to the base station.

[0030] Based on the aforementioned signal and cost models, a joint optimization objective function and constraints are constructed. The objective of the joint optimization objective function is to minimize the overall load of the first user equipment. Critical machine user equipment performing computationally intensive tasks can achieve lower offloading latency by choosing M2M offloading, resulting in a shorter offloading distance. However, the local machine device gateway has fewer computing resources than the edge cloud. Furthermore, when critical machine user equipment within the same cluster all choose M2M offloading mode, these users will share the computing resources available to the local machine device gateway, resulting in a higher load and impacting the user's quality of service. In this embodiment, through joint optimization of power allocation, computing resource allocation, and power IoT task allocation, the overall load of all critical machine user equipment with computationally intensive tasks in the network is minimized, while simultaneously meeting the QoS requirements of other user types. The overall load can be defined as the weighted sum of computational energy consumption and latency, or a comprehensive indicator of energy efficiency and latency. Constraints include, but are not limited to, latency constraints requiring task computation to be completed within a specified time limit, edge server and base station computing resource capacity constraints, wireless link bandwidth constraints, and user equipment battery energy consumption constraints. These constraints ensure that the optimization results are feasible and engineering-problem-proof.

[0031] The joint optimization objective function and constraints are expressed by the following formula: Where T represents the total transmission time, M represents the total number of users, and C1 and C2 are the power upload constraints of the power IoT terminals. This represents the signal power received by the edge server from the m-th user. This represents the maximum signal power received by the edge server from the m-th user. This represents the signal power received by the base station from the m-th user. This represents the maximum signal power received by the base station from the m-th user; C3 and C4 are computing power constraints. Indicates that the edge server is assigned to Computing resources This indicates that edge servers can be assigned to The maximum computing power; Assigned to BS Computing resources This indicates that BS can be assigned to The maximum computing power; C5 and C6 represent upload latency constraints, respectively; This indicates the total number of uninstalled tasks. This indicates the transmission rate of the edge server. This indicates the maximum receive latency of the edge server. This indicates the amount of tasks unloaded to the BS (Browser / Server) area. Indicates the transmission rate of the BS. C7 and C8 indicate the maximum reception delay of the BS; C7 and C8 indicate that there will be no interruption during signal transmission. This is expressed as the received signal-to-interference-plus-noise ratio (SIR / NDR) of the edge server. This is expressed as the minimum received signal-to-interference-plus-noise ratio (SIN / NDR) for the edge server. This is expressed as the received signal-to-interference-plus-noise ratio (SIR) of the BS. C9 represents the minimum received signal-to-interference-plus-noise ratio (SIN / NDR) for the BS; C9 is the computational task constraint. These represent the network operation cost model for local computing, the network operation cost model for task offloading to edge servers, and the network operation cost model and cost model for task offloading to base stations, respectively.

[0032] Based on deep reinforcement learning, the joint optimization objective function is solved to obtain the optimal offloading resource allocation strategy. Since the optimization problem is complex, to avoid the curse of dimensionality problem faced by traditional reinforcement learning methods, such as Q-learning, which prevents each agent from obtaining enough samples to traverse every state and action choice, resulting in poor algorithm performance or even non-convergence, a deep neural network is used to estimate the Q-value, rather than traversing and calculating all Q-functions.

[0033] The deep reinforcement learning method is based on a multi-agent deep Q-network; Based on deep reinforcement learning, the joint optimization objective function is solved to obtain the optimal offloading resource allocation strategy, including: Set the state space, action space, and reward function of the multi-agent deep Q-network; The input information of the state space includes task data volume, computing power information, channel gain, interference information, and QoS indicators; in the defined computing task offloading scenario, a single agent in a time slot The system environment's state observations mainly consist of its own computational task state information, computing power information, local machine energy efficiency, local machine gateway computing power information, channel gain information, interference information, and QoS indicator information of other users in the network, defined as follows: in, The amount of data for the computation task in time slot t; The computing power available for power IoT devices, edge servers, and BS; Channel gain information for each device; Interference experienced by power Internet of Things (IoT) devices; For maximum delay, For the training iteration at this time; Let be the exploration probability of the greedy strategy.

[0034] The action space includes several actions, which include offloading mode, spectrum subband, and transmit power. The joint task offloading and resource allocation scheme involves the selection of computational offloading mode, spectrum subband, and transmit power allocation for critical machine user equipment. Therefore, each agent will, based on the above state information, in the time slot... use - Greedy strategy, based on probability Randomly select the calculation offload mode, spectrum subband, and transmit power ratio, or use probability. We select the optimal action selection strategy that maximizes the state-action value function at the current moment, and define the action (resource) set as follows: The reward function is constructed based on minimizing the overall load and satisfying QoS. In the joint task offloading and resource allocation problem described in the optimization problem, there are two objectives: minimizing the overall load of all critical machine user equipment with computationally intensive tasks in the network; and simultaneously satisfying the QoS requirements of other types of users. Since the goal of the reinforcement learning method in each time slot is to maximize the immediate reward, in this embodiment, the reward function is expressed using the following formula: in, Represents the reward function, This represents the network operating cost of power Internet of Things (IoT) device m. This represents the network operating cost of local computing. This indicates the power penalty for edge servers. Indicates the BS power penalty term. This represents the signal-to-interference-plus-noise ratio (SIR) penalty term for edge servers. This indicates the BS signal-to-interference-to-noise ratio penalty term.

[0035] Based on the reward function, calculate the Q value corresponding to each time slot input information and several actions, and determine the action with the largest Q value as the optimal unloading resource allocation strategy.

[0036] Furthermore, in this embodiment, the multi-agent deep Q-network is trained using a preset training network and a target network, wherein the preset training network and the target network have the same deep neural network structure. The training updates the parameters of the training network using the stochastic gradient descent method and optimizes the training network based on the least squares loss function, so that the output value of the training network approximates the target network value.

[0037] Training iterations begin with a random initial environment state (determined by the user's initial location, initial channel state information, initial computational task size, computational resources, etc.). In each time slot of each iteration, the user's computational task size and small-scale channel fading between links cause changes in the environment state, prompting each agent to adjust its action selection strategy.

[0038] The distributed joint task offloading and resource allocation algorithm based on multi-agent DQN utilizes asynchronously updated Q-networks and a target Q-network, along with an experience replay pool, to train multiple agents to learn joint task offloading and resource allocation strategies. Each agent has two identical DQNs, referred to as the training network and the target network, respectively. The agent's state value in each time slot serves as the input to the training network, and the output of the training network is the value function of all action choices in that state. To train the agents, two networks are set up: a training network and a target network. The agent's state value in each time slot serves as the input to the training network, and the value function serves as the output. The state space and action space of each time slot are stored in the experience pool. The network parameters are updated using stochastic gradient descent. The least squares function for training the network is: For the target network value In each time slot, the environment changes due to variations in computational task size, channel conditions, and the different actions taken by all agents. Each agent represents this environment transition as a tuple. And it is stored in its own experience replay pool. Figure 3 The training process and reward value convergence curve of one embodiment of the present invention are shown, as follows: Figure 3 As shown, this paper illustrates the performance trends of different numbers of key terminal devices during model training. By randomly and repeatedly drawing small batches of data from the experience replay pool to train the neural network parameters, not only is the utilization rate of experience sample data improved, but the strong correlation between continuous experience data samples is also broken, thereby stabilizing the learning process. Table 1 shows the algorithm of the training process; Table 1 Algorithm for Training Process Randomly initialize the DQN parameters for each agent. and Initialize the size of the experience replay pool for each agent. for to do Reset network environment for to do for to do intelligent agent Obtain environmental state observations intelligent agent according to Greedy strategy for selecting actions intelligent agent After executing the action, obtain the environmental state observation value for the next time slot. endfor All agents will receive a shared reward value after completing their selected actions. Regenerate the computational task for each agent and update the small-scale fading of the channel. for to do intelligent agent storage To its own experience replay pool endfor endfor for to do intelligent agent Randomly select a size from the replay pool of one's own experience. small batch experience samples intelligent agent Update the trained network parameters by minimizing the loss function. endfor if for to do intelligent agent Update its own target network parameters to make endfor end if During the implementation phase, in each time slot, the agent selects the action that maximizes its Q-value, as provided by DQN, based on the state data it receives. Furthermore, in this phase, the state data, iteration count, and greed factor for each time slot are set to the values ​​from the last round of training. After action selection, all agents communicate using their chosen computational task offloading mode, spectral subband, and transmit power values. Figure 4 This diagram illustrates the relationship between the overall system computing load and the number of critical devices according to an embodiment of the present invention; as shown... Figure 4 As shown, the system load levels under multi-task density are compared by different offloading strategies. Figure 5 The graph shows a trend of the unloading mode selection ratio as a function of the number of CMTCDs according to an embodiment of the present invention. Figure 5 As shown, the percentage changes of local computing, M2M offloading, and edge cloud offloading under different load conditions. Figure 6 The system energy efficiency variation curves of different algorithms under the condition of increasing number of critical equipment are shown, such as... Figure 6 As shown, this embodiment demonstrates its resource adaptability and energy efficiency advantages in high-density scenarios.

[0039] This invention, by constructing signal and cost models, comprehensively characterizes the transmission characteristics and resource consumption of tasks under different offloading modes, ensuring the accuracy and completeness of the modeling. Through intelligent decision-making, it rationally allocates computing and communication resources, effectively reducing the overall computing and communication burden on critical user equipment. While ensuring QoS requirements, it improves the success rate and response speed of task offloading, and shortens task completion latency. It maintains good performance under dynamic channel environments, varying task loads, and interference conditions, fully leveraging the computing power advantages of local devices, edge servers, and base stations to achieve efficient resource scheduling and utilization.

[0040] Example 2 A power Internet of Things (IoT) edge computing task offloading and resource management system, comprising: First building module: Building a signal model, which is used to describe the transmission characteristics when tasks are offloaded to edge servers and base stations; The second building module: building a cost model, which includes a network operation sub-cost model for local computing, a network operation sub-cost model for task offloading to edge servers, and a network operation sub-cost model for task offloading to base stations; The third construction module: Based on the signal model and cost model, a joint optimization objective function and constraints are constructed. The objective of the joint optimization objective function is to minimize the overall load of the first user equipment. Solution module: Based on deep reinforcement learning, the joint optimization objective function is solved to obtain the optimal unloading resource allocation strategy.

[0041] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

[0042] The above description does not provide detailed explanations of the technical aspects of each layer's patterning and etching. However, those skilled in the art should understand that various methods existing in the prior art can be used to form layers and regions of the desired shape. Furthermore, to form the same structure, those skilled in the art can also design methods that are not entirely identical to those described above.

[0043] The present invention has been described above with reference to embodiments thereof. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

[0044] Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions, and modifications can be made to the embodiments of the present invention without departing from the spirit and scope of the invention.

[0045] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for offloading and managing edge computing tasks and resources in the power Internet of Things, characterized in that, include: Construct a signal model to describe the transmission characteristics when tasks are offloaded to edge servers and base stations; A cost model is constructed, which includes a network operation sub-cost model for local computing, a network operation sub-cost model for task offloading to edge servers, and a network operation cost model for task offloading to base stations; Based on the signal model and cost model, a joint optimization objective function and constraints are constructed. The objective of the joint optimization objective function is to minimize the overall load of the first user equipment. Based on deep reinforcement learning, the joint optimization objective function is solved to obtain the optimal unloading resource allocation strategy.

2. The method for offloading and managing edge computing tasks in the power Internet of Things according to claim 1, characterized in that, The signal model is represented by the following formula: Based on the aforementioned signal model, the received signal-to-interference-plus-noise ratio (SIR) of the base station and the edge server is expressed using the following formula: in Indicates the transmission power. Indicates the communication channel. Represents the uploaded task. The variance of additive white Gaussian noise is represented by m, where m represents the user ID and e represents the edge server. This indicates the signal received by the edge server. This represents the signal power received by the edge server from the m-th user. This represents the channel from user m to the edge server. This indicates the signal uploaded by user m to the edge server. Indicates user The signal power uploaded to the edge server Indicates user Channel to the edge server Indicates user The signal uploaded to the edge server, where 'c' represents the base station. This indicates the signal received by the base station. This represents the signal power received by the base station from the m-th user. This represents the channel from user m to the base station. This represents the signal uploaded by user m to the base station. Indicates user The signal power uploaded to the base station, Indicates user Channel to base station Indicates user The signal is uploaded to the base station; The received signal-to-interference-plus-noise ratio (SIR) of the edge server. This represents the received signal-to-interference-plus-noise ratio (SIR) of the base station.

3. The method for offloading and managing edge computing tasks in the power Internet of Things according to claim 2, characterized in that, The locally computed network operation cost model is expressed using the following formula: in, This represents the network operating cost of local computing. Indicates the effective switching capacitor of the CPU. Indicates power Internet of Things (IoT) devices Time slot The periodic frequency of time, Indicates the first The number of CPU cycles required for an application in a power IoT device to execute 1 bit of data. This indicates the local queue to be executed. This indicates the cost of local computation.

4. The method for offloading and managing edge computing tasks in the power Internet of Things according to claim 3, characterized in that, The network operation cost model for offloading tasks to edge servers includes the first link transmission cost and the edge server usage cost. The first link transmission cost is expressed using the following formula: in, This represents the transmission cost of the first link. Indicates in time slot Internal users Select the Unload the edge servers. For users signal transmission power, This represents the total number of tasks offloaded to the edge server. Indicates the transmission rate; The cost of using the edge server is expressed using the following formula: in, This indicates the cost of using edge servers. Indicates the time slot length. Indicates time slot No. An edge IoT device is assigned to the user. Computing resources.

5. The method for offloading and managing edge computing tasks in the power Internet of Things according to claim 4, characterized in that, The network operation sub-cost model for offloading tasks to the base station includes the second link transmission cost and the base station's computing cost; The transmission cost of the second link is expressed using the following formula: in, Indicates the transmission cost of the second link. for Fixed transmission rate to BS, The amount of work to be unloaded to BS; in, This represents the computing cost of the base station. Assigned to BS computing resources Indicates the time slot length.

6. The method for offloading and managing edge computing tasks in the power Internet of Things according to claim 5, characterized in that, The joint optimization objective function and constraints are expressed by the following formula: Where T represents the total transmission time, M represents the total number of users, and C1 and C2 are the power upload constraints of the power IoT terminals. This represents the signal power received by the edge server from the m-th user. This represents the maximum signal power received by the edge server from the m-th user. This represents the signal power received by the base station from the m-th user. This represents the maximum signal power received by the base station from the m-th user; C3 and C4 are computing power constraints. Indicates that the edge server is assigned to Computing resources This indicates that edge servers can be assigned to The maximum computing power; Assigned to BS Computing resources This indicates that BS can be assigned to The maximum computing power; C5 and C6 represent upload latency constraints, respectively; This indicates the total number of uninstalled tasks. This indicates the transmission rate of the edge server. This indicates the maximum receive latency of the edge server. This indicates the amount of tasks unloaded to the BS (Browser / Server) area. Indicates the transmission rate of the BS. C7 and C8 indicate the maximum reception delay of the BS; C7 and C8 indicate that there will be no interruption during signal transmission. This is expressed as the received signal-to-interference-plus-noise ratio (SIR / NDR) of the edge server. This is expressed as the minimum received signal-to-interference-plus-noise ratio (SIN / NDR) for the edge server. This is expressed as the received signal-to-interference-plus-noise ratio (SIR) of the BS. C9 represents the minimum received signal-to-interference-plus-noise ratio (SIN / NDR) for the BS; C9 is the computational task constraint. These represent the network operation cost model for local computing, the network operation cost model for task offloading to edge servers, and the network operation cost model and cost model for task offloading to base stations, respectively.

7. The method for offloading and managing edge computing tasks in the power Internet of Things according to claim 5, characterized in that, The deep reinforcement learning method is based on a multi-agent deep Q-network; Based on deep reinforcement learning, the joint optimization objective function is solved to obtain the optimal offloading resource allocation strategy, including: Set the state space, action space, and reward function of the multi-agent deep Q-network; The input information of the state space includes task data volume, computing power information, channel gain, interference information, and QoS indicators. The action space includes several actions, including unloading mode, spectrum sub-band, and transmit power; The reward function is constructed based on the objective of minimizing the overall load and satisfying QoS. Based on the reward function, calculate the Q value corresponding to each time slot input information and several actions, and determine the action with the largest Q value as the optimal unloading resource allocation strategy.

8. The method for offloading and managing edge computing tasks in the power Internet of Things according to claim 7, characterized in that, The reward function is expressed using the following formula: in, Represents the reward function, This represents the network operating cost of power Internet of Things (IoT) device m. This represents the network operating cost of local computing. This indicates the power penalty for edge servers. Indicates the BS power penalty term. This represents the signal-to-interference-plus-noise ratio (SIR) penalty term for edge servers. This indicates the BS signal-to-interference-to-noise ratio penalty.

9. The method for offloading and managing edge computing tasks in the power Internet of Things according to claim 7, characterized in that, The multi-agent deep Q-network is trained using a preset training network and a target network, and the preset training network and the target network have the same deep neural network structure. The training updates the parameters of the training network using the stochastic gradient descent method and optimizes the training network based on the least squares loss function, so that the output value of the training network approximates the target network value.

10. A power Internet of Things (IoT) edge computing task offloading and resource management system, characterized in that, include: First building module: Building a signal model, which is used to describe the transmission characteristics when tasks are offloaded to edge servers and base stations; The second building module: building a cost model, which includes a network operation sub-cost model for local computing, a network operation sub-cost model for task offloading to edge servers, and a network operation sub-cost model for task offloading to base stations; The third construction module: Based on the signal model and cost model, a joint optimization objective function and constraints are constructed. The objective of the joint optimization objective function is to minimize the overall load of the first user equipment. Solution module: Based on deep reinforcement learning, the joint optimization objective function is solved to obtain the optimal unloading resource allocation strategy.

Images