Resource-aware intelligent edge gateway system and operation method

Abstract

The invention belongs to the technical field of edge computing, and discloses a resource-aware intelligent edge gateway system and an operation method. The intelligent edge gateway comprises a user equipment service and protocol conversion unit, a computing resource sensing unit, a gateway equipment service unit, a wireless charging unit, an equipment management and control unit, a data storage unit, a computing task unloading decision unit and a task computing unit. Energy, calculation and transmission resource information of user equipment and gateway equipment is obtained through a resource sensing technology, a deep reinforcement learning algorithm is combined, the information is applied to unloading decision and resource allocation of a system, and charging and unloading time, an unloading strategy and resource allocation are continuously optimized through learning, so that the resource allocation efficiency is improved. And in combination with multi-region intelligent edge gateway cooperative computing, the time delay of a computing task and the power consumption of the user equipment are reduced, and the user requirements are met.

Description

Technical Field

[0001] The present invention belongs to the field of edge computing technology, and specifically relates to a resource-aware intelligent edge gateway system and an operation method. Background Art

[0002] Recent developments in IoT technology have enabled the intelligent and autonomous control of many important industrial and commercial systems, such as smart power grids, unmanned farms, and smart home automation, greatly improving work efficiency and saving labor costs. However, IoT devices are limited by strict device size and production costs, and usually carry limited-capacity batteries and energy-saving low-performance processors, which makes them unable to support more and more new applications that require high-performance computing. These applications also have the same characteristics, that is, they generate massive amounts of real-time data that need to be processed, which brings serious delays and network congestion to IoT systems. To solve the above problems, academia and industry have begun to pay attention to mobile edge computing (MEC) technology, which integrates servers on wireless access points and base stations to achieve a large number of computing tasks on large-scale low-power devices. On the other hand, wireless power transmission (WPT) technology solves the problem of limited battery capacity by deploying dedicated energy transmitters to charge devices equipped with remote energy harvesting in the form of wireless broadcast energy. At present, wireless power mobile edge computing (WPMEC), which combines the above two technologies, has become a new paradigm for realizing self-sustainable mobile computing for delay-sensitive and computation-intensive applications. In a typical wireless charging mobile edge computing system, a hybrid access point (AP) integrating a mobile edge computing server and an energy transmitter can wirelessly charge a large number of low-power wireless devices such as sensors and wearable devices, and these devices rely on the harvested energy to perform local computing or offload tasks to the hybrid access point (AP) to perform computing tasks.

[0003] Current research on wireless charging edge computing mainly focuses on the task offloading and resource allocation problems between a single wireless charging edge server and multi-user devices. Many solutions have been proposed for indicators such as latency and energy consumption, such as user cooperation, drone assistance, and reflective surface assistance. However, wireless charging mobile edge computing still has problems of unbalanced resource allocation and insufficient energy utilization. In order to meet the task latency requirements, a large number of tasks are offloaded to edge servers. Due to the lack of collaboration between individual servers, the server load is overloaded, even exceeding the maximum computing capacity, resulting in the phenomenon of discarding tasks, which will greatly affect the stability and efficiency of the entire system. On the other hand, due to the limitations of the half-duplex characteristics of user devices, most of the current solutions mainly consider the balance between indicators such as computing latency and overall power consumption under the condition of full consumption of collected energy, so as to minimize the overall latency indicator or power consumption indicator. In particular, when facing a surge in data volume, some devices may be unable to perform local computing or task offloading due to insufficient energy acquisition. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a resource-aware intelligent edge gateway system and operation method composed of multiple gateway devices and multiple users. The energy, computing and transmission resource information of user devices and gateway devices are obtained through resource perception technology, and combined with a deep reinforcement learning algorithm, it is applied to the unloading decision and resource allocation of the system. Through learning, the charging and unloading time, unloading strategy and resource allocation are continuously optimized, and the intelligent edge gateways in multiple regions are combined for collaborative computing to reduce the latency of computing tasks and the power consumption of user devices to meet user needs.

[0005] In order to achieve the above object, the present invention is achieved through the following technical solutions:

[0006] The present invention is a resource-aware intelligent edge gateway system, which includes a plurality of wireless charging intelligent edge gateways, wherein the intelligent edge gateway includes a user device service and protocol conversion unit, a computing resource awareness unit, a gateway device service unit, a wireless charging unit, a device management and control unit, a data storage unit, a computing task offloading decision unit, and a task computing unit;

[0007] The user device service and protocol conversion unit is used to connect user devices of multiple communication protocols, and perform data identification, data analysis, and data encapsulation on the original data input by the user device, convert it into data of the specified type and structure of the intelligent edge gateway, and send the parameters of the computing task in the data to the computing resource perception unit. At the same time, the user device service and protocol conversion unit sends the device information when the user device is connected and the device power perceived at a fixed time. Send to the device management and control unit;

[0008] The computing resource sensing unit is responsible for sending a gateway computing resource sensing command to the gateway device service unit after receiving the computing task request of the first device, the computing resource usage and communication status information of other intelligent edge gateways in the resource-aware intelligent edge gateway system, and forwarding the device information and gateway information to the computing task offloading decision unit;

[0009] The gateway device service unit is used to send gateway device information perception commands and calculation results, receive gateway device requests and information, including gateway device ID, calculation waiting delay The transmission rate between the transmission gateway and the intelligent gateway in the system

[0010] The wireless charging unit is used to send wireless energy signals to provide wireless energy to user devices within the range;

[0011] The device management and control unit is used to store the device information sent when the user device accesses, assign a gateway device ID to the accessed user device through the device management and control unit, and control the wireless charging unit to be turned on and off through the device management and control unit;

[0012] The data storage unit is used to store computing task related data and computing resource perception data;

[0013] The computing task offloading decision unit is used to decide on the offloading of computing tasks to the current edge computing gateway or an edge computing gateway other than the current edge computing gateway in the intelligent edge gateway system, i.e., an intelligent gateway in another area, not referring to gateways of other types or models, as well as the time allocation of user device task offloading and wireless charging, the size allocation of local computing and offloading of user device computing tasks, the energy consumption allocation of user devices for local computing and offloading, and the CPU frequency of user devices for local computing;

[0014] The task calculation unit is used to calculate the offloaded computing task, obtain the calculation result and send the task source to the user equipment service and protocol conversion unit or the gateway equipment service unit.

[0015] A further improvement of the present invention is that the computing task includes the user device ID, the computing task data volume d n , the maximum tolerable delay of the task T n，max , the number of CPU cycles required to calculate 1 bit of data, Φ, the energy collected in the previous frame And the power of the user's device

[0016] A further improvement of the present invention is that the device information includes the maximum calculation frequency f n，max, maximum transmission power p n，max , Maximum battery capacity And the device power The user equipment is powered by a battery, has a wireless energy collection module, and is connected to an edge computing gateway via a wireless link.

[0017] The present invention provides an operation method of a resource-aware intelligent edge gateway system, which specifically comprises the following steps:

[0018] Step 1: The user device generates a computing task and sends a message containing the user device ID and the computing task data volume d to the intelligent edge gateway. n , the number of CPU cycles required to calculate 1 bit of data in the task Φ, the maximum tolerable delay of the task T n，max , the energy collected in the previous frame and device power The offloading task request is received and the response from the smart edge gateway is waited for;

[0019] Step 2: The intelligent edge gateway receives the task offloading request sent by the user equipment, converts it into data of a specified type and structure through the user equipment service and protocol conversion unit, and identifies it as a task offloading request;

[0020] Step 3: The user equipment service and protocol conversion unit converts the gateway device ID in the task offloading request and the calculation task data volume d n , the number of CPU cycles required to calculate 1 bit of data in the task Φ, the maximum tolerable delay of the task T n，max , the energy collected in the previous frame and device power Encapsulate it into a data packet and pass it to the computing resource perception unit, and start listening for computing offload requests. The listening time interval is recorded as a listening time frame;

[0021] Step 4: The computing resource sensing unit receives the data packet, generates a computing resource sensing command, and sends a computing resource sensing request to the remaining edge gateways in the intelligent edge gateway system through the gateway device service unit to obtain their computing waiting delay. Gateways i To Gateways j The transmission rate After the perception is completed, the data is immediately encapsulated into a data packet and passed to the computing task offloading decision unit and waits for the end of the listening time frame. After the listening time frame ends, the computing resource perception unit will receive the received data packet containing the gateway device ID and the computing task data volume d n , the number of CPU cycles required to calculate 1 bit of data in the task Φ, the maximum tolerable delay of the task T n，max , the energy collected in the previous frame and device power User equipment ID and channel gain h n After the data packets are uniformly encapsulated, they are passed to the computing task offloading decision unit;

[0022] Step 5: The computing task offloading decision unit determines the amount of computing task data d of the user device. n , Equipment power Channel gain h n And the computational latency of the gateway in the intelligent edge gateway system Transmission speed between gateways The MCMHD algorithm is used to calculate the optimal wireless charging time ratio coefficient τ for the objective function. n , the user locally calculates the CPU frequency f n , signal transmission power p n and uninstall destination gateway x n ;

[0023] Step 6: The computing task offloading decision unit encapsulates the completed offloading decision into two data packets containing different information, and sends them to different user devices and gateways through the user device service and protocol conversion unit and the gateway device service unit respectively. After the user device service and protocol conversion unit of the remaining gateway devices receives the offloading decision, it adds the user device information to be offloaded to the local calculation as a temporary user device to the device management and control unit, and at the same time adds the computing task to the computing waiting queue of the task computing unit. Queue in

[0024] Step 7: After receiving the offloading decision of the wireless charging edge computing gateway, the user device adjusts the working parameters of the device and offloads the calculation part d″ according to the calculation decision. n If it is 0, it determines whether the computing task needs to be offloaded. If it needs to be offloaded, the computing task is split into local computing parts d′ according to the offloading decision. n and offload the computational part d″ n , if not required, only the local calculation part d′ n , local calculation part d′ n Start computing and offloading the computing part in the computing unit of the user device itself d" n Start to unload to the smart gateway through the wireless component, and in the unloading calculation part d″ n After the unloading is completed, the charging time starts;

[0025] Step 8: After receiving the unloading task data of the user device, the user device service and protocol conversion unit of the wireless charging edge computing gateway checks the destination gateway device ID. If it is the gateway device ID of the wireless charging edge computing gateway, it is added to the computing waiting queue of the task computing unit. And cache it to the data storage unit, otherwise add it to the gateway device service unit and continue to forward it to the unloading destination gateway. At the same time, after the user device completes the unloading within the time specified by the unloading decision, it starts to collect wireless energy according to the time period specified by the unloading decision and performs wireless charging;

[0026] Step 9: Unload the calculation part d″ n After the task calculation is completed, it is sent back to the user device according to the source gateway device ID and user ID;

[0027] Step 10: After receiving the calculation result of the offloaded part task, the user equipment merges it with the local calculation result;

[0028] Step 11. After the calculation results are merged, the wireless energy harvesting task continues to complete the decision-making allocation of the remaining time. After the wireless energy harvesting time period is completed, it is determined whether there is a calculation task. If there is a task, steps 1 to 11 are repeated. If not, it is detected whether the battery power is 100%. If the battery power does not reach 100%, charging continues until the next calculation task is generated or the power reaches 100%.

[0029] A further improvement of the present invention is that in step 1, the energy Defined as:

[0030]

[0031] Where μ is the energy conversion coefficient, p s is the energy transmission power of the wireless charging module of the smart edge gateway, τ n is the wireless charging time ratio coefficient, T f is the length of a time frame, c n is the distance between the user equipment and the edge gateway, and α is the energy conversion efficiency index.

[0032] A further improvement of the present invention is that in step 5, the objective function is:

[0033]

[0034] in, Calculate the latency locally for the user device. Offload latency for tasks, Using local computation latency for all tasks, The local computing power consumption of the user device, Offloading power consumption for tasks, is the energy harvesting size of all tasks using local computing, N is the total number of user devices requesting computing task offloading, n is the number of user devices requesting computing task offloading, Processing delay for the gateway, including calculating waiting delay Processing delay and computing task transmission delay is the energy harvesting size, Energy consumption of local computation for all tasks.

[0035] A further improvement of the present invention is that in step 5, the wireless charging time proportional coefficient τ is obtained by using the MCMHD algorithm n , the user locally calculates the CPU frequency f n , signal transmission power p n and uninstall destination gateway x n The training process includes the following steps:

[0036] Step 5.1, initialize the Actor training network μ(S) to generate a set of continuous actions con_a;

[0037] Step 5.2: Initialize the Critic training network Q(S,con_a*,dis_a) to generate discrete actions dis_a and estimate action values ​​Q_values;

[0038] Step 5.3, initialize the Actor target network μ'(S) and the Critic target network Q'(S,con_a*,dis_a) as copies of the training network to stabilize learning and participate in action value estimation;

[0039] Step 5.4, initialize the experience replay pool Rp, which is used to store experience data;

[0040] Step 5.5, initialize state;

[0041] Step 5.6: Check whether the termination condition is met. If the maximum number of iterations is reached, end the training; otherwise, continue;

[0042] Step 5.7, use the Actor training network μ(S) and the current state state to generate a set of continuous actions con_a;

[0043] Step 5.8, combine the continuous action con_a set and the current state state, and use the Critic training network Q(S, con_a*, dis_a) to select the discrete action dis_a;

[0044] Step 5.9, combine the continuous action con_a set and the discrete action dis_a to obtain the best continuous action con_a*;

[0045] Step 5.10, interact with the system environment Env of the MCMHD algorithm, and obtain the next state next_state and reward reward through the state state, con_a*, dis_a and the system environment Env of the MCMHD algorithm. Env is a technical indicator class designed according to the system model and reward reward of the MCMHD algorithm;

[0046] Step 5.11, store the experience state, con_a*, dis_a, reward, and next_state into the experience replay pool Rp;

[0047] Step 5.12: If the experience replay pool Rp has accumulated enough data of the set minimum batch size for training, extract a batch of data batch_memory from it;

[0048] Step 5.13, use the state in batch_memory and the Actor target network μ'(S) to predict the continuous action set next_con_a of the next state;

[0049] Step 5.14, use the continuous action set next_con_a of the next state and next_state in batch_memory, as well as the Critic training network Q(S, con_a*, dis_a) and the Critic target network Q′(S, con_a*, dis_a) to calculate the action value next_Q_values ​​set of the next state;

[0050] Step 5.15, obtain the next state discrete action next_dis_a and the maximum next state action value next_Q_values* through the next state action value set next_Q_values;

[0051] Step 5.16, calculate the target action value targets_Q_values, the target action value targets_Q_values ​​is the immediate reward reward plus the discounted next_Q_values*;

[0052] Step 5.17, use the Critic network Q(S,con_a*,dis_a) and the state state in batch_memory, the continuous action con_a set and the discrete action dis_a to calculate the predicted action value Q_values;

[0053] Step 5.18. Calculate the Critic's loss, that is, the difference between the predicted action value Q_values ​​and the target action value targets_Q_values, and perform backpropagation to update the Critic network.

[0054] Step 5.19, use the Critic network Q(S,con_a*,dis_a) to evaluate the state in batch_memory and the action value predicted by the Actor network μ(S), calculate the loss of the Actor, backpropagate and update the Actor network;

[0055] Step 5.20: Use the soft update strategy to update the parameters of the target network so that the parameters of the target network are biased towards the parameters of the training network.

[0056] Step 5.21, the environment state is updated, and the state is updated to next_state;

[0057] Step 5.22: Repeat steps 5.6 to 5.21, and repeat the entire process until the termination condition is met.

[0058] A further improvement of the present invention is that: in step 7, after the task is split, the local calculation part d′ n Immediately calculate locally on the user's device, offloading the calculation part d″ n At the same time, it is offloaded to the intelligent edge gateway in decision-making.

[0059] The beneficial effects of the present invention are:

[0060] The present invention combines wireless charging with edge computing technology to offload computing tasks of different user devices to the intelligent edge gateway for computing, and complete a charging process at the same time;

[0061] The offloading strategy adopts the processing mode of collaborative computing between devices and intelligent edge gateways and collaborative computing between intelligent edge gateways;

[0062] The present invention senses the resources of user devices including energy, computing and transmission, and adopts a multi-agent hybrid decision algorithm (MCMHD) that combines continuous action space and discrete action space for computing tasks proposed by different user devices and combined with the wireless charging and remaining power of the devices, to achieve task offloading and wireless charging time balance of multiple devices under gateway management, optimal resource allocation and offloading scheme for device computing task allocation, and minimize the weighted sum of task delay and power consumption of user devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is an architectural diagram of the resource-aware intelligent edge gateway system of the present invention.

[0064] Figure 2 The figure is a block diagram of the composition of the intelligent edge gateway system of the present invention.

[0065] Figure 3It is a flow chart of the overall operation of the system of the present invention.

[0066] Figure 4 This is a flowchart of the task offloading operation on the user equipment side of the present invention.

[0067] Figure 5 This is a flow chart of the task offloading operation on the edge gateway side of the present invention.

[0068] Figure 6 It is the flow chart of the MCMHD algorithm of the present invention.

[0069] Figure 7 This is a diagram showing the convergence effect of the MCMHD algorithm in the system of the present invention.

[0070] Figure 8 This is a comparison chart of the convergence effects of the MADDPG algorithm, the DDPG algorithm, and the MADDPG algorithm in this system.

[0071] Fig. 9 The figure is a comparison chart of the task completion rate and device power-off rate of the MCMHD algorithm, the DDPG algorithm, and the MADDPG algorithm.

[0072] Fig.10 It is a comparison chart of the impact of average completion delay of the MCMHD algorithm of the present invention, the DDPG algorithm and the MADDPG algorithm.

[0073] Fig.11 It is a comparison chart of the average completion energy consumption impact of the MCMHD algorithm of the present invention, the DDPG algorithm and the MADDPG algorithm.

[0074] Fig.12 This is a comparison chart of the impact of task size on average delay between the present invention and the non-offloading system, cloud offloading system and single gateway offloading system.

[0075] Fig.13 This is a comparison chart of the impact of the gateway computing capacity on the average computing delay of tasks between the present invention and the non-offloading system, the cloud offloading system and the single gateway offloading system. DETAILED DESCRIPTION

[0076] The following will disclose the embodiments of the present invention with drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. That is to say, in some embodiments of the present invention, these practical details are not necessary.

[0077] like Figure 1 As shown, the present invention provides a resource-aware intelligent edge gateway system, which includes several wireless charging intelligent edge gateways, such as Figure 2As shown, the intelligent edge gateway includes a user device service and protocol conversion unit, a computing resource perception unit, a gateway device service unit, a wireless charging unit, a device management and control unit, a data storage unit, a computing task offloading decision unit, and a task computing unit, which are used for computing resource perception, computing task offloading decision, task computing, data storage, wireless charging control, and device management functions.

[0078] The user device service and protocol conversion unit is used to connect user devices of communication protocols such as Ethernet, Wi-Fi, Zigbee, GPRS, GPS, 3G, 4G, 5G, REST / HTTP, MQTT, CoAP, DDS, AMQP, etc., and perform data identification, data analysis, and data encapsulation on the original data input by the user device, and convert it into data of the specified type and structure of the intelligent edge gateway, and send the parameters of the computing task in the data to the computing resource perception unit. At the same time, the user device service and protocol conversion unit sends the device information of the user device when it is connected and the device power perceived at a fixed time. Send to the device management and control unit;

[0079] The computing resource perception unit is responsible for sending a gateway computing resource perception command to the gateway device service unit after receiving the computing task request of the first device, computing resource usage and communication status information of other intelligent edge gateways in the resource-aware intelligent edge gateway system, and forwarding the device information and gateway information to the computing task offloading decision unit. The computing task includes the user device ID, computing task data volume d n , the maximum tolerable delay of the task T n , the number of CPU cycles required to calculate 1 bit of data, Φ, the energy collected in the previous frame And the power of the user's device

[0080] The gateway device service unit is used to send gateway device information perception commands and calculation results, receive gateway device requests and information, including gateway device ID, calculation waiting delay The transmission rate between the transmission gateway and the intelligent gateway in the system The device information includes the maximum calculation frequency f n，max , maximum transmission power p n，max , Maximum battery capacity And the device power The user equipment is powered by a battery, has a wireless energy collection module, and is connected to an edge computing gateway via a wireless link.

[0081] The wireless charging unit is used to send wireless energy signals to provide wireless energy to user devices within the range;

[0082] The device management and control unit is used to store the device information sent when the user device accesses, assign a gateway device ID to the accessed user device through the device management and control unit, and control the wireless charging unit to be turned on and off through the device management and control unit;

[0083] The data storage unit is used to store computing task related data and computing resource perception data;

[0084] The computing task offloading decision unit is used to decide whether to offload computing tasks to the current edge computing gateway or an edge computing gateway other than the current edge computing gateway in the intelligent edge gateway system, as well as the time allocation of user device task offloading and wireless charging, the size allocation of local computing and offloading of user device computing tasks, the energy consumption allocation of user devices for local computing and offloading, and the CPU frequency of user devices for local computing;

[0085] The task calculation unit is used to calculate the offloaded computing task, obtain the calculation result and send the task source to the user equipment service and protocol conversion unit or the gateway equipment service unit.

[0086] The present invention also provides a method for operating a resource-aware intelligent edge gateway system, such as Figure 3 As shown in the overall system flow chart, there are several intelligent edge gateways in the resource-aware intelligent edge gateway system. Each intelligent edge gateway controls the wireless charging process and computing task offloading process of the user device.

[0087] The user-side operation process (including steps 1, 7, 10 and 11) is as follows Figure 4 The gateway operation process (including steps 2-68-9) is as follows Figure 5 , where the number of intelligent edge gateways depends on the specific system. This embodiment is only for distance description.

[0088] The specific operation method of the present invention specifically comprises the following steps:

[0089] Step 1: User device u i Generate a computing task and send a data packet containing the user device ID and the computing task data volume d to the wireless charging intelligent edge gateway. n , the number of CPU cycles required to calculate 1 bit of data in the task Φ, the maximum tolerable delay of the task T n,max , the energy collected in the previous frame and device power The offloading task request is sent and the wireless charging smart edge gateway is waiting for the j The response, where the energy size Defined as:

[0090]

[0091] Where μ is the energy conversion coefficient, p s is the energy transmission power of the wireless charging module of the smart edge gateway, τ n is the wireless charging time ratio coefficient, T f is the length of a time frame, c n is the distance between the user equipment and the edge gateway, and α is the energy conversion efficiency index.

[0092] like Figure 5 As shown, step 2, intelligent edge gateway s j A task offloading request sent by a user device is received, converted into data of a specified type and structure by a user device service and protocol conversion unit, and identified as a task offloading request;

[0093] Step 3: The user equipment service and protocol conversion unit converts the gateway device ID in the task offloading request and the calculation task data volume d n , the number of CPU cycles required to calculate 1 bit of data in the task Φ, the maximum tolerable delay of the task T n，max , the energy collected in the previous frame and device power Encapsulate it into a data packet and pass it to the computing resource perception unit, and start listening for computing offload requests. The listening time interval is recorded as a listening time frame;

[0094] Step 4: The computing resource sensing unit receives the data packet, generates a computing resource sensing command, and sends a computing resource sensing request to other remaining edge gateways in the intelligent edge gateway system through the gateway device service unit to obtain their computing waiting delay. Gateways i To Gateways j The transmission rate (0 if no connection) After the perception is completed, the data is immediately encapsulated into a data packet and passed to the computing task offloading decision unit and waits for the end of the listening time frame. Since the listening time frame is very small, it is mainly to ensure that all task offloading requests can be received at the beginning of the time frame, so it can be ignored when optimizing the objective function. After the listening time frame ends, the computing resource perception unit will receive the received packet containing the gateway device ID and the computing task data volume d n , the number of CPU cycles required to calculate 1 bit of data in the task Φ, the maximum tolerable delay of the task T n，max , the energy collected in the previous frame and device power User equipment ID and channel gain h n After the data packets are uniformly encapsulated, they are passed to the computing task offloading decision unit;

[0095] Step 5: The computing task offloading decision unit determines the amount of computing task data d of the user device. n , Equipment power Channel gain h n And the computational latency of the gateway in the intelligent edge gateway system Transmission speed between gateways The MCMHD algorithm is used to calculate the optimal wireless charging time ratio coefficient τ for the objective function. n , the user locally calculates the CPU frequency f n , signal transmission power p n and uninstall destination gateway x n , wireless charging time proportional coefficient τ n , wireless charging time proportional coefficient τ n And the signal transmission power p n is a continuous variable, unloading destination gateway x n is a discrete variable, and the objective function is:

[0096]

[0097] in, Calculate the latency locally for the user device. Offload latency for tasks, Using local computation latency for all tasks, The local computing power consumption of the user device, Offloading power consumption for tasks, is the energy harvesting size of all tasks using local computing, N is the total number of user devices requesting computing task offloading, n is the number of user devices requesting computing task offloading, Processing delay for the gateway, including calculating waiting delay Processing delay and computing task transmission delay is the energy harvesting size, Energy consumption of local computation for all tasks.

[0098] Take the device u1 in the smart edge gateway s1 area as an example. r i is the uninstall rate of the user device, Indicates that u1 belongs to the smart edge gateway s1 and is assigned to the smart edge gateway s1, which means that it does not need to be unloaded to other gateways for the second time and calculates the waiting delay Gateway task processing delay Gateway task transmission delay All tasks use local computing latency Power consumption Task offloading power consumption Energy harvesting size

[0099] Furthermore, taking the device u2 in the smart edge gateway s1 area as an example, It means that u2 belongs to smart edge gateway s1 and is assigned to smart edge gateway s5, which means that it needs to be unloaded to gateway s5 for the second time and the waiting delay is calculated. Gateway task processing delay Gateway task transmission delay Path is the best path from s1 to s5 in the routing table, and hop is the number of hops on the path;

[0100] Step 6: The computing task offloading decision unit encapsulates the completed offloading decision into two data packets containing different information, and sends them to different user devices and gateways through the user device service and protocol conversion unit and the gateway device service unit respectively. After the user device service and protocol conversion unit of the remaining gateway devices receives the offloading decision, it adds the user device information to be offloaded to the local calculation as a temporary user device to the device management and control unit, and at the same time adds the computing task to the computing waiting queue of the task computing unit. Queue in

[0101] Step 7: After the user device receives the offloading decision from the wireless charging edge computing gateway, Figure 3 The process shown in the figure adjusts the working parameters of the equipment and unloads the calculation part d″ according to the calculation decision n If it is 0, it determines whether the computing task needs to be offloaded. If it needs to be offloaded, the computing task is split into local computing parts d′ according to the offloading decision. n and offload the computational part d″ n , if not required, only the local calculation part d′ n , local calculation part d′ n Start computing and offloading the computing part in the computing unit of the user device itself d" n Start to unload to the smart gateway through the wireless component, and in the unloading calculation part d″ n After the unloading is completed, the charging time starts. After the task is split, the local calculation part d′ n Immediately calculate locally on the user's device, offloading the calculation part d″ n At the same time, it is offloaded to the intelligent edge gateway in decision-making.

[0102] Step 8: After receiving the unloading task data of the user device, the user device service and protocol conversion unit of the wireless charging edge computing gateway checks the destination gateway device ID. If it is the gateway device ID of the wireless charging edge computing gateway, it is added to the computing waiting queue of the task computing unit. And cache it to the data storage unit, otherwise add it to the gateway device service unit and continue to forward it to the unloading destination gateway. At the same time, after the user device completes the unloading within the time specified by the unloading decision, it starts to collect wireless energy according to the time period specified by the unloading decision and performs wireless charging;

[0103] Step 9: Unload the calculation part d″ n After the task calculation is completed, it is sent back to the user device according to the source gateway device ID and user ID;

[0104] Step 10: After receiving the calculation result of the offloaded part task, the user equipment merges it with the local calculation result;

[0105] Step 11. After the calculation results are merged, the wireless energy harvesting task continues to complete the decision-making allocation of the remaining time. After the wireless energy harvesting time period is completed, it is determined whether there is a calculation task. If there is a task, steps 1 to 11 are repeated. If not, it is detected whether the battery power is 100%. If the battery power does not reach 100%, charging continues until the next calculation task is generated or the power reaches 100%.

[0106] The present invention adopts a multi-agent hybrid decision algorithm (MCMHD) based on the improved MADDPG that combines continuous action space and discrete action space. The present invention adopts a multi-agent centralized training and decentralized execution mode. Each user device acts as an Actor module and shares a Critic module. In this way, each user device can share continuous variables in the decision and coordinate discrete variables at the same time. The state in the algorithm includes the task data volume d n , Equipment power Wireless link channel gain h n , the energy collected by the user device in the previous time frame Computational latency of each intelligent edge gateway The action action includes the continuous variable wireless charging time proportional coefficient τ n 、User device CPU calculation frequency f n And the signal transmission power p n And discrete variable offloading destination gateway selection The constructed reward consists of four parts. The first part is the objective function reward r1, which is the weighted sum of the normalized delay and energy consumption with respect to the battery power. The second part is the task processing timeout penalty r2, which is the penalty for the task processing delay exceeding the maximum tolerable delay of the task. The third part is the task processing energy consumption exceeding penalty r3, which is the fourth part of the battery power is too low penalty, which is the remaining power of the device. The penalty r4 is obtained when the battery level is below the minimum threshold. Among them, r3 introduces the penalty r4 for the battery level of the device. S-shaped growth function Energy collected in the same time frame as the user device The energy consumption threshold of this computing task is used as the product of the energy consumption threshold. The energy consumption threshold allows the agent to receive different violation penalties at different power levels, and combined with the device power in r1 The weights of latency and energy consumption in the objective function are adjusted as weight coefficients, so that the present invention can dynamically adjust the computing performance and energy consumption of the user device. The special reward composition makes the user device under the MCMHD algorithm pay more attention to latency when the battery power is good, and pay more attention to energy consumption when the battery power is low. Under different conditions, the computing performance and energy consumption of the device are dynamically adjusted.

[0107] The present invention uses the MCMHD algorithm to deal with problems with continuous action and discrete action space. The specific process is as follows: Figure 6 As shown, the following steps are included:

[0108] Step 5.1, initialize the Actor training network μ(S) to generate a set of continuous actions con_a;

[0109] Step 5.2: Initialize the Critic training network Q(S,con_a*,dis_a) to generate discrete actions dis_a and estimate action values ​​Q_values;

[0110] Step 5.3, initialize the Actor target network μ'(S) and the Critic target network Q'(S,con_a*,dis_a) as copies of the training network to stabilize learning and participate in action value estimation;

[0111] Step 5.4, initialize the experience replay pool Rp, which is used to store experience data;

[0112] Step 5.5, initialize state;

[0113] Step 5.6: Check whether the termination condition is met. If the maximum number of iterations is reached, end the training; otherwise, continue;

[0114] Step 5.7, use the Actor training network μ(S) and the current state state to generate a set of continuous actions con_a;

[0115] Step 5.8, combine the continuous action con_a set and the current state state, and use the Critic training network Q(S, con_a*, dis_a) to select the discrete action dis_a;

[0116] Step 5.9, combine the continuous action con_a set and the discrete action dis_a to obtain the best continuous action con_a*;

[0117] Step 5.10, interact with the system environment Env of the MCMHD algorithm, and obtain the next state next_state and reward reward through the state state, con_a*, dis_a and the system environment Env of the MCMHD algorithm. Env is a technical indicator class designed according to the system model and reward reward of the MCMHD algorithm;

[0118] Step 5.11, store the experience state, con_a*, dis_a, reward, and next_state into the experience replay pool Rp; Step 5.12, if the experience replay pool Rp accumulates enough data of the set minimum batch size for training, extract a batch of data batch_memory from it;

[0119] Step 5.13, use the state in batch_memory and the Actor target network μ'(S) to predict the continuous action set next_con_a of the next state;

[0120] Step 5.14, use the continuous action set next_con_a of the next state and next_state in batch_memory, as well as the Critic training network Q(S, con_a*, dis_a) and the Critic target network Q′(S, con_a*, dis_a) to calculate the action value next_Q_values ​​set of the next state;

[0121] Step 5.15, obtain the next state discrete action next_dis_a and the maximum next state action value next_Q_values* through the next state action value set next_Q_values;

[0122] Step 5.16, calculate the target action value targets_Q_values, the target action value targets_Q_values ​​is the immediate reward reward plus the discounted next_Q_values*;

[0123] Step 5.17, use the Critic network Q(S,con_a*,dis_a) and the state state in batch_memory, the continuous action con_a set and the discrete action dis_a to calculate the predicted action value Q_values;

[0124] Step 5.18. Calculate the Critic's loss, that is, the difference between the predicted action value Q_values ​​and the target action value targets_Q_values, and perform backpropagation to update the Critic network.

[0125] Step 5.19, use the Critic network Q(S,con_a*,dis_a) to evaluate the state in batch_memory and the action value predicted by the Actor network μ(S), calculate the loss of the Actor, backpropagate and update the Actor network;

[0126] Step 5.20: Use the soft update strategy to update the parameters of the target network so that the parameters of the target network are slightly biased towards the parameters of the training network.

[0127] Step 5.21, the environment state is updated, and the state is updated to next_state;

[0128] Step 5.22: Repeat steps 5.6 to 5.21, and repeat the entire process until the termination condition is met.

[0129] The above is the wireless charging time ratio coefficient τ obtained by the MCMHD algorithm n , the user locally calculates the CPU frequency f n , signal transmission power p n and the decision of the offloading destination gateway x n training process.

[0130] In order to verify the present invention, the simulation experiment is provided as follows:

[0131] The simulation system includes 8 user devices and 5 intelligent gateway devices. One intelligent gateway device is in the current area, which is responsible for transmitting wireless energy to the 8 user devices and making decision-making task offloading requests. The 8 user devices need to receive the wireless energy transmitted by an intelligent gateway device and offload the generated computing tasks to the intelligent gateway specified in the decision. The computing tasks of the user devices in the simulation use the same task size and maximum tolerable delay, and the mobile range is specified to be within 10 to 20 meters from the intelligent gateway device in the current area. The simulation tested the performance of the system in processing the same computing tasks 100 times in a row, and compared it with the systems based on the DDPG algorithm and the MADDPG algorithm. Figure 7 This is a diagram showing the convergence effect of the MCMHD algorithm in the system of the present invention, where the reward value reaches above 1.3, which is close to the maximum reward value of 1.5 set by the system. Figure 8 The convergence effect of the MADDPG algorithm and the DDPG algorithm in this system is compared. It can be clearly seen that the improved MCMHD algorithm converges faster than the original MADDPG algorithm and the explored reward value is higher. Although the DDPG algorithm converges faster, the maximum reward value explored is lower. Due to the decision strategy of finding a higher reward, Fig. 9 It can be clearly seen that the MCMHD algorithm achieves a higher task completion rate and a lower device power-off rate. Fig.10 and Fig.11 It also shows that the MCMHD algorithm has discovered that the decision-making strategy is more adaptable to changes in task size. Fig.12 and Fig.13 This demonstrates the advantage of the MCMHD algorithm system in terms of average task computation delay compared to the non-offloading system, cloud-based task offloading system, and single intelligent gateway system, where the vertical axis average delay ratio is defined as the average delay ratio of the MCMHD algorithm system to other mode systems.

[0132] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A resource-aware intelligent edge gateway system, characterized in that: The intelligent edge gateway system includes several wireless charging intelligent edge gateways, and the intelligent edge gateway includes a user device service and protocol conversion unit, a computing resource perception unit, a gateway device service unit, a wireless charging unit, a device management and control unit, a data storage unit, a computing task offloading decision unit, and a task computing unit; The user device service and protocol conversion unit is used to connect user devices of multiple communication protocols, and perform data identification, data analysis, and data encapsulation on the original data input by the user device, convert it into data of the specified type and structure of the intelligent edge gateway, and send the parameters of the computing task in the data to the computing resource perception unit. At the same time, the user device service and protocol conversion unit sends the device information when the user device is connected and the device power perceived at a fixed time. Send to the device management and control unit; The computing resource sensing unit is responsible for sending a gateway computing resource sensing command to the gateway device service unit after receiving the computing task request of the first device, the computing resource usage and communication status information of other intelligent edge gateways in the resource-aware intelligent edge gateway system, and forwarding the device information and gateway information to the computing task offloading decision unit; The gateway device service unit is used to send gateway device information perception commands and calculation results, receive gateway device requests and information, including gateway device ID, calculation waiting delay The transmission rate between the transmission gateway and the intelligent gateway in the system The wireless charging unit is used to send wireless energy signals to provide wireless energy to user devices within the range; The device management and control unit is used to store the device information sent when the user device accesses, assign a gateway device ID to the accessed user device through the device management and control unit, and control the wireless charging unit to be turned on and off through the device management and control unit; The data storage unit is used to store computing task related data and computing resource perception data; The computing task offloading decision unit is used to decide whether to offload computing tasks to the current edge computing gateway or an edge computing gateway other than the current edge computing gateway in the intelligent edge gateway system, as well as the time allocation of user device task offloading and wireless charging, the size allocation of local computing and offloading of user device computing tasks, the energy consumption allocation of user devices for local computing and offloading, and the CPU frequency of user devices for local computing; The task calculation unit is used to calculate the offloaded computing task, obtain the calculation result and send the task source to the user equipment service and protocol conversion unit or the gateway equipment service unit.

2. A resource-aware intelligent edge gateway system according to claim 1, characterized in that: The computing task includes the user device ID, the computing task data volume d n , the maximum tolerable delay of the task T n，max , the number of CPU cycles required to calculate 1 bit of data, Φ, the energy collected in the previous frame And the user's device power 3. A resource-aware intelligent edge gateway system according to claim 1, characterized in that: The device information includes the maximum calculation frequency f n，max , maximum transmission power p n，max , Maximum battery capacity And the device power 4. A resource-aware intelligent edge gateway system according to claim 1, characterized in that: The user equipment is powered by a battery, has a wireless energy collection module, and is connected to an edge computing gateway via a wireless link.

5. A method for operating a resource-aware intelligent edge gateway system according to any one of claims 1 to 4, characterized in that: In the resource-aware intelligent edge gateway system, there are several intelligent edge gateways, each of which controls the wireless charging process and computing task offloading process of the user device, and the secondary offloading of the intelligent edge gateway realizes the collaboration between the intelligent edge gateways. Specifically, the operation method specifically includes the following steps: Step 1: The user device generates a computing task and sends a message containing the user device ID and the computing task data volume d to the intelligent edge gateway. n , the number of CPU cycles required to calculate 1 bit of data in the task Φ, the maximum tolerable delay of the task T n，max , the energy collected in the previous frame and device power The offloading task request is received and the response from the smart edge gateway is waited for; Step 2: The intelligent edge gateway receives the task offloading request sent by the user equipment, converts it into data of a specified type and structure through the user equipment service and protocol conversion unit, and identifies it as a task offloading request; Step 3: The user equipment service and protocol conversion unit converts the gateway device ID in the task offloading request and the calculation task data volume d n , the number of CPU cycles required to calculate 1 bit of data in the task Φ, the maximum tolerable delay of the task T n，max , the energy collected in the previous frame and device power Encapsulate it into a data packet and pass it to the computing resource perception unit, and start listening for computing offload requests. The listening time interval is recorded as a listening time frame; Step 4: The computing resource sensing unit receives the data packet, generates a computing resource sensing command, and sends a computing resource sensing request to the remaining edge gateways in the intelligent edge gateway system through the gateway device service unit to obtain their computing waiting delay. Gateways i To Gateways j The transmission rate After the perception is completed, the data is immediately encapsulated into a data packet and passed to the computing task offloading decision unit and waits for the end of the listening time frame. After the listening time frame ends, the computing resource perception unit will receive the received data packet containing the gateway device ID and the computing task data volume d n , the number of CPU cycles required to calculate 1 bit of data in the task Φ, the maximum tolerable delay of the task T n，max , the energy collected in the previous frame and device power User equipment ID and channel gain h n After the data packets are uniformly encapsulated, they are passed to the computing task offloading decision unit; Step 5: The computing task offloading decision unit determines the amount of computing task data d of the user device. n , Equipment power Channel gain h n And the computational latency of the gateway in the intelligent edge gateway system Transmission speed between gateways The MCMHD algorithm is used to calculate the optimal wireless charging time ratio coefficient τ for the objective function. n , the user locally calculates the CPU frequency f n , signal transmission power p n and uninstall destination gateway x n ; Step 6: The computing task offloading decision unit encapsulates the completed offloading decision into two data packets containing different information, and sends them to different user devices and gateways through the user device service and protocol conversion unit and the gateway device service unit respectively. After the user device service and protocol conversion unit of the remaining gateway devices receives the offloading decision, it adds the user device information to be offloaded to the local calculation as a temporary user device to the device management and control unit, and at the same time adds the computing task to the computing waiting queue of the task computing unit. Queue in Step 7: After receiving the offloading decision of the wireless charging edge computing gateway, the user device adjusts the working parameters of the device and offloads the calculation part d″ according to the calculation decision. n If it is 0, it determines whether the computing task needs to be offloaded. If it needs to be offloaded, the computing task is split into local computing parts d′ according to the offloading decision. n and offload the computational part d″ n , if not required, only the local calculation part d′ n , local calculation part d′ n Start computing and offloading the computing part in the computing unit of the user device itself d" n Start to unload to the smart gateway through the wireless component, and in the unloading calculation part d″ n After the unloading is completed, the charging time starts; Step 8: After receiving the unloading task data of the user device, the user device service and protocol conversion unit of the wireless charging edge computing gateway checks the destination gateway device ID. If it is the gateway device ID of the wireless charging edge computing gateway, it is added to the computing waiting queue of the task computing unit. And cache it to the data storage unit, otherwise add it to the gateway device service unit and continue to forward it to the unloading destination gateway. At the same time, after the user device completes the unloading within the time specified by the unloading decision, it starts to collect wireless energy according to the time period specified by the unloading decision and performs wireless charging; Step 9: Unload the calculation part d″ n After the task calculation is completed, it is sent back to the user device according to the source gateway device ID and user ID; Step 10: After receiving the calculation result of the offloaded part task, the user equipment merges it with the local calculation result; Step 11. After the calculation results are merged, the wireless energy harvesting task continues to complete the decision-making allocation of the remaining time. After the wireless energy harvesting time period is completed, it is determined whether there is a calculation task. If there is a task, steps 1 to 11 are repeated. If not, it is detected whether the battery power is 100%. If the battery power does not reach 100%, charging continues until the next calculation task is generated or the power reaches 100%.

6. The method for operating a resource-aware intelligent edge gateway system according to claim 5, characterized in that: In step 1, the energy Defined as: Among them, μ is the energy conversion coefficient, p s is the energy transmission power of the wireless charging module of the smart edge gateway, τ n is the wireless charging time ratio coefficient, T f is the length of a time frame, c n is the distance between the user equipment and the edge gateway, and α is the energy conversion efficiency index.

7. The method for operating a resource-aware intelligent edge gateway system according to claim 5, characterized in that: In step 5, the objective function is: in, Calculate the latency locally for the user device. Offload latency for tasks, Using local computation latency for all tasks, The local computing power consumption of the user device, Offloading power consumption for tasks, is the energy harvesting size of all tasks using local computing, N is the total number of user devices requesting computing task offloading, n is the number of user devices requesting computing task offloading, Processing delay for the gateway, including calculating waiting delay Processing delay and computing task transmission delay is the energy harvesting size, Energy consumption of local computation for all tasks.

8. The method for operating a resource-aware intelligent edge gateway system according to claim 5, characterized in that: The MCMHD algorithm adopts a multi-agent centralized training and decentralized execution mode. Each user device acts as an Actor module and shares a Critic module. The state in the algorithm includes the task data volume d n , Equipment power Wireless link channel gain h n , the amount of energy collected by the user device in the previous time frame Computational latency of each intelligent edge gateway The action action includes the continuous variable wireless charging time proportional coefficient τ n 、User device CPU calculation frequency f n And the signal transmission power p n And discrete variable offloading destination gateway selection The constructed reward consists of four parts. The first part is the objective function reward r1, which is the weighted sum of the normalized delay and energy consumption with respect to the battery power. The second part is the task processing timeout penalty r2, which is the penalty for the task processing delay exceeding the maximum tolerable delay of the task. The third part is the task processing energy consumption exceeding penalty r3. The fourth part is the low battery penalty, which is the remaining device power. The penalty r4 is obtained when the energy consumption of task processing exceeds the minimum power threshold, and the penalty r3 is introduced for the device power. The S-shaped growth function and the energy collected by the user device in the previous time frame The product of is used as the energy consumption threshold of this computing task. The energy consumption threshold allows the agent to receive different violation penalties at different power levels, and combines the power of the device in the objective function reward r1 It is used as a weight coefficient to adjust the weights of delay and energy consumption in the objective function.

9. The method for operating a resource-aware intelligent edge gateway system according to claim 5 or 8, characterized in that: In step 5, the wireless charging time ratio coefficient τ is obtained using the MCMHD algorithm. n , the user locally calculates the CPU frequency f n , signal transmission power p n and uninstall destination gateway x n The training process includes the following steps: Step 5.1, initialize the Actor training network μ(S) to generate a set of continuous actions con_a; Step 5.2: Initialize the Critic training network Q(S,con_a*,dis_a) to generate discrete actions dis_a and estimate action values ​​Q_values; Step 5.3, initialize the Actor target network μ'(S) and the Critic target network Q'(S,con_a*,dis_a) as copies of the training network to stabilize learning and participate in action value estimation; Step 5.4, initialize the experience replay pool Rp, which is used to store experience data; Step 5.5, initialize state; Step 5.6: Check whether the termination condition is met. If the maximum number of iterations is reached, end the training; Otherwise, continue; Step 5.7, use the Actor training network μ(S) and the current state state to generate a set of continuous actions con_a; Step 5.8, combine the continuous action con_a set and the current state state, and use the Critic training network Q(S, con_a*, dis_a) to select the discrete action dis_a; Step 5.9, combine the continuous action con_a set and the discrete action dis_a to obtain the best continuous action con_a*; Step 5.10, interact with the system environment Env of the MCMHD algorithm, and obtain the next state next_state and reward reward through the state state, con_a*, dis_a and the system environment Env of the MCMHD algorithm. Env is a technical indicator class designed according to the system model and reward reward of the MCMHD algorithm; Step 5.11, store the experience state, con_a*, dis_a, reward, and next_state into the experience replay pool Rp; Step 5.12: If the experience replay pool Rp has accumulated enough data of the set minimum batch size for training, extract a batch of data batch_memory from it; Step 5.13, use the state in batch_memory and the Actor target network μ'(S) to predict the continuous action set next_con_a of the next state; Step 5.14, use the continuous action set next_con_a of the next state and next_state in batch_memory, as well as the Critic training network Q(S, con_a*, dis_a) and the Critic target network Q′(S, con_a*, dis_a) to calculate the action value next_Q_values ​​set of the next state; Step 5.15, obtain the next state discrete action next_dis_a and the maximum next state action value next_Q_values* through the next state action value set next_Q_values; Step 5.16, calculate the target action value targets_Q_values, the target action value targets_Q_values ​​is the immediate reward reward plus the discounted next_Q_values*; Step 5.17, use the Critic network Q(S,con_a*,dis_a) and the state state in batch_memory, the continuous action con_a set and the discrete action dis_a to calculate the predicted action value Q_values; Step 5.

18. Calculate the Critic's loss, that is, the difference between the predicted action value Q_values ​​and the target action value targets_Q_values, and perform backpropagation to update the Critic network. Step 5.19, use the Critic network Q(S,con_a*,dis_a) to evaluate the state in batch_memory and the action value predicted by the Actor network μ(S), calculate the loss of the Actor, backpropagate and update the Actor network; Step 5.20: Use the soft update strategy to update the parameters of the target network so that the parameters of the target network are biased towards the parameters of the training network. Step 5.21, the environment state is updated, and the state is updated to next_state; Step 5.22: Repeat steps 5.6 to 5.21, and repeat the entire process until the termination condition is met.

10. The method for operating a resource-aware intelligent edge gateway system according to claim 1, characterized in that: In step 7, after the task is split, the local calculation part d′ n Immediately calculate locally on the user's device, offloading the calculation part d″ n At the same time, it is offloaded to the intelligent edge gateway in decision-making.

Images