Task unloading and wireless energy transmission method and system

Abstract

The invention belongs to the technical field of wireless communication, and discloses a task unloading and wireless energy transmission method and system, and the method comprises the steps: building a communication and calculation model between a UAV (Unmanned Aerial Vehicle) and a ground Internet of Things user, including a time-varying channel model for communication, a WPT (Wireless Power Transfer) channel model for modeling by using a free space loss model, and a WPT channel model for modeling by using a free space loss model; a calculation and transmission model of UAV and ground Internet of Things users; the method comprises the following steps: designing a basic framework for performing task unloading and WPT on a ground Internet of Things user by using UAV under an edge computing scene assisted by the UAV, and performing system modeling according to the framework to obtain a mathematical model of the whole system; designing an optimization problem, and defining a target function as the weighted sum of the total throughput of the system and the energy consumption of the UAV; modeling the whole framework as a Markov process, designing a reasonable state action space, and designing a reasonable reward function by using a target function; and a task unloading decision and wireless energy transmission are optimized while the total throughput of the system and the UAV energy consumption are constrained by using a double-delay depth deterministic strategy gradient algorithm.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of wireless communication technology, and in particular relates to a task offloading and wireless energy transmission method and system. Background Art

[0002] With the rapid advancement of 5G technology, IoT users have shown a trend of gridding and widespread distribution in daily life and natural environment. The popularization of IoT technology has greatly changed people's lifestyles, significantly improved the degree of intelligence, and enabled people to manage daily affairs more efficiently through various smart devices. However, due to the design limitations of these IoT devices, the capacity of their built-in batteries is limited, and computing resources are relatively scarce. As devices continue to generate and collect massive amounts of data, the problem of insufficient task processing capabilities has become increasingly prominent. These tasks are often highly computationally intensive and extremely sensitive to delays. Although some IoT devices are equipped with external energy conversion functions, such as solar or wind energy modules, which can theoretically alleviate the problem of insufficient power to a certain extent, due to the uncertainty of the natural environment, these devices may encounter various challenges during actual operation. For example, sandstorms or lightning weather may significantly reduce the efficiency of the energy conversion module, making it impossible for the device to fully utilize external resources for self-charging. In addition, the gradual consumption of battery power during long-term operation, especially in extreme environments, may cause the device power to drop rapidly, seriously affecting its normal function. When the power of the IoT device is in urgent need and cannot be replenished in time by the energy conversion module, the need to replace the battery follows. However, due to the large number of IoT devices and their wide distribution, battery replacement is not only a time-consuming and laborious task, but also brings a huge workload to the maintenance team and increases operating costs. Especially in remote or difficult-to-reach areas, the difficulty of battery replacement is further increased, which greatly affects the stability and sustainability of the IoT system.

[0003] Unmanned aerial vehicles (UAVs) have attracted much attention due to their flexibility and easy deployment. They can be used as edge servers in the air to provide offloading and a series of services for ground users. Since UAVs fly at a high altitude, there is a high probability that they can establish a line of sight (LOS) communication link with users. They are also more flexible in the air and can provide better service quality in some extreme disaster environments (floods, earthquakes, volcanic eruptions). Nan Zhao et al. studied the offloading services provided by UAVs in the air for ground IoT users, and optimized the optimization objectives by optimizing the trajectory and power of UAVs. Although these technologies have alleviated the dilemma of insufficient computing resources for ground IoT devices to a certain extent, the battery power problem has always been a thorny challenge. Although ground IoT devices can effectively complete complex task processing with the support of external computing resources, the limited battery capacity restricts the continuous operation of these devices. With the increase in the amount of tasks and the extension of the running time, the gradual consumption of battery power becomes a key factor affecting its performance and reliability. Once the battery power is insufficient, the normal function of the device may not be maintained, which in turn affects the stability and efficiency of the entire IoT system.

[0004] In view of the above analysis, the technical problem that urgently needs to be solved in the existing technology is: once the battery power of the existing device is low, the normal function of the device may not be maintained, which in turn affects the stability and efficiency of the entire Internet of Things system. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a task offloading and wireless energy transmission method and system to solve the problem that ground IoT devices are unable to perform effective and stable task processing calculations due to insufficient computing resources and limited battery capacity in complex natural environments and time-varying wireless channels. This method improves the information processing capabilities of ground IoT users in complex situations while maintaining system stability and better system utility.

[0006] The present invention is implemented as follows: a task offloading and wireless energy transmission method, comprising the following steps:

[0007] S1. Based on the changing environmental conditions and complex electromagnetic interference in the real environment, the communication and computing model between UAV and ground IoT users was established, including the time-varying channel model for communication, the WPT channel model modeled by the free space loss model, and the computing and transmission model between UAV and ground IoT users;

[0008] S2. Based on the channel and computing model proposed in S1, a basic framework for using UAV to perform task offloading and WPT for ground IoT users in the UAV-assisted edge computing scenario is jointly designed. System modeling is performed based on this framework to obtain the mathematical model of the entire system. Under this framework, at the start of each time slot, the UAV will provide reasonable task offloading and WPT decisions for ground IoT users;

[0009] S3. Based on the basic framework of using UAV to offload tasks and perform WPT for ground IoT users in the UAV-assisted edge computing scenario proposed in S2, the energy consumption of UAV and ground IoT devices and the total system throughput are calculated. The CPU computing frequency, transmission power, task size, and channel status of UAV and ground IoT users are jointly considered. By jointly formulating the optimization problem using energy consumption and total system throughput, the objective function is defined as the weighted sum of total system throughput and UAV energy consumption. The objective function is expressed as follows:

[0010]

[0011] in represents the total energy consumption of the UAV in time interval t, D all (t) represents the total system throughput, w1 and w2 represent two weight coefficients, one positive and one negative;

[0012] S4. Model the entire system framework as a Markov process, design a reasonable state and action space, and use the objective function to design a reasonable reward function;

[0013] S5. Use deep reinforcement learning TD3 algorithm to optimize the objective function, while constraining the total system throughput and UAV energy consumption to optimize task offloading decisions and wireless energy transmission.

[0014] Furthermore, S1 specifically includes:

[0015] S11. Establish a service scenario consisting of a UAV equipped with an edge computing server and a WPT radio frequency device and multiple ground users. Each ground user is equipped with an energy receiving device with the same frequency as the UAV radio frequency device and a rechargeable battery to store the collected energy. Both the UAV and the ground user are equipped with a single CPU and a certain amount of memory space to store computing tasks;

[0016] S12. Convert the continuous time operation mechanism into time slots of equal length, with the time interval of each time slot being Τ, and the time slot set being τ = {1, 2, 3, ..., i, ..., T}. Considering the real scenario, the randomly arriving computing tasks are made to obey the exponential distribution, and each task is independent and identically distributed;

[0017] In S1, it is considered that the UAV in the air equipped with edge servers and wireless radio frequency charging devices provides services to the IoT users on the ground. The UAV in the air is represented by A and its three-dimensional coordinates are € A =(x A ,y A ,z A ), the ground IoT user is represented by t and its three-dimensional coordinates are € t =(x t ,y t ,z t ), so the distance between the UAV in the air and the IoT user on the ground can be expressed as D t Indicates that D t =||X A -X t ||.

[0018] Furthermore, in step S1, considering the variability of the actual situation and the randomness of the channel, a communication channel between the UAV and the ground IoT user is established, aiming to use the UAV in the air to provide services to the ground users. The UAV generally stays at an altitude of several kilometers, and the probability of being blocked by the ground IoT user is small, so there will be a relatively stable communication quality and low latency. It is further considered that the communication link between the UAV and the ground IoT user is mainly dominated by the line-of-sight (LOS) channel. In order to minimize the communication interference between each IoT user on the ground and the UAV in the air, the communication between the UAV and a single IoT user adopts the orthogonal frequency division multiple access (OFDMA) protocol. The channel state is static in a time slot, but dynamic in different time slots. It is assumed that the average channel gain follows the path loss model:

[0019]

[0020] in represents the average channel gain between the UAV and the tth IoT user on the ground, A d represents the antenna gain of each IoT user, f c Indicates the signal carrier frequency, d c represents the path loss index, and the channel gain h between each IoT user and the UAV A,t (t) obeys independent and identically distributed Rayleigh distribution.

[0021] Furthermore, in step S1, the WPT channel is modeled. It is considered that the link between UAV and ground IoT users for WPT is dominated by the LOS (Line of Sight) link. The free space loss model is used to model the WPT transmission link:

[0022]

[0023] in represents the energy received by the ground IoT device at time t, v t ∈(0,1) represents the wireless charging conversion efficiency of the tth ground IoT device, g t It represents the channel gain of the tth ground IoT device. It represents the WPT action of UAV to the tth terrestrial IoT device, and indicates how much energy to be transmitted to the terrestrial IoT device.

[0024] Furthermore, in step S1, the channel transmission rate R between the UAV and the tth ground IoT device can be obtained according to the Shannon formula: t :

[0025]

[0026] Where B represents the total channel bandwidth, φ represents the power spectral density of the noise signal, and P t represents the signal transmission power of the tth ground IoT device to the UAV, T represents the number of ground IoT users. Since the amount of data obtained after calculation by the MEC server is very small, the consumption of the task results after the MEC server offloads the calculation is ignored. According to the channel transmission rate obtained above, represents the amount of tasks uploaded by the t-th ground IoT user at time t. This amount of tasks is determined by the task queue size of the local user, the offloading decision made by the reinforcement learning agent, and the computing tasks generated by the ground IoT user at time t. Therefore, the task upload time and task upload energy It is expressed as:

[0027]

[0028] Furthermore, in steps S2 and S3, a UAV-based task offloading and WPT framework is designed for ground IoT users. Specifically, the energy consumption of UAV and ground IoT devices and the overall throughput of the system are calculated. In this process, the CPU computing frequency, transmission power, task size, and channel status of UAV and ground IoT users are comprehensively considered. By jointly analyzing the energy consumption and the total system throughput, an optimization problem is formulated, and its objective function is defined as the weighted sum of the total system throughput and the UAV energy consumption. The objective function is expressed as follows:

[0029]

[0030] in represents the total energy consumption of the UAV in time interval t, in It means that the UAV calculates the energy consumption at time t. It is expressed as:

[0031]

[0032] Where k represents the computing energy efficiency factor, which is related to the CPU chip structure of the edge server equipped by the UAV, and f A represents the CPU calculation frequency of UAV, and D all (t) represents the total throughput of the system at time t, and They represent the local computing tasks of the ground IoT user and the UAV at time t. It is worth noting that the tasks processed by the UAV in each time slot are taken from its own memory space for storing tasks, rather than the tasks uploaded by each ground user in each time slot. The local computing tasks of the UAV and the local IoT user need to comply with the following conditions:

[0033]

[0034] where α t (t) and α A (t) are the task queue storage sizes of ground IoT devices and aerial UAVs, represents the energy consumed by ground IoT users for computing at time t, Only when UAV and ground IoT devices meet the above conditions and The following equation is satisfied:

[0035]

[0036] where f A Indicates the CPU calculation frequency of the UAV local, Θ t and θ A They represent the number of CPUs required for ground IoT devices and UAVs to run 1 bit respectively.

[0037] Furthermore, in step S4, according to the basic framework of using UAV to perform task offloading and WPT for ground IoT users in the UAV-assisted edge computing scenario proposed in S2, the UAV-assisted task offloading and WPT framework is modeled as an MDP decision process (s, a, r, p, γ), where s represents the environmental state including time-varying channel gain, the size of randomly arriving tasks, and the task queue of ground IoT users, a represents the agent action including task offloading and WPT decision, r represents the reward brought by the agent performing one step of action, p is the state transition function, γ is the discount factor of reward r, γ∈(0,1), and the total utility of the system is only related to the current state and action, but not to the actions and states at other times.

[0038] Further, in step S5, according to the MDP process constructed in S4, in order to cope with the continuous action space, the deep reinforcement learning TD3 algorithm is used to optimize the objective function, and the task offloading decision and wireless energy transmission are optimized while constraining the total system throughput and UAV energy consumption. The TD3 algorithm is a deep reinforcement learning algorithm based on policy gradient, which mainly includes two Q networks, one π network and three target networks. In step S5, the main steps of the deep reinforcement learning algorithm TD3 are as follows:

[0039] (1) Initialize various parameters, UAV-assisted edge computing environment, agent network, and build the experience replay pool buffer;

[0040] (2) Select an action. The policy network π generates an action and adds exploration noise. The action is executed in the current state to obtain a reward r and the next state s'.

[0041] (3) Update the experience pool and store the current state s, action a, reward r, and next state s' into the experience buffer buffer;

[0042] (4) Sampling from the experience buffer: randomly sample a small batch (s, a, r, s') from the experience buffer;

[0043] (5) Calculate the target action, use the target policy network π_target to generate the target action a' for the next state s', and add noise;

[0044] (6) Calculate the target Q value, use the target network Q1_target and Q2_target to calculate the target Q value, Q'(s'a') = min(Q1'(s',a'), Q'2(s',a')), and use TD-target to approximate the output of the Q network y'(t) = r+γQ'(s',a');

[0045] (7) Using the TD algorithm to update the Q network, the loss function TD-error of the two Q networks is expressed as follows: Where y(t) = Q1(s,a) or Q2(s,a). After obtaining the TD-error, the gradient of the TD-error is back-propagated, and the Adam optimization algorithm is used to update the neural network parameters of the Q network. When the Q network is updated, the policy gradient algorithm is used to update the π network in a fixed iteration cycle. The following is the derived policy gradient formula for updating the π network:

[0046]

[0047] Where a is the action predicted by the π network when the state is S;

[0048] (8) Use soft update to stabilize the parameters of the target network and slowly migrate the parameters of the main network to the target network, thereby avoiding overestimation of the valuation function and improving the stability of the system. The soft update of the neural network parameters is as follows:

[0049] θ'=δθ+(1-δ)θ',

[0050] Where θ' represents the parameters of the target network, and θ represents the parameters of the main network;

[0051] (9) Repeat steps (2) to (8) and continue iterating until the algorithm converges.

[0052] Another object of the present invention is to provide a task offloading and wireless energy transmission system for implementing the task offloading and wireless energy transmission method, comprising:

[0053] The communication and computing model building module builds the communication and computing model between UAV and ground IoT users based on the changing environmental conditions and complex electromagnetic interference in the real environment, including the time-varying channel model for communication, the WPT channel model modeled using the free space loss model, and the computing and transmission model between UAV and ground IoT users;

[0054] The basic framework acquisition module of task offloading and WPT is designed based on the channel and computing model to jointly design the basic framework of using UAV to perform task offloading and WPT for ground IoT users in the UAV-assisted edge computing scenario. The system is modeled according to this framework to obtain the mathematical model of the entire system. Under this framework, at the start of each time slot, the UAV will provide reasonable task offloading and WPT decisions for ground IoT users;

[0055] The objective function module is based on the basic framework of using UAV to offload tasks and perform WPT on ground IoT users in the UAV-assisted edge computing scenario. The energy consumption of UAV and ground IoT devices and the total system throughput are calculated. The CPU computing frequency, transmission power, task size, and channel status of UAV and ground IoT users are jointly considered. By jointly formulating the optimization problem using energy consumption and total system throughput, the objective function is defined as the weighted sum of total system throughput and UAV energy consumption. The objective function is expressed as follows:

[0056]

[0057] in represents the total energy consumption of the UAV in time interval t, D all (t) represents the total system throughput, w1 and w2 represent two weight coefficients, one positive and one negative;

[0058] The reward function module models the entire system framework as a Markov process, designs a reasonable state and action space, and uses the objective function to design a reasonable reward function;

[0059] The objective function optimization module uses the deep reinforcement learning TD3 algorithm to optimize the objective function, optimizing task offloading decisions and wireless energy transmission while constraining the total system throughput and UAV energy consumption.

[0060] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the task offloading and wireless energy transmission method.

[0061] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the task offloading and wireless energy transmission method.

[0062] Another object of the present invention is to provide an information data processing terminal, which includes the task offloading and wireless energy transmission system.

[0063] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0064] First, compared with the prior art, the UAV assistance task offloading and wireless energy transmission method based on deep reinforcement learning proposed in the present invention has the following advantages:

[0065] (1) By introducing the deep reinforcement learning algorithm, the system can dynamically adjust the task offloading and energy transmission strategies according to environmental changes, thereby improving its adaptability in complex natural environments and time-varying wireless channels.

[0066] (2) Effectively balance the computing resources and energy consumption of ground IoT devices, achieve the optimal trade-off between energy consumption and system throughput through a multi-objective optimization algorithm, and improve the overall utility of the system.

[0067] (3) By adopting adaptive strategies, the system can maintain stable operation under uncertain conditions and effectively respond to challenges brought by environmental and channel changes.

[0068] (4) Through the online learning mechanism, the system can continuously update and optimize its policy model as time goes by and the environment changes, thereby maintaining long-term stability and efficient operation.

[0069] (5) Through reasonable energy management and wireless energy transmission, the dependence of ground IoT devices on battery replacement is reduced, greatly reducing manual maintenance costs.

[0070] Second, the present invention proposes a UAV-assisted task offloading and wireless power transfer (WPT) method, which aims to solve the communication and computing problems between UAV and ground IoT users in complex real-world environments. By establishing a communication and computing framework including a time-varying channel model and a WPT channel model, a joint optimization system for task offloading and energy transfer is constructed. In this system, UAV provides task offloading and energy replenishment for ground IoT users through wireless power transfer (WPT), and the overall throughput and energy consumption of the system are jointly optimized through mathematical models, ultimately aiming to maximize system performance.

[0071] Specifically, the present invention designs a joint optimization problem, taking the energy consumption of UAV and the total system throughput as the objective function, and optimizes it through deep reinforcement learning (TD3 algorithm). The key to task offloading decision and energy transmission is to dynamically adjust according to the channel state, task volume, CPU frequency, etc. in different time slots, so that the system can achieve higher task processing capabilities while lowering energy consumption.

[0072] By introducing the Markov decision process (MDP) to model the decision-making process of the entire system, reasonable state, action and reward functions are defined to ensure efficient task offloading and energy transfer under different environmental conditions. As the core part of reinforcement learning, the TD3 algorithm approximates and updates the objective function through the dual Q network and policy network, so that the system maintains stable performance in a dynamic environment.

[0073] The present invention significantly improves the task offloading efficiency and energy transmission capacity of UAV in complex environments. By combining the policy gradient method with the deep reinforcement learning algorithm, it achieves an optimization effect that cannot be solved by the existing technology, significantly reduces the energy consumption of the system, and achieves technological progress in task offloading and WPT.

[0074] Third, the present invention solves multiple technical problems in the practical industrial applications of UAV-assisted task offloading and wireless power transmission (WPT) in the prior art. First, in the traditional UAV task offloading and energy transmission system, it is often difficult to ensure the communication quality and system stability in the face of complex environmental conditions and dynamically changing channel states. By constructing a time-varying channel model and a WPT channel model, the present invention enhances the communication stability between UAV and ground IoT users, effectively copes with complex electromagnetic interference and channel randomness problems, and improves the reliability of task transmission.

[0075] Secondly, the existing UAV energy consumption optimization methods are relatively limited and it is difficult to balance the total system throughput and the energy efficiency of the UAV. This paper proposes a joint optimization strategy based on the total system throughput and UAV energy consumption, and uses deep reinforcement learning (TD3 algorithm) to adjust and optimize task offloading and wireless energy transmission in real time, so that the system can achieve the maximum task processing capability under the premise of low energy consumption. This improvement greatly improves the overall performance of the system.

[0076] Third, the traditional task offloading scheme lacks flexibility in dealing with dynamically changing environments and tasks, resulting in problems such as delayed system response and unbalanced resource allocation. The present invention dynamically adjusts the resource allocation between UAV and ground IoT users by modeling the task offloading and wireless energy transmission framework as a Markov decision process (MDP), ensuring efficient task offloading and energy transmission under different environmental conditions, and significantly improving the response speed and adaptability of the system.

[0077] Finally, the present invention has achieved significant technological progress in UAV-assisted edge computing scenarios in industrial applications, especially in the fields of Internet of Things, big data processing and smart devices. Through the combination of deep reinforcement learning algorithms and optimization problems, the system can achieve intelligent decision-making and resource optimization in complex industrial environments, reduce UAV energy consumption, improve the system's task offloading efficiency and energy transmission capacity, and provide an effective technical solution for the large-scale application of UAV in actual industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a flow chart of the task offloading and wireless energy transmission method provided by an embodiment of the present invention.

[0080] Figure 2This is a drone-assisted edge computing scenario diagram provided by an embodiment of the present invention.

[0081] Figure 3 It is the average reward graph for each round of training of the reinforcement learning algorithms TD3 and DDPG provided in the embodiments of the present invention.

[0082] Figure 4 It is a curve diagram of the total system throughput provided by an embodiment of the present invention.

[0083] Figure 5 It is a UAV energy consumption diagram provided by an embodiment of the present invention.

[0084] Figure 6 It is a structural diagram of the task offloading and wireless energy transmission system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0086] The present invention provides a method for optimizing the task offloading decision of ground IoT users and wireless energy transmission (WPT) by using DRL in a UAV-assisted edge computing scenario. Firstly, based on the real scenario and taking into account the time-varying wireless channel and the complex natural environment, the UAV-assisted edge computing scenario is systematically modeled; secondly, the time and energy consumption of the UAV and ground IoT users for computing and offloading are calculated with the help of the created system model; then, a reasonable optimization function is constructed according to the formulated system model and system simulation quantity; the entire system framework is modeled as a Markov decision process, and a reasonable state-action space and reward function are formulated; finally, a deep reinforcement learning algorithm is used to comprehensively optimize the formulated optimization function, so as to optimize the task offloading decision and wireless energy transmission while constraining the total system throughput and UAV energy consumption.

[0087] like Figure 1 and Figure 2 As shown, the present invention provides a task offloading and wireless energy transmission method based on deep reinforcement learning in an edge computing scenario assisted by a drone, which specifically includes the following steps:

[0088] S1. According to the changing environmental conditions and complex electromagnetic interference in the real environment, the communication and computing model between UAV and ground IoT users is established, including the time-varying channel model for communication, the WPT channel model modeled by the free space loss model, and the computing and transmission model between UAV and ground IoT users. The model is as follows Figure 2 As shown;

[0089] S11. Establish a service scenario consisting of a UAV equipped with an edge computing server and a WPT radio frequency device and multiple ground users. Each ground user is equipped with an energy receiving device with the same frequency as the UAV radio frequency device and a rechargeable battery to store the collected energy. Both the UAV and the ground user are equipped with a single CPU and a certain amount of memory space to store computing tasks;

[0090] S12. Convert the continuous time operation mechanism into time slots of equal length, with the time interval of each time slot being Τ, and the time slot set being τ = {1, 2, 3, ..., i, ..., T}. Considering the real scenario, the randomly arriving computing tasks are made to obey the exponential distribution, and each task is independent and identically distributed;

[0091] like Figure 2 As shown, the UAV in the air is represented by A and its three-dimensional coordinates are € A =(x A ,y A ,z A ), the ground IoT user is represented by t and its three-dimensional coordinates are € t =(x t ,y t ,z t ), so the distance between the UAV in the air and the IoT user on the ground can be expressed as D t Indicates that D t =||X A -X t ||; Considering the variability of the actual situation and the randomness of the channel, a communication channel between UAV and ground IoT users is established, aiming to use UAV in the air to provide services to ground users. UAVs generally stay at an altitude of several kilometers, and the probability of being blocked by ground IoT users is small, so there will be relatively stable communication quality and low latency. It is believed that the communication link between UAV and ground IoT users is mainly dominated by the line-of-sight (LOS) channel. In order to minimize the communication interference between each IoT user on the ground and the UAV in the air, the communication between UAV and a single IoT user adopts the orthogonal frequency division multiple access (OFDMA) protocol. The channel state is static in a time slot, but dynamic in different time slots. It is assumed that the average channel gain follows the path loss model:

[0092]

[0093] in represents the average channel gain between the UAV and the tth IoT user on the ground, A d represents the antenna gain of each IoT user, f c Indicates the signal carrier frequency, d crepresents the path loss index, and the channel gain h between each IoT user and the UAV A,t (t) obeys independent and identically distributed Rayleigh distribution.

[0094] Then, the WPT channel was modeled. It was considered that the link between UAV and ground IoT users for WPT was dominated by the LOS (Line of Sight) link. The free space loss model was used to model the WPT transmission link:

[0095]

[0096] in represents the energy received by the ground IoT device at time t, v t ∈(0,1) represents the wireless charging conversion efficiency of the tth ground IoT device, g t It represents the channel gain of the tth ground IoT device. It represents the WPT action of UAV to the tth terrestrial IoT device, and indicates how much energy to be transmitted to the terrestrial IoT device.

[0097] After obtaining the distance and channel gain between the UAV and the ground IoT user, the channel transmission rate R between the UAV and the tth ground IoT device can be obtained according to the Shannon formula: t :

[0098]

[0099] Where B represents the total channel bandwidth, φ represents the power spectral density of the noise signal, and P t represents the signal transmission power of the tth ground IoT device to the UAV, T represents the number of ground IoT users. Since the amount of data obtained after calculation by the MEC server is very small, the consumption of the task results after the MEC server offloads the calculation is ignored. According to the channel transmission rate obtained above, represents the amount of tasks uploaded by the t-th ground IoT user at time t. This amount of tasks is determined by the task queue size of the local user, the offloading decision made by the reinforcement learning agent, and the computing tasks generated by the ground IoT user at time t. Therefore, the task upload time and task upload energy It is expressed as:

[0100]

[0101] S2. Based on the channel and computing model proposed in S1, a basic framework for using UAV to perform task offloading and WPT for ground IoT users in the UAV-assisted edge computing scenario is jointly designed. System modeling is performed based on this framework to obtain the mathematical model of the entire system. Under this framework, at the start of each time slot, the UAV will provide reasonable task offloading and WPT decisions for ground IoT users;

[0102] S3. Based on the basic framework of using UAV to perform task offloading and WPT for terrestrial IoT users in the UAV-assisted edge computing scenario proposed in S2, the energy consumption of UAV and terrestrial IoT devices and the total system throughput are calculated. The CPU computing frequency, transmission power, task size and channel status of UAV and terrestrial IoT users are jointly considered. By jointly formulating the optimization problem using energy consumption and total system throughput, the objective function is defined as the weighted sum of the total system throughput and UAV energy consumption.

[0103] A UAV-based task offloading and WPT framework is designed. Specifically, the energy consumption of UAV and ground IoT devices and the overall throughput of the system are calculated. In this process, the CPU computing frequency, transmission power, task size, and channel status of UAV and ground IoT users are comprehensively considered. By jointly analyzing the energy consumption and the total system throughput, an optimization problem is formulated, and its objective function is defined as the weighted sum of the total system throughput and the UAV energy consumption. The objective function is expressed as follows:

[0104]

[0105] in represents the total energy consumption of the UAV in time interval t, in It means that the UAV calculates the energy consumption at time t. It is expressed as:

[0106]

[0107] Where k represents the computing energy efficiency factor, which is related to the CPU chip structure of the edge server equipped by the UAV, and f A represents the CPU calculation frequency of UAV, and D all (t) represents the total throughput of the system at time t, and They represent the local computing tasks of the ground IoT user and the UAV at time t. It is worth noting that the tasks processed by the UAV in each time slot are taken from its own memory space for storing tasks, rather than the tasks uploaded by each ground user in each time slot. The local computing tasks of the UAV and the local IoT user need to comply with the following conditions:

[0108]

[0109] where α t (t) and α A (t) are the task queue storage sizes of ground IoT devices and aerial UAVs, represents the energy consumed by ground IoT users for computing at time t, Only when UAV and ground IoT devices meet the above conditions and The following equation is satisfied:

[0110]

[0111] where f A Indicates the CPU calculation frequency of the UAV local, Θ t and θ A They represent the number of CPUs required for ground IoT devices and UAVs to run 1 bit respectively;

[0112] S4. Model the entire system framework as a Markov process, design a reasonable state and action space, and use the objective function to design a reasonable reward function;

[0113] The UAV-assisted task offloading and WPT framework is modeled as an MDP decision process (s, a, r, p, γ), where s represents the environmental state including the time-varying channel gain, the size of randomly arriving tasks, and the task queue of ground IoT users, a represents the agent action including task offloading and WPT decisions, r represents the reward brought by the agent performing one step of action, p is the state transition function, γ is the discount factor of the reward r, γ∈(0,1), and the total utility of the system is only related to the current state and action, but not to the actions and states at other times.

[0114] S5. Use the deep reinforcement learning TD3 algorithm to optimize the objective function, while constraining the total system throughput and UAV energy consumption to optimize task offloading decisions and wireless energy transmission;

[0115] According to the MDP process constructed in S4, in order to cope with the continuous action space, the deep reinforcement learning TD3 algorithm is used to optimize the objective function, while constraining the total system throughput and UAV energy consumption to optimize the task offloading decision and wireless energy transmission. The TD3 algorithm is a deep reinforcement learning algorithm based on policy gradient, which mainly includes two Q networks, one π network and three target networks. In step S5, the main steps of the deep reinforcement learning algorithm TD3 are as follows:

[0116] (1) Initialize various parameters, UAV-assisted edge computing environment, agent network, and build the experience replay pool buffer;

[0117] (2) Select an action. The policy network π generates an action and adds exploration noise. The action is executed in the current state to obtain a reward r and the next state s'.

[0118] (3) Update the experience pool and store the current state s, action a, reward r, and next state s' into the experience buffer buffer;

[0119] (4) Sampling from the experience buffer: randomly sample a small batch (s, a, r, s') from the experience buffer;

[0120] (5) Calculate the target action, use the target policy network π_target to generate the target action a' for the next state s', and add noise;

[0121] (6) Calculate the target Q value, use the target network Q1_target and Q2_target to calculate the target Q value, Q'(s'a') = min(Q1'(s',a'), Q'2(s',a')), and use TD-target to approximate the output of the Q network y'(t) = r+γQ'(s',a');

[0122] (7) Using the TD algorithm to update the Q network, the loss function TD-error of the two Q networks is expressed as follows: Where y(t) = Q1(s,a) or Q2(s,a). After obtaining the TD-error, the gradient of the TD-error is back-propagated, and the Adam optimization algorithm is used to update the neural network parameters of the Q network. When the Q network is updated, the policy gradient algorithm is used to update the π network in a fixed iteration cycle. The following is the derived policy gradient formula for updating the π network:

[0123]

[0124] Where a is the action predicted by the π network when the state is S;

[0125] (8) Use soft update to stabilize the parameters of the target network and slowly migrate the parameters of the main network to the target network, thereby avoiding overestimation of the valuation function and improving the stability of the system. The soft update of the neural network parameters is as follows:

[0126] θ'=δθ+(1-δ)θ',

[0127] Where θ' represents the parameters of the target network, and θ represents the parameters of the main network;

[0128] (9) Repeat steps (2) to (8) and continue iterating until the algorithm converges.

[0129] like Figure 3 As shown in the figure, it represents the entire learning process of the TD3 and DDPG algorithms. The actor and critic networks of the Agent in the two algorithms respectively use the sigmoid activation function. It can be seen that in the initial stage of model training, the average rewards of the two Agents gradually increase. This is because the Agent continuously improves its own neural network in the process of interaction with the environment, so that the action output of the π network is aligned with the direction of increasing rewards. After comparison, the overall convergence reward of the TD3 algorithm is greater than that of the DDPG algorithm. After the model converges, a higher average reward will be obtained. This is because the TD3 algorithm reduces the over-estimation bias of the environment Q value by introducing double Q learning, thereby improving the stability and performance of training.

[0130] like Figure 4 As shown in the figure, it shows the total system throughput curve obtained by using the TD3 algorithm and DDPG algorithm agents that have converged in training to interact with the test environment for a round of 5000 steps. The number of users is 10 and the task arrival rate is 2.4Mbps. By comparison, when the task arrival rate is 2.4Mbps, both algorithms can maintain system stability and keep the system's task queue in a relatively reasonable state. However, it is worth noting that when the time step is 0-1000 steps, the system throughput caused by the TD3 algorithm has a larger average value than the other two comparison algorithms, resulting in the pressure on the task storage space of the system using the TD3 algorithm agent being less than that of the DDPG algorithm.

[0131] like Figure 5 As shown in the figure, it shows the UAV energy consumption diagram obtained by using the TD3 algorithm and DDPG algorithm agents that have been trained to converge and interacting with the test environment for a round of 5000 steps. By comparison, the average energy consumption of the system using the TD3 algorithm is slightly smaller than that of the DDPG algorithm. This shows that the TD3 algorithm can more effectively balance the system throughput and energy consumption when dealing with task offloading and wireless energy transmission problems, and achieve better energy consumption management in complex environments. In addition, due to its dual Critic network structure, the TD3 algorithm can effectively reduce the deviation of Q-value estimation, thereby further improving energy efficiency.

[0132] like Figure 6 As shown, the task offloading and wireless energy transmission system provided by the embodiment of the present invention includes:

[0133] The communication and computing model building module builds the communication and computing model between UAV and ground IoT users based on the changing environmental conditions and complex electromagnetic interference in the real environment, including the time-varying channel model for communication, the WPT channel model modeled using the free space loss model, and the computing and transmission model between UAV and ground IoT users;

[0134] The basic framework acquisition module of task offloading and WPT is designed based on the channel and computing model to jointly design the basic framework of using UAV to perform task offloading and WPT for ground IoT users in the UAV-assisted edge computing scenario. The system is modeled according to this framework to obtain the mathematical model of the entire system. Under this framework, at the start of each time slot, the UAV will provide reasonable task offloading and WPT decisions for ground IoT users;

[0135] The objective function module is based on the basic framework of using UAV to offload tasks and perform WPT on ground IoT users in the UAV-assisted edge computing scenario. The energy consumption of UAV and ground IoT devices and the total system throughput are calculated. The CPU computing frequency, transmission power, task size, and channel status of UAV and ground IoT users are jointly considered. By jointly formulating the optimization problem using energy consumption and total system throughput, the objective function is defined as the weighted sum of total system throughput and UAV energy consumption. The objective function is expressed as follows:

[0136]

[0137] in represents the total energy consumption of the UAV in time interval t, D all (t) represents the total system throughput, w1 and w2 represent two weight coefficients, one positive and one negative;

[0138] The reward function module models the entire system framework as a Markov process, designs a reasonable state and action space, and uses the objective function to design a reasonable reward function;

[0139] The objective function optimization module uses the deep reinforcement learning TD3 algorithm to optimize the objective function, optimizing task offloading decisions and wireless energy transmission while constraining the total system throughput and UAV energy consumption.

[0140] An application embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor performs the steps of the task unloading and wireless energy transmission method.

[0141] An application embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the task unloading and wireless energy transmission method.

[0142] An application embodiment of the present invention provides an information data processing terminal, which includes a task offloading and wireless energy transmission system.

[0143] The following are two embodiments of the present invention in industrial applications:

[0144] 1. UAV task offloading and wireless energy transmission applications in smart agriculture

[0145] In smart agriculture, unmanned aerial vehicles (UAVs) are often used for tasks such as large-scale farmland monitoring, pest and disease detection, and soil composition analysis. However, faced with vast farmland, the battery power of ground sensors and monitoring equipment is easily exhausted, and the amount of mission data is large, and the calculation and transmission efficiency become bottlenecks. Through the method of the present invention, UAV can not only offload tasks to ground sensors and process big data in real time, but also charge ground equipment through wireless power transmission (WPT) to extend its working life. This application significantly improves the continuous working ability of the agricultural monitoring system, reduces the cost of frequent battery replacement, and improves the efficiency of farmland monitoring and data processing.

[0146] 2. IoT equipment maintenance and energy transmission applications in smart cities

[0147] In the Internet of Things (IoT) applications of smart cities, the operation of environmental sensors, traffic monitoring equipment and public facilities in the city depends on electricity and network connections. However, due to the wide distribution of equipment and dispersed geographical locations, energy supply and communication issues often become bottlenecks. The present invention introduces UAV-assisted task offloading and wireless energy transmission systems. The UAV can transmit energy to IoT devices distributed in different areas of the city during cruising, while reducing local computing pressure through task offloading, and uploading large amounts of data to the cloud or edge servers for processing. This application improves the maintenance efficiency of smart city equipment, reduces maintenance costs, and ensures the continuity and real-time nature of urban environmental data.

[0148] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0149] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A task offloading and wireless energy transmission method, characterized in that: The steps include: S1. Based on the changing environmental conditions and complex electromagnetic interference in the real environment, the communication and computing model between UAV and ground IoT users was established, including the time-varying channel model for communication, the WPT channel model modeled by the free space loss model, and the computing and transmission model between UAV and ground IoT users; S2. Based on the channel and computing model proposed in S1, a basic framework for using UAV to perform task offloading and WPT for ground IoT users in the UAV-assisted edge computing scenario is jointly designed. System modeling is performed based on this framework to obtain the mathematical model of the entire system. Under this framework, at the start of each time slot, the UAV will provide reasonable task offloading and WPT decisions for ground IoT users; S3. Based on the basic framework of using UAV to offload tasks and perform WPT for ground IoT users in the UAV-assisted edge computing scenario proposed in S2, the energy consumption of UAV and ground IoT devices and the total system throughput are calculated. The CPU computing frequency, transmission power, task size, and channel status of UAV and ground IoT users are jointly considered. By jointly formulating the optimization problem using energy consumption and total system throughput, the objective function is defined as the weighted sum of total system throughput and UAV energy consumption. The objective function is expressed as follows: in represents the total energy consumption of the UAV in time interval t, D all (t) represents the total system throughput, w1 and w2 represent two weight coefficients, one positive and one negative; S4. Model the entire system framework as a Markov process, design a reasonable state and action space, and use the objective function to design a reasonable reward function; S5. Use deep reinforcement learning TD3 algorithm to optimize the objective function, while constraining the total system throughput and UAV energy consumption to optimize task offloading decisions and wireless energy transmission.

2. The task offloading and wireless energy transmission method according to claim 1, characterized in that: S1 specifically includes: S11. Establish a service scenario consisting of a UAV equipped with an edge computing server and a WPT radio frequency device and multiple ground users. Each ground user is equipped with an energy receiving device with the same frequency as the UAV radio frequency device and a rechargeable battery to store the collected energy. Both the UAV and the ground user are equipped with a single CPU and a certain amount of memory space to store computing tasks; S12. Convert the continuous time operation mechanism into time slots of equal length, with the time interval of each time slot being Τ, and the time slot set being τ = {1, 2, 3, ..., i, ..., T}. Considering the real scenario, the randomly arriving computing tasks are made to obey the exponential distribution, and each task is independent and identically distributed; In S1, it is considered that the UAV in the air equipped with edge servers and wireless radio frequency charging devices provides services to the IoT users on the ground. The UAV in the air is represented by A and its three-dimensional coordinates are The three-dimensional coordinates of the IoT user on the ground are represented by t. So the distance between the UAV in the air and the IoT user on the ground can be expressed as D t Indicates that D t =||X A -X t ||.

3. The task offloading and wireless energy transmission method according to claim 1, characterized in that: In step S1, considering the variability of the actual situation and the randomness of the channel, the communication channel between the UAV and the ground IoT user is established, aiming to use the UAV in the air to provide services to the ground users. The UAV generally stays at an altitude of several kilometers, and the probability of being blocked by the ground IoT users is small, so there will be a relatively stable communication quality and low latency. It is believed that the communication link between the UAV and the ground IoT user is mainly dominated by the line-of-sight (LOS) channel. In order to minimize the communication interference between each IoT user on the ground and the UAV in the air, the communication between the UAV and a single IoT user adopts the orthogonal frequency division multiple access (OFDMA) protocol. The channel state is static in a time slot, but dynamic in different time slots. It is assumed that the average channel gain follows the path loss model: in represents the average channel gain between the UAV and the tth IoT user on the ground, A d represents the antenna gain of each IoT user, f c Indicates the signal carrier frequency, d c represents the path loss index, and the channel gain h between each IoT user and the UAV A,t (t) obeys independent and identically distributed Rayleigh distribution.

4. The task offloading and wireless energy transmission method according to claim 1, characterized in that: In step S1, the WPT channel is modeled. It is considered that the link between UAV and ground IoT users for WPT is dominated by the LOS (Line of Sight) link. The free space loss model is used to model the WPT transmission link: in represents the energy received by the ground IoT device at time t, v t ∈(0,1) represents the wireless charging conversion efficiency of the tth ground IoT device, g t It represents the channel gain of the tth ground IoT device. It represents the WPT action of UAV to the tth terrestrial IoT device, indicating how much energy to transmit to the terrestrial IoT device.

5. The task offloading and wireless energy transmission method according to claim 1, characterized in that: In step S1, the channel transmission rate R between the UAV and the tth ground IoT device can be obtained according to the Shannon formula: t : Where B represents the total channel bandwidth, φ represents the power spectral density of the noise signal, and P t represents the signal transmission power of the tth ground IoT device to the UAV, T represents the number of ground IoT users. Since the amount of data obtained after calculation by the MEC server is very small, the consumption of the task results after the MEC server offloads the calculation is ignored. According to the channel transmission rate obtained above, represents the amount of tasks uploaded by the t-th ground IoT user at time t. This amount of tasks is determined by the size of the local user's task queue, the offloading decision made by the reinforcement learning agent, and the computing tasks generated by the ground IoT user at time t. Therefore, the task upload time and task upload energy It is expressed as:

6. The task offloading and wireless energy transmission method according to claim 1, characterized in that: In steps S2 and S3, a UAV-based task offloading and WPT framework is designed for ground IoT users. Specifically, the energy consumption of UAV and ground IoT devices and the overall throughput of the system are calculated. In this process, the CPU computing frequency, transmission power, task size, and channel status of UAV and ground IoT users are comprehensively considered. By jointly analyzing the energy consumption and the total system throughput, an optimization problem is formulated, and its objective function is defined as the weighted sum of the total system throughput and the UAV energy consumption. The objective function is expressed as follows: in represents the total energy consumption of the UAV in time interval t, in It means that the UAV calculates the energy consumption at time t. It is expressed as: Where k represents the computing energy efficiency factor, which is related to the CPU chip structure of the edge server equipped by the UAV, and f A represents the CPU calculation frequency of UAV, and D all (t) represents the total throughput of the system at time t, and They represent the local computing tasks of the ground IoT user and the UAV at time t. It is worth noting that the tasks processed by the UAV in each time slot are taken from its own memory space for storing tasks, rather than the tasks uploaded by each ground user in each time slot. The local computing tasks of the UAV and the local IoT user need to comply with the following conditions: where α t (t) and α A (t) are the task queue storage sizes of ground IoT devices and aerial UAVs, represents the energy consumed by ground IoT users for computing at time t, Only when UAV and ground IoT devices meet the above conditions and The following equation is satisfied: where f A Indicates the CPU calculation frequency of the UAV local, Θ t and θ A They represent the number of CPUs required for ground IoT devices and UAVs to run 1 bit respectively.

7. The task offloading and wireless energy transmission method according to claim 1, characterized in that: In step S4, according to the basic framework of using UAV to perform task offloading and WPT for ground IoT users in the UAV-assisted edge computing scenario proposed in S2, the UAV-assisted task offloading and WPT framework is modeled as an MDP decision process (s, a, r, p, γ), where s represents the environmental state including time-varying channel gain, the size of randomly arriving tasks, and the task queue of ground IoT users, a represents the agent action including task offloading and WPT decision, r represents the reward brought by the agent performing one step of action, p is the state transition function, γ is the discount factor of reward r, γ∈(0,1), and the total utility of the system is only related to the current state and action, but not to the actions and states at other times.

8. The task offloading and wireless energy transmission method according to claim 1, characterized in that: In step S5, according to the MDP process constructed in S4, in order to cope with the continuous action space, the deep reinforcement learning TD3 algorithm is used to optimize the objective function, and the task offloading decision and wireless energy transmission are optimized while constraining the total system throughput and UAV energy consumption. The TD3 algorithm is a deep reinforcement learning algorithm based on policy gradient, which mainly includes two Q networks, one π network and three target networks. In step S5, the main steps of the deep reinforcement learning algorithm TD3 are as follows: (1) Initialize various parameters, UAV-assisted edge computing environment, agent network, and build the experience replay pool buffer; (2) Select an action. The policy network π generates an action and adds exploration noise. The action is executed in the current state to obtain a reward r and the next state s'. (3) Update the experience pool and store the current state s, action a, reward r, and next state s' into the experience buffer; (4) Sampling from the experience buffer: randomly sample a small batch (s, a, r, s') from the experience buffer; (5) Calculate the target action, use the target policy network π_target to generate the target action a' for the next state s', and add noise; (6) Calculate the target Q value, use the target network Q1_target and Q2_target to calculate the target Q value, Q'(s'a') = min(Q1'(s',a'), Q'2(s',a')), and use TD-target to approximate the output of the Q network y'(t) = r+γQ'(s',a'); (7) Using the TD algorithm to update the Q network, the loss function TD-error of the two Q networks is expressed as follows: Where y(t) = Q1(s,a) or Q2(s,a). After obtaining the TD-error, the gradient of the TD-error is back-propagated, and the Adam optimization algorithm is used to update the neural network parameters of the Q network. When the Q network is updated, the policy gradient algorithm is used to update the π network in a fixed iteration cycle. The following is the derived policy gradient formula for updating the π network: Where a is the action predicted by the π network when the state is S; (8) Use soft update to stabilize the parameters of the target network and slowly migrate the parameters of the main network to the target network, thereby avoiding overestimation of the valuation function and improving the stability of the system. The soft update of the neural network parameters is as follows: θ'=δθ+(1-δ)θ', Where θ' represents the parameters of the target network, and θ represents the parameters of the main network; (9) Repeat steps (2) to (8) and continue iterating until the algorithm converges.

9. A task offloading and wireless energy transmission system for implementing the task offloading and wireless energy transmission method according to any one of claims 1 to 8, characterized in that: include: The communication and computing model building module builds the communication and computing model between UAV and ground IoT users based on the changing environmental conditions and complex electromagnetic interference in the real environment, including the time-varying channel model for communication, the WPT channel model modeled using the free space loss model, and the computing and transmission model between UAV and ground IoT users; The basic framework acquisition module of task offloading and WPT is designed based on the channel and computing model to jointly design the basic framework of using UAV to perform task offloading and WPT for ground IoT users in the UAV-assisted edge computing scenario. The system is modeled according to this framework to obtain the mathematical model of the entire system. Under this framework, at the start of each time slot, the UAV will provide reasonable task offloading and WPT decisions for ground IoT users; The objective function module is based on the basic framework of using UAV to offload tasks and perform WPT on ground IoT users in the UAV-assisted edge computing scenario. The energy consumption of UAV and ground IoT devices and the total system throughput are calculated. The CPU computing frequency, transmission power, task size, and channel status of UAV and ground IoT users are jointly considered. By jointly formulating the optimization problem using energy consumption and total system throughput, the objective function is defined as the weighted sum of total system throughput and UAV energy consumption. The objective function is expressed as follows: in represents the total energy consumption of the UAV in time interval t, D all (t) represents the total system throughput, w1 and w2 represent two weight coefficients, one positive and one negative; The reward function module models the entire system framework as a Markov process, designs a reasonable state and action space, and uses the objective function to design a reasonable reward function; The objective function optimization module uses the deep reinforcement learning TD3 algorithm to optimize the objective function, optimizing task offloading decisions and wireless energy transmission while constraining the total system throughput and UAV energy consumption.

10. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the task offloading and wireless energy transmission method according to any one of claims 1 to 8.

Images