Unmanned aerial vehicle assisted calculation unloading method and system and computer storage medium

Abstract

The invention provides an unmanned aerial vehicle assisted calculation unloading method and system and a computer storage medium, and belongs to the field of edge calculation based on unmanned aerial vehicles. The method comprises the following steps: establishing an unmanned aerial vehicle assisted calculation unloading model, wherein the model comprises an Internet of Things equipment layer, an edge layer and a cloud layer; in the calculation unloading model assisted by the unmanned aerial vehicle, adopting # imgabs0 # to represent whether the unmanned aerial vehicle u provides service for the Internet of Things equipment k in the time slot t; establishing a local Internet of Things equipment calculation scene, an unmanned aerial vehicle auxiliary edge calculation scene and a cloud-edge-end calculation scene; with the aim of minimizing the weighted sum of delay and energy consumption of all tasks, obtaining an unmanned aerial vehicle which is most suitable for serving the Internet of Things equipment, and calculating an optimal calculation unloading scene by the unmanned aerial vehicle; and completing calculation unloading at the optimal unloading scene. According to the method, the energy consumption speed during network computing task unloading is reduced, and it is ensured that delay requirements of related computing tasks are effectively met.

Description

Technical Field

[0001] The present invention belongs to the field of drone-based edge computing, and specifically relates to a drone-assisted computing offloading method, system and computer storage medium. Background Art

[0002] With the development of the Internet of Things, more and more IoT devices are being added to the Internet of Things, including various sensors, high-definition monitors, augmented reality (AR), virtual reality (VR) and digital twin services; cloud computing provides efficient, flexible and scalable computing, storage and management resources for the Internet of Things, and provides reliable backend support and data analysis for IoT devices. However, with the increasing number of IoT devices, the generation and processing of data has become more and more complicated; in order to obtain a better application experience, edge computing technology is generally used to improve the response speed of applications. However, edge computing is an open platform that integrates network, computing, storage and application core capabilities on the side close to the source of objects or data, providing the nearest service nearby and initiating it on the edge side, so as to generate faster network service response.

[0003] However, emerging latency-sensitive applications such as AR, VR, and digital twins generate a large amount of data. Considering the ultra-long transmission distance, if all data is transmitted to the cloud server, there will still be huge challenges in meeting the latency requirements of the application. At the edge of the network, drones can act as edge servers, equivalent to network repeaters, thereby providing some computing services for IoT devices. This setting can significantly reduce the overall computing latency and save limited energy for IoT devices.

[0004] In the prior art, drone-based edge computing uses drones to complete complex computing tasks at the edge of the network to achieve the comprehensive optimization of task computing latency and total energy consumption of IoT devices. For example, a drone-assisted edge computing network task offloading solution based on traditional algorithms (heuristics) proposed by some scholars has a simple design, easy implementation, and relatively low computational complexity, and can run on resource-constrained IoT devices. However, traditional algorithms have many drawbacks when making task offloading decisions. Traditional algorithms usually rely on predefined rules and experience and lack adaptability; in addition, such algorithms are usually inefficient in solving large-scale and high-dimensional problems, and it is difficult to find the global optimal solution within a limited time.

[0005] The Chinese patent application number CN202410760607.3 discloses a method for optimizing the task offloading of a drone swarm in collaboration with mobile edge computing based on D2D. A mobile edge computing network model including a terminal user layer, an edge layer, and a cloud layer is established. Based on the model, a transmission model, a computing model, and an energy consumption model are established. Combined with the D2D offloading technology, the utilization rate of drone computing resources is improved, while the transmission energy consumption is reduced and energy and channel resources are saved, thereby reducing the burden on the edge server on a single drone. Specifically, by jointly optimizing the task allocation ratio, transmission power, and flight parameters of the drone, the MASAC algorithm and the genetic algorithm are used to iterate and obtain the optimal computing task offloading decision and drone location deployment decision. However, the above scheme takes into account the factors of how the IoT device selects which drone in the drone swarm, and the delay problem of the system in task offloading optimization is not well solved; in addition, the system uses the MASAC algorithm to jointly optimize the computing task allocation and communication resource management, which is difficult to train and requires more computing resources.

[0006] For another example, the Chinese patent with application number CN202310257281.8 discloses a method and device for unloading edge computing tasks of drones for power inspection. By establishing two models, namely, a control model for controlling the flight of drones and a computing task model for different power IoT devices in different time slots, efficient edge computing task unloading of ground power IoT devices is completed, service quality is improved and task delay is reduced. Specifically, for the control model, the flight control parameters of the power inspection drone in the target time slot, the position of each power IoT device and the edge computing task amount of each power IoT device are taken as inputs of the model, and the flight control parameters of the power inspection drone in the target time slot can be obtained. For the computing task model of the power IoT device, the task unloading rate of each power IoT device in the time slot is determined based on the results of the control model, the edge computing task amount of the time slot, and the available computing resources of each power IoT device. In this process, the training sample data is generated by using the generative adversarial network to train the task unloading method based on the MATD3 algorithm, so as to minimize the energy consumption of the inspection drone in a time slot and the average delay of processing edge computing tasks. However, the above scheme has the following shortcomings: the designed drone power inspection system does not involve cloud-edge-end collaboration, and the computing tasks are concentrated on the power Internet of Things devices. The technical complexity is relatively low and the effect is poor; the drone is only used as an information collection carrier, and can neither undertake computing tasks nor serve as the main body of the decision-making task offloading scheme, which limits the overall decision-making efficiency and accuracy of the designed system; the designed system completely ignores the energy consumption of the power Internet of Things devices. If the power Internet of Things devices lack power supply, some power Internet of Things devices are at risk of running out of energy too quickly, which seriously limits the application of this patent in passive wireless scenarios and has poor universality. Summary of the invention

[0007] In view of the above-mentioned defects or deficiencies in the prior art, the present invention aims to provide a drone-assisted computing offloading method, system and computer storage medium to minimize the energy consumption rate of IoT devices and drones as much as possible, and ensure that the delay requirements of related computing tasks are effectively met.

[0008] In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solution:

[0009] In a first aspect, an embodiment of the present invention provides a method for unmanned aerial vehicle-assisted computing offloading, the method comprising the following steps:

[0010] Step S1, establishing a drone-assisted computing offloading model, the model includes an IoT device layer, an edge layer, and a cloud layer; wherein,

[0011] The IoT device layer includes several IoT devices with different battery capacities, and the IoT devices are immovable, represented by k = {1, 2, ... K}. In time slot t, IoT device k continuously generates computing tasks m, expressed by the following formula: m = {1, 2, ... M}, and reports computing requests to the drone; the size of task m is given by express;

[0012] The edge layer includes a group of drones, represented by u={1, 2, ... U}, which provide communication relay and on-the-way computing for IoT devices; edge computing servers are deployed on the drones, and the edge computing servers are used to determine offloading scenarios and provide computing offloading in edge layer computing scenarios;

[0013] The cloud layer includes high-performance servers, supercomputing centers and / or artificial intelligence computing groups, which receive IoT data forwarded from drones, provide high-quality services for computing-intensive tasks, and transmit computing results back to drones through base stations;

[0014] Step S2, in the UAV-assisted computational offloading model, adopt Indicates whether drone u provides service to IoT device k in time slot t; if IoT device k is connected to drone u, then otherwise

[0015] There are the following limitations on the connection between drones and IoT devices:

[0016] i) There are no remaining tasks to be processed at the end of time slot t;

[0017] ii) Each IoT device can only connect to one drone in the same time slot;

[0018] The formula is:

[0019]

[0020] In formula (1), U is the total number of drones; Represents any value;

[0021] Step S3, under the drone-assisted computing offloading model, three task computing offloading scenarios are established, including: local IoT device computing scenario, drone-assisted edge computing scenario, and cloud-edge-end computing scenario; and a single computing task is completely processed in one scenario;

[0022] Step S4, based on the three established task computing offloading scenarios, with the goal of minimizing the weighted sum of delays and energy consumption of all tasks, the transmission power and task processing parameters of the IoT device, the transmission power and task processing parameters of the drone, the available energy of the drone and the IoT device, the size of the task to be processed and the tolerable delay are combined to obtain the drone that is most suitable for serving the IoT device, and the drone calculates the optimal computing offloading scenario;

[0023] Step S5, completing the calculation offloading in the optimal offloading scenario.

[0024] As a preferred embodiment of the present invention, in step S2, in order to realize that each IoT device can only be connected to one drone in the same time slot, the three-dimensional coordinates of the drone u are represented as L u (t) = {X u (t),Y u (t),H u}, X u (t), Y u (t) and H u are the X, Y, and Z coordinates of the drone u; IoT devices are immovable, and the three-dimensional coordinates are represented by L k (t) = {X k ,Y k ,H k}; where X k , Y k , H u and H k is a constant; and:

[0025] ‖L u (t)-L k (t)‖≤d max (twenty four)

[0026] In formula (2), d max is the distance threshold between the IoT device and the drone;

[0027] If the distance between the IoT device and the drone exceeds d max , drones will deny IoT devices access.

[0028] As a preferred embodiment of the present invention, the calculation process in the local IoT device calculation scenario in step S3 is as follows:

[0029] set up Indicates whether task m is processed by the local IoT device k; if task m is processed by the local IoT device, then otherwise

[0030] The delay from the generation to the completion of task m only considers the computational delay, as shown in formula (3):

[0031]

[0032] In formula (3), is the total latency of local IoT device k processing task m, is the time consumed by the local IoT device k to process task m, f k represents the CPU frequency of the local IoT device k, C k Indicates the CPU cycles required to process 1 bit of data;

[0033] Weighted energy consumption of the whole process for:

[0034]

[0035] In formula (4), Ω k is the effective switched capacitance of the IoT device k, ω k is the importance coefficient of IoT device k, and all local IoT devices share the same parameters in the model.

[0036] As a preferred embodiment of the present invention, the calculation process in the drone-assisted edge computing scenario in step S3 is as follows:

[0037] set up Indicates whether task m is processed by the drone; if task m is processed by the drone, then otherwise

[0038] In this scenario, the computing task is processed by the edge computing server deployed on the drone; IoT device k generates task m and sends the raw data to the edge computing server. The data is transmitted to the corresponding UAV u, which then processes the data and sends the result data to the corresponding UAV u in time slot t. Return to the IoT device, therefore, the drone-assisted edge computing model The total delay is given by

[0039]

[0040] In formula (5), and They represent the data sending delay, result receiving delay and drone processing delay of task m in IoT device k respectively;

[0041] Drone processing delay It is expressed as:

[0042]

[0043] In formula (6), C u The CPU cycles required for drone u to process 1 bit of data, f u The frequency of the drone CPU;

[0044] The delay in sending data and receiving results is expressed as:

[0045]

[0046] In formula (7), and are the uplink and downlink transmission rates between drone u and IoT device k, and:

[0047]

[0048] In formula (8), B up and B down They are used to represent the bandwidth of uplink and downlink respectively; h m (t) represents the propagation loss, and They represent the transmission power of IoT devices and drones respectively; and denote the terminal Gaussian noise power of the UAV and IoT device respectively;

[0049] Weighted energy consumption of the whole process Energy transmitted by IoT devices Drones transmit energy and processing energy Composed and obtained by:

[0050]

[0051] In formula (9), Ω u is the effective switching capacitance of UAV u, ωu is the importance coefficient of UAV u, and all UAVs share the same parameters; ω k is the importance coefficient of IoT device k.

[0052] As a preferred embodiment of the present invention, the calculation process in the cloud-edge-device computing scenario in step S3 is as follows:

[0053] set up Indicates whether task m is processed on the cloud server; if task m is processed on the cloud server, then otherwise

[0054] In this scenario, IoT device k creates a computing task m and sends raw data to drone u; the drone then relays the data to the cloud server through the base station; once the cloud server completes the task processing, it sends the result back to the drone, which eventually returns the result to IoT device k within time slot t; the total delay of the whole process is It can be obtained by the following formula:

[0055]

[0056] In formula (10), and It is the end-to-end total transmission delay and total reception delay of data from IoT devices to cloud servers; the processing delay of cloud servers is considered as 0, and Given by the following formula

[0057]

[0058] In formula (11), ξ m is the propagation delay from the cloud to the base station, Indicates the size of the calculation result returned by the cloud server; and are the uplink and downlink transmission rates between drone u and IoT device k, and represents the uplink transmission rate and downlink transmission rate between the drone and the base station; the weighted energy consumption of the process The following formula is used to obtain

[0059]

[0060] In formula (12), and The energy consumption of the cloud is the additive transmission energy consumption of the drone and IoT device when sending raw data and receiving results. is regarded as 0, and the rest are obtained by:

[0061]

[0062] In formula (13), and Represent the transmission power of IoT devices and drones respectively, is the transmission power of the UAV to the base station, ω k is the importance coefficient of IoT device k, ω u is the importance coefficient of UAV u.

[0063] As a preferred embodiment of the present invention, step S4 connects the IoT device to an appropriate drone to complete the networking and the drone decides the appropriate task unloading location to minimize the weighted sum of the delay and energy consumption of all tasks, which is expressed as problem P0:

[0064]

[0065] In formula (14), and They respectively indicate whether task m is processed in the local IoT device, drone, or cloud server; and They represent the total delay of local IoT devices, drones, and cloud servers respectively; and Respectively represent the weighted energy consumption of local IoT devices, drones, and cloud servers; ω t represents the delay coefficient;

[0066] In problem P0, the selection of connected drones by IoT devices and the decision of task offloading are coupled with each other, and problem P0 is decomposed into multiple drone selection sub-problems P1 and task offloading decision sub-problems P2;

[0067] and,

[0068] The subproblem P1 is to minimize the sum of the weighted transmission powers of IoT devices and drones by connecting IoT devices to appropriate drones in each time slot t, as follows:

[0069]

[0070] In formula (15), P u,k (t) represents the UAV transmission power, P k,u (t) represents the transmission power of the IoT device and remains unchanged within a given time slot t; the output result of sub-problem P1 is the set Display and confirm the connection status between IoT devices and drones;

[0071] The subproblem P2 minimizes the weighted sum of energy consumption and delay of all tasks performed by the same UAV u in time slot t, and its formula is:

[0072]

[0073] Results of the IoT device clustering phase Significantly affects the input of subproblem P2; the output of subproblem P2 is or Indicates whether task m is processed by a local IoT device, a drone, or a cloud server.

[0074] As a preferred embodiment of the present invention, the sub-problem P1 is solved by using an IoT device clustering algorithm based on simulated annealing; the solution process includes:

[0075] Step S411, establish an optimization function and align the optimization function with sub-problem P1, that is

[0076]

[0077] Step S412: input the drone set U, IoT device set K, and drone location L u (t) and IoT device location L k (t);

[0078] Step S413: Randomly initialize a random set Annealing temperature T0 and cooling coefficient α, get the initial optimization function

[0079] Step S414, generate a new Get a new optimization function And calculate

[0080] Step S415: if ΔF1<0, then accept the change; otherwise, according to the Metropolis rule, the probability of accepting the change is determined by the following formula:

[0081]

[0082] Step S416, updating the annealing temperature T1 so that the latest temperature T1 = α*T1; α represents the cooling rate;

[0083] Step S417, determining whether the annealing temperature T1 drops below a specified cooling threshold; if so, proceeding to step S418; if not, returning to step S415;

[0084] Step S418: Output the optimized result set Determine which IoT devices and their associated tasks the drone u will handle within time slot t.

[0085] As a preferred embodiment of the present invention, the sub-problem P2 is solved by using a task offloading decision algorithm based on a proximal policy optimization algorithm PPO; the solution process includes:

[0086] Step S421, input the available energy of the drone u and the IoT device Actor Network Learning Rate Critic network learning rate CLIP parameters ε, discount factor γ, and computational requirements G for all tasks, the total number of episodes is W, and the actor network is iterated N times and the critic network is iterated I times in each episode;

[0087] Step S422, initializing the actor network parameters θ and the commentator network parameters

[0088] Step S423, collect the trajectory (s q ,a q ,r q ,done q ,s q+1 ,…,a Q ,r Q ), using the current policy π(a|s; θ ω ), let π old =π(a|s;θ ω );

[0089] Step S424, in order to improve learning stability and reduce variance, calculate the advantage function for each state s(q) And is given by:

[0090]

[0091] In formula (21), r q Indicates that the trajectory is in state s q The reward obtained under γ q′-q represents the discount factor from state q to state q′; r q′ Indicates that the trajectory is in state s q′ The reward obtained under V π (s q ; φ) represents the estimated value of the state s by the neural network using the parameter φ q The value function of

[0092] Step S425, the actor network randomly selects an action from the activity set of the current state, interacts with the environment and obtains the next state and reward function;

[0093] Step S426, calculate the actor network sampling rate, and obtain it by the following formula:

[0094]

[0095] In formula (23), Indicates the state s in the nth training q The sampling rate ratio, π(a q |s q ,θ n ) indicates the use of parameter θ n The new actor network, π(a q |s q ,θ old ) indicates that the parameter θ is still used ω The original actor network;

[0096] Step S427, calculate the actor network loss function, whose objective function is

[0097]

[0098] Use the Adam optimizer to update the parameters related to θ;

[0099] loss Obtained by the following formula:

[0100]

[0101] Update the actor network parameters θ,π(a|s;θ ω )=π(a|s;θ n );

[0102] Step S428: The commentator network updates its parameters using the Adam optimizer To improve the accuracy of evaluating the actor network behavior; its loss function The calculation of depends on the long-term reward provided by the environment:

[0103]

[0104] The parameters of the critic network are updated by using the loss function

[0105] Step S429, determine whether the maximum number of iterations W has been reached; if not, return to step S423; if so, output the result task calculation offloading scenario

[0106] In a second aspect, an embodiment of the present invention further provides a drone-assisted computing offloading system, the system comprising: an Internet of Things device layer, an edge layer, a cloud layer, a drone connection module, and a scene selection module; wherein,

[0107] The IoT device layer includes several IoT devices with different battery capacities, and the IoT devices are immovable, represented by k = {1, 2, ... K}. In time slot t, IoT device k continuously generates computing tasks m, expressed by the following formula: m = {1, 2, ... M}, and reports computing requests to the drone; the size of task m is given by It is also used for computing offloading in local IoT device computing scenarios.

[0108] The edge layer includes a group of drones, represented by u={1, 2, ... U}, which provide communication relay and on-the-way computing for IoT devices; edge computing servers are deployed on the drones, and the edge computing servers are used to determine offloading scenarios and provide computing offloading in edge layer computing scenarios;

[0109] The cloud layer includes high-performance servers, supercomputing centers and / or artificial intelligence computing groups, which are used to receive IoT data forwarded from drones, perform computing offloading in cloud-edge-end computing scenarios, and transmit computing results back to drones through base stations;

[0110] The drone connection module is defined in Indicates whether drone u provides service to IoT device k in time slot t; if IoT device k is connected to drone u, then otherwise

[0111] The scenario selection module is used to combine the three established task computing offloading scenarios, with the goal of minimizing the weighted sum of delays and energy consumption of all tasks, the transmission power and task processing parameters of the IoT device, the transmission power and task processing parameters of the drone, the available energy of the drone and the IoT device, the size of the task to be processed and the tolerable delay, and the drone connection module to obtain the drone that is most suitable for serving the IoT device, and the drone calculates the optimal computing offloading scenario; then, based on the selected optimal computing offloading scenario, the computing offloading function in the corresponding networking device layer, edge layer or cloud layer is started.

[0112] In a third aspect, an embodiment of the present invention further provides a computing storage medium, wherein the medium stores the drone-assisted computing offloading system as described above.

[0113] The technical solution provided by the embodiment of the invention has the following beneficial effects:

[0114] The unmanned aerial vehicle-assisted computing offloading method, system and computer storage medium provided in the embodiment of the present invention enable the Internet of Things devices in remote areas to perform computing, networking, long-distance communication, etc., and can run delay-sensitive applications in remote areas. With the flexibility and easy deployment of unmanned aerial vehicles, it can become a mobile edge server, avoiding the long-distance transmission of data between the Internet of Things devices in remote areas and the cloud server, thereby reducing data transmission and computing delays, improving the task processing speed, and improving the stability of communication. At the same time, the present invention can maximize the working life of the Internet of Things devices. In remote areas, the Internet of Things devices can send data to unmanned aerial vehicles at suitable locations. For some data with high delay tolerance and large computing volume, it can be arranged to be processed on airborne servers or cloud servers, thereby reducing the transmission power of the Internet of Things devices, reducing energy consumption, and extending the service life. In addition, the present invention can more effectively process massive computing tasks. The unmanned aerial vehicle can reasonably select the computing location according to the requirements of the task and the size of the data. The local Internet of Things devices, airborne servers and cloud high-performance servers work together to significantly improve data processing and computing efficiency, reduce delays, and optimize resource utilization.

[0115] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0116] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0117] Figure 1 is a flow chart of a drone-assisted calculation method in an embodiment of the present invention;

[0118] Figure 2 Schematic diagram of cloud-edge-end collaborative computing architecture in a drone-assisted computing offloading method in an embodiment of the present invention;

[0119] Figure 3 is a flow chart of the simulated annealing algorithm used for sub-problem P1 in the embodiment of the present invention;

[0120] Figure 4 It is a flow chart of the proximal strategy optimization algorithm adopted for sub-problem P2 in the embodiment of the present invention. DETAILED DESCRIPTION

[0121] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations. It should be noted that the embodiments of the present invention and the features in the embodiments can also be combined with each other without conflict.

[0122] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In the description of the present invention, the terms "first", "second", "third", "fourth", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0123] The present invention aims at the extensive computing, networking and long-distance communication needs of IoT devices in remote areas, and solves the problem that emerging applications in remote areas such as digital twins and augmented reality have strict requirements on service delays and generate a huge amount of data to be processed. At the same time, in view of the fact that IoT devices in remote areas are subject to limited energy reserves and computing power, and it is difficult for them to independently complete massive computing tasks within a specified time, a drone-assisted computing offloading method, system and computer storage medium are proposed, and a three-layer, two-stage drone-assisted edge computing network task offloading architecture is designed. A corresponding networking model and task offloading model are proposed for the architecture, so as to improve data processing and computing efficiency and optimize resource utilization.

[0124] like Figure 1 As shown, the drone-assisted computing offloading method described in the embodiment of the present invention includes the following steps:

[0125] Step S1, establish a UAV-assisted computational offloading model, such as Figure 2 As shown in Figure 1, the model includes an IoT device layer, an edge layer, and a cloud layer. The IoT device layer includes several IoT devices with different battery capacities, and the IoT devices are immovable, represented by k = {1, 2, ... K}. In time slot t, IoT device k continuously generates computing tasks m, expressed by the following formula: m = {1, 2, ... M}, and reports computing requests to the drone. The size of task m is given by Represented; the edge layer includes a group of drones, represented by u={1, 2, ... U}, which provide communication relay and on-the-way computing for IoT devices; an edge computing server is deployed on the drone, and the edge computing server is used to determine the offloading scenario and provide computing offloading in the edge layer computing scenario; the cloud layer includes high-performance servers, supercomputing centers and / or artificial intelligence computing groups, which receive IoT data forwarded from drones, provide high-quality services for computing-intensive tasks, and transmit the computing results back to the drones through base stations.

[0126] Step S2, in the UAV-assisted computational offloading model, adopt Indicates whether drone u provides service to IoT device k in time slot t; if IoT device k is connected to drone u, then otherwise

[0127] For the connection between drones and IoT devices, there are the following restrictions: i) there are no remaining pending tasks at the end of time slot t, ii) each IoT device can only be connected to one drone in the same time slot, i.e.

[0128]

[0129] The three-dimensional coordinates of UAV u are expressed as L u (t) = {X u (t),Y u (t),H u}, where X u (t), Y u (t) and H u are the X, Y, and Z coordinates of the drone u; IoT devices are immovable, and the 3D coordinates are represented by L k (t) = {X k ,Y k ,H k}; where X k , Y k , H u and H k is a constant. To avoid excessive energy consumption in data transmission, the maximum distance between the IoT device and the drone should not exceed d max , given by

[0130] ‖L u (t)-L k (t)‖≤d max (46)

[0131] If the maximum distance between the IoT device and the drone exceeds d max,The drone will deny access to IoT devices.,The drone’s coverage areas partially overlap to improve service reliability.

[0132] Step S3, under the drone-assisted computing offloading model, three task computing offloading scenarios are established, including: local IoT device computing scenario, drone-assisted edge computing scenario, and cloud-edge-end computing scenario; and a single computing task is completely processed in one scenario.

[0133] In this step, due to the limited computing power of IoT devices, considering factors such as task size, latency tolerance, IoT device power, and drone power, the task is placed in three different locations for computing offload, namely local IoT device computing, drone-assisted edge computing, and cloud-edge-end collaborative computing.

[0134] For local IoT device computing scenarios:

[0135] set up Indicates whether task m is processed by the local IoT device k; if task m is processed by the local IoT device, then otherwise

[0136] The delay from task m generation to completion only considers the computational delay, which can be obtained as

[0137]

[0138] In formula (3), is the total latency of the local IoT device processing task m, is the time consumed by the local IoT device k to process task m. The CPU frequency of IoT device k is represented by f k Indicates that C k represents the CPU cycles required to process 1 bit of data. Therefore, the weighted energy consumption of the whole process is

[0139]

[0140] In formula (4), Ω k is the effective switched capacitance of the IoT device k, ω k is the importance coefficient of IoT device k. It is assumed that all local IoT devices share the same parameters in this architecture.

[0141] For drone-assisted edge computing scenarios:

[0142] set up Indicates whether task m is processed by the drone; if task m is processed by the drone, then otherwise

[0143] In this scenario, the computing task is processed by the edge computing server deployed on the drone. IoT device k generates task m and sends the raw data to the The data is transmitted to the corresponding UAV u, which then processes the data and sends the result data to the corresponding UAV u in time slot t. Return to the IoT device, therefore, the drone-assisted edge computing model The total delay is given by

[0144]

[0145] In formula (5), and They represent the data sending delay, result receiving delay and drone processing delay of task m in IoT device k. For convenience, the propagation delay is ignored. In addition, the processing delay It can be expressed as

[0146]

[0147] C u The CPU cycles required for the drone to process 1 bit of data, f u is the frequency of the drone CPU, the sending delay and receiving delay can be expressed as:

[0148]

[0149] In formula (7), and is the uplink and downlink transmission rates between u and k, given by

[0150]

[0151] In formula (8), B up and B down They are used to represent the bandwidth of the uplink and downlink, respectively. The propagation loss is represented by h m (t) represents the transmission power of IoT devices and drones. and The terminal Gaussian noise power of drones and IoT devices is represented by and The weighted energy consumption of the whole process is given as Energy transmitted by IoT devices Drones transmit energy and processing energy Composition, and obtained by

[0152]

[0153] In formula (9), Ω u is the effective switching capacitance of UAV u, ω u are important coefficients for UAV u. All UAVs also share the same parameters.

[0154] For cloud-edge-end computing scenarios:

[0155] set up Indicates whether task m is processed on the cloud server; if task m is processed on the cloud server, then otherwise

[0156] In this scenario, IoT device k creates a computing task m and sends raw data to drone u. The drone then relays the data to the cloud server through the base station. Once the cloud server completes the task processing, it sends the result back to the drone, which eventually returns the result to IoT device k within time slot t. The total delay of the whole process is It can be obtained by the following formula:

[0157]

[0158] In formula (10), and It is the total end-to-end sending delay and total receiving delay of data from IoT devices to cloud servers. The processing delay of cloud servers is considered as 0, and Given by the following formula

[0159]

[0160] In formula (11), ξ m is the propagation delay from the cloud to the base station, Indicates the size of the calculation result returned by the cloud server. The uplink and downlink rates between the drone and the base station are and The weighted energy consumption of this process is It can be obtained by the following formula

[0161]

[0162] In formula (12), and are the additive transmission energy consumption of the drone and IoT device when sending raw data and receiving results, respectively. is regarded as 0, and the rest are obtained by:

[0163]

[0164] In formula (13), is the transmission power of the drone to the base station.

[0165] Step S4, based on the three established task computing offloading scenarios, under the constraints, combined with the IoT device transmission power and task processing parameters, the UAV transmission power and task processing parameters, the available energy of the UAV and the IoT device, the size of the task to be processed and the tolerable delay, etc., obtain the most suitable UAV and the best computing offloading scenario for serving the IoT device, so as to minimize the weighted sum of the delay and energy consumption of all tasks.

[0166] In this step, the IoT devices are connected to the appropriate drones to complete the networking and the drones decide the appropriate task unloading location to minimize the weighted sum of the delay and energy consumption of all tasks, which is expressed as problem P0:

[0167]

[0168] In formula (14), and They respectively indicate whether task m is processed in the local IoT device, drone, or cloud server; and They represent the total delay of local IoT devices, drones, and cloud servers respectively; and Respectively represent the weighted energy consumption of local IoT devices, drones, and cloud servers; ω t represents the delay coefficient;

[0169] In problem P0, the IoT device’s selection of connected drones and task offloading decisions are coupled with each other, similar to a multi-agent competition problem.

[0170] In this step, in order to simplify the solution process, problem P0 is decomposed into multiple drone selection subproblems P1 and task offloading decision subproblems P2. Subproblems P1 and P2 can be regarded as corresponding optimization problems in the IoT device clustering stage and network operation stage, respectively.

[0171] The subproblem P1 is to minimize the sum of the weighted transmission powers of IoT devices and drones by connecting IoT devices to appropriate drones in each time slot t, as follows:

[0172]

[0173] In formula (15), P u,k (t) represents the UAV transmission power, P k,u (t) represents the transmission power of the IoT device and remains unchanged within a given time slot t. The output of subproblem P1 is the set Displays the connection status between the IoT device and the drone.

[0174] The subproblem P2 minimizes the weighted sum of energy consumption and delay of all tasks performed by the same UAV u in time slot t, and its formula is:

[0175]

[0176] Results of the IoT device clustering phase Significantly affects the input of subproblem P2. The output of subproblem P2 is or Indicates whether task m is processed by a local IoT device, a drone, or a cloud server.

[0177] In a specific embodiment, the sub-problem P1 is solved using an IoT device clustering algorithm based on simulated annealing. Figure 3 As shown, the solution process includes:

[0178] Step S411, establish an optimization function and align the optimization function with sub-problem P1, that is

[0179]

[0180] Step S412: input the drone set U, IoT device set K, and drone location L u (t) and IoT device location L k (t);

[0181] Step S413: Randomly initialize a random set Annealing temperature T0 and cooling coefficient α, get the initial optimization function

[0182] Step S414, generate a new Get a new optimization function And calculate

[0183] Step S415, if ΔF1<0, then accept the change. Otherwise, according to the Metropolis rule, the probability of accepting the change is determined by the following formula:

[0184]

[0185] Step S416, updating the annealing temperature T1 so that the latest temperature T1 = α*T1; α represents the cooling rate;

[0186] Step S417, determining whether the annealing temperature T1 drops below a specified cooling threshold; if so, proceeding to step S418; if not, returning to step S415;

[0187] Step S418: Output the optimized result set Determine which IoT devices and their associated tasks the drone u will handle within time slot t.

[0188] Furthermore, in a specific embodiment, for sub-problem P2, Figure 4 As shown in the figure, the task offloading decision algorithm based on the proximal policy optimization algorithm (PPO) is used to solve the problem, in which the actor network and the commentator network interact with the environment to find the optimal computing offloading scenario. In order to save storage resources, the algorithm does not set up a storage pool, but adopts a sampling and training method. The algorithm is based on the current policy π(a|s; θ ω ) interacts stably with the environment until it reaches a completion state, thus generating a complete trajectory. The environment will then take Q steps (s q ,a q ,r q ,done q ,s q+1 ,…,s Q ,r Q ) is fed back to the two networks. Regarding the tuple, the state space consists of the available energy of the drone and the IoT device, the size of the task to be processed in step q, and the tolerable delay, i.e. The action space is At each step q, the reward function is defined as the inverse of the weighted sum of latency and energy consumption:

[0189]

[0190] The proposed algorithm adopts CLIP-based importance sampling, where the CLIP function is defined as:

[0191]

[0192] The specific steps are as follows:

[0193] Step S421, input the available energy of the drone u and the IoT device Actor Network Learning Rate Critic network learning rate CLIP parameters ε, discount factor γ, and computational requirements G for all tasks, the total number of episodes is W, and the actor network is iterated N times and the critic network is iterated I times in each episode;

[0194] Step S422, initializing the actor network parameters θ and the commentator network parameters

[0195] Step S423, collect the trajectory (s q ,a q ,r q ,done q ,s q+1 ,…,s Q ,r Q ), using the current policy π(a|s; θ ω ), let π old =π(a|s;θ ω );

[0196] Step S424, in order to improve learning stability and reduce variance, calculate the advantage function for each state s(q) And is given by:

[0197]

[0198] In formula (21), r q Indicates that the trajectory is in state s q The reward obtained under γ q′-q represents the discount factor from state q to state q′; r q′ Indicates that the trajectory is in state s q′ The reward obtained under V π (s q ; φ) represents the estimated value of the state s by the neural network using the parameter φ q The value function of

[0199] Step S425, the actor network randomly selects an action from the activity set of the current state, interacts with the environment and obtains the next state and reward function;

[0200] Step S426, calculate the actor network sampling rate, and obtain it by the following formula

[0201]

[0202] In formula (23), Indicates the state s in the nth training q The sampling rate ratio, π(a q |s q ,θ n ) indicates the use of parameter θ n The new actor network, π(a q |s q ,θ old ) indicates that the parameter θ is still used ω The original actor network.

[0203] Step S427, calculate the actor network loss function, whose objective function is

[0204]

[0205] Use the Adam optimizer to update the parameters related to θ; the loss Obtained by the following formula

[0206]

[0207] Update the actor network parameters θ,π(a|s;θ ω )=π(a|s;θ n );

[0208] Step S428: The commentator network updates its parameters using the Adam optimizer To improve the accuracy of evaluating the actor network behavior; its loss function The calculation of depends on the long-term reward provided by the environment:

[0209]

[0210] The parameters of the critic network are updated by using the loss function

[0211] Step S429, determine whether the maximum number of iterations W has been reached; if not, return to step S423; if so, output the result task calculation offloading scenario

[0212] Step S5, completing the computation offloading at the optimal offloading scenario.

[0213] In this step, the calculation offloading is completed according to the calculation offloading methods in the three different scenarios defined in step S3.

[0214] Based on the same idea, an embodiment of the present invention also provides a drone-assisted edge computing offloading system, the system comprising: an IoT device layer, an edge layer, a cloud layer, a drone connection module and a scene selection module; wherein,

[0215] The IoT device layer includes several IoT devices with different battery capacities, and the IoT devices are immovable, represented by k = {1, 2, ... K}. In time slot t, IoT device k continuously generates computing tasks m, expressed by the following formula: m = {1, 2, ... M}, and reports computing requests to the drone; the size of task m is given by It is also used for computing offloading in local IoT device computing scenarios.

[0216] The edge layer includes a group of drones, represented by u={1, 2, ... U}, which provide communication relay and on-the-way computing for IoT devices; edge computing servers are deployed on the drones, and the edge computing servers are used to determine offloading scenarios and provide computing offloading in edge layer computing scenarios;

[0217] The cloud layer includes high-performance servers, supercomputing centers and / or artificial intelligence computing groups, which are used to receive IoT data forwarded from drones, perform computing offloading in cloud-edge-end computing scenarios, and transmit computing results back to drones through base stations;

[0218] The drone connection module is defined in Indicates whether drone u provides service to IoT device k in time slot t; if IoT device k is connected to drone u, then otherwise

[0219] The scenario selection module is used to combine the three established task computing offloading scenarios, with the goal of minimizing the weighted sum of delays and energy consumption of all tasks, the transmission power and task processing parameters of the IoT device, the transmission power and task processing parameters of the drone, the available energy of the drone and the IoT device, the size of the task to be processed and the tolerable delay, and the drone connection module to obtain the drone that is most suitable for serving the IoT device, and the drone calculates the optimal computing offloading scenario; then, based on the selected optimal computing offloading scenario, the computing offloading function in the corresponding networking device layer, edge layer or cloud layer is started.

[0220] In this embodiment, each module is implemented by a processor, and a memory is appropriately added when storage is required. Among them, the processor can be but is not limited to a microprocessor MPU, a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, etc. The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0221] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0222] It should also be noted that the drone-assisted computing offloading system described in this embodiment corresponds to the drone-assisted computing offloading method, and the description and limitation of the method are also applicable to the system and will not be repeated here.

[0223] An embodiment of the present invention also provides a computing storage medium, which stores a drone-assisted computing offloading system.

[0224] It can be seen from the above technical solutions that the unmanned aerial vehicle-assisted calculation offloading method, system and computer storage medium provided in the embodiment of the present invention enable remote area Internet of Things devices to perform calculations, networking, long-distance communications, etc., and can run delay-sensitive applications in remote areas. With the flexibility and easy deployment of unmanned aerial vehicles, it can become a mobile edge server, avoiding the long-distance transmission of data between remote area Internet of Things devices and cloud servers, thereby reducing data transmission and calculation delays, improving task processing speed, and improving communication stability. At the same time, the present invention can maximize the working life of Internet of Things devices. In remote areas, Internet of Things devices can send data to unmanned aerial vehicles at appropriate locations. For some data with high delay tolerance and large calculation amount, they can be arranged on airborne servers or cloud servers for processing, thereby reducing the transmission power of Internet of Things devices, reducing energy consumption, and extending service life. In addition, the present invention can more effectively process massive computing tasks. Unmanned aerial vehicles can reasonably select computing locations according to task requirements and data size. Local Internet of Things devices, airborne servers and cloud high-performance servers work together to significantly improve data processing and computing efficiency, reduce delays, and optimize resource utilization.

[0225] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. It is not intended to limit the scope of the invention claimed for protection, but only represents the preferred embodiment of the present invention. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the inventive concept. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

Claims

1. A drone-assisted computation offloading method, characterized in that: The method comprises the following steps: Step S1, establishing a drone-assisted computing offloading model, the model includes an IoT device layer, an edge layer, and a cloud layer; wherein, The IoT device layer includes several IoT devices with different battery capacities, and the IoT devices are immovable, represented by k = {1, 2, ... K}. In time slot t, IoT device k continuously generates computing tasks m, expressed by the following formula: m = {1, 2, ... M}, and reports computing requests to the drone; the size of task m is given by express; The edge layer includes a group of drones, represented by u={1, 2, ... U}, which provide communication relay and on-the-way computing for IoT devices; edge computing servers are deployed on the drones, and the edge computing servers are used to determine offloading scenarios and provide computing offloading in edge layer computing scenarios; The cloud layer includes high-performance servers, supercomputing centers and / or artificial intelligence computing groups, which receive IoT data forwarded from drones, provide high-quality services for computing-intensive tasks, and transmit computing results back to drones through base stations; Step S2, in the UAV-assisted computational offloading model, adopt Indicates whether drone u provides service to IoT device k in time slot t; if IoT device k is connected to drone u, then otherwise There are the following limitations on the connection between drones and IoT devices: i) There are no remaining tasks to be processed at the end of time slot t; ii) Each IoT device can only connect to one drone in the same time slot; The formula is: In formula (1), U is the total number of drones; Represents any value; Step S3, under the drone-assisted computing offloading model, three task computing offloading scenarios are established, including: local IoT device computing scenario, drone-assisted edge computing scenario, and cloud-edge-end computing scenario; and a single computing task is completely processed in one scenario; Step S4, based on the three established task computing offloading scenarios, with the goal of minimizing the weighted sum of delays and energy consumption of all tasks, the transmission power and task processing parameters of the IoT device, the transmission power and task processing parameters of the drone, the available energy of the drone and the IoT device, the size of the task to be processed and the tolerable delay are combined to obtain the drone that is most suitable for serving the IoT device, and the drone calculates the optimal computing offloading scenario; Step S5, completing the calculation offloading in the optimal offloading scenario.

2. The drone-assisted computing offloading method according to claim 1, characterized in that: In step S2, in order to ensure that each IoT device can only be connected to one drone in the same time slot, the three-dimensional coordinates of drone u are represented as L u (t) = {X u (t),Y u (t),H u }, X u (t), Y u (t) and H u are the X, Y, and Z coordinates of the drone u; IoT devices are immovable, and the three-dimensional coordinates are represented by L k (t) = {X k ,Y k ,H k }; where X k , Y k , H u and H k is a constant; and: ‖L u (t)-L k (t)‖≤d max (2) In formula (2), d max is the distance threshold between the IoT device and the drone; If the distance between the IoT device and the drone exceeds d max , drones will deny IoT devices access.

3. The drone-assisted computing offloading method according to claim 1, characterized in that: The calculation process in the local IoT device calculation scenario in step S3 is as follows: set up Indicates whether task m is processed by the local IoT device k; if task m is processed by the local IoT device, then otherwise The delay from the generation to the completion of task m only considers the computational delay, as shown in formula (3): In formula (3), is the total latency of local IoT device k processing task m, is the time consumed by the local IoT device k to process task m, f k represents the CPU frequency of the local IoT device k, C k Indicates the CPU cycles required to process 1 bit of data; Weighted energy consumption of the whole process for: In formula (4), Ω k is the effective switched capacitance of the IoT device k, ω k is the importance coefficient of IoT device k, and all local IoT devices share the same parameters in the model.

4. The drone-assisted computing offloading method according to claim 1, characterized in that: The calculation process in the drone-assisted edge computing scenario in step S3 is as follows: set up Indicates whether task m is processed by the drone; if task m is processed by the drone, then otherwise In this scenario, the computing task is processed by the edge computing server deployed on the drone; IoT device k generates task m and sends the raw data to the edge computing server. The data is transmitted to the corresponding UAV u, which then processes the data and sends the result data to the corresponding UAV u in time slot t. Return to the IoT device, therefore, the drone-assisted edge computing model The total delay is given by In formula (5), and They represent the data sending delay, result receiving delay and drone processing delay of task m in IoT device k respectively; Drone processing delay It is expressed as: In formula (6), C u The CPU cycles required for drone u to process 1 bit of data, f u The frequency of the drone CPU; The delay in sending data and receiving results is expressed as: In formula (7), and are the uplink and downlink transmission rates between drone u and IoT device k, and: In formula (8), B up and B down They are used to represent the bandwidth of uplink and downlink respectively; h m (t) represents the propagation loss, and They represent the transmission power of IoT devices and drones respectively; and denote the terminal Gaussian noise power of the UAV and IoT device respectively; Weighted energy consumption of the whole process Energy transmitted by IoT devices Drones transmit energy and processing energy Composed and obtained by: In formula (9), Ω u is the effective switching capacitance of UAV u, ω u is the importance coefficient of UAV u, and all UAVs share the same parameters; ω k is the importance coefficient of IoT device k.

5. The drone-assisted computing offloading method according to claim 1, characterized in that: The calculation process in the cloud-edge-device computing scenario in step S3 is as follows: set up Indicates whether task m is processed on the cloud server; if task m is processed on the cloud server, then otherwise In this scenario, IoT device k creates a computing task m and sends raw data to drone u; the drone then relays the data to the cloud server through the base station; once the cloud server completes the task processing, it sends the result back to the drone, which eventually returns the result to IoT device k within time slot t; the total delay of the whole process is It can be obtained by the following formula: In formula (10), and It is the end-to-end total transmission delay and total reception delay of data from IoT devices to cloud servers; the processing delay of cloud servers is considered as 0, and Given by the following formula In formula (11), ξ m is the propagation delay from the cloud to the base station, Indicates the size of the calculation result returned by the cloud server; and are the uplink and downlink transmission rates between drone u and IoT device k, and represents the uplink transmission rate and downlink transmission rate between the drone and the base station; the weighted energy consumption of the process The following formula is used to obtain In formula (12), and The energy consumption of the cloud is the additive transmission energy consumption of the drone and IoT device when sending raw data and receiving results. is regarded as 0, and the rest are obtained by: In formula (13), and Represent the transmission power of IoT devices and drones respectively, is the transmission power of the UAV to the base station, ω k is the importance coefficient of IoT device k, ω u is the importance coefficient of UAV u.

6. The drone-assisted computing offloading method according to claim 2, characterized in that: Step S4 completes the networking by connecting the IoT devices to the appropriate drones and letting the drones decide the appropriate task offloading location to minimize the weighted sum of the delay and energy consumption of all tasks, which is expressed as problem P0: In formula (14), and They respectively indicate whether task m is processed in the local IoT device, drone, or cloud server; and They represent the total delay of local IoT devices, drones, and cloud servers respectively; and Respectively represent the weighted energy consumption of local IoT devices, drones, and cloud servers; ω t represents the delay coefficient; In problem P0, the selection of connected drones by IoT devices and the decision of task offloading are coupled with each other, and problem P0 is decomposed into multiple drone selection sub-problems P1 and task offloading decision sub-problems P2; and, The subproblem P1 is to minimize the sum of the weighted transmission powers of IoT devices and drones by connecting IoT devices to appropriate drones in each time slot t, as follows: st(1),(2)(15) In formula (15), P u,k (t) represents the UAV transmission power, P k,u (t) represents the transmission power of the IoT device and remains unchanged within a given time slot t; the output result of sub-problem P1 is the set Display and confirm the connection status between IoT devices and drones; The subproblem P2 minimizes the weighted sum of energy consumption and delay of all tasks performed by the same UAV u in time slot t, and its formula is: Results of the IoT device clustering phase Significantly affects the input of subproblem P2; the output of subproblem P2 is or Indicates whether task m is processed by a local IoT device, a drone, or a cloud server.

7. The drone-assisted computing offloading method according to claim 6, characterized in that: The sub-problem P1 is solved by using an IoT device clustering algorithm based on simulated annealing; the solution process includes: Step S411, establish an optimization function and align the optimization function with sub-problem P1, that is Step S412: input the drone set U, IoT device set K, and drone location L u (t) and IoT device location L k (t); Step S413: Randomly initialize a random set Annealing temperature T0 and cooling coefficient α, get the initial optimization function Step S414, generate a new Get a new optimization function And calculate Step S415: if ΔF1<0, then accept the change; otherwise, according to the Metropolis rule, the probability of accepting the change is determined by the following formula: Step S416, updating the annealing temperature T1 so that the latest temperature T1 = α*T1; α represents the cooling rate; Step S417, determining whether the annealing temperature T1 drops below a specified cooling threshold; if so, proceeding to step S418; if not, returning to step S415; Step S418: Output the optimized result set Determine which IoT devices and their associated tasks the drone u will handle within time slot t.

8. The drone-assisted computing offloading method according to claim 6, characterized in that: The sub-problem P2 is solved by using a task offloading decision algorithm based on the proximal policy optimization algorithm PPO; the solution process include: Step S421, input the available energy ζ of the drone u and the IoT device u (q), Actor Network Learning Rate Critic network learning rate CLIP parameters ε, discount factor γ, and computational requirements G for all tasks, the total number of episodes is W, and the actor network is iterated N times and the critic network is iterated I times in each episode; Step S422, initializing the actor network parameters θ and the commentator network parameters Step S423, collect the trajectory (s q ,a q ,r q ,done q ,s q+1 ,…,s Q ,r Q ), using the current policy π(a|s; θ ω ), let π old =π(a|s;θ ω ); Step S424, in order to improve learning stability and reduce variance, calculate the advantage function for each state s(q) And is given by: In formula (21), r q Indicates that the trajectory is in state s q The reward obtained under γ q′-q represents the discount factor from state q to state q′; r q′ Indicates that the trajectory is in state s q′ The reward obtained under V π (s q ; φ) represents the estimated value of the state s by the neural network using the parameter φ q The value function of Step S425, the actor network randomly selects an action from the activity set of the current state, interacts with the environment and obtains the next state and reward function; Step S426, calculate the actor network sampling rate, and obtain it by the following formula: In formula (23), Indicates the state s in the nth training q The sampling rate ratio, π(a q |s q ,θ n ) indicates the use of parameter θ n The new actor network, π(a q |s q ,θ old ) indicates that the parameter θ is still used ω The original actor network; Step S427, calculate the actor network loss function, whose objective function is Use the Adam optimizer to update the parameters related to θ; loss Obtained by the following formula: Update the actor network parameters θ,π(a|s;θ ω )=π(a|s;θ n ); Step S428: The commentator network updates its parameters using the Adam optimizer To improve the accuracy of evaluating the actor network behavior; its loss function The calculation of depends on the long-term reward provided by the environment: The parameters of the critic network are updated by using the loss function Step S429, determine whether the maximum number of iterations W has been reached; if not, return to step S423; if so, output the result task calculation offloading scenario 9. A drone-assisted computing offloading system, characterized in that: The system includes: an IoT device layer, an edge layer, a cloud layer, a drone connection module, and a scene selection module; wherein, The IoT device layer includes several IoT devices with different battery capacities, and the IoT devices are immovable, represented by k = {1, 2, ... K}. In time slot t, IoT device k continuously generates computing tasks m, expressed by the following formula: m = {1, 2, ... M}, and reports computing requests to the drone; the size of task m is given by It is also used for computing offloading in local IoT device computing scenarios. The edge layer includes a group of drones, represented by u={1, 2, ... U}, which provide communication relay and on-the-way computing for IoT devices; edge computing servers are deployed on the drones, and the edge computing servers are used to determine offloading scenarios and provide computing offloading in edge layer computing scenarios; The cloud layer includes high-performance servers, supercomputing centers and / or artificial intelligence computing groups, which are used to receive IoT data forwarded from drones, perform computing offloading in cloud-edge-end computing scenarios, and transmit computing results back to drones through base stations; The drone connection module is defined in Indicates whether drone u provides service to IoT device k in time slot t; if IoT device k is connected to drone u, then otherwise The scenario selection module is used to combine the three established task computing offloading scenarios, with the goal of minimizing the weighted sum of delays and energy consumption of all tasks, the transmission power and task processing parameters of the IoT device, the transmission power and task processing parameters of the drone, the available energy of the drone and the IoT device, the size of the task to be processed and the tolerable delay, and the drone connection module to obtain the drone that is most suitable for serving the IoT device, and the drone calculates the optimal computing offloading scenario; then, based on the selected optimal computing offloading scenario, the computing offloading function in the corresponding networking device layer, edge layer or cloud layer is started.

10. A computing storage medium, characterized in that: The medium stores the drone-assisted computing offloading system as described in claim 9.

Images