Unmanned aerial vehicle auxiliary task unloading method based on block flow prediction

Abstract

The invention belongs to the technical field of task offloading, and discloses an unmanned aerial vehicle auxiliary task offloading method based on block flow prediction. In order to simultaneously solve the problems of unmanned aerial vehicle deployment and task unloading caused by frequent change of user distribution, a block traffic prediction module, a task unloading module and an unmanned aerial vehicle trajectory optimization module are designed. Inputting user distribution time sequence information of the current time slot to obtain predicted future user distribution information; an optimal unloading decision is made according to relevant parameters of the current task, the user and the unmanned aerial vehicle to minimize delay and energy consumption of the user; a multi-agent reinforcement learning algorithm is used, an environment is built on the basis of a block flow prediction module and a task unloading module, and then environment parameter data is collected for training so as to obtain an unmanned aerial vehicle deployment scheme to optimize the flight path of the unmanned aerial vehicle. The method provided by the invention not only solves the problem of task unloading decision making in a scene with frequent change of user distribution, but also solves the problems of unmanned and deployment, and is superior to other existing methods.

Description

Technical Field

[0001] The present invention relates to the technical field of task offloading, and in particular to a method for unmanned aerial vehicle-assisted task offloading based on block traffic prediction. Background Art

[0002] With the rapid growth of mobile computing demand, traditional cloud computing is unable to cope with application scenarios that require real-time response due to its network latency and bandwidth limitations. At the same time, although edge computing reduces latency by offloading computing tasks to servers closer to users, it has high deployment and maintenance costs and limited coverage. The development of drone technology provides a new solution to this challenge. They can be deployed quickly, move flexibly, and provide services in areas without infrastructure. They can serve as mobile edge computing nodes in the air to provide computing resources for ground users. The development of 5G and 6G technologies provides technical support for efficient communication between drones and ground users. In addition, the widespread deployment of IoT devices generates a large amount of data that needs to be processed and analyzed. Drones can serve as mobile data processing centers to reduce the burden on ground networks. In scenarios such as environmental monitoring and disaster response, drones can quickly arrive at the scene, collect data and perform preliminary processing so that the command center can make decisions quickly. However, existing drone-assisted task offloading work has neglected in-depth exploration of scenarios where user distribution frequently changes, such as city centers. In such scenarios, the location and number of users will change over time, which will increase the difficulty of real-time deployment of drones and the difficulty of making task offloading decisions. In addition, due to the limited battery capacity of drones, how to complete more computing tasks under limited energy is a challenge. Summary of the invention

[0003] In view of the above problems, the present invention specifically proposes a UAV-assisted task offloading method based on block traffic prediction. The main idea is to predict the future user distribution to deploy the UAV flight trajectory to improve the service quality.

[0004] The technical solution of the present invention is as follows: a UAV-assisted task offloading method based on block traffic prediction, mainly composed of three parts: a block traffic prediction module, a task offloading module and a UAV trajectory optimization module;

[0005] The block traffic prediction module divides the target area into multiple sub-areas of equal size, and counts the traffic data of each sub-area respectively, and obtains the future traffic information of each sub-area through the time series prediction algorithm; the traffic information of the sub-area is used by the drone trajectory optimization module as the state of the reinforcement learning environment;

[0006] The task offloading module defines the weighted sum of the average delay and average energy consumption of the task as the utility of the system according to the dynamic characteristics of the user distribution of the scene, and uses this as an evaluation indicator. The entire offloading problem is modeled as an optimization problem, and a task offloading decision algorithm is designed to solve the optimization problem.

[0007] The UAV trajectory optimization module uses a multi-agent reinforcement learning algorithm to build an environment based on the block traffic prediction module and the task offloading module, collects environmental parameter data for training to obtain the UAV deployment plan, thereby optimizing the UAV's flight trajectory.

[0008] The block traffic prediction module is as follows:

[0009] The target area is divided into multiple sub-areas of equal size, and the number of users in each sub-area is counted respectively. The current time is combined as the user flow data of each block and expressed in the form of time series data. Then, the time series prediction algorithm is continuously trained according to the content of the historical data set. The results predicted by the time series prediction algorithm are used to input into the UAV trajectory optimization module.

[0010] The time series prediction algorithm specifically embeds the historical user traffic data of each sub-region, performs multi-head attention, normalizes, and calculates the loss after forward propagation of the feedforward network, and then backpropagates to update the network parameters until the training is completed.

[0011] The task offloading module is specifically as follows:

[0012] According to the system model and the real-world dataset, the entire UAV auxiliary task offloading is modeled as an optimization problem with the goal of minimizing the weighted sum of system delay and energy consumption; the optimization problem is as follows:

[0013]

[0014] in is the utility of the system, which represents the weighted sum of the average delay and average energy consumption of the task; the variables in constraint C1 represents the offloading decision and is also the decision variable of the optimization problem. The constraint indicates that the task can only be executed on one device; constraint C2 indicates that the execution time of the task does not exceed the deadline of the task; constraint C3 indicates that the number of users covered by the drone must be less than or equal to its coverage threshold. UAV stands for UAV j The number of users covered, is the upper limit of the drone’s coverage. Similarly, constraint C4 indicates that the number of users covered by the server must be less than or equal to its upper limit of coverage;

[0015] The unloading decision algorithm is used to solve the current optimal task unloading decision variables according to the system state in a certain time slot to maximize the current system utility. After obtaining the UAV trajectory, the same optimization problem is used to solve the system utility as an evaluation index.

[0016] The steps of the offloading decision algorithm are as follows: first, the offloading decision variables of all tasks are initialized to 0, and all tasks are executed locally by default at the beginning, and then all users are traversed to determine in turn whether the user is covered by a server that has not reached the service limit; when the user is covered by a server that has not reached the service limit, the user's task is offloaded to the corresponding server, and the corresponding offloading decision variable is set to 1, otherwise, it is further determined whether the user is covered by a drone that has not reached the service limit; when the user is covered by a drone that has not reached the service limit, the user's task is offloaded to the drone, and the corresponding offloading decision variable is set to 1, otherwise the task is executed locally; the drone will not execute the offloaded task immediately, but will first determine whether the server has reached the service limit. If not, the task will continue to be offloaded to the server for execution, and the corresponding offloading decision variable will be set to 1. When the service limit is reached, the task is directly executed on the drone.

[0017] The UAV trajectory optimization module is as follows:

[0018] The UAV trajectory optimization module is solved based on a multi-agent reinforcement learning algorithm, and a reinforcement learning environment is constructed by combining the contents of the block traffic prediction module and the task offloading module, and data is collected for training the reinforcement learning algorithm model.

[0019] The future user traffic data obtained in the block traffic prediction module is used to perform an average operation on the predicted time series data in blocks in the drone trajectory optimization module, and the average value is used as the state of the reinforcement learning environment;

[0020] The content of the task offloading module solves the offloading decision during the operation of the reinforcement learning environment and uses the corresponding system utility as an indicator to verify the learning effect of the algorithm;

[0021] According to the reward sharing mechanism, the agent’s reward R j It is defined as the sum of global coverage, that is, Where N t is the total number of users in time slot t; the target deployment position of the drone in the next time slot is encoded as a one-hot vector, which is used as the action of the drone to continuously provide services to users while the drone flies to the next target.

[0022] Beneficial effects of the present invention: In summary, the present invention provides a comprehensive and efficient method to solve the problem of drone-assisted task offloading in scenarios where user distribution changes frequently. The present invention proposes a block traffic prediction module to predict future user traffic distribution, and cooperates with the reinforcement learning algorithm in the drone trajectory optimization module to obtain the optimal deployment strategy for drones. In addition, a task offloading module is proposed to define the weighted sum of the average delay and average energy consumption of the task as the system utility, which provides an accurate evaluation standard for the task offloading problem in scenarios where user distribution changes frequently, and is used to verify the effect of the drone deployment strategy. The method proposed by the present invention significantly improves the performance of the overall network and the quality of service to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is the overall framework diagram of the UAV-assisted task offloading method based on block traffic prediction.

[0024] Figure 2 It is a schematic diagram of the system model.

[0025] Figure 3 It is a schematic diagram comparing the training performance of CTPUTO (the method of the present invention) and the baseline algorithm.

[0026] Figure 4 It is a schematic diagram of the comparison of evaluation performance of CTPUTO (the method of the present invention) and the baseline algorithm.

[0027] Figure 5 It is the system utility diagram under different numbers of drones.

[0028] Figure 6 It is a schematic diagram comparing the training performances of CTPUTO (the method of the present invention) and MAT.

[0029] Figure 7 It is a schematic diagram comparing the evaluation performances of CTPUTO (the method of the present invention) and MAT. DETAILED DESCRIPTION

[0030] The specific implementation modes of the present invention are described in detail below.

[0031] Taking into account that current research work has neglected the in-depth discussion of UAV-assisted task offloading in scenarios where user distribution frequently changes, such as city centers, and aiming at the problems of difficulty in deploying UAVs and making task offloading decisions in such scenarios, a UAV-assisted task offloading method based on block traffic prediction was designed with reinforcement learning and time series prediction as the theoretical basis.

[0032] The present invention is mainly implemented based on three modules. First, the block traffic prediction module is used to obtain the predicted future user distribution information by inputting the user distribution timing information of the current time slot; secondly, the task offloading module is used to make the best offloading decision according to the parameters of the current task, user and drone to minimize the user's delay and energy consumption; finally, the drone trajectory optimization module uses a multi-agent reinforcement learning algorithm and uses the environmental parameter data collected by the previous two modules for training in order to obtain the drone deployment plan to optimize the drone's flight trajectory. In general, the method proposed in the present invention not only solves the problem of task offloading decision-making in scenarios where user distribution changes frequently, but also optimizes the drone's flight trajectory.

[0033] The present invention proposes a UAV-assisted task offloading method based on block traffic prediction. The overall system framework is as follows: Figure 1 The overall system framework includes a block traffic prediction module, a task offloading module, and a drone trajectory optimization module. The system simulates user information based on the data set to obtain task offloading decisions and drone deployment in real time.

[0034] First, the present invention analyzes the existing problem, namely, the problem of task offloading and drone deployment in scenarios where user distribution changes frequently, which has not been solved in existing engineering applications. On the one hand, the existing methods ignore the challenges brought by this scenario, and only consider the problem in some simple scenarios with user mobility. The number of users in such scenarios generally does not change much, and most of them use methods related to user trajectory prediction to solve the problem. However, in scenarios where user distribution changes frequently, the number and location of users will change a lot over time. If methods related to user trajectory prediction are used, the computational complexity will be greatly increased, thereby achieving poor results. On the other hand, due to the limitations of communication resources and drone battery capacity, it becomes very difficult to ensure service quality and reduce drone energy consumption under limited resources, which requires a more efficient resource allocation method. Therefore, the current mainstream drone-assisted task offloading methods cannot meet the challenges brought by scenarios where user distribution changes frequently. Under the premise of meeting the problems of task offloading and drone deployment in such actual engineering scenarios, the present invention improves service quality and reduces drone energy consumption as much as possible to achieve more efficient and reliable drone-assisted task offloading.

[0035] (1) Block traffic prediction module based on time series prediction

[0036] Because the current method based on trajectory prediction is not suitable for scenarios where user distribution changes frequently, the present invention proposes a method based on block traffic prediction to deal with this scenario. The key part is the block traffic prediction module, which first divides the target area into multiple small areas of equal size, and counts the number of users in each small area and combines the current time slot information as the user traffic data of each block, and expresses it in the form of time series data; then calls the time series prediction algorithm model trained with historical data to obtain the predicted multiple time series data as user distribution information for a period of time in the future; finally, these time series data are averaged according to each block to obtain future block traffic information, and cooperate with the task unloading module and the drone trajectory optimization module to perform drone deployment operations. The advantage of this is that there is no need to perform trajectory prediction for each user, which can greatly reduce the computational complexity, and can obtain user distribution information for a period of time in the future, just like knowing in advance where the user will appear in the next time, which can provide important data for drone deployment more efficiently.

[0037] (2) Task offloading module

[0038] Since the tasks generated in the scenario where the user distribution changes frequently will change with the location and number of users, and the parameters of the task itself also change dynamically, how to make offloading decisions to always ensure the best service quality in the ever-changing environment has become a difficult problem. The present invention designs a task offloading module to solve this problem. In this module, Figure 2 The system model shown models the weighted sum minimization problem of system average delay and energy consumption, and proposes a task offloading decision algorithm to solve it.

[0039] First, the present invention constructs a Figure 2 The UAV-assisted task offloading system model shown is a three-layer edge computing network, which includes three types of devices, namely, user terminal devices on the ground, edge servers at the edge of the region, and UAVs acting as mobile edge servers in the air. In the scenario constructed by the present invention, the running time is divided into T time slots (1, 2, ..., t, ..., T). In time slot t, the number of terminal devices is set to N according to the content of the data set. t , each device has its own computing resource size And a task will be generated where i∈{1,2,…,N t}, is the size of the computing resources required to perform the task, is the data size required for the task to be transmitted in the wireless channel, is the deadline for mission completion. The number of drones is U t, each drone has its own computing resources It is expressed as the number of CPU cycles that can be executed per second, where j∈{1,2,…,U t There is only one edge server in the scenario, and its computing resources are C e , which can satisfy the parallel execution of multiple tasks. The terminal device MU in the system i 、UAV j and edge servers have their own location coordinates, respectively and (x e ,y e ,0) to indicate.

[0040] Secondly, the offloading decision variables are set to model the task offloading optimization problem, which mainly involves solving variables such as computing delay and energy consumption, transmission delay and energy consumption, and computing delay and energy consumption of the server. It can be executed locally on the terminal device or offloaded to the drone or edge server covering it. The drone can choose to execute the received task on the drone itself or continue to offload it to the server. The present invention sets the relevant offloading decision variables for the specific selection. They represent the offloading decision variables of tasks from terminal devices to edge servers, from terminal devices to drones, and from drones to edge servers. 0 means no offloading and 1 means offloading. They are initialized to 0 when each task is generated, because tasks can only be executed on one device. And only after the task is offloaded to the drone can it be offloaded to the server. According to the above analysis, specific task offloading can be divided into the following four situations:

[0041] 1) Task local execution

[0042] Task If executed locally, And only the computational delay and computational energy consumption of the local execution of the task need to be considered, among which the computational delay of the local execution of the task The energy consumption is calculated by dividing the computing resources required by the task by the computing resources of the terminal device. The energy required per unit of computation is multiplied by the computational resource size of the task, so they can be defined by the following formula:

[0043]

[0044] in, It is the terminal device MU iThe size of the computing resources is expressed in terms of the number of CPU cycles that can be run per unit time, e d It is the amount of energy consumed by executing a unit CPU cycle on the terminal device.

[0045] 2) The task is offloaded from the terminal device to the drone for execution

[0046] Task From the terminal device MU i Offloading to UAV j If executed on In this process, the transmission delay of the task from the terminal device to the drone needs to be considered. Energy consumption and the computational latency required to execute on the drone Energy consumption First, calculate the wireless transmission rate by the following formula

[0047]

[0048] Where W is the channel bandwidth between the terminal device and the drone, P d is the transmitting power of the terminal equipment, N0 is the background noise power, is the channel gain between the terminal device and the drone, where μ0 is the fading component, k is the channel path loss index, is the distance between the terminal device and the drone. The transmission delay and energy consumption can then be calculated using the following formula:

[0049]

[0050] The transmission delay is calculated by dividing the data size required for task transmission by the transmission rate obtained above, and the transmission energy consumption is calculated by multiplying the transmission power of the terminal device by the transmission delay. Finally, after receiving the task, the drone will execute it, and the calculation delay and energy consumption are defined by the following formula:

[0051]

[0052] in, It is a UAV j The computing resource size, e u It is the amount of energy consumed by executing a unit CPU cycle on the drone.

[0053] 3) Tasks are offloaded from terminal devices to edge servers for execution

[0054] Similar to the previous case, the task From the terminal device MU i If it is offloaded to the edge server for execution, According to the previous formula (3), the transmission rate can be calculated The transmission delay is then defined by the following formula: Transmission energy consumption Computational delays on the server and calculate energy consumption

[0055]

[0056] Among them, C e is the computing resource size of the server, e e It is the amount of energy consumed by executing a unit CPU cycle on the server.

[0057] 4) Tasks are offloaded from drones to edge servers for execution

[0058] When the task After being offloaded from the terminal device to the drone, it is further offloaded to the edge server for execution. In addition to considering the transmission delay from the terminal device to the drone, Energy consumption In addition, the transmission delay from the drone to the server must also be considered. Transmission energy consumption and the latency of computations performed on the server and calculate energy consumption They are defined by the following formulas:

[0059]

[0060] in, is the transmission rate between the drone and the edge server, P u is the transmission power of the drone. Taking the above four situations into consideration, the total delay T for task completion is defined by the following formula: tot The total energy consumption E of the terminal equipment tot :

[0061]

[0062] In addition, since the number of users in the target scene is constantly changing, in order to provide an effective evaluation index, the present invention defines the utility of the system by the following formula:

[0063]

[0064] Among them, c1 and c2 are weighted factors of delay and energy consumption, respectively, which are hyperparameters. The system utility is the negative value of the weighted sum of the average delay and average energy consumption of system task offloading.

[0065] In summary, the present invention defines the following optimization problem:

[0066]

[0067] Where i∈{1,2,…,N t}, j∈{1,2,…,U t}, constraint C1 means that the task can only be executed on one device, constraint C2 means that the execution time of the task does not exceed the deadline of the task, and constraint C3 means that the number of users covered by the drone must be less than or equal to its coverage threshold. UAV stands for UAV j The number of users covered, is the upper limit of the drone’s coverage. Similarly, constraint C4 indicates that the number of users covered by the server must be less than or equal to its upper limit. Finally, after the problem modeling is completed, the task offloading decision algorithm is used to solve the value of the offloading decision variable, and then the optimal utility of the system is obtained.

[0068] (3) UAV trajectory optimization module based on multi-agent reinforcement learning

[0069] In order to cope with the challenge of difficult drone deployment in scenarios where user distribution frequently changes, the present invention proposes a drone trajectory optimization module, which is solved based on a multi-agent reinforcement learning algorithm. It combines the contents of the previous block traffic prediction module and task offloading module to construct a reinforcement learning environment, and then collects data for training the reinforcement learning algorithm model.

[0070] In the UAV-assisted task offloading system, the offloading utility of the system depends only on the current state of the system. Therefore, the UAV-assisted task offloading problem can be modeled as a Markov decision process suitable for multi-agent reinforcement learning, which is expressed as <U,S,[A j ] j∈U ,[Obs j ] j∈U ,[R j ] j∈U ,σ>, where U is the set of agents, S is the current state of the system, and A j It is an agent j The action space, Obs j It is an agent j The observed value, R j It is an agent j The reward is , σ∈[0,1] represents the discount factor. The following will introduce the definition of the above concepts in the present invention in detail:

[0071] 1) Agent Set U: Each UAV acts as an agent, learns deployment strategies to optimize the system offloading cost, and takes optimal actions according to the algorithm strategy.

[0072] 2) System state S: The present invention divides the target area into G blocks of equal size, and takes the average number of users in each block as the current state of the system, which is expressed as follows:

[0073] s(δ)={N1(δ),N2(δ),…,N G (δ)} (20)

[0074] 3) Action Space A j : The present invention encodes the target deployment position of the UAV in the next time slot into a one-hot vector, which is used as the action space of the UAV.

[0075] 4) Observation Obs j :Each UAV concatenates its own location code, its own ID, service threshold and system status to obtain its own local observation value. When the prediction module is introduced, the state is replaced by the state of the next time slot, then the UAV j Observation value Obs j It is expressed as:

[0076]

[0077] 5) Reward R j : In the drone-assisted unloading scenario proposed in this invention, the utility of the system is directly related to the deployment of drones. In other words, in the entire scenario, the more users the drones and servers serve, the smaller the total consumption of the system. Due to the reward sharing mechanism, the present invention divides the agent's reward R j It is defined as the sum of global coverage, and its expression is:

[0078]

[0079] After that, the environment is built and the UAV trajectory optimization algorithm is trained. The training process includes initializing system parameters and environment, interacting with the environment to obtain training sample data, and randomly extracting data for strategy update. Finally, simulation is performed to evaluate the performance of the algorithm, where the evaluation indicator of the simulation is the utility of the system.

[0080] The hardware and software configuration environment of the present invention is: the operating system is Ubuntu 22.04, the deep learning framework is PyTorch, the CPU is i5-10700F, the memory is 16G, and the graphics card is NVIDIA GeForce GTX 1070.

[0081] Step 1: Implement the contents of each module.

[0082] The present invention proposes three modules in total, namely, a block traffic prediction module, a task offloading module and a UAV trajectory optimization module. The following will explain how to implement these modules in detail.

[0083] The first step is to implement the block traffic prediction module, which is based on the time series prediction algorithm. The present invention uses the Shanghai Telecom base station data set, which is a data set generated from the real world, for simulation. For this purpose, the data set is preprocessed to make it a new data set containing timestamps, user IDs and user locations. Through screening and counting operations, a time series changing data set of the number of users contained in each block area is obtained, which is used to train the block traffic prediction algorithm.

[0084] In the selection of the time series prediction algorithm, the present invention selected a transformer-based time series prediction algorithm, which only selected the encoder for training, encoded the time series and the data vectors of each block, and then obtained the model through the forward propagation and back propagation of the network layers such as the multi-layer attention mechanism, normalization, and feedforward network. In order to simulate the real situation, the present invention divides the data set into two parts according to time. The front part is used as historical data to train the model, and the back part is used for the training and verification of the subsequent drone trajectory optimization algorithm.

[0085] The second step is to implement the task offloading module. Since this module is not independent like the block traffic prediction module, it is part of the subsequent drone trajectory optimization algorithm environment, so the code implementation of this module needs to be placed in the subsequent multi-agent reinforcement learning environment to implement the specific delay and energy consumption calculation process, and the following algorithm will be called in the algorithm verification phase to solve the offloading decision variable value and the corresponding system utility size.

[0086]

[0087]

[0088] The third step is to implement the drone trajectory optimization module. After the environment is built, it can be trained with the multi-agent reinforcement learning algorithm. Based on the multi-agent advantage decomposition theorem, the present invention models the multi-agent reinforcement learning training problem into a serialized decision problem, and introduces the transformer architecture to solve it. Figure 1 The right side of the diagram shows the architecture of the algorithm, which includes two parts: the encoder and the decoder, and the decisions of the agent are made serially one by one, which can greatly reduce the time complexity.

[0089] In addition, the previous two modules serve this module. The model trained in the block traffic prediction module can be directly called in this module to calculate the system state of the environment. The task offloading module exists to model and implement the reinforcement learning environment. In the testing phase, the multi-agent reinforcement learning algorithm model trained in the drone trajectory optimization module can be called to solve the deployment plan, and then the offloading decision algorithm can be used to solve the offloading decision variable value and system utility to test the algorithm performance.

[0090] Step 2: Train the algorithm model.

[0091] After implementing all the contents proposed by the present invention, you can start training the algorithm model. From the previous content, it can be known that the present invention needs to train two models, namely the time series prediction algorithm model in the block traffic prediction module and the multi-agent reinforcement learning algorithm model in the drone trajectory optimization module. Considering that the time series prediction algorithm is needed in the multi-intelligence reinforcement learning algorithm, but if the two models are trained collaboratively, it will increase the difficulty of training. For this reason, the present invention adopts a decentralized training method to keep the service interface between the algorithms consistent, and then train them separately. The training of the time series prediction algorithm model is relatively conventional. A part of the data set is used as historical data for training, and normal forward propagation and back propagation are performed to calculate the loss function to train it. The multi-agent reinforcement learning algorithm of the present invention is not the same as the traditional multi-agent reinforcement learning algorithm. It is a centralized training algorithm. The specific traffic is shown in the following algorithm:

[0092]

[0093]

[0094] Step 3: Perform simulation to verify algorithm performance.

[0095] First, the system parameters used in the present invention are introduced. The drone-assisted task offloading network considered in the present invention consists of a single edge server, multiple unmanned aerial vehicles, and a large number of terminal devices, and is simulated based on the Shanghai Telecom base station data set. According to the data of the above data set, the location coordinates of all devices are expressed in latitude and longitude values. Without further explanation, the following parameters are used to simulate the system. The location of the edge server is (31.24, 121.46), and the number and location of the terminal devices change dynamically according to the real data of the data set. Each terminal device has the same computing power (1GHz) and transmission power (0.1W). In addition, the length of each time slot is one second. The number of drones in the scene defaults to 4. The computing power of each drone is 10GHz, the transmission power is 0.2W, the coverage range is limited to 50, and the maximum length of the task queue is 20. Finally, the background noise and channel path loss index of the environment are -100dBm / Hz and -4, respectively.

[0096] Secondly, the present invention selects MAPPO, MAPPO algorithm with Beta distribution, ABMAPPO and MATD3 as baseline algorithms for comparative experiments, and names the algorithm proposed by the present invention as CTPUTO, and the algorithm without adding a prediction module is recorded as MAT. Figure 3 and Figure 4 The performance comparison between CTPUTO and the baseline algorithm under the same experimental conditions is shown. Figure 3 The different learning capabilities of each algorithm during the training phase are described. From the reward trajectory, it can be seen that compared with the baseline algorithm, CTPUTO exhibits a faster convergence speed and a more stable convergence state, which is beneficial for solving complex scenarios. On the other hand, the present invention evaluates the actual performance of the algorithm, such as Figure 4 As shown in Figure 2, the CTPUTO algorithm achieves the highest system utility and a stable system utility interval, which is the same as the performance in the training phase. This proves the high efficiency of the CTPUTO algorithm.

[0097] Finally, the present invention conducted an ablation experiment. Figure 5 As shown, in the UAV-assisted task offloading scenario considered in the present invention, as the number of UAVs increases, the system utility gradually increases, proving that UAVs play a significant auxiliary role in task offloading. Figure 6 and Figure 7 The training and evaluation comparison results without and with the prediction module are shown, as Figure 6 As shown in , CTPUTO with the prediction module added converges faster during the training phase, and the average value of the reward converges is also larger. Figure 7 As shown in Figure 2, the average system cost of the CTPUTO algorithm using the block traffic prediction module is also lower during the evaluation phase. In summary, it can be concluded that the block traffic prediction module is very efficient and the CTPUTO using the block traffic prediction module can better cope with the challenges brought by the scenario where user distribution changes frequently.

Claims

1. A UAV-assisted task offloading method based on block traffic prediction, characterized in that: It mainly consists of three parts: block traffic prediction module, task offloading module and drone trajectory optimization module; The block traffic prediction module divides the target area into multiple sub-areas of equal size, and counts the traffic data of each sub-area respectively, and obtains the future traffic information of each sub-area through the time series prediction algorithm; the traffic information of the sub-area is used by the drone trajectory optimization module as the state of the reinforcement learning environment; The task offloading module defines the weighted sum of the average delay and average energy consumption of the task as the utility of the system according to the dynamic characteristics of the user distribution of the scene, and uses this as an evaluation indicator. The entire offloading problem is modeled as an optimization problem, and a task offloading decision algorithm is designed to solve the optimization problem. The UAV trajectory optimization module uses a multi-agent reinforcement learning algorithm to build an environment based on the block traffic prediction module and the task offloading module, collects environmental parameter data for training to obtain the UAV deployment plan, thereby optimizing the UAV's flight trajectory.

2. The method for unloading unmanned aerial vehicle-assisted tasks based on block traffic prediction according to claim 1 is characterized in that: The block traffic prediction module is as follows: The target area is divided into multiple sub-areas of equal size, and the number of users in each sub-area is counted respectively. The current time is combined as the user flow data of each block and expressed in the form of time series data. Then, the time series prediction algorithm is continuously trained according to the content of the historical data set. The results predicted by the time series prediction algorithm are used to input into the UAV trajectory optimization module.

3. The method for unloading unmanned aerial vehicle-assisted tasks based on block traffic prediction according to claim 2 is characterized in that: The time series prediction algorithm specifically embeds the historical user traffic data of each sub-region, performs multi-head attention, normalizes, and calculates the loss after forward propagation of the feedforward network, and then backpropagates to update the network parameters until the training is completed.

4. The method for unloading unmanned aerial vehicle-assisted tasks based on block traffic prediction according to claim 1 is characterized in that: The task offloading module is specifically as follows: According to the system model and the real-world dataset, the entire UAV auxiliary task offloading is modeled as an optimization problem with the goal of minimizing the weighted sum of system delay and energy consumption; the optimization problem is as follows: in is the utility of the system, which represents the weighted sum of the average delay and average energy consumption of the task; the variables in constraint C1 represents the offloading decision and is also the decision variable of the optimization problem. The content of the constraint indicates that the task can only be executed on one device; constraint C2 indicates that the execution time of the task does not exceed the deadline of the task; Constraint C3 means that the number of users covered by the drone must be less than or equal to its coverage threshold, where UAV j The number of users covered, is the upper limit of the drone’s coverage. Similarly, constraint C4 indicates that the number of users covered by the server must be less than or equal to its upper limit of coverage; The unloading decision algorithm is used to solve the current optimal task unloading decision variables according to the system state in a certain time slot to maximize the current system utility. After obtaining the UAV trajectory, the same optimization problem is used to solve the system utility as an evaluation index.

5. The method for unloading unmanned aerial vehicle-assisted tasks based on block traffic prediction according to claim 4 is characterized in that: The steps of the offloading decision algorithm are as follows: first, the offloading decision variables of all tasks are initialized to 0, and all tasks are executed locally by default at the beginning, and then all users are traversed to determine in turn whether the user is covered by a server that has not reached the service limit; when the user is covered by a server that has not reached the service limit, the user's task is offloaded to the corresponding server, and the corresponding offloading decision variable is set to 1, otherwise, it is further determined whether the user is covered by a drone that has not reached the service limit; when the user is covered by a drone that has not reached the service limit, the user's task is offloaded to the drone, and the corresponding offloading decision variable is set to 1, otherwise the task is executed locally; the drone will not execute the offloaded task immediately, but will first determine whether the server has reached the service limit. If not, the task will continue to be offloaded to the server for execution, and the corresponding offloading decision variable will be set to 1. When the service limit is reached, the task is directly executed on the drone.

6. The method for unloading unmanned aerial vehicle-assisted tasks based on block traffic prediction according to claim 1 is characterized in that: The UAV trajectory optimization module is as follows: The UAV trajectory optimization module is solved based on a multi-agent reinforcement learning algorithm, and a reinforcement learning environment is constructed by combining the contents of the block traffic prediction module and the task offloading module, and data is collected for training the reinforcement learning algorithm model.

7. The method for unloading unmanned aerial vehicle-assisted tasks based on block traffic prediction according to claim 6 is characterized in that: The future user traffic data obtained in the block traffic prediction module is used to perform an average operation on the predicted time series data in blocks in the drone trajectory optimization module, and the average operation is used as the system state of the reinforcement learning environment; The content of the task offloading module solves the offloading decision during the operation of the reinforcement learning environment and uses the corresponding system utility as an indicator to verify the learning effect of the algorithm; According to the reward sharing mechanism, the agent’s reward R j It is defined as the sum of global coverage, that is, Where N t is the total number of users in time slot t; the target deployment position of the drone in the next time slot is encoded as a one-hot vector, which is used as the action of the drone to continuously provide services to users while the drone flies to the next target.

Images