Deep neural network task-oriented collaborative reasoning and unmanned aerial vehicle trajectory optimization algorithm

Abstract

The invention relates to the technical field of unmanned aerial vehicle assisted edge computing, and discloses a deep neural network task-oriented collaborative reasoning and unmanned aerial vehicle trajectory optimization algorithm, which comprises the following steps of S1, establishing a multi-unmanned aerial vehicle assisted mobile edge computing system model; s2, providing an optimal segmentation point selection algorithm, selecting an optimal segmentation point for each user task, and determining a task unloading proportion; s3, providing an unmanned aerial vehicle and ground user matching algorithm, and allocating computing resources of the unmanned aerial vehicle; and S4, modeling the unmanned aerial vehicle trajectory optimization and ground user transmitting power selection problem as a Markov decision process, providing a dynamic DNN task unloading and trajectory optimization algorithm based on deep reinforcement learning, and obtaining the optimal trajectory of the unmanned aerial vehicle and the transmitting power of the ground user in each time slot by training an intelligent agent. According to the algorithm, rewards can be continuously improved, and delay and energy consumption of DNN tasks in the MEC system are effectively reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV)-assisted edge computing, and specifically to a collaborative inference and UAV trajectory optimization algorithm for deep neural network tasks. Background Art

[0002] In an edge computing environment, terminal devices such as smartphones and Internet of Things (IoT) sensors usually have limited computing power, small storage capacity, and short battery life, yet they need to execute complex AI tasks such as image recognition and speech processing. The data generated by these devices is huge. If all of it is transmitted to the edge server or the cloud for processing, it will result in high network latency and communication costs. Deep neural network (DNN) has a complex structure and numerous parameters, demanding extremely high computing resources, and is vulnerable to network conditions and device performance during data transmission, leading to high transmission latency.

[0003] To solve this problem, existing technologies have proposed task offloading and collaborative inference strategies. Task offloading transfers some computing tasks from local devices to the edge server or the cloud to relieve the local burden; collaborative inference improves the inference efficiency through multi-device cooperation. However, the existing methods still have the following problems: The task segmentation is not fine: Traditional methods mostly adopt binary offloading (fully local or fully offloaded), unable to dynamically select the optimal segmentation point according to the DNN structure, resulting in low resource utilization. The resource allocation is unfair: The UAV resource allocation does not consider task priorities and real-time requirements, easily causing resource waste or task blocking. The trajectory optimization is insufficient: Existing UAV trajectory planning is mostly based on a static environment and cannot adapt to dynamic task distribution and network changes, making it difficult to globally optimize energy consumption and latency. Therefore, a collaborative inference and optimization method that can dynamically segment tasks, fairly allocate resources, and intelligently plan trajectories has become a worthy research content. Based on this, the present invention proposes a collaborative inference and UAV trajectory optimization algorithm for deep neural network tasks. Summary of the Invention

[0004] The purpose of the present invention is to provide a collaborative inference and UAV trajectory optimization algorithm for deep neural network tasks to solve the problems raised in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A collaborative inference and UAV trajectory optimization algorithm for deep neural network tasks, including the following steps:

[0006] Step S1: Establish a multi-UAV-assisted mobile edge computing system model, including a communication model, a time delay model, and an energy consumption model, and define the system optimization goal as minimizing the weighted sum of system energy consumption and task delay;

[0007] Step S2: Propose an optimal partitioning point selection algorithm (OPPS) based on the structural features of DNN tasks to select the optimal partitioning point for each user task and determine the task offloading ratio;

[0008] Step S3: Propose a UAV and ground user matching algorithm based on fairness factors to allocate the computing resources of UAVs;

[0009] Step S4: Model the UAV trajectory optimization and ground user transmit power selection problems as a Markov decision process (MDP), and propose a dynamic DNN task offloading and trajectory optimization algorithm (DPDTS) based on deep reinforcement learning (DRL). Train the agent to obtain the optimal trajectory of the UAV and the transmit power of the ground user in each time slot.

[0010] Preferably: The specific process of the DNN task optimal partitioning point selection algorithm OPPS in step S2 is as follows. After initializing the set of optimal partitioning points, for each UAV, add the user equipment within the signal coverage range of each UAV to the set. For each ground user in the set, select the layer that minimizes the weighted sum of task delay and energy consumption as the optimal partitioning point, and store the set of partitioning points of the users within the current UAV range in the global set.

[0011] Preferably: The matching algorithm between UAVs and ground users in step S3 includes the following steps: Initialize the user offloading decision and UAV computing resource allocation strategy in the current time slot. Obtain the global optimal partitioning point strategy according to the algorithm in step S2. For each UAV, obtain the set of optimal partitioning points of the tasks of the served users, and calculate the weighted difference between the energy consumption and time delay when the task is processed entirely locally and when it is offloaded. Where represents the delay of the part before the optimal partitioning point of each task when executed locally, represents the total time delay of task execution, represents the energy consumption of the part before the optimal partitioning point of each task when executed locally, Calculate the total energy consumed by the task of user i in the nth time slot, sort the elements in the set in descending order, set the user's offloading decision variable reasonably, allocate the computing resources of the UAV, and set a priority-based allocation weight, which represents the degree of urgent demand for resources by the task, that is:

[0012]

[0013] And according to the fairness weight factor ψ i (n), obtain the resource allocation ratio of UAV j:

[0014]

[0015] Preferably, the definitions of the state space, action space, and reward function of the UAV trajectory optimization algorithm DPDTS in step S4 are as follows:

[0016] State space : The state of the agent is represented as: which respectively represent the position coordinates of each UAV, the task arrival rate, and the task type of each user equipment in each time slot;

[0017] Action space : The action of the agent is defined as: which are respectively the vector velocity of each UAV and the transmission power of the user to offload tasks in each time slot;

[0018] Reward function : The reward of the agent is defined as: The first part is the opposite of the system optimization objective, and the second part is the penalty for the UAV approaching or exceeding the active range.

[0019] Preferably, the UAV trajectory optimization algorithm DPDTS in step S4 includes four steps: experience collection, offline replay, update of the policy network, and update of the target network. The specific process is as follows. First, by interacting with the task scenario, a certain number of experience samples (state space action space reward function state space ) are collected and stored in the experience replay buffer. Then, a small batch L b of experience samples is randomly sampled from it for updating the policy network and the target network. Based on the target network, is calculated under the state :

[0020]

[0021] where is a truncated random exploration noise that follows a Gaussian distribution. The Critic network uses temporal difference to update the network parameters:

[0022]

[0023]

[0024] where γ is the discount factor of the future Q value. The parameters θ1 and θ2 of the Critic main network are updated through the cost functions and For the update, the Actor main network adopts delayed update. Every d steps, the parameters φ of the Actor network are updated through deterministic policy gradient:

[0025]

[0026] Finally, the target network is updated in a soft update manner. By introducing the learning rate τ, the old target network parameters and new parameters are weighted averaged and then assigned to the three target networks:

[0027]

[0028]

[0029] This process will be repeated continuously until the maximum training steps are reached.

[0030] Compared with the prior art, the beneficial effects of the present invention are:

[0031] The present invention combines collaborative inference and task offloading, and proposes a trajectory optimization algorithm DPDTS based on user scheduling and task splitting point selection, aiming to achieve the rationality of resource allocation and the minimization of system energy consumption and total delay. DPDTS is a reinforcement learning decision-making method based on the optimal splitting point of DNN and fairness factor, which can continuously improve the reward and effectively reduce the delay and energy consumption of DNN tasks in the MEC system. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is the algorithm flowchart of the present invention;

[0033] Figure 2 is the system model diagram of DNN task offloading;

[0034] Figure 3 is the end-to-end delay of each optional splitting point of AlexNet under different resource allocation ratios;

[0035] Figure 4 is the optimal splitting point of three DNN tasks under different resource allocation ratios;

[0036] Figure 5 is the algorithm framework diagram of the trajectory optimization algorithm DPDTS of the present invention;

[0037] Figure 6 is the cumulative reward convergence diagram of DPDTS and other benchmark schemes;

[0038] Figure 7 is the relationship between the total energy consumption and the average time delay and the coverage radius of the UAV;

[0039] Figure 8Relationship between total energy consumption and average delay and the number of ground devices;

[0040] Figure 9 Relationship between total energy consumption and average delay and communication bandwidth. Specific implementation manners

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0042] Embodiment

[0043] Please refer to Figure 1 , a collaborative inference and UAV trajectory optimization algorithm for deep neural network tasks in the figure, including the following steps:

[0044] Step S1: Establish a multi-UAV-assisted mobile edge computing system model. There are K UAVs in the network, and each UAV is equipped with an MEC server. Multiple UAVs cooperate to provide computing offloading services for M user devices on the ground. As shown in the attached Figure 2 figure, it includes a communication model, a delay model, and an energy consumption model. The system optimization goal is defined as minimizing the weighted sum of system energy consumption and task delay;

[0045] Step S2: To reduce system delay and energy consumption, a DNN optimal partition point selection algorithm OPPS is designed. Figure 3 shows the end-to-end delay of each optional segmentation point of the AlexNet network under different resource allocation ratios b i,j (n). It can be seen that the end-to-end delays corresponding to different partitioning points show a consistent trend under five resource allocation ratios. Figure 4 shows the optimal segmentation points of three DNN tasks under different resource allocation ratios b i,j (n). It can be observed that when the allocated resource ratio is greater than 20%, the optimal segmentation points of AlexNet, VGG-16, and ResNet-34 remain unchanged;

[0046] In the OPPS algorithm, after inputting the task arrival rate w i , task type t i , the output data size D of each layer l,ti , and the number of CPU cycles C required for neuron calculation in each layer l,ti , first, the optimal segmentation point set G needs to be initialized. Then, for each UAV j, the user devices within its signal coverage range are added to the set H j, then select the best splitting point for each ground user in the set. To minimize the system overhead, it is necessary to select the layer that minimizes the weighted sum of task latency and energy consumption as the best splitting point, and set the splitting point set G of the users within the current UAV range j and store it in the global set G. Eventually, the best splitting points of all user tasks at the nth moment will be obtained;

[0047] Step S3: In each time slot, the ground users within the signal coverage range of the UAV can choose to offload their tasks to the UAV. However, due to limited computing resources, each UAV can serve at most Z ground users. Therefore, it is necessary to select matching users for each UAV and dynamically allocate resources to obtain a smaller system overhead. For this purpose, the present invention proposes a matching algorithm for UAVs and ground users. First, initialize the user offloading decision and the UAV computing resource allocation strategy Then, the optimal splitting point selection algorithm (OPPS) in Step S2 obtains the global task splitting strategy G. For each UAV, it is necessary to first obtain the set G of the best splitting points of the tasks of the served users j , and then calculate the weighted difference between the energy consumption and latency when the task is processed entirely locally and when it is offloaded, that is:

[0048]

[0049] where represents the latency of the part before the best splitting point of each task executed locally, represents the total task execution latency, represents the energy consumption of the part before the best splitting point of each task executed locally, Calculate the total energy consumed by the task of user i in the nth time slot,

[0050] and put the user number i and into the service queue Q that can be selected by the current UAV j . Then, according to values, sort the pairs in Q j in descending order. Each UAV serves at most Z ground users in each time slot. Select Z users from Q j to serve, and set the offloading decision variable α of the user i (n) = j. Note that when the service queues Q j of different UAVs have intersections, the ground users must be matched so that For the smaller UAV j, after determining the service queue of the current UAV, it is also necessary to reasonably allocate the computing resources of the UAV to improve the system utility. Therefore, based on the CPU frequency required when the task is completely processed locally, the algorithm sets a priority-based allocation weight, indicating the urgent demand degree of the task for resources, that is:

[0051]

[0052] And according to the fairness weight factor ψ i (n), the resource allocation ratio of UAV j is obtained:

[0053]

[0054] Through the iteration of the above process, the offloading decision of each ground user and the computing resource allocation strategy of each UAV can be finally obtained;

[0055] Step S4: After solving the problem of selecting the best segmentation point of the DNN task and the matching scheduling problem between the UAV and the ground user, the optimal offloading decision and the UAV server resource allocation ratio of each time slot can be obtained. In order to solve the best flight trajectory of the UAV and the user transmission power allocation, a dynamic DNN task offloading and trajectory optimization (DPDTS) algorithm is proposed, and MDP is used to construct a simplified sub-problem.

[0056] Figure 5 The algorithm framework of DPDTS is shown. This algorithm combines the OPPS algorithm and the matching scheduling algorithm of the above-mentioned steps S2 and S3, and obtains the optimal trajectory of the UAV and the transmission power of the ground user in each time slot by training the agent. It is divided into four steps: experience collection, offline replay, update policy network, and update target network. First, by interacting with the task scenario, a certain number of experience samples (state space action space reward function state space ) are collected and stored in the experience replay buffer, and then a small batch L b of experience samples is randomly sampled from it for updating the policy network and the target network. Based on the target network, under the state calculate :

[0057]

[0058] where is a truncated random exploration noise that follows a Gaussian distribution and can make the value function smoother. The Critic network uses temporal difference to update the network parameters:

[0059]

[0060]

[0061] Among them, γ is the discount factor of the future Q value, and the parameters θ1 and θ2 of the Critic main network are obtained through the cost function and are updated to minimize the error between the evaluation value and the target value. The Actor main network adopts delayed update. Every d steps, the parameters φ of the Actor network are updated through the deterministic policy gradient:

[0062]

[0063] Finally, the target network is updated in a soft update manner. By introducing the learning rate τ, the old target network parameters and the new parameters are weighted and averaged, and then assigned to the three target networks:

[0064] θ′ i' ←τθ i' +(1 - τ)θ′ i' , i′ = 1, 2

[0065] φ'←τφ+(1 - τ)

[0066] This process will be repeated continuously until the maximum number of training steps is reached. During this process, the agent will be able to give the best actions that can meet the requirements of the objective function in the given environment.

[0067] Next, a simulation experiment is carried out on the algorithm of the present invention. In the multi-UAV-assisted MEC scenario, 4 UAVs act as aerial base stations and fly within their activity range at a constant altitude of 50 m. The ground user equipment is distributed in an area of 400 m × 400 m. The maximum flight speed V of the UAV max = 2 m / s, and their initial positions are randomly generated at the activity boundary. There are three types of DNN inference tasks for users, namely ResNet-34, VGG-16, and AlexNet. The number of CPU cycles C l,ti required for each layer of DNN calculation and the output data size D l,ti are obtained by combining the inference delay and computing power of the local device.

[0068] The parameter settings are as follows: the number of ground users M = 50, the signal coverage radius d max of each UAV = 50 m, the communication bandwidth B = 10 MHz, as Figure 6As shown in the figure, as the number of training steps increases, the strategy of DPDTS gradually improves, and the performance fluctuation shows an obvious downward trend. Eventually, a relatively stable cumulative reward value can be obtained, which indicates that it has been trained as an agent that can make the best real-time decisions. Since DPDTS adopts a partial offloading strategy based on the structural characteristics of DNN tasks, determines the scheduling strategy through a fair weight factor, and can also adapt to the dynamic environment to make the best flight decisions, the cumulative reward of DPDTS is higher and the convergence speed is faster than other comparison algorithms, which reduces the system energy consumption and task latency to the greatest extent.

[0069] As shown in the appendix Figure 7 As shown in the figure, when the coverage radius of the UAV changes, the total system energy consumption and the average task processing delay of the ground users generated by DPDTS and the comparison scheme are statistically analyzed. The result data are the averages obtained by different schemes under 5 EP trainings. The larger the UAV signal coverage radius, the more ground users can offload high-complexity tasks to the UAV for execution. Therefore, the system energy consumption and the average task delay show a downward trend. Since DPDTS comprehensively considers the offloading strategy of DNN tasks, UAV resource scheduling, and trajectory optimization, the energy consumption under different coverage radius conditions is always the lowest. Therefore, it can continuously and efficiently perform collaborative inference and computational offloading for ground users. Especially when the radius is 50m, the total energy consumed by DPDTS during the service time is 1536.9J, which is reduced by 22.6%, 35.5%, 17.6%, and 54.2% compared with DTS, RAT, EE-PPO, and DPRT, respectively. In addition, since DPDTS adopts DNN segmentation technology to significantly reduce the task transmission delay, the average task processing delay is the lowest. Especially when the UAV signal coverage radius is 50m, the average delay of each user is 253.7ms, which is 24.8%, 36.8%, 19.26%, and 49.1% faster than other comparison schemes, respectively.

[0070] As Figure 8 As shown in the figure, a step-by-step analysis statistically analyzes the system cumulative energy consumption and the average task processing delay of the ground users obtained by DPDTS and other schemes when the number of ground user devices increases. Since the computing resources of the UAV are limited, the more ground users there are, the more data needs to be processed locally. Therefore, both the system energy consumption and the average task delay tend to increase. Since DPDTS can dynamically obtain the task segmentation point that minimizes the end-to-end overhead, and can flexibly adapt to the position distribution of ground devices, generate task situations, and make optimal flight actions, its performance is the most stable compared with other schemes, and the system energy consumption under different numbers of users is the lowest.

[0071] Figure 9It shows the total system power consumption and the average latency of tasks obtained by DPDTS and other comparison schemes under different communication bandwidths. The larger the communication bandwidth, the higher the data transmission rate, and the lower the transmission latency and transmission energy consumption required for task offloading. Therefore, both energy consumption and latency show a downward trend. It can be observed that DPDTS exhibits superior energy consumption and latency performance under various bandwidth conditions. Its main advantage lies in its ability to intelligently decide the flight path of the drone, task scheduling, and the transmission power allocation of user equipment, thereby achieving global optimization. Through precise policy learning and dynamic adjustment, DPDTS effectively improves the efficiency of task completion, significantly reduces the energy consumption and task latency of the system, and at the same time guarantees the quality of user services. Compared with DTS, RAT, EE-PPO, and DPRT, the average total system energy consumption is reduced by 35.1%, 49.2%, 23.9%, and 66.4%, and the average latency of user tasks is reduced by 27.5%, 31.7%, 18.2%, and 57.1%.

[0072] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0073] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A collaborative inference and UAV trajectory optimization algorithm for deep neural network tasks, characterized in that, It includes the following steps: Step S1: Establish a multi-UAV-assisted mobile edge computing system model, including a communication model, a delay model, and an energy consumption model. Define the system optimization objective as minimizing the weighted sum of system energy consumption and task delay; Step S2: Propose an optimal partitioning point selection algorithm (OPPS) based on the DNN task structure characteristics to select the optimal partitioning point for each user task and determine the task offloading ratio; Step S3: Propose a UAV and ground user matching algorithm based on a fairness factor to allocate the computing resources of the UAVs; Step S4: Model the UAV trajectory optimization and ground user transmit power selection problems as a Markov decision process (MDP), and propose a dynamic DNN task offloading and trajectory optimization algorithm (DPDTS) based on deep reinforcement learning (DRL). Obtain the optimal trajectory of the UAV and the transmit power of the ground user for each time slot by training the agent.

2. The collaborative inference and UAV trajectory optimization algorithm for deep neural network tasks according to claim 1, wherein: The specific process of the DNN task optimal partitioning point selection algorithm OPPS in step S2 is as follows. After initializing the set of optimal partitioning points, for each UAV, add the user equipment within the signal coverage range of each UAV to the set. For each ground user in the set, select the layer that minimizes the weighted sum of task delay and energy consumption as the optimal partitioning point, and store the set of partitioning points of the users within the current UAV range into the global set.

3. A collaborative inference and UAV trajectory optimization algorithm for deep neural network tasks according to claim 2, characterized in that: The UAV and ground user matching algorithm in step S3 includes the following steps: Initialize the user offloading decision and UAV computing resource allocation strategy for the current time slot. Obtain the global optimal splitting point strategy according to the algorithm in step S2. For each UAV, obtain the set of optimal splitting points for the tasks of the served users, and calculate the weighted difference in energy consumption and delay when the tasks are processed entirely locally and offloaded where represents the delay of the part before the optimal splitting point of each task executed locally, represents the total task execution delay, represents the energy consumption of the part before the optimal splitting point of each task executed locally, Calculate the total energy consumed by the task of user i in the nth time slot, sort the elements in the set in descending order, set the offloading decision variable of the user reasonably, allocate the computing resources of the UAV, and set a priority-based allocation weight to represent the urgent demand degree of the task for resources, that is: and according to the fairness weight factor ψ i (n) to obtain the resource allocation ratio of UAV j:

4. A collaborative inference and UAV trajectory optimization algorithm for deep neural network tasks according to claim 3, characterized in that: The definitions of the state space, action space, and reward function of the UAV trajectory optimization algorithm DPDTS in step S4 are: State space The state of the agent is represented as: which respectively represent the position coordinates of each UAV, the task arrival rate and task type of each user equipment in each time slot; Action space The actions of the agent are defined as: They are the vector velocity of each UAV and the transmit power of the user to offload tasks in each time slot respectively; Reward function The reward of the agent is defined as: The first part is the opposite of the system optimization goal, and the second part is the penalty for the drone approaching or exceeding the activity range.

5. The collaborative inference and UAV trajectory optimization algorithm for deep neural network tasks according to claim 4, characterized in that: The drone trajectory optimization algorithm DPDTS in step S4 includes four steps: experience collection, offline playback, updating the policy network, and updating the target network. The specific process is as follows. First, by interacting with the task scenario, a certain number of experience samples (state space action space reward function state space ) are collected and stored in the experience replay buffer. Then, a small batch L b of experience samples is randomly sampled from it for updating the policy network and the target network. Based on the target network, under the state calculate Among them, is a truncated random exploration noise that follows a Gaussian distribution. The Critic network uses temporal difference to update the network parameters: where γ is the discount factor of the future Q value, and the parameters θ1 and θ2 of the Critic main network are updated through the cost functions and The Actor main network adopts delayed updates. Every d steps, the parameters φ of the Actor network are updated through the deterministic policy gradient: Finally, update the target network in a soft update manner. By introducing a learning rate τ, perform a weighted average of the old target network parameters and the new parameters, and then allocate them to the three target networks: This process will be repeated continuously until the maximum number of training steps is reached.

Images