A method for unmanned aerial vehicle (UAV) swarm-assisted vehicle task offloading in intelligent transportation environments

Abstract

This invention belongs to the field of intelligent transportation technology and discloses a vehicle task offloading method assisted by drone swarms in intelligent transportation environments. This method targets a traffic light intersection scenario consisting of multiple intelligent connected vehicles and drone swarms. The drone with the most battery power in the swarm is designated as the cluster leader drone, and the remaining drones are designated as edge service drones. A DQN algorithm is deployed on the cluster leader drone. The DQN algorithm design can monitor the battery level of the edge service drones in real time and dynamically adjust the reward function based on battery consumption, fully utilizing the computing resources of the edge service drones to complete more tasks with limited battery power. This invention can also determine the task type based on the amount of computation generated by the vehicles and apply rewards and penalties accordingly, enabling the agent to adopt an offloading strategy corresponding to the task type, fully utilizing computing resources, reducing communication resource waste, and improving task completion rate.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent transportation and unmanned aerial vehicle cluster assisted vehicle networking edge computing task offloading, and particularly relates to a vehicle task offloading method assisted by an unmanned aerial vehicle cluster in an intelligent transportation environment. BACKGROUND

[0002] With the iterative development of vehicle networking technology, automobile manufacturers equip automobiles with more and more sensors. One kind is a perception type sensor for monitoring external environment information to support an assisted driving function, such as a vehicle-mounted camera, and the like, which generates an assisted driving type task. Another kind is an interactive type sensor for meeting the demand of an in-vehicle entertainment function, such as an entertainment display screen, and the like, which generates an in-vehicle entertainment type task. The assisted driving type task is characterized by strict latency requirement, small data volume, and large task computation. The in-vehicle entertainment type task is characterized by loose latency requirement, large data volume, and small task computation. Only the computing capability of the automobile itself cannot meet the processing requirements of the two types of tasks at the same time, and transmitting the two types of tasks to the cloud for processing will occupy a large amount of communication resources and generate high communication latency. Therefore, this type of problem has become a key obstacle to restricting the development of vehicle networking technology in a higher performance and more intelligent direction. Unmanned aerial vehicle cluster assisted edge computing as a novel computing method can effectively solve the above problems because unmanned aerial vehicles are deployed closer to the data source.

[0003] In an intelligent transportation environment, for example, a congested red light intersection, the driving environment is relatively complex, and the perception type sensor of the automobile generates a large amount of data that needs to be processed. Unmanned aerial vehicle cluster assisted edge computing has a unique advantage. At this time, the strong maneuvering capability of the unmanned aerial vehicle can be used to quickly deploy at the congested red light intersection. The automobile offloads the tasks that it cannot process within the specified time to the unmanned aerial vehicle, and lets the unmanned aerial vehicle assist the automobile in processing. This method shortens the communication distance and improves the data processing and response speed. However, if the automobile offloads all tasks to the unmanned aerial vehicle for processing, the computing resources of the automobile will be wasted, and the computing pressure of the unmanned aerial vehicle and the communication pressure in the region will be increased. Therefore, for unmanned aerial vehicle cluster assisted edge computing, different types of tasks generated by the automobile are determined to offload which tasks and how many tasks will achieve better results.

[0004] Currently in the unmanned aerial vehicle assisted intelligent transportation scene, it is common to deploy a deep reinforcement learning algorithm on the unmanned aerial vehicle to make different unloading strategies, vehicle transmission power allocation and unmanned aerial vehicle computing resource allocation schemes by interacting with the environment. Vehicle transmission power allocation and unmanned aerial vehicle computing resource allocation are similar, the more power or computing resource allocation, the faster the transmission or processing speed, but the energy consumption will also increase. Traditional deep reinforcement learning algorithms generally take the minimum energy consumption or the lowest latency as the optimization target, rarely consider how the unmanned aerial vehicle processes more tasks under the condition of limited power, resulting in that the unmanned aerial vehicle as an edge node cannot be fully utilized. At the same time, for different types of tasks generated by the vehicle, the existing literature does not consider encouraging the vehicle to adopt an unloading strategy suitable for its type, resulting in increased communication resource consumption and low task completion rate. SUMMARY

[0005] The purpose of the present application is to propose a vehicle task offloading method assisted by unmanned aerial vehicle cluster in an intelligent transportation environment. The method is for an intelligent transportation scene composed of multiple intelligent connected vehicles, multiple unmanned aerial vehicle formations and traffic lights at a traffic light intersection. An improved task offloading method based on the DQN algorithm is proposed. In the design of the DQN algorithm, the model can be dynamically adjusted from optimizing overall latency performance to optimizing overall energy consumption performance according to the power consumption of the edge service unmanned aerial vehicle, fully utilizing the computing resources of the edge service unmanned aerial vehicle to complete more tasks under the condition of limited power. In addition, the task type can be determined according to the task computation amount generated by the vehicle. Different unloading strategies are encouraged according to the task type through reward and punishment, which can fully utilize the computing resources, reduce the waste of communication resources and improve the task completion rate.

[0006] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0007] A vehicle task offloading method assisted by unmanned aerial vehicle cluster in an intelligent transportation environment, comprising the following steps:

[0008] Step 1. Build a scene of a traffic light intersection composed of multiple intelligent connected vehicles, multiple unmanned aerial vehicle formations, traffic lights and pedestrians; the unmanned aerial vehicle formation is composed of a cluster head unmanned aerial vehicle and a plurality of edge service unmanned aerial vehicles;

[0009] Select the unmanned aerial vehicle with the most power in the unmanned aerial vehicle formation as the cluster head unmanned aerial vehicle, and the remaining unmanned aerial vehicles as edge service unmanned aerial vehicles. The cluster head unmanned aerial vehicle is deployed with a DQN algorithm, and the edge service unmanned aerial vehicles are used to receive and process the tasks offloaded by the vehicles;

[0010] Step 2. The cluster head unmanned aerial vehicle is used to collect information from the vehicles and the edge service unmanned aerial vehicles, establish an environment state according to the collected information and input the environment state into the DQN algorithm, and make corresponding unloading decisions and resource allocation decisions;

[0011] In the DQN algorithm design process, the weights of the delay and the energy consumption in the reward function are dynamically adjusted according to the real-time monitored power size of the edge unmanned aerial vehicle, so that the delay is paid more attention to when the power is high, and the energy consumption is paid more attention to when the power is low.

[0012] Meanwhile, the task type, i.e., an auxiliary driving type or an in-vehicle entertainment type task, is determined according to the calculation amount of the task generated by the automobile, and different unloading decisions are encouraged by increasing / decreasing the size of the reward function according to the different task types.

[0013] The unloading decision includes automobile local processing, full unloading or partial unloading to the edge unmanned aerial vehicle for processing.

[0014] Step 3. The cluster head unmanned aerial vehicle feeds back the corresponding unloading decision, the edge service unmanned aerial vehicle computing resource allocation decision and the automobile transmission power allocation to the automobile and the edge service unmanned aerial vehicle through the DQN algorithm.

[0015] In addition, on the basis of the vehicle task unloading method assisted by the unmanned aerial vehicle cluster in the intelligent transportation environment, the application further proposes a corresponding unmanned aerial vehicle cluster assisted vehicle task unloading system in the intelligent transportation environment, which adopts the following scheme:

[0016] A vehicle task unloading system assisted by an unmanned aerial vehicle cluster in an intelligent transportation environment includes a plurality of intelligent networked vehicles and a plurality of unmanned aerial vehicle formations, and the unmanned aerial vehicle formation is composed of a cluster head unmanned aerial vehicle and a plurality of edge service unmanned aerial vehicles. The cluster head unmanned aerial vehicle includes a memory and one or more processors. The executable code is stored in the memory. When the processor executes the executable code, the steps of the vehicle task unloading method assisted by the unmanned aerial vehicle cluster in the intelligent transportation environment are implemented.

[0017] In addition, on the basis of the vehicle task unloading method assisted by the unmanned aerial vehicle cluster in the intelligent transportation environment, the application further proposes a computer readable storage medium having a program stored thereon; the program is executed by the processor to implement the steps of the vehicle task unloading method assisted by the unmanned aerial vehicle cluster in the intelligent transportation environment.

[0018] The application has the following advantages:

[0019] As described above, this invention discloses a vehicle task offloading method assisted by a drone swarm in an intelligent transportation environment. This method targets an intelligent transportation scenario consisting of multiple intelligent connected vehicles, a swarm of drones, and traffic lights at a traffic light intersection. The method selects the drone with the highest battery level in the swarm as the cluster leader drone, and the remaining drones as edge service drones. The cluster leader drone does not participate in the specific task computation stage. A DQN algorithm is deployed on the cluster leader drone, which collects information from the vehicles and edge service drones and makes corresponding offloading and resource allocation decisions. The edge service drones receive and process the tasks unloaded by the vehicles. During the DQN-based edge computing task offloading process, this invention can monitor the battery level of the edge service drones in real time and dynamically adjust the weights of latency and energy consumption in the reward function based on battery consumption. This allows for a focus on latency when battery is high and on energy consumption when battery is low, dynamically shifting the model's focus from optimizing latency performance to optimizing energy consumption performance. This fully utilizes the computing resources of the edge service drones, enabling them to complete more tasks with limited battery power. Furthermore, this invention can determine the task type (i.e., driver assistance or in-vehicle entertainment tasks) based on the computational load generated by the vehicle, and provide rewards or penalties by increasing or decreasing the reward function value according to different task types. This allows the agent to adopt an unloading strategy corresponding to the task type, making full use of computing resources, reducing communication resource waste, and improving task completion rate. The method of this invention has a simple design principle and a concise and clear process design, making it particularly suitable for use in intelligent transportation environments such as traffic light intersections where drone swarms assist in unloading tasks from vehicles. Attached Figure Description

[0020] Figure 1 This is a flowchart of a vehicle task offloading method assisted by a drone swarm in an intelligent transportation environment, as described in this embodiment of the invention.

[0021] Figure 2 This is a scenario diagram of a vehicle task offloading method assisted by a drone swarm in an intelligent transportation environment, as described in an embodiment of the present invention.

[0022] Figure 3 This is a flowchart of an improved task unloading method based on the DQN algorithm in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram illustrating the average number of tasks completed in a specific embodiment of the present invention. Detailed Implementation

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0025] Example 1

[0026] This embodiment describes a vehicle task offloading method assisted by a drone swarm in an intelligent transportation environment. This method relies on a drone swarm to assist the vehicle in making task offloading and resource allocation decisions, addressing the problem that in congested traffic light intersections, vehicles relying solely on their own computing power cannot simultaneously meet the processing requirements of assisted driving and entertainment tasks. Furthermore, during the vehicle task offloading process, this invention fully considers how drones can handle more tasks with limited battery power, making full use of the computing resources of edge service drones to complete more tasks under limited battery capacity. In addition, depending on the type of task generated by the vehicle, this invention encourages the agent to adopt an offloading strategy appropriate to the task type through a reward and penalty system, which can fully utilize computing resources, reduce communication resource waste, and improve task completion rate.

[0027] like Figure 1 As shown, the vehicle task offloading method assisted by UAV swarms in an intelligent transportation environment includes the following steps:

[0028] Step 1. Construct a scene of a traffic light intersection consisting of multiple intelligent connected vehicles, a formation of multiple drones, and traffic lights. The drone formation consists of a cluster leader drone and several edge service drones.

[0029] The drone with the most battery power in the drone formation is selected as the cluster leader drone, and the remaining drones are edge service drones. An improved task offloading method based on the DQN algorithm is deployed on the cluster leader drone.

[0030] The cluster-head drone is used to collect information from vehicles and edge service drones, and makes corresponding offloading and resource allocation decisions based on the DQN algorithm. The cluster-head drone does not participate in the specific task calculation process.

[0031] Edge service drones are used to receive and process tasks related to unloading vehicles.

[0032] like Figure 2 The image depicts a traffic light intersection scene consisting of one intelligent connected car, a formation of J drones, traffic lights, and pedestrians. (Car assembly) for:

[0033] (1)

[0034] The coordinates of car i in a three-dimensional Cartesian coordinate system for:

[0035] (2)

[0036] in and Represents the horizontal coordinate of car i. Let V represent the height of car i. The car coordinate set V is:

[0037] (3)

[0038] The set of task information generated by car i for:

[0039] (4)

[0040] in This represents the computational load generated by the vehicle (i), measured in bits per cycle.

[0041] This parameter is related to the task type: in-vehicle entertainment tasks mainly involve video encoding, compression, and transmission. The main tasks in driver assistance systems involve image processing, and the corresponding... .

[0042] This represents the amount of tasks generated by car i, in bits. This represents the maximum completion latency of the task generated by car i, in seconds. Therefore, the computational cost required for each task generated by car i is... for:

[0043] (5)

[0044] drone collection for:

[0045] (6)

[0046] The coordinates of UAV j in a three-dimensional Cartesian coordinate system for:

[0047] (7)

[0048] in and This represents the horizontal coordinate of drone j. This represents the altitude of drone j. The set of drone coordinates. for:

[0049] (8)

[0050] The drone swarm consists of a leader drone and several edge service drones, with the drone having the most battery power serving as the leader drone and the rest serving as edge service drones.

[0051] The cluster-head drone's role is to collect information from vehicles and edge service drones, make corresponding offloading and resource allocation decisions, and provide feedback to the vehicles and edge service drones. The cluster-head drone does not participate in specific task calculations; its maximum battery power... It has a capacity of 100Wh. The role of the edge service drone is to receive and process tasks unloaded from vehicles.

[0052] Information sent from edge service drone j to cluster head drone for:

[0053] (9)

[0054] in, This indicates the remaining battery power of edge service drone j, and the maximum battery power of edge service drone j. It is 100Wh. The computing resources allocated to edge service drone j for a certain car i are as follows: .

[0055] Then calculate the resource allocation matrix. for:

[0056] (10)

[0057] Information sent by car i to cluster head drone for:

[0058] (11)

[0059] in, This represents the computing resources of vehicle i itself. The unloading decision made by the cluster-headed drone on vehicle i. for:

[0060] (12)

[0061] in, This indicates that the vehicle is processed locally. This indicates that the entire process will be offloaded to edge service drones for handling. This indicates partial unloading processing. All vehicle unloading decision matrix. Represented as:

[0062] (13)

[0063] The communication model of this invention is modeled as follows:

[0064] The edge service drones fly at a fixed altitude of 30 meters, with a 40-meter interval between two edge service drones. The effective communication range of the edge service drones is 50 meters, and the antenna sector angle is 120 degrees.

[0065] Edge service drones and cars can communicate with cluster-head drones.

[0066] Since the car and edge service drones only report information to the cluster head drone, and the cluster head drone only provides feedback on decisions to the car, the entire process does not involve task transmission, so its transmission latency is negligible.

[0067] Euclidean distance between cars and edge service drones for:

[0068] (14)

[0069] For effective communication to occur between cars and drones, the following needs to be met. . This represents the transmit power sent by vehicle i to edge service drone j, and the transmit power allocation set for vehicle i. for:

[0070] (15)

[0071] This represents the maximum transmit power of the vehicle. The speed at which vehicle i transmits the task to edge service drone j. for:

[0072] (16)

[0073] Where B is the channel bandwidth. Here, k is the reference channel gain, T is the Boltzmann constant, and F is the channel fading factor. This is the path loss index.

[0074] The latency and energy consumption model of the cluster-head UAV is as follows:

[0075] The cluster-headed drone calculates the energy consumption of tasks generated by the vehicle. for:

[0076] (17)

[0077] in It is the energy consumption coefficient of the drone chip. The number of cycles required to execute DQN, The computing power of the cluster-head drone; the energy consumption of the cluster-head drone in handling all tasks generated by the vehicle. for:

[0078] (18)

[0079] Computational latency of cluster-headed drones processing tasks generated by vehicles. for:

[0080] (19)

[0081] Latency of cluster-headed drones processing all automotive-generated tasks for:

[0082] (20)

[0083] Hovering power of cluster-headed drones for:

[0084] (twenty one)

[0085] in It is an aerodynamic constant. For the quality of drones, It is air density. It is the rotational area of ​​the drone's rotor; the hovering energy consumption of the cluster-headed drone during the time it takes to calculate the task generated by car i. for:

[0086] (twenty two)

[0087] The latency and energy consumption model for the task's local computation in the vehicle is modeled as follows:

[0088] If the task is processed only locally, the computing power of vehicle i is The vehicle will dedicate all its computing power to this task, reducing computational latency. for:

[0089] (twenty three)

[0090] Computational energy consumption of vehicle i in processing its own tasks for:

[0091] (twenty four)

[0092] in It is the energy consumption coefficient of automotive chips.

[0093] The latency and energy consumption model for offloading all tasks to the edge service drone is as follows:

[0094] The tasks generated by car i Full offloading of transmission latency to edge service drone j for:

[0095] (25)

[0096] in This indicates the speed at which car i transmits tasks to edge service drone j.

[0097] Energy consumption of car i transmitting tasks to edge service drone j for:

[0098] (26)

[0099] definition The edge service drone j allocates computing resources to a task generated by a certain car i, and the latency of the edge service drone j computing the task generated by car i is... for:

[0100] (27)

[0101] Computational energy consumption for processing this task for:

[0102] (28)

[0103] hovering power of edge service drone j for:

[0104] (29)

[0105] Hovering energy consumption of edge service drones j processing tasks generated by vehicles i during the time :

[0106] (30)

[0107] The latency and energy consumption model for the task offloading to the edge service drone is modeled as follows:

[0108] The maximum latency that the task generated by car i can accept is Therefore, the size of the task processed locally by the car is obtained. for:

[0109] (31)

[0110] Therefore, the task size processed by the edge service drone j for:

[0111] (32)

[0112] Automotive i-processing tasks computation delay for:

[0113] (33)

[0114] Automotive i-processing tasks Computational energy consumption for:

[0115] (34)

[0116] Car i will task Transmission latency to edge service drone j for:

[0117] (35)

[0118] Car i will task Transmission power consumption of the drone j transmitting data to the edge service for:

[0119] (36)

[0120] Edge service drone processing tasks computation delay for:

[0121] (37)

[0122] Edge service drone processing tasks Computational energy consumption for:

[0123] (38)

[0124] Edge service drone processing tasks Hovering energy consumption over time for:

[0125] (39)

[0126] Find the total energy consumption of a certain task. for:

[0127] (40)

[0128] Find the total latency of a certain task. for:

[0129] (41)

[0130] Where parameters , , These correspond to decisions for local processing, full uninstallation, and partial task uninstallation, respectively.

[0131] Construct the following optimization problem:

[0132] (42)

[0133] in Let the objective function to be optimized be defined, with the following constraints:

[0134] The distance between the edge service drone and the car is less than 50 meters, that is:

[0135] (43)

[0136] The sum of computing resources allocated to edge service drones is less than their maximum computing resources, that is:

[0137] (44)

[0138] The transmission power allocated to the car is less than the maximum transmission power, that is:

[0139] (45)

[0140] The total latency is less than the maximum latency limit, that is:

[0141] (46)

[0142] The judgment condition can only be 0 or 1, that is:

[0143] (47)

[0144] in, The sum of the three values ​​is 1, so only one value is 1, and the other two are 0.

[0145] (48)

[0146] The computational and hovering power consumption of the cluster-headed drone is less than its maximum power consumption, that is:

[0147] (49)

[0148] The computing and hovering power consumption of edge service drones is less than their maximum power consumption, that is:

[0149] (50)

[0150] in This indicates the maximum battery capacity of the edge service drone. This is the maximum transmission power of the car; This represents the transmit power that vehicle i sends to edge service drone j.

[0151] The above environmental modeling reveals that the problem is a non-convex optimization problem. Establishing it as a Markov decision process and using the DQN solution is a highly effective approach.

[0152] Step 2. The cluster-head drone is used to collect information from vehicles and edge service drones, establish environmental states based on the collected information, and input the environmental states into the DQN algorithm to make corresponding offloading and resource allocation decisions.

[0153] Vehicle information includes the vehicle's location, the computational load of the task generated, the size of the task data, the maximum allowable latency of the task, and the vehicle's local computing resources; information about edge service drones includes the drone's battery level, location, and remaining computing resources.

[0154] In the design of the DQN algorithm, the weights of latency and energy consumption in the reward function are dynamically adjusted according to the real-time monitoring of the battery level of the edge drone, so as to pay more attention to latency when the battery is high and more attention to energy consumption when the battery is low.

[0155] At the same time, the task type is determined based on the amount of computation generated by the car, i.e., driver assistance or in-car entertainment tasks. Depending on the task type, different unloading decisions are encouraged by increasing or decreasing the size of the reward function.

[0156] Deep Q-Network (DQN) is an algorithm that combines deep learning and reinforcement learning. According to existing research, the DQN algorithm performs well in handling tasks with large input dimensions.

[0157] Deploying the DQN algorithm on a cluster-head drone can transmit environmental information to the drone, which can then observe the state of the environment, choose to perform an action, evaluate the quality of the action based on the reward it receives, and continuously iterate to obtain actions with higher rewards.

[0158] Within the framework of Markov decision processes, the state space, action space, and reward are elaborated in detail:

[0159] state space ( The state space set is constructed as follows:

[0160] (51)

[0161] The cluster-headed drone collects information transmitted from vehicle i and edge service drone j to construct a state space, from which we can obtain... .

[0162] After performing the action and receiving feedback, the state space will be updated to obtain... .

[0163] Action space ( The action space set is constructed as follows:

[0164] (52)

[0165] Action A consists of three parts: vehicle transmission power, offloading strategy, and computing resources allocated to edge service drones. The vehicle's transmission power should meet the following requirements. Resource allocation for edge service drones should meet the following requirements. ,in Maximum computing resources for edge-service drones.

[0166] Unloading decisions include three types: vehicle-local processing Fully offload drone processing to edge services Partial task uninstallation processing These three decisions should satisfy the constraints shown in formulas (47) and (48).

[0167] Based on these three actions in the action space, the latency and energy consumption consumed when making a certain unloading decision can be calculated.

[0168] award( The design of the reward function can guide the DQN algorithm to develop in the expected direction during runtime.

[0169] The reward function of this invention incorporates a power weighting factor for monitoring edge service drones during its design. Used to determine the proportion of remaining battery power of edge service drones Calculated using a linear mapping function The calculation formula is as follows:

[0170] (53)

[0171] (54)

[0172] in This indicates the edge service drone. Remaining battery power, Indicating edge service drones Maximum battery capacity, This indicates the number of drones in a drone formation. . This indicates an inverse relationship; the more battery power an edge service drone has remaining, the better. The smaller the value, the better.

[0173] The power weighting factor is obtained from the above formula. The weighting factor is inversely proportional to the battery level, so the more battery power the edge service drone has remaining, the higher the weighting factor becomes. The closer it is to 0; the delay weight is obtained based on this change relationship. and energy consumption weight expression:

[0174] (55)

[0175] (56)

[0176] Let the normalized total delay and total energy consumption be denoted as follows: and Then the reward function for:

[0177] (57)

[0178] like Figure 2 This demonstrates an edge computing task offloading process based on DQN, where information sent from vehicles and edge service drones to the cluster head drone is used as environmental information input, and the intelligent agent establishes a state space based on the input environmental information. Then, an ε-greedy strategy is used to select actions, and a reward function is calculated based on the actions performed. and the state space Action A, Reward and the state space after the action The data is added to the experience pool. The agent randomly selects a batch of data from the experience pool and feeds it into the target network and Q network. It calculates the Q-value and target value, and uses these to calculate the loss. Then, it updates the Q network using gradient descent, and updates the target network every K steps. By iteratively updating in this manner, the unloading decision and resource allocation results are obtained.

[0179] Specifically, the processing flow of the improved task offloading method based on DQN in this embodiment is as follows:

[0180] Step 2.1. Initialize the Q network, target network, and experience pool.

[0181] Step 2.2. Establish environment information for the input information. .

[0182] Step 2.3. Start the loop.

[0183] Step 2.4. Select actions using an ε-greedy strategy. .

[0184] Step 2.5. Calculate the total energy consumption based on the selected actions. Total delay parameters , , , , , , , , , .

[0185] Calculated , The parameters were then obtained through normalization. and , and This is the result after normalization.

[0186] Step 2.6. Monitor the battery level of the edge service drone in real time and calculate... , , , and .

[0187] Step 2.7. Calculate the reward function .

[0188] Step 2.8. Set the task type judgment value ,in .

[0189] if If the task is classified as a driver assistance task, Then reduce the reward function ;

[0190] if If the task is classified as an in-car entertainment task, then... Then add a reward function. .

[0191] Step 2.9. State Space Action A, Reward and the state space after the action Add to the experience pool.

[0192] Step 2.10. Randomly select a batch of data from the experience pool and input it into the Q network and the target network to calculate the Q value and the target value.

[0193] Step 2.11. Calculate the loss based on the Q network and the target network.

[0194] Step 2.12. Update the Q-network using gradient descent.

[0195] Step 2.13. Update the target network every K steps.

[0196] Step 2.14. Obtain the task offloading decision and the allocation results of vehicle transmission power and computing resources.

[0197] Step 2.15. End the loop.

[0198] Within this action space, an ε-greedy strategy is used to select three actions: vehicle transmission power, offloading strategy, and computational resources allocated to the edge service drone. Vehicle transmission power is determined by the vehicle based on the data size and latency requirements, whether the task is fully or partially offloaded. Higher transmission power results in faster data transmission but also higher energy consumption. Computational resources allocated to the edge service drone are determined by the latency requirements and computational workload of the task, whether the task is partially or fully offloaded. More resources allocated result in less time consumed but higher energy consumption. The offloading strategy is selected based on the computational workload of the task generated by the vehicle. The size is used to determine the task type. (Decision value) ,like If the task is classified as a driver assistance task, The task is classified as in-vehicle entertainment. Assisted driving tasks are characterized by high computational requirements but relatively small data volumes and strict latency requirements. Since the computing power of edge service drones exceeds that of the vehicle, and offloading these tasks entirely or partially does not consume excessive communication resources, a full or partial offloading strategy is more suitable for this type of task. In-vehicle entertainment tasks, on the other hand, require less computation but have relatively large data volumes and more relaxed latency requirements. Offloading these tasks would consume excessive communication resources, but since latency requirements are not high, processing them locally within the vehicle saves overall resources. Therefore, a fully local processing offloading strategy is more suitable for this type of task.

[0199] Different unloading strategies are encouraged for different types of tasks. When a task is classified as an assisted driving task, if the agent makes a decision to handle all tasks locally, the reward is reduced, prompting the agent to make a decision to fully or partially unload tasks. When a task is classified as an in-vehicle entertainment task, if the agent makes a decision to handle all tasks locally, the reward is increased, prompting the agent to make a decision to handle tasks locally within the vehicle. This invention determines the task type based on the amount of computation generated by the vehicle and provides corresponding rewards and penalties for different task types. This allows the agent to adopt an unloading strategy corresponding to the task type, making full use of computing resources, reducing communication resource waste, and thus improving the task completion rate.

[0200] In the design of the reward function, the intelligent system detects the battery level of the edge service drone in real time, and obtains a weighting factor for the battery level through a linear mapping function based on the battery level. Based on the power weighting factor Calculate the delay weight and energy consumption weight and normalized delay and energy consumption respectively with and Multiply and add them together to get the reward function. The expression describes how, when the edge service drone has a high battery level, the weighting factor for battery power is smaller, and the weighting factor for latency, as determined by the formula, is greater than that for energy consumption. Therefore, the agent rewards actions that process tasks quickly, allowing it to handle more tasks in a shorter time; in other words, the agent prioritizes latency performance. As the edge service drone's battery level decreases, the weighting factor for battery power... Consequently, the energy consumption weight, derived from the formula, is greater than the latency weight. The agent will then reward actions that consume less overall energy, enabling it to handle as many tasks as possible with limited power consumption. In other words, the agent prioritizes overall energy efficiency. This invention fully utilizes the computing resources of edge service drones to complete more tasks with limited power.

[0201] Step 3. The cluster-head drone will use the DQN algorithm to make corresponding offloading decisions, edge service drone computing resource allocation decisions, and vehicle transmission power allocation decisions, and then feed them back to the vehicle and the edge service drone.

[0202] The vehicle will make corresponding unloading decisions and transmission power allocation according to the instructions of the cluster head drone. The edge service drone will allocate computing resources to the corresponding tasks according to the instructions of the cluster head drone and feed the computing results back to the vehicle.

[0203] In designing the reward function of the DQN algorithm, this invention dynamically adjusts the model from optimizing overall latency performance to optimizing overall energy consumption performance based on the power consumption of the edge service drone. This allows for full utilization of the edge service drone's computing resources, enabling it to complete more tasks with limited power. Furthermore, this invention can determine the task type based on the computational load generated by the vehicle, distinguishing between driver assistance tasks and in-vehicle entertainment tasks. Based on the vehicle's task type, different offloading strategies are encouraged, maximizing computing resources, reducing communication resource waste, and improving task completion rates.

[0204] Now, suppose there is a drone platoon consisting of three drones and forty vehicles at a traffic light intersection. In the drone platoon, one drone is the leader and two are edge service drones. The leader drone collects and processes information, and feeds back the resulting task offloading strategy, vehicle transmit power allocation, and resource allocation results to the vehicles and edge service drones.

[0205] In a simulation involving forty vehicles, each vehicle generates only one task, and the task type, its computational complexity, and data size are randomly generated. The simulation results are as follows: Figure 4 As shown. Figure 4 The horizontal axis represents the number of iterations, and the vertical axis represents the average number of tasks completed. With environmental information input, after 1200 rounds of iterative training, the average number of completed tasks stabilized at around 35, meaning the average task completion rate stabilized at around 88%, verifying the effectiveness of the method of this invention.

[0206] This invention can adjust the optimization direction of the task offloading strategy in real time based on the remaining battery power of the edge service drone. When the drone has sufficient power, the reward function prioritizes guiding the model to optimize the latency performance of the overall task execution, thereby improving system response speed and service quality. When the drone's battery power decreases, the model automatically shifts to optimizing overall energy consumption performance based on the battery change, thus extending the effective service time of the drone under energy constraints and ensuring the continuous operation of the system. This mechanism effectively improves the utilization efficiency of computing resources, enabling more task offloading and processing operations to be completed even in environments with limited edge resources. In addition, this invention introduces a task feature-based type discrimination mechanism, which can dynamically determine whether a task is a driver assistance task or an in-vehicle entertainment task based on the amount of computation generated by the ground vehicle terminal. The system applies differentiated guidance to the reward function according to the different task types, which not only improves the task completion rate and resource matching accuracy, but also effectively reduces unnecessary communication resource waste and enhances the scalability and service capabilities of the system in complex multi-task scenarios.

[0207] Example 2

[0208] This embodiment 2 describes a vehicle task unloading system assisted by a drone swarm in an intelligent transportation environment, which includes multiple intelligent connected vehicles and a formation of multiple drones, wherein the drone formation consists of a cluster leader drone and several edge service drones; the cluster leader drone includes a memory and one or more processors.

[0209] The memory stores executable code; when the processor executes the executable code, it implements the steps of the vehicle task offloading method assisted by UAV swarm in the intelligent transportation environment described in Embodiment 1 above.

[0210] Example 3

[0211] This embodiment 3 describes a computer-readable storage medium storing a program that, when executed by a processor, is used to implement the steps of the vehicle task offloading method assisted by a drone swarm in the intelligent transportation environment described in embodiment 1 above.

[0212] The computer-readable storage medium can be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc.

[0213] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

[0214] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. A method for vehicle task offloading assisted by a UAV cluster in an intelligent transportation environment, characterized in that it comprises the following steps: Step 1. Building a scene of a red-light crossroad composed of multiple intelligent connected vehicles, a UAV formation, and red lights; the UAV formation is composed of a cluster head UAV and a plurality of edge service UAVs; Step 2. The cluster head UAV collects information from the vehicles and the edge service UAVs, establishes an environment state according to the collected information, and inputs the environment state into the DQN algorithm to make corresponding offloading decisions and resource allocation decisions; In the design process of the DQN algorithm, the weights of the delay and energy consumption in the reward function are dynamically adjusted according to the real-time monitoring of the edge UAV's power size, so as to pay more attention to the delay when the power is high and pay more attention to the energy consumption when the power is low; At the same time, the task type is judged according to the task calculation amount generated by the vehicle, i.e., an auxiliary driving type or an in-vehicle entertainment type task, and different offloading decisions are encouraged by increasing / decreasing the size of the reward function according to the different task types; The offloading decisions include local processing by the vehicle, full offloading, or partial offloading to the edge service UAV for processing; Step 3. The cluster head UAV feeds back the corresponding offloading decisions made by the DQN algorithm, the edge service UAV computing resource allocation decisions, and the vehicle transmit power allocation to the vehicle and the edge service UAV. In step 1, the vehicle's information includes the vehicle's position, the generated task calculation amount, the task data size, the maximum allowed delay of the task, and the vehicle's local computing resources; The following optimization problem is constructed: ; represents the objective function to be optimized; represents the total energy consumption of a task, represents the total latency of a task; represents the number of cars; the constraint is: Edge service drone and car distance less than 50 meters, i.e. ; ; allocating computing resources to an edge service drone j for a certain car i; computing resources for an edge service drone j; ; transmission power for vehicle i to send to edge service drone j; maximum transmission power for car; less than the maximum limit latency i.e. ; The decision condition can only be 0 and 1, ; The sum of the three values is 1, so only one value is 1, and the other two are 0, , , respectively correspond to the decisions of local processing, full offloading processing and partial task offloading processing.​ ; the computation delay of the cluster head unmanned aerial vehicle processing the task generated by the car i, the hovering energy consumption of the cluster head unmanned aerial vehicle in the time of computing the task generated by the car i; the maximum power ; Maximal energy of the edge serving drone; Computational energy of the edge serving drone j to process the tasks generated by the car i, Hovering energy of the edge serving drone j in time to process the tasks generated by the car i, speed of the car i to transmit the tasks to the edge serving drone j is: ； wherein is the channel bandwidth, is the reference channel gain, is the Boltzmann constant, is the temperature, is the channel fading factor, is the path loss exponent; The calculation formula is as follows: ； wherein and denotes the horizontal coordinate of the car i, denotes the height of the car i; and denotes the horizontal coordinate of the drone j, denotes the height of the drone j; denotes the set of cars, denotes the set of drones; The information of the edge service UAV includes the edge service UAV's power, position, and remaining computing resources. 2.The method of claim 1, wherein, In step 1, the process of building the red-light crossroad scene is as follows: In step 2, the improved task offloading method based on the DQN algorithm is as follows: 3.The method of claim 1, wherein, Step 2.

1. Initialize the Q network, target network, and experience pool; Definition of the set of cars is: ; coordinates of car i in a three-dimensional Cartesian coordinate system is: ; definition of the set of car coordinates is: ; denotes the coordinates of car i; defining a set of task information generated by the car i for: ; wherein is the amount of task computation generated by car i, in bit / cycles; denotes the amount of task generated by car i, in bit; denotes the maximum completion latency of the task generated by car i, in seconds; So each task generated by the car i corresponds to the amount of computation required To: ; Defining a set of drones To: ; The coordinates of the UAV j correspond to the coordinates of the three-dimensional Cartesian coordinate system are: ; defining a set of coordinates of the drone is: ; wherein represents the coordinates of the drone; The cluster-headed drone does not participate in specific task calculations; its maximum battery power... It is 100Wh; Edge service drone j sends information to cluster head drone To: ; represents the remaining power of the edge service drone j, the maximum power of the edge service drone j is 100 Wh, represents the computing resources of the edge service drone j, the computing resources of the edge service drone j allocated to a certain car i is ; then the computational resource allocation matrix is: ; The car i sends information to the cluster head unmanned aerial vehicle To: ; wherein represents the computing resources of the car i itself; Unloading decision made by cluster head drone on car i To: ; All car offload decision matrix is represented as: . 4.The method of claim 1, wherein, In step 2, the reward function is designed to introduce a power weight factor to monitor the edge service drone , according to the proportion of the remaining power of the edge service drone , calculated by using a linear mapping function , the calculation formula is as follows: ； ； wherein representing edge service drones remaining power, representing edge service drones maximum power, representing the number of drones in the drone formation, , representing an inverse relationship; wherein the more residual power the edge service drone has, the smaller the value of the number of values. From the above formula, the electric quantity weight factor is inversely proportional to the electric quantity, so when the edge service UAV has more electric quantity left, the weight factor is closer to 0; according to the change relationship, the delay weight and the energy consumption weight are expressed as: ； ； Let the normalized total latency and total energy consumption be denoted as and respectively, then the reward function is: . 5.The method of claim 4, wherein, Step 2.

3. Start the loop; Step 2.

10. Randomly select a batch of data from the experience pool and input them into the Q network and the target network to calculate the Q value and the target value; Step 2.

2. Establishing context information for input information ; wherein denotes the coordinates of the car i, denotes the computing resources of the car i itself, is the amount of task computation generated by the car i; denotes the amount of electricity left for the edge service drone j, is the computing resource of the edge service drone j; denotes the position of the drone, denotes the amount of tasks generated by the car i; denotes the maximum completion delay of the task generated by the car i; Step 2.

11. Calculate the loss according to the Q network and the target network; Step 2.

4. Select action using ε-greedy policy ; wherein the offloading decision made by the cluster head drone for the car i is: ; denotes the transmit power of the vehicle i to the edge service drone j; denotes the computing resource allocated by the edge service drone j to a certain car i; wherein , ; Step 2.

5. Calculate the parameters of total energy consumption and total latency according to the selected action and obtain , , , , , , are the normalized results; Step 2.

6. Monitor the drone battery level of the edge service in real time and calculate , , , and ; Step 2.

7. Calculate reward function ; Step 2.

8. Set task type decision value wherein ; If : if the task is determined to be an assisted driving type task, and if , then the reward function is reduced; If : if the task is determined to be an in-vehicle entertainment type task, then the reward function is increased ; Step 2.

9. State space , action A, reward and state space after action Put into experience pool; Step 2.

12. Update the Q network using the gradient descent method; Step 2.

13. Update the target network every K steps; Step 2.

14. Obtain the task offloading decisions and the allocation results of the vehicle transmit power and computing resources; Step 2.

15. End the loop. The delay and energy consumption model for task local computation in the vehicle is as follows: The delay and energy consumption model for full offloading of the task to the edge service UAV is as follows: 6.The method of claim 5, wherein, In step 2.5, the total energy consumption is calculated. Total delay The parameters include , , , , , , , , , The solution process and meaning of each parameter are as follows: Energy consumption of the cluster head unmanned aerial vehicle computing car i generated by the task is: ; wherein is the energy consumption coefficient of the unmanned aerial vehicle chip, is the number of cycles required to execute DQN, is the computing power of the cluster head unmanned aerial vehicle; The cluster head unmanned aerial vehicle processes the energy consumption of all the tasks generated by the cars To: ; Cluster head unmanned aerial vehicle processes computing delay of task generated by automobile i To: ; Latency of cluster head unmanned aerial vehicle to process all car-generated tasks To: ; Hover power of cluster head unmanned aerial vehicle is: ; wherein is an aerodynamic constant, is the mass of the unmanned aerial vehicle, is the air density, is the rotor area of the unmanned aerial vehicle; Hovering energy consumption of cluster head unmanned aerial vehicle in computing the time of tasks generated by the car i To: ; The delay and energy consumption model for partial offloading of the task to the edge service UAV is as follows: If the task is only processed locally, the computing power of the vehicle i is ; The vehicle will provide all the computing power to this task, computing latency To: ; Vehicle i handles the computational energy consumption of tasks it generates itself To: ; wherein is the energy consumption coefficient of the automotive chip; ​ Car i will generate the task Transmission latency for full offloading to edge service drone j For: ; wherein denotes the speed at which the car i transfers the task to the edge service drone j; Automobile i will transmit the task to the energy consumption of edge service drone j To: ; Definitions is the computing resource of the edge service drone j assigned to the task generated by a certain car i, the latency of the edge service drone j computing the task generated by the car i is: ; The processing of this task corresponds to the computing energy consumption To: ; Hover power for edge service drone j To: ; Edge service drone j handles hover energy consumption within the time of tasks generated by car i : ; ​ The maximum time delay accepted by the task generated by the car i is ; thus the size of the task processed locally by the car i is : ; So, the edge service drone j handles the task size is: ; Automobile i handles tasks Computational latency Is: ; Automobile i handles tasks Computational energy consumption Is: ; Car i will task Transmission latency to edge service drone j To: ; Car i will task Transmission energy consumption to edge service drone j For: ; Edge service drone j handles task Computational latency Is: ; Edge service drone j handles tasks Computational energy consumption Is: ; Edge service drone j handles tasks Hovering energy consumption over time Is: ; total energy consumption for a certain task for: ； total latency for a certain task for: 。 7. A system for UAV swarm-assisted vehicle task offloading in an intelligent transportation environment, comprising a plurality of intelligent connected vehicles and a UAV formation, the UAV formation comprising a cluster head UAV and a plurality of edge service UAVs; the cluster head UAV comprising a memory and one or more processors; the memory storing executable code; characterized in that the executable code, when executed by the one or more processors, is configured to cause the cluster head UAV to perform steps of the method for UAV swarm-assisted vehicle task offloading in an intelligent transportation environment according to any one of claims 1 to 6.

8. A computer readable storage medium having stored thereon a program; characterized in that, the program, when executed by the one or more processors, is configured to cause the cluster head UAV to perform steps of the method for UAV swarm-assisted vehicle task offloading in an intelligent transportation environment according to any one of claims 1 to 6.

Images