A method for offloading computing tasks in Internet of Vehicles based on digital twins

Abstract

The present invention relates to the field of electronic technology, and discloses a method for unloading computing tasks in a vehicle network based on digital twins. The method uses a digital twin model to simulate the operating status of each drone and each vehicle, obtains twin data of multiple time slots, and constructs an objective function and constraint conditions based on the twin data. The objective function is constructed with the goal of minimizing the total system delay. The objective function is solved using digital twin data of multiple drones and vehicles, constraint conditions, and a preset reinforcement learning model to obtain the association between vehicles and drones when the system delay is minimized and the task unloading ratio of each vehicle. Based on the solution results, the vehicle is controlled to unload computing tasks. The method provided by the present invention controls the vehicle to unload computing tasks based on the solution results of the objective function, effectively optimizes the association between drones and vehicles and the task unloading ratio, and reduces the delay of the entire system.

Description

Technical Field

[0001] The present invention relates to the field of electronic technology, and in particular to a method for offloading computing tasks in an Internet of Vehicles based on digital twins. Background Art

[0002] With the development of various terminal devices and the fifth generation of mobile communication technology, new IoT applications such as smart wearable devices, online car-hailing, and autonomous driving have begun to play an increasingly important role in the fields of medical care and transportation. Due to the limited available resources and processing power of IoT devices, they face unprecedented challenges in mobile computing. Mobile Edge Computing (MEC) technology uses the powerful computing power of nearby edge servers to deploy computing and caching functions to wireless access networks closer to users, so that mobile terminal devices only need to offload local application services to MEC servers at the edge of the network through wireless networks (such as cellular networks, WiFi, Bluetooth, etc.) to enjoy high-performance network services, reducing the high latency, network congestion, and high energy consumption caused by large amounts of data calculations. However, when IoT devices and edge servers are deployed on a large scale in the MEC system, it is particularly important to design efficient computing offloading strategies and perform reasonable resource allocation due to the massive connections and high dynamics of the network.

[0003] Recently, drones equipped with MEC servers have been used to provide computing services for mobile terminals and have been widely discussed in industry and academia as a new technology, providing great convenience for IoT applications to implement time-sensitive and computing-intensive services. The drone-based edge network can not only minimize the energy consumption of drones and IoT devices through optimal offloading decisions, but also reduce the maximum latency by optimizing the association between drone base stations and users, caching strategies, computing capacity allocation, and drone locations.

[0004] In related technologies, due to the limited computing power of the vehicle itself, the computing speed can be effectively improved by offloading the vehicle's computing tasks to drones equipped with MEC servers. However, when each vehicle in a multi-drone-assisted vehicle network system offloads computing tasks, it generally offloads a certain proportion of computing tasks to the corresponding associated drones for calculation based on pre-established offloading rules. It is not possible to flexibly coordinate the drones associated with the vehicle and the proportion of offloaded tasks, resulting in low overall computing efficiency of the system. Summary of the invention

[0005] In view of this, the present invention provides a method for unloading vehicle computing tasks based on digital twins to solve the problem of low overall system computing efficiency in the related art of unloading vehicle computing tasks based on pre-established rules.

[0006] In a first aspect, the present invention provides a method for unloading computing tasks in an Internet of Vehicles based on digital twins, which is applied to a data center, wherein the data center is communicatively connected to a plurality of drones, and each drone is communicatively connected to a plurality of vehicles. The method comprises: constructing digital twin models of a plurality of drones and a plurality of vehicles; simulating the operating status of each drone and each vehicle using the digital twin model to obtain twin data of a plurality of time slots; constructing an objective function and constraint conditions based on the twin data of a plurality of time slots, wherein the objective function is constructed with the minimum total system delay as the goal, and the constraint conditions include first constraint information of the association relationship between each vehicle and different drones, second constraint information of the number of drones associated with each vehicle, and third constraint information of the task unloading ratio of each vehicle, and the objective function is used to characterize the association relationship between each vehicle and different drones, the correlation between the task unloading ratio of each vehicle and the total system delay; solving the objective function using the twin data of a plurality of time slots, the first constraint information, the second constraint information, the third constraint information, and a preset reinforcement learning model to obtain the target association relationship between each vehicle and different drones and the target computing task unloading ratio of each vehicle; controlling the corresponding vehicle to unload computing tasks based on the target association information between each vehicle and different drones and the target computing task unloading ratio.

[0007] The vehicle network computing task unloading method based on digital twin provided by the present invention constructs digital twin models of multiple drones and multiple vehicles, uses the digital twin models to simulate the operating status of each drone and each vehicle, obtains twin data of multiple time slots, and constructs an objective function and constraint conditions based on the twin data. The objective function is constructed with the goal of minimizing the total system delay. The constraint conditions include first constraint information of the association relationship between each vehicle and different drones, second constraint information of the number of drones associated with each vehicle, and third constraint information of the task unloading ratio of each vehicle. The objective function is used to characterize the association relationship between each vehicle and different drones, the correlation between the task unloading ratio of each vehicle and the total system delay. The objective function is solved using the twin data of multiple time slots, the first constraint information, the second constraint information, the third constraint information, and the preset reinforcement learning model to obtain the target association relationship between each vehicle and different drones and the target computing task unloading ratio of each vehicle; based on the target association information between each vehicle and different drones and the target computing task unloading ratio, the corresponding vehicle is controlled to unload the computing task. The method provided by the present invention constructs an objective function with the goal of minimizing the total system delay, and solves the objective function using digital twin data of multiple drones and vehicles, constraints, and a preset reinforcement learning model to obtain the association between vehicles and drones and the task offloading ratio of each vehicle when the system delay is minimized. Based on the solution results, the vehicle is controlled to offload computing tasks, effectively optimizing the association between drones and vehicles and the task offloading ratio, thereby reducing the delay of the entire system.

[0008] In an optional embodiment, the twin data includes the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, the computing task volume information generated by each vehicle, the computing power information of each vehicle, and the computing power information of each drone.

[0009] In an optional embodiment, the total system delay is used to characterize the sum of the computing delay and the transmission delay. The computing delay is determined by the computing task volume information generated by each vehicle, the computing power information of each vehicle, the computing power information of each drone, and the computing task offloading ratio of each vehicle. The transmission delay is determined by the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, and the computing task offloading ratio of each vehicle.

[0010] In an optional embodiment, the objective function is solved using twin data, first constraint information, second constraint information, and third constraint information and a preset reinforcement learning model to obtain the target association relationship between each vehicle and different drones and the target computing task unloading ratio of each vehicle, including: determining the state space of each vehicle in the corresponding time slot based on the computing task volume information corresponding to different vehicles in each time slot, the channel gain between each vehicle and different drones, and the channel gain between each drone and a data center; determining the action space of each vehicle based on the first constraint information, the second constraint information, and the third constraint information, the action space of each vehicle being used to characterize the multiple computing task unloading ratios of the corresponding vehicle under the constraint conditions and the association relationship between the vehicle and the drone corresponding to each computing task unloading ratio; using the state space of different vehicles in each time slot, the action space of each vehicle, the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and a data center, the computing task volume information generated by each vehicle, the computing power information of each vehicle, and the computing power information of each drone, the objective function is iteratively solved to obtain a solution result; determining the target association relationship between each vehicle and different drones and the target computing task unloading ratio of each vehicle based on the solution result.

[0011] In an optional embodiment, the computing delay includes a first computing delay of each vehicle itself and a second computing delay for each vehicle to offload computing tasks to a drone for computing; the transmission delay includes a first transmission delay for each vehicle to offload computing tasks and transmit them to an associated drone, a second transmission delay generated when the drone uploads computing task information to a data center, and a third transmission delay for the data center to send decision information; the first computing delay is determined by the computing task volume information, computing power information, and computing task offloading ratio generated by the corresponding vehicle; the second computing delay is determined by the computing power information, the computing task volume information, and the computing task offloading ratio generated by the corresponding drone; the first transmission delay is determined by the first data transmission rate between the corresponding vehicle and different drones and the computing task offloading ratio of the corresponding vehicle; the second transmission delay is determined by the second data transmission rate between the corresponding drone and the data center and the size of the computing task information of the corresponding drone.

[0012] In an optional embodiment, the first data transmission rate corresponding to each vehicle and different drones is determined by the following steps: obtaining the flight altitude information of each drone, the position information of each drone, the position information of each vehicle, the channel power gain, the Rayleigh fading coefficient corresponding to each vehicle and different drones, the communication power corresponding to each vehicle and different drones, the Gaussian white noise power and the channel bandwidth; determining the free space fading coefficient corresponding to each vehicle and different drones according to the flight altitude information of each drone, the position information of each drone, the position information of the corresponding vehicle and the channel power gain; determining the channel gain corresponding to each vehicle and different drones according to the free space fading coefficient and the Rayleigh fading coefficient corresponding to each vehicle and different drones; determining the signal-to-noise ratio corresponding to each vehicle and different drones according to the communication power, channel gain and Gaussian white noise power corresponding to each vehicle and different drones; determining the first data transmission rate corresponding to each vehicle and different drones based on the signal-to-noise ratio and the channel bandwidth corresponding to each vehicle and different drones.

[0013] In an optional embodiment, the second data transmission rate between each UAV and the data center is determined by the following steps: obtaining the communication power between the UAV and the data center; determining the channel gain between the corresponding UAV and the data center based on the location information, flight altitude and channel power gain of each UAV; determining the second data transmission rate between the corresponding UAV and the data center based on the channel gain, communication power, the Gaussian white noise power and the channel bandwidth between each UAV and the data center. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 is a flowchart of a method for offloading computing tasks of an Internet of Vehicles based on digital twins according to an embodiment of the present invention;

[0016] Figure 2 is a flow chart of another method for offloading computing tasks of an Internet of Vehicles based on digital twins according to an embodiment of the present invention;

[0017] Figure 3 is a comparative schematic diagram of a specific example of an embodiment of the present invention;

[0018] Figure 4 is a comparative schematic diagram of another specific example of the embodiment of the present invention;

[0019] Figure 5 It is a structural block diagram of a vehicle networking computing task unloading device based on digital twin according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0021] In related technologies, due to the limited computing power of the vehicle itself, the computing speed can be effectively improved by offloading the vehicle's computing tasks to drones equipped with MEC servers. However, when each vehicle in a multi-drone-assisted vehicle network system offloads computing tasks, it generally offloads a certain proportion of computing tasks to the corresponding associated drones for calculation based on pre-established offloading rules. It is not possible to flexibly coordinate the drones associated with the vehicle and the proportion of offloaded tasks, resulting in low overall computing efficiency of the system.

[0022] Digital Twins (DT) generate digital images of the physical world through physical entities, sensor updates, and historical operating status data. At the same time, digital twin technology simulates the operating status of physical entities in the digital world and predicts the operating status of physical entities. Digital twins can also be used in communications and networks, such as cloud computing and edge computing. These features have gradually attracted widespread attention from academia and industry at home and abroad. In the digital world, intelligent optimization methods and real-time data provided by DT are used to make accurate and timely unloading decisions and minimize latency. To this end, DT provides a new approach to solving high-complexity problems in wireless networks.

[0023] In view of this, the embodiment of the present application provides a method for unloading computing tasks of Internet of Vehicles based on digital twins, which uses a data center to realize the unloading of vehicle computing tasks. The method provided by the present invention constructs an objective function with the minimum total system delay as the goal, solves the objective function using digital twin data of multiple drones and vehicles, constraints, and a preset reinforcement learning model, obtains the association between vehicles and drones when the system delay is minimum, and the task unloading ratio of each vehicle, controls the vehicle to unload computing tasks based on the solution results, effectively optimizes the association between drones and vehicles and the task unloading ratio, thereby reducing the delay of the entire system.

[0024] According to an embodiment of the present invention, an embodiment of a method for offloading computing tasks of an Internet of Vehicles based on digital twins is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0025] In this embodiment, a method for offloading computing tasks of an Internet of Vehicles based on digital twins is provided, which is applied to a data center, wherein the data center is connected to multiple drones for communication, and each drone is connected to multiple vehicles for communication. Figure 1 is a flowchart of a method for offloading computing tasks of an Internet of Vehicles based on digital twins according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0026] Step S101, construct digital twin models of multiple drones and multiple vehicles.

[0027] For example, the embodiment of the present application does not limit the number of drones and vehicles, and those skilled in the art can determine it according to their needs. In the embodiment of the present application, multiple drones and multiple vehicles form a multi-drone network system, whose physical network consists of a data center and multiple drones and vehicles, where the number of drones is M, and the definition is Denotes the set of drones, the number of vehicles is K, and the definition is Represents a collection of vehicles, the location of the data center is at the origin O, and the vehicles and drones are connected to the data center through wireless channels to complete data transmission. Each drone is equipped with a MEC server, and the vehicle offloads computing tasks to the drone for calculation. During the movement of vehicles and drones, each vehicle generates computing tasks, and all tasks are independent of each other. The vehicle offloads part of the tasks to the associated drone for operation. The drone uploads the computing task information offloaded by the vehicle to the data center. The digital twin system in the data center stores and updates the relevant information of the drones and vehicles. The digital twin system generates corresponding decision information based on the relevant information and sends it to the drones and vehicles. In the embodiment of the present application, based on the physical entities, sensor updates, and historical operating status data of multiple drones and multiple vehicles in the physical network, a digital mirror of the physical world is generated to obtain digital twin models of multiple vehicles.

[0028] Step S102: Use the digital twin model to simulate the operating status of each drone and each vehicle to obtain twin data of multiple time slots.

[0029] For example, in the embodiment of the present application, a time slot is the minimum unit for transmitting circuit-switched summary information, and the duration of each time slot is the same. The set of time slots is represented as In time slot n, drones and vehicles are relatively stationary. Twin data includes the operating data of each drone and each vehicle and the information transmission capability between drones and vehicles. The operating status data of each vehicle includes vehicle location information, computing tasks and computing capabilities generated by the vehicle, etc. The operating data of drones includes the flight altitude of drones, drone location information, computing capabilities, etc. Digital twins can fully store historical information, hardware configuration and other data of network devices, and can also interact with network devices through mechanisms such as real-time updates. Therefore, the digital twin of drone m in time slot n can be shown as follows (1):

[0030]

[0031] Among them, DT m [n] represents the digital twin of drone m in time slot n, f m [n] represents the estimated computing capacity of drone m in time slot n, represents the deviation between the actual computing capacity of UAV m in time slot n and the estimated computing capacity of UAV m stored in the digital twin;

[0032] The digital twin of vehicle m in time slot n can be expressed as follows:

[0033]

[0034] Among them, DT k [n] represents the digital twin of vehicle k in time slot n, f k [n] represents the estimated computing power of vehicle k in time slot n, Represents the deviation value between the actual computing capacity of vehicle k in time slot n and the estimated computing capacity of vehicle k stored in the digital twin.

[0035] Step S103, constructing the objective function and constraints based on the twin data of multiple time slots.

[0036] Exemplarily, the objective function is constructed with the goal of minimizing the total system delay, and the constraints include the first constraint information of the association relationship between each vehicle and different drones, the second constraint information of the number of drones associated with each vehicle, and the third constraint information of the task unloading ratio of each vehicle. The objective function is used to characterize the association relationship between each vehicle and different drones in different time slots, the correlation between the task unloading ratio of each vehicle in different time slots and the total system delay. In the embodiment of the present application, the constraint condition can be shown as follows (3):

[0037]

[0038] in, Indicates whether there is a correlation between vehicle k and drone m, β k [n] represents the calculation ratio of vehicle k’s own calculation tasks, 1-β k [n] represents the ratio of vehicle k offloading computing tasks to the associated UAV, C1 represents the first constraint information, C2 represents the second constraint information, and C3 represents the third constraint information.

[0039] Step S104, using the twin data of multiple time slots, the first constraint information, the second constraint information and the third constraint information and the preset reinforcement learning model to solve the objective function, and obtain the target association relationship between each vehicle and different drones and the target calculation task unloading ratio of each vehicle.

[0040] Exemplarily, in an embodiment of the present application, the preset reinforcement learning model may include but is not limited to a proximal policy optimization (PPO) model. The PPO algorithm model solves the objective function to obtain the target association relationship and task unloading ratio between each vehicle and different drones in different time slots when the total system delay is minimized. Since solving the objective function includes calculations and transmission delays in multiple time slots, the proximal policy optimization (PPO) algorithm has better training efficiency and robustness than other deep reinforcement learning algorithms. The two-layer PPO network considers the dynamic and time-varying edge network digital twin model and the physical network mapping error, and can better solve the adaptive optimization problem. The PPO algorithm runs in a data center. The system inputs drone and vehicle information into the PPO algorithm, optimizes the delay of the entire system, and finally sends the optimized information back to the drone and vehicle.

[0041] Step S105, based on the target association information between each vehicle and different UAVs and the target computing task unloading ratio, the corresponding vehicle is controlled to unload the computing task.

[0042] Exemplarily, in an embodiment of the present application, based on the target association relationships and task offloading ratios between each vehicle and different drones in different time slots, the corresponding vehicle is controlled to offload the computing tasks to the associated drone for computing.

[0043] The digital twin-based Internet of Vehicles computing task offloading method provided in this embodiment constructs an objective function with the goal of minimizing the total system delay, and solves the objective function using the digital twin data of multiple drones and vehicles, constraints, and a preset reinforcement learning model to obtain the association between vehicles and drones and the task offloading ratio of each vehicle when the system delay is minimized. Based on the solution results, the vehicle is controlled to offload computing tasks, which effectively optimizes the association between drones and vehicles and the task offloading ratio, thereby reducing the delay of the entire system.

[0044] In this embodiment, a method for offloading computing tasks of an Internet of Vehicles based on digital twins is provided, which is applied to a data center, wherein the data center is connected to multiple drones for communication, and each drone is connected to multiple vehicles for communication. Figure 2 is a flowchart of a method for offloading computing tasks of an Internet of Vehicles based on digital twins according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0045] Step S201: construct digital twin models of multiple drones and multiple vehicles. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0046] Step S202: Use the digital twin model to simulate the operating status of each drone and each vehicle to obtain twin data of multiple time slots. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0047] Step S203, construct objective functions and constraints based on the twin data of multiple time slots. The objective function is constructed with the goal of minimizing the total system delay. The constraints include the first constraint information of the association relationship between each vehicle and different drones, the second constraint information of the number of drones associated with each vehicle, and the third constraint information of the task unloading ratio of each vehicle. The objective function is used to characterize the association relationship between each vehicle and different drones, the correlation between the task unloading ratio of each vehicle and the total system delay. For details, please refer to Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0048] Step S204, using the twin data of multiple time slots, the first constraint information, the second constraint information, the third constraint information and the preset reinforcement learning model to solve the objective function, and obtain the target association relationship between each vehicle and different drones and the target calculation task unloading ratio of each vehicle.

[0049] In some optional embodiments, the twin data includes the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, the computing task volume information generated by each vehicle, the computing power information of each vehicle, and the computing power information of each drone.

[0050] In some optional embodiments, the total system delay is used to characterize the sum of the computing delay and the transmission delay. The computing delay is determined by the computing task volume information generated by each vehicle, the computing power information of each vehicle, the computing power information of each drone, and the computing task unloading ratio of each vehicle. The transmission delay is determined by the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, and the computing task unloading ratio of each vehicle.

[0051] For example, in the embodiment of the present application, the vehicle uploads the computing task to the associated drone, the drone uploads the information of the computing task uploaded by the vehicle to the digital center, the digital center sends the decision information to the drone and the vehicle, and the drone calculates the computing task through the overall planning given by the digital center. Therefore, the total system delay can be determined by the following formula (4):

[0052]

[0053] Where T represents the total system delay, T com[n] represents the computation delay in time slot n, T trans [n] indicates the calculation of the transmission delay within time slot n.

[0054] The relationship between drones and vehicles is The ratio of vehicle-to-UAV task offloading is Taking the total delay of the system as the optimization goal, the relationship between the UAV and the vehicle and the impact of the task offloading ratio from the vehicle to the UAV on the total delay of the system are studied. The objective function can be expressed as

[0055] In some optional embodiments, the computing delay includes a first computing delay of each vehicle itself and a second computing delay for each vehicle to offload computing tasks to a drone for computing; the transmission delay includes a first transmission delay for each vehicle to offload computing tasks and transmit them to an associated drone, a second transmission delay for the drone to upload computing task information to a data center, and a third transmission delay for the data center to send decision information; the first computing delay is determined by the computing task volume information, computing power information, and computing task offloading ratio generated by the corresponding vehicle; the second computing delay is determined by the computing power information, the computing task volume information, and the computing task offloading ratio generated by the corresponding drone; the first transmission delay is determined by the first data transmission rate between the corresponding vehicle and different drones and the computing task offloading ratio of the corresponding vehicle; the second transmission delay is determined by the second data transmission rate between the corresponding drone and the data center and the size of the computing task information of the corresponding drone.

[0056] Exemplarily, in the embodiment of the present application, the calculation delay in time slot n can be determined by the following formula (5):

[0057]

[0058] Among them, T com [n] represents the computation delay within time slot n, represents the actual computing delay time slot (first computing delay) from vehicle k unloading computing tasks to drone n in time slot n, represents the actual computational delay of car k itself in time slot n (second computational delay);

[0059] Determined by the following formula (6):

[0060]

[0061] in, represents the actual computational delay time slot in time slot n where vehicle k offloads the computational task to drone m for computation. represents the calculated value of the corresponding computing delay of vehicle k unloading the computing task to drone m in time slot n, It represents the error between the calculated value and the true value of the corresponding calculation delay of vehicle k unloading to drone m in time slot n;

[0062] Determined by the following formula (7):

[0063]

[0064] Among them, β k [n] represents the calculation ratio of vehicle k’s own calculation tasks, 1-β k [n] represents the ratio of vehicle k to offload computing tasks to associated drones. Each vehicle k can only be associated with one drone m. k [n] represents the amount of computing task data generated by vehicle k in time slot n, C k [n] represents the number of CPU cycles required to complete a 1-bit computation task, f m [n] represents the estimated computing capacity of drone m in time slot n, Indicates whether vehicle k and drone m are associated, when When , vehicle k and drone m can communicate with each other, and drone m receives the computing task unloaded by vehicle k. When vehicle k and drone m cannot communicate, drone m does not receive the computing task unloaded by vehicle k. For any vehicle k, the association relationship with a drone satisfies

[0065] Determined by the following formula (8):

[0066]

[0067] in, It represents the deviation between the actual computing power of UAV m in time slot n and the estimated computing power of UAV m stored in the digital twin. The meanings of other variables are not repeated here.

[0068] Determined by the following formula (9):

[0069]

[0070] in, represents the true value of the calculation delay of car k itself in time slot n, represents the calculated value of the calculated delay of car k in time slot n, ΔT k [n] represents the deviation between the actual value and the calculated value of the calculated delay of car k itself in time slot n.

[0071] Determined by the following formula (10):

[0072]

[0073] Among them, f k [n] represents the estimated computing capacity of vehicle k in time slot n. The meanings of other variables are not repeated here.

[0074] ΔT k [n] is determined by the following formula (11):

[0075]

[0076] in, It represents the deviation between the actual computing power of vehicle k in time slot n and the estimated computing power of vehicle k stored in the digital twin. The meanings of other variables are not repeated here.

[0077] The transmission delay in time slot n can be determined by the following equation (12):

[0078]

[0079] Among them, T trans [n] represents the transmission delay in time slot n, represents the first transmission delay of the offloaded computing task from vehicle k to the associated UAV m, represents the second transmission delay generated when UAV m uploads the computing task information to data center O, T d [n] represents the time when the data center issues a decision in time slot n.

[0080] Calculated by the following formula (13):

[0081]

[0082] in, represents the first data transmission rate between vehicle k and drone m. The meanings of other variables refer to the relevant description and will not be repeated here.

[0083] Calculated by the following formula (14):

[0084]

[0085] in, represents the data transmission rate between UAV m and the process of uploading mission information to data center O, Indicates the size of the computing task information uploaded by the associated vehicle to the drone m.

[0086] Specifically, the above step S204 includes:

[0087] Step S2041, based on the computing task information generated by different vehicles in each time slot, the channel gain between each vehicle and different drones, and the channel gain between each drone and the data center, determine the state space of each vehicle in the corresponding time slot.

[0088] Exemplarily, in an embodiment of the present application, a Markov decision process (MDP) is an optimal decision process for a stochastic dynamic system based on Markov theory. Based on each observed state, the decision maker selects an action from the available action set to make a decision, and obtains its state transition probability based on the newly observed state. The mathematical description of MDP can be expressed as {s, a, r, p}, where s is defined as the state of the agent, a is defined as the action performed in each state, r is defined as the reward in this state, and p represents the state transition probability of taking action a in the current state s. In an embodiment of the present application, the optimization problem is set as an MDP problem. At the decision time n, the twin model state space maintained by the data center includes the vehicle's computing task D k [n] and channel gain and The state space can be expressed as follows:

[0089]

[0090] Step S2042, based on the first constraint information, the second constraint information and the third constraint information, the action space of each vehicle is determined, and the action space of each vehicle is used to characterize the multiple computing task offloading ratios of the corresponding vehicle under the constraint conditions and the association relationship between the vehicle and the drone corresponding to each computing task offloading ratio.

[0091] For example, in the embodiment of the present application, in time slot n, in order to minimize the system delay, the decision variables need to be adjusted to adapt to the dynamic time-varying nature of the drone network, and the data center needs to adjust the proportion β of the car's own calculation k [n] and the connection relationship between drones and vehicles The action space can be expressed as follows:

[0092]

[0093] Step S2043, utilizes the state space of different vehicles in each time slot, the action space of each vehicle, the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, the computing task volume information generated by each vehicle, the computing power information of each vehicle, and the computing power information of each drone to iteratively solve the objective function and obtain a solution.

[0094] For example, in the embodiment of the present application, when solving the objective function, transmission and calculation operations are performed according to the state s and action a of the entire system in each time slot n. The system will give rewards according to the current state and the selected action. The goal of the reward is to reduce the delay of the entire system. Therefore, the reward function can be designed according to the objective function, as shown in the following formula (17):

[0095] r(s,a)=-T (17)

[0096] Among them, r(s,a) represents the reward function, and T represents the total system delay;

[0097] The PPO algorithm consists of an Actor neural network and a Critic neural network. It is a reinforcement learning algorithm in which an intelligent agent follows the current strategy to select actions and updates the strategy based on these experiences, that is, a policy-based reinforcement learning algorithm. However, the update strategy of the PPO algorithm is relatively conservative compared to other reinforcement learning algorithms. Therefore, this algorithm needs to use the ratio between the current strategy and the previous strategy to measure the change in strategy. The strategy ratio is controlled within the range of (1+ε, 1-ε). ε is a hyperparameter that can be set to 0.1. 1+ε approaches 0, which means that the PPO algorithm removes strategies that are too far away from the previous strategy. Assuming there is a function f(x) and distributions p(x) and q(x), the importance sampling is defined as follows (18):

[0098]

[0099] in, represents the importance weight to balance the gap between the two distributions, E x～p [f(x)] represents the importance sampling result;

[0100] The optimization problem solved by the PPO algorithm is that the objective function needs to be maximized under the constraint of the strategy update size, as shown in the following formula (19):

[0101]

[0102] in, It indicates that the empirical average is used to estimate the mean between multiple sampled samples. The constraint condition is to limit the information divergence of the new and old strategies to a certain range so that the update will not diverge too much. θ is the neural network parameter, represents the estimator of the advantage function at time t, δ represents the maximum value of the sample mean, and KL represents the KL divergence; for the optimization problem (14), based on the existing strategy π θ And the strategy before the update And use KL divergence to calculate the previous strategy and the current strategy π θ , and the state s at this time t and action a t , the restriction condition of formula (18) can explain the distance between the two strategy distributions. Therefore, we first make a linear approximation to the optimization objective and a quadratic approximation to the restriction condition, and then use the conjugate gradient algorithm to efficiently solve it. The optimization objective can be shown in the following formula (20):

[0103]

[0104] Here the PPO algorithm uses the clip method and defines r t (θ) is the probability ratio Therefore, t (θ old )=1. The PPO algorithm maximizes a proxy objective, as shown in the following equation (21):

[0105]

[0106] The CPI here refers to conservative policy iteration. If it is not restricted, each step will produce a very large policy update, so it is necessary to limit this ratio to close to 1.

[0107] The process of solving the objective function can be shown in Table 1.

[0108] Step S2044, based on the solution result, determine the target association relationship between each vehicle and different drones and the target calculation task offloading ratio of each vehicle.

[0109] Exemplarily, based on the solution results, each vehicle is controlled to offload the task to the associated drone for calculation.

[0110] Table 1

[0111]

[0112] Step S205: Based on the target association information between each vehicle and different drones and the target computing task unloading ratio, the corresponding vehicle is controlled to unload the computing task. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0113] In some optional implementations, the first data transmission rates respectively corresponding to each vehicle and different drones are determined by the following steps:

[0114] Step a1, obtain the flight altitude information of each UAV, the location information of each UAV, the location information of each vehicle, the channel power gain, the Rayleigh fading coefficient corresponding to each vehicle and different UAVs, the communication power corresponding to each vehicle and different UAVs, the Gaussian white noise power and the channel bandwidth.

[0115] For example, in the embodiment of the present application, with the data center as the origin, for a three-dimensional Cartesian coordinate system, the coordinates of the drone m in the time slot n are shown in the following formula (22):

[0116]

[0117] Where H represents the flight altitude of the drone.

[0118] The coordinates of vehicle m in time slot n are shown in equation (23):

[0119]

[0120] Step a2, determining the free space fading coefficients respectively corresponding to the corresponding vehicle and different drones based on the flight altitude information of each drone, the location information of each drone, the location information of the corresponding vehicle, and the channel power gain.

[0121] For example, in the embodiment of the present application, the free space fading coefficient between the vehicle k and the drone m can be determined by the following formula (24):

[0122]

[0123] in, represents the free space fading coefficient from vehicle k to UAV m in time slot n, H represents the flight altitude of the UAV, and γ 0 Indicates the channel power gain at the reference distance.

[0124] Step a3, determining the channel gains respectively corresponding to the corresponding vehicles and the different UAVs according to the free space fading coefficients and Rayleigh fading coefficients respectively corresponding to the corresponding vehicles and the different UAVs.

[0125] For example, in the embodiment of the present application, a quasi-static block fading channel model is adopted between the vehicle and the UAV, which means that the channel remains unchanged within the block and changes in different blocks. The ground mobile device and the UAV will have a strong channel loss link, and since the UAV has a small-scale fading due to scattering at a certain height, the channel gain between the vehicle k and the UAV m is determined by the following formula (25):

[0126]

[0127] in, represents the free space fading coefficient from vehicle k to drone m in time slot n, is the Rayleigh fading coefficient from vehicle k to drone m in time slot n, which is determined by the following formula (26);

[0128]

[0129] where ψ is the deterministic line-of-sight component with modulus 1, It is represented as a circularly symmetric complex Gaussian random variable with zero mean and unit variance and random scattered components. is the Rayleigh factor.

[0130] Step a4, determining the signal-to-noise ratios between the corresponding vehicles and the different drones according to the communication power, channel gain and Gaussian white noise power respectively corresponding to each vehicle and the different drones.

[0131] For example, in the embodiment of the present application, the signal-to-noise ratio between vehicle k and drone m can be determined by the following formula (27):

[0132]

[0133] in, represents the signal-to-noise ratio between vehicle k and drone m in time slot n, represents the communication power between vehicle k and drone m in time slot n, σ 2 represents the Gaussian white noise power.

[0134] Step a4, based on the signal-to-noise ratio and channel bandwidth respectively corresponding to each vehicle and different drones, determining the first data transmission rate respectively corresponding to each vehicle and different drones.

[0135] Exemplarily, in the embodiment of the present application, the first data transmission rate between vehicle k and drone m is determined by the following formula (28):

[0136]

[0137] Among them, B represents the channel bandwidth, and the meanings of other variables are not repeated here.

[0138] In some optional implementations, the second data transmission rate between each drone and the data center is determined by the following steps:

[0139] Step b1, obtaining the communication power between each drone and the data center.

[0140] For example, in the embodiments of the present application, it is defined is the communication power between UAV m and data center O.

[0141] Step b2: determining the channel gain between the corresponding drone and the data center based on the location information, flight altitude, and channel power gain of each drone.

[0142] For example, in the embodiment of the present application, the wireless channel model between the drone and the data center adopts a free path loss model, and the channel gain between each drone and the data center is determined by the following formula (29):

[0143]

[0144] in, represents the channel gain from UAV m to data center O in time slot n.

[0145] Step b3, determining a second data transmission rate between the corresponding drone and the data center according to the channel gain, communication power, the Gaussian white noise power and the channel bandwidth between each drone and the data center.

[0146] Exemplarily, in the embodiment of the present application, the second data transmission rate between each drone and the data center is determined by the following formula (30).

[0147]

[0148] The following is an explanation of the vehicle networking computing task offloading method based on digital twin provided by the present invention through a specific embodiment.

[0149] Example:

[0150] In this embodiment, the python and pytorch frameworks are used to simulate the computing task offloading scheme of the multi-UAV Internet of Vehicles system. Considering that two UAVs and four vehicles assigned with MEC servers are distributed in the area, the flight altitude of the UAV is 200 meters. In any time slot, the communication power of the UAV, and other parameter configurations are shown in Table 2.

[0151] Table 2

[0152]

[0153] In order to verify the superiority of the PPO algorithm provided by this embodiment, the proposed PPO algorithm is compared with the greedy algorithm (greedy) and the policy gradient algorithm (Policy Gradient, PG) in experiments.

[0154] Comparing the convergence of PPO and PG algorithms, the larger the reward, the lower the task processing delay. Regardless of whether the PPO algorithm or the PG algorithm is used, the rewards obtained tend to converge. Based on the comparison results, it can be determined that the reward for the convergence of the PPO algorithm is better than the reward for the convergence of the PG algorithm. Since this method runs in the data center, the data center allocates the optimal offloading plan to drones and vehicles, which can ensure the online operation of the offloading plan and the availability of the model, meeting the requirements of minimizing delay.

[0155] Figure 3 The changes in delay between the proposed scheme and the comparative scheme under different power conditions when the digital twin error rate is 5% are compared. Under different numbers of drones, the system delay of the proposed algorithm is smaller than the system delay of the comparative algorithm. As the power of the drone increases, the average delay of the system tends to gradually decrease. When the number of vehicles is 2 and the power is 0.1w, the delay of the proposed scheme is 1.12 times the delay calculated by the policy gradient algorithm and 1.21 times the delay calculated by the greedy algorithm. When the number of drones is 4 and the power is 0.1w, the calculation delay of the proposed scheme is 1.07 times that of the policy gradient algorithm. These results prove that the proposed algorithm is superior to the comparative algorithm and has a greater advantage in delay optimization.

[0156] Figure 4 The effects of the computing task size and the error rate between the digital twin and the entity on the delay are analyzed. The offloading method we proposed has better performance than the PG algorithm. When the task volume range is constant, the error of the digital twin is positively correlated with the delay. A larger error rate means that the estimated delay of the system will be higher than the actual delay, so the actual delay of the system will be less than the estimated value. On the other hand, when the digital twin error remains unchanged, as the task volume increases, the corresponding delay will also increase.

[0157] In this embodiment, a vehicle networking computing task unloading device based on digital twin is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0158] This embodiment provides a vehicle networking computing task offloading device based on digital twins, such as Figure 5 As shown, including:

[0159] The first construction module 501 is used to construct digital twin models of multiple drones and multiple vehicles;

[0160] The simulation module 502 is used to simulate the operating status of each UAV and each vehicle using the digital twin model to obtain twin data of multiple time slots;

[0161] The second construction module 503 is used to construct an objective function and constraint conditions based on the twin data of multiple time slots. The objective function is constructed with the goal of minimizing the total system delay. The constraint conditions include first constraint information of the association relationship between each vehicle and different drones, second constraint information of the number of drones associated with each vehicle, and third constraint information of the task offloading ratio of each vehicle. The objective function is used to characterize the association relationship between each vehicle and different drones, the correlation between the task offloading ratio of each vehicle and the total system delay;

[0162] A solution module 504 is used to solve the objective function by using the twin data of multiple time slots, the first constraint information, the second constraint information, the third constraint information and the preset reinforcement learning model to obtain the target association relationship between each vehicle and different drones and the target calculation task unloading ratio of each vehicle;

[0163] The control module 505 is used to control the corresponding vehicle to unload the computing task based on the target association information between each vehicle and different drones and the target computing task unloading ratio.

[0164] In some optional embodiments, the twin data includes the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, the computing task volume information generated by each vehicle, the computing power information of each vehicle, and the computing power information of each drone.

[0165] In some optional embodiments, the total system delay is used to characterize the sum of the computing delay and the transmission delay. The computing delay is determined by the computing task volume information generated by each vehicle, the computing power information of each vehicle, the computing power information of each drone, and the computing task unloading ratio of each vehicle. The transmission delay is determined by the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, and the computing task unloading ratio of each vehicle.

[0166] In some optional implementations, the control module 505 includes:

[0167] The first determination submodule is used to determine the state space of each vehicle in the corresponding time slot based on the computing task volume information generated by different vehicles in each time slot, the channel gain between each vehicle and different drones, and the channel gain between each drone and the data center;

[0168] A second determination submodule is used to determine the action space of each vehicle based on the first constraint information, the second constraint information and the third constraint information, wherein the action space of each vehicle is used to characterize the unloading ratios of multiple computing tasks of the corresponding vehicle under the constraint conditions and the association relationship between the vehicle and the drone corresponding to each computing task unloading ratio;

[0169] A solution submodule is used to iteratively solve the objective function by using the state space of different vehicles in each time slot, the action space of each vehicle, the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, the computing task volume information generated by each vehicle, the computing capacity information of each vehicle, and the computing capacity information of each drone to obtain a solution result;

[0170] The third determination submodule is used to determine the target association relationship between each vehicle and different drones and the target calculation task offloading ratio of each vehicle based on the solution result.

[0171] In some optional embodiments, the computing delay includes a first computing delay of each vehicle itself and a second computing delay for each vehicle to offload computing tasks to a drone for computing; the transmission delay includes a first transmission delay for each vehicle to offload computing tasks and transmit them to an associated drone, a second transmission delay for the drone to upload computing task information to a data center, and a third transmission delay for the data center to send decision information; the first computing delay is determined by the computing task volume information, computing power information, and computing task offloading ratio generated by the corresponding vehicle; the second computing delay is determined by the computing power information, the computing task volume information, and the computing task offloading ratio generated by the corresponding drone; the first transmission delay is determined by the first data transmission rate between the corresponding vehicle and different drones and the computing task offloading ratio of the corresponding vehicle; the second transmission delay is determined by the second data transmission rate between the corresponding drone and the data center and the size of the computing task information of the corresponding drone.

[0172] In some optional implementations, the first data transmission rates respectively corresponding to each vehicle and different drones are determined by the following steps:

[0173] Obtain the flight altitude information of each UAV, the location information of each UAV, the location information of each vehicle, the channel power gain, the Rayleigh fading coefficients corresponding to each vehicle and different UAVs, the communication power corresponding to each vehicle and different UAVs, the Gaussian white noise power and the channel bandwidth;

[0174] According to the flight altitude information of each UAV, the position information of each UAV, the position information of the corresponding vehicle and the channel power gain, the free space fading coefficients corresponding to the corresponding vehicle and different UAVs are determined;

[0175] According to the free space fading coefficient and Rayleigh fading coefficient corresponding to each vehicle and different UAVs, the channel gains corresponding to each vehicle and different UAVs are determined;

[0176] According to the communication power, channel gain and Gaussian white noise power respectively corresponding to each vehicle and different UAVs, the signal-to-noise ratios respectively corresponding to the corresponding vehicles and different UAVs are determined;

[0177] Based on the signal-to-noise ratio and the channel bandwidth respectively corresponding to each vehicle and different drones, the first data transmission rates respectively corresponding to the corresponding vehicle and different drones are determined.

[0178] In some optional implementations, the second data transmission rate between each drone and the data center is determined by the following steps:

[0179] Obtain the communication power between the drone and the data center;

[0180] Determine the channel gain between the corresponding drone and the data center according to the location information, flight altitude and channel power gain of each drone;

[0181] Determine a second data transmission rate between the corresponding drone and the data center according to the channel gain, communication power, Gaussian white noise power and channel bandwidth between each drone and the data center

[0182] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0183] The digital twin-based Internet of Vehicles computing task offloading device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0184] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for offloading computing tasks of Internet of Vehicles based on digital twins, characterized in that: Applied to a data center, the data center is communicatively connected to a plurality of drones, each drone is communicatively connected to a plurality of vehicles, the method comprises: Build digital twin models of multiple drones and multiple vehicles; The digital twin model is used to simulate the operating status of each UAV and each vehicle to obtain twin data of multiple time slots; An objective function and constraint conditions are constructed based on the twin data of the multiple time slots. The objective function is constructed with the goal of minimizing the total system delay. The constraint conditions include first constraint information on the association relationship between each vehicle and different drones, second constraint information on the number of drones associated with each vehicle, and third constraint information on the task unloading ratio of each vehicle. The objective function is used to characterize the association relationship between each vehicle and different drones, the correlation between the task unloading ratio of each vehicle and the total system delay. The constraint conditions are shown in the following formula: in, Indicates vehicle k and drones m Is there any correlation between Indicates vehicle k The proportion of computing tasks itself, Indicates vehicle k The ratio of offloading computing tasks to associated drones, C1 represents the first constraint information, C2 represents the second constraint information, and C3 represents the third constraint information; The objective function is solved by using the twin data of the multiple time slots, the first constraint information, the second constraint information, the third constraint information, and a preset reinforcement learning model to obtain a target association relationship between each vehicle and different drones and a target calculation task offloading ratio of each vehicle; Based on the target association information between each vehicle and different drones and the target computing task unloading ratio, the corresponding vehicle is controlled to unload the computing task; The twin data includes the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, the computing task volume information generated by each vehicle, the computing power information of each vehicle, and the computing power information of each drone. The computing power information of each vehicle includes the estimated computing power of the corresponding vehicle and the deviation value between the actual computing power and the estimated computing power of the vehicle stored in the digital twin. The computing power information of each drone includes the estimated computing power of the corresponding drone and the deviation value between the actual computing power and the estimated computing power of the drone stored in the digital twin. The total system delay is used to characterize the sum of the calculation delay and the transmission delay. The calculation delay is determined by the calculation task amount information generated by each vehicle, the calculation capacity information of each vehicle, the calculation capacity information of each drone, and the calculation task unloading ratio of each vehicle. The transmission delay is determined by the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, and the calculation task unloading ratio of each vehicle. The objective function is , Determined by the following formula: in, represents the total system delay, Indicates time slot n The computational delay within Indicates the calculation time slot n The transmission delay within Determined by the following formula: in, Indicates time slot n Inside vehicle k Offloading computing tasks to drones n The first calculation delay for calculation, Indicates time slot n Inside car k The second computing delay of the drone itself is M , the number of vehicles is K， It is expressed as: in, Indicates time slot n Inside vehicle k Offloading computing tasks to drones m The calculated value corresponding to the calculation delay, Indicates time slot n Inside vehicle k Offloading to drone m The error between the calculated value and the true value of the corresponding calculation delay, Determined by the following formula: in, Indicates vehicle k The proportion of computing tasks itself, Indicates vehicle k In time slot n The amount of computing task data generated within Indicates the number of CPU cycles required to complete a 1-bit computation task. Indicates time slot n Internal drone m The estimated computing power of Indicates vehicle k and drones m Is there any correlation between When the vehicle k and drones m The drones can communicate with each other. m Receiving vehicles k Offloaded computing tasks, when When the vehicle k and drones m No communication is possible between them; Determined by the following formula: in, Indicates time slot n Internal drone m The real computing power and storage of drones in digital twins m The deviation value between the estimated computing power of Determined by the following formula: in, Indicates time slot n Inside car k The calculated value of the calculation delay, Indicates time slot n Inside car k The deviation between the actual value and the calculated value of the self-calculated delay, It is expressed as: in, Indicates time slot n Inside vehicle k Estimated computing power; It is expressed as: in, Indicates time slot n Inside vehicle k The real computing power and storage of the vehicle in the digital twin k The deviation value between the estimated computing power of Determined by the following formula: in, Indicates vehicle k Offload computing tasks to associated drones m The first transmission delay, Indicates drone m Upload computing task information to the data center The second transmission delay generated is Indicates time slot n The time when the internal data center issues the decision, Determined by the following formula: in, Indicates vehicle k With drones m a first data transmission rate between; Determined by the following formula: in, Indicates drone m Upload task information to the data center The data transfer rate during the process, Indicates that the associated vehicle is uploaded to the drone m The computing task information size, the relationship between the drone and the vehicle is , the ratio of vehicle-to-UAV task offloading is ; The step of solving the objective function by using the twin data, the first constraint information, the second constraint information, the third constraint information and the preset reinforcement learning model to obtain the target association relationship between each vehicle and different drones and the target calculation task offloading ratio of each vehicle includes: The state space of each vehicle in the corresponding time slot is determined based on the computing task information generated by different vehicles in each time slot, the channel gain between each vehicle and different drones, and the channel gain between each drone and the data center; Determine the action space of each vehicle based on the first constraint information, the second constraint information and the third constraint information, wherein the action space of each vehicle is used to represent the multiple computing task offloading ratios of the corresponding vehicle under the constraint conditions and the association relationship between the vehicle and the drone corresponding to each computing task offloading ratio; The objective function is iteratively solved by using the state space of different vehicles in each time slot, the action space of each vehicle, the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, the computing task volume information generated by each vehicle, the computing capacity information of each vehicle, and the computing capacity information of each drone to obtain a solution result; Determine the target association relationship between each vehicle and different drones and the target calculation task offloading ratio of each vehicle based on the solution result; The state space of different vehicles in each time slot is expressed as: in, represents the state space, and represents the channel gain, Indicates vehicle k The computational tasks represents a collection of drones, represents a collection of vehicles; The action space of each vehicle is expressed as: in, represents the action space, Indicates vehicle k The proportion of self-calculation, Indicates vehicle k and drones m Is there any correlation between them? The step of iteratively solving the objective function by using the state space of different vehicles in each time slot, the action space of each vehicle, the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, the computing task volume information generated by each vehicle, the computing capacity information of each vehicle, and the computing capacity information of each drone to obtain the solution result includes: The objective function is iteratively solved by utilizing the state space of different vehicles in each time slot, the action space of each vehicle, the first data transmission rate corresponding to each vehicle and different drones, the second data transmission rate between each drone and the data center, the computing task volume information generated by each vehicle, the computing power information of each vehicle, the computing power information of each drone, the reward function and the PPO algorithm to obtain a solution result. The reward function is established with the goal of reducing the delay of the entire system. The reward function is: in, and Represents each time slot separately n The state and actions of the entire system within represents the reward function, Indicates the total system delay; The PPO algorithm consists of an Actor neural network and a Critic neural network. The PPO algorithm solves the objective function to maximize the optimization objective under the constraint of the strategy update size. The optimization objective is expressed as: ， in, It indicates that the empirical average is used to estimate the mean between multiple sampled samples. The constraint condition aims to limit the information divergence of the new and old strategies to a certain range so that the update will not diverge too much. are neural network parameters, represents the advantage function in time t The estimated amount, represents the maximum value of the sample mean, and KL represents the KL divergence; Indicates the policy before update. Indicates the existing strategy, and Indicates time t status and actions; Make a linear approximation of the optimization objective, make a quadratic approximation of the constraints, and then use the conjugate gradient algorithm to efficiently solve it. The optimization objective can be expressed as follows: The PPO algorithm uses the clip method and defines Probability ratio ,therefore , the PPO algorithm maximizes a proxy objective as shown below: Among them, CPI refers to conservative strategy iteration; The process of solving the objective function includes: Initialize Actor network parameters, Critic network parameters, and training times , maximum time step 1000, the computing power of the car , the computing power of drones ; Initialization state s ; According to the status s Get Action a , the computational latency of the computing system and transmission delay , calculate the total delay of the system , calculate the reward , update status s , update the Actor network and Critic network parameters until the maximum number of training times is reached.

2. The method according to claim 1, characterized in that The calculation delay includes the first calculation delay of each vehicle itself and the second calculation delay of each vehicle unloading the calculation task to the drone for calculation. The transmission delay includes the first transmission delay of each vehicle unloading the calculation task and transmitting it to the associated drone, the second transmission delay generated by the drone uploading the calculation task information to the data center, and the third transmission delay of the data center sending the decision information. The first calculation delay is determined by the calculation task amount information, calculation capacity information and calculation task unloading ratio generated by the corresponding vehicle. The second calculation delay is determined by the calculation capacity information of the corresponding drone, the calculation task amount information and the calculation task unloading ratio generated by the corresponding vehicle. The first transmission delay is determined by the first data transmission rate between the corresponding vehicle and different drones and the calculation task unloading ratio of the corresponding vehicle. The second transmission delay is determined by the second data transmission rate between the corresponding drone and the data center and the size of the calculation task information of the corresponding drone.

3. The method according to claim 1, characterized in that The first data transmission rates respectively corresponding to each vehicle and different drones are determined by the following steps: Obtain the flight altitude information of each UAV, the location information of each UAV, the location information of each vehicle, the channel power gain, the Rayleigh fading coefficients corresponding to each vehicle and different UAVs, the communication power corresponding to each vehicle and different UAVs, the Gaussian white noise power and the channel bandwidth; Determine the free space fading coefficients corresponding to the corresponding vehicle and different drones according to the flight altitude information of each drone, the location information of each drone, the location information of the corresponding vehicle, and the channel power gain; According to the free space fading coefficient and Rayleigh fading coefficient corresponding to each vehicle and different UAVs, the channel gains corresponding to each vehicle and different UAVs are determined; Determine the signal-to-noise ratios between the corresponding vehicles and the different drones according to the communication powers, channel gains, and Gaussian white noise powers respectively corresponding to the communication powers between the vehicles and the different drones; Based on the signal-to-noise ratio and the channel bandwidth respectively corresponding to each vehicle and different drones, the first data transmission rates respectively corresponding to the corresponding vehicle and different drones are determined.

4. The method according to claim 3, characterized in that The second data transmission rate between each drone and the data center is determined by the following steps: Obtain the communication power between the drone and the data center; Determine the channel gain between the corresponding drone and the data center according to the location information, flight altitude and channel power gain of each drone; The second data transmission rate between the corresponding drone and the data center is determined according to the channel gain, communication power, the Gaussian white noise power and the channel bandwidth between each drone and the data center.

Images