Vehicle MEC task unloading and resource allocation optimization method and system

Abstract

The invention relates to a vehicle MEC task unloading and resource allocation optimization method and system. The method comprises the following steps: establishing a road scene in which an unmanned aerial vehicle assists in vehicle edge calculation; defining a resource matrix and time delay according to the road scene; comprehensively proposing an optimization problem of minimizing task processing time delay based on a task unloading mode; describing a task unloading decision process through an MDP model; communication links between the network connection vehicle and the road side unit and the unmanned aerial vehicle in the communication coverage range where the network connection vehicle is located are established, and the total time delay of the calculation task and the size of cache resources are calculated; generating a task unloading decision and a resource allocation strategy by using a DDPG algorithm; and optimizing a task unloading decision and a resource allocation strategy in real time to obtain an optimal unloading decision and an optimal resource allocation strategy. Compared with the prior art, the method has the advantages that the resource utilization rate is increased, the system time delay is reduced, the method adapts to complex scenes of vehicle high-speed movement and network load dynamic change, and continuous and reliable calculation service is provided for time delay sensitive vehicle-mounted application.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation systems, and more particularly to a method and system for optimizing vehicle MEC task offloading and resource allocation. Background Art

[0002] In recent years, the Internet of Vehicles (IoV), as a key component of intelligent transportation systems, has received extensive attention. The IoV connects vehicles, infrastructure, and pedestrians through wireless communication technologies to achieve information sharing and transmission, improving traffic safety and efficiency. With the development of technology, applications such as autonomous driving, intelligent traffic management, and in-vehicle entertainment services have put forward higher requirements for the IoV. These applications need to process a large amount of real-time data, most of which are computationally intensive and latency-sensitive, posing severe challenges to the computing power and communication network of the system.

[0003] Traditional cloud computing architectures usually rely on remote data centers for data processing, but this mode is difficult to meet the requirements of the IoV for low latency and high reliability. Especially in an environment with complex network conditions and dynamically changing user demands, the latency caused by long-distance data transmission may affect the system performance and user experience. Therefore, Multi-Access Edge Computing (MEC), as an emerging computing paradigm, provides an effective solution by sinking computing resources to the network edge. MEC can process data closer to the vehicle, significantly reducing latency and improving real-time performance and reliability.

[0004] However, the traditional IoV MEC system architecture still has many limitations. First, the coverage of ground infrastructure is limited, especially in remote areas or regions with complex terrain, where the availability and stability of network signals are insufficient to meet the requirements of wide-area coverage. Second, traditional fixed MEC nodes lack flexibility and cannot quickly respond and adjust to cope with emergencies or temporarily increased traffic demands. In addition, the deployment and maintenance costs of the fixed architecture are high, and it is difficult to flexibly expand or reconfigure to adapt to the dynamically changing network environment. These limitations restrict the efficiency and adaptability of traditional MEC systems in practical applications.

[0005] Meanwhile, how to efficiently perform task offloading and resource allocation in a dynamic and complex environment has become a key issue. Most existing studies adopt traditional mathematical optimization and heuristic algorithms. Although these methods perform well in specific scenarios, they are insufficient in adaptability when dealing with the vehicle networking scenarios with real-time changes and strong uncertainties. For example, patent application CN117336750A discloses a computing offloading method and system based on service caching technology in a vehicle networking edge computing architecture. This method establishes a network topology of the vehicle networking architecture based on service caching technology, and constructs a problem model of the vehicle networking architecture based on the network topology of the vehicle networking architecture; constructs a road link model by using the movement trajectory information of vehicles, and pre-deploys service caches by using a service cache deployment algorithm based on the road link model; for each real-time computing task in the network topology of the vehicle networking architecture, uses a greedy strategy-based computing offloading algorithm for service caching to solve the problem model of the vehicle networking architecture and obtain the forwarding path of the computing task. This method, based on the caching deployment algorithm of the greedy strategy and the static road model, cannot adapt to the real-time changing network load and vehicle trajectories. Using the greedy strategy may fall into a local optimum, and it is difficult to balance the quality of service (QoS) and fairness, especially in complex scenarios. Moreover, this method depends on the coverage range of fixed RSUs, and vehicles may experience service interruptions due to signal attenuation in the edge area or at the junction of RSUs. Summary of the Invention

[0006] The purpose of the present invention is to overcome the above-mentioned defects existing in the prior art and provide a method and system for optimizing vehicle MEC task offloading and resource allocation, which improves resource utilization rate, reduces system latency, adapts to complex scenarios of high-speed vehicle movement and dynamic network load changes, and provides continuous and reliable computing services for latency-sensitive in-vehicle applications.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A method for optimizing vehicle MEC task offloading and resource allocation includes the following steps:

[0009] Establish a road scenario for drone-assisted vehicle edge computing, where the number of lanes in the road scenario for drone-assisted vehicle edge computing is fixed, roadside units are arranged beside the lanes, the drones fly at a constant speed under the coverage area of the roadside units and provide resource access for vehicles, and mobile edge computing servers are deployed on the roadside units and drones;

[0010] Define the transmission delay, computing delay of vehicles, drones and roadside units, and the vehicle-drone movement mode according to the road scenario for drone-assisted vehicle edge computing, and define a multi-dimensional resource matrix of the roadside unit and the drone, where the multi-dimensional resource matrix includes computing resources, caching resources and spectrum resources;

[0011] Based on three offloading modes: executing computing tasks locally on the vehicle, offloading computing tasks to the mobile edge computing server on the roadside unit, and offloading computing tasks to the mobile edge computing server on the unmanned aerial vehicle, an optimization problem of minimizing the task processing delay with the coordination of computing resources, cache resources, and spectrum resources is comprehensively proposed;

[0012] Describe the task offloading decision-making process through a Markov decision process model, and define the multi-dimensional state space, action space, and reward function of the Markov decision process model;

[0013] Establish communication links between the vehicle and the roadside unit and the unmanned aerial vehicle within its communication coverage respectively, and calculate the total delay of the computing task and the size of the cache resources;

[0014] Use the deep deterministic policy gradient algorithm to construct an Actor-Critic network, and guide the Actor-Critic network to achieve the goal of minimizing the task processing delay through the reward function, and input the multi-dimensional state space and action space to generate a task offloading decision and resource allocation strategy;

[0015] Sample and train the parameters of the Actor-Critic network through an experience replay pool, and optimize the task offloading decision and resource allocation strategy in real time to obtain the optimal offloading decision and resource allocation strategy.

[0016] Furthermore, after the vehicle generates a computing task, it sends a resource access request to the mobile edge computing server. The mobile edge computing server makes a vehicle association and resource allocation decision based on the received resource access request and returns it to the vehicle. The resource access request includes the detailed information of the computing task and the driving state of the vehicle. The vehicle association and resource allocation decision includes performing calculations locally on the vehicle, offloading the computing task to the mobile edge computing server on the roadside unit, or offloading the computing task to the mobile edge computing server on the unmanned aerial vehicle, as well as the proportions of spectrum resources, computing resources, and cache resources allocated to the vehicle.

[0017] Furthermore, the computing tasks generated by the vehicle are:

[0018] D i =(c i ,d i ,T i max )

[0019] where D i is the computing task of the i-th vehicle, c i is the number of CPU cycles required for the i-th vehicle to complete the computing task, d i is the data size of the computing task generated by the i-th vehicle, and T imax The maximum latency allowed for the computing task generated for the i-th vehicle.

[0020] Further, after receiving the vehicle association and resource allocation decision, the vehicle offloads the task to the associated mobile edge computing server according to the ratios of the spectrum resources, computing resources, and caching resources allocated to the vehicle.

[0021] Further, when the vehicle is located at the junction of the coverage areas of two roadside units or at the edge of the roadside unit coverage area, the computing task is offloaded to the mobile edge computing server on the unmanned aerial vehicle.

[0022] Further, the optimization problem of minimizing the task processing latency is:

[0023]

[0024]

[0025] In the formula, H is the unit step function, T i is the total latency of the computing task, T i max is the maximum latency allowed for the computing task, is the caching resource allocated by the MEC server to the i-th vehicle, d i is the size of the computing task, is the decision parameter for the vehicle to locally execute the computing task offloading, is the decision parameter for offloading the computing task to the roadside unit, is the decision parameter for offloading the computing task to the unmanned aerial vehicle, is the ratio of the spectrum resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle, is the ratio of the computing resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle, is the ratio of the caching resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle, is the ratio of the spectrum resources allocated by the mobile edge computing server deployed on the u-th unmanned aerial vehicle to the i-th vehicle, is the ratio of the computing resources allocated by the mobile edge computing server deployed on the u-th unmanned aerial vehicle to the i-th vehicle, is the ratio of the caching resources allocated by the mobile edge computing server deployed on the u-th unmanned aerial vehicle to the i-th vehicle.

[0026] Further, when performing a computing task locally on the vehicle, if the cache resources provided locally by the vehicle are equal to the size of the computing task and the computing task needs to be offloaded to the mobile edge computing server, the cache resources allocated by the mobile edge computing server to the vehicle are greater than the size of the computing task generated by this vehicle. The cache resources allocated by the mobile edge computing server to the vehicle are:

[0027]

[0028] In the formula, is the cache resource of the mobile edge computing server deployed on the m-th roadside unit, is the cache resource of the mobile edge computing server deployed on the u-th unmanned aerial vehicle.

[0029] Further, the total delay of the computing task is:

[0030]

[0031] In the formula, T i is the total delay of the computing task, is the offloading decision parameter for the vehicle to perform the computing task locally. T i loc is the delay caused by the vehicle performing the computing task locally, is the offloading decision parameter for offloading the computing task to the roadside unit. T i m is the delay generated by offloading the computing task to the mobile edge computing server on the roadside unit, is the offloading decision parameter for offloading the computing task to the unmanned aerial vehicle. T i u is the delay generated by offloading the computing task to the mobile edge computing server on the unmanned aerial vehicle.

[0032] Further, the delay caused by the vehicle performing the computing task locally is:

[0033]

[0034] In the formula, c i is the number of CPU cycles required to complete the computing task generated by the i-th vehicle. Co i is the local computing power of the i-th vehicle;

[0035] The delay generated by offloading the computing task to the mobile edge computing server on the roadside unit is:

[0036]

[0037] In the formula, The computing delay generated by the mobile edge computing server for offloading computing tasks to the m-th roadside unit The transmission delay generated by the mobile edge computing server for offloading computing tasks to the m-th roadside unit The proportion of computing resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle The computing resources of the mobile edge computing server deployed on the m-th roadside unit, d i The data size of the computing tasks generated by the i-th vehicle, r i m The maximum data transmission rate between the i-th vehicle and the mobile edge computing server deployed on the m-th roadside unit;

[0038] The delay generated by the mobile edge computing server for offloading computing tasks to the unmanned aerial vehicle is:

[0039]

[0040] In the formula, The computing delay generated by the mobile edge computing server for offloading computing tasks to the u-th unmanned aerial vehicle The transmission delay generated by the mobile edge computing server for offloading computing tasks to the u-th unmanned aerial vehicle The proportion of computing resources allocated by the mobile edge computing server deployed on the u-th unmanned aerial vehicle to the i-th vehicle The computing resources of the mobile edge computing server deployed on the u-th unmanned aerial vehicle, r i u The maximum data transmission rate between the i-th vehicle and the mobile edge computing server deployed on the u-th unmanned aerial vehicle.

[0041] According to another aspect of the present invention, there is provided a vehicle MEC task offloading and resource allocation optimization method system, including

[0042] A road scene establishment module for establishing a road scene of unmanned aerial vehicle-assisted vehicle edge computing. The number of lanes in the road scene of unmanned aerial vehicle-assisted vehicle edge computing is fixed. Roadside units are arranged beside the lanes. The unmanned aerial vehicle flies at a constant speed under the coverage area of the roadside unit and provides resource access for vehicles. Mobile edge computing servers are deployed on the roadside unit and the unmanned aerial vehicle;

[0043] A road scene definition module, which is used to define the transmission delay, computing delay, and vehicle-UAV motion mode of vehicles, UAVs, and roadside units according to the road scene of the vehicle-assisted vehicle edge computing, and define the multi-dimensional resource matrix of the roadside unit and the UAV, where the multi-dimensional resource matrix includes computing resources, caching resources, and spectrum resources;

[0044] An optimization problem formulation module, which comprehensively formulates an optimization problem of minimizing the task processing delay with the cooperation of the computing resources, caching resources, and spectrum resources based on three offloading modes: executing computing tasks locally on the vehicle, offloading computing tasks to the mobile edge computing server on the roadside unit, and offloading computing tasks to the mobile edge computing server on the UAV;

[0045] A Markov decision process model definition module, which is used to describe the task offloading decision process through a Markov decision process model, and define the multi-dimensional state space, action space, and reward function of the Markov decision process model;

[0046] A communication link establishment module, which is used to establish communication links between the vehicle and the roadside unit and the UAV within its communication coverage respectively, and calculate the total delay of the computing task and the size of the caching resources;

[0047] A task offloading decision and resource allocation strategy generation module, which is used to construct an Actor-Critic network by using the deep deterministic policy gradient algorithm, guide the Actor-Critic network to achieve the goal of minimizing the task processing delay through the reward function, and generate a task offloading decision and resource allocation strategy by inputting the multi-dimensional state space and action space;

[0048] A task offloading decision and resource allocation strategy optimization module, which is used to sample and train the Actor-Critic network parameters through an experience replay pool, and optimize the task offloading decision and resource allocation strategy in real time to obtain the optimal offloading decision and resource allocation strategy.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. The present invention introduces a UAV as a node for dynamic mobile edge computing, constructs a flexible edge computing network by combining the UAV and the roadside unit, models the multi-dimensional state space and action space through a Markov decision process, uses the deep deterministic policy gradient algorithm to generate a task offloading decision and resource allocation strategy in real time, dynamically optimizes the cooperation mechanism of three modes: local computing, roadside unit offloading, and UAV offloading, realizes the real-time optimization of the task offloading strategy and resource allocation, ensures the continuity and efficiency of the computing task during the vehicle movement, and can adapt to complex scenarios of high-speed vehicle movement and dynamic network load changes.

[0051] 2. For the vehicle tasks in the coverage edge or intersection area of the RSU, the present invention forcibly enables UAV offloading to avoid the delay fluctuation caused by signal attenuation. Meanwhile, it comprehensively manages spectrum, computing, and caching resources, restricts the total task processing delay and cache capacity, and guides the policy network through a reward function to achieve the goal of minimizing delay. At the same time, it minimizes both delay and energy consumption, significantly improves resource utilization efficiency, reduces task processing delay, and can provide continuous and reliable computing services for delay-sensitive vehicular applications. Description of the Drawings

[0052] Figure 1 It is a schematic flowchart of a method for optimizing vehicle MEC task offloading and resource allocation proposed by the present invention;

[0053] Figure 2 It is a schematic diagram of a road scene for UAV-assisted vehicle edge computing;

[0054] Figure 3 It is a schematic flowchart of the DDPG algorithm. Detailed Embodiment

[0055] The present invention will be described in detail below with reference to the drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives detailed implementation manners and specific operation processes, but the protection scope of the present invention is not limited to the following embodiments.

[0056] Involved English abbreviations:

[0057] Mobile edge computing: Mobile edge computing, MEC

[0058] Unmanned Aerial Vehicle: Unmanned Aerial Vehicle, UAV

[0059] Road Side Unit: Road Side Unit, RSU

[0060] Deep Deterministic Policy Gradient: Deep Deterministic Policy Gradient, DDPG

[0061] Embodiment 1

[0062] This embodiment provides a method for optimizing vehicle MEC task offloading and resource allocation, as Figure 1 shown, including the following steps:

[0063] S1. Establish a road scene for UAV-assisted vehicle edge computing, as Figure 2As shown in the figure, the number of lanes in the road scenario of UAV-assisted vehicle edge computing is fixed. Roadside units are set beside the lanes. The UAV flies at a constant speed under the coverage area of the roadside unit and provides resource access for vehicles. Mobile edge computing servers are deployed in the roadside unit and the UAV.

[0064] If it is necessary to offload the computing task to the MEC server, the vehicle first sends a resource access request to the roadside unit and UAV covering it. After receiving the access permission and resource allocation result of the corresponding MEC server, the computing task will be offloaded to the associated MEC server through the allocated spectrum resources. All computing tasks owned by each vehicle adopt a full offloading strategy. The connected vehicle can only choose to calculate the entire computing task locally by the vehicle or offload the entire computing task to the MEC server located at the roadside unit or the UAV for calculation. At the same time, the vehicle in the overlapping area between the roadside unit and the UAV where the MEC server is deployed can only be associated and offload its task to one of the MEC servers.

[0065] S2. Define the transmission delay, computing delay of the vehicle, UAV and roadside unit, and the vehicle-UAV motion mode according to the road scenario of UAV-assisted vehicle edge computing, and define the multi-dimensional resource matrix of the roadside unit and the UAV. The multi-dimensional resource matrix includes computing resources, cache resources and spectrum resources.

[0066] The road scenario of UAV-assisted vehicle edge computing includes several intelligent connected vehicles. The vehicles run unidirectionally in a straight line at a constant speed, and new vehicles are randomly generated. The entire road scenario of UAV-assisted vehicle edge computing works in discrete time with equal-length time slots. The computing tasks generated by the vehicles are:

[0067] D i =(c i ,d i ,T i max )

[0068] In the formula, D i is the computing task of the i-th vehicle, c i is the CPU cycles required for the i-th vehicle to complete the computing task, d i is the data size of the computing task generated by the i-th vehicle, and T i max is the maximum delay allowed for the computing task generated by the i-th vehicle.

[0069] After the vehicle generates a computing task, it sends a resource access request to the mobile edge computing server. The mobile edge computing server makes vehicle association and resource allocation decisions based on the received resource access request and returns them to the vehicle. The resource access request includes the detailed information of the computing task and the driving state of the vehicle. The vehicle association and resource allocation decisions include performing calculations locally on the vehicle, offloading the computing task to the mobile edge computing server on the roadside unit, or offloading the computing task to the mobile edge computing server on the unmanned aerial vehicle, as well as the proportions of spectrum resources, computing resources, and caching resources allocated to the vehicle. After receiving the vehicle association and resource allocation decisions, the vehicle offloads the task to the associated mobile edge computing server according to the proportions of spectrum resources, computing resources, and caching resources allocated to the vehicle.

[0070] The resource matrix of the MEC server deployed at the m-th roadside unit is expressed as:

[0071]

[0072] In the formula, is the spectrum resource of the mobile edge computing server deployed on the m-th roadside unit, is the computing resource of the mobile edge computing server deployed on the m-th roadside unit, is the caching resource of the mobile edge computing server deployed on the m-th roadside unit.

[0073] S3. Based on the three offloading modes of executing the computing task locally on the vehicle, offloading the computing task to the mobile edge computing server on the roadside unit, and offloading the computing task to the mobile edge computing server on the unmanned aerial vehicle, an optimization problem of minimizing the task processing delay with the coordination of computing resources, caching resources, and spectrum resources is comprehensively proposed.

[0074] An appropriate amount of caching resources needs to be pre-allocated for each resource access request so that all data related to the computing task can be cached for task processing. For the resource access request sent by the i-th vehicle, let be the proportion of the caching resources allocated to the MEC server located at the roadside unit, and let be the proportion of the caching resources allocated to the MEC server on the unmanned aerial vehicle. When the i-th vehicle offloads the task to the MEC server, it needs to satisfy:

[0075] 1. The total delay generated by the computing task is less than the maximum allowable delay of the computing task, that is:

[0076]

[0077] In the formula, T i is the total delay of the computing task, is the offloading decision parameter for the vehicle to execute the computing task locally, T i loc is the latency caused by the vehicle executing the computing task locally is the offloading decision parameter for offloading the computing task to the roadside unit, T i m is the latency generated by the mobile edge computing server when offloading the computing task to the roadside unit is the offloading decision parameter for offloading the computing task to the drone, T i u is the latency generated by the mobile edge computing server when offloading the computing task to the drone

[0078] 2. When the vehicle executes the computing task locally, if the cache resource provided by the vehicle locally is equal to the size of the computing task, and when the computing task needs to be offloaded to the mobile edge computing server, the cache resource allocated by the mobile edge computing server to the vehicle is greater than the size of the computing task generated by this vehicle. The cache resource allocated by the mobile edge computing server to the vehicle is:

[0079]

[0080] In the formula, is the cache resource of the mobile edge computing server deployed on the m-th roadside unit is the cache resource of the mobile edge computing server deployed on the u-th drone, d i is the data size of the computing task generated by the i-th vehicle

[0081] Since the resources available to the MEC server are limited, it is crucial to allocate them effectively. The purpose of the offloading decision is to minimize the total time consumption for completing the vehicle computing task

[0082] The optimization problem of minimizing the task processing latency is:

[0083]

[0084] In the formula, H is the unit step function, T i is the total latency of the computing task, T i max is the maximum latency allowed for the computing task is the cache resource allocated by the MEC server to the i-th vehicle, d i is the size of the computing task is the offloading decision parameter for the vehicle to execute the computing task locally is the offloading decision parameter for offloading the computing task to the roadside unit is the offloading decision parameter for offloading the computing task to the drone The proportion of spectrum resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle The proportion of computing resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle The proportion of cache resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle The proportion of spectrum resources allocated by the mobile edge computing server deployed on the u-th unmanned aerial vehicle to the i-th vehicle The proportion of computing resources allocated by the mobile edge computing server deployed on the u-th unmanned aerial vehicle to the i-th vehicle The proportion of cache resources allocated by the mobile edge computing server deployed on the u-th unmanned aerial vehicle to the i-th vehicle. T i ≤T i max Indicates that the delay generated by executing the computing task cannot be higher than the maximum delay allowed for the computing task Indicates that the cache resources occupied by the computing task must be greater than the size of the computing task generated by this user Indicates that for the offloading decision parameters, three 0-1 variables are used Indicates that the connected vehicle can only choose to perform the entire task locally or offload the entire task to the MEC server located at the roadside unit or unmanned aerial vehicle for computing Indicates that the proportion of all resources allocated by the MEC server to the i-th vehicle is less than 1 Indicates that the sum of the proportions of spectrum resources allocated by the MEC server located at the roadside unit (or on the unmanned aerial vehicle) to the i-th vehicle is 1. Indicates that the sum of the proportions of computing resources allocated by the MEC server located at the roadside unit (or on the unmanned aerial vehicle) to the i-th vehicle is 1. Indicates that the sum of the proportions of spectrum resources allocated by the MEC server located at the roadside unit (or on the unmanned aerial vehicle) to the i-th vehicle is 1

[0085] S4. Describe the task offloading decision process through the Markov decision process model, and define the multi-dimensional state space, action space, and reward function of the Markov decision process model

[0086] The multi-dimensional state space of the Markov decision process (MDP) model includes the following multi-dimensional features: the location information of all vehicles, such as the location information of vehicle i The location information of all unmanned aerial vehicles, such as the location information of unmanned aerial vehicle u The computing tasks generated by all vehicles, such as the computing tasks generated by vehicle i The observation space of the MEC server deployed at the roadside unit is the location information and computing task information of all vehicle users within its coverage. Similarly, the observation space of the MEC server deployed on the drone is the location information and computing task information of all vehicles within its coverage, as well as its own location information.

[0087] The action space of the Markov decision process (MDP) model is represented as follows:

[0088] The three 0-1 variables representing the offloading strategy are relaxed to real variables, i.e., The constraint conditions are kept unchanged, i.e.,

[0089] When , the i-th vehicle selects to compute the task locally; when , the i-th vehicle offloads the task data to the MEC server located at the roadside unit for computing; when , the i-th vehicle offloads the task data to the MEC server located on the drone for computing and processing. Given that each vehicle's computing task adopts a full offloading strategy, the connected vehicle can only choose to either compute the entire task locally or offload the entire task to the MEC server located at the roadside unit or on the drone for computing. Therefore, when , and When , and When and That is,

[0090] The largest value in is the offloading destination. For example, if the task is offloaded to the MEC server deployed on the u-th drone. Therefore, the action vector contains the following multi-dimensional features: the offloading strategy vector of all vehicle tasks, such as the offloading strategy vector of the i-th vehicle the resource allocation vector of all MEC servers, such as the resource allocation vector of the MEC server located at the m-th roadside unit the resource allocation vector of the MEC server located on the u-th drone

[0091] When the vehicle is at the junction of the coverage areas of two roadside units or at the edge of the roadside unit coverage area, it is forced to select drone offloading Offload the computing task to the mobile edge computing server on the drone.

[0092] The reward space of the Markov decision process (MDP) model is represented as follows:

[0093]

[0094] where r i delay is the delay reward, T i max is the maximum delay allowed for the computing task, T i is the total delay of the computing task, r i cache is the cache reward, is the cache resource allocated to the i-th vehicle, d i is the size of the computing task.

[0095] S5. Establish communication links between the vehicle and the roadside unit and the UAV within its communication coverage respectively, and calculate the total delay of the computing task and the size of the cache resource.

[0096] The formula for the maximum data transfer rate between the vehicle and the MEC server deployed at the m-th roadside unit is:

[0097]

[0098] where is the bandwidth ratio allocated by the MEC server deployed at the m-th roadside unit to vehicle i, is the bandwidth allocated to the wireless device of vehicle i, σ 2 is the background noise power of the vehicle networking environment, P i is the signal transmission power of vehicle i, is the channel gain between vehicle i and the MEC server deployed at the m-th roadside unit.

[0099] The formula for the maximum data transfer rate between the vehicle user and the MEC server deployed on the u-th UAV is:

[0100]

[0101] where is the bandwidth ratio allocated by the MEC server deployed on the u-th UAV to vehicle i, is the bandwidth allocated to the wireless device of vehicle i, σ 2 is the background noise power of the vehicle networking environment, P i is the signal transmission power of vehicle i, is the channel gain between the vehicle and the MEC server deployed on the u-th UAV.

[0102] The total delay of the computing task is:

[0103]

[0104] where T i is the total time delay of the computing task, is the offloading decision parameter for the vehicle to locally execute the computing task, and T i loc is the time delay caused by the vehicle locally executing the computing task, is the offloading decision parameter for offloading the computing task to the roadside unit, and T i m is the time delay generated by the mobile edge computing server when the computing task is offloaded to the roadside unit, is the offloading decision parameter for offloading the computing task to the unmanned aerial vehicle, and T i u is the time delay generated by the mobile edge computing server when the computing task is offloaded to the unmanned aerial vehicle.

[0105] The time delay caused by the vehicle locally executing the computing task is:

[0106]

[0107] where c i is the CPU cycles required to complete the computing task generated by the i-th vehicle, and Co i is the local computing power of the i-th vehicle;

[0108] The time delay generated by the mobile edge computing server when the computing task is offloaded to the roadside unit is:

[0109]

[0110] where is the computing time delay generated by the mobile edge computing server when the computing task is offloaded to the m-th roadside unit, is the transmission time delay generated by the mobile edge computing server when the computing task is offloaded to the m-th roadside unit, is the proportion of computing resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle, is the computing resources of the mobile edge computing server deployed on the m-th roadside unit, and d i is the data size of the computing task generated by the i-th vehicle, and r i m is the maximum data transmission rate between the i-th vehicle and the mobile edge computing server deployed on the m-th roadside unit;

[0111] The time delay generated by the mobile edge computing server when the computing task is offloaded to the unmanned aerial vehicle is:

[0112]

[0113] Wherein, is the computing delay generated by the mobile edge computing server that offloads the computing task to the u-th drone, is the transmission delay generated by the mobile edge computing server that offloads the computing task to the u-th drone, is the proportion of computing resources allocated by the mobile edge computing server deployed on the u-th drone to the i-th vehicle, is the computing resource of the mobile edge computing server deployed on the u-th drone, r i u is the maximum data transmission rate between the i-th vehicle and the mobile edge computing server deployed on the u-th drone.

[0114] S6. Use the deep deterministic policy gradient algorithm to construct an Actor-Critic network, and through the reward function, guide the Actor-Critic network to achieve the goal of minimizing the task processing delay, and input the multi-dimensional state space and action space to generate task offloading decisions and resource allocation strategies.

[0115] Use such as Figure 3 The DDPG algorithm shown processes multi-dimensional states, trains neural network parameters, and optimizes task offloading strategies and resource allocation. The specific process of the DDPG algorithm is as follows:

[0116] First, initialize the policy gradient θ of the policy network (Actor) μ and the Q gradient θ of the value network (Critic) Q as well as their respective target network parameters θ μ′ and θ Q′ . At the same time, initialize the experience replay pool. Train the agent iteratively in the road scenario of drone-assisted vehicle edge computing, and the MEC server independently learns the determined service strategy after iteration. Secondly, simulate the parameters of the vehicle network environment, randomly generate a state s t ∈S, set it as the initial state, start training within a time period, and during the training, execute the current policy network parameters θ μ . Select an action at a t = μ(s t |θ μ ) to determine the offloading location. Then form a quadruple (s t , a t , r t , s t+1)Stored in the buffer. When the buffer memory is full, the Critic network uses the experience replay technique. During this time slot, the records stored in the replay tuple A small batch of data in the tuple needs to be randomly sampled for training and updating. Then, the parameters of the Critic network are updated by minimizing the loss function. The minimum loss function in the judgment process is:

[0117]

[0118] In the formula, θ Q is the Q-gradient of the value network, χ is the size of the sampling batch, s t is the state at the current moment, a t is the action at the current moment, y t is the target Q value.

[0119] Among them, the Q value of the target network is calculated by the Critic network:

[0120] y t = r t + γQ′(s t+1 , μ′(s t+1 |θ μ′ )|θ Q′ )

[0121] In the formula, r t is the reward at the current moment, s t+1 is the state at the next moment, μ′ is the target actor network, θ μ′ is the parameter of the target policy network, θ Q′ is the parameter of the target critic network.

[0122] Then, the parameters of the Actor network are updated using the policy gradient, expressed as:

[0123]

[0124] In the formula, is the gradient of the Actor network parameter θ μ , and the goal is to maximize the long-term return expectation J, indicating the gradient of the action μ(s t |θ μ ) output by the Actor network with respect to its own parameter θ μ .

[0125] Finally, the soft update method is used to update the target network parameters to:

[0126] θ μ′ ← τθ μ +(1 - τ)θ μ′

[0127] θ Q′ ← τθ Q +(1 - τ)θ Q′

[0128] where θ μ′ is the target policy network parameter, τ is the soft update coefficient, and θ μ is the policy gradient, and θ Q′ is the target Q - network parameter, and θ Q is the Q - gradient.

[0129] S7. Sample and train the Actor - Critic network parameters through the experience replay pool, and optimize the task offloading decision and resource allocation strategy in real - time to obtain the optimal offloading decision and resource allocation strategy.

[0130] Combined with the DDPG algorithm, the vehicle offloading decision and resource allocation method can be specifically implemented as the following steps: Use all initial data, including vehicle location, resource status, task queue, etc. to obtain the initial state. Obtain the current action and the next state. Allocate resources to each computing task and calculate the total delay. Obtain the reward based on the delay constraint and resource utilization rate and obtain the new state. Store the state, action, reward, and next state, etc. in the experience replay pool. Optimize the offloading decision and resource allocation by randomly sampling from the experience replay pool to train the neural network.

[0131] Embodiment 2

[0132] This embodiment provides a vehicle MEC task offloading and resource allocation optimization method and system, including

[0133] A road scene establishment module, which is used to establish a road scene for drone - assisted vehicle edge computing. The number of lanes in the road scene for drone - assisted vehicle edge computing is fixed. Road - side units are set beside the lanes. The drone flies at a constant speed under the coverage area of the road - side unit and provides resource access for the vehicle. Mobile edge computing servers are deployed on the road - side unit and the drone;

[0134] A road scene definition module, which is used to define the transmission delay, computing delay, and vehicle - drone motion mode of the vehicle, drone, and road - side unit according to the road scene for drone - assisted vehicle edge computing, and define the multi - dimensional resource matrix of the road - side unit and the drone. The multi - dimensional resource matrix includes computing resources, cache resources, and spectrum resources;

[0135] An optimization problem formulation module, which, based on three offloading modes: executing the computing task locally on the vehicle, offloading the computing task to the mobile edge computing server on the road - side unit, and offloading the computing task to the mobile edge computing server on the drone, comprehensively formulates an optimization problem of minimizing the task processing delay with the collaboration of computing resources, cache resources, and spectrum resources;

[0136] A Markov decision process model definition module, which is used to describe the task offloading decision process through a Markov decision process model, and define the multi-dimensional state space, action space and reward function of the Markov decision process model;

[0137] A communication link establishment module, which is used to establish communication links between a vehicle and the roadside units and unmanned aerial vehicles within its communication coverage respectively, and calculate the total delay of the computing task and the size of the cache resource;

[0138] A task offloading decision and resource allocation strategy generation module, which is used to construct an Actor-Critic network by using the deep deterministic policy gradient algorithm, guide the Actor-Critic network to achieve the goal of minimizing the task processing delay through the reward function, and generate a task offloading decision and resource allocation strategy by inputting the multi-dimensional state space and action space;

[0139] A task offloading decision and resource allocation strategy optimization module, which is used to sample and train the Actor-Critic network parameters through an experience replay pool, optimize the task offloading decision and resource allocation strategy in real time, and obtain the optimal offloading decision and resource allocation strategy.

[0140] The rest is the same as in Embodiment 1.

[0141] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations according to the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field of the present invention based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the protection scope determined by the claims.

Claims

1. An optimization method for vehicle MEC task offloading and resource allocation, characterized in that, Including the following steps: Establish a road scenario for drone-assisted vehicular edge computing. The number of lanes in the road scenario for drone-assisted vehicular edge computing is fixed. Roadside units are set beside the lanes. The drone flies at a constant speed under the coverage area of the roadside unit and provides resource access for vehicles. Mobile edge computing servers are deployed on the roadside unit and the drone; Define the transmission delay, computing delay, and vehicle-drone motion mode of vehicles, drones, and roadside units according to the road scenario for drone-assisted vehicular edge computing, and define the multi-dimensional resource matrix between the roadside unit and the drone. The multi-dimensional resource matrix includes computing resources, caching resources, and spectrum resources; Based on three offloading modes of executing computing tasks locally on the vehicle, offloading computing tasks to the mobile edge computing server on the roadside unit, and offloading computing tasks to the mobile edge computing server on the drone, comprehensively propose an optimization problem of minimizing the task processing delay with the coordination of computing resources, caching resources, and spectrum resources; Describe the task offloading decision-making process through a Markov decision process model, and define the multi-dimensional state space, action space, and reward function of the Markov decision process model; Establish communication links between the vehicle and the roadside unit and the drone within its communication coverage respectively, and calculate the total delay of the computing task and the size of the caching resources; Use the deep deterministic policy gradient algorithm to construct an Actor-Critic network, and guide the Actor-Critic network to achieve the goal of minimizing the task processing delay through the reward function, and input the multi-dimensional state space and action space to generate task offloading decisions and resource allocation strategies; Sample and train the parameters of the Actor-Critic network through an experience replay pool, and optimize the task offloading decisions and resource allocation strategies in real time to obtain the optimal offloading decisions and resource allocation strategies.

2. The vehicle MEC task offloading and resource allocation optimization method according to claim 1, wherein After the vehicle generates a computing task, it sends a resource access request to the mobile edge computing server. The mobile edge computing server makes vehicle association and resource allocation decisions based on the received resource access request and returns them to the vehicle. The resource access request includes the detailed information of the computing task and the driving state of the vehicle. The vehicle association and resource allocation decisions include computing locally on the vehicle, offloading the computing task to the mobile edge computing server on the roadside unit, or offloading the computing task to the mobile edge computing server on the drone, as well as the proportions of spectrum resources, computing resources, and caching resources allocated to the vehicle.

3. The vehicle MEC task offloading and resource allocation optimization method according to claim 2, wherein, The computing tasks generated by the vehicle are: D i = (c i , d i , T i max ) where D i is the computing task of the i-th vehicle, and c i is the CPU cycles required for the i-th vehicle to complete the computing task, d i is the data size of the computing task generated by the i-th vehicle, and T i max is the maximum latency allowed for the computing task generated by the i-th vehicle.

4. The vehicle MEC task offloading and resource allocation optimization method according to claim 2, wherein After the vehicle receives the vehicle association and resource allocation decisions, it offloads the task to the associated mobile edge computing server according to the proportions of spectrum resources, computing resources, and caching resources allocated to the vehicle.

5. The vehicle MEC task offloading and resource allocation optimization method according to claim 1, characterized in that When the vehicle is at the junction of the coverage areas of two roadside units or at the edge of the coverage area of a roadside unit, it offloads the computing task to the mobile edge computing server on the drone.

6. The vehicle MEC task offloading and resource allocation optimization method according to claim 1, characterized in that, The optimization problem of minimizing the task processing delay is: where H is the unit step function, and T i is the total time delay of the computing task, and T i max is the maximum time delay allowed for the computing task, is the caching resource allocated by the MEC server to the i-th vehicle, and d i is the size of the computing task, is the offloading decision parameter for the i-th vehicle to execute the computing task locally, is the offloading decision parameter for offloading the computing task to the roadside unit, is the offloading decision parameter for offloading the computing task to the unmanned aerial vehicle, is the proportion of spectrum resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle, is the proportion of computing resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle, is the proportion of caching resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle, is the proportion of spectrum resources allocated by the mobile edge computing server deployed on the u-th unmanned aerial vehicle to the i-th vehicle, is the proportion of computing resources allocated by the mobile edge computing server deployed on the u-th unmanned aerial vehicle to the i-th vehicle, is the proportion of caching resources allocated by the mobile edge computing server deployed on the u-th unmanned aerial vehicle to the i-th vehicle.

7. The vehicle MEC task offloading and resource allocation optimization method according to claim 6, characterized in that, When a computing task is executed locally on a vehicle, if the cache resources provided locally by the vehicle are equal to the size of the computing task, and when the computing task needs to be offloaded to a mobile edge computing server, the cache resources allocated by the mobile edge computing server to the vehicle are greater than the size of the computing task generated by this vehicle. The cache resources allocated by the mobile edge computing server to the vehicle are as follows: wherein, is the cache resource of the mobile edge computing server deployed on the m-th roadside unit, is the cache resource of the mobile edge computing server deployed on the u-th unmanned aerial vehicle.

8. The vehicle MEC task offloading and resource allocation optimization method according to claim 6, characterized in that The total delay of the computing task is: where, T i is the total delay of the computing task, is the offloading decision parameter for the vehicle to locally execute the computing task, and T i loc is the delay caused by the vehicle locally executing the computing task, is the offloading decision parameter for offloading the computing task to the roadside unit, and T i m is the delay generated by the mobile edge computing server when offloading the computing task to the roadside unit, is the offloading decision parameter for offloading the computing task to the drone, and T i u is the delay generated by the mobile edge computing server when offloading the computing task to the drone.

9. The vehicle MEC task offloading and resource allocation optimization method according to claim 7, wherein The delay caused by the vehicle executing the computing task locally is: where c i is the CPU cycles required to complete the computing tasks generated by the i-th vehicle, and Co i is the local computing power of the i-th vehicle; The delay generated by offloading the computing task to the mobile edge computing server on the roadside unit is: Wherein, is the computing delay generated by the mobile edge computing server that offloads the computing task to the m-th roadside unit; is the transmission delay generated by the mobile edge computing server that offloads the computing task to the m-th roadside unit; is the proportion of computing resources allocated by the mobile edge computing server deployed on the m-th roadside unit to the i-th vehicle; is the computing resources of the mobile edge computing server deployed on the m-th roadside unit, d i is the data size of the computing task generated by the i-th vehicle, r i m is the maximum data transmission rate between the i-th vehicle and the mobile edge computing server deployed on the m-th roadside unit; The delay generated by offloading the computing task to the mobile edge computing server on the unmanned aerial vehicle is: Wherein, is the computing delay generated by the mobile edge computing server for offloading the computing task to the u-th drone, is the transmission delay generated by the mobile edge computing server for offloading the computing task to the u-th drone, is the proportion of computing resources allocated by the mobile edge computing server deployed on the u-th drone to the i-th vehicle, is the computing resources of the mobile edge computing server deployed on the u-th drone, r i u is the maximum data transmission rate between the i-th vehicle and the mobile edge computing server deployed on the u-th drone.

10. A method and system for optimizing vehicle MEC task offloading and resource allocation, characterized in that, including A road scene establishment module, configured to establish a road scene for unmanned aerial vehicle-assisted vehicle edge computing. The number of lanes in the road scene for unmanned aerial vehicle-assisted vehicle edge computing is fixed. Roadside units are arranged beside the lanes. The unmanned aerial vehicle flies at a constant speed under the coverage area of the roadside unit and provides resource access for the vehicle. Mobile edge computing servers are deployed on the roadside units and the unmanned aerial vehicle; A road scene definition module, configured to define the transmission delay, computing delay, and vehicle-unmanned aerial vehicle motion mode of the vehicle, unmanned aerial vehicle, and roadside unit according to the road scene for unmanned aerial vehicle-assisted vehicle edge computing, and define a multi-dimensional resource matrix of the roadside unit and the unmanned aerial vehicle. The multi-dimensional resource matrix includes computing resources, cache resources, and spectrum resources; An optimization problem formulation module, based on three offloading modes of executing the computing task locally on the vehicle, offloading the computing task to the mobile edge computing server on the roadside unit, and offloading the computing task to the mobile edge computing server on the unmanned aerial vehicle, comprehensively formulates an optimization problem of minimizing the task processing delay with the collaboration of the computing resources, cache resources, and spectrum resources; A Markov decision process model definition module, configured to describe the task offloading decision process through a Markov decision process model, and define the multi-dimensional state space, action space, and reward function of the Markov decision process model; A communication link establishment module, configured to establish communication links between the vehicle and the roadside unit and the unmanned aerial vehicle within its communication coverage area respectively, and calculate the total delay of the computing task and the size of the cache resources; A task offloading decision and resource allocation strategy generation module, configured to use the deep deterministic policy gradient algorithm to construct an Actor-Critic network, guide the Actor-Critic network to achieve the goal of minimizing the task processing delay through the reward function, and input the multi-dimensional state space and action space to generate a task offloading decision and resource allocation strategy; A task offloading decision and resource allocation strategy optimization module, configured to sample and train the Actor-Critic network parameters through an experience replay pool, and optimize the task offloading decision and resource allocation strategy in real time to obtain an optimal offloading decision and resource allocation strategy.

Images