Dual-Layer UAV-Assisted Vehicular Network Task Offloading and Resource Scheduling Method and Medium

Abstract

A method and medium for double - layer UAV - assisted task offloading and resource scheduling in vehicle - to - everything (V2X) networks. Aiming at the problems of limited computing and storage capabilities of vehicles and roadside units in existing vehicle - edge computing, and the challenge that traditional fixed infrastructure is difficult to meet the requirements of high real - time and low energy consumption under high - load conditions, a double - layer UAV - assisted V2X network architecture is adopted, including upper - layer relay UAVs and lower - layer computing UAVs, which work together to share the high - load computing tasks that roadside units cannot handle. The present invention introduces the age of information as an index to measure data freshness, and constructs an optimization model for task offloading and resource scheduling based on the age of information. The multi - agent double - delay deep deterministic policy gradient algorithm is used to solve the model, effectively optimizing the load balance of the roadside unit RSU in a dynamic V2X environment, and reducing the age of system task information and processing energy consumption.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle networking, and particularly relates to a method and a storage medium for double-layer unmanned aerial vehicle-assisted task offloading and resource scheduling in vehicle networking. Background Art

[0002] With the rapid development of vehicle networking and the sixth-generation mobile communication technology, emerging applications such as autonomous driving and real-time video-assisted navigation have emerged continuously. These applications require vehicles to process a large amount of data and complete complex computing tasks, such as computationally intensive tasks or latency-sensitive tasks like real-time traffic information analysis, road obstacle recognition, and path planning. However, traditional in-vehicle computing resources and storage capabilities often cannot meet these requirements.

[0003] As an emerging computing architecture, vehicle edge computing moves computing and storage resources from cloud servers far away from vehicles to edge nodes such as roadside units close to vehicles, thereby effectively reducing data transmission latency and significantly improving service efficiency. However, although vehicle edge computing provides a more flexible way of computing resource allocation, traditional fixed infrastructure and roadside units still face some challenges, especially under high load. First, due to the high-speed mobility of vehicles and the temporal fluctuations of traffic flow, fixed edge servers cannot meet the actual needs of vehicle networking during certain periods. For example, during the traffic peak, the traffic flow may be several times that of normal times, and the computing tasks generated by vehicles may exceed the carrying capacity of the edge server, resulting in overloading of the edge server and thus affecting the service quality. To alleviate this situation, traditional methods usually increase the number of roadside units, but this will increase the deployment cost and resource consumption. Especially in some areas where it is difficult to lay optical cables, the deployment of edge servers will be greatly restricted. In addition, how to continue to provide efficient computing and forwarding services when the network is congested or the edge server fails is also a key problem faced by vehicle edge computing.

[0004] To solve the above problems, in recent years, using unmanned aerial vehicles as dynamic edge service nodes has gradually become a solution. Unmanned aerial vehicles, with their flexible deployment capabilities, mobility, and relatively high computing and transmission capabilities, can provide additional computing and communication services in special areas or periods. When the ground communication infrastructure is limited, fails, or when the network is congested, unmanned aerial vehicles can dynamically adjust their flight trajectories and task offloading strategies according to the traffic flow, the distribution of computing tasks, and the coverage requirements to provide higher-quality services. By introducing unmanned aerial vehicle-assisted vehicle networking, timely offloading and effective allocation of tasks can be achieved, avoiding problems such as delays or service interruptions caused by overloading of fixed infrastructure. At the same time, unmanned aerial vehicles can also quickly respond to changes in the network environment, break through the limitations of static infrastructure, and provide instant computing resources in specific areas or periods, significantly enhancing the scalability and adaptability of the vehicle networking system.

[0005] In addition to computing offloading and resource scheduling, the real-time nature and timeliness of data in the vehicle-to-everything (V2X) network are also crucial. In many real-time applications, the freshness of information directly affects the accuracy and safety of decision-making. Traditional performance evaluations of V2X networks often focus on metrics such as latency, bandwidth, and throughput, but these metrics cannot comprehensively reflect the timeliness of information. As an important metric for measuring the freshness of information, the age of information has gradually become a key reference in the optimization of V2X networks in recent years. The age of information is defined as the time from when the information is generated to when it is received. The larger the age of information, the more stale or even outdated the information may be, which may lead to incorrect decisions or operations. In the V2X network environment, due to the high-speed movement of vehicles and the dynamically changing network environment, how to reduce the age of information to improve the freshness of information and reduce the backlog of information has become an important goal in optimizing computing offloading and resource scheduling in V2X networks. Summary of the Invention

[0006] A two-layer unmanned aerial vehicle (UAV)-assisted task offloading and resource scheduling method, device, and storage medium proposed by the present invention can at least solve one of the technical problems in the background art.

[0007] To achieve the above objective, the present invention adopts the following technical solutions:

[0008] A two-layer UAV-assisted task offloading and resource scheduling method for V2X networks includes the following steps:

[0009] Step 1: Construct a scenario for two-layer UAV-assisted task offloading and resource scheduling in V2X networks.

[0010] Step 2: Respectively establish a task processing time model, an energy consumption model, and an age of information model according to different task calculation locations; and based on different offloading methods, combined with the allocation strategies of transmission power, computing resources, and bandwidth resources, construct an optimization problem aiming to minimize the weighted sum of the average age of information and energy consumption in the system.

[0011] Step 3: According to the established models, design a method for solving; propose a two-layer UAV-assisted task offloading and resource scheduling algorithm MATD3-DLUTORS based on MATD3 to solve the multi-user multi-server offloading problem.

[0012] On the other hand, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to execute the steps of the above method.

[0013] On yet another hand, the present invention also discloses a computer device including a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the above method.

[0014] As can be seen from the above technical solutions, the method and medium for task offloading and resource scheduling in a two-layer UAV-assisted vehicle network of the present invention are related to the processing of computationally intensive tasks and latency-sensitive tasks in a vehicle network. Aiming at the problems of limited computing and storage capabilities of vehicles and roadside units in existing vehicle edge computing, and the challenge that traditional fixed infrastructure is difficult to meet the requirements of high real-time and low energy consumption under high load, the present invention adopts a two-layer UAV-assisted vehicle network architecture. This architecture includes upper-layer relay UAVs and lower-layer computing UAVs, which work together to share the high-load computing tasks that cannot be processed by roadside units. The present invention introduces the age of information as an index to measure data freshness, and constructs an optimization model for task offloading and resource scheduling based on the age of information. The present invention uses the multi-agent double-delay deep deterministic policy gradient algorithm to solve the model, and effectively optimizes the load balancing of the roadside unit RSU in a dynamic vehicle network environment, reducing the system task age of information and processing energy consumption.

[0015] The present invention proposes a two-layer UAV-assisted vehicle network architecture based on the age of information. By introducing upper-layer relay UAVs and lower-layer computing UAVs, it collaboratively processes computationally intensive and latency-sensitive tasks that are difficult for high-load roadside units RSU to handle alone. For the task offloading and resource scheduling problem in this scenario, the present invention constructs a mixed-integer non-linear programming optimization problem with the goal of minimizing the weighted sum of the system average age of information and average energy consumption, and proposes a two-layer UAV-assisted vehicle network task offloading and resource scheduling algorithm MATD3-DLUTORS based on MATD3, successfully achieving an effective reduction in the system average age of information and energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a two-layer UAV-assisted computing network architecture;

[0017] Figure 2 is the AoI evolution process;

[0018] Figure 3 is the two-layer UAV-assisted vehicle network task offloading and resource scheduling algorithm MATD3-DLUTORS;

[0019] Figure 4 is a performance comparison chart of different algorithms under different numbers of vehicles. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention.

[0021] Such asFigure 1 As shown in the figure, the method for task offloading and resource scheduling of a two-layer unmanned aerial vehicle (UAV)-assisted vehicular network in this embodiment includes a scenario description module, a model establishment module, and a model solution module, and comprises the following steps:

[0022] S1. Establish a two-layer UAV-assisted vehicular network model and a task scheduling model;

[0023] S2. Establish a vehicle task execution model for the two-layer UAV-assisted vehicular network to obtain the final task processing time and energy consumption;

[0024] S2.1. The vehicle performs task calculation locally to obtain the local processing time and total energy consumption;

[0025] S2.2. The vehicle selects the roadside unit (RSU) to access according to the signal-to-noise ratio and the current load condition of the roadside unit RSU, and offloads the task to the roadside unit RSU for calculation or relays it to the corresponding UAV;

[0026] S2.2.1. When the roadside unit RSU is in a normal state, the computing tasks generated by the vehicle will be processed on the roadside unit RSU;

[0027] S2.2.2. When the roadside unit RSU is in a high-load state, if the vehicle task is a delay-sensitive task, it will be directly calculated by the lower-layer UAV;

[0028] S2.2.3. When the roadside unit RSU is in a high-load state, if the vehicle task is a computation-intensive task, the upper-layer UAV will relay the task to a distant idle roadside unit RSU for processing;

[0029] S3. Construct an Age of Information (AoI) model for the system according to different task processing methods;

[0030] S4. Construct an optimization problem according to the network model, task scheduling model, task execution model, and AoI model. The goal is to minimize the weighted sum of the system average Age of Information and average task processing energy consumption by jointly optimizing the task offloading strategy, computing resource allocation strategy, transmission power, and bandwidth allocation under the premise of meeting the task delay constraint and UAV movement restriction;

[0031] S5. Reformulate the optimization problem as a Markov decision process, and propose a task offloading and resource scheduling algorithm based on MATD3 - MATD3-DLUTORS to solve this problem.

[0032] The following is a specific description:

[0033] The scenario description module is mainly used to construct a scenario for task offloading and resource scheduling of a two-layer UAV-assisted vehicular network. As Figure 1 shown, the present invention considers a network consisting of The urban topology covered by each roadside unit (RSU), and define the set of roadside units (RSUs) as . Each roadside unit (RSU) is equipped with an edge server for processing the computing tasks generated by vehicles. Suppose there are vehicles driving on the road, , and denote as the set of vehicles. In addition, the system time in the present invention is discretized into a series of time slots , and the duration of each time slot is . Within each time slot, each vehicle generates a computing task, which can be processed locally on the vehicle or offloaded to the roadside unit (RSU) via the 5G uplink for processing. On this basis, the present invention designs a two-layer drone-assisted vehicle network architecture to share and process the computing tasks that cannot be completed by the overloaded roadside units (RSUs). The task allocation and coordination between the roadside units (RSUs) and the drones are carried out through the 5G network. Among them, the upper-layer relay drones are located at a higher position and are responsible for relaying the computationally intensive tasks to the distant idle roadside units (RSUs) for execution; while the lower-layer computing drones carry edge servers and hover at a lower altitude to directly process the latency-sensitive tasks. Define the set of drones as , where is the set of upper-layer relay drones, is the set of lower-layer computing drones.

[0034] Overloaded The location can be expressed as , the upper-layer relay drone and the lower-layer computing drone The locations in time slot can be expressed as and respectively, and .

[0035] In the present invention, according to the characteristics of the tasks, the computing tasks generated by vehicles can be divided into two categories: computationally intensive tasks and latency-sensitive tasks. Computationally intensive tasks usually require a large amount of computing resources to complete, while latency-sensitive tasks usually require completion within an extremely short time. To ensure the task completion rate, for those tasks with ultra-low latency requirements and large computing resource requirements, the present invention classifies them as latency-sensitive tasks. It should be noted that both computationally intensive tasks and latency-sensitive tasks must be completed within their respective maximum tolerable latencies.

[0036] Based on the above task characteristics, the present invention describes the computing task generated by vehicle as a four-tuple vector . Among them, Is a computing task The number of CPU cycles required for a single-bit data For the task The input data size For the task The maximum completion delay that can be tolerated Indicates the task Type Indicates that the task is a compute-intensive task Indicates that the task is a latency-sensitive task. The task Can be locally computed by the vehicle Or offloaded to the roadside unit RSU via the V2R wireless communication link for processing. Among them, the access mechanism between the vehicle and the roadside unit RSU is determined based on the signal-to-noise ratio and the current load of the roadside unit RSU. Specifically, if the vehicle is only within the coverage area of one roadside unit RSU, it can choose to locally complete the computing task or offload it to this roadside unit RSU for processing; if the vehicle is located in the overlapping area of the coverage ranges of multiple roadside unit RSUs, it is necessary to comprehensively consider the signal-to-noise ratio and the current load of each roadside unit RSU, and select the roadside unit RSU with better signal quality and lighter load for task offloading, so as to optimize the utilization rate of computing resources and the task processing efficiency

[0037] To balance the signal-to-noise ratio and the load of the roadside unit RSU, the present invention introduces a comprehensive scoring function Taking the vehicle And As an example, its comprehensive scoring function Is defined as

[0038]

[0039] Among them Is the signal-to-noise ratio between the vehicle And ; Is The current load ratio Is The current computing load Is The maximum computing power And Are the weight coefficients of the signal-to-noise ratio and the load ratio respectively

[0040] To avoid performance bottlenecks caused by the overload of the roadside unit RSU, the present invention introduces a roadside unit RSU load ratio threshold To represent the upper limit of the ratio of the computing load of the roadside unit RSU to its maximum computing power, and introduces an unmanned aerial vehicle to share and process excessive computing tasks on the high-load roadside unit RSU

[0041] Due to the differences in traffic flow density in different regions, the task offloading requirements in different regions are also different. If only upper-layer drones are deployed, the high-load roadside units (RSUs) in the regions with high traffic flow density cannot process delay-sensitive tasks in a timely manner, making it difficult to meet the computing requirements of vehicle users. If only lower-layer computing drones are deployed, due to the occlusion of surrounding buildings, they cannot communicate with the idle RSUs at a long distance through line of sight (LoS), and the resources of the RSUs in the regions with low traffic flow density are difficult to be fully utilized. It can be seen that compared with the single-layer architecture, adopting a two-layer drone architecture can not only achieve more efficient computing task offloading services and stronger computing performance, but also can meet different traffic environment requirements, improve the utilization rate of RSU resources, and achieve load balancing of RSUs. Therefore, the present invention adopts a two-layer drone-assisted vehicle networking architecture to optimize computing task offloading, improve the system response speed, and effectively balance the computing load of RSUs.

[0042] Specifically, when the load ratio of a certain RSU does not exceed the load ratio threshold, that is , it means that the RSU is in a normal state, and the computing tasks unloaded by the vehicle to this RSU will be calculated by the RSU. When the load ratio of a certain RSU exceeds the load ratio threshold, that is , it means that the RSU is already in a high-load state. The system will dynamically select an appropriate drone for assisted computing or task forwarding according to the delay requirements and computing resource requirements of the current task, ensuring that the system can take effective measures when the load of the RSU is high but not overloaded, thereby avoiding a sharp decline in performance.

[0043] In addition, to ensure that there is always a sufficient safety distance between all drones, a minimum distance is set to avoid potential drone collision risks. Therefore, the positions of any two drones and ( and ) at any time should satisfy the distance constraint

[0044]

[0045] Define the vehicle offloading strategy set . When , it means that the vehicle offloads the task to for computing or relays it to the drone. When , it means that the vehicle does not offload the task to above. In particular, indicates that the vehicle selects a local processing task , indicates the vehicle generates a computing task not to be processed locally.

[0046] To ensure that each vehicle can only select to locally process the computing tasks it generates or offload the tasks to at most one roadside unit (RSU), there is

[0047]

[0048] In addition, the present invention uses to represent the auxiliary decision-making set of the upper-layer relay UAV, represents the auxiliary decision-making set of the lower-layer computing UAV. When , the upper-layer relay UAV relays the computationally intensive tasks from the high-load to a distant roadside unit (RSU) for processing; conversely, when , the high-load does not offload the task to the upper-layer UAV for relaying. Similarly, when , the lower-layer computing UAV directly provides computing services for the high-load ; conversely, when , the high-load does not offload the task to the lower-layer UAV for computing.

[0049] The model establishment module mainly establishes a task processing time model, an energy consumption model, and an age of information model according to different task calculation positions, and models the system average age of information and the weighted sum of energy consumption minimization problem according to different offloading methods and the allocation of transmission power, computing resources, and bandwidth resources.

[0050] (1) The vehicle performs local computing

[0051] If the vehicle selects to locally process the computing task , that is, , the computing time and energy consumption required for it to execute the task are

[0052]

[0053]

[0054] where is the effective capacitance constant of the vehicle; For vehicles The computing resources allocated to local computing tasks cannot exceed the value of the vehicle Maximum computing power ,Right now .

[0055] (2) The vehicle offloads the task to the roadside unit RSU

[0056] Assuming that the wireless communication between the vehicle and the roadside unit RSU is based on orthogonal frequency division multiple access, in which different users are separated in the frequency domain, the vehicle arrive The uplink task transmission rate can be expressed as

[0057]

[0058] In the formula, Indicates the vehicle is assigned It should be noted that the total bandwidth allocated to all vehicles must not exceed the total bandwidth used by the system for V2R communication. ,Right now

[0059]

[0060] set up For vehicles The maximum transmission power of the vehicle Transmit power Should meet

[0061]

[0062] Therefore, the vehicle The task Uninstall to The transmission delay is

[0063]

[0064] vehicle Upload computing tasks The corresponding transmission energy consumption is

[0065]

[0066] 2.1) Tasks are calculated on the roadside unit RSU

[0067] Execute the task The computational delay and computational energy consumption are

[0068]

[0069]

[0070] Among them, represents the computing resources allocated for the task, and its value cannot exceed the maximum available computing resources, that is ; ; is the effective capacitance constant of the roadside unit RSU.

[0071] Similar to most previous work, the size of the output data volume of the calculation result is much smaller than the size of the original input data. Therefore, the backhaul delay and the corresponding energy consumption of the return result are negligible in the present invention.

[0072] 2.2) Delay-sensitive tasks are calculated on the lower-layer UAVs

[0073] When the computing load ratio of the edge server deployed on exceeds the load threshold , if the time slot is a computing task offloaded by the vehicle and is a delay-sensitive task, that is , the high load will offload the task to the lower-layer computing UAV for calculation.

[0074] When the UAV flies at a certain altitude and communicates with the roadside unit RSU, the communication channel is mainly affected by the LoS transmission path. Therefore, the free space path loss model is used for the channel between the lower-layer computing UAV and the roadside unit RSU. During the time slot , the lower-layer computing UAV hovers above the high load to receive and execute the delay-sensitive task . The probability that they can establish a LoS channel is

[0075]

[0076] Among them, and are parameters related to the environment; is the time slot The elevation angle between the lower-layer computing UAV and the high load . Therefore, The uplink channel gain from to the lower-layer computing UAV

[0077]

[0078] Among them, is the unit channel gain when the distance between the roadside unit RSU and the UAV is 1 m and the transmission power is 1 W; the parameter is the attenuation factor of the non-line-of-sight channel. Therefore, offloading the delay-sensitive task to the lower-layer computing UAV has an effective transmission rate of

[0079]

[0080] Among them, represents the transmission bandwidth between and the lower-layer computing UAV represents the transmission power from to the lower-layer computing UAV The value shall not exceed the maximum transmission power of , that is,

[0081]

[0082] Offloading the task to the lower-layer computing UAV The transmission delay and transmission energy consumption can be respectively expressed as

[0083]

[0084]

[0085] The lower-layer computing UAV assists in processing the high-load task The computing delay is

[0086]

[0087] Among them, is the CPU frequency of the processor of the lower-layer computing UAV . It is assumed in the present invention that the computing capabilities of all lower-layer UAVs are the same. Further, it can be obtained that the UAV consumes energy for computing the task as

[0088]

[0089] In the formula, is the effective capacitance coefficient of the edge server chip carried by the lower-layer computing UAV.

[0090] In addition, for the high-load , the lower-layer computing UAV maintains a hovering state within a limited time to receive and process data. The lower-layer UAV 's hovering power can be expressed as

[0091]

[0092] where is the mass of the lower-layer UAV; is the radius of the lower-layer UAV's propeller; is the number of the lower-layer UAV's propellers. Accordingly, the lower-layer UAV 's hovering energy consumption is expressed as follows

[0093]

[0094] In summary, the lower-layer computing UAV receives and calculates high-load offloaded latency-sensitive tasks The total latency and total energy consumption of the process can be expressed as

[0095]

[0096]

[0097] 2.3) The compute-intensive tasks are forwarded by the upper-layer relay UAV to the available roadside unit RSU far away for computing

[0098] When the computing load ratio of the edge server deployed on exceeds the load threshold , if the time slot is the compute-intensive task offloaded by the vehicle , that is , the high-load will offload the task to the upper-layer relay UAV , and then relay it to the with idle computing resources far away for computing.

[0099] Since the flight altitude of the upper-layer relay UAV is much higher than the roadside unit RSU, ground buildings and other obstacles, the communication link between the upper-layer relay UAV and the roadside unit RSU can be almost completely regarded as LoS propagation. Therefore, the time slot The high-load to the upper-layer relay UAV channel gain and the upper-layer relay UAV to the far away ​Channel gain Can be respectively expressed as

[0100]

[0101]

[0102] Therefore The task Unloaded to the upper-layer relay UAV The transmission rate is

[0103]

[0104] The upper-layer relay UAV To The transmission rate can be expressed as

[0105]

[0106] Among them And Are respectively With the upper-layer relay UAV Between and the UAV With The transmission bandwidth between; And Are respectively To the upper UAV The transmit power and the upper UAV To The transmit power. In order to ensure the reasonable allocation of communication resources, for And the upper UAV The transmit powers are respectively set with the following constraints

[0107]

[0108]

[0109] Among them Is the maximum transmit power of the upper-layer relay UAV . In addition, since the upper-layer relay UAV and the lower-layer computing UAV share 5G wireless resources, the total transmission bandwidth between the roadside unit RSU and all UAVs cannot exceed the total bandwidth Allocated to the communication between the roadside unit RSU and the UAVs, that is

[0110]

[0111] Therefore The task Unloaded to the upper-layer relay UAV The transmission delay and transmission energy consumption are respectively

[0112]

[0113]

[0114] The upper-layer relay UAV transmits the task to The required transmission delay and transmission energy consumption are respectively

[0115]

[0116]

[0117] The computing task The computing delay and computing energy consumption are respectively

[0118]

[0119]

[0120] Among them, is the computing resources allocated for task processing, and its value cannot exceed the maximum available computing resources of .

[0121] In addition, the propulsion power consumption of the upper-layer relay UAV can be expressed as

[0122]

[0123] Among them, represents the moving speed of the upper-layer relay UAV during uniform linear motion, and are respectively the blade profile power and induction power of the upper-layer relay UAV in the hovering state. Assume that the communication coverage radius of the UAV is , so the upper-layer relay UAV the actual moving distance is

[0124]

[0125] Among them, is the horizontal distance between the high load and the available in the distance, then the upper-layer relay UAV the actual moving time is . Therefore, the time slot Upper relay UAV The propulsion energy consumption of

[0126]

[0127] In addition, the hovering power of the upper relay UAV is

[0128]

[0129] Among them, is the mass of the upper UAV; is the radius of the upper UAV propeller; is the number of upper UAV propellers. Therefore, the upper relay UAV in the time slot The hovering energy consumption within is

[0130]

[0131] In summary, the upper relay UAV assists the high load to process computationally intensive tasks The total delay and total energy consumption during the process are respectively

[0132]

[0133]

[0134] AoI model: AoI is a metric used to characterize the freshness of data. It can reflect the "staleness" of data in the system and thus measure the timeliness of information. In applications with extremely high real-time requirements such as vehicle-to-everything (V2X), autonomous driving, and intelligent manufacturing, the data on which the system makes decisions must be the latest. Outdated data may not only lead to incorrect decisions but may even cause system failures. By introducing AoI, the system can dynamically sense and determine whether the data is still of reference value and select to prioritize the processing of data with a larger information age to ensure that the data can be processed or transmitted in a timely manner, thereby optimizing task scheduling and the overall system performance.

[0135] Therefore, this section constructs an AoI model for optimizing task offloading and resource scheduling. Its core lies in enabling the system to real-time sense the AoI status of tasks and dynamically adjust task scheduling decisions according to the changes in AoI to achieve the optimization of information freshness. The present invention considers the evolution process of the AoI model in the zero-waiting state and assumes that the state is updated at the sampling time and . If the task is processed locally by the vehicle, the processing time of the task is the local computing time. At this time ; If the task is calculated by the roadside unit RSU, the total processing time of the task is the sum of the transmission delay and the processing delay. At this time ; When the task is calculated by the lower-layer UAV, the total task processing time is equal to the sum of the task transmission delay from the vehicle to the roadside unit RSU, the task transmission delay from the roadside unit RSU to the lower-layer UAV, and the calculation delay of the lower-layer UAV. Then ; When the task is relayed by the upper-layer UAV to the distant roadside unit RSU for calculation, the total task processing time is equal to the sum of the task transmission delay from the vehicle to the roadside unit RSU, the task transmission delay from the roadside unit RSU to the upper-layer relay UAV, the movement delay of the UAV, the transmission delay from the upper-layer relay UAV to the distant roadside unit RSU, and the calculation delay of the distant roadside unit RSU. At this time . Since the waiting delay is not considered in the present invention, at the completion of the current task , the system can immediately enter the next state, that is, the state update. Assume that at any time , the last state update time is , then the instantaneous AoI in the AoI model can be expressed as

[0136]

[0137] where represents the age of information of the latest information held by the current system, that is, the time interval since the last state update. When the system does not perform any state updates, the AoI increases linearly with time. However, if the system is updated at a certain moment, the AoI value will drop sharply. Therefore, the evolution of the AoI presents a sawtooth shape, as shown in Figure 2 .

[0138] This paper sets sampling points and calculates the long-term average AoI using the following formula

[0139]

[0140] In the AoI change curve, a drop point is formed when each task is completed. The AoI change regions before and after it enclose a right trapezoid. The area of this trapezoid reflects the impact of different tasks on the cumulative AoI of the system. According to different task processing methods, this area can be calculated using the following formula

[0141]

[0142] Through long-term sampling, the average AoI over a long period can be calculated, and its value is equal to the ratio of the total area enclosed by the AoI change curve to the total duration, that is

[0143]

[0144] The goal of this invention is to optimize the vehicle task offloading strategy by jointly optimizing the task delay constraint and the UAV movement restriction condition. , computing resource allocation strategy , transmit power set And bandwidth allocation strategy ,minimize the weighted sum of the system’s average information age and average task processing energy consumption to ensure that information can be processed in a timely manner and energy consumption is managed effectively.

[0145] Handle vehicles according to different treatment methods Generate computing tasks Total time and total energy consumption Can be expressed as

[0146]

[0147]

[0148] It should be noted that the task The completion time of must not exceed its maximum tolerable completion delay, that is, In addition, the average energy consumption of all vehicle generation computing tasks is for

[0149]

[0150] Therefore, the optimization problem can be formulated as

[0151]

[0152] Among them, the constraints Indicates vehicle The generated computing tasks can only be calculated locally, or at most offloaded to a roadside unit RSU for calculation or relay; Indicates that the computing resources allocated to the vehicle's local processing of computing tasks cannot exceed its own maximum computing capacity; and Respectively and distant ones with idle resources For vehicles The computing resources allocated for generating computing tasks shall not exceed the current maximum computing capacity available to the roadside unit RSU; Indicates vehicle The transmission power used for V2R transmission cannot exceed its maximum transmission power; and Guaranteed high load Offload tasks to the upper-layer relay drones and the lower-layer computing drones The transmission power cannot exceed its maximum transmission power; Ensure that the upper-layer relay drones forward tasks to available ones far away The transmission power cannot exceed its own maximum transmission power; It is stipulated that the total communication bandwidth allocated to all vehicles shall not exceed the total bandwidth of the system for V2R communication ; It is stipulated that the total communication bandwidth between the roadside unit RSU and the double-layer drones shall not exceed the total bandwidth allocated to the communication link between the roadside unit RSU and the drones ; It is stipulated that the position of the upper-layer relay drones must be higher than that of the lower-layer computing drones; Ensure that at any moment , the distance between any two drones must not be less than the minimum safety distance; It is stipulated that the completion time of the task shall not exceed the maximum tolerable completion delay of the task; It is stipulated that the weighting factor and must be non-negative.

[0153] Based on the above model, the present invention describes a method for computing offloading and resource scheduling of a double-layer drone-assisted vehicle network based on the age of information.

[0154] The model solving module mainly designs a method to solve according to the model established by the model establishment module. Since this problem is a mixed-integer non-linear programming problem, traditional optimization methods face the problems of low computational efficiency and being prone to falling into local optimal solutions in the case of large scale. In addition, the strong dynamics of the double-layer drone-assisted vehicle network environment leads to a huge action dimension space for this optimization problem. More difficultly, considering the task execution delay and information transmission overhead, it is not realistic to obtain global state information.

[0155] Therefore, in response to this challenge, the present invention proposes a double-layer drone-assisted vehicle network task offloading and resource scheduling algorithm MATD3-DLUTORS based on MATD3 to solve the multi-user multi-server offloading problem. Its cooperative learning mechanism enables each agent to take coordinated actions in different states to achieve system-level goals. This algorithm can learn and adapt to the optimal resource scheduling strategy in a dynamic environment, thereby effectively handling non-convex optimization problems and improving the solving efficiency. Next, the present invention will introduce the Markov process and the double-layer drone-assisted vehicle network computing offloading and resource scheduling MATD3-DLUTORS algorithm based on the MATD3 algorithm in sequence.

[0156] MDP process:

[0157] The present invention first reformulates the problem P of task offloading and resource scheduling in a two-layer UAV-assisted vehicular network into a Markov decision process, and represents it with a quadruple The specific meanings of each element are as follows:

[0158] (1) Agent set : In the two-layer UAV-assisted vehicular network system of the present invention, the agent set represents all the entities that participate in decision-making and interact with the shared environment. Therefore, the agent set includes N vehicles, M roadside units (RSUs), O lower-layer computing UAVs, and U upper-layer relay UAVs, with a total of . Therefore, the agent set can be specifically represented as

[0159] . Each agent has its own state space and action space , and takes corresponding actions according to the current state and policy, and completes the computing task offloading and resource scheduling through continuous interaction.

[0160] (2) State space : At time slot , all agents continuously explore the environment to obtain the current environmental state information , which mainly includes: the set of task data volumes to be transmitted during task offloading ; the set of CPU cycles required to complete a unit bit of task data ; the set of maximum tolerable delays of tasks ; the set of task types ; time slot , the signal-to-noise ratio set of vehicles to roadside units (RSUs) ; time slot , the computing load set of roadside units (RSUs) . Therefore, the state space can be represented as

[0161]

[0162] (3) Action space : The action space of the agent covers the agent's offloading decision, computing resource allocation, and the set of transmit power and bandwidth allocation during data transmission. Therefore, the action space of the agent can be represented as:

[0163]

[0164] Among them, is the time slot Vehicle unloading decision set; and respectively represent the time slot Set of computing resources allocated by the agent to process tasks generated by vehicles; and are the transmission power and transmission bandwidth sets from the vehicle to the roadside unit RSU, respectively; and respectively represent the time slot Set of transmission power and transmission bandwidth from the high-load roadside unit RSU to the lower-layer computing UAV; and respectively represent the time slot Set of transmission power and transmission bandwidth from the high-load roadside unit RSU to the upper-layer relay UAV; and respectively represent the time slot Set of transmission power and transmission bandwidth from the upper-layer relay UAV to the distant roadside unit RSU.

[0165] (4) Reward function : To solve the optimization problem P under multiple constraint conditions, the design of the reward function includes four parts: age of information, task processing energy consumption, task completion time, and distance limit.

[0166] First of all, the goal of the optimization problem P is to minimize the weighted function of the long-term average age of information and average task processing energy consumption of the system. Therefore, the age of information and energy consumption penalty terms are added to the reward function, which are expressed as follows

[0167]

[0168]

[0169] In the formula, and are the reward weight coefficients of the age of information and energy consumption, respectively, used to adjust their importance in the reward function.

[0170] In addition, in order to meet the constraint condition , that is, to ensure that the completion delay of the task does not exceed the maximum tolerable delay of the task, the reward function needs to consider the delay limit penalty. When the task processing time exceeds its maximum tolerable delay, the delay penalty is triggered:

[0171]

[0172] Among them, is the indicator function. When the task Processing time exceeds its maximum tolerable completion time delay Take the value of 1 when it does, otherwise 0; is the penalty coefficient used to control the penalty intensity when the time delay limit is exceeded.

[0173] Finally, to ensure the safety of the multi-UAV environment, the reward function considers the penalty for distance constraints. When the distance between any two UAVs is less than the minimum safety distance a distance penalty is triggered:

[0174]

[0175] where is an indicator function that takes the value of 1 when the distance between UAV and is less than the minimum safety distance otherwise 0; is the penalty coefficient used to control the penalty intensity when the distance limit is exceeded. Therefore, the reward function for time slot can be expressed as

[0176]

[0177] Double-Layer UAV-Assisted Vehicular Network Task Offloading and Resource Scheduling Algorithm Based on MATD3

[0178] MATD3 is an extension of the multi-agent deep deterministic policy gradient (MADDPG) algorithm. By training each agent, this algorithm can capture the non-linear relationship between vehicle computing task offloading, computing resource allocation, transmit power allocation, and bandwidth allocation, learn the trade-off between different objectives, and thus find the optimal solution. By continuously interacting with the environment to update the model, this algorithm can generate decisions within a short time to meet real-time requirements and is more suitable for solving multi-agent task offloading and resource scheduling problems.

[0179] Based on the MATD3 algorithm, the present invention proposes a double-layer UAV-assisted vehicular network task offloading and resource scheduling algorithm MATD3-DLUTORS, as Figure 3 shown. In this algorithm, each vehicle, each roadside unit (RSU), and each UAV act as an independent agent, responsible for collecting observation information from the double-layer UAV-assisted vehicular network system. Subsequently, based on these observations, each agent determines and executes the optimal action according to the reward.

[0180] Similar to the MADDPG algorithm, the MATD3-DLUTORS algorithm adopts the Actor-Critic architecture of multi-agent reinforcement learning. Among them, the Actor network selects actions based on the current agent state, and the Critic network is responsible for evaluating the value of the current state or action. The goal of the Actor network is to learn a mapping function that can generate the optimal strategy of the agent based on the centralized training and distributed execution paradigm. The Critic network is deployed in the cloud server, and the Critic network of each agent is updated in the centralized training mode. In this mode, the input of the Critic network of the agent not only includes its own observations and actions , but also includes the observations and actions of all other agents, and this information is obtained through the communication between vehicles, drones, and the roadside unit RSU and the cloud server. The input of the Critic network is represented as

[0181]

[0182] The Critic network can consider the behaviors of other agents during the training process, so as to better evaluate actions and improve the stability of learning strategies in dynamic environments. In the distributed execution stage, the Actor network can output the optimal action only based on its own observations and the trained model network parameters.

[0183] The main difference from the MADDPG algorithm is that the MATD3-DLUTORS algorithm adopts a double delayed update mechanism to improve the stability of the learning process. Each agent adopts the TD3 algorithm, which includes an Actor network with as the parameter and two Critic networks with and as the parameters and . Among them, the Critic network and the Critic network run independently, and the smaller value of their outputs is used to provide an estimate of the Q value. This mechanism helps to stabilize the learning process and improve the accuracy of Q value estimation. In addition, to enhance the stability of the learning process, the MATD3-DLUTORS algorithm adopts a target Actor network and target Critic networks and .

[0184] The present invention summarizes the specific steps of this algorithm in Algorithm 1. First, initialize the six neural network parameters and the experience replay buffer of each agent ​. The MATD3-DLUTORS algorithm uses experience replay to improve sample efficiency and adds noise to the output of the Actor network to promote exploration. The action of the agent can be expressed as:

[0185]

[0186] where is the noise with a mean of 0 and a standard deviation of , and the noise decreases as the number of training iterations increases. By adopting the target policy smoothing technique, that is, introducing random noise into the target action, the exploration of the policy can be increased, the dependence on the Q-function error can be reduced, the predicted value of the Critic network can be made more accurate, and the stability of the algorithm can be improved.

[0187] Each agent executes the action , receives the next observation and the immediate reward , and stores the experience in its experience replay buffer to update the network parameters. The training process of the neural network alternates with the above-mentioned environment interaction process. When the number of experiences stored in the replay buffer reaches a sufficient amount, the neural network will be trained. Then each agent randomly samples from mini-batch transition samples . To update the parameters of the Actor network and the Critic network, the MATD3-DLUTORS algorithm adopts the policy gradient method, and its loss function aims to minimize the difference between the predicted Q-value and the actual Q-value. The loss function is defined as:

[0188]

[0189] where is the target value. Since the TD3 algorithm introduces the double-delayed learning mechanism and the double-Q network architecture, two Q-functions are learned in parallel, and the smaller value predicted by these two Q-functions is used as the target Q-value, thus effectively alleviating the problem of overestimating the Q-value. Therefore, the target value can be expressed as

[0190]

[0191] where and ; is the discount factor, which represents the emphasis on the rewards obtained at future moments. The smaller the discount factor , the more it indicates caring about the rewards in the current state; conversely, the discount factor The larger it is, the more attention is paid to the reward of the future state.

[0192] The Actor network uses the deterministic policy gradient of Q-values to maximize the action value, and its gradient can be expressed as:

[0193]

[0194] Then, each agent updates the network parameters. The update formulas for the Actor network and the two Critic networks are as follows:

[0195]

[0196]

[0197] Among them, is the learning rate. After each round of network parameter update, the parameters of the target network need to be updated using the soft update method. The specific method is as follows:

[0198]

[0199]

[0200] Among them, is the update speed of the soft update of the target Actor network and the target Critic network.

[0201] As shown in Algorithm 1, the MATD3-DLUTORS algorithm sets Z training rounds. Step 1 initializes the parameters of the Actor network, the parameters of the Critic network, and the experience replay pool; Step 2 initializes the simulation parameters and the environment of the two-layer UAV-assisted vehicle networking system; Steps 3-5 represent the operations at the beginning of each training cycle, including resetting the environmental state, re-initializing the simulation parameters of the two-layer UAV-assisted vehicle networking system, and setting the action exploration noise. Among them, the introduction of the action exploration noise is beneficial for the agent to better explore the action space and avoid falling into local optimal solutions; Steps 8-9, the agent selects an action according to the observed state , obtains the reward value and the next state ; Steps 10-13 describe the experience replay pool mechanism, that is, if the experience in the experience replay pool is not full, the experience is stored in the experience replay pool, otherwise the older experience tuple in the experience replay pool is replaced by the experience; Steps 18-21, a certain number of samples are drawn from the experience replay pool, and the target value and the loss function are calculated based on these sample data, and then the parameters of the Actor network and the Critic network are updated, and the soft update method is used to update the parameters of the three target networks.

[0202]

[0203] Beneficial effects:

[0204] An embodiment of the present invention proposes a two - layer UAV - assisted vehicular network architecture based on age of information. By introducing upper - layer relay UAVs and lower - layer computing UAVs, it collaboratively processes computationally intensive and latency - sensitive tasks that are difficult for high - load roadside units (RSUs) to handle alone. Aiming at the task offloading and resource scheduling problems in this scenario, the present invention constructs a mixed - integer non - linear programming optimization problem with the goal of minimizing the weighted sum of the system's average age of information and average energy consumption, and proposes a two - layer UAV - assisted vehicular network task offloading and resource scheduling algorithm MATD3 - DLUTORS based on MATD3, successfully achieving an effective reduction in the system's average age of information and energy consumption.

[0205] Through simulation experiments, the present invention verifies the superiority of the MATD3 - DLUTORS algorithm in terms of information freshness, energy consumption control, and load balancing of roadside units (RSUs). The experimental results show that the two - layer UAV architecture can more efficiently share and process computing tasks in a dynamic vehicular network environment, thereby reducing the system's average AoI and energy consumption, and at the same time optimizing the resource utilization of roadside units (RSUs). Compared with existing benchmark methods, the MATD3 - DLUTORS algorithm shows stronger adaptability and more superior performance in various scenarios. Especially in high - load and large - scale vehicle environments, it demonstrates its excellent robustness and efficiency.

[0206] For example, Figure 4 Three figures respectively show the performance of five schemes in terms of average AoI, average energy consumption, and the weighted sum of the two under different numbers of vehicles. The simulation results show that as the number of vehicles increases, the average AoI, average energy consumption, and the weighted sum of the two of all algorithms show an upward trend. Among them, the MATD3 - DLUTORS algorithm performs the most prominently in the comprehensive performance of average energy consumption and information timeliness. When the number of vehicles increases, the growth of energy consumption and age of information under this algorithm is the steadiest. In contrast, the performance of the MADQN and MADDPG algorithms is slightly inferior to that of the MATD3 - DLUTORS algorithm. For example, when the number of vehicles is 50, the weighted sum of the average AoI and average energy consumption of the MATD3 - DLUTORS algorithm is 7.4% and 14.6% lower than those of the MADDPG algorithm and the MADQN algorithm, respectively. In addition, the strategies of using only lower - layer or upper - layer UAVs perform significantly worse as the number of vehicles increases. Specifically, the scheme of only deploying lower - layer UAVs performs better in terms of information timeliness, but the energy consumption increases rapidly, resulting in an impact on the comprehensive performance; the scheme of only deploying upper - layer UAVs performs relatively stably in terms of energy consumption and comprehensive performance, but the information timeliness is poor.

[0207] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which when executed by a processor causes the processor to execute the steps of the above method.

[0208] In yet another aspect, the present invention also discloses a computer device including a memory and a processor, where the memory stores a computer program, which when executed by the processor causes the processor to execute the steps of the above method.

[0209] In another embodiment provided by the present application, there is also provided a computer program product containing instructions, which when running on a computer causes the computer to execute any of the dual-layer UAV-assisted vehicle network task offloading and resource scheduling methods in the above embodiments.

[0210] It can be understood that the systems, devices, and storage media provided by the embodiments of the present invention correspond to the methods provided by the embodiments of the present invention. Explanations, examples, and beneficial effects of related content can refer to the corresponding parts in the above methods.

[0211] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrating one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state disk (SSD)).

[0212] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0213] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiment.

[0214] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for task offloading and resource scheduling in a two - layer UAV - assisted vehicle - to - everything network, characterized in that, It includes the following steps: Step 1: Construct a scenario for the task offloading and resource scheduling of a two - layer UAV - assisted vehicle - to - everything (V2X) network; Step 2: According to different task calculation locations, establish a task processing time model, an energy consumption model, and an age - of - information model respectively; and based on different offloading methods, combined with the allocation strategies of transmit power, computing resources, and bandwidth resources, construct an optimization problem aiming to minimize the weighted sum of the system - average age of information and energy consumption; Step 3: According to the established models, design a method for solving; propose a MATD3 - based two - layer UAV - assisted V2X task offloading and resource scheduling algorithm MATD3 - DLUTORS to solve the multi - user multi - server offloading problem; In Step 1, the scenario construction for the task offloading and resource scheduling of a two - layer UAV - assisted V2X network is as follows: Consider a city topology covered by roadside units (RSUs), and define the set of RSUs as . Each RSU is equipped with an edge server to process the computing tasks generated by vehicles. Suppose there are vehicles running on the road, , and denote as the set of vehicles. Discretize the system time into a series of time slots , and the duration of each time slot is . Within each time slot, each vehicle generates a computing task, which is processed locally on the vehicle or offloaded to an RSU via the 5G uplink for processing. Design a two - layer UAV - assisted vehicle - to - everything (V2X) architecture to share and process computational tasks that cannot be completed by high - load roadside units (RSUs). The task assignment and coordination between the RSUs and UAVs are carried out through a 5G network. Among them, the upper - layer relay UAVs are located at a higher position and are responsible for relaying computationally intensive tasks to distant idle RSUs for execution; while the lower - layer computing UAVs carry edge servers and hover at a lower altitude to directly process latency - sensitive tasks; define the UAV set as , where is the set of upper - layer relay UAVs, is the set of lower - layer computing UAVs; High load is represented as , the upper relay UAV and the lower computing UAV at time slot are respectively represented as and , and ; According to the characteristics of tasks, the computing tasks generated by vehicles are divided into two categories: computationally intensive tasks and latency - sensitive tasks; Whether it is a computationally intensive task or a latency - sensitive task, it must be completed within its maximum tolerable latency; Based on the above task characteristics, the vehicle generated computational tasks are described as a four-tuple vector ; where is the number of CPU cycles required for the computational task per unit bit of data; is the size of the input data for the task ; is the maximum completion time delay that the task can tolerate; represents the type of the task , indicating that the task is a compute-intensive task, indicating that the task is a latency-sensitive task; Task by the vehicle locally calculated or offloaded to the roadside unit RSU via the V2R wireless communication link for processing; among them, the access mechanism between the vehicle and the roadside unit RSU is determined based on the signal-to-noise ratio and the current load of the roadside unit RSU; To balance the signal-to-noise ratio and the load of roadside unit (RSU), a comprehensive scoring function is introduced , taking the vehicle and as an example. Its comprehensive scoring function is defined as: Among them, is the vehicle and the signal-to-noise ratio between; is the current load ratio of, is the current computing load, is the maximum computing power; and are the weight coefficients of the signal-to-noise ratio and the load ratio respectively; Introduce the load ratio threshold of roadside unit (RSU) to represent the upper limit of the ratio of the calculated load of the roadside unit (RSU) to its maximum computing capacity, and introduce drones to share and process excessive computing tasks on high-load roadside units (RSUs); Adopt a two-layer UAV-assisted vehicle networking architecture to optimize the offloading of computing tasks, improve the system response speed, and effectively balance the computing load of roadside units (RSUs); specifically, when the load ratio of a certain roadside unit (RSU) does not exceed the load ratio threshold, that is it indicates that the roadside unit (RSU) is in a normal state, and the computing tasks offloaded by vehicles to this roadside unit (RSU) will be calculated by the roadside unit (RSU); when the load ratio of a certain roadside unit (RSU) exceeds the load ratio threshold, that is it indicates that the roadside unit (RSU) is already in a high-load state. The system will dynamically select an appropriate UAV for assisted computing or task forwarding according to the delay requirements and computing resource requirements of the current task, ensuring that the system can take effective measures when the load of the roadside unit (RSU) is high but not overloaded, thereby avoiding a sharp decline in performance; Step 2 includes establishing a vehicle task execution model for a two - layer UAV - assisted V2X network to obtain the final task processing time and energy consumption, specifically including: S2.1: The vehicle performs task calculation locally to obtain the local processing time and total energy consumption; S2.2: The vehicle selects the roadside unit (RSU) to access according to the signal - to - noise ratio and the current load of the roadside unit RSU, and offloads the task to the roadside unit RSU for calculation or relays it to the corresponding UAV; S2.2.1: When the roadside unit RSU is in a normal state, the computing tasks generated by the vehicle will be processed on the roadside unit RSU; S2.2.2: When the roadside unit RSU is in a high - load state, if the vehicle task is a latency - sensitive task, it will be directly calculated by the lower - layer UAV; S2.2.3: When the roadside unit RSU is in a high - load state, if the vehicle task is a computationally intensive task, the upper - layer UAV will relay the task to a distant idle roadside unit RSU for processing; Step 3 includes: proposing a MATD3 - based two - layer UAV - assisted V2X task offloading and resource scheduling algorithm MATD3 - DLUTORS to solve the multi - user multi - server offloading problem, and its cooperative learning mechanism enables each agent to take collaborative actions in different states to achieve system - level goals; First, the problem P of task offloading and resource scheduling in a two-layer UAV-assisted vehicle network is reformulated as a Markov decision process and represented by a quadruple The specific meanings of each element are as follows: Agent set : In the two - layer UAV - assisted vehicle - to - everything (V2X) system of the present invention, the agent set represents all the entities that participate in decision - making and interact with the shared environment; the agent set contains N vehicles, M roadside units (RSUs), O lower - layer computing UAVs, and U upper - layer relay UAVs, with a total number of ; the agent set is specifically represented as ; each agent has its own state space and action space , and takes corresponding actions according to the current state and policy, and completes the computing task offloading and resource scheduling through continuous interaction; State space : At time slot , all agents continuously explore the environment to obtain the current environmental state information , including: the set of task data volumes to be transmitted when task offloading ; the set of CPU cycles required to complete a unit bit of task data ; the set of maximum tolerable latencies of tasks ; the set of task types ; time slot , the signal-to-noise ratio set of vehicle to roadside unit RSU ; time slot , the computing load set of roadside unit RSU ; state space is expressed as: Action space : The action space of the agent covers the agent's offloading decision, computing resource allocation, and the set of transmit power and bandwidth allocation during data transmission; therefore, the action space of the agent is expressed as: Among them, is a time slot vehicle unloading decision set; and respectively represent the set of computing resources allocated by the intelligent agent to process the tasks generated by vehicles in the time slot ; and are respectively the transmission power and transmission bandwidth set from the vehicle to the roadside unit RSU; and respectively represent the set of transmission power and transmission bandwidth from the high-load roadside unit RSU to the lower-layer computing UAV in the time slot ; and respectively represent the set of transmission power and transmission bandwidth from the high-load roadside unit RSU to the upper-layer relay UAV in the time slot ; and respectively represent the set of transmission power and transmission bandwidth from the upper-layer relay UAV to the remote roadside unit RSU in the time slot ; Reward function : To solve the optimization problem P under multiple constraint conditions, the design of the reward function includes four parts: age of information, task processing energy consumption, task completion time, and distance limit; First, the objective of the optimization problem P is to minimize the weighted function of the long - term average age of information and average task - processing energy consumption of the system; therefore, an age - of - information and energy - consumption penalty term is added to the reward function, which is expressed as follows: In the formula, and are respectively the reward weight coefficients of the information age and the energy consumption, which are used to adjust their importance in the reward function; To ensure that the completion delay of a task does not exceed the maximum tolerable delay of the task, the reward function needs to consider the delay limit penalty; when the processing time of a task exceeds its maximum tolerable delay, a delay penalty is triggered: Among them, is an indicator function. When the task processing time exceeds its maximum tolerable completion time delay it takes the value of 1, otherwise 0; is a penalty coefficient used to control the penalty intensity when the time delay limit is exceeded; Finally, to ensure the safety of the multi-UAV environment, the reward function considers the penalty for distance limits; when the distance between any two UAVs is less than the minimum safety distance , a distance penalty is triggered: Among them, is an indicator function, which takes the value of 1 when the distance between the UAV and is less than the minimum safety distance , and 0 otherwise; is a penalty coefficient used to control the penalty intensity when the distance limit is exceeded; Therefore, the reward function of time slot is expressed as 。 2. The method for dual-layer UAV-assisted vehicle network task offloading and resource scheduling according to claim 1, wherein: Step one includes setting a minimum distance to avoid potential risks of drone collisions; Therefore, any two drones and , and at any given time should satisfy the distance constraint Define the vehicle offloading policy set , when , it means that the vehicle offloads the task to for computing or relays it to the UAV; when , it means that the vehicle does not offload the task to ; means that the vehicle selects to locally process the task , means that the vehicle generates a computing task and does not process it locally.

3. The method for dual-layer UAV-assisted vehicle network task offloading and resource scheduling according to claim 2, wherein: Step 1 also includes: To ensure that each vehicle can only choose to process the computing tasks generated by itself locally or offload the tasks to at most one roadside unit RSU, then there is In addition, use to represent the auxiliary decision-making set of the upper-layer relay UAV, and represents the auxiliary decision-making set of the lower-layer computing UAV; when the upper-layer relay UAV relays the compute-intensive task from the high-load to the remote roadside unit RSU for processing; conversely, when the high-load does not offload the task to the upper-layer UAV for relaying; when the lower-layer computing UAV directly provides computing services for the high-load ; conversely, when the high-load does not offload the task to the lower-layer UAV for computing.

4. The method for dual-layer UAV-assisted vehicle network task offloading and resource scheduling according to claim 1, characterized in that: Step 2 includes: (1) The vehicle performs local calculation If the vehicle selects to locally process the computing task , that is , the computing time and energy consumption required for it to execute the task are wherein, is the effective capacitance constant of the vehicle; is the computing resources allocated to the local computing task by the vehicle, and its value cannot exceed the maximum computing power of the vehicle , that is ; (2) The vehicle offloads the task to the roadside unit RSU Assume that the wireless communication between the vehicle and the roadside unit (RSU) is based on orthogonal frequency division multiple access (OFDMA), where different users are separated in the frequency domain. Then, the to uplink task transmission rate of the vehicle is expressed as wherein, represents the bandwidth resource allocated to the vehicle ; the total bandwidth allocated to all vehicles shall not exceed the total bandwidth of the system for V2R communication , that is Let be the maximum transmission power of the vehicle . Then the transmission power of the vehicle shall satisfy: Therefore, the vehicle unloads the task to The transmission delay is as follows: Vehicle Upload calculation task The corresponding transmission energy consumption is: 2.1) The task is calculated on the roadside unit RSU Execute the task The computing latency and computing energy consumption are respectively as follows: Among them, represents the computing resources allocated for the task whose value cannot exceed the maximum available computing resources, that is ; is the effective capacitance constant of the roadside unit RSU; 2.2) The latency - sensitive task is calculated on the lower - layer UAV When deployed on the edge server, if the computing load ratio exceeds the load threshold and the time slot is for a computing task unloaded by a vehicle and the task is delay-sensitive, that is when the load is high the task will be unloaded to the lower-layer computing drone for computing; ​ When the UAV flies at a set altitude and communicates with the roadside unit (RSU), the communication channel is mainly affected by the LoS transmission path. Therefore, the free space path loss model is used for the channel between the lower-layer computing UAV and the RSU; within the time slot the lower-layer computing UAV hovers above the high load to receive and execute latency-sensitive tasks , and the probability that they can establish a LoS channel is: Among them, and are environment-related parameters; is a time slot The lower-layer computing UAV and the high load The elevation angle between them; therefore, To the lower-layer computing UAV The uplink channel gain is expressed as: Among them, is the unit channel gain when the distance between the roadside unit RSU and the UAV is 1 m and the transmission power is 1 W; the parameter is the attenuation factor of the non-line-of-sight channel; therefore, offload the latency-sensitive task to the lower-layer computing UAV The effective transmission rate is: Among them, represents the transmission bandwidth between and the lower-layer computing UAV; represents the transmission power to the lower-layer computing UAV and this value shall not exceed the maximum transmission power of i.e., Unload the task to the lower-layer computing UAV The transmission delay and transmission energy consumption are respectively expressed as: Lower-layer computing UAV Auxiliary high load Processing tasks The computing delay of is as follows: Among them, is the CPU frequency of the processor of the lower-layer computing UAV. Assuming that the computing capabilities of all lower-layer UAVs are the same, the UAV computing task computing task The energy consumed is: In the formula, is the effective capacitance coefficient of the edge server chip carried by the lower-layer computing UAV; In addition, for high load , the lower-layer computing UAV maintains a hovering state within a limited time to receive and process data. The hovering power of the lower-layer UAV is expressed as: Among them, is the mass of the lower-layer drone; is the radius of the propeller of the lower-layer drone; is the number of propellers of the lower-layer drone; correspondingly, the hovering energy consumption of the lower-layer drone is expressed as follows: In summary, the lower-layer computing UAV receives and computes high-load offloaded latency-sensitive tasks The total latency and total energy consumption of the process are respectively expressed as: 2.3) The computationally intensive task is forwarded by the upper - layer relay UAV to a distant available roadside unit RSU for calculation When the computing load ratio of the edge server deployed on exceeds the load threshold and if the time slot is for a computing task unloaded by a vehicle and the computing task is a compute-intensive task, that is , the high-load will offload the task to the upper-layer relay UAV and then relay it to a distant with idle computing resources for computing; Time slot High load To the upper-layer relay UAV Channel gain And the upper-layer relay UAV To a distance Channel gain Are respectively expressed as: Therefore, offload the task to the upper-layer relay UAV The transmission rate is Upper relay UAV to The transmission rate is expressed as: Among them, and are respectively the transmission bandwidth between the upper-layer relay UAV and the UAV and ; and are respectively the transmission power from the UAV to the upper-layer UAV and the transmission power from the upper-layer UAV to ; To ensure the reasonable allocation of communication resources, for and the upper-layer UAV the transmission powers are respectively set with the following constraints Among them, is the maximum transmit power of the upper-layer relay UAV ; in addition, since the upper-layer relay UAV and the lower-layer computing UAV share 5G wireless resources, the total transmission bandwidth between the roadside unit RSU and all UAVs cannot exceed the total bandwidth allocated for the communication between the roadside unit RSU and the UAVs , that is Therefore, the time slot unloads the task to the upper-layer relay UAV with the transmission delay and transmission energy consumption respectively as follows: Upper relay UAV Forward the task to The required transmission delay and transmission energy consumption are respectively: Computing task The computing latency and computing energy consumption are respectively as follows: Among them, is the computing resource for processing task allocation, and its value cannot exceed the maximum available computing resource, that is ; In addition, the propulsion power consumption of the upper-layer relay UAV is expressed as: Among them, represents the moving speed when the upper relay UAV moves at a uniform linear speed, and are respectively the blade profile power and the induction power of the upper relay UAV in the hovering state; Let the communication coverage radius of the UAV be , so the upper-layer relay UAV actual moving distance is: Among them, is the high load and the horizontal distance available from afar , then the actual moving time of the upper-layer relay UAV is ; Therefore, the time slot for the upper-layer relay UAV The propulsion energy consumption is: In addition, the hovering power of the upper-layer relay UAV is as follows: Among them, is the mass of the upper - layer UAV; is the radius of the propeller of the upper - layer UAV; is the number of propellers of the upper - layer UAV; Therefore, the upper - layer relay UAV in the time slot the hovering energy consumption within is: In summary, the upper-layer relay UAV assists the high payload to process computationally intensive tasks The total delay and total energy consumption of the process are respectively 。 5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 4.

Images