A method for offloading vehicular network-dependent tasks based on physical benchmark-guided long-term differential learning

By introducing physical benchmarks and differential learning methods into the vehicle-to-everything (V2X) environment, the counterfactual physical cost of candidate execution locations is evaluated and differential learning is performed. This solves the problems of learning difficulty and insufficient information utilization in V2X task offloading under high dynamic environments, and achieves more stable and efficient task offloading decisions.

CN123114680APending Publication Date: 2026-11-17SOUTHWEAT UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611382241.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-08
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing vehicle-to-everything (V2X) task offloading methods struggle to adapt quickly to state changes in highly dynamic and non-stationary environments. The lack of physical benchmark guidance leads to significant learning difficulties and insufficient utilization of candidate execution location information, impacting the stability and efficiency of offloading decisions.

Method used

The physical benchmark-guided long-term differential learning method determines candidate execution positions through counterfactual physical cost evaluation, reconstructs the network using differential value for long-term differential learning, and combines simulated and real execution samples for network training, thereby improving sample utilization and decision adaptability.

Benefits of technology

It achieves more stable learning and task offloading decisions in highly dynamic vehicle networking environments, improves sample utilization efficiency and decision adaptability, reduces learning difficulty, and improves the stability and efficiency of offloading.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123114680A_ABST
    Figure CN123114680A_ABST
Patent Text Reader

Abstract

This invention discloses a method for offloading vehicular network (V2N) dependent tasks based on physical benchmark-guided long-term differential learning. The method determines the current subtask to be processed based on dynamic remaining critical timeliness, latency urgency, and subsequent release benefits. Counterfactual physical evaluations are performed on candidate execution locations at local, edge, and cloud levels to obtain feasibility, physical execution cost, and physical process description. A physical benchmark is selected to guide the differential value reconstruction network to learn the long-term differential components of candidate locations, and the target offloading location is determined based on the reconstruction value. Simulated candidate action samples are generated using isolated state replicas and updated together with real samples. This invention addresses the problems of balancing execution cost and long-term impact in highly dynamic V2N environments, the difficulty of value learning, and insufficient utilization of candidate action information. It can guide long-term differential learning using current physical costs, improving the stability of value learning and the efficiency of sample utilization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of edge computing technology for vehicle-to-everything (V2X) networks. More specifically, it relates to a method for offloading V2X-dependent tasks based on physical benchmark-guided long-term differential learning. Background Technology

[0002] With the rapid development of vehicle-to-everything (V2X) technology, applications such as autonomous driving, high-definition maps, intelligent navigation, and vehicle-to-infrastructure (V2I) communication generate a large number of computationally intensive and latency-sensitive tasks. Limited by the computing power and energy capacity of onboard terminals, vehicles can offload tasks to Roadside Units (RSUs) or cloud servers to alleviate local computing pressure. However, vehicle movement, wireless channel fluctuations, dynamic task arrival, and changes in RSU resource and queue status cause V2X edge computing systems to exhibit significant high dynamic and non-stationary characteristics. In this environment, the transmission latency, resource waiting time, computation latency, and energy consumption of different offloading actions continuously change with the system state.

[0003] In task offloading technology, research methods are mainly divided into two categories: one is traditional optimization methods, which model the task offloading problem as a multi-objective optimization problem and solve the optimal offloading strategy through methods such as convex optimization, linear programming, and heuristic algorithms. The main methods include greedy algorithms, particle swarm optimization, genetic algorithms, and game theory methods. The other category is task offloading methods based on deep reinforcement learning, which mainly include deep reinforcement learning algorithms such as deep Q-networks (DQN), deep deterministic policy gradient (DDPG), and proximal policy optimization (PPO). These methods can adapt to dynamic network environments and learn the optimal offloading strategy by interacting with the environment.

[0004] Palaniappan et al. [Palaniappan SC, Ponnuswamy P P. Task offloading inedge computing using integrated particle swarm optimization and genetic algorithm[J]. Advances in Science and Technology. Research Journal, 2025, 19(1): 371-380.] proposed a task offloading method that integrates particle swarm optimization and genetic algorithm. This method searches for candidate offloading schemes through particle swarm optimization and enhances the search capability by combining crossover and mutation operations of the genetic algorithm, thereby optimizing task allocation, resource utilization, latency, and energy consumption. This method is suitable for static or quasi-static offloading optimization scenarios, but it mainly relies on a pre-designed fitness function and iterative search process, making it difficult to quickly reflect real-time state changes in vehicle-to-everything (V2X) networks, such as changes in edge node queues, communication link fluctuations, vehicle movement, and task deadline pressures.

[0005] Paknejad et al. [Paknejad M, Moshiri PF, Simsek M, et al. Meetingdeadlines in motion: Deep RL for real-time task offloading in vehicular edgenetworks[C] / / 2025 16th International Conference on Network of the Future.IEEE, 2025: 134-141.] proposed a deep reinforcement learning method for real-time task offloading in vehicular edge networks. This method can adapt to dynamic vehicular network environments to a certain extent, and its offloading actions mainly rely on the action values ​​learned by the reinforcement learning network based on historical interactions. However, in highly dynamic and non-stationary vehicular network environments, there are still problems such as unstable training and slow convergence.

[0006] Existing vehicle-to-everything (V2X) task offloading methods still suffer from the following shortcomings in highly dynamic and non-stationary environments: First, offloading methods based on current latency, energy consumption, or overall cost, while quickly determining the optimal execution position, rarely consider the impact of current decisions on subsequent task execution and resource status. Deep reinforcement learning methods, although capable of considering long-term benefits, primarily learn their offloading strategies through historical interactions. When vehicle movement, communication links, and resource loads change rapidly, existing experience may differ from the current operating state, requiring gradual adjustments through new interactions. Second, existing deep reinforcement learning methods lack guidance for network learning using current physical execution costs. As vehicle positions, communication links, and resource statuses continuously change, the network needs to simultaneously adjust its estimates of current costs and long-term impacts, increasing learning difficulty. Third, during deep reinforcement learning training, typically only one offloading action is executed and feedback obtained at a time, failing to fully utilize information from other candidate actions under the same conditions, resulting in low training sample utilization. Therefore, a V2X task offloading method is needed that can determine a physical benchmark based on the current system state, use this benchmark to guide the network in learning the long-term differences of candidate offloading actions, and fully utilize information from other candidate actions. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies. In the context of highly dynamic and non-stationary vehicle network environments, the offloading strategy needs to constantly adapt to state changes, the current execution cost and long-term impact are difficult to learn together, and the candidate execution location information is not fully utilized. This invention provides a vehicle network-dependent task offloading method based on physical benchmark-guided long-term differential learning. When faced with vehicle movement, link changes and continuous fluctuations in resource load, it can achieve more stable learning, more full utilization of samples, and more adaptive task offloading decisions.

[0008] To achieve the above-mentioned objectives, this invention provides a method for offloading vehicle-to-everything (V2X) dependent tasks based on physical benchmark-guided long-term differential learning, with the following specific steps:

[0009] S1: Based on the current system status of the vehicle-to-everything (V2X) edge computing system and the dynamic remaining critical timeliness, latency urgency, and subsequent release benefits of ready subtasks, determine the subtasks to be processed.

[0010] S2: In the current system state, perform counterfactual physical cost evaluation on the candidate execution locations of the current subtask to be processed in the vehicle local, edge node and cloud, and obtain the feasibility, counterfactual physical execution cost and physical process description of each candidate execution location;

[0011] S3: Using the feasible candidate execution position with the lowest counterfactual physical execution cost as the physical benchmark, calculate the counterfactual physical execution cost difference and physical process description difference of the remaining feasible candidate execution positions relative to the physical benchmark; determine the long-term difference component by the differential value reconstruction network, and combine the long-term difference component with the counterfactual physical execution cost of the corresponding candidate execution position to form the reconstruction value;

[0012] S4: Determine the feasible candidate execution location with the greatest reconstruction value as the target unloading location, and execute the corresponding target unloading scheme;

[0013] S5: During training, taking the current system state as the common branch starting point, simulated unloading based on each feasible candidate execution position is executed in mutually isolated simulated state replicas, and a finite number of subsequent steps are performed. The generated simulated execution samples and real execution samples are used together to update the differential value reconstruction network; and the unloading phase of the next subtask is performed.

[0014] As a further optimization of the present invention, in step S1, the vehicle network system includes a vehicle, a roadside unit, and a cloud server. The vehicle communicates with its currently associated roadside unit via a wireless link, and the roadside units communicate with each other and with the cloud server via wired links.

[0015] The vehicle-to-everything (V2X) edge computing system includes a set of vehicles, a set of roadside units, and a cloud server; the set of vehicles is represented as... The roadside unit set is represented as Cloud server is denoted as ;

[0016] Each vehicle is equipped with a local computing processor and a wireless communication interface, enabling it to execute subtasks locally and upload subtask data to the currently associated roadside unit via a wireless link. Each roadside unit is equipped with an edge computing server, uplink wireless resources, downlink wireless resources, and a wired communication interface to receive data uploaded by vehicles, execute edge computing tasks, and deliver computing results to vehicles. The cloud server is equipped with cloud computing resources and is connected to each roadside unit via a wired network.

[0017] In step S1, the current system state of the vehicle-to-everything (V2X) edge computing system is obtained. The current system state is represented as follows:

[0018]

[0019] in, It indicates the communication link status, which is determined based on the vehicle's current location, the relationship between the vehicle and the roadside unit, and the occupancy status of wireless and wired links. It is used to characterize the communication conditions corresponding to the current data upload, dependent data transmission, and result delivery. It indicates the status of dependent tasks, which is determined based on the execution records of dependent tasks and their subtasks, including task release time, deadline, dependencies between subtasks, current execution status of each subtask, and the location of output data from completed subtasks; Indicates the status of vehicles and roadside unit resources; The current system state indicates the task queue and resource waiting status; the current system state is dynamically updated as the vehicle moves, subtasks are executed, data is transmitted, and resources are released, and is reacquired at each decision moment; the current system state is processed by feature extraction and normalization to form a fixed-dimensional state input, which is used for subsequent selection of subtasks to be processed and evaluation of candidate execution positions.

[0020] As a further optimization of the present invention, step S1, determining the current subtask to be processed, specifically includes:

[0021] The dependent tasks are represented as a directed acyclic graph:

[0022]

[0023] in, Represents a set of subtasks. Indicates the first Sub-tasks Indicates the number of subtasks. Represents the set of subtask dependencies; if This indicates a subtask. Depends on subtasks Subtask Waiting for subtasks Execution can only proceed after the process is complete and the results are usable.

[0024] Subtask The sets of predecessor subtasks and successor subtasks are represented as follows:

[0025]

[0026]

[0027] Based on the current execution status of each subtask, from the subtasks that have not yet been completed, select those subtasks whose predecessor subtasks have all been completed and whose results are usable, forming a set of ready subtasks. :

[0028]

[0029] in, This represents the set of subtasks that have been completed and whose results are available.

[0030] For ready subtasks Based on the estimated execution time of the ready subtask and the longest unfinished execution chain from the ready subtask to the exit node of the dependent task, calculate the original dynamic remaining critical timeout:

[0031]

[0032] in, For subtasks The original dynamic remaining critical timeframe, Subtasks Estimated execution time;

[0033] The original dynamic remaining critical timeframe is normalized to obtain the dynamic remaining critical timeframe. :

[0034]

[0035] in, To prevent positive numbers with a denominator of zero;

[0036] Calculate the time urgency of ready subtasks based on the time margin between the deadline of the dependent task, the current decision time, and the original dynamic remaining critical timeframe. :

[0037]

[0038] in, Subtasks The deadline of the dependent task. Indicates the current decision-making moment. This is a parameter representing the urgency of time delay. For the Sigmoid function;

[0039] Without altering the actual task state, temporarily make the ready subtask... Mark a task as completed, compare the set of ready subtasks before and after the marking, and determine the set of successor subtasks that are newly transitioned to the ready state solely because of the completion of that ready subtask. ;

[0040] Based on the computational complexity and dependent data volume of the newly transitioned ready subtasks, determine the subtasks. Original subsequent release benefits :

[0041]

[0042] in, The computational cost of subsequent subtasks. For reference, calculate the frequency. For subtasks and successor subtask The amount of dependent data between them The normalization parameter depends on the amount of data; the original subsequent release revenue is normalized to obtain the subsequent release revenue. :

[0043]

[0044] Based on the dynamic remaining critical timeliness, latency urgency, and subsequent release benefits, calculate the scheduling score of the ready subtask:

[0045]

[0046] in, Subtasks The scheduling score, , and These represent the corresponding weight coefficients;

[0047] The ready subtask with the highest scheduling score is identified as the current subtask to be processed. , is represented as:

[0048] .

[0049] As a further optimization of the present invention, step S2, determining the feasibility, counterfactual physical execution cost, and physical process description quantity of each candidate execution location, specifically includes the following:

[0050] Define the current subtask to be processed. In the current system state The following is the set of candidate execution locations This includes candidate execution locations corresponding to the vehicle's local location, edge nodes, and the cloud, whereby the candidate execution locations are denoted as... , The values ​​are 1, 2, and 3, representing the vehicle's local location, edge node, and cloud location, respectively.

[0051] When candidate execution location If the required communication conditions, vehicle dwell time conditions, or computing resource conditions are not met, the candidate execution location is marked as infeasible; otherwise, it is marked as feasible, and a feasibility indicator is determined. ;

[0052] The current system state is determined based on the feasibility identifier. The following is a set of feasible candidate execution locations :

[0053]

[0054] in, This indicates the feasibility of a candidate execution location; a value of 1 indicates feasibility, and a value of 0 indicates infeasibility.

[0055] For each feasible candidate execution position Based on the current subtask to be processed The location of the predecessor subtask result, the status of the communication link, the status of computing resource occupancy, and the status of the task queue are used to determine the corresponding physical execution process, and to predict the data transmission latency, resource waiting latency, computing latency, and dynamic energy consumption.

[0056] When the current subtask is pending Feasible candidate execution locations When the vehicle is an edge node or cloud, it transmits data through the uplink wireless link of the currently associated roadside unit. When it is necessary to return the result to the vehicle, the result data is transmitted through the downlink wireless link of the currently associated roadside unit.

[0057] Uplink wireless transmission rate Represented as:

[0058]

[0059] Downlink wireless transmission rate Represented as:

[0060]

[0061] in, and These represent the allocated uplink and downlink sub-channel bandwidths, respectively. and These represent the uplink and downlink transmit powers, respectively. Indicates the time of the vehicle's relationship with the currently associated roadside unit. Channel gain, Indicates noise power;

[0062] The wireless transmission rate is updated in segments according to the vehicle's location at preset channel update intervals; for feasible candidate execution locations... any wireless transmission phase The conditions for its transmission completion are expressed as follows:

[0063]

[0064] The corresponding number The actual wireless transmission duration of each wireless transmission phase is represented by for:

[0065]

[0066] in, Indicates the wireless transmission phase The upward or downward direction, This indicates the amount of data transmitted during this stage. and They represent the first The actual start and end times of each wireless transmission phase. Indicates time Instantaneous wireless transmission rate;

[0067] The start time of the wireless transmission phase is determined based on the combined availability time of the roadside unit's wireless sub-channel and the vehicle's wireless communication interface; when the wireless transmission cannot be completed before the vehicle leaves the coverage area of ​​the currently associated roadside unit, the corresponding candidate execution location is marked as infeasible; and the transmission risk caused by vehicle movement is determined based on the relationship between the expected completion time of the wireless transmission and the vehicle's remaining dwell time.

[0068] For the feasible candidate execution locations corresponding to edge nodes and the cloud, the subtasks to be processed Upload latency of source input data The calculation method is as follows:

[0069]

[0070] in, Indicates feasible candidate execution positions During the corresponding physical execution process, the currently pending subtask is uploaded. The source input data includes the set of uplink wireless transmission stages; the upload latency of the source input data corresponding to the feasible candidate execution location on the vehicle is zero; in addition to the uplink wireless link from the vehicle to the currently associated roadside unit, the feasible candidate execution location on the cloud also needs to go through the wired link from the roadside unit to the cloud server, and the corresponding wired transmission latency is included in the wired route transmission latency.

[0071] set up The current subtask to be processed The set of direct precursor tasks, Indicates a precursor mission The output data reaches the feasible candidate execution position. At the corresponding execution node, the time when all predecessor dependency data of the currently pending subtask arrives at the execution node is represented as follows:

[0072]

[0073] in, Indicates the currently pending subtasks. The moment of entering the ready state;

[0074] The corresponding predecessor dependency data wait latency is expressed as follows: ,in ,when At that time, subtask There is no need to wait for predecessor dependency data, and its dependency data transmission latency is zero; when the result of a predecessor subtask is located on the same execution node as the currently pending subtask, the result of the predecessor subtask does not need to be transmitted through a communication link. Take the moment when the result of the predecessor subtask reaches a usable state; when the two are located on different execution nodes. The moment when the result of the preceding subtask is completed and dependent data transmission is obtained;

[0075] set up Indicates the currently pending subtasks. The source input data reaches the feasible candidate execution position. The time at which the source input data and all predecessor dependency data arrive at the execution node corresponding to the execution node is represented as:

[0076]

[0077] For feasible candidate execution locations corresponding to the vehicle's local location, if the source input data is already located locally on the vehicle, then:

[0078] For feasible candidate execution locations The wired link traversed during the corresponding physical execution process has a wired route transmission delay. Represented as:

[0079]

[0080] in, Indicates feasible candidate execution positions The set of wired links that correspond to the physical execution process. Subtasks In the The amount of data transmitted on each link Indicates the first The bandwidth of the wired link, This indicates the propagation delay of the wired link; data for the corresponding candidate execution location in the cloud must be transmitted via the wired route between the currently associated roadside unit and the cloud server, and vehicles are not allowed to communicate directly with the cloud server;

[0081] Feasible candidate execution positions The resource wait latency in the corresponding physical execution process is expressed as:

[0082]

[0083] in, Indicates feasible candidate execution positions Includes the complete set of wireless transmission stages. Indicates the first The moment when a wireless transmission phase satisfies the preceding transmission conditions and enters the corresponding wireless resource queue. Indicates the first The moment when a wired transmission stage satisfies the preceding transmission conditions and enters the corresponding wired link resource queue. Indicates feasible candidate execution positions The corresponding execution node begins executing the currently pending subtask. The timing of transmission start is determined by the joint reservation method between the wireless sub-channel and the vehicle wireless interface, and the waiting time is calculated only once.

[0084] set up Indicates the currently pending subtasks. Number of processor cycles required Indicates feasible candidate execution positions The computation latency is expressed as follows, corresponding to the processor frequency of the execution node:

[0085]

[0086] The time of completion of the calculation is expressed as:

[0087]

[0088] For currently pending subtasks executed locally on the vehicle, and for non-exit subtasks that do not require delivering the final result to the vehicle, the completion time is the time when the result reaches a usable state.

[0089] For exit subtasks with candidate execution locations at edge nodes or in the cloud, the computation results still need to be transmitted to the vehicle; the corresponding downlink latency is expressed as follows:

[0090]

[0091] in, Indicates the currently pending subtasks. At feasible candidate execution locations Execution occurs when the result reaches a state where the application is usable; when there is no need to deliver the result to the vehicle, ;

[0092] For each feasible candidate execution position Based on the current subtask to be processed The moment of entering the ready state The moment when its result reaches the application's usability state Calculate the counterfactual execution delay :

[0093]

[0094] Feasible candidate execution positions The corresponding dynamic energy consumption is expressed as follows:

[0095]

[0096] in, Indicates the effective switched capacitor coefficient;

[0097] The dynamic energy consumption generated by wireless communication is expressed as:

[0098]

[0099] in, Indicates the first The transmission power corresponding to each wireless transmission phase.

[0100] Feasible candidate execution positions The energy consumption of wired link transmission during the corresponding physical execution process is expressed as follows:

[0101]

[0102] in, Indicates the first The energy consumed by a wired link to transmit a unit of data. Indicates the first The amount of data transmitted over a single wired link;

[0103] For each feasible candidate execution position Based on the dynamic computing energy consumption, wireless communication energy consumption, and wired link transmission energy consumption generated during the corresponding physical execution process, the counterfactual execution energy consumption is determined. :

[0104]

[0105] The counterfactual execution delay Counterfactual execution energy consumption Upload latency, resource waiting latency, computation latency, wired route transmission latency, result downlink latency, dependent data transmission latency, and transmission risks caused by vehicle movement are used to subsequently determine the counterfactual physical execution cost corresponding to the candidate execution location and construct the physical process description quantity.

[0106] Each feasible candidate execution location is calculated based on the counterfactual execution delay and counterfactual energy consumption. Physical execution cost , is represented as:

[0107]

[0108] in, and These represent the delay weighting coefficient and the energy consumption weighting coefficient, respectively. and These represent the normalized values ​​of time delay and energy consumption, respectively. This indicates the pre-defined invalid cost corresponding to an infeasible candidate execution location;

[0109] Record the physical execution amount in the physical execution process corresponding to each candidate execution location, and identify feasible candidate execution locations. physical process description quantity :

[0110]

[0111] in, This indicates the transmission risk characteristics caused by vehicle movement;

[0112] The feasibility indicators, counterfactual physical execution costs, and physical process descriptions of each candidate execution location are combined to obtain the current subtask to be processed. The corresponding counterfactual physical cost assessment result set is represented as follows:

[0113]

[0114] in, The counterfactual physical cost assessment result set corresponding to the current subtask to be processed is then input into step S3 to determine the physical benchmark and physical process difference description quantity.

[0115] As a further optimization of the present invention, in step S3, the differential value reconstruction network calculates the reconstruction value of each unloading action, specifically including:

[0116] In the current system state Next, state encoding is performed to obtain the latent state vector. :

[0117]

[0118] in, Represents the state coding function. Indicates the state coding parameters;

[0119] From feasible candidate execution positions In this process, the candidate execution location with the lowest counterfactual physical execution cost is determined as the physical benchmark. :

[0120]

[0121] Each feasible candidate execution position is constructed based on the physical execution difference between each candidate execution position and the physical baseline action. Comparative characteristics of actions , is represented as:

[0122]

[0123] in, Indicates feasible candidate execution positions With physical benchmark The difference in counterfactual physical execution costs between them Indicates feasible candidate execution positions The cost ranking features, Indicates feasible candidate execution positions An indicator of whether it is the unloading action with the lowest physical execution cost;

[0124] Based on the state latent vector and action comparison features, the shared value of the action set, the original residual advantage of the candidate execution position, and the action-level gating coefficient are calculated to construct the shared value branch. Motion residual advantage branch and action-level gating branches , respectively represented as:

[0125]

[0126]

[0127]

[0128] in, This indicates that the set of actions shares value. Indicates feasible candidate execution positions The original residual advantage, Indicates feasible candidate execution positions Action-level gating coefficient, , and These represent the shared value branch function, the action residual advantage branch function, and the action-level gated branch function, respectively. , and These represent the network parameters of the shared value branch, the action residual advantage branch, and the action-level gating branch, respectively. This represents the aggregated representation of physical process descriptors after processing by the action comparison layer. Represents a nonlinear gated function;

[0129] For feasible candidate execution locations The original residual advantage is centralized to obtain the centralized action residual advantage. :

[0130]

[0131] Before the network starts operating, the output layers of the shared value branch and the action residual advantage branch are zero-initialized to ensure that the network initialization satisfies the following:

[0132]

[0133] The reconstruction value of each feasible candidate execution location during network initialization is determined by the corresponding physical execution cost;

[0134] An upper limit is set on the magnitude of the difference in action residual advantage between legal candidate execution positions, expressed as:

[0135]

[0136] in, This indicates the upper limit of the preset action residual advantage difference magnitude;

[0137] Feasible candidate execution positions Corresponding long-term difference components for:

[0138]

[0139] For each feasible candidate execution location The value of reconstruction is calculated by combining its counterfactual physical execution cost and long-term difference component:

[0140]

[0141] in, Indicates the currently pending subtasks. In the current system state Select feasible candidate execution locations The value of reconstruction.

[0142] As a further optimization of the present invention, in step S4, determining the target unloading location specifically includes:

[0143] Compare each feasible candidate execution location Based on the reconstruction value, the feasible candidate execution location with the greatest reconstruction value is determined as the target unloading location:

[0144]

[0145] Specifically, when two or more feasible candidate execution locations have the same maximum reconfiguration value, and this includes a physical baseline, the value is considered to be one of the highest possible values. At that time, the physical reference will be given priority. The target uninstallation location has been determined.

[0146] To explain the physical meaning of the reconstructed value selection relative to the physical benchmark, regarding the physical benchmark... Each feasible candidate execution location other than Determine its long-term differential increment relative to the physical reference:

[0147]

[0148] And determine its counterfactual physical execution cost difference relative to the physical benchmark:

[0149]

[0150] Based on the definition of refactoring value, feasible candidate execution locations Relative to physical reference The value difference of the reconstruction is:

[0151]

[0152] when At that time, feasible candidate execution positions The reconstruction value does not exceed the reconstruction value of the physical baseline; when At that time, feasible candidate execution positions The corresponding long-term differential increment can compensate for the increased counterfactual physical execution cost relative to the physical baseline, and the reconstruction value of the feasible candidate execution location exceeds the reconstruction value of the physical baseline.

[0153] When the long-term differential increments of all feasible candidate execution locations other than the physical benchmark do not exceed the corresponding counterfactual physical execution cost increments, the physical benchmark is determined as the target unloading location; when there is a feasible candidate execution location whose long-term differential increment exceeds the corresponding counterfactual physical execution cost increment, the feasible candidate execution location with the highest reconstruction value is determined as the target unloading location; a target unloading scheme is formed based on the physical execution process corresponding to the target unloading location.

[0154] Write the communication resource usage, computing resource usage, and task status changes corresponding to the target offloading scheme into the real system.

[0155] As a further optimization of the present invention, in step S5, the generation of the simulated branch sample specifically includes:

[0156] Before writing the target unloading scheme described in S4 into the real system, the current system state is... The common branch starting point is fixed at each feasible candidate execution position; for each feasible candidate execution position... Copy the current system state This yields a copy of the corresponding simulated state:

[0157]

[0158] in, Indicates feasible candidate execution positions The corresponding simulated state copy, This represents a system state replication function; each simulated state copy is isolated from the real system state, and changes in communication resource usage, computing resource usage, task queue changes, and dependent task state changes in the copy are not written to the real system.

[0159] At feasible candidate execution locations In the corresponding simulated state replica, the feasible candidate execution position is set as the first unload decision, and the subsequent process is carried out according to the event process of dependent data transmission, communication resource occupation, computing resource waiting, subtask calculation, result return and subsequent subtask release, so as to obtain the subsequent decision state, stage feedback and task termination identifier corresponding to the first unload decision.

[0160] When the dependent task has not terminated, in subsequent decision states, the feasible candidate execution position with the lowest counterfactual physical execution cost is selected from the feasible candidate execution positions in the simulation state replica to continue execution in a finite number of steps; simulation progress stops when the preset maximum number of simulation steps is reached or the dependent task terminates; the maximum number of simulation steps is recorded as... And determine the cumulative simulation feedback based on the stage feedback generated by each simulation execution. The calculation method is as follows:

[0161]

[0162] in, Indicates the discount factor. Indicates the simulated state copy number The phase feedback generated during the execution of each simulation; This represents the actual number of steps advanced by the simulated cumulative feedback, and its value is determined by... The conditions for task termination are determined together; when the dependent task has not terminated and Maximum number of simulated steps not reached If the system continues to simulate propulsion, it will continue; otherwise, it will stop simulating propulsion.

[0163] Based on the current system state The simulation execution sample is constructed by initial unloading decision, cumulative simulation feedback, simulation end status and task termination flag, and actual simulation steps. :

[0164]

[0165] The simulated execution samples and the real execution samples generated in the real system by the target unloading scheme in S4 are jointly stored in the experience replay pool for training and updating the differential value reconstruction network. The update method is as follows:

[0166]

[0167] in, This represents the experience replay pool. This represents the actual execution sample corresponding to the target unloading scheme. This indicates the operational status after a short-term implementation. Indicates the task termination flag. This represents the actual number of steps taken corresponding to the simulated cumulative feedback.

[0168] As a further optimization of the present invention, step S5 specifically includes the training and updating of the differential value reconstruction network:

[0169] From the experience replay pool The Middle Each training sample is represented as:

[0170]

[0171] in, Indicates the first training samples, Indicates the initial running status of the sample. This indicates the unloading action corresponding to the sample. Indicates the end of the process. Indicates the task termination flag. This indicates the actual number of steps progressed corresponding to the cumulative feedback; if the first... If each training sample is a real execution sample, then This indicates the stage feedback generated during this actual execution. If the... If each training sample is a simulated execution sample, then This indicates the cumulative feedback of the corresponding simulation branch;

[0172] Based on the communication link occupancy, computational resource waiting intensity, dependent data transmission volume, and connection changes caused by vehicle movement during the physical execution process corresponding to the training samples, the dynamic risk value of the training samples is determined, and the sample priority weight is determined based on the time difference error and dynamic risk value of the training samples.

[0173]

[0174] in, Indicates the first The sample priority weights of each training sample Indicates the first Temporal difference error of each training sample Indicates the first The dynamic risk value of each training sample. This represents the dynamic risk weighting coefficient. This indicates preventing positive numbers with a priority weight of zero. This represents the priority weight adjustment coefficient; training samples are obtained from the empirical replay pool according to the sampling probability that is positively correlated with the sample priority weight;

[0175] Build parameters are Online reconstruction of value networks and parameters are Target Reconstruction of Value Network ;

[0176] When the When the dependent task corresponding to each training sample has not terminated, in the end-of-progress state Next, following step S1, determine the current subtask to be processed and form a set of feasible candidate execution locations. The online reconstruction value network selects the feasible candidate execution position with the highest online reconstruction value from the set of feasible candidate execution positions as the subsequent action. The selection method is as follows:

[0177]

[0178] Calculate the value network based on the target. The training target value is calculated as follows:

[0179]

[0180] in, Indicates the first The training target value for each training sample. Indicates the relationship with the first The discount term corresponding to the actual number of steps advanced for each training sample. Indicates the end of the process;

[0181] The time difference error is calculated based on the training target value and the reconstructed value of the online reconstructed value network. :

[0182]

[0183] The total loss is constructed based on the time difference loss, action-level gating constraint loss, low-risk baseline preservation constraint loss, and high-risk gating opening constraint loss:

[0184]

[0185] in, Indicates the total loss. This indicates the number of samples in a single training batch. Represents the error loss function. This indicates the action-level gating constraint loss. Low-risk benchmarks maintain constrained losses. This indicates the loss from opening the high-risk gating constraint. , , These represent the weighting coefficients of the corresponding loss terms;

[0186] The parameters of the online reconstructed value network are updated according to minimizing the total loss, as follows:

[0187]

[0188] in, Indicates the learning rate;

[0189] When the number of online value network updates is an integer multiple of the target value network update cycle, the parameters of the online value network will be synchronized to the target value network.

[0190]

[0191] The sample priority weights of the corresponding training samples are then re-determined based on the updated time difference error.

[0192] Compared with the prior art, the advantages of this invention, which employs the above-mentioned technical methods, are as follows:

[0193] 1) By performing counterfactual physical evaluations on each candidate execution location under the same system state, the feasibility, physical execution cost, and physical process description of each candidate execution location can be obtained. This can reflect the actual execution status of different candidate execution locations under the current communication, computing, and resource state in a timely manner, providing the current physical basis for subsequent offloading decisions.

[0194] 2) Using the feasible candidate execution position with the lowest counterfactual physical execution cost as the physical benchmark, the differential value reconstruction network is guided to learn the long-term difference component of the candidate execution position. This separates the immediate physical cost from the long-term operational impact, reduces the difficulty of value learning, and improves the stability of value estimation and network training.

[0195] 3) Use mutually isolated state replicas to simulate each feasible candidate execution position in a finite number of steps, and use the generated simulated execution samples and real execution samples together for network updates, increasing the training samples of different candidate actions in the same state and improving the utilization efficiency of candidate action information and training samples. Attached Figure Description

[0196] Figure 1 This is a flowchart illustrating a specific implementation method of the Internet of Vehicles (IoV) dependent task offloading method based on physical benchmark-guided long-term differential learning.

[0197] Figure 2 This is a flowchart illustrating the specific framework of the present invention: a method for unloading dependent tasks in the Internet of Vehicles based on physical benchmark-guided long-term differential learning.

[0198] Figure 3 This is a cost comparison chart of the present invention and the comparative method;

[0199] Figure 4 This is a comparison chart of the total task response latency of the present invention and the comparative method;

[0200] Figure 5 This is a comparison chart of the total dynamic energy consumption of the present invention and the comparative method;

[0201] Figure 6 This is a comparison chart of the number of training interactions between the present invention and the comparison method; Detailed Implementation

[0202] To better illustrate the technical effects of the present invention, specific embodiments of the present invention are described below with reference to the accompanying drawings, so that those skilled in the art can better understand the present invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the present invention will be omitted here.

[0203] Example

[0204] The specific implementation flow of the vehicle-to-everything (V2X) dependent task offloading method in the embodiments is as follows: Figure 1 As shown; its specific framework and process are as follows: Figure 2 As shown. Using SUMO traffic simulation data as an example, the advantages of the method of the present invention are illustrated. The specific steps are as follows:

[0205] S1: Using the SUMO traffic simulation software, a vehicle-to-everything (V2X) traffic environment is built, a DAG generator is set up to generate the DAG tasks required for the experiment, and the current system state is obtained. The process includes the following steps:

[0206] Step 1: Build a vehicle-to-everything (V2X) traffic environment in the SUMO traffic simulation software. Assume that a roadside unit server is deployed every 400m along a 2km long two-way road. The vehicle speed is randomly generated from 0 to 20km / h, 20 to 40km / h, or 40 to 60km / h according to the scenario type, to simulate low-speed, general urban driving, and fast-switching scenarios, respectively, and is randomly distributed on the road. A linear motion model is used to describe the vehicle motion state, and the vehicle position is dynamically updated according to the initial position, direction of travel, and speed.

[0207] The computing frequency of the vehicle's local processor is set to 1.1–1.8 GHz, the computing frequency of the roadside unit edge server is set to 6–12 GHz, and the computing frequency of the cloud server is set to 60–130 GHz. Each roadside unit is configured with uplink and downlink wireless resources, with a total uplink and downlink bandwidth of 8–18 MHz. Roadside units are connected to each other and to the cloud server via wired links, with a wired link bandwidth of 100–700 Mbit / s.

[0208] Step 2: Use a DAG generator to generate vehicle application tasks. Randomly select a topology from chain structure, branch-convergence structure, diamond structure, Map-Reduce structure and random hierarchical structure, and establish directed dependencies between subtasks according to the selected topology.

[0209] In Easy load scenarios, each dependent task contains 3-5 subtasks; in Medium load scenarios, it contains 5-8 subtasks; and in Hard load scenarios, it contains 8-12 subtasks. For each subtask, the source input data volume, processor cycles, and output data volume are generated. The source input data volume ranges from 0.5 to 35 Mbit depending on the scenario difficulty; the processor cycles are determined based on the source input data volume and the unit data computation frequency; and the output data volume is generated proportionally to the source input data volume. For adjacent subtasks with dependencies, 0.2 to 60 Mbit of dependency data is generated.

[0210] Step 3: Generate task release and deadline times. The first dependent task is released at the start of the simulation. Subsequent dependent tasks are generated using single-arrival, approximate Poisson arrival, periodic arrival with random perturbation, or burst arrival methods. The deadline time for each dependent task is determined based on its release time, remaining critical path estimation time, and preset deadline relaxation coefficient, ensuring that the generated scenario covers different operating conditions such as resource abundance, wireless resource contention, deadline pressure, and vehicle movement across roadside units.

[0211] Once generated, the vehicle movement status, roadside unit and cloud server configuration, dependent tasks, task release events and random seeds are saved as a scenario list, and the scenario list is used to identify that the same initial environment can be generated repeatedly in each simulation experiment.

[0212] Step 4: Obtain the current system state of the vehicle-to-everything (V2X) edge computing system. The current system state is represented as follows:

[0213]

[0214] in, It indicates the status of the communication link, which is determined based on the vehicle's current location, the relationship between the vehicle and the roadside unit, and the occupancy status of the wireless and wired links; It indicates the status of dependent tasks, which is determined based on the execution records of dependent tasks and their subtasks, including task release time, deadline, dependencies between subtasks, current execution status of each subtask, and the location of output data from completed subtasks; Indicates the status of vehicles and roadside unit resources; This indicates the task queue and resource waiting status. The current system state is dynamically updated as the vehicle moves, subtasks are executed, data is transmitted, and resources are released, and is reacquired at each decision point. After feature extraction and normalization, the current system state forms a fixed-dimensional state input, which is used for subsequent selection of subtasks to be processed and evaluation of candidate execution locations.

[0215] In the current dependent task, candidate subtasks are determined according to dynamic priority among all ready subtasks, as follows:

[0216] For ready subtasks Based on the estimated execution time of the ready subtask and the longest unfinished execution chain from the ready subtask to the exit node of the dependent task, calculate the original dynamic remaining critical timeout:

[0217]

[0218] in, For subtasks The original dynamic remaining critical timeframe, Subtasks Estimated execution time;

[0219] The original dynamic remaining critical timeframe is normalized to obtain the dynamic remaining critical timeframe. :

[0220]

[0221] in, To prevent positive numbers with a denominator of zero, 1.0 × 10⁻⁶ is used in this example. -9 ;

[0222] Based on the time margin between the deadline of the dependent task, the current decision time, and the original dynamic remaining critical timeframe, calculate the chain-level latency urgency of each ready subtask. :

[0223]

[0224] in, Subtasks The deadline of the dependent task. Indicates the current decision-making moment. To maintain the urgency scale parameter, a value of 1.0 is used in this embodiment. For the Sigmoid function;

[0225] Without altering the actual task state, temporarily make the ready subtask... Mark a task as completed, compare the set of ready subtasks before and after the marking, and determine the set of successor subtasks that are newly transitioned to the ready state solely because of the completion of that ready subtask. ;

[0226] Based on the computational complexity and dependent data volume of the successor subtasks that have newly transitioned to the ready state, the ready subtasks are determined. Original subsequent release benefits :

[0227]

[0228] in, The computational cost of subsequent subtasks. For reference calculation frequency, 1.0 × 10⁻⁶ is used in this example. 9 Hz, For subtasks and successor subtask The amount of dependent data between them To normalize the data volume, a normalization parameter of 1.0 × 10 is used in this example. 6 bit; normalize the original subsequent release profit to obtain the subsequent release profit. :

[0229]

[0230] Based on the dynamic remaining critical timeliness, latency urgency, and subsequent release benefits, calculate the scheduling score for each ready subtask:

[0231]

[0232] in, Subtasks The scheduling score, , and These represent the corresponding weight coefficients, and their sum is 1. In the example, they are taken as 0.5, 0.3, and 0.2 respectively.

[0233] The ready subtask with the highest scheduling score is identified as the current subtask to be processed. , is represented as:

[0234]

[0235] Communication and computation parameters include: channel update interval of 0.01 s; noise power. 1.0×10 −13 W; the reference channel gain is 1.0 × 10⁻⁶. −3 The path loss index is 3.0; the minimum calculation distance is 1.0 m; and the vehicle uplink transmit power is... The RSU downlink total transmit power is 0.18–0.28 W. The power consumption is 1.0 W; the wired link propagation delay is 0.001 s; and the power consumption per bit of the wired link is 1.0 × 10⁻⁶. −9 J / bit; Channel gain for , This represents the distance between the vehicle and the currently associated roadside unit, in meters; the effective switched capacitor coefficient is: =2.0×10 −29 , =1.0×10 −30 , =3.0×10 −32 .

[0236] S2: Reconstruct candidate execution locations and determine their feasibility, counterfactual physical execution cost, and physical process description quantity, specifically including:

[0237] Candidate execution positions Determine its feasibility when the candidate execution location is selected. If the required communication conditions, vehicle dwell time conditions, or computing resource conditions are not met, the candidate execution location is marked as infeasible; otherwise, it is marked as feasible, and a feasibility indicator is determined. ;

[0238] The current system state is determined based on the feasibility identifier. The following is a set of feasible candidate execution locations :

[0239]

[0240] in, This indicates the feasibility of a candidate execution location; a value of 1 indicates feasibility, and a value of 0 indicates infeasibility.

[0241] For each feasible candidate execution position Based on the current subtask to be processed The location of the predecessor subtask result, the status of the communication link, the status of computing resource occupancy, and the status of the task queue are used to determine the corresponding physical execution process, and to predict the data transmission latency, resource waiting latency, computing latency, and dynamic energy consumption.

[0242] For each feasible candidate execution position Based on the time when the currently pending subtask enters the ready state The moment when its result reaches the application's usability state Calculate the counterfactual execution delay :

[0243]

[0244] For each feasible candidate execution position Based on the dynamic computing energy consumption, wireless communication energy consumption, and wired link transmission energy consumption generated during the corresponding physical execution process, the counterfactual execution energy consumption is determined. :

[0245]

[0246] The counterfactual execution delay Counterfactual execution energy consumption Upload latency, resource waiting latency, computation latency, wired route transmission latency, result downlink latency, dependent data transmission latency, and transmission risks caused by vehicle movement are used to subsequently determine the counterfactual physical execution cost corresponding to the candidate execution location and construct the physical process description quantity;

[0247] Each feasible candidate execution location is calculated based on the counterfactual execution delay and counterfactual energy consumption. Physical execution cost , is represented as:

[0248]

[0249] in, and These represent the delay weighting coefficient and the energy consumption weighting coefficient, respectively. In this example, they are set to 0.5 and 0.5, respectively. and These represent the time delay normalization factor and the energy consumption normalization factor, respectively. In this example, they are both taken as 1 and 0.1. This represents the preset invalid cost corresponding to the infeasible candidate execution location, which is 1.0 × 10 in this example. 9 ;

[0250] Record the physical execution amount in the physical execution process corresponding to each candidate execution location, and construct feasible candidate execution locations. physical process description quantity :

[0251]

[0252] in, This indicates the transmission risk characteristics caused by vehicle movement;

[0253] The feasibility indicators, counterfactual physical execution costs, and physical process descriptions of each candidate execution location are combined to obtain the current subtask to be processed. The corresponding counterfactual physical cost assessment result set is represented as follows:

[0254]

[0255] in, The counterfactual physical cost assessment result set corresponding to the current subtask to be processed is then input into step S3 to determine the physical benchmark and physical process difference description quantity.

[0256] S3: The differential value reconstruction network calculates the reconstruction value of each unloading action, and the specific steps are as follows:

[0257] In the current system state Next, state encoding is performed to obtain the latent state vector. :

[0258]

[0259] in, Represents the state coding function. Indicates the state coding parameters;

[0260] From feasible candidate execution positions In this process, the candidate execution location with the lowest counterfactual physical execution cost is determined as the physical benchmark. :

[0261]

[0262] Each feasible candidate execution position is constructed based on the physical execution difference between each candidate execution position and the physical baseline action. Comparative characteristics of actions , is represented as:

[0263]

[0264] in, Indicates feasible candidate execution positions With physical benchmark The difference in counterfactual physical execution costs between them Indicates feasible candidate execution positions The cost ranking features, Indicates feasible candidate execution positions An indicator of whether it is the unloading action with the lowest physical execution cost;

[0265] Based on the state latent vector and action comparison features, the shared value of the action set, the original residual advantage of the candidate execution position, and the action-level gating coefficient are calculated to construct the shared value branch. Motion residual advantage branch and action-level gating branches , respectively represented as:

[0266]

[0267]

[0268]

[0269] in, This indicates that the set of actions shares value. Indicates feasible candidate execution positions The original residual advantage, Indicates feasible candidate execution positions Action-level gating coefficient, , and These represent the shared value branch function, the action residual advantage branch function, and the action-level gated branch function, respectively. , and These represent the network parameters of the shared value branch, the action residual advantage branch, and the action-level gating branch, respectively. This represents the aggregated representation of physical process descriptors after processing by the action comparison layer. Represents a nonlinear gated function;

[0270] For feasible candidate execution locations The original residual advantage is centralized to obtain the centralized action residual advantage. :

[0271]

[0272] Before the network starts operating, the output layers of the shared value branch and the action residual advantage branch are zero-initialized to ensure that the network initialization satisfies the following:

[0273]

[0274] The reconstruction value of each feasible candidate execution location during network initialization is determined by the corresponding physical execution cost;

[0275] An upper limit is set on the magnitude of the difference in action residual advantage between legal candidate execution positions, expressed as:

[0276]

[0277] in, This indicates the preset upper limit of the motion residual advantage difference magnitude, which is set to 32 in this example;

[0278] Feasible candidate execution positions Corresponding long-term difference components for:

[0279]

[0280] For each feasible candidate execution location The value of reconstruction is calculated by combining its counterfactual physical execution cost and long-term difference component:

[0281]

[0282] in, Indicates the currently pending subtasks. In the current system state Select feasible candidate execution locations The value of reconstruction;

[0283] S4: Select the feasible candidate execution location with the greatest refactoring value as the target unloading location, specifically including: comparing each feasible candidate execution location. Based on the reconstruction value, the feasible candidate execution location with the greatest reconstruction value is determined as the target unloading location:

[0284]

[0285] Specifically, when two or more feasible candidate execution locations have the same maximum reconfiguration value, and this includes a physical baseline, the value is considered to be one of the highest possible values. At that time, the physical reference will be given priority. The target uninstallation location has been determined.

[0286] For physical benchmarks Each feasible candidate execution location other than Determine its long-term differential increment relative to the physical reference:

[0287]

[0288] And determine its counterfactual physical execution cost difference relative to the physical benchmark:

[0289]

[0290] Feasible candidate execution positions Relative to physical reference The value difference of the reconstruction is:

[0291]

[0292] When the long-term differential increments of all feasible candidate execution locations other than the physical benchmark do not exceed the corresponding counterfactual physical execution cost increments, the physical benchmark is determined as the target unloading location; when there is a feasible candidate execution location whose long-term differential increment exceeds the corresponding counterfactual physical execution cost increment, the feasible candidate execution location with the highest reconstruction value is determined as the target unloading location; a target unloading scheme is formed based on the physical execution process corresponding to the target unloading location.

[0293] Write the communication resource usage, computing resource usage, and task status changes corresponding to the target offloading scheme into the real system.

[0294] S5: The process of generating and training the simulated branch samples specifically includes:

[0295] Before writing the target unloading scheme described in S4 into the real system, the current system state is... The common branch starting point is fixed at each feasible candidate execution position; for each feasible candidate execution position... Copy the current system state This yields a copy of the corresponding simulated state:

[0296]

[0297] in, Indicates feasible candidate execution positions The corresponding simulated state copy, This represents a system state replication function; each simulated state copy is isolated from the real system state, and changes in communication resource usage, computing resource usage, task queue changes, and dependent task state changes in the copy are not written to the real system.

[0298] At feasible candidate execution locations In the corresponding simulated state replica, the feasible candidate execution position is set as the first unload decision, and the subsequent process is carried out according to the event process of dependent data transmission, communication resource occupation, computing resource waiting, subtask calculation, result return and subsequent subtask release, so as to obtain the subsequent decision state, stage feedback and task termination identifier corresponding to the first unload decision.

[0299] When the dependent task has not terminated, in subsequent decision states, the feasible candidate execution position with the lowest counterfactual physical execution cost is selected from the feasible candidate execution positions in the simulation state replica to continue execution in a finite number of steps; simulation progress stops when the preset maximum number of simulation steps is reached or the dependent task terminates; the maximum number of simulation steps is recorded as... In this embodiment, the value is 2, and the cumulative simulation feedback is determined based on the stage feedback generated by each simulation execution. The calculation method is as follows:

[0300]

[0301] in, This represents the discount factor, which is set to 1.0 in this example. Indicates the simulated state copy number The phase feedback generated during the execution of each simulation; This represents the actual number of steps advanced by the simulated cumulative feedback, and its value is determined by... The conditions for task termination are determined together; when the dependent task has not terminated and Maximum number of simulated steps not reached If the system continues to simulate propulsion, it will continue; otherwise, it will stop simulating propulsion.

[0302] Based on the current system state The simulation execution sample is constructed by initial unloading decision, cumulative simulation feedback, simulation end status and task termination flag, and actual simulation steps. :

[0303]

[0304] The simulated execution samples and the real execution samples generated in the real system by the target unloading scheme in S4 are jointly stored in the experience replay pool for training and updating the differential value reconstruction network. The update method is as follows:

[0305]

[0306] in, This represents the experience replay pool. This represents the actual execution sample corresponding to the target unloading scheme. This indicates the operational status after a short-term implementation. Indicates whether the task has been terminated. This represents the actual number of steps taken corresponding to the simulated cumulative feedback.

[0307] The training and update process of the differential value reconstruction network is as follows:

[0308] From the experience replay pool The Middle Each training sample is represented as:

[0309]

[0310] in, Indicates the first training samples, Indicates the initial running status of the sample. This indicates the unloading action corresponding to the sample. Indicates the end of the process. Indicates whether the task has been terminated. This indicates the actual number of steps progressed corresponding to the cumulative feedback; if the first... If each training sample is a real execution sample, then This indicates the stage feedback generated during this actual execution. If the... If each training sample is a simulated execution sample, then This indicates the cumulative feedback of the corresponding simulation branch;

[0311] Based on the communication link occupancy, computational resource waiting intensity, dependent data transmission volume, and connection changes caused by vehicle movement during the physical execution process corresponding to the training samples, the dynamic risk value of the training samples is determined, and the sample priority weight is determined based on the time difference error and dynamic risk value of the training samples.

[0312]

[0313] in, Indicates the first The sample priority weights of each training sample Indicates the first Temporal difference error of each training sample Indicates the first The dynamic risk value of each training sample. This represents the dynamic risk weighting coefficient, which is set to 1.0 in this example. This indicates a positive number to prevent a priority weight of zero; in this example, it is taken as 1.0 × 10. −6 , This represents the priority weight adjustment coefficient, which is 0.6 in this example; training samples are obtained from the empirical replay pool according to the sampling probability that is positively correlated with the sample priority weight;

[0314] Build parameters are Online reconstruction of value networks and parameters are Target Reconstruction of Value Network ;

[0315] When the When the dependent task corresponding to each training sample has not terminated, in the end-of-progress state Next, following step S1, determine the current subtask to be processed and form a set of feasible candidate execution locations. The online reconstruction value network selects the feasible candidate execution position with the highest online reconstruction value from the set of feasible candidate execution positions as the subsequent action. The selection method is as follows:

[0316]

[0317] Calculate the value network based on the target. The training target value is calculated as follows:

[0318]

[0319] in, Indicates the first The training target value for each training sample. Indicates the relationship with the first The discount term corresponding to the actual number of steps advanced for each training sample. Indicates the end of the process;

[0320] The time difference error is calculated based on the training target value and the reconstructed value of the online reconstructed value network. :

[0321]

[0322] The total loss is constructed based on the time difference loss, action-level gating constraint loss, low-risk baseline preservation constraint loss, and high-risk gating opening constraint loss:

[0323]

[0324] in, Indicates the total loss. This indicates the number of samples in a single training batch. Represents the error loss function. This indicates the action-level gating constraint loss. Low-risk benchmarks maintain constrained losses. This indicates the loss from opening the high-risk gating constraint. , , These represent the weighting coefficients of the corresponding loss terms;

[0325] The parameters of the online reconstructed value network are updated according to minimizing the total loss, as follows:

[0326]

[0327] in, Indicates the learning rate;

[0328] When the number of online value network updates is an integer multiple of the target value network update cycle, the parameters of the online value network will be synchronized to the target value network.

[0329]

[0330] And the sample priority weights of the corresponding training samples are re-determined based on the updated time difference error;

[0331] In this embodiment, the network training parameters are: 1000 training epochs, 8192 experience replay pool capacity, 64 warm-up samples, and batch size. The learning rate is 64. 1.0×10 −3 The target network update cycle is 64, and the training update interval is once for each real decision. , and The values ​​are 0.01, 0.05, and 0.02, respectively.

[0332] To verify the effectiveness of the proposed task offloading method, relevant research methods were selected, including: OSMG method, PSO+GA method (Palaniappan SC, Ponnuswamy PP. Task Offloading in Edge Computing Using Integrated Particle Swarm Optimization and Genetic Algorithm[J]. Advances in Science and Technology. Research Journal, 2025, 19(1): 371-380.), and Dynamic DQN method (Paknejad M, Moshiri PF, Simsek M, et al. Meeting Deadlines in Motion: Deep RL for Real-Time Task Offloading in Vehicular Edge Networks[C] / / 202516th International Conference on Network of the Future (NoF). IEEE, 2025:134-141.), the D3QN method, and the PG-D3QN method of this invention were compared in Easy, Medium, and Hard load scenarios, respectively. The OSMG method directly determines the unloading position based on the lowest current physical cost, belonging to a single-step decision-making method that requires no training; the PSO+GA method obtains the task unloading position through heuristic search, belonging to a traditional heuristic optimization method; the Dynamic DQN and D3QN methods learn task unloading strategies through continuous interaction with the environment, belonging to deep reinforcement learning methods. To ensure fairness in the comparison, the joint cost of latency and energy consumption, total task response latency, and total dynamic energy consumption were used as performance evaluation indicators, and the training efficiency of each reinforcement learning method was evaluated by the number of training interactions. The calculation of each indicator is as follows:

[0333] The combined cost of time delay and energy consumption is calculated as follows:

[0334]

[0335] The total task response latency is calculated as follows:

[0336]

[0337] The total dynamic energy consumption is calculated as follows:

[0338]

[0339] in, and These are the time delay and energy consumption normalization coefficients, respectively, which are set to 1 and 0.1 in this example; and The weights are time delay and energy consumption, respectively, and in this embodiment, they are set to 0.5 and 0.5, respectively.

[0340] Figure 3 , Figure 4 , Figure 5 The table lists the joint latency-energy cost, total task response latency, and total dynamic energy consumption of different methods in three task scenarios. In the Easy scenario, the performance of OSMG, Dynamic DQN, D3QN, and PG-D3QN methods is basically similar, indicating that single-step physics decision-making can achieve good offloading effects when the scenario complexity is low and the task dependency is weak. However, the PSO+GA method, because it needs to iteratively search to determine the offloading scheme at each step, produces higher joint latency-energy cost and total task response latency in this scenario. In the Medium scenario, as task dependency and resource competition increase, the performance of OSMG begins to decline, and deep reinforcement learning methods generally perform better, especially Dynamic DQN. The DQN method achieves a slightly lower total task response latency, while the PG-D3QN method has the lowest total dynamic energy consumption and latency-energy joint cost. Although the PSO+GA method can improve some offloading decisions through global search, its overall performance is still lower than that of reinforcement learning methods. In the Hard scenario, the performance gap between the methods further widens. The PG-D3QN method achieves better results in terms of latency-energy joint cost, total task response latency, and total dynamic energy consumption, showing a more significant advantage over the OSMG and D3QN methods. Its overall performance is also superior to the PSO+GA method, indicating that the method of this invention has better comprehensive optimization capabilities under conditions of high scenario complexity and strong task dependency. In summary, the method of this invention maintains good performance in simple scenarios and can still further optimize offloading decisions by leveraging the long-term influence between tasks in scenarios with high scenario complexity and strong task dependency.

[0341] Figure 6The number of training interactions required for several reinforcement learning methods is presented. In the Easy scenario, task coupling and resource competition are weak, and the long-term impact of different unloading actions on subsequent data migration and resource status is limited. The current physical cost can accurately reflect the merits of the current action. The PG-D3QN method requires no additional training interactions, while the D3QN and Dynamic DQN methods require 8880 and 57600 interactions, respectively. In the Medium and Hard scenarios, the number of training interactions required by the PG-D3QN method is significantly less than that of the Dynamic DQN and D3QN methods. This is because the method of this invention uses counterfactual physical cost to guide the network to learn the long-term effects not considered by the physical model, thereby reducing random exploration during the training process and exhibiting better adaptability in highly dynamic environments.

[0342] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A method for offloading vehicular network-dependent tasks based on physical benchmark-guided long-term differential learning, characterized in that, Includes the following steps: S1: Based on the current system status of the vehicle-to-everything (V2X) edge computing system and the dynamic remaining critical timeliness, latency urgency, and subsequent release benefits of ready subtasks, determine the subtasks to be processed. S2: In the current system state, perform counterfactual physical cost evaluation on the candidate execution locations of the current subtask to be processed in the vehicle local, edge node and cloud, and obtain the feasibility, counterfactual physical execution cost and physical process description of each candidate execution location; S3: Using the feasible candidate execution position with the lowest counterfactual physical execution cost as the physical benchmark, calculate the counterfactual physical execution cost difference and physical process description difference of the remaining feasible candidate execution positions relative to the physical benchmark; The long-term difference component is determined by the differential value reconstruction network, and the long-term difference component is combined with the counterfactual physical execution cost of the corresponding candidate execution location to form the reconstruction value; S4: Determine the feasible candidate execution location with the greatest reconstruction value as the target unloading location, and execute the corresponding target unloading scheme; S5: During training, taking the current system state as the common branch starting point, simulated unloading based on each feasible candidate execution position is executed in mutually isolated simulated state replicas, and a finite number of subsequent steps are performed. The generated simulated execution samples and real execution samples are used together to update the differential value reconstruction network; and the unloading phase of the next subtask is performed.

2. The method for offloading vehicle-to-everything (V2X) dependent tasks based on physical benchmark-guided long-term differential learning according to claim 1, characterized in that, Step S1 determines the current subtask to be processed, specifically including: Based on the current system status of the vehicle-to-everything (V2X) edge computing system Select the subtasks from the subtasks that have not yet been uninstalled, and choose the subtasks from the subtasks where all predecessors have been completed, and designate them as ready subtasks. For each ready subtask, determine the dynamic remaining critical timeframe based on the estimated execution time of the ready subtask and the longest unfinished execution chain from the ready subtask to the exit node of the dependent task; The time margin of the dynamic remaining critical timeliness is calculated based on the deadline of the dependent task and the decision time of the currently ready subtask, and the time urgency of the ready subtask is determined. In the current system state Without changing the actual task state, assuming that the ready subtask has been completed, determine the successor subtask that becomes ready only because the ready subtask has been completed, and calculate the subsequent release benefit of the ready subtask based on the computational amount and dependent data amount of the successor subtask. Based on the dynamic remaining critical timeliness, latency urgency, and subsequent release benefits, calculate the scheduling score for each ready subtask: in, Indicates ready subtasks The scheduling score, , and Representing ready subtasks The dynamic remaining critical timeliness, time delay urgency, and subsequent release benefits, , and These represent the corresponding weight coefficients; The ready subtask with the highest scheduling score is identified as the current subtask to be processed. .

3. The method for offloading vehicle-to-everything (V2X) dependent tasks based on physical benchmark-guided long-term differential learning according to claim 1, characterized in that, In step S2, the feasibility, counterfactual physical execution cost, and physical process description quantity of each candidate execution location are determined, specifically including: In the current system state Below, only the currently pending subtasks will be processed. The candidate execution locations are used as variables for counterfactual physical cost evaluation; the set of candidate execution locations is denoted as... This includes candidate execution locations corresponding to the vehicle's local location, edge nodes, and the cloud, whereby the candidate execution locations are denoted as... , The values ​​are 1, 2, and 3, representing the vehicle's local location, edge node, and cloud location, respectively. For each candidate execution location Based on the current subtask to be processed The location of the predecessor subtask result, the communication link status, the computing resource occupancy status, and the task queue status are used to determine the physical execution process from the predecessor subtask result to the current subtask reaching a usable state. Execute at each candidate location The corresponding physical execution process undergoes a counterfactual physical cost assessment: if a candidate execution location lacks the necessary communication conditions, vehicle dwell time conditions, or computing resource conditions, the candidate execution location is marked as infeasible; otherwise, it is marked as feasible, and a feasibility indicator is determined. ; The current system state is determined based on the feasibility identifier. The following is a set of feasible candidate execution locations : in, This indicates the feasibility of candidate execution locations, with a value of 1 indicating feasibility and 0 indicating infeasibility. For each feasible candidate execution location... Based on the time when the currently pending subtask enters the ready state The moment when its result reaches a usable state Determine the counterfactual execution delay : For each feasible candidate execution location The energy consumption is dynamically calculated based on the corresponding physical execution process. Wireless communication power consumption Energy consumption of wired link transmission Determine the energy consumption of counterfactual enforcement. : For each feasible candidate execution location The counterfactual physical execution cost corresponding to the candidate execution location is determined based on the counterfactual execution latency and counterfactual execution energy consumption. : in, and These represent the delay weighting coefficient and the energy consumption weighting coefficient, respectively. and These represent the normalized values ​​of time delay and energy consumption, respectively. This indicates the pre-defined invalid cost corresponding to an infeasible candidate execution location; Based on the upload latency, resource waiting latency, computation latency, routing transmission latency, result downlink latency, dependent data transmission latency, and transmission risks caused by vehicle movement in the physical execution process corresponding to each candidate execution location, a corresponding physical process descriptor is constructed. ; The feasibility of each candidate execution location, the counterfactual physical execution cost, and the physical process description quantity are used as the counterfactual physical cost assessment results and input into step S3 to determine the physical baseline and the physical process difference description quantity.

4. The method for offloading vehicle-to-everything (V2X) dependent tasks based on physical benchmark-guided long-term differential learning according to claim 1, characterized in that, In step S3, the long-term difference components are determined and reconstructed value is formed, specifically including: In the current system state Next, state encoding is performed to obtain the latent state vector. : in, Represents the state coding function. Indicates the state coding parameters; From feasible candidate execution positions In this process, the candidate execution location with the lowest counterfactual physical execution cost is determined as the physical benchmark. : For each feasible candidate execution location According to its physical process description quantity Physical benchmark The physical process description quantity, calculating the counterfactual physical execution cost difference. Difference between physical process descriptors ; the hidden state vector Difference in physical execution cost Difference between physical process descriptors Input a differential value reconstruction network, and use the differential value reconstruction network to determine feasible candidate execution locations. Relative to physical reference The original long-term difference component and differential gating coefficients are obtained, and the original long-term difference component is adjusted using the differential gating coefficients to obtain feasible candidate execution positions. long-term difference The reconstruction value is calculated based on the counterfactual physical execution cost and the long-run difference component. in, Indicates the currently pending subtasks. In the current system state Select feasible candidate execution locations The value of reconstruction.

5. The method for offloading vehicle-to-everything (V2X) dependent tasks based on physical benchmark-guided long-term differential learning according to claim 4, characterized in that, In step S4, determining the target unloading location specifically includes: Compare each feasible candidate execution location Based on the reconstruction value, the feasible candidate execution location with the greatest reconstruction value is determined as the target unloading location: Specifically, when two or more feasible candidate execution locations have the same maximum reconfiguration value, and this includes a physical baseline, the value is considered to be one of the highest possible values. At that time, the physical reference will be given priority. The target unloading location is determined, excluding the physical reference. Then, according to the preset priority order of vehicle local, edge node and cloud, the highest priority feasible candidate execution location is selected as the target unloading location; A target unloading plan is formed based on the physical execution process corresponding to the target unloading location, and the target unloading plan is submitted to the real system for execution.

6. The method for offloading vehicle-to-everything (V2X) dependent tasks based on physical benchmark-guided long-term differential learning according to claim 1, characterized in that, In step S5, the generation of the simulated execution sample specifically includes: Before writing the target unloading scheme described in S4 into the real system, the current system state is... The common branch starting point is fixed at each feasible candidate execution position; for each feasible candidate execution position... Copy the current system state This yields a copy of the corresponding simulated state: in, Indicates feasible candidate execution positions The corresponding simulated state copy, This represents a system state replication function; each simulated state copy is isolated from the real system state, and changes in communication resource usage, computing resource usage, task queue changes, and dependent task state changes in the copy are not written to the real system. At feasible candidate execution locations In the corresponding simulated state replica, the feasible candidate execution position is set as the first unload decision, and the subsequent process is carried out according to the event process of dependent data transmission, communication resource occupation, computing resource waiting, subtask calculation, result return and subsequent subtask release, so as to obtain the subsequent decision state, stage feedback and task termination identifier corresponding to the first unload decision. When the dependent task has not terminated, in subsequent decision states, the feasible candidate execution position with the lowest counterfactual physical execution cost is selected from the feasible candidate execution positions in the simulation state replica to continue execution in a finite number of steps; simulation progress stops when the preset maximum number of simulation steps is reached or the dependent task terminates; the maximum number of simulation steps is recorded as... And determine the cumulative simulation feedback based on the stage feedback generated by each simulation execution. The calculation method is as follows: in, Indicates the discount factor. Indicates the simulated state copy number The phase feedback generated during the execution of each simulation; This represents the actual number of steps advanced by the simulated cumulative feedback, and its value is determined by... The conditions for task termination are determined together; when the dependent task has not terminated and Maximum number of simulated steps not reached If the simulation continues, the system will continue to advance; otherwise, the simulation will stop. Based on the current system state The simulation execution sample is constructed by initial unloading decision, cumulative simulation feedback, simulation end status and task termination flag, and actual simulation steps. : The simulated execution samples and the real execution samples generated in the real system by the target unloading scheme in S4 are jointly stored in the experience replay pool for training and updating the differential value reconstruction network. The update method is as follows: in, This represents the experience replay pool. This represents the actual execution sample corresponding to the target unloading scheme. This indicates the operational status after a short-term implementation. Indicates the task termination flag. This represents the actual number of steps taken corresponding to the simulated cumulative feedback.

7. The method for offloading vehicle-to-everything (V2X) dependent tasks based on physical benchmark-guided long-term differential learning according to claim 6, characterized in that, In step S5, the update of the differential value reconstruction network specifically includes: Experience Replay Pool The Middle Each training sample is represented as: in, Indicates the first training samples, Indicates the initial running status of the sample. This indicates the unloading action corresponding to the sample. Indicates the end of the process. Indicates the task termination flag. This indicates the actual number of steps progressed corresponding to the cumulative feedback; if the first... If each training sample is a real execution sample, then This indicates the stage feedback generated during this actual execution. If the... If each training sample is a simulated execution sample, then This indicates the cumulative feedback of the corresponding simulation branch; Based on the communication link occupancy, computational resource waiting intensity, dependent data transmission volume, and connection changes caused by vehicle movement during the physical execution process corresponding to the training samples, the dynamic risk value of the training samples is determined, and the sample priority weight is determined based on the time difference error and dynamic risk value of the training samples. in, Indicates the first The sample priority weights of each training sample Indicates the first Temporal difference error of each training sample Indicates the first The dynamic risk value of each training sample. This represents the dynamic risk weighting coefficient. This indicates preventing positive numbers with a priority weight of zero. This represents the priority weight adjustment coefficient; training samples are obtained from the empirical replay pool according to the sampling probability that is positively correlated with the sample priority weight; Build parameters are Online reconstruction of value networks and parameters are Target Reconstruction of Value Network ; When the When the dependent task corresponding to each training sample has not terminated, in the end-of-progress state Next, following step S1, determine the current subtask to be processed and form a set of feasible candidate execution locations. The online reconstruction value network selects the feasible candidate execution position with the highest online reconstruction value from the set of feasible candidate execution positions as the subsequent action. The selection method is as follows: Calculate the value network based on the target. The training target value is calculated as follows: in, Indicates the first The training target value for each training sample. Indicates the relationship with the first The discount term corresponding to the actual number of steps advanced for each training sample. Indicates the end of the process; The time difference error is calculated based on the training target value and the reconstructed value of the online reconstructed value network. : The total loss is constructed based on the time difference loss, action-level gating constraint loss, low-risk baseline preservation constraint loss, and high-risk gating opening constraint loss: in, Indicates the total loss. This indicates the number of samples in a single training batch. Represents the error loss function. This indicates the action-level gating constraint loss. Low-risk benchmarks maintain constrained losses. This indicates the loss from opening the high-risk gating constraint. , , These represent the weighting coefficients of the corresponding loss terms; The parameters of the online reconstructed value network are updated according to minimizing the total loss, as follows: in, Indicates the learning rate; When the number of online value network reconstruction updates is an integer multiple of the target value network reconstruction update cycle, the parameters of the online value network reconstruction will be synchronized to the target value network reconstruction. The sample priority weights of the corresponding training samples are then re-determined based on the updated time difference error.