Internet of vehicles task offloading method and system based on graph perception of global load

Abstract

The present application relates to the technical field of Internet of Vehicles, and proposes an Internet of Vehicles task offloading method and system based on graph perception global load. The method acquires vehicle task information and queue state information under the current decision time slot, combines vehicle task offloading information and the connection relationship between vehicles and MEC servers in the preset historical time slot, uses a graph neural network to extract global load features representing historical load and vehicle-MEC graph structure, constructs an enhanced state vector, so that the offloading decision reflects the vehicle's current task state, queue waiting condition and overall system load distribution; further, based on the global load features, the target node load state corresponding to the candidate offloading action is evaluated, and the action that may cause node congestion is eliminated; finally, the enhanced state vector is input into a pre-trained deep reinforcement learning network, and the offloading decision is determined according to the value of the filtered candidate action, thereby reducing the risk of node congestion and queue waiting, and improving the rationality of task offloading and the balance of load utilization.

Description

Technical Field

[0001] This invention relates to the field of vehicle networking technology, specifically to a method and system for offloading vehicle networking tasks based on graph perception global load. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of vehicle-to-everything (V2X) technology, vehicles need to handle an increasing number of perception, communication, navigation, and collaborative control tasks during operation. Due to the limited local computing resources and storage capacity of onboard devices, relying solely on the vehicle itself to complete complex task processing is insufficient to meet the low latency, low power consumption, and high reliability requirements of time-sensitive applications. Therefore, it is necessary to improve existing task processing methods to enhance task processing efficiency in V2X environments, reduce task execution latency and system energy consumption, and improve overall resource utilization.

[0004] Existing devices or methods mainly employ local vehicle computing, offloading tasks to remote cloud servers, or allowing individual vehicles to independently make offloading decisions based on their own states. Due to the limited local computing power of vehicles and the distance between remote cloud servers and vehicles, tasks are susceptible to network fluctuations, increased transmission distance, and increased data volume during transmission, offloading, and result return. Furthermore, in scenarios where tasks occur randomly and in varying numbers, the task states, computing resources, and communication resources of each vehicle are interdependent. Existing methods typically only consider the locally optimal decisions of a single vehicle, failing to comprehensively consider the global resource coordination and globally optimal offloading requirements of the entire vehicle network system. This results in problems such as significant task processing latency, high energy consumption, unreasonable resource allocation, and difficulty in achieving globally optimal offloading decisions. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a method and system for task offloading in the Internet of Vehicles (IoV) based on graph-aware global load. It utilizes graph neural networks (GNNs) to extract global load features of the IoV system and performs joint optimization with a deep reinforcement learning strategy. This approach comprehensively considers the overall system load when making task offloading decisions, thereby improving task scheduling efficiency and enhancing system load balancing performance.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of this invention provides a method for offloading vehicle network tasks based on graph-aware global load, comprising the following steps: Obtain task information and queue status information of each vehicle in the vehicle-to-everything (V2X) system during the current decision-making time slot; Obtain task unloading information for each vehicle within a preset historical time slot, construct a historical data matrix, and construct an adjacency matrix based on the connection relationship between vehicles and MEC servers in the vehicle networking system. The historical data matrix and the adjacency matrix are input into the graph neural network for feature aggregation and feature transformation to obtain global load features that characterize the historical load and the vehicle-MEC graph structure. The problem of solving vehicle task unloading decision is constructed as a Markov decision process, and the task information, queue state information and global load characteristics are used to form the enhanced state vector of the vehicle's current time slot. The load status of the target node corresponding to the candidate offloading action is evaluated by global load characteristics, and the candidate actions that are expected to cause congestion of the target node are eliminated to obtain the filtered candidate actions. The enhanced state vector is input into a pre-trained deep reinforcement learning network to obtain the value evaluation results of each candidate action after screening. Based on the value evaluation results, the unloading decision scheme for the current task is determined.

[0007] A further technical solution involves inputting the historical data matrix and adjacency matrix into a graph neural network for feature aggregation and transformation to obtain global load features characterizing the historical load and vehicle-MEC graph structure. This process includes the following: Perform input alignment processing on the historical data matrix and the adjacency matrix; The aligned historical data matrix and adjacency matrix are then subjected to graph structure feature aggregation processing. A multilayer perceptron is used to perform feature transformation on the aggregated features to obtain the transformed features. The transformed features are processed by an activation function to obtain the activated feature matrix. The obtained activated feature matrix is ​​processed by a dropout layer to obtain the global load features.

[0008] A further technical solution uses task information, queue status information, and global load characteristics as states, task unloading decisions and edge computing resource allocation decisions as action spaces, and the weighted cost of task completion latency and energy consumption as optimization objectives to construct a Markov decision process for vehicle task unloading decisions.

[0009] A further technical solution involves constructing constraints on the optimization objective of a Markov decision process for vehicle task unloading decisions, including: Task information content The maximum allowed workload must not be exceeded. The edge computing resources allocated to vehicle tasks must not exceed the total computing power of the MEC server; each task can only choose one of two methods: local processing on the vehicle or offloading to the MEC server for processing; the corresponding processing method decision variable is a binary variable used to indicate whether the processing method is selected.

[0010] A further technical solution involves evaluating the load status of the target node corresponding to the candidate offloading action based on global load characteristics, eliminating candidate actions that are expected to cause congestion in the target node, and obtaining the filtered candidate actions, including the following steps: In the current decision time slot, the global load features output by the graph neural network are aggregated through a readout function to obtain a system-level load vector. : Iterate through all candidate unload actions corresponding to the current task, and determine the target node for each candidate action. Based on system-level load vectors The load status of the target node corresponding to each candidate action is estimated to obtain the load estimate. The filtered candidate actions are obtained by comparing the estimate with the set threshold.

[0011] Further technical solutions involve using the current global load characteristics during the training process of deep reinforcement learning networks. Aggregation yields system-level load vectors According to the system-level load vector The system congestion level and load balancing level are weighted with task latency and energy consumption to construct a reward function.

[0012] A further technical solution involves using graph neural networks to obtain the global load features at the next time step during the training process of deep reinforcement learning networks. The aggregation yields the system-level load vector. Then, the future congestion level of the system is calculated, and the future congestion level is introduced as a penalty term into the target Q value to construct the graph-aware target Q value.

[0013] A further technical solution involves performing graph-aware priority experience sampling during the training process of deep reinforcement learning networks. Specifically, based on the traditional priority experience replay mechanism based on temporal difference error, global load features extracted by graph neural networks are introduced and aggregated to obtain a system-level load vector. The sampling priority of experience samples is determined according to the system congestion level at the corresponding time of the experience sample and the change amplitude of the system-level load vector at adjacent time points, and training samples are extracted from the experience replay pool.

[0014] A further technical solution involves combining the reinforcement learning Q-network loss, load prediction error loss, and graph structure consistency constraint loss during the training process of the deep reinforcement learning network to construct a joint loss function, and then using the joint loss function to collaboratively optimize the network parameters and the parameters of the graph neural network.

[0015] A second aspect of the present invention provides a vehicle networking task offloading system based on graph-aware global load, comprising: The acquisition module is configured to acquire the task information and queue status information of each vehicle in the vehicle-to-everything (V2X) system under the current decision time slot; The graph construction module is configured to obtain the task unloading information of each vehicle within a preset historical time slot, construct a historical data matrix, and construct an adjacency matrix based on the connection relationship between vehicles and MEC servers in the vehicle networking system. The feature extraction module is configured to input the historical data matrix and the adjacency matrix into the graph neural network for feature aggregation and feature transformation, so as to obtain global load features that characterize the historical load and the vehicle-MEC graph structure. The enhanced state construction module is configured to construct the vehicle task unloading decision problem into a Markov decision process, and to construct the enhanced state vector of the vehicle's current time slot by combining task information, queue state information and global load characteristics. The action filtering module is configured to evaluate the load status of the target node corresponding to the candidate offloading action based on global load characteristics, eliminate candidate actions that are expected to cause congestion of the target node, and obtain the filtered candidate actions. The evaluation module is configured to input the enhanced state vector into a pre-trained deep reinforcement learning network to obtain the value evaluation results of each candidate action after screening, and to determine the unloading decision scheme for the current task based on the value evaluation results.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention, based on vehicle task information and queue status information in the current decision time slot, combines task unloading information of each vehicle in a preset historical time slot with the connection relationship between the vehicle and the MEC server. It utilizes a graph neural network to extract global load features representing the historical load and the vehicle-MEC graph structure. These global load features, along with task information and queue status information, form an enhanced state vector. This allows task unloading decisions to simultaneously reflect the vehicle's current task status, queue waiting status, and the overall system load distribution, avoiding unloading decisions based solely on the local state of a single vehicle. Simultaneously, by evaluating the target node load status corresponding to candidate unloading actions using global load features and eliminating candidate actions expected to cause congestion at the target node, it reduces the likelihood of tasks continuing to be unloaded to high-load nodes. Furthermore, the enhanced state vector is input into a pre-trained deep reinforcement learning network. Based on the value evaluation results of each selected candidate action, the unloading decision scheme for the current task is determined. This comprehensively considers task status, queue status, and global load status during task unloading, thereby reducing the risk of node congestion and queuing caused by concentrated task unloading and improving the rationality of task unloading decisions and the overall load utilization balance in the vehicle-to-everything (V2X) system.

[0017] The advantages of the present invention, as well as its additional advantages, will be described in detail in the following specific embodiments. Attached Figure Description

[0018] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.

[0019] Figure 1 This is a flowchart of the vehicle network task offloading method based on graph perception global load according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the vehicle-to-everything (V2X) edge computing task transmission and processing flow according to Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the reinforcement learning process of the deep reinforcement learning network in the graph-aware global load-based vehicle network task offloading method of Embodiment 1 of the present invention. Figure 4 This is a schematic diagram of the convergence process of the loss function of the method (GADRLO) in the simulation experiment of Embodiment 1 of the present invention; Figure 5 This is a comparison chart of the average time delay of different methods in the simulation experiment of Embodiment 1 of the present invention; Figure 6 This is a comparison chart of the average energy consumption of different methods in the simulation experiment of Embodiment 1 of the present invention; Figure 7This is a comparison chart of the average cost of different methods in the simulation experiment of Embodiment 1 of the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0022] It should be noted that the terminology used herein is for describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. It should be noted that, without conflict, the various embodiments and features within those embodiments can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0023] Example 1 In one or more of the technical solutions disclosed in the embodiments, such as Figures 1 to 7 As shown, a graph-aware global load balancing method for vehicle-to-everything (V2X) task offloading is applied to a V2X system comprising base stations, MEC servers, and multiple vehicle nodes, and includes the following steps: Step 1: Obtain the task information and queue status information of each vehicle in the vehicle-to-everything (V2X) system under the current decision time slot; Step 2: Obtain the task unloading information of each vehicle within the preset historical time slot, construct the historical data matrix H(t), and construct the adjacency matrix B according to the connection relationship between the vehicle and the MEC server in the vehicle network system; Step 3: Input the historical data matrix H(t) and the adjacency matrix B into the graph neural network for feature aggregation and feature transformation to obtain the global load features representing the historical load and vehicle-MEC graph structure. ; Step 4: Construct the vehicle task unloading decision problem as a Markov decision process, incorporating task information, queue state information, and global load characteristics. The enhanced state vector that constitutes the vehicle's current time slot; Step 5: Through global load characteristics The load status of the target node corresponding to the candidate offloading action is evaluated, and the candidate actions that are expected to cause congestion of the target node are eliminated to obtain the filtered candidate actions; Step 6: Input the enhanced state vector into the pre-trained deep reinforcement learning network to obtain the value evaluation results of each candidate action after screening, and determine the unloading decision scheme for the current task based on the value evaluation results. After determining the unloading decision scheme through steps 1 to 6 above, this embodiment determines whether the current task is to be processed locally in the vehicle or unloaded to the target MEC server for processing, and can obtain the task unloading result under the current decision time slot.

[0024] This embodiment, through the aforementioned technical solution, obtains vehicle task information and queue status information under the current decision time slot. It then combines this with task unloading information for each vehicle within a preset historical time slot and the connection relationship between the vehicle and the MEC server. A graph neural network is used to extract global load features representing the historical load and the vehicle-MEC graph structure. These global load features, along with task information and queue status information, constitute an enhanced state vector. This allows task unloading decisions to simultaneously reflect the vehicle's current task status, queue waiting status, and the overall system load distribution, avoiding unloading decisions based solely on the local status of a single vehicle. Simultaneously, the global load features are used to evaluate the target node load status corresponding to candidate unloading actions, eliminating candidate actions expected to cause congestion at the target node, thus reducing the likelihood of tasks continuing to be unloaded to high-load nodes. Furthermore, the enhanced state vector is input into a pre-trained deep reinforcement learning network. Based on the value evaluation results of each selected candidate action, the unloading decision scheme for the current task is determined. This allows for comprehensive consideration of task status, queue status, and global load status during task unloading, thereby reducing the risk of node congestion and queuing caused by concentrated task unloading and improving the rationality of task unloading decisions and the overall load utilization balance in the vehicle-to-everything (V2X) system.

[0025] like Figure 1 As shown, in step 1, the vehicle network edge computing environment of the vehicle network system includes multiple vehicles and at least one mobile edge computing MEC server, wherein the tasks generated by the vehicles can be processed locally in the vehicle or offloaded to the corresponding MEC server for processing. Optionally, the vehicle's task information includes at least the task data volume and task computing resource requirements; the vehicle status information includes at least the vehicle's local computing queue waiting time, the vehicle's transmission queue waiting time, and the length of the MEC server queue corresponding to the vehicle; the MEC status information includes at least the MEC server's computing power, available computing resources, and current load status. This embodiment uses an urban traffic scenario in a vehicle-to-everything (V2X) edge computing environment as an example. The V2X system consists of a base station (BS) deploying a mobile edge computing server (MEC) and several vehicles, denoted as I. In this traffic scenario, vehicles continuously travel on the road and make path selections when they reach an intersection. When a vehicle reaches an intersection, it can choose to turn left, turn right, or go straight through the intersection, with corresponding probabilities of […]. , and And satisfy The movement of vehicles within the road network is determined by the aforementioned probabilities, creating a dynamically changing traffic environment.

[0026] During each time interval, the vehicle may generate new computational tasks. Each task can be computed locally by the vehicle or offloaded to the MEC server for execution. The vehicle needs to select the appropriate task processing method based on the current system state to reduce system energy consumption while ensuring task completion efficiency. Since the vehicle's location, task arrival, and system load state all change over time, task offloading decisions need to be continuously made in a dynamic environment. Therefore, to characterize the vehicle's decision-making process at different times, this embodiment models the task offloading problem as a Markov Decision Process (MDP), thereby enabling the learning of the optimal task offloading strategy in a dynamic environment.

[0027] In step 2, the task unloading information of each vehicle within a preset historical time slot is obtained, and a historical data matrix H(t) is constructed. Specifically, within a preset historical time window, T consecutive historical time slots are selected, and the task unloading information of each vehicle to each MEC server in each time slot is obtained, including the task data volume or task load value; a two-dimensional matrix is ​​constructed, where each row of the two-dimensional matrix represents a historical time slot, each column represents the task queue between the vehicle and the MEC server, and the element value of the two-dimensional matrix is ​​the load information of the task queue between the vehicle and the MEC server, thus constructing the historical data matrix H(t), as shown in Table 1. Table 1. Example of a historical data matrix;

[0028] The task offload load information, referred to as the task load value, can include the amount of task data, the number of tasks, and the load intensity value after standardization within a unit time slot.

[0029] In this embodiment, to describe the connection relationship between vehicle nodes and MEC server nodes in the vehicle networking system, an adjacency matrix B is constructed as follows: a two-dimensional matrix is ​​constructed as the adjacency matrix, where the rows of the two-dimensional matrix correspond to vehicle nodes and the columns correspond to MEC server nodes; if vehicle node i and MEC server node j have a task offloading connection relationship, then the matrix element B(i,j) of the adjacency matrix takes the value of 1, otherwise it takes the value of 0.

[0030] like Figure 3 As shown, in the constructed graph structure, vehicle nodes and MEC nodes Solid lines can be used to represent communication edges, while dashed lines can represent edges where direct communication is not possible. In step 3, the historical data matrix H(t) and the adjacency matrix B are input into the graph neural network (GNN) for feature aggregation and feature transformation to obtain the global load features characterizing the historical load and the vehicle-MEC graph structure. The process includes the following: Step 31: Perform input alignment processing on the historical data matrix H(t) and the adjacency matrix B; The historical data matrix H(t) and adjacency matrix B obtained in step 2 are subjected to dimension matching and input alignment processing to make the vehicle-MEC task queue load information in the historical data matrix correspond to the vehicle node and MEC server node connection relationship in the adjacency matrix, so as to obtain input data that can be used for graph neural network calculation.

[0031] Step 32, Feature Aggregation: Perform graph structure feature aggregation processing on the aligned historical data matrix H(t) and the adjacency matrix B; Specifically, the connection relationships between vehicle nodes and MEC server nodes represented by the adjacency matrix B are applied to the historical data matrix H(t) to obtain the aggregated feature matrix. : (1); Step 33, Feature Transformation: Using a multilayer perceptron, feature transformation is performed on the aggregated features to obtain the transformed features; Taking a two-layer perceptron as an example, the aggregated feature matrix obtained in step 3.2 is... The first layer of the multilayer perceptron is input and subjected to a linear transformation to obtain the first layer output matrix. : (2); The first layer output matrix The input to the second layer of the multilayer perceptron undergoes further linear transformation to obtain the output matrix of the second layer. : (3); In the formula, , These are the weight matrices for layers 1 and 2, respectively. , These are the bias vectors for layers 1 and 2, respectively.

[0032] Step 34: Transform the features The activation function is applied to obtain the activated feature matrix; Optionally, the activation function can be the ReLU activation function to enhance the nonlinearity of the graphical feature representation and obtain the activated feature matrix. : (4); In the formula, This is the ReLU activation function.

[0033] Step 35: Process the obtained activated feature matrix Global load characteristics are obtained through dropout layer processing. ; Dropout layers can suppress overfitting in the network, allowing the activated feature matrix to... Element-wise multiplication with the mask matrix M yields the global load characteristics. Let the probability of discarding be... Then we have: (5); in, This involves element-wise multiplication; M is a mask matrix where elements are either 0 or 1, and the probability of discarding an element is set to p. Each element has a probability of... If we set it to 0, then the probability is... Set it to 1.

[0034] Specifically, the graph neural network used in this implementation is a graph convolutional network (GCN). Step 4: Construct the vehicle task unloading decision problem into a Markov decision process, including the following steps: Step 41: For the communication process of vehicle task offloading to edge server, construct the vehicle network system transmission model for vehicle task offloading, including the total transmission delay and communication energy consumption generated by the task during the offloading transmission phase; wherein, the total transmission delay includes transmission queue waiting time and wireless transmission time, and the communication energy consumption is determined according to the vehicle-side wireless transmission power and task transmission duration. Specifically, vehicle i generates a task in time slot t. When the vehicle chooses to offload the task to the MEC server for processing, the task first enters the vehicle-side transmission queue and is sent to the base station via a wireless link, and then forwarded by the base station to the MEC server. The latency of the task offloading phase mainly includes two parts: transmission queue waiting time and wireless transmission time. The energy consumption mainly comes from the wireless transmission process on the vehicle side.

[0035] The wireless communication rate between the vehicle and the base station can be expressed as: (6); in, For bandwidth, It's the transmission power. It is channel gain. It's noise.

[0036] When a vehicle offloads a task to the MEC server via a transmission queue, the transmission queue's transmission rate is... ,Task The time required to wait in the transmission queue before transmission. : (7); In the formula: In time slot The time slots during which tasks that previously arrived in the transmission queue were processed. ; If the task does not need to wait before transmission, then .

[0037] In the time slot The time slot for processing tasks arriving in the transmission queue for: (8); In the formula: For the task From vehicles The time required to transmit to the MEC server, ,in The amount of information in the task; The round-up operator indicates that, since the system uses a discrete time slot mechanism, continuous transmission time needs to be mapped to integer time slots. This indicates the wireless transmission rate between vehicle i and the MEC server; It represents the duration of a single discrete time slot.

[0038] Total time required for the task during the transmission phase : (9); In the time slot ,vehicle Energy consumption generated when tasks are transmitted to the MEC server via the transmission queue: (10); In the formula, Transmit power to the vehicle.

[0039] The above model can be used to calculate the transmission delay and communication energy consumption generated during the task unloading process, providing a theoretical basis for the optimization of subsequent task unloading strategies and system performance analysis.

[0040] Step 42: For the processing methods of local computing and edge computing for vehicle tasks, construct local computing models and edge computing models respectively; Step 421: Construct a local computing model, including the total computing latency and vehicle-side computing energy consumption generated when the task is processed locally in the vehicle. The total local computing latency includes the local computing queue waiting time and the local computing service time. The local computing energy consumption is determined by the vehicle CPU computing power and the duration of task computing. Set up vehicles The local computing power is When the task When processed locally on the vehicle, the data is entered into the vehicle's computation queue in a FIFO manner.

[0041] Task The time that needs to wait in the computation queue before being processed. The calculation formula is: (11); In the formula, In time slot The time slots during which tasks that previously arrived in the computation queue were processed. ; In the time slot The time slot for tasks arriving in the computation queue to complete processing. The calculation formula is: (12); In the formula, For the task The time required for local processing ,in Task processing density represents the number of CPU cycles required to process a unit of information. It represents the duration of a single discrete time slot.

[0042] Total time required for task to be processed locally on the vehicle for: (13); In the time slot ,vehicle Energy consumption generated by task processing through the computation queue for: (14); In the formula, Calculate the power for the vehicle's CPU.

[0043] Step 421: Construct an edge computing model, including edge computing latency and energy consumption generated during task processing on the MEC server. Specifically, when vehicle i chooses to offload a task to the MEC server for processing, the MEC server allocates computing resources using a generalized processor sharing approach based on the number of task queues currently in execution within the time slot. Specifically, the edge computing resources allocated to each task queue are determined based on the total computing power of the MEC server and the current number of task queues.

[0044] After completing the wireless transmission, the task enters the corresponding task queue in the MEC server. Assume the vehicle... The corresponding MEC server queue in the time slot The length is The queue is in a task-execution state if at least one task is being processed, or if a task has arrived at the queue. .set up For time slots The number of task queues in the MEC server. In each time slot, the computing resources of the MEC server are allocated to the task queues using a generalized processor sharing method. Assume the computing power of the MEC server is... Then the MEC server can allocate computing resources to each task queue as the number of task queues changes. ; The waiting time that the MEC server needs in the MEC server queue before processing a task. for: (15); In the formula, In time slot The time slots during which tasks that previously arrived in the MEC server queue were processed. .

[0045] In the time slot The time slot for tasks arriving in the MEC server queue to complete processing. for: (16); In the formula, The computing service time required for MEC to process tasks. The MEC server flexibly allocates computing resources based on the number of task queues, so the computing resources obtained by the same task queue may not be the same in different time slots.

[0046] A further technical solution involves calculating the computation service time of a task on the MEC server based on the amount of task information, task processing density, and allocated edge computing resources. It can be determined according to the following different situations: (1) If This indicates that the task can be fully processed within that time slot. The formula for calculation is: (17); in, This indicates the number of CPU cycles required to process a unit of data, i.e., the computing resources required to process a unit of data. It represents the duration of a single discrete time slot.

[0047] (2) When the computing resources allocated to the task queue by the MEC server in different time slots change, the remaining workload of the task is updated time slot by time slot according to the remaining computing workload of the task and the computing resources allocated to each time slot until the task is completed, thereby obtaining the edge computing service time of the task. ; like This indicates that the task can only process a portion of the data within that time slot. If discriminant one is satisfied: This indicates that after processing in the previous time slot, the task can be completely processed within this time slot. The formula for calculation is: (18); Otherwise, if the discriminant is not true, proceed to the next time slot for processing until the current task is completed.

[0048] In case (2) The formula for calculation is: (19); In the formula, n is the number of comparisons between discriminant 1 and 0, and n≥2.

[0049] Total time required for the task to be processed on the MEC server: (20); In the time slot ,vehicle Energy consumption generated by tasks processed by the MEC server: (twenty one); In the formula, This refers to the computing power of the MEC server.

[0050] Furthermore, the total task delay is determined based on the task processing method. Total energy consumption : (twenty two); The resulting delay (twenty three); Different task processing methods differ in terms of computational latency and energy consumption. A reasonable task offloading strategy is needed to select between local computing and edge computing to optimize the overall system performance.

[0051] Because the vehicle-to-everything (V2X) environment is highly dynamic, the vehicle's location, task arrival status, and the resource status of edge servers all change constantly over time. Therefore, the task offloading decision at each moment is affected by the current system state. To characterize this dynamic decision-making process, this embodiment models the V2X task offloading problem as an optimal decision problem under a Markov Decision Process (MDP). Through the MDP model, the vehicle can select an appropriate offloading strategy based on the current system state and gradually optimize the overall system performance during the continuous decision-making process.

[0052] Step 43: Construct a Markov decision process based on the transmission model and computation model: using task information, queue state information, and global load characteristics. As the state, the action space is defined by task offloading decisions and edge computing resource allocation decisions, and the optimization objective is a Markov decision process for vehicle task offloading decisions. Task completion latency and energy consumption are calculated using a transmission model and a computation model. In this embodiment, task information, queue status information, and global load characteristics are used. As a state, a state space is constructed; queue state information can include the vehicle local computing queue state, the vehicle transmission queue state, and the current number of task queues on the MEC server. The MDP provides a comprehensive description of the vehicle's state in an environment-oriented manner. In this embodiment, the vehicle... In the time slot The state is represented as: (twenty four); in, This represents the task information generated by vehicle i in time slot t. Indicates the vehicle's local computation queue status. Indicates the status of the vehicle transmission queue. This indicates the current number of task queues on the MEC server. This represents the global load representation of the system obtained after processing by the graph neural network in step 3.

[0053] Obtain partial vehicle status information This includes vehicle position and speed, task size and deadline, as well as vehicle computing power and current load information; a state representation is constructed based on the vehicle's local state information. ; In this embodiment, This is used to characterize the overall load state of the current vehicle-to-everything (V2X) edge computing system, and will play a further role in subsequent solution processes such as reward correction, candidate action selection, target Q-value construction, and experience-first sampling; unlike task offloading methods that only rely on local state information, it introduces... Subsequently, the vehicle can comprehensively consider the overall load changes of the system during the decision-making process, thereby improving the globality of the task unloading strategy.

[0054] The action space is defined by task offloading decisions and edge computing resource allocation decisions, which represent the behavior and decisions of tasks.

[0055] set up For vehicles In the time slot The uninstallation actions taken at that time involve uninstallation decisions to determine whether to uninstall the task. It can be represented as: (25); in: Indicates how the task is executed; Indicates the target to be unloaded from the task.

[0056] Configurable, when When =0, it indicates that the vehicle is performing computational tasks locally; when When =1, it means that the vehicle will offload the task to the MEC server; Indicates the MEC server number selected for the vehicle; The action space can be represented as: ; ; Where M1 represents the number of MEC servers in the system.

[0057] Furthermore, the optimization objective of constructing the Markov decision process for vehicle task unloading decision is: to minimize task completion delay and energy consumption as the optimization objective; Constraints include: Task information content The maximum allowed workload must not be exceeded. The corresponding constraint is C1; edge computing resources allocated to vehicle tasks. It must not exceed the total computing power of the MEC server. The corresponding constraint is C2; each task can only choose one of two methods: local processing on the vehicle or offloading to the MEC server for processing, corresponding to constraint C3; the corresponding processing method decision variable is a binary variable used to indicate whether the processing method is selected, corresponding to constraint C4; The problem description of Markov decision processes is as follows: P1:

[0058] P2: (26); st C1:

[0059] C2:

[0060] C3:

[0061] C4:

[0062] In the formula: optimization problem P1 is to minimize the average latency of task completion; optimization problem P2 is to minimize the average energy consumption of task completion; constraint C1 is that the amount of information in the task cannot exceed the specified maximum amount of information; constraint C2 is that, in edge computing, the computing resources allocated to the task queue cannot exceed the total computing power of the MEC server; constraints C3 and C4 indicate that there are only two choices after the task is generated. To process locally on the vehicle, This is for unloading to the MEC server.

[0063] In traditional reinforcement learning methods, vehicle agents typically search and evaluate directly within the complete action space. However, in the vehicle-to-everything (V2X) edge computing environment, some actions are not actually reasonable under the current system state. For example, when a MEC node is already under high load, continuing to offload tasks to that node often leads to additional queuing latency and energy consumption; similarly, when local vehicle resources are nearing saturation, continuing to execute locally may not be the optimal decision. Therefore, before selecting actions, it is necessary to pre-screen the action space using global load information to eliminate obviously inferior candidate actions in advance, thereby reducing ineffective exploration and improving policy learning efficiency.

[0064] Further, step 5 is candidate action screening: the load status of the target node corresponding to the candidate offloading action is evaluated based on global load characteristics, and candidate actions that are expected to cause congestion of the target node are eliminated to obtain the screened candidate actions, including the following steps: Step 51: In the current decision time slot, output the global load features from the graph neural network. The system-level load vector is obtained by aggregation through the readout function. : (27); in, It characterizes the overall congestion level and load distribution of the system.

[0065] Step 52: Traverse all candidate unloading actions corresponding to the current task, and determine the target node corresponding to each candidate action; Specifically, when the candidate action is local processing, the target node is the local computing node of the current vehicle; when the candidate action is unloading processing, the target node is the selected MEC server node.

[0066] Step 53: Based on system-level load vector The load state of the target node corresponding to each candidate action is estimated to obtain the load estimate. Candidate actions are obtained by comparing them with a set threshold.

[0067] The method for estimating the load estimate is as follows: for each candidate action 'a', determine the target MEC node corresponding to that action, and extract the system-level load vector from the graph neural network output. Extract the load status and resource availability status of the target node, and combine this with the computing resources required by the current task to calculate the estimated load of the target node corresponding to the candidate action. Its expression is: ; in, This represents the load estimate of the target node corresponding to candidate action a; This indicates the current load rate of target node j; This indicates the current task queue load of target node j; This indicates the computing resources required by vehicle i for its current task; This represents the available computing resources for target node j in time slot t; β is the weighting coefficient, and 0 ≤ β ≤ 1.

[0068] when If the load exceeds the set threshold, the candidate action is determined to be likely to cause congestion on the target node and is removed; otherwise, the candidate action is retained in the filtered candidate action set.

[0069] This embodiment is based on graph-level load vectors. To estimate the potential load changes on the target node caused by different actions, the action gating function is defined as follows: (28); in, This represents the target node load estimate corresponding to action a under global graph representation conditions. This is the load threshold.

[0070] when When =0, it means that the action may cause the target node to be in a congested state, so it is not included in the current candidate action set.

[0071] The resulting set of actions after filtering is as follows: (29); This mechanism allows for selection from a set of actions more likely to yield reasonable returns during subsequent solution processes, thereby improving the effectiveness of decision-making and the stability of training.

[0072] In step 6, the deep reinforcement learning network is the core learning model used to realize vehicle task offloading decision. It is built based on deep reinforcement learning (DRL) and learns the optimal task offloading strategy under different system states by interacting with the vehicle network edge computing environment.

[0073] Specifically, the deep reinforcement learning network adopts a value function-based decision-making mechanism, using the enhanced state vector constructed in step 4 as input, and evaluates the value of each candidate action in the candidate action set after screening in step 5, in order to determine the optimal unloading decision scheme for the current task.

[0074] In this embodiment, the deep reinforcement learning network adopts a double deep Q-network (DDQN), which includes an evaluation network and a target network; An evaluation network is used to estimate the value of each candidate action in the current state, and its input is the system state vector. The output is the Q value corresponding to each action: Optionally, the evaluation network employs a deep neural network to approximate the solution of the state-action value function; The target network is used to calculate the target Q-value during training. Its network structure is the same as that of the evaluation network, and the network parameters are evaluated synchronously according to a preset update cycle to improve the stability of the training process and reduce value estimation oscillations.

[0075] In step 6, the enhanced state vector will be... Input the pre-trained deep reinforcement learning network to obtain the value evaluation results of each candidate action after screening. Based on the value evaluation results, determine the unloading decision scheme for the current task. Input the augmented state vector into the deep reinforcement learning network, and use the evaluation network to calculate the action value corresponding to each candidate action after screening in step 5, to obtain the value evaluation result of the current task, i.e., the value evaluation result. : (30); in, This represents the set of candidate actions formed after the candidate actions are filtered in step 5. This indicates the evaluation of network parameters. Indicates the action to be performed in the current enhanced state. The valuation results; Based on the value assessment results of each candidate action, the action with the highest value is selected as the final unloading decision for the current task, and the corresponding task processing operation is executed.

[0076] Traditional task offloading reward functions often focus primarily on two local metrics: task processing latency and energy consumption. While these can reflect the execution quality of individual tasks, they fail to capture the impact of a decision on the overall system operation. In a vehicle-to-everything (V2X) edge computing environment, focusing solely on the local gains of the current task may lead to a concentration of tasks being offloaded to some MEC servers, resulting in server congestion and resource utilization imbalances.

[0077] Further technical solutions involve using the current global load characteristics during the training process of deep reinforcement learning networks. Aggregation yields system-level load vectors According to the system-level load vector The congestion level and load balancing level of the computing system are weighted with task latency and energy consumption to construct a graph-aware reward function. Specifically, based on task latency and energy consumption, a system congestion penalty term and a load balancing incentive term are introduced to construct a graph-aware reward function: (31); In the formula, and These represent the weighting coefficients for the delay term and the energy consumption term, respectively. Indicates the degree of system congestion; Indicates the system load balancing level. and These are the congestion penalty weight and the equilibrium incentive weight, respectively.

[0078] In this embodiment, by constructing a graph-aware reward function, the reward function is calculated after the agent makes a task processing decision during training. This allows the deep reinforcement learning network to simultaneously consider task latency, task energy consumption, and the overall system load during policy learning. Compared to methods that construct reward functions solely based on task latency and energy consumption, this embodiment reduces the probability of tasks being concentratedly offloaded to high-load MEC servers, reduces server congestion risk, improves the balance of edge computing resource utilization, and enhances the stability of the task offloading strategy in dynamic vehicular network environments.

[0079] One specific implementation method, system congestion level The calculation method is as follows: ; Where: M2 represents the total number of MEC server nodes in the system; This represents the length of the task queue for the m-th MEC server in time slot t. This represents the current resource load rate of the m-th MEC server; and Let be the weighting coefficients, and satisfy: =1; Resource load rate: ; in: This represents the computing resources allocated to the m-th MEC server within time slot t; This represents the maximum computing power of the m-th MEC server.

[0080] One specific implementation method is the system load balancing level. The calculation method is as follows: Let the resource load rate of the m-th MEC server in time slot t be... As shown in the previous formula, the system average load rate for: ; ; The objective Q-value refers to the value of the target action used to guide the evaluation of network parameter updates during deep reinforcement learning network training. In traditional DQN or DDQN methods, the objective Q-value is typically determined by both the immediate reward and the maximum action value in the next state. While this approach reflects the immediate benefit of the current action and the potential reward in future states, it does not explicitly consider the impact of future system load changes on the value of the current offloading decision. In the edge computing environment of connected vehicles, when a task is offloaded to a heavily loaded MEC server, it may lead to increased server queue length, exacerbated system congestion, and resource utilization imbalances in the next time step, thus affecting the processing efficiency of subsequent tasks. Relying solely on the traditional objective Q-value for value updates can easily make the model insufficiently sensitive to long-term load evolution trends in the system.

[0081] Therefore, this embodiment further introduces the global load characteristics of the next moment in the process of constructing the target Q value, estimates the future system load state, and introduces the future congestion risk into the target Q value calculation process. This allows the agent to consider both the immediate benefits and the future system load changes when updating the value of the current action, thereby improving the accuracy of the target Q value assessment and reducing the risk of performance degradation due to server congestion during task unloading.

[0082] A further technical solution involves using graph neural networks to obtain the global load features at the next time step during the training process of deep reinforcement learning networks. The aggregation yields the system-level load vector. Then, the future congestion level of the system is calculated, and the future congestion level is used as a penalty term to introduce the target Q value, thus constructing a graph-aware target Q value; The next time-map representation is further introduced into the construction of the target Q value. The corresponding future load estimate, thus forming the Q-value of the graph-aware target. : (32); in, Indicates the immediate reward at the current moment; As a discount factor, Q represents the set of candidate actions after being filtered by graph representation at the next time step; To represent the target network in the next state Next action The value of the output action Indicates the target network parameters; This indicates the future system congestion level, calculated using the same method as the current system congestion level, based on the global load characteristics output by the graphical neural network at the next time step. It is used to characterize the impact of the current offloading decision on the system load state at the next time step. This represents the future congestion penalty coefficient.

[0083] In this embodiment, by constructing a graph-aware target Q-value, the deep reinforcement learning network, when updating the value of the current action, considers not only the current immediate reward and the estimated value of the next state action, but also the potential future system congestion risk caused by the current unloading action. Compared to the traditional approach of constructing target values ​​based solely on immediate rewards and the value of the next state action, this embodiment reduces the probability of the model selecting unloading actions with high short-term gains but potentially leading to subsequent server congestion. It improves the task unloading strategy's ability to perceive system load evolution trends, and enhances the long-term optimization capability and global load balancing capability of the strategy learning.

[0084] In the training of deep reinforcement learning networks, the experience replay mechanism is used to store experience samples generated by the interaction between the agent on the vehicle and the environment, and to draw samples from the experience replay pool for training when the network parameters are updated. Traditional experience replay methods typically use uniform random sampling, meaning that each experience sample is drawn with equal probability. However, in the task offloading scenario of vehicle-to-everything (V2X) edge computing, different experience samples have different importance for policy optimization. For example, experience samples generated under high congestion conditions of MEC servers, or under scenarios of drastic changes in system load, often better reflect the difficulties of task offloading decisions in complex dynamic environments. If uniform random sampling is still used, these key experience samples may not be fully utilized, thus affecting the training efficiency and policy convergence effect of the deep reinforcement learning network.

[0085] A further technical solution involves performing graph-aware priority experience sampling during the training process of the deep reinforcement learning network. Specifically, based on the traditional priority experience replay mechanism based on temporal difference error, global load features extracted by graph neural networks are introduced. These features are then combined with the system congestion level at the corresponding time of the experience sample and the load change magnitude at adjacent time points to jointly determine the sampling priority of the experience samples. Priority is given to selecting key experience samples with high congestion or large system load changes for training, thereby improving the utilization efficiency of key samples and enhancing the training efficiency and policy convergence effect of the deep reinforcement learning network.

[0086] Building upon the traditional priority-based empirical replay method based on time-series differential errors, this paper further introduces graph representations to depict congestion intensity and load variation amplitude, defining the priority of empirical samples as follows: (33); in, Temporal difference error is used to characterize the deviation between the current evaluation network output value and the target Q value. This indicates the level of system congestion at the moment the experience was generated. This indicates the magnitude of change represented by the graph at adjacent time points. , , These are weighting coefficients. Higher priority indicates that the sample better reflects the system's decision-making characteristics under critical conditions, and therefore should be sampled more frequently. This mechanism allows the training process to focus more on complex, high-value empirical samples, thereby improving sample utilization efficiency.

[0087] In this embodiment, by constructing a graph-aware priority experience sampling mechanism, the training process can focus more on experience samples that are of high value for optimizing the task offloading strategy. Compared with uniform random experience sampling, this embodiment can improve the utilization rate of experience samples under conditions of high congestion, drastic load changes, and large prediction errors. This allows the deep reinforcement learning network to learn the offloading patterns in complex vehicle-to-everything (V2X) edge computing environments more quickly, thereby improving sample utilization efficiency, accelerating model convergence, and enhancing the stability and robustness of the task offloading strategy under dynamic load scenarios.

[0088] During the training of deep reinforcement learning networks, if a graph neural network branch is used only as a state feature extraction module, its parameters are mainly updated indirectly by relying on the main loss of reinforcement learning. While this approach enables the graph neural network to learn a certain load representation capability, it is difficult to guarantee that the extracted graph features can fully support the modeling of future load changes and the optimization of task offloading strategies. Without additional constraint mechanisms, the graph neural network tends to be biased towards learning current state features, making it difficult to fully explore the spatiotemporal structural information that plays an important role in long-term task scheduling decisions.

[0089] A further technical solution involves combining the reinforcement learning Q-network loss, load prediction error loss, and graph structure consistency constraint loss during the training process of the deep reinforcement learning network to construct a joint loss function, and then using the joint loss function to collaboratively optimize the network parameters and the parameters of the graph neural network.

[0090] Specifically, the joint loss function includes reinforcement learning Q network loss, load prediction error loss, and graph structure consistency constraint loss.

[0091] set up The future load vector predicted by the GNN branch. For the actual observed load in the environment, the load prediction error loss is defined as: (34); This loss is used to improve the GNN branch's ability to predict future system load changes.

[0092] To maintain the continuity and stability of adjacent time-phase embeddings in the structure, define Figure 1 Consistency constraint loss is: (35); Here, Φ(.) represents the temporal mapping function, used to characterize the evolutionary relationship of graph embeddings between adjacent time points. This constraint can prevent unreasonable and drastic fluctuations in graph features over time.

[0093] Finally, the joint loss function is obtained by weighting and summing the reinforcement learning Q-network loss, load prediction error loss, and graph structure consistency constraint loss: (36); in, This represents the Q-network loss in reinforcement learning. and The balancing coefficient is used. This joint optimization mechanism enables the GNN branches to not only undertake the task of state representation, but also to predict future loads and... Figure 1 Consistency constraints actively participate in policy network training, thereby improving global load modeling capabilities and decision robustness.

[0094] In this embodiment, by constructing a joint loss function, the graph neural network branches are no longer merely static feature extraction modules, but actively participate in the training process of the deep reinforcement learning network through load prediction error loss and graph structure consistency constraint loss. Therefore, the global load representation extracted by the graph neural network can not only be used to enhance state inputs, but also further support modeling future load changes, constructing target Q-values, and optimizing task offloading strategies, thereby improving the long-term optimization capability of task scheduling strategies and the system's load balancing capability.

[0095] In one specific implementation, to avoid the auxiliary loss from excessively interfering with the main reinforcement learning task, the gradients of the graph neural network branch parameters are pruned during training to satisfy the following: (37); in, Here are the GNN parameters, and δ is the gradient threshold. This represents the gradient of the joint loss function with respect to the branch parameters of the graph neural network; through the above design, the global load characteristics... It not only participates in state construction, but also directly participates in the optimization objective of policy learning, thereby achieving collaborative training of the GNN branch and the reinforcement learning branch.

[0096] In this embodiment, by introducing graph representation information into multiple stages such as action selection, reward construction, value update, and experience sample sampling, the global load information extracted by the graph neural network is no longer merely a supplementary feature to the state input, but deeply involved in the generation, evaluation, and optimization of task offloading strategies. Specifically, in the action selection stage, using graph representation to filter offloading actions corresponding to highly congested nodes can reduce the search for invalid actions and lower the probability of tasks being offloaded to overloaded MEC servers; in the reward construction stage, by introducing system congestion penalty terms and load balancing incentive terms, the agent can be guided to reduce task latency and energy consumption while taking into account the overall system resource utilization efficiency; in the value update stage, by introducing future congestion risk into the target Q value, the model can consider the subsequent load evolution trend of the system when updating the value of the current action; in the experience sampling stage, by increasing the sampling probability of experience samples under high congestion and drastic load change states, the model's learning ability for complex key scenarios can be enhanced.

[0097] Therefore, this embodiment enables the collaborative optimization of graph neural networks and deep reinforcement learning networks, allowing global load information to permeate the entire process of task offloading strategy learning. Compared to methods that rely solely on local vehicle states for offloading decisions, this embodiment effectively reduces the risk of server congestion caused by concentrated task offloading, improves load balancing among MEC servers, reduces task queuing time, lowers overall latency and energy consumption during task processing, and enhances the stability, robustness, and long-term optimization capabilities of the offloading strategy under dynamic vehicle quantity changes and sudden task load scenarios.

[0098] Further technical solutions involve the training process of a network model composed of graph neural networks and deep reinforcement learning networks, such as... Figure 3 As shown, it includes the following steps: Step S1: Initialize training parameters and obtain the current training state; Initialize the vehicle-to-everything (V2X) edge computing training environment by setting the number of vehicles, the number of MEC servers, vehicle computing power, MEC server computing power, wireless communication parameters, task arrival parameters, time slot length, and maximum number of training rounds.

[0099] Initialize and evaluate network parameters θ, target network parameters θ Graph Neural Network Branch Parameters Set the priority experience replay pool M3, and set the discount factor, learning rate, exploration rate, batch sample size, and target network update cycle.

[0100] In each time slot t of each training round, obtain the current vehicle task status, vehicle local computing queue status, transmission queue status, MEC server queue status, server load status, and wireless link status.

[0101] Step S2: Based on the current training state, calculate the global load characteristics using a graph neural network. Then, feature aggregation is performed to obtain the system-level load vector. And combine the local state of the vehicle to obtain the enhanced state vector; the method is the same as step 4; Step S3: Through global load characteristics The load status of the target node corresponding to the candidate offloading action is evaluated, and the candidate actions that are expected to cause congestion of the target node are eliminated to obtain the filtered candidate actions; Step S4: Input the enhanced state vector into the evaluation network of the deep reinforcement learning network to obtain the action value of each candidate action, and select the unloading action for the current task based on the ε-greedy policy; Among them, the ε-greedy strategy is a classic action selection strategy used in reinforcement learning to balance exploration and exploitation; Step S5: Execute the action and calculate the reward. Execute the current action. If local processing is selected, the task enters the vehicle local calculation queue. If unloading processing is selected, the task enters the transmission queue and is unloaded to the corresponding MEC server for processing. Calculate the task latency and task energy consumption based on the task execution result, and calculate the system congestion level and load balancing level based on the system-level load vector to obtain the graph-aware reward. See formula (31). Step S6: Generate experience samples: After the action is executed, obtain the state of the next time slot and calculate the global load characteristics of the next time slot. System-level load vector Combine the enhanced state vector of the next time slot with the empirical sample and store it in the empirical replay pool M3. Step S7: When the number of samples in the experience replay pool reaches the preset training conditions, calculate the sample priority according to the graph perception priority experience sampling method (Formula 33), and extract a small batch of training samples from the experience replay pool according to the sample priority. Step S8: Calculate the Q-value of the graph perception target: For the extracted small batch of training samples, calculate the Q-value of the graph perception target based on the next state action value output by the target network and the system congestion level at the next moment, as shown in Formula 32. Step S9: Calculate the joint loss based on the current action value output by the evaluation network and the Q value of the graph-aware target, and prune the gradients of the graph neural network branch parameters; update the evaluation network parameters and graph neural network branch parameters based on the joint loss, and update the target network parameters according to the preset update cycle; iterate the training until the network training stopping condition is met to obtain the trained joint network.

[0102] To illustrate the effectiveness of this embodiment, a simulation experiment was conducted, and the details are as follows; The simulation experiment simulates a vehicle-to-everything (V2X) edge computing scenario, which includes 10 vehicles and 1 MEC server node. The vehicles move in the road network and randomly generate computing tasks. Each task can be processed locally or offloaded to the MEC server for execution.

[0103] To ensure the stability of the experimental results, each experiment was run for 1000 training episodes, and the average of the results from multiple experiments was taken as the final result. The reinforcement learning model used the Adam optimizer for parameter updates, with a discount factor set to 0.99 and a learning rate set to 0.01. The experience replay pool had a capacity of 200, and the target network was updated every 500 steps.

[0104] The baseline method used in the comparative experiment is as follows: (1) Local computing (LC): Each vehicle uses its local computing power to process all the tasks generated.

[0105] (2) Full offloading (FO): All tasks generated by each vehicle are offloaded to the MEC server for processing.

[0106] (3) Random decision (RD): Each vehicle processes the generated task randomly, either locally or offloaded to the MEC server with the same probability.

[0107] (4) Traditional double deep Q-network (DDQN) algorithm: Each vehicle processes the task according to the unloading decision made by the DDQN algorithm.

[0108] Figure 3 The diagram shows the loss function variation curve of the GADRLO algorithm during training. It can be observed that in the early stages of training, the loss function fluctuates significantly because the reinforcement learning model needs to continuously explore different action combinations. As the number of training epochs increases, the model gradually learns a more stable task offloading strategy, and the loss function begins to gradually decrease. When the simulation training reaches approximately 310 epochs, the downward trend of the loss function slows down significantly; when the training reaches approximately 520 epochs, the loss function basically stabilizes.

[0109] This result shows that as the training process continues, the reinforcement learning model can gradually converge to a stable policy space, thereby achieving effective learning of vehicle-to-everything (V2X) task offloading decisions.

[0110] Compared with traditional reinforcement learning algorithms that rely solely on local state information, the method in this embodiment (GADRLO) introduces global load features extracted by graph neural networks, enabling the agent to obtain more comprehensive environmental information during the decision-making process, thereby helping to improve the stability and convergence efficiency of policy learning.

[0111] The performance is analyzed below through comparative experiments; 1) Comparison of average task latency; As the system task generation rate changes, the average task latency varies under different methods as follows: Figure 5 As shown.

[0112] When the task generation rate is low, the average task latency differences among the methods are not significant. This is because, with a small number of tasks, the system can complete task processing relatively quickly regardless of the strategy employed. However, as the number of tasks gradually increases, the average latency of the local computation method (LC) grows rapidly. This is due to the limited local computing power of the vehicle; as the number of tasks continues to increase, a large number of tasks need to wait in the queue for processing, resulting in a significant increase in system latency.

[0113] The Full Offload (FO) method can fully utilize the computing power of the MEC server when the number of tasks is small, resulting in low average latency. However, as the number of tasks increases, a large number of tasks are offloaded to the MEC server, causing congestion in the server's computing queue and a rapid increase in average latency.

[0114] In contrast, RD, DDQN, and the GADRLO method proposed in this embodiment can dynamically select between local and edge computing, thereby effectively alleviating the task queuing problem. The GADRLO method, in particular, introduces a graph neural network to extract global system load information and combines it with deep reinforcement learning for decision-making, enabling tasks to be more rationally allocated to different computing nodes. Therefore, it can maintain a low average latency even when the number of tasks increases. When the task generation rate is high, compared with LC, FO, RD, and DDQN methods, such as... Figure 5 As shown, the average task latency of GADRLO in this embodiment is reduced by approximately 88.9%, 49.9%, 27.6%, and 9.4%, respectively.

[0115] 2) Average energy consumption comparison; The variation of system average energy consumption with task generation rate under different methods is as follows: Figure 6 As shown.

[0116] For the LC method, since tasks are always executed locally on the vehicle, system energy consumption is mainly determined by the vehicle's computing power. As the number of tasks increases, a large number of tasks wait in the queue for processing, and the waiting process does not generate additional energy consumption, so the average energy consumption shows a gradual downward trend.

[0117] For the FO method, all tasks are offloaded to the MEC server for processing. When the number of tasks is small, the server can complete task processing quickly, resulting in low energy consumption. However, as the number of tasks increases, server resources are shared by a large number of tasks, increasing task processing time and leading to a significant increase in overall system energy consumption.

[0118] The RD and DDQN methods can achieve task load distribution to a certain extent, thus slowing down the rate of energy consumption growth. However, due to the lack of accurate perception of the overall system load status, their decision-making still involves a certain degree of randomness. In contrast, the GADRLO method utilizes graph neural networks to extract global system load characteristics and dynamically adjusts task offloading strategies through reinforcement learning, enabling tasks to be more rationally distributed between vehicles and MEC servers, thereby effectively reducing overall system energy consumption. When the number of tasks is high, GADRLO reduces average energy consumption by approximately 43.6%, 25.4%, and 4.2% compared to the FO, RD, and DDQN methods, respectively.

[0119] This performance improvement is primarily attributed to the global load modeling capabilities of graph neural networks and the graph-aware reinforcement learning strategy optimization mechanism introduced in this paper. By comprehensively considering the overall resource distribution of the system during the task offloading decision process, GADRLO can effectively avoid tasks being concentrated and offloaded to a single server node, thereby achieving more balanced resource allocation and more efficient system scheduling.

[0120] To comprehensively evaluate system performance, this paper further compares the performance of different methods in terms of overall system cost. The overall cost considers both task processing latency and system energy consumption, and its variation is shown below. Figure 7 As shown.

[0121] Because the LC method relies solely on the vehicle's local computing power, as the number of tasks increases, a large number of tasks wait in the queue for processing, leading to a significant increase in system latency and thus the highest overall cost. While the FO method can utilize MEC servers for computation, the server load becomes too high when the number of tasks is large, resulting in a simultaneous increase in processing latency and energy consumption; its overall cost is second only to the LC method.

[0122] In comparison, the RD and DDQN methods can alleviate system congestion to some extent, thus their overall cost is significantly lower than that of a single processing strategy. Among them, the DDQN method optimizes the task offloading strategy through reinforcement learning, achieving better performance in complex and dynamic environments.

[0123] The GADRLO method proposed in this embodiment combines graph neural networks and deep reinforcement learning, enabling the agent to fully consider the overall system load during the decision-making process, thereby reducing task latency and system energy consumption. Experimental results show that the GADRLO method achieves the lowest overall system cost under different task load conditions. When the number of tasks is high, compared with other methods, the overall cost of GADRLO is reduced by approximately 74.5%, 46.1%, 26.3%, and 6.2%, respectively.

[0124] Example 2 Based on Embodiment 1, this embodiment provides a vehicle-to-everything (V2X) task offloading system based on graph-aware global load balancing, including: The acquisition module is configured to acquire the task information and queue status information of each vehicle in the vehicle-to-everything (V2X) system under the current decision time slot; The graph construction module is configured to obtain the task unloading information of each vehicle within a preset historical time slot, construct a historical data matrix H(t), and construct an adjacency matrix B based on the connection relationship between the vehicle and the MEC server in the vehicle networking system. The feature extraction module is configured to input the historical data matrix H(t) and the adjacency matrix B into the graph neural network for feature aggregation and feature transformation, thereby obtaining global load features characterizing the historical load and the vehicle-MEC graph structure. ; The enhanced state construction module is configured to construct the vehicle task unloading decision problem as a Markov decision process, incorporating task information, queue state information, and global load characteristics. The enhanced state vector that constitutes the vehicle's current time slot; The action filtering module is configured to use global load characteristics. The load status of the target node corresponding to the candidate offloading action is evaluated, and the candidate actions that are expected to cause congestion of the target node are eliminated to obtain the filtered candidate actions; The evaluation module is configured to input the enhanced state vector into a pre-trained deep reinforcement learning network to obtain the value evaluation results of each candidate action after screening, and to determine the unloading decision scheme for the current task based on the value evaluation results.

[0125] It should be noted that each module in this embodiment corresponds one-to-one with each step in embodiment 1, and their specific implementation process is the same, so it will not be repeated here.

[0126] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0127] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for offloading vehicle-to-everything (V2X) tasks based on graph-aware global load balancing, characterized in that: Includes the following steps: Obtain task information and queue status information of each vehicle in the vehicle-to-everything (V2X) system during the current decision-making time slot; Obtain task unloading information for each vehicle within a preset historical time slot, construct a historical data matrix, and construct an adjacency matrix based on the connection relationship between vehicles and MEC servers in the vehicle networking system. The historical data matrix and the adjacency matrix are input into the graph neural network for feature aggregation and feature transformation to obtain global load features that characterize the historical load and the vehicle-MEC graph structure. The problem of solving vehicle task unloading decision is constructed as a Markov decision process, and the task information, queue state information and global load characteristics are used to form the enhanced state vector of the vehicle's current time slot. The load status of the target node corresponding to the candidate offloading action is evaluated by global load characteristics, and the candidate actions that are expected to cause congestion of the target node are eliminated to obtain the filtered candidate actions. The enhanced state vector is input into a pre-trained deep reinforcement learning network to obtain the value evaluation results of each candidate action after screening. Based on the value evaluation results, the unloading decision scheme for the current task is determined. Using task information, queue status information, and global load characteristics as states, task unloading decisions and edge computing resource allocation decisions as action spaces, and minimizing task completion latency and energy consumption as optimization objectives, a Markov decision process for vehicle task unloading decisions is constructed. The constraints on the optimization objective of the Markov decision process for vehicle task unloading decision-making include: The amount of task information must not exceed the maximum allowed amount of task information; Edge computing resources allocated to vehicle tasks must not exceed the total computing power of the MEC server; each task can only choose one of two methods: local processing on the vehicle or offloading to the MEC server for processing; the corresponding processing method decision variable is a binary variable, used to indicate whether the processing method is selected; during the training of the deep reinforcement learning network, based on the current global load characteristics... Aggregation yields system-level load vectors According to the system-level load vector The congestion level and load balancing level of the computational system are weighted with task latency and energy consumption to construct a reward function; during the training of the deep reinforcement learning network, a graph neural network is used to obtain the global load characteristics of the next time step. The aggregation yields the system-level load vector. The system then calculates the future congestion level and incorporates this future congestion level as a penalty term into the target Q-value to construct the graph-aware target Q-value. During the deep reinforcement learning network training process, graph-aware priority experience sampling is performed. Specifically, based on the traditional priority experience replay mechanism based on temporal difference error, global load features extracted by the graph neural network are introduced and aggregated to obtain a system-level load vector. The sampling priority of the experience samples is determined according to the system congestion level at the corresponding time and the change amplitude of the system-level load vector at adjacent time. Training samples are then extracted from the experience replay pool. During the deep reinforcement learning network training process, the reinforcement learning Q-network loss, load prediction error loss, and graph structure consistency constraint loss are jointly constructed to build a joint loss function. This joint loss function is then used to collaboratively optimize the evaluation network parameters and the graph neural network parameters.

2. The vehicular network task offloading method based on graph perception global load as described in claim 1, characterized in that, The process of inputting the historical data matrix and adjacency matrix into a graph neural network for feature aggregation and transformation to obtain global load features characterizing the historical load and vehicle-MEC graph structure includes the following: Perform input alignment processing on the historical data matrix and the adjacency matrix; The aligned historical data matrix and adjacency matrix are then subjected to graph structure feature aggregation processing. A multilayer perceptron is used to perform feature transformation on the aggregated features to obtain the transformed features. The transformed features are processed by an activation function to obtain the activated feature matrix. The obtained activated feature matrix is ​​processed by a dropout layer to obtain the global load features.

3. The vehicular network task offloading method based on graph perception global load as described in claim 1, characterized in that, The load status of the target node corresponding to the candidate offloading action is evaluated by global load characteristics. Candidate actions that are expected to cause congestion of the target node are eliminated, and the filtered candidate actions are obtained. The process includes the following steps: In the current decision time slot, the global load features output by the graph neural network are aggregated through a readout function to obtain a system-level load vector. : Iterate through all candidate unload actions corresponding to the current task, and determine the target node for each candidate action. Based on system-level load vectors The load status of the target node corresponding to each candidate action is estimated to obtain the load estimate. The filtered candidate actions are obtained by comparing the estimate with the set threshold.

4. A vehicle-to-everything (V2X) task offloading system based on graph-aware global load balancing, employing the V2X task offloading method based on graph-aware global load balancing as described in any one of claims 1-3, characterized in that... include: The acquisition module is configured to acquire the task information and queue status information of each vehicle in the vehicle-to-everything (V2X) system under the current decision time slot; The graph construction module is configured to obtain the task unloading information of each vehicle within a preset historical time slot, construct a historical data matrix, and construct an adjacency matrix based on the connection relationship between vehicles and MEC servers in the vehicle networking system. The feature extraction module is configured to input the historical data matrix and the adjacency matrix into the graph neural network for feature aggregation and feature transformation, so as to obtain global load features that characterize the historical load and the vehicle-MEC graph structure. The enhanced state construction module is configured to construct the vehicle task unloading decision problem into a Markov decision process, and to construct the enhanced state vector of the vehicle's current time slot by combining task information, queue state information and global load characteristics. The action filtering module is configured to evaluate the load status of the target node corresponding to the candidate offloading action based on global load characteristics, eliminate candidate actions that are expected to cause congestion of the target node, and obtain the filtered candidate actions. The evaluation module is configured to input the enhanced state vector into a pre-trained deep reinforcement learning network to obtain the value evaluation results of each candidate action after screening, and to determine the unloading decision scheme for the current task based on the value evaluation results.

Images