Internet of vehicles computing power scheduling algorithm and system based on graph neural network and deep reinforcement learning

Abstract

The invention relates to an Internet of Vehicles computing power scheduling method based on a graph neural network and deep reinforcement learning, and the method comprises the steps: S1, collecting the state information of network entities in the Internet of Vehicles and the association information between the entities in real time, and constructing a dynamic time-space attribute graph; s2, inputting the dynamic space-time attribute graph into a pre-trained graph neural network encoder, and outputting a node embedding feature set containing a high-order topological relation and global graph embedding features through multi-layer message passing and feature aggregation; s3, the node embedding feature set and / or the global graph embedding feature are / is used as the state of a deep reinforcement learning agent and input to a strategy network, the computing power scheduling action at the current moment is output, and the computing power scheduling action comprises the steps of assigning an unloading target node for a to-be-processed computing task and distributing corresponding computing and communication resources; s4, dispatching actions are distributed to the corresponding network entities to be executed, environment feedback is collected and used for model updating and next round of dispatching, and the method has the advantages of improving dispatching efficiency, adaptability and system energy efficiency and the like.

Description

A computing power scheduling algorithm and system for vehicle-to-everything (V2X) networks based on graph neural networks and deep reinforcement learning Technical Field

[0001] This invention relates to the fields of vehicle networking, mobile edge computing and artificial intelligence, and in particular to a vehicle networking computing power scheduling algorithm and system based on graph neural networks and deep reinforcement learning. Background Technology

[0002] The rapid development of intelligent connected vehicles has spurred the emergence of massive data-intensive and computationally intensive applications, such as collaborative perception, real-time reconstruction of high-precision maps, and immersive in-vehicle entertainment. These applications place extremely high demands on computing resources and response latency, which are difficult to meet with the limited computing power of a single vehicle. Vehicle-to-everything (V2X) computing power scheduling technology aims to offload in-vehicle tasks to network edge facilities or remote cloud centers, becoming a key to breaking through the bottleneck of single-vehicle computing power.

[0003] Existing computing power scheduling schemes mainly include methods based on optimization theory and methods based on traditional machine learning. Optimization theory-based methods typically seek optimal solutions under simplified and assumed static models, making it difficult to adapt to drastic changes in network topology and dynamic fluctuations in resource status caused by high-speed vehicle movement. Traditional machine learning-based methods rely on large amounts of labeled data, and their model generalization ability is limited, making them unable to cope with unseen new scenarios.

[0004] In recent years, deep reinforcement learning (DRL) has been introduced into this field due to its powerful sequential decision-making and interactive learning capabilities. However, most existing DRL-based solutions model the vehicle-to-everything (V2X) environment as a "flat" state vector, using multilayer perceptrons or convolutional neural networks for processing. This approach has a fundamental flaw: it cannot explicitly characterize and utilize the complex graph-structured relationships between vehicles and between vehicles and infrastructure, formed by geographical location, communication links, and task dependencies. The V2X is essentially a dynamic graph; ignoring its topology will prevent DRL agents from performing effective spatial relationship reasoning, limiting decision-making to a local view and hindering global collaborative optimization.

[0005] This application provides a vehicle network computing power scheduling algorithm and system based on graph neural networks and deep reinforcement learning to solve the above-mentioned problems. Summary of the Invention

[0006] The main objective of this invention is to address the problem that most existing DRL-based solutions model the vehicle-to-everything (V2X) environment as a "flat" state vector, using multilayer perceptrons or convolutional neural networks for processing. This approach has a fundamental flaw: it cannot explicitly characterize and utilize the complex graph-structured relationships between vehicles and between vehicles and infrastructure, formed by geographical location, communication links, and task dependencies. The V2X is essentially a dynamic graph; ignoring its topology will prevent DRL agents from performing effective spatial relationship reasoning, limiting decision-making to a local view and hindering global collaborative optimization.

[0007] This invention provides a vehicle-to-everything (V2X) computing power scheduling algorithm and system based on graph neural networks and deep reinforcement learning. The V2X computing power scheduling algorithm based on graph neural networks and deep reinforcement learning includes:

[0008] S1: Real-time acquisition of state information and inter-entity association information of network entities in the vehicle network to construct a dynamic spatiotemporal attribute graph; S2: Inputting the dynamic spatiotemporal attribute graph into a pre-trained graph neural network encoder, and outputting a set of node embedding features and global graph embedding features containing high-order topological relationships through multi-layer message passing and feature aggregation; S3: Using the set of node embedding features and / or global graph embedding features as the state of the deep reinforcement learning agent, inputting it into the policy network, and outputting the computing power scheduling action at the current moment, wherein the computing power scheduling action includes specifying the unloading target node for the computing task to be processed and allocating corresponding computing and communication resources; S4: Distributing the scheduling action to the corresponding network entities for execution, and collecting environmental feedback for model updates and the next round of scheduling.

[0009] Optionally, in step S1, the node includes intelligent vehicles, roadside units (RSUs), mobile edge computing servers (MECs), and cloud servers; the node characteristics include at least three of the following: computing power, remaining energy, storage space, location, speed, and task queue status; the edge is established based on communication reachability or task dependency; the edge characteristics include at least two of the following: link latency, bandwidth, packet loss rate, and distance.

[0010] Optionally, in step S2, the graph neural network encoder uses a graph attention network or a message passing neural network; the message passing process is as follows: for each node, information from its neighborhood is aggregated to obtain a message, and then its hidden state is updated by combining the node's own features. After K layers of iteration, the final node embedding is obtained; the global graph embedding is obtained by performing graph pooling or a global attention mechanism on all node embeddings.

[0011] Optionally, the deep reinforcement learning agent adopts a single-agent framework with centralized training and centralized execution, using the Deep Deterministic Policy Gradient (DDPG) or Double Delay Deep Deterministic Policy Gradient (TD3) algorithm; or adopts a multi-agent framework with centralized training and distributed execution, using the Multi-Agent Deep Reinforcement Learning (MADDPG) algorithm or the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm.

[0012] Optionally, in step S3, the reward function R_t of the deep reinforcement learning agent is a weighted summation form, including: a negative average task processing delay term, a negative total system energy consumption term, a negative network congestion index term, a positive task completion success rate term, and a penalty term for violating constraints.

[0013] Optionally, it also includes an offline training phase, which includes: in a simulation environment, using historical or simulated vehicle network dynamic data to jointly train or train the graph neural network encoder and the deep reinforcement learning policy network in stages until the cumulative reward converges and the optimal policy network parameters are obtained.

[0014] Secondly, this application also provides a vehicle-to-everything (V2X) computing power scheduling system based on graph neural networks and deep reinforcement learning, used in the V2X computing power scheduling method based on graph neural networks and deep reinforcement learning as described above. The system includes: a dynamic graph construction module, used to collect real-time state information of network entities and association information between entities in the V2X network to construct a dynamic spatiotemporal attribute graph; a graph neural network encoding module, used to input the dynamic spatiotemporal attribute graph into a pre-trained graph neural network encoder, and output a set of node embedding features containing high-order topological relationships and global graph embedding features through multi-layer message passing and feature aggregation; and a deep reinforcement learning decision engine module, used to use the node embedding feature set and... The system uses global graph embedding features as the state of the deep reinforcement learning agent, inputs them into the policy network, and outputs the computing power scheduling action at the current moment. The computing power scheduling action includes specifying the unloading target node for the computing task to be processed and allocating the corresponding computing and communication resources. The policy execution and communication coordination module is used to distribute the scheduling action to the corresponding network entities for execution and collect environmental feedback for model updates and the next round of scheduling. The model training and update module is used to perform joint training or phased training of the graph neural network encoder and the deep reinforcement learning policy network in a simulation environment using historical or simulated vehicle network dynamic data until the cumulative reward converges and the optimal policy network parameters are obtained.

[0015] Optionally, the deep reinforcement learning decision engine module may be deployed on a regional mobile edge computing server or in the cloud for centralized decision-making; or, the graph neural network encoding module and the deep reinforcement learning decision engine module may be distributed and deployed in each intelligent vehicle, with each vehicle making distributed decisions based on a local view.

[0016] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described above.

[0017] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described above.

[0018] This application introduces graph neural networks into the vehicle-to-everything (V2X) scheduling framework, enabling agents to "understand" network topology and relationships between entities, thereby making collaborative decisions based on global structural information and overcoming the problem of missing relationship modeling in traditional DRL schemes. By encoding vehicle mobility and link dynamics as temporal changes in graph structures, the model can intrinsically learn and predict network dynamics, exhibiting strong robustness to topology changes and sudden traffic surges without the need for manual reconfiguration. It effectively avoids local optima and improves overall system performance. It supports both centralized and distributed deployment modes. The graph neural network's generalization ability to graph sizes allows the trained model to be applied to road scenarios of different scales and densities, demonstrating strong scalability. Simulation verification shows that compared to baseline methods based on optimization theory and traditional DRL methods, the proposed solution can reduce average task processing latency by 20%-40% and improve overall system energy efficiency by 15%-30%, while maintaining a higher task completion rate and a more balanced network load. Attached Figure Description

[0019] Figure 1 is a flowchart of the present invention.

[0020] Figure 2 is a schematic diagram of the structure of the present invention. Detailed Implementation

[0021] This invention provides a computing power scheduling algorithm and system for vehicle-to-everything (V2X) networks based on graph neural networks and deep reinforcement learning. The V2X computing power scheduling algorithm includes: S1: Real-time acquisition of state information and inter-entity association information of network entities in the V2X network to construct a dynamic spatiotemporal attribute graph; S2: Inputting the dynamic spatiotemporal attribute graph into a pre-trained graph neural network encoder, and outputting a set of node embedding features and global graph embedding features containing high-order topological relationships through multi-layer message passing and feature aggregation; S3: Using the set of node embedding features and / or global graph embedding features as the state of the deep reinforcement learning agent, inputting it into the policy network, and outputting the computing power scheduling action at the current moment. The computing power scheduling action includes specifying the unloading target node for the computing task to be processed and allocating corresponding computing and communication resources; S4: Distributing the scheduling action to the corresponding network entities for execution, and collecting environmental feedback for model updates and the next round of scheduling. The main purpose of this invention is to solve the problem that in the prior art, most traditional deep reinforcement learning-based schemes model the V2X environment as a "flat" state vector and use multi-layer perceptrons or convolutional neural networks for processing. This approach has a fundamental flaw: it cannot explicitly characterize and utilize the complex graph structure relationships formed between vehicles and between vehicles and infrastructure due to geographical location, communication links, and task dependencies. The Internet of Vehicles (IoV) is essentially a dynamic graph; ignoring its topology will prevent deep reinforcement learning agents from performing effective spatial relationship reasoning, limiting decision-making to local views and hindering global collaborative optimization.

[0022] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] Referring to Figures 1-2 in the specification, this embodiment provides a vehicle-to-everything (V2X) computing power scheduling algorithm and system based on graph neural networks and deep reinforcement learning. The V2X computing power scheduling algorithm includes: S1: Real-time acquisition of state information and inter-entity association information of network entities in the V2X to construct a dynamic spatiotemporal attribute graph; S2: Inputting the dynamic spatiotemporal attribute graph into a pre-trained graph neural network encoder, and outputting a set of node embedding features and global graph embedding features containing high-order topological relationships through multi-layer message passing and feature aggregation; S3: Using the set of node embedding features and / or global graph embedding features as the state of the deep reinforcement learning agent, inputting it into the policy network, and outputting the computing power scheduling action at the current moment. The computing power scheduling action includes specifying the unloading target node for the computing task to be processed and allocating corresponding computing and communication resources; S4: Distributing the scheduling action to the corresponding network entities for execution, and collecting environmental feedback for model updates and the next round of scheduling.

[0024] In step S1, nodes include intelligent vehicles, roadside units (RSUs), mobile edge computing servers (MECs), and cloud servers; node characteristics include at least three of the following: computing power, remaining energy, storage space, location, speed, and task queue status; edges are established based on communication reachability or task dependencies; edge characteristics include at least two of the following: link latency, bandwidth, packet loss rate, and distance.

[0025] Dynamic spatiotemporal attribute graph It represents the state of the vehicle-to-everything (V2X) network at time t. Its construction follows these rules: node set The node types for the construction rules include: intelligent vehicles (Vehicle), roadside units (RSU), mobile edge computing servers (MEC), and cloud servers (Cloud).

[0026] Node admission rules: Vehicle nodes: Only vehicles that are included in the communication range and whose computing power can be scheduled.

[0027] Infrastructure nodes: fixed locations, permanently present on the map.

[0028] Dynamic maintenance: Based on the communication heartbeat mechanism (such as every...) The node set is updated every second, and nodes that leave the communication range or become invalid are removed.

[0029] edge set The conditions for establishing the construction rule edge (based on communication reachability) are as follows: if the communication distance between two nodes is less than or equal to their maximum communication radius, and the current signal-to-noise ratio (SNR) is higher than the threshold, then an undirected edge is established in the graph.

[0030] Task dependencies can also be used as conditions for edge establishment. For example, when the output of a preceding task is the input of a subsequent task, a directed edge is established between the nodes to which the two tasks belong.

[0031] Dynamic edge updates: Predicting the link's duration based on node location, speed, and communication status, and updating it periodically (e.g., every...). Update the edge set every second.

[0032] If the link quality is below the threshold (e.g., packet loss rate greater than 5%), the edge will be temporarily removed.

[0033] Node attribute matrix The construction rules for each node Corresponding to an attribute vector It includes at least three of the following characteristics: Computational: CPU computing power (GHz), remaining memory (GB), and task queue length.

[0034] Communication-related: Transmit power, Received signal strength (RSSI).

[0035] State type: position coordinates (x, y), instantaneous velocity, remaining energy (e.g., for a vehicle).

[0036] Task type: Current task type and estimated completion time.

[0037] The attribute collection frequency is synchronized with the graph update frequency to ensure spatiotemporal consistency.

[0038] Edge attribute matrix Construction rules for each edge Corresponding to an attribute vector This includes: link performance: latency (ms), available bandwidth (MHz), and packet loss rate (%).

[0039] Physical relationships: Euclidean distance, relative velocity.

[0040] Topological relationships: edge types (vehicle-vehicle, vehicle-road, road-cloud).

[0041] Edge attributes are obtained through periodic network probing or estimation using historical statistical models.

[0042] Spatiotemporal dynamics are reflected in the rule-based time sliding window mechanism: (Figure) It not only includes the current state, but can also fuse recent multi-frame graph structures through a sliding time window to form a spatiotemporal graph sequence. Provided for GNN processing.

[0043] Topology change prediction: Based on vehicle trajectory prediction models (such as Kalman filtering), the node positions and edge existence probabilities at the next time step are estimated as prior information for dynamic graph updates.

[0044] Summary of graph construction process and data collection: Status and link information are collected in real time through vehicle-mounted OBU, RSU and central controller.

[0045] Graph initialization: Initialize nodes and edges based on communication relationships and task dependencies.

[0046] Periodic updates: Update node attributes, edge sets, and edge attributes at fixed intervals (e.g., 100ms).

[0047] Anomaly handling: Real-time correction of graph structure triggered by abnormal events such as communication interruption and node exit.

[0048] In step S2, the graph neural network encoder uses a graph attention network or a message passing neural network. The message passing process is as follows: for each node, the information of its neighborhood is aggregated to obtain a message, and then its hidden state is updated by combining the node's own features. After K layers of iteration, the final node embedding is obtained. The global graph embedding is obtained by performing graph pooling operation or global attention mechanism on all node embeddings.

[0049] The deep reinforcement learning agent adopts a single-agent framework with centralized training and centralized execution, using the Deep Deterministic Policy Gradient (DDPG) or the Double Delay Deep Deterministic Policy Gradient (TD3) algorithm; or adopts a multi-agent framework with centralized training and distributed execution, using the Multi-Agent Deep Reinforcement Learning (MADDPG) algorithm or the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm.

[0050] In step S3, the reward function R_t of the deep reinforcement learning agent is a weighted summation, including: a negative average task processing delay term, a negative total system energy consumption term, a negative network congestion index term, a positive task completion success rate term, and a penalty term for violating constraints.

[0051] It also includes an offline training phase, which includes: in a simulation environment, using historical or simulated vehicle network dynamic data to jointly train or train the graph neural network encoder and the deep reinforcement learning policy network in stages until the cumulative reward converges and the optimal policy network parameters are obtained.

[0052] Secondly, this embodiment also provides a vehicle-to-everything (V2X) computing power scheduling system based on graph neural networks and deep reinforcement learning, used for the aforementioned V2X computing power scheduling method based on graph neural networks and deep reinforcement learning. The system includes: a dynamic graph construction module for real-time acquisition of state information of network entities and association information between entities in the V2X network, constructing a dynamic spatiotemporal attribute graph; a graph neural network encoding module for inputting the dynamic spatiotemporal attribute graph into a pre-trained graph neural network encoder, outputting a set of node embedding features containing high-order topological relationships and global graph embedding features through multi-layer message passing and feature aggregation; and a deep reinforcement learning decision engine module for using the node embedding feature set and / or... Alternatively, global graph embedding features can be used as the state of the deep reinforcement learning agent, inputting into the policy network and outputting the computing power scheduling action at the current moment. The computing power scheduling action includes specifying the unloading target node for the computing task to be processed and allocating the corresponding computing and communication resources. The policy execution and communication coordination module is used to distribute the scheduling action to the corresponding network entity for execution and collect environmental feedback for model updates and the next round of scheduling. The model training and update module is used to jointly train or stage-by-stage train the graph neural network encoder and the deep reinforcement learning policy network in a simulation environment using historical or simulated vehicle network dynamic data until the cumulative reward converges and the optimal policy network parameters are obtained.

[0053] This includes deploying a deep reinforcement learning decision engine module on a regional mobile edge computing server or in the cloud for centralized decision-making; or distributing a graph neural network encoding module and a deep reinforcement learning decision engine module across various intelligent vehicles, with each vehicle making distributed decisions based on a local view.

[0054] Thirdly, this embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described above.

[0055] Fourthly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0056] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for scheduling computing power in vehicle-to-everything (V2X) networks based on graph neural networks and deep reinforcement learning, characterized in that, include: S1: Real-time acquisition of state information and relationship information between network entities in the vehicle network to construct a dynamic spatiotemporal attribute graph; S2: Inputting the dynamic spatiotemporal attribute graph into a pre-trained graph neural network encoder, and outputting a set of node embedding features and global graph embedding features containing high-order topological relationships through multi-layer message passing and feature aggregation; S3: Using the set of node embedding features and / or global graph embedding features as the state of the deep reinforcement learning agent, inputting it into the policy network, and outputting the computing power scheduling action at the current moment, wherein the computing power scheduling action includes specifying the unloading target node for the computing task to be processed and allocating corresponding computing and communication resources; S4: Distribute the scheduling action to the corresponding network entity for execution, and collect environmental feedback for model updates and the next round of scheduling.

2. The vehicle network computing power scheduling method based on graph neural networks and deep reinforcement learning according to claim 1, characterized in that, In step S1, the nodes include intelligent vehicles, roadside units (RSUs), mobile edge computing servers (MECs), and cloud servers; the node characteristics include at least three of the following: computing power, remaining energy, storage space, location, speed, and task queue status; the edges are established based on communication reachability or task dependencies; the edge characteristics include at least two of the following: link latency, bandwidth, packet loss rate, and distance.

3. The vehicle network computing power scheduling method based on graph neural networks and deep reinforcement learning according to claim 2, characterized in that, In step S2, the graph neural network encoder uses a graph attention network or a message passing neural network; the message passing process is as follows: for each node, information from its neighborhood is aggregated to obtain a message, and then its hidden state is updated by combining the node's own features. After K layers of iteration, the final node embedding is obtained; the global graph embedding is obtained by performing graph pooling or a global attention mechanism on all node embeddings.

4. The vehicle network computing power scheduling method based on graph neural networks and deep reinforcement learning according to claim 3, characterized in that, The deep reinforcement learning agent adopts a single-agent framework with centralized training and centralized execution, using the Deep Deterministic Policy Gradient (DDPG) or Double Delay Deep Deterministic Policy Gradient (TD3) algorithm; or adopts a multi-agent framework with centralized training and distributed execution, using the Multi-Agent Deep Reinforcement Learning (MADDPG) algorithm or the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm.

5. The vehicle network computing power scheduling method based on graph neural networks and deep reinforcement learning according to claim 4, characterized in that, In step S3, the reward function of the deep reinforcement learning agent is a weighted summation form, including: a negative average task processing delay term, a negative total system energy consumption term, a negative network congestion index term, a positive task completion success rate term, and a penalty term for violating constraints.

6. The vehicle network computing power scheduling method based on graph neural networks and deep reinforcement learning according to claim 5, characterized in that, It also includes an offline training phase, which includes: in a simulation environment, using historical or simulated vehicle network dynamic data to jointly train or train the graph neural network encoder and the deep reinforcement learning policy network in stages until the cumulative reward converges and the optimal policy network parameters are obtained.

7. A vehicle-to-everything (V2X) computing power scheduling system based on graph neural networks and deep reinforcement learning, used in the V2X computing power scheduling method based on graph neural networks and deep reinforcement learning as described in any one of claims 1-6, characterized in that, include: The dynamic graph construction module is used to collect the state information of network entities and the association information between entities in the Internet of Vehicles in real time, and construct a dynamic spatiotemporal attribute graph; the graph neural network encoding module is used to input the dynamic spatiotemporal attribute graph into a pre-trained graph neural network encoder, and output a set of node embedding features containing high-order topological relationships and global graph embedding features through multi-layer message passing and feature aggregation. The deep reinforcement learning decision engine module is used to input the state of the deep reinforcement learning agent into the policy network using the node embedding feature set and / or global graph embedding features as the state, and output the computing power scheduling action at the current time. The computing power scheduling action includes specifying the unloading target node for the computing task to be processed and allocating the corresponding computing and communication resources. The policy execution and communication coordination module is used to distribute the scheduling action to the corresponding network entities for execution and collect environmental feedback for model updates and the next round of scheduling. The model training and update module is used to perform joint training or phased training of the graph neural network encoder and the deep reinforcement learning policy network in a simulation environment using historical or simulated vehicle network dynamic data until the cumulative reward converges and the optimal policy network parameters are obtained.

8. The vehicle-to-everything (V2X) computing power scheduling system based on graph neural networks and deep reinforcement learning according to claim 7, characterized in that, The deep reinforcement learning decision engine module is deployed on a regional mobile edge computing server or in the cloud for centralized decision-making; or, the graph neural network encoding module and the deep reinforcement learning decision engine module are distributed and deployed in each intelligent vehicle, with each vehicle making distributed decisions based on a local view.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-6.

Images