A cloud-edge-end collaborative double-target path planning method and system based on spatiotemporal graph reinforcement learning

Abstract

The application relates to a cloud-edge-end collaborative double-target path planning method and system based on spatiotemporal graph reinforcement learning, and belongs to the technical field of target path planning. The method comprises the following steps: step 1, collecting urban traffic operation data; step 2, pre-processing and fusing the collected urban traffic operation data on a cloud side to form a road network spatiotemporal representation which simultaneously describes the correlation relationship of a traffic state space and the time evolution characteristics; step 3, learning to obtain a road congestion index reflecting the current congestion degree and the future evolution trend of a road section; step 4, based on the generated road congestion index, a double-target path planning model is constructed in combination with the travel preference of a vehicle individual; step 5, the cloud side generates an optimal driving path by using a heuristic path search algorithm according to the constructed double-target path planning model; and step 6, the collaborative optimization of individual travel demand and global traffic efficiency in a dynamic traffic environment is realized. The application realizes the collaborative improvement of individual travel efficiency and global traffic relief.

Description

Technical Field

[0001] This invention relates to a cloud-edge-device collaborative dual-objective path planning method and system based on spatiotemporal graph reinforcement learning, belonging to the field of objective path planning technology. Background Technology

[0002] In modern intelligent transportation systems, path planning technology is widely used in vehicle navigation and traffic flow guidance. By providing vehicles with reasonable driving routes, it optimizes the utilization of traffic resources and improves traffic efficiency.

[0003] In existing technologies, vehicle route planning is typically solved using graph theory models, such as Dijkstra's algorithm and A* algorithm, which are classic shortest path algorithms. These methods abstract the road network into a graph structure and use metrics such as distance or travel time as optimization objectives to calculate the optimal travel path for vehicles. These algorithms have advantages such as clear structure and high computational efficiency, and are therefore widely used in navigation systems and traffic management systems.

[0004] In recent years, with the development of vehicle-to-everything (V2X) technology and edge computing technology, intelligent transportation systems have gradually formed a cloud-edge-device collaborative architecture. Through data interaction between vehicle terminals, roadside units, and cloud platforms, multi-source traffic information can be collected and processed in real time, thus providing a richer data foundation for traffic condition perception and route planning decisions. This collaborative computing model provides new technical conditions for achieving more intelligent and refined traffic management.

[0005] However, existing route planning methods still have certain limitations in complex and highly dynamic urban traffic environments. For example, some methods struggle to fully utilize the dynamic changes in traffic conditions across time and space, resulting in limited adaptability of the planning results to changes in the traffic environment. Furthermore, many methods primarily use a single indicator as the optimization objective, lacking a comprehensive consideration of the relationship between individual travel preferences and overall traffic efficiency. In addition, in large-scale urban road networks, frequent global route recalculations incur high computational costs, making it difficult to meet the needs of real-time route planning. Therefore, it is necessary to propose a route planning method that can deeply model the spatiotemporal relationships of road networks and take into account both global traffic management and user travel preferences, in order to improve the overall operational efficiency of urban transportation systems. Summary of the Invention

[0006] In current urban transportation systems, road traffic conditions exhibit significant spatiotemporal dynamic changes. Existing vehicle path planning methods are insufficient in modeling the spatiotemporal relationships of road networks and struggle to effectively coordinate individual travel preferences with system traffic regulation, thus exacerbating regional traffic congestion. To address these issues, this invention proposes a cloud-edge-device collaborative dual-objective path planning method and system based on spatiotemporal graph reinforcement learning.

[0007] This method relies on a cloud-edge-device collaborative architecture to achieve real-time interaction of multi-source traffic data. It utilizes a spatiotemporal graph reinforcement learning model integrating graph attention networks and near-end policy optimization to perceive the real-time status of the road network and predict short-term evolution trends, generating a road congestion index. Based on this, the invention constructs a dual-objective path search mechanism, coupling the learned road congestion index with individual travel preferences. This satisfies users' personalized path needs while achieving dynamic traffic flow management across the entire network. Furthermore, addressing the computational pressure caused by frequent path recalculation in large-scale dynamic environments, the invention designs an adaptive rerouting strategy. Through a triggered path update mechanism, it reduces the system's computational overhead while ensuring real-time congestion response.

[0008] The technical solution of this invention is as follows: A cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning includes the following steps: Step 1: The edge-side roadside unit collects urban traffic operation data, namely vehicle operation information and road traffic status information within the coverage area, including vehicle location, instantaneous speed, travel time, traffic flow and road space occupancy rate, and uploads the urban traffic operation data to the cloud server. At the same time, it sends real-time traffic status information and planned route instructions to vehicle terminals within the coverage area. Step 2: The cloud preprocesses and integrates the collected urban traffic operation data. Based on the line graph model, a graph neural network is constructed to map the physical road segments in the urban traffic network into graph nodes. The directed connection edges between graph nodes represent the feasible driving relationship between vehicles in different road segments. Based on the historical time window, a dynamic sequence is constructed to form a spatiotemporal representation of the road network that simultaneously describes the spatial correlation and temporal evolution characteristics of traffic status. Step 3: Based on the constructed graph neural network, an attention mechanism is added to perform deep modeling of traffic congestion. The attention mechanism aggregates the spatiotemporal features of adjacent road segments and extracts the spatial dependencies between roads. At the same time, a long short-term memory network is introduced to encode the time series features of traffic status and capture the dynamic changes of traffic flow. The spatiotemporal features extracted by the graph attention network are input into the proximal policy optimization reinforcement learning framework. Through continuous interaction with the simulation environment and policy updates, a road congestion index reflecting the current congestion level and future evolution trend of road segments is learned. Step 4: Based on the road congestion index generated in Step 3, and combined with the travel preferences of individual vehicles, a dual-objective path planning model is constructed. In this model, the individual travel cost is weighted by the user's preference for travel distance and travel time, and the global traffic management objective is achieved by introducing the road congestion index. The dual-objective path planning model simultaneously considers individual travel efficiency and road congestion level, comprehensively generates the cost function of path search, and adjusts the relative importance of individual travel efficiency and road congestion level in path planning through balancing parameters. Step 5: Based on the dual-objective path planning model constructed in Step 4, the cloud uses a heuristic path search algorithm to generate the optimal driving path and returns the planning results to the edge server, which then sends the plan to the vehicle terminal for execution. Step 6: During vehicle travel, the roadside unit continuously collects real-time traffic data and uploads it to the cloud. The cloud monitors changes in the road congestion index of each road segment. When the road congestion index of some road segments exceeds a preset threshold or the traffic condition changes significantly, a dynamic route adjustment mechanism is triggered. This mechanism identifies the set of target vehicles affected by congestion and replans the remaining routes of the affected vehicles based on the updated road congestion index. Through a selective local route update strategy, the route planning results adaptively respond to real-time changes in the traffic environment, achieving coordinated optimization of individual travel needs and overall traffic efficiency in a dynamic traffic environment.

[0009] According to a preferred embodiment of the present invention, the cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning is applied to a vehicle-road-cloud integrated intelligent transportation system, which includes a three-layer architecture: cloud server, roadside unit, and vehicle. The cloud server is responsible for aggregating traffic operation indicators uploaded by various roadside units; based on traffic perception data, the cloud server runs a network-wide vehicle route planning algorithm and outputs the optimal driving route strategy. Roadside units are deployed at road intersections, and their set is represented as follows: Each roadside unit is connected to the cloud server via a wireless link; the roadside unit is responsible for collecting traffic information within its coverage area and uploading it to the cloud periodically; at the same time, the roadside unit is also responsible for sending trajectory planning instructions generated by the cloud to vehicles within its communication range; The vehicles include three types: connected vehicles, buses, and emergency vehicles; each vehicle is assigned a priority level. In addition, each vehicle is associated with a preference weight vector to determine its routing behavior; for each vehicle Preference vector Defined as: (1); in and These respectively reflect the driver's preference for driving distance and driving time; the weights satisfy... ,and ; Each vehicle The features are represented by the state vector as follows: (2); in and They represent the start and end points, respectively. This represents the vehicle speed, which follows a normal distribution. This represents the average speed of vehicles traveling within a road segment. The variance representing the vehicle's speed. Priority This refers to the path preference factor, i.e., the preference vector.

[0010] According to a preferred embodiment of the present invention, step 2 includes the following specific implementation process: Using line graph model , This represents the set of nodes in a line graph model. Each node in Corresponding to a road segment in the original physical road network ; This represents the set of directed edges in a line graph model. Used to characterize adjacent road segments in a physical road network and The topological connections between them and the vehicle steering relationship; if the vehicle can move from the physical road segment Drive directly into the physical road section Then in the corresponding line graph model node and There exists a directed edge between them; the following indices are defined to characterize road traffic state information: 1) Average travel time: for a given road segment Considering time The previous length was time window Based on the time window The distance a vehicle travels and its average speed are defined as follows: (3); in, Indicates within the time window Average driving speed within the area This indicates that the vth vehicle is on the road segment. The driving distance on the road The total number of vehicles on the road segment; This indicates the length of the time window; therefore, the average travel time. The derivation is as follows: (4); in The physical length of the road segment; 2) Free-flow travel time: Defines the free-flow travel time. The time required for a vehicle to traverse a road segment under ideal, uncongested conditions, constrained only by the segment length and speed limit: (5); in, Represents the free-flow velocity; 3) Space occupancy rate: This refers to the percentage of space occupied. Defined as the ratio of the total space occupied by vehicles to the length of the road segment. : (6); in For vehicles The length of the car body, Maintain a safe following distance.

[0011] According to a preferred embodiment of the present invention, step 3, the extraction process of spatiotemporal features includes: The road network is modeled using graph attention networks, with road segments as the basic modeling unit. This modeling process depicts the spatial dependencies between adjacent road segments and the temporal evolution of traffic conditions. The spatiotemporal features of each road segment are extracted and divided into two parts: historical state features and future traffic prediction features. Section Historical state feature sequence Defined as: (7); in, Indicates road segment The average speed of vehicles on the road Indicates the number of vehicles on the road segment. Indicates the occupancy rate of road space. Indicates the average travel time of the road segment; For any road segment Constructing future traffic flow sequences: ;in, Indicates road segment In the future The projected traffic flow within each time step; the dynamic sequence includes historical state characteristic sequences and future traffic flow sequences; Calculate vehicles Arrival Section Expected time The calculation method is as follows: (8); in, This represents the set of road segments included in the vehicle's current planned route. For road section Free circulation time; After obtaining the vehicle arrival time, further statistics are calculated on traffic flow in each future time step; Road sections within a time step Expected flow for: (9); in, For indicator functions, This indicates that the planned path includes road segments. A collection of vehicles; Indicates vehicle Arrival Section The estimated time; Feature encoding is performed on historical state feature sequences and future traffic flow sequences, respectively; including: First, the Long Short-Term Memory (LSTM) network is used to process the historical state feature sequence. Encode to capture historical feature vectors of traffic flow changes over time. : (10); Using a multilayer perceptron (MLP) to analyze future traffic flow sequences Perform nonlinear mapping: (11); in, Represents a non-linear activation function; This represents the future traffic feature vector obtained by nonlinear mapping of the future traffic flow sequence. The weight matrix of the nonlinear mapping layer is represented by... Indicates road segment Future traffic flow sequences This represents the bias term of the nonlinear mapping layer; Subsequently, the two types of feature vectors are concatenated to form a joint temporal feature representation. : (12); After obtaining the joint temporal feature representation, the importance weights of historical traffic information and future traffic information are adaptively learned through an attention mechanism. The importance weights are calculated as follows: (13); in, This represents the learnable weight matrix. This is the attention weight vector; Finally, the historical feature vector and the future traffic feature vector are weighted and fused based on the obtained attention weights to obtain the road segment. Temporal feature enhancement representation : (14); in, The attention weight score represents the historical feature vector. This represents the attention weight score of the future traffic feature vector; Spatial feature modeling using line graph structures; set up Represents a node exist The neighbor set in the road segment, that is, the vehicle passes through the road segment The set of downstream road segments that continue to be entered; based on the neighborhood structure, a graph attention mechanism is used to adaptively aggregate the information of neighboring road segments; nodes Its neighboring nodes Attention coefficient between The calculation is as follows: (15); in, The characteristic transformation matrix, This represents the transpose of the spatial attention weight vector. and Representing road segments Adjacent road sections Temporal feature enhancement representation, This represents the linear rectification activation function with leakage coefficient; Subsequently, the attention coefficients are normalized to obtain the node... Its neighboring nodes attention weights : (16); in, Represents a node and Attention coefficient between nodes Belongs to node exist Neighbor set , Represents an exponential function; Obtaining attention weights Then, the features of neighboring nodes are weighted and aggregated to obtain the spatiotemporal fusion representation of the road segment. : (17); in, Represents a non-linear activation function; Finally, the spatiotemporal feature representations of all road segments are summarized to form the spatiotemporal feature matrix of the entire road network. : (18); The spatiotemporal representation of the road network includes the spatiotemporal fusion representation of road segments in formula (17). And the spatiotemporal characteristic matrix of the entire road network in formula (18) .

[0012] According to a preferred embodiment of the present invention, in step 3, the congestion index modeling based on near-end strategy optimization specifically includes the following steps: In a reinforcement learning environment, the spatiotemporal feature matrix of the entire road network is used. As input to the environment state, the state space The spatiotemporal feature matrix of the entire road network is as follows: (19); in, The spatiotemporal feature matrix represents the overall road network and is used to characterize the overall traffic operation status of the road network at the current moment; Action space Defined as a continuous vector: (20); in, Indicates road segment The congestion index, By assigning different congestion indices to different road sections, potential congested road sections can be actively avoided, thereby guiding vehicles to choose more reasonable driving routes.

[0013] The strategy is evaluated using a delayed reward mechanism, and the reward function is... Defined as: (twenty one); in Indicates road segment Average travel time Indicates the number of road segments in the road network. This indicates the number of road segments that exceed the congestion threshold. This is the penalty constant for severe congestion.

[0014] According to a preferred embodiment of the present invention, learning to obtain a road congestion index reflecting the current level of congestion and its future evolution trend includes: First, obtain the spatiotemporal feature matrix of the entire road network at the current moment. As input for environmental status; Then, the spatiotemporal feature matrix The policy network in the trained PPO algorithm is input into the nonlinear transformation of multiple neurons to map high-dimensional traffic features into continuous vectors corresponding to the number of road segments; the output layer constrains the values ​​in the interval [0,1] to generate the dynamic road congestion index corresponding to each road segment.

[0015] According to a preferred embodiment of the present invention, the construction process of the dual-objective path planning model specifically includes: From the starting point to the node Cumulative path cost Defined as: (twenty two); in, Represents the distance from the starting node to the node. The collection of road segments traversed; and Representing road sections Normalized length and average travel time, parameters and Indicates vehicle The preference weights for travel distance and travel time, parameters This represents a balance coefficient between travel preferences and congestion control. Indicates road section The congestion index; Heuristic functions Defined as: (twenty three); in, Represents a node With vehicles Euclidean distance between destinations The free-flow velocity of the road; Therefore, in the bi-objective path planning model, nodes The total cost function is expressed as: (twenty four); By continuously selecting the node with the smallest total cost function value for expansion, the optimal path from the starting point to the destination, i.e. the optimal driving path, is gradually obtained through the search.

[0016] According to a preferred embodiment of the present invention, step 6 includes the following specific implementation process: An adaptive replanning strategy is employed to achieve dynamic response to traffic congestion by jointly controlling the trigger frequency of path replanning and the vehicles involved in the replanning process; this includes: First, congested road segments in the current road network are identified based on the road congestion index. When the congestion index of a road segment exceeds a preset threshold, it is determined to be a congested road segment. Defined as: (25); in, Indicates road section The congestion index, The congestion determination threshold is the threshold value for determining congestion. The range of values ​​is ; The execution interval of route replanning is dynamically adjusted based on the current number of congested road sections. It is negatively correlated with the degree of road network congestion, defined as: (26); in, This represents the maximum allowable replanning interval under free-flow conditions. The minimum replanning interval is set to avoid frequent path oscillations. This is the sensitivity coefficient; Assumption This represents the set of road segments included in the vehicle's current planned path. Vehicles whose paths intersect with congested road segments are selected as candidate vehicles. : (27); This represents the set of road segments included in the vehicle's current planned path; After obtaining the candidate vehicle set, further filtering is performed based on the spatial distance between the vehicles and the congested road segments; for any congested road segment... Vehicles that meet the following conditions are considered to be affected by traffic congestion. : (28); in, Indicates vehicle Current location information and road segment The spatial distance between them is the shortest Euclidean distance from the vehicle's current location to the road segment; Distance threshold The determination is performed using an adaptive method, and the expression is: (29); in, Based on the replanning of distance, Indicates time Section The congestion index, This represents the maximum value of the congestion index. Ultimately, the collection of all vehicles affected by congestion. Represented as: (30); in, It is affected by the road section The set of vehicles affected by congestion is used to obtain the union of these sets of vehicles to get all vehicles in the road network affected by congestion. Only for sets The vehicle performs a route replanning operation.

[0017] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning.

[0018] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning.

[0019] A cloud-edge-device collaborative dual-objective path planning system based on spatiotemporal graph reinforcement learning includes: The urban traffic operation data acquisition module is configured to: collect urban traffic operation data from the edge-side roadside units, namely vehicle operation information and road traffic status information within the coverage area, including vehicle location, instantaneous speed, travel time, traffic flow and road space occupancy rate, and upload the urban traffic operation data to the cloud server, while simultaneously sending real-time traffic status information and planned route instructions to vehicle terminals within the coverage area; The preprocessing and fusion module is configured to: preprocess and fuse the collected urban traffic operation data in the cloud; construct a graph neural network based on the line graph model to map the physical road segments in the urban traffic network into graph nodes; the directed connection edges between graph nodes represent the feasible driving relationship between vehicles in different road segments; and construct a dynamic sequence based on historical time windows to form a spatiotemporal representation of the road network that simultaneously describes the spatial correlation and temporal evolution characteristics of traffic status. The road congestion index acquisition module is configured to: add an attention mechanism to the constructed graph neural network to perform deep modeling of traffic congestion status; aggregate the spatiotemporal features of adjacent road segments through the attention mechanism to extract the spatial dependencies between roads; simultaneously introduce a long short-term memory network to encode the time series features of traffic status and capture the dynamic changes of traffic flow; input the spatiotemporal features extracted by the graph attention network into the proximal policy optimization reinforcement learning framework; and learn to obtain a road congestion index that reflects the current congestion level and future evolution trend of road segments through continuous interaction with the simulation environment and policy updates. The dual-objective path planning model construction module is configured to: construct a dual-objective path planning model based on the generated road congestion index and combined with the travel preferences of individual vehicles. The individual travel cost is weighted by the user's preference for travel distance and travel time, and the global traffic management objective is achieved by introducing the road congestion index. The dual-objective path planning model simultaneously focuses on individual travel efficiency and road congestion level, comprehensively generates a cost function for path search, and adjusts the relative importance of individual travel efficiency and road congestion level in path planning through balancing parameters. The optimal driving route generation module is configured as follows: the cloud uses a heuristic path search algorithm to generate the optimal driving route based on the constructed dual-objective path planning model, and returns the planning results to the edge server, which then sends the route to the vehicle terminal for execution. The collaborative optimization module is configured as follows: during vehicle travel, the roadside unit continuously collects real-time traffic data and uploads it to the cloud, while the cloud monitors changes in the road congestion index of each road segment; when the road congestion index of some road segments exceeds a preset threshold or the traffic condition changes significantly, a dynamic path adjustment mechanism is triggered to identify the set of target vehicles affected by congestion and replan the remaining paths of the affected vehicles based on the updated road congestion index; through a selective local path update strategy, the path planning results adaptively respond to real-time changes in the traffic environment, achieving collaborative optimization of individual travel needs and global traffic efficiency in a dynamic traffic environment.

[0020] The beneficial effects of this invention are as follows: The cloud-edge-device collaborative dual-objective path planning method and system based on spatiotemporal graph reinforcement learning proposed in this invention has the following significant advantages: 1. This invention overcomes the limitations of traditional methods that only use a single indicator as the optimization objective by constructing a dual-objective path planning model that integrates individual travel preferences and road congestion index. While meeting users' personalized needs for travel distance or time, the system guides vehicles to detour around congested sections through the congestion index, achieving a synergistic improvement in individual travel efficiency and overall traffic management, and avoiding competitive occupation of local road network resources due to convergent path decisions.

[0021] 2. This invention employs Graph Attention Network (GAT) and Long Short-Term Memory Network (LSTM) to extract spatiotemporal features of traffic conditions and introduces Proximal Policy Optimization (PPO) for congestion index modeling. This method can not only perceive the current road network congestion status but also predict its short-term evolution trend, providing a forward-looking decision-making basis for route planning and improving the system's adaptability to dynamic traffic environments.

[0022] 3. This invention relies on a three-layer collaborative architecture of cloud-edge-device to achieve an efficient closed loop from data acquisition and state modeling to path planning and command delivery. The edge-side roadside unit is responsible for real-time data acquisition and command delivery, while the cloud undertakes complex computing tasks, which reduces the computing pressure on a single node and ensures the real-time performance and scalability of the system.

[0023] 4. To address the computational burden caused by frequent replanning in dynamic environments, this invention designs an adaptive rerouting triggering mechanism. The system only updates the local routes for affected vehicles when it detects significant changes in the state of critical road segments, avoiding global recalculation, significantly reducing computational overhead, while maintaining rapid response to traffic fluctuations, thus improving the system's practicality and stability in large-scale road networks. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the workflow of the cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning of the present invention; Figure 2 This is the road network structure diagram used in the simulation; Figure 3 It is the convergence curve of the model training; Figure 4 This is a comparison chart of the method of the present invention with other methods in terms of average travel time under different traffic scales; Figure 5 This is a comparison chart of the method of the present invention with other methods in terms of the number of congested road sections under different traffic scales. Detailed Implementation

[0025] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0026] Terminology Explanation: 1. Graph Attention Network (GAT), a graph neural network incorporating an attention mechanism, is a deep learning model specifically designed for processing graph-structured data. It introduces an attention mechanism into traditional graph neural networks, aiming to handle interactions between nodes by dynamically assigning weights. This mechanism allows the model to automatically calculate the importance scores of different neighbors to the current node based on the correlation between nodes, rather than simply averaging or summing information when aggregating neighbor node information.

[0027] 2. Long Short-Term Memory (LSTM) network is a special type of recurrent neural network specifically designed to solve long-distance dependency problems. It uses a unique gating structure to control the flow of information, which can effectively alleviate the gradient vanishing or gradient explosion phenomena that occur when traditional recurrent neural networks process long sequence data.

[0028] 3. The Multilayer Perceptron (MLP) is a classic feedforward artificial neural network architecture, consisting of an input layer, one or more hidden layers, and an output layer. Neurons between layers are fully connected, meaning every neuron in each layer is connected to all neurons in the previous layer. By introducing nonlinear activation functions in the hidden layers, the MLP can learn and simulate any complex nonlinear mapping relationship.

[0029] 4. The policy network (Actor) in the PPO algorithm is a neural network used to output the probability distribution of actions based on the environmental state.

[0030] Example 1 A cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning, such as Figure 1 As shown, it includes the following steps: Step 1: The edge-side roadside unit collects urban traffic operation data, namely vehicle operation information and road traffic status information within the coverage area, including vehicle location, instantaneous speed, travel time, traffic flow and road space occupancy rate, and uploads the urban traffic operation data to the cloud server. At the same time, it sends real-time traffic status information and planned route instructions to vehicle terminals within the coverage area. Step 2: The cloud preprocesses and fuses the collected urban traffic operation data, and constructs a graph neural network based on the line graph model (adding an attention mechanism to the graph neural network to make it a graph attention network) to map the physical road segments in the urban traffic network into graph nodes. The directed connection edges between graph nodes represent the feasible driving relationship between vehicles in different road segments, and a dynamic sequence is constructed based on historical time windows, thus forming a spatiotemporal representation of the road network that simultaneously describes the spatial correlation and temporal evolution characteristics of traffic states. Step 3: Based on the constructed graph neural network, an attention mechanism is added to perform deep modeling of traffic congestion. The attention mechanism aggregates the spatiotemporal features of adjacent road segments and extracts the spatial dependencies between roads. At the same time, a long short-term memory network is introduced to encode the time series features of traffic status and capture the dynamic changes of traffic flow. The spatiotemporal features extracted by the graph attention network are input into the proximal policy optimization (PPO) reinforcement learning framework. Through continuous interaction with the simulation environment and policy updates, a road congestion index reflecting the current congestion level and future evolution trend of road segments is learned. Step 4: Based on the road congestion index generated in Step 3, and combined with the travel preferences of individual vehicles, a dual-objective path planning model is constructed. In this model, the individual travel cost is weighted by the user's preference for travel distance and travel time, and the global traffic management objective is achieved by introducing the road congestion index. The dual-objective path planning model simultaneously considers individual travel efficiency and road congestion level, comprehensively generates the cost function of path search, and adjusts the relative importance of individual travel efficiency and road congestion level in path planning through balancing parameters. Step 5: Based on the dual-objective path planning model constructed in Step 4, the cloud uses a heuristic path search algorithm to generate the optimal driving path and returns the planning results to the edge server, which then sends the plan to the vehicle terminal for execution. Step 6: During vehicle travel, the roadside unit continuously collects real-time traffic data and uploads it to the cloud. The cloud monitors changes in the road congestion index of each road segment. When the road congestion index of some road segments exceeds a preset threshold or the traffic condition changes significantly, a dynamic route adjustment mechanism is triggered. This mechanism identifies the set of target vehicles affected by congestion and replans the remaining routes of the affected vehicles based on the updated road congestion index. Through a selective local route update strategy, the route planning results adaptively respond to real-time changes in the traffic environment while avoiding the system overhead caused by large-scale global recalculation, thus achieving coordinated optimization of individual travel needs and global traffic efficiency in a dynamic traffic environment.

[0031] Example 2 The difference between the cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning described in Example 1 and the one described in Example 1 is as follows: The cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning is applied to the vehicle-road-cloud integrated intelligent transportation system, which includes a three-layer architecture: cloud server, roadside unit and vehicle. As the core control node of the system, the cloud server is responsible for aggregating traffic operation indicators uploaded by various roadside units, including parameters such as traffic flow, average vehicle travel time, and road space occupancy. Based on traffic perception data, the cloud server runs a network-wide vehicle path planning algorithm and outputs the optimal driving path strategy.

[0032] Roadside units are deployed at road intersections, and their set is represented as follows: Each roadside unit (RSU) is connected to the cloud server via a wireless link. The RSU is responsible for collecting traffic information within its coverage area and uploading it to the cloud periodically. At the same time, the RSU is also responsible for sending trajectory planning instructions generated by the cloud to vehicles within its communication range. The vehicles include three types: connected vehicles, buses, and emergency vehicles; each vehicle is assigned a priority level. The smaller the value, the higher the priority of the route planning. Emergency vehicles have the highest priority, followed by buses, and connected vehicles have the lowest priority. Furthermore, each vehicle is associated with a preference weight vector to determine its routing behavior; for each vehicle... Preference vector Defined as:

[0033] (1); in and These respectively reflect the driver's preference for driving distance and driving time; the weights satisfy... ,and ; Each vehicle The features are represented by the state vector as follows: (2); in and They represent the start and end points, respectively. This represents the vehicle speed, which follows a normal distribution. This represents the average speed of vehicles traveling within a road segment. The variance representing the vehicle's speed. Priority This refers to the path preference factor, i.e., the preference vector.

[0034] The specific implementation process of step 2 includes: To depict the traffic flow relationships between road segments, this invention employs a line graph model. , This represents the set of nodes in a line graph model. Each node in Corresponding to a road segment in the original physical road network (referring to actual roads). ; This represents the set of directed edges in a line graph model. Used to characterize adjacent road segments in a physical road network and The topological connections between them and the vehicle steering relationship; if the vehicle can move from the physical road segment Drive directly into the physical road section Then in the corresponding line graph model node and There exists a directed edge between them; the following indices are defined to characterize road traffic state information: 1) Average travel time: for a given road segment Considering time The previous length was time window Based on the time window The distance a vehicle travels and its average speed are defined as follows: (3); in, Indicates within the time window Average driving speed within the area This indicates that the vth vehicle is on the road segment. The driving distance on the road The total number of vehicles on the road segment; This indicates the length of the time window, defining the historical time range used to calculate the average driving time. Therefore, the average driving time... The derivation is as follows:

[0035] (4); in The physical length of the road segment; 2) Free-flow travel time: Defines the free-flow travel time. The time required for a vehicle to traverse a road segment under ideal, uncongested conditions, constrained only by the segment length and speed limit: (5); in, Represents the free-flow velocity; corresponding to the design speed limit of the road.

[0036] 3) Space occupancy rate: To indicate the degree of traffic congestion, space occupancy rate is used. Defined as the ratio of the total space occupied by vehicles to the length of the road segment. : (6); in For vehicles The length of the car body, Maintain a safe following distance.

[0037] Step 3, the process of extracting spatiotemporal features, includes: In this invention, a graph attention network is used to model the road network, with road segments as the basic modeling unit. The spatial dependency between adjacent road segments and the temporal evolution characteristics of traffic state are characterized. The spatiotemporal features of each road segment are extracted and divided into two parts: historical state features and future traffic prediction features. Section Historical state feature sequence Defined as: (7); in, Indicates road segment The average speed of vehicles on the road Indicates the number of vehicles on the road segment. Indicates the occupancy rate of road space. Indicates the average travel time of the road segment; Furthermore, to improve the foresight of traffic state modeling, this invention incorporates short-term future traffic information into the traffic state features to describe the potential changes in traffic flow across road segments over a future period. Specifically, for any road segment... Constructing future traffic flow sequences: ;in, Indicates road segment In the future The projected traffic flow is calculated within a given time step; this flow is obtained statistically from the projected arrival times of vehicles. The dynamic sequence includes historical state characteristic sequences and future traffic flow sequences.

[0038] Calculate vehicles Arrival Section Expected time The calculation method is as follows: (8); in, This represents the set of road segments included in the vehicle's current planned route. For road section Free circulation time; After obtaining the vehicle arrival time, further statistics are calculated on traffic flow in each future time step; Road sections within a time step Expected flow for: (9); in, For indicator functions, This indicates that the planned path includes road segments. A collection of vehicles; Indicates vehicle Arrival Section The estimated time; To further model the temporal correlation of traffic states, this invention performs feature encoding on historical state feature sequences and future traffic flow sequences, including: First, the Long Short-Term Memory (LSTM) network is used to process the historical state feature sequence. Encode to capture historical feature vectors of traffic flow changes over time. : (10); At the same time, in order to make future traffic flow sequences consistent with historical feature vectors Within the same feature space, this invention employs a multilayer perceptron (MLP) to analyze future traffic flow sequences. Perform nonlinear mapping: (11); in, Represents a non-linear activation function; This represents the future traffic feature vector obtained by nonlinear mapping of the future traffic flow sequence. The weight matrix of the nonlinear mapping layer is represented by... Indicates road segment Future traffic flow sequences This represents the bias term of the nonlinear mapping layer; Subsequently, the two types of feature vectors are concatenated to form a joint temporal feature representation. : (12); After obtaining the joint temporal feature representation, the importance weights of historical traffic information and future traffic information are adaptively learned through an attention mechanism. The importance weights are calculated as follows: (13); in, This represents the learnable weight matrix. This is the attention weight vector, used to characterize the contribution of historical feature vectors and future traffic feature vectors to the current traffic state modeling.

[0039] Finally, the historical feature vector and the future traffic feature vector are weighted and fused based on the obtained attention weights to obtain the road segment. Temporal feature enhancement representation : (14); in, The attention weight score represents the historical feature vector. This represents the attention weight score of the future traffic feature vector; After obtaining the enhanced temporal feature representation of each road segment, in order to further characterize the spatial interaction relationship between different road segments in the road network, this invention further performs spatial feature modeling through a line graph structure. set up Represents a node exist The neighbor set in the road segment, that is, the vehicle passes through the road segment The set of downstream road segments that continue to be entered; based on the neighborhood structure, this invention uses a graph attention mechanism to adaptively aggregate neighbor road segment information; nodes Its neighboring nodes Attention coefficient between The calculation is as follows: (15); in, The characteristic transformation matrix, This represents the transpose of the spatial attention weight vector. and Representing road segments Adjacent road sections Temporal feature enhancement representation, This represents a linear rectified activation function with a leakage coefficient; it is used to retain a certain non-zero output when the input is negative.

[0040] Subsequently, the attention coefficients are normalized to obtain the node... Its neighboring nodes attention weights : (16); in, Represents a node and Attention coefficient between nodes Belongs to node exist Neighbor set , Represents an exponential function; Obtaining attention weights Then, the features of neighboring nodes are weighted and aggregated to obtain the spatiotemporal fusion representation of the road segment. : (17); in, Represents a non-linear activation function; Finally, the spatiotemporal feature representations of all road segments are summarized to form the spatiotemporal feature matrix of the entire road network. : (18); The spatiotemporal representation of the road network includes the spatiotemporal fusion representation of road segments in formula (17). And the spatiotemporal characteristic matrix of the entire road network in formula (18) .

[0041] In step 3, the congestion index modeling based on near-end strategy optimization is specifically implemented as follows: After obtaining the spatiotemporal traffic state representation of the road network, this invention further constructs a congestion perception decision-making mechanism based on Proximity Policy Optimization (PPO), which generates a dynamic road congestion index based on the current traffic state, thereby providing an adaptive basis for adjusting road weights for subsequent route planning. This decision-making mechanism establishes a mapping relationship between traffic state and road congestion level through reinforcement learning, enabling the system to dynamically adjust the congestion weights of each road segment according to the real-time traffic environment.

[0042] In a reinforcement learning environment, the spatiotemporal feature matrix of the entire road network is used. As input to the environment state, the state space The spatiotemporal feature matrix of the entire road network is as follows: (19); in, The spatiotemporal feature matrix represents the overall road network and is used to characterize the overall traffic operation status of the road network at the current moment; Action space Defined as a continuous vector: (20); in, Indicates road segment The congestion index, By assigning different congestion indices to different road sections, potential congested road sections can be actively avoided, thereby guiding vehicles to choose more reasonable driving routes.

[0043] To characterize the long-term impact of route decisions on the transportation system, this invention employs a delayed reward mechanism to evaluate the strategy, with the reward function... Defined as: (twenty one); in Indicates road segment Average travel time Indicates the number of road segments in the road network. This indicates the number of road segments that exceed the congestion threshold. This represents the penalty constant for severe congestion. By designing this reward function, additional penalties can be imposed on severely congested road segments while optimizing average travel time, thereby guiding the path planning algorithm to learn a more balanced traffic allocation scheme. Through iterative training of the policy network, a dynamic road congestion index for each road segment can be output based on the current traffic state, providing a decision-making basis for subsequent path planning.

[0044] Learn to obtain a road congestion index that reflects the current level of congestion and its future trends, including: First, obtain the spatiotemporal feature matrix of the entire road network at the current moment. As input for environmental status; Then, the spatiotemporal feature matrix The policy network in the PPO algorithm after input training acts as the decision-making brain. Through the nonlinear transformation of multiple layers of neurons, it maps high-dimensional traffic features into continuous vectors corresponding to the number of road segments. In order to ensure that the output results are logical, the output layer uses a specific activation function to constrain the values ​​in the range of [0,1], thereby generating a dynamic road congestion index corresponding to each road segment.

[0045] The generated congestion index is fed back to the path planning layer in real time. As a dynamic cost factor in the subsequent A* heuristic search algorithm, it directly rewrites the travel cost of road segments, thereby guiding vehicles to low-risk road segments. At this point, the traffic environment (and the average travel time of the road network) will be reflected in the new traffic distribution based on this decision.

[0046] The evaluation network of the PPO algorithm scores the current output congestion index scheme. The system calculates the advantage value of the current decision based on the reward function defined by formula (21).

[0047] Through extensive iterative training, the policy network gradually learns the mapping relationship between the spatiotemporal evolution of the road network and the overall system efficiency. Ultimately, when the model is deployed, only the current spatiotemporal feature matrix needs to be input, and the policy network can instantly output a road congestion index reflecting the current level of congestion and future trends.

[0048] The construction process of the dual-objective path planning model specifically includes: To balance individual travel preferences with the overall road network congestion control needs, this invention adopts a method based on The algorithm employs a bi-objective path planning strategy. This is achieved through traditional... The algorithm incorporates a road congestion index into its path cost function and combines it with the personalized travel preferences of vehicles to construct a preference-weighted path cost model, thereby achieving global control of traffic flow while meeting individual travel needs.

[0049] Specifically, from the starting point to the node Cumulative path cost Defined as: (twenty two); in, Represents the distance from the starting node to the node. The collection of road segments traversed; and Representing road sections Normalized length and average travel time, parameters and Indicates vehicle The preference weights for driving distance and travel time are used to characterize the differences in travel preferences among different drivers. Parameters This represents a balance coefficient between travel preferences and congestion control. Indicates road section The congestion index; this coefficient is dynamically generated by the reinforcement learning strategy.

[0050] By introducing a congestion-aware term into the path cost function The proposed path planning model can suppress path selection behaviors that may exacerbate traffic concentration during the path search process. Because... It also reflects the current traffic operation status of the road segment and the short-term congestion evolution trend. The route planning process is guided at the global level to avoid road segments with potential overload risks, thereby achieving adaptive dispersion of traffic flow in the road network.

[0051] exist In the algorithm, the heuristic function is used to estimate the remaining cost from the current node to the target node. The heuristic function... Defined as:

[0052] (twenty three); in, Represents a node With vehicles Euclidean distance between destinations The free-flow velocity of the road; this heuristic function provides a reasonable estimate of the remaining travel time for the search process.

[0053] Therefore, in the bi-objective path planning model, nodes The total cost function is expressed as: (twenty four); By continuously selecting the node with the smallest total cost function value for expansion, the optimal path from the starting point to the destination, i.e. the optimal driving path, is gradually obtained through the search.

[0054] The specific implementation process of step 6 includes: To effectively alleviate traffic congestion while reducing the computational overhead of frequent global path replanning, this invention employs an adaptive replanning strategy. This strategy jointly regulates the trigger frequency of path replanning and the vehicles involved in the replanning process, achieving a dynamic response to traffic congestion while avoiding unnecessary computational costs. This includes:

[0055] First, congested road segments in the current road network are identified based on the road congestion index. When the congestion index of a road segment exceeds a preset threshold, it is determined to be a congested road segment. Defined as: (25); in, Indicates road section The congestion index, The congestion determination threshold is the threshold value for determining congestion. The range of values ​​is In this embodiment, it is preferred that... When the dynamic congestion index of a road segment When the threshold is exceeded, the road segment is identified as a potentially congested segment, and an additional penalty is imposed on its total cost function during route planning.

[0056] Based on this, the execution interval of route replanning is dynamically adjusted according to the current number of congested road sections. It is negatively correlated with the degree of road network congestion, defined as: (26); in, This represents the maximum allowable replanning interval under free-flow conditions. The minimum replanning interval is set to avoid frequent path oscillations. This is a sensitivity coefficient used to adjust the system's response to changes in congestion. When the number of congested road segments increases, the system automatically shortens the replanning interval, thereby increasing the path update frequency; when the road network is in a low-congestion state, the replanning cycle is extended to reduce unnecessary computational overhead.

[0057] To further reduce the system's computational burden, route replanning is not performed simultaneously on all vehicles, but rather selectively on vehicles that may be affected by congestion. Assume... This represents the set of road segments included in the vehicle's current planned path. Vehicles whose paths intersect with congested road segments are selected as candidate vehicles. :

[0058] (27); This represents the set of road segments included in the vehicle's current planned path; After obtaining the candidate vehicle set, further filtering is performed based on the spatial distance between the vehicles and the congested road segments; for any congested road segment... Vehicles that meet the following conditions are considered to be affected by traffic congestion. : (28); in, Indicates vehicle Current location information and road segment The spatial distance between them is the shortest Euclidean distance from the vehicle's current location to the road segment; Distance threshold The determination is performed using an adaptive method, and the expression is: (29); in, Based on the replanning of distance, Indicates time Section The congestion index, This represents the maximum value of the congestion index; as congestion worsens, the distance threshold increases accordingly, thereby expanding the range of vehicles requiring route adjustments.

[0059] Ultimately, the collection of all vehicles affected by congestion. Represented as: (30); in, It is affected by the road section The set of vehicles affected by congestion is used to obtain the union of these sets of vehicles to get all vehicles in the road network affected by congestion. Only for sets The vehicles in the process perform route replanning operations. Specifically, this includes:

[0060] (1) The cloud acquires the set of affected vehicles in real time and extracts the overall spatiotemporal feature matrix of the road network simultaneously. .

[0061] (2) Input the overall spatiotemporal feature matrix of the road network into the pre-trained PPO policy network, and re-output the road congestion index through nonlinear mapping.

[0062] (3) Inject the generated road congestion index into the bi-objective path planning model, and perform heuristic optimal path search with the current position of the vehicle as the starting node.

[0063] (4) Extract the route to avoid high-risk congestion areas and send the instruction to the target vehicle terminal for execution.

[0064] By jointly controlling the replanning trigger frequency and the scope of replanning objects, the overall computational cost of the path planning algorithm is significantly reduced while ensuring the traffic diversion effect, thereby achieving efficient dynamic path planning in a large-scale road network environment.

[0065] Compared with existing traffic guidance methods, this invention introduces spatiotemporal traffic feature modeling, reinforcement learning congestion index generation, and a dual-objective path optimization mechanism, enabling the system to dynamically adjust road weights according to real-time traffic conditions and improve the overall traffic efficiency of the road network while taking into account the individual travel needs of vehicles.

[0066] Specifically, firstly, this invention constructs a spatiotemporal traffic state representation of the road network and models the spatial relationships between roads using graph attention networks. Simultaneously, it combines time-series traffic features to characterize the dynamic evolution of traffic states, thereby achieving accurate representation of road traffic conditions. Secondly, based on the obtained spatiotemporal traffic features of the road network, this invention introduces a reinforcement learning decision-making mechanism based on proximal policy optimization (PPO). By learning the mapping relationship between traffic states and road congestion levels, a dynamic road congestion index is generated, enabling adaptive adjustment of road traffic weights. Furthermore, this invention designs a dual-objective path planning algorithm that integrates individual travel efficiency and global congestion control objectives. Combined with an adaptive path replanning mechanism, it adjusts the paths only for vehicles affected by congestion, thereby reducing system computational overhead while ensuring traffic diversion effects and achieving efficient path guidance in large-scale road network environments.

[0067] To verify the effectiveness of the proposed method in urban road network traffic control, a simulation experimental environment was constructed in the traffic simulation platform SUMO, and real-time data interaction between the path planning algorithm and the traffic simulation system was realized through the TraCI interface, thereby completing the collaborative simulation of vehicle path decision-making and traffic state updates.

[0068] The simulation scenario is based on a portion of the urban road network in Fengxian District, Shanghai. The specific structure is determined by… Figure 2 As shown. To evaluate the performance of the algorithm under different traffic demand conditions, the experiment set four traffic demand scales of 900, 1200, 1500 and 1800 vehicles to simulate the road network operation under different traffic load levels.

[0069] To verify the performance advantages of the proposed method, the following three path planning methods were selected as comparison algorithms in the experiment: 1) Static shortest time method: The vehicle calculates the shortest travel time path based on Dijkstra's algorithm when it departs, and keeps the path unchanged throughout the entire journey, without considering the impact of changes in road network traffic conditions on path selection.

[0070] 2) Heuristic shortest path method: Vehicle route planning adopts... The algorithm completes, using road length as the cost function for path search. During vehicle travel, road weights remain fixed and are not dynamically updated based on real-time traffic conditions.

[0071] 3) Shortest-time guidance method: The system identifies potentially congested road sections through a congestion detection mechanism and triggers a route guidance strategy. Vehicles adopt a time-cost-based approach. The algorithm regenerates alternative routes, periodically replans vehicle routes at preset time intervals, and recalculates vehicle travel routes based on updated road weights.

[0072] In terms of performance evaluation, the experiment comprehensively analyzed the method of this invention from three aspects: algorithm training convergence, road traffic efficiency, and road network congestion. Road traffic efficiency was measured by statistically analyzing the average vehicle travel time, while road network congestion was evaluated by statistically analyzing the number of congested roads in the network.

[0073] Figure 3 The convergence of the method of this invention during the training process is demonstrated. It can be observed that as the training progresses, the algorithm's reward value gradually stabilizes, indicating that the reinforcement learning model can progressively learn effective path control strategies, achieving reasonable adjustment of road traffic conditions. This result shows that the method of this invention can characterize the spatiotemporal traffic state of roads, achieving accurate characterization and stable estimation of the road network traffic operation state, thereby providing reliable road condition information support for subsequent path planning.

[0074] Figure 4 This paper compares the average travel time of the method of this invention with that of a benchmark method under different traffic volume scales. As traffic demand increases, the overall operational pressure on the road network gradually increases, and the average travel time of all methods shows an upward trend. However, compared with the comparative methods, the method of this invention consistently maintains a lower average travel time. This indicates that the present invention can effectively improve road traffic efficiency and mitigate the impact of congestion on the overall traffic system under continuously increasing traffic demand.

[0075] Figure 5 This paper compares the number of congested road segments in the present invention with those of comparative methods under different traffic demand scales. It shows that as the number of vehicles increases, the number of congested road segments in the road network generally increases, while the present invention maintains a low number of congested road segments under all traffic scales. This indicates that the present invention can more rationally guide traffic flow at the road network level, effectively disperse traffic flow, avoid excessive congestion on some key road segments, and thus improve the overall road network operating efficiency.

[0076] Simulation results show that the proposed method achieves good performance under different traffic demand scenarios. With the increase in the number of vehicles, the average travel time of traditional path planning methods increases significantly, while the proposed method can effectively disperse traffic flow through congestion perception and path optimization mechanisms, thereby reducing the average travel time of vehicles and alleviating road congestion.

[0077] Furthermore, the changes in the number of congested roads demonstrate that the proposed method can, to some extent, prevent excessive traffic flow concentration on certain key roads, thereby achieving a more balanced traffic flow distribution. Experimental results validate the effectiveness of this method in improving traffic efficiency and alleviating congestion.

[0078] Example 3 A computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning as described in Embodiment 1 or 2.

[0079] Example 4 A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning as described in Embodiment 1 or 2.

[0080] Example 5 A cloud-edge-device collaborative dual-objective path planning system based on spatiotemporal graph reinforcement learning includes: The urban traffic operation data acquisition module is configured to: collect urban traffic operation data from the edge-side roadside units, namely vehicle operation information and road traffic status information within the coverage area, including vehicle location, instantaneous speed, travel time, traffic flow and road space occupancy rate, and upload the urban traffic operation data to the cloud server, while simultaneously sending real-time traffic status information and planned route instructions to vehicle terminals within the coverage area; The preprocessing and fusion module is configured to: preprocess and fuse the collected urban traffic operation data in the cloud; construct a graph neural network based on the line graph model (add an attention mechanism to the graph neural network to make it a graph attention network) to map the physical road segments in the urban traffic network into graph nodes; the directed connection edges between graph nodes represent the feasible driving relationship between vehicles in different road segments; and construct a dynamic sequence based on historical time windows to form a spatiotemporal representation of the road network that simultaneously describes the spatial correlation of traffic state and the temporal evolution characteristics. The road congestion index acquisition module is configured to: add an attention mechanism to the constructed graph neural network to perform deep modeling of traffic congestion status; aggregate the spatiotemporal features of adjacent road segments through the attention mechanism to extract the spatial dependencies between roads; simultaneously introduce a long short-term memory network to encode the time-series features of traffic status and capture the dynamic changes in traffic flow; input the spatiotemporal features extracted by the graph attention network into the proximal policy optimization (PPO) reinforcement learning framework, and learn to obtain a road congestion index that reflects the current congestion level and future evolution trend of road segments through continuous interaction with the simulation environment and policy updates; The dual-objective path planning model construction module is configured to: construct a dual-objective path planning model based on the generated road congestion index and combined with the travel preferences of individual vehicles. The individual travel cost is weighted by the user's preference for travel distance and travel time, and the global traffic management objective is achieved by introducing the road congestion index. The dual-objective path planning model simultaneously focuses on individual travel efficiency and road congestion level, comprehensively generates a cost function for path search, and adjusts the relative importance of individual travel efficiency and road congestion level in path planning through balancing parameters. The optimal driving route generation module is configured as follows: the cloud uses a heuristic path search algorithm to generate the optimal driving route based on the constructed dual-objective path planning model, and returns the planning results to the edge server, which then sends the route to the vehicle terminal for execution. The collaborative optimization module is configured as follows: during vehicle travel, roadside units continuously collect real-time traffic data and upload it to the cloud. The cloud monitors changes in the road congestion index of each road segment. When the road congestion index of some road segments exceeds a preset threshold or the traffic condition changes significantly, a dynamic path adjustment mechanism is triggered. This mechanism identifies the set of target vehicles affected by congestion and replans the remaining paths of the affected vehicles based on the updated road congestion index. Through a selective local path update strategy, the path planning results adaptively respond to real-time changes in the traffic environment while avoiding the system overhead caused by large-scale global recalculation, thus achieving collaborative optimization of individual travel needs and global traffic efficiency in a dynamic traffic environment.

Claims

1. A cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning, characterized in that, Includes the following steps: Step 1: The edge-side roadside unit collects urban traffic operation data, namely vehicle operation information and road traffic status information within the coverage area, including vehicle location, instantaneous speed, travel time, traffic flow and road space occupancy rate, and uploads the urban traffic operation data to the cloud server. At the same time, it sends real-time traffic status information and planned route instructions to vehicle terminals within the coverage area. Step 2: The cloud preprocesses and integrates the collected urban traffic operation data. Based on the line graph model, a graph neural network is constructed to map the physical road segments in the urban traffic network into graph nodes. The directed connection edges between graph nodes represent the feasible driving relationship between vehicles in different road segments. Based on the historical time window, a dynamic sequence is constructed to form a spatiotemporal representation of the road network that simultaneously describes the spatial correlation and temporal evolution characteristics of traffic status. Step 3: Based on the constructed graph neural network, an attention mechanism is added to perform deep modeling of traffic congestion. The attention mechanism aggregates the spatiotemporal features of adjacent road segments and extracts the spatial dependencies between roads. At the same time, a long short-term memory network is introduced to encode the time series features of traffic status and capture the dynamic changes of traffic flow. The spatiotemporal features extracted by the graph attention network are input into the proximal policy optimization reinforcement learning framework. Through continuous interaction with the simulation environment and policy updates, a road congestion index reflecting the current congestion level and future evolution trend of road segments is learned. Step 4: Based on the road congestion index generated in Step 3, and combined with the travel preferences of individual vehicles, a dual-objective path planning model is constructed. In this model, the individual travel cost is weighted by the user's preference for travel distance and travel time, and the global traffic management objective is achieved by introducing the road congestion index. The dual-objective path planning model simultaneously considers individual travel efficiency and road congestion level, comprehensively generates the cost function of path search, and adjusts the relative importance of individual travel efficiency and road congestion level in path planning through balancing parameters. Step 5: Based on the dual-objective path planning model constructed in Step 4, the cloud uses a heuristic path search algorithm to generate the optimal driving path and returns the planning results to the edge server, which then sends the plan to the vehicle terminal for execution. Step 6: During vehicle travel, the roadside unit continuously collects real-time traffic data and uploads it to the cloud. The cloud monitors changes in the road congestion index of each road segment. When the road congestion index of some road segments exceeds a preset threshold or the traffic condition changes significantly, a dynamic route adjustment mechanism is triggered. This mechanism identifies the set of target vehicles affected by congestion and replans the remaining routes of the affected vehicles based on the updated road congestion index. Through a selective local route update strategy, the route planning results adaptively respond to real-time changes in the traffic environment, achieving coordinated optimization of individual travel needs and overall traffic efficiency in a dynamic traffic environment.

2. The cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning according to claim 1, characterized in that, The cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning is applied to the vehicle-road-cloud integrated intelligent transportation system, which includes a three-layer architecture: cloud server, roadside unit and vehicle. The cloud server is responsible for aggregating traffic operation indicators uploaded by various roadside units; based on traffic perception data, the cloud server runs a network-wide vehicle route planning algorithm and outputs the optimal driving route strategy. Roadside units are deployed at road intersections, and their set is represented as follows: Each roadside unit is connected to the cloud server via a wireless link; the roadside unit is responsible for collecting traffic information within its coverage area and uploading it to the cloud periodically; at the same time, the roadside unit is also responsible for sending trajectory planning instructions generated by the cloud to vehicles within its communication range; The vehicles include three types: connected vehicles, buses, and emergency vehicles; each vehicle is assigned a priority level. In addition, each vehicle is associated with a preference weight vector to determine its routing behavior; For vehicles Preference vector Defined as: （1）； in and These respectively reflect the driver's preference for driving distance and driving time; the weights satisfy... ,and ; Each vehicle The features are represented by the state vector as follows: （2）； in and They represent the start and end points, respectively. This represents the vehicle speed, which follows a normal distribution. This represents the average speed of vehicles traveling within a road segment. The variance representing the vehicle's speed. Priority This refers to the path preference factor, i.e., the preference vector.

3. The cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning according to claim 1, characterized in that, The specific implementation process of step 2 includes: Using line graph model , This represents the set of nodes in a line graph model. Each node in Corresponding to a road segment in the original physical road network ; This represents the set of directed edges in a line graph model. Used to characterize adjacent road segments in a physical road network and The topological connections between them and the vehicle steering relationship; if the vehicle can move from the physical road segment Drive directly into the physical road section Then in the corresponding line graph model node and There exists a directed edge between them; the following indices are defined to characterize road traffic state information: 1) Average travel time: for a given road segment Considering time The previous length was time window Based on the time window The distance a vehicle travels and its average speed are defined as follows: （3）； in, Indicates within the time window Average driving speed within the area This indicates that the vth vehicle is on the road segment. The driving distance on the road The total number of vehicles on the road segment; This indicates the length of the time window; therefore, the average travel time. The derivation is as follows: （4）； in The physical length of the road segment; 2) Free-flow travel time: Defines the free-flow travel time. The time required for a vehicle to traverse a road segment under ideal, uncongested conditions, constrained only by the segment length and speed limit: （5）； in, Represents the free-flow velocity; 3) Space occupancy rate: This refers to the percentage of space occupied. Defined as the ratio of the total space occupied by vehicles to the length of the road segment. : （6）； in For vehicles The length of the car body, Maintain a safe following distance.

4. The cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning according to claim 1, characterized in that, Step 3, the process of extracting spatiotemporal features, includes: The road network is modeled using graph attention networks, with road segments as the basic modeling unit. This modeling process depicts the spatial dependencies between adjacent road segments and the temporal evolution of traffic conditions. The spatiotemporal features of each road segment are extracted and divided into two parts: historical state features and future traffic prediction features. Section Historical state feature sequence Defined as: （7）； in, Indicates road segment The average speed of vehicles on the road Indicates the number of vehicles on the road segment. Indicates the occupancy rate of road space. Indicates the average travel time of the road segment; For any road segment Constructing future traffic flow sequences: ;in, Indicates road segment In the future The projected traffic flow within each time step; the dynamic sequence includes historical state characteristic sequences and future traffic flow sequences; Calculate vehicles Arrival Section Expected time The calculation method is as follows: （8）； in, This represents the set of road segments included in the vehicle's current planned route. For road section Free circulation time; After obtaining the vehicle arrival time, further statistics are calculated on traffic flow in each future time step; Road sections within a time step Expected flow for: （9）； in, For indicator functions, This indicates that the planned path includes road segments. A collection of vehicles; Indicates vehicle Arrival Section The estimated time; Feature encoding is performed on historical state feature sequences and future traffic flow sequences, including: First, the Long Short-Term Memory (LSTM) network is used to process the historical state feature sequence. Encode to capture historical feature vectors of traffic flow changes over time. : （10）； Using a multilayer perceptron (MLP) to analyze future traffic flow sequences Perform nonlinear mapping: （11）； in, Represents a non-linear activation function; This represents the future traffic feature vector obtained by nonlinear mapping of the future traffic flow sequence. The weight matrix of the nonlinear mapping layer is represented by... Indicates road segment Future traffic flow sequences This represents the bias term of the nonlinear mapping layer; Subsequently, the two types of feature vectors are concatenated to form a joint temporal feature representation. : （12）； After obtaining the joint temporal feature representation, the importance weights of historical traffic information and future traffic information are adaptively learned through an attention mechanism. The importance weights are calculated as follows: （13）； in, This represents the learnable weight matrix. This is the attention weight vector; Finally, the historical feature vector and the future traffic feature vector are weighted and fused based on the obtained attention weights to obtain the road segment. Temporal feature enhancement representation : （14）； in, The attention weight score represents the historical feature vector. This represents the attention weight score of the future traffic feature vector; Spatial feature modeling using line graph structures; set up Represents a node exist The neighbor set in the road segment, that is, the vehicle passes through the road segment The set of downstream road segments that continue to be entered; based on the neighborhood structure, a graph attention mechanism is used to adaptively aggregate the information of neighboring road segments; nodes Its neighboring nodes Attention coefficient between The calculation is as follows: （15）； in, The characteristic transformation matrix, This represents the transpose of the spatial attention weight vector. and Representing road segments Adjacent road sections Temporal feature enhancement representation, This represents the linear rectification activation function with leakage coefficient; Subsequently, the attention coefficients are normalized to obtain the node... Its neighboring nodes Attention weights : （16）； in, Represents a node and Attention coefficient between nodes Belongs to node exist Neighbor set , Represents an exponential function; Obtaining attention weights Then, the features of neighboring nodes are weighted and aggregated to obtain the spatiotemporal fusion representation of the road segment. : （17）； in, Represents a non-linear activation function; Finally, the spatiotemporal feature representations of all road segments are summarized to form the spatiotemporal feature matrix of the entire road network. : （18）； The spatiotemporal representation of the road network includes the spatiotemporal fusion representation of road segments in formula (17). And the spatiotemporal characteristic matrix of the entire road network in formula (18) .

5. The cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning according to claim 1, characterized in that, In step 3, the congestion index modeling based on near-end strategy optimization is specifically implemented as follows: In a reinforcement learning environment, the spatiotemporal feature matrix of the entire road network is used. As input to the environment state, the state space The spatiotemporal feature matrix of the entire road network is as follows: （19）； in, The spatiotemporal feature matrix represents the overall road network and is used to characterize the overall traffic operation status of the road network at the current moment; Action space Defined as a continuous vector: （20）； in, Indicates road segment The congestion index, ; The strategy is evaluated using a delayed reward mechanism, and the reward function is... Defined as: （21）； in Indicates road segment Average travel time Indicates the number of road segments in the road network. This indicates the number of road segments that exceed the congestion threshold. This is the penalty constant for severe congestion.

6. The cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning according to claim 1, characterized in that, Learn to obtain a road congestion index that reflects the current level of congestion and its future trends, including: First, obtain the spatiotemporal feature matrix of the entire road network at the current moment. As input for environmental status; Then, the spatiotemporal feature matrix The policy network in the trained PPO algorithm is input into the nonlinear transformation of multiple neurons to map high-dimensional traffic features into continuous vectors corresponding to the number of road segments; the output layer constrains the values ​​in the [0,1] interval to generate the dynamic road congestion index corresponding to each road segment. The construction process of the dual-objective path planning model specifically includes: From the starting point to the node Cumulative path cost Defined as: （22）； in, Represents the distance from the starting node to the node. The collection of road segments traversed; and Representing road sections Normalized length and average travel time, parameters and Indicates vehicle The preference weights for travel distance and travel time, parameters This represents a balance coefficient between travel preferences and congestion control. Indicates road section The congestion index; Heuristic functions Defined as: （23）； in, Represents a node With vehicles Euclidean distance between destinations The free-flow velocity of the road; Therefore, in the bi-objective path planning model, nodes The total cost function is expressed as: （24）； By continuously selecting the node with the smallest total cost function value for expansion, the optimal path from the starting point to the destination, i.e. the optimal driving path, is gradually obtained through the search.

7. A cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning according to any one of claims 1-6, characterized in that, The specific implementation process of step 6 includes: An adaptive replanning strategy is employed to achieve dynamic response to traffic congestion by jointly controlling the trigger frequency of path replanning and the vehicles involved in the replanning process; this includes: First, congested road segments in the current road network are identified based on the road congestion index. When the congestion index of a road segment exceeds a preset threshold, it is determined to be a congested road segment. Defined as: （25）； in, Indicates road section The congestion index, The congestion determination threshold is the threshold value for determining congestion. The range of values ​​is ; The execution interval of route replanning is dynamically adjusted based on the current number of congested road sections. It is negatively correlated with the degree of road network congestion, defined as: （26）； in, This represents the maximum allowable replanning interval under free-flow conditions. The minimum replanning interval is set to avoid frequent path oscillations. This is the sensitivity coefficient; Assumption This represents the set of road segments included in the vehicle's current planned path. Vehicles whose paths intersect with congested road segments are selected as candidate vehicles. : （27）； This represents the set of road segments included in the vehicle's current planned path; After obtaining the candidate vehicle set, further filtering is performed based on the spatial distance between the vehicles and the congested road segments; for any congested road segment... Vehicles that meet the following conditions are considered to be affected by traffic congestion. : （28）； in, Indicates vehicle Current location information and road segment The spatial distance between them is the shortest Euclidean distance from the vehicle's current location to the road segment; Distance threshold The determination is performed using an adaptive method, and the expression is: （29）； in, Based on the replanning of distance, Indicates time Section The congestion index, This represents the maximum value of the congestion index. Ultimately, the collection of all vehicles affected by congestion. Represented as: （30）； in, It is affected by the road section The set of vehicles affected by congestion is used to obtain the union of these sets of vehicles to get all vehicles in the road network affected by congestion. Only for sets The vehicle performs a route replanning operation.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the cloud-edge-device collaborative dual-objective path planning method based on spatiotemporal graph reinforcement learning as described in any one of claims 1-7.

10. A cloud-edge-device collaborative dual-objective path planning system based on spatiotemporal graph reinforcement learning, characterized in that, include: The urban traffic operation data acquisition module is configured to: collect urban traffic operation data from the edge-side roadside units, namely vehicle operation information and road traffic status information within the coverage area, including vehicle location, instantaneous speed, travel time, traffic flow and road space occupancy rate, and upload the urban traffic operation data to the cloud server, while simultaneously sending real-time traffic status information and planned route instructions to vehicle terminals within the coverage area; The preprocessing and fusion module is configured to: preprocess and fuse the collected urban traffic operation data in the cloud; construct a graph neural network based on the line graph model to map the physical road segments in the urban traffic network into graph nodes; the directed connection edges between graph nodes represent the feasible driving relationship between vehicles in different road segments; and construct a dynamic sequence based on historical time windows to form a spatiotemporal representation of the road network that simultaneously describes the spatial correlation and temporal evolution characteristics of traffic status. The road congestion index acquisition module is configured to: add an attention mechanism to the constructed graph neural network to perform deep modeling of traffic congestion status; aggregate the spatiotemporal features of adjacent road segments through the attention mechanism to extract the spatial dependencies between roads; simultaneously introduce a long short-term memory network to encode the time series features of traffic status and capture the dynamic changes of traffic flow; input the spatiotemporal features extracted by the graph attention network into the proximal policy optimization reinforcement learning framework; and learn to obtain a road congestion index that reflects the current congestion level and future evolution trend of road segments through continuous interaction with the simulation environment and policy updates. The dual-objective path planning model construction module is configured to: construct a dual-objective path planning model based on the generated road congestion index and combined with the travel preferences of individual vehicles. The individual travel cost is weighted by the user's preference for travel distance and travel time, and the global traffic management objective is achieved by introducing the road congestion index. The dual-objective path planning model simultaneously focuses on individual travel efficiency and road congestion level, comprehensively generates a cost function for path search, and adjusts the relative importance of individual travel efficiency and road congestion level in path planning through balancing parameters. The optimal driving route generation module is configured as follows: the cloud uses a heuristic path search algorithm to generate the optimal driving route based on the constructed dual-objective path planning model, and returns the planning results to the edge server, which then sends the route to the vehicle terminal for execution. The collaborative optimization module is configured as follows: during vehicle travel, the roadside unit continuously collects real-time traffic data and uploads it to the cloud, while the cloud monitors changes in the road congestion index of each road segment; when the road congestion index of some road segments exceeds a preset threshold or the traffic condition changes significantly, a dynamic path adjustment mechanism is triggered to identify the set of target vehicles affected by congestion and replan the remaining paths of the affected vehicles based on the updated road congestion index; through a selective local path update strategy, the path planning results adaptively respond to real-time changes in the traffic environment, achieving collaborative optimization of individual travel needs and global traffic efficiency in a dynamic traffic environment.

Images