Large-scale road network traffic control method based on deep reinforcement learning large model

Abstract

The application relates to a large-scale road network traffic control method based on a deep reinforcement learning large model and belongs to the technical field of intelligent traffic control. The method comprises the following steps: perceiving real-time multi-modal road network information including urban road intersections, highway entrance ramps and emergency lanes, generating a space-time fusion representation vector representing the current traffic network state by fusing a space graph construction method and a time series embedding method; splicing the space-time fusion representation vector and a historical state memory to serve as input, performing state feature distillation on a backbone network of a pre-trained large language model to enhance state representation, and outputting a traffic control decision through a policy network with a hierarchical action space; and guiding and optimizing the training process of the deep reinforcement learning algorithm through cross-modal knowledge transfer and a progressive curriculum learning strategy to improve the model training efficiency and generalization ability. The application improves the generalization performance and accuracy of the control strategy while ensuring real-time response speed.

Description

Technical Field

[0001] This invention belongs to the field of intelligent traffic control technology and relates to a large-scale road network traffic management method based on a deep reinforcement learning model. Background Technology

[0002] With the accelerating pace of global urbanization and the continuous growth of motor vehicle ownership, traffic signal control in large-scale road networks is playing an increasingly central role in modern Intelligent Transportation Systems (ITS). As a complex, nonlinear, and dynamic system, the operational efficiency of urban transportation networks directly impacts regional economic development, social energy consumption, and the quality of life for residents. However, traditional single-point control or small-scale coordination signal control methods are no longer sufficient to effectively address the complex spatiotemporal dynamics of traffic flow in large-scale road networks. This has led to a series of serious problems, including the normalization and cross-regional spread of traffic congestion, a significant increase in vehicle travel delays, and a continuous rise in exhaust emissions. Particularly in megacities with thousands of intersections, traffic flow exhibits strong coupling and nonlinear characteristics, necessitating a systematic and efficient global control strategy to collaboratively optimize traffic operation across the entire region. Furthermore, the rise of vehicle-to-everything (V2X) and autonomous driving technologies has placed higher demands on the accuracy and latency of traffic signal control, making it a key infrastructure supporting the development of next-generation intelligent connected transportation systems and possessing profound strategic significance for achieving efficient, safe, and green urban transportation.

[0003] Looking at the development history of traffic signal control technology, the existing technology system can be mainly divided into the following stages, but each stage has significant technical bottlenecks:

[0004] The first generation is the fixed timing control system. This type of system is represented by early electromechanical signal controllers, whose core is to control the switching of signal phases using a pre-set, fixed timing scheme. Its hardware structure typically includes a power module, a mechanical timing device, and a relay switch assembly. The main drawback of this technology is its complete lack of real-time traffic flow perception, making it unable to dynamically adjust the timing scheme according to changes in traffic demand. When traffic flow fluctuates significantly, this fixed control mode can result in up to 40% of green light time being wasted, greatly reducing the efficiency of intersections.

[0005] The second generation is the inductive control system. This system uses vehicle detectors such as loop detectors buried near the stop line at intersections, combined with a programmable logic controller (PLC), to appropriately extend the current green light phase based on real-time vehicle arrival data. However, this type of system is essentially still a single-point optimization strategy, with its decision-making scope limited to a single intersection, lacking effective coordination with adjacent intersections. Therefore, in arterial roads or regional road networks, it is prone to causing "green wave" interruptions, resulting in frequent starts and stops between adjacent intersections. Real-world data shows that using this technology can lead to differences in vehicle queue lengths of over 35% between adjacent intersections, failing to achieve smooth regional traffic flow.

[0006] The third generation is the centralized adaptive system. These systems typically employ a three-tier architecture of "detector-area controller-central server," aiming to achieve coordinated control within a region. For example, the well-known SCOOT system utilizes a central host such as the IBM System / 370 to run the TRANSYT traffic model for online simulation and timing optimization. However, this approach faces severe computational complexity issues. As the number of controlled intersections increases, the computational burden grows exponentially. When the control scale exceeds 500 intersections, the delay in optimization decisions can exceed 15 minutes, failing to meet the requirements of real-time control. Another similar system, RHODES, introduces a rolling time-domain optimization strategy, but its effectiveness is highly dependent on the accuracy of the origin-end point (OD) matrix. In practical applications, the estimation error of the OD matrix is ​​generally around 30%, directly affecting the accuracy of the control effect. Furthermore, most of these systems employ a fixed phase sequence design, limiting the flexibility of signal control and resulting in an actual green light time utilization rate of only about 78.3%.

[0007] As an improvement over centralized control, distributed intelligent control systems (such as multi-agent signal control) have also been proposed. This approach uses frameworks like JADE, attempting to achieve coordination through autonomous negotiation between intersection controllers. However, its main bottleneck lies in communication latency. In urban environments, the communication latency can reach 300 milliseconds per kilometer. When the system scales up (e.g., more than 10 intersections), the accumulated communication and negotiation time will result in a decision-making time exceeding 5 seconds, making it difficult to synchronize control between intersections and thus weakening the coordination effect.

[0008] In recent years, with the development of artificial intelligence technology, reinforcement learning has been applied to the field of signal control. Early Q-learning algorithms learned by defining a state space (such as queue length, arrival flow) and an action space (such as phase switching). However, this method encounters the "curse of dimensionality" when the state variables exceed 20 dimensions, and the Q-value table becomes abnormally large, making it difficult for the model to converge. To solve this problem, deep reinforcement learning (DRL) schemes have been proposed, such as using a three-layer fully connected network to approximate the Q-value function. While this improves the ability to extract high-dimensional features, it brings new problems: first, it requires millions or even tens of millions of interaction data to train a basically effective policy; second, its "black box" nature leads to poor policy interpretability, making it difficult to verify and deploy using traditional traffic engineering theory.

[0009] In summary, existing large-scale road network signal control technologies suffer from the following shortcomings that urgently need to be addressed in multiple aspects, including data acquisition, model algorithms, system implementation, and spatiotemporal modeling:

[0010] At the data acquisition level: Sensors have inherent limitations. For example, geomagnetic sensors have a false detection rate as high as 15% in rainy or snowy weather, while radar has a false detection rate of 10-15% in multi-lane scenarios. Furthermore, the data sampling periods (ranging from 20ms to 5s) and data formats (ASCII / binary) of various devices are highly inconsistent, posing significant challenges to multi-source data fusion. In areas where 5G networks are not fully covered, data transmission delays can reach 2-5 seconds, severely impacting the real-time performance of control.

[0011] At the model and algorithm level: Traditional optimization methods (such as linear programming) take too long to solve large-scale problems (such as timing schemes for 200 intersections), sometimes exceeding 45 minutes. Machine learning methods require traffic experts to manually define more than 30 traffic flow features, resulting in high feature engineering costs and difficulty in generalization. Deep reinforcement learning methods suffer from unstable training processes and slow convergence speeds, typically requiring thousands of iterations to achieve stable performance in simulation environments like SUMO.

[0012] At the system implementation level: hardware upgrades are costly, and the cost of upgrading a single smart intersection is high. Real-time control of thousands of intersections requires up to 50 TFLOPS of computing power. In addition, the compatibility upgrade cycle between the new system and existing industry standard protocols (such as NTCIP 1202v3) is lengthy, typically taking 3-5 years.

[0013] In terms of spatiotemporal modeling and control flexibility: traditional methods often use macroscopic traffic flow parameters (volume, speed, occupancy) for modeling, failing to capture microscopic driving behavior characteristics. Temporal models such as Long Short-Term Memory (LSTM) networks struggle to effectively handle the topological heterogeneity of road networks, while Graph Neural Networks (GNNs) have limited ability to model long-range spatiotemporal dependencies. Existing methods typically divide the control area into fixed sub-regions (e.g., 8-16 intersections in the SCATS system), a static division that cannot adapt to dynamically changing traffic flow patterns. Regarding real-time performance and flexibility, Model Predictive Control (MPC) methods require solving non-convex optimization problems, resulting in computational latency that grows exponentially with scale; while rule-based control strategies (e.g., fuzzy logic) require manual maintenance of thousands of rules. Furthermore, commonly used hard-coded rules (e.g., minimum green light time of 5 seconds, strict phase conflict avoidance) lack flexible constraint mechanisms, leading to a 15-20% decrease in system flexibility when responding to emergencies.

[0014] Therefore, it is urgent to develop a new traffic signal control method to overcome the many limitations of existing technologies in terms of data, models, computing power, and systems. Summary of the Invention

[0015] In view of this, the purpose of this invention is to provide a large-scale road network traffic management method based on a deep reinforcement learning large-scale model, solving the core technical challenges of traditional traffic management systems in achieving high-precision spatiotemporal perception, intelligent real-time decision-making, and cross-scenario knowledge transfer in large-scale complex road networks. Furthermore, based on this, a multimodal road network information perception coding system, a large-model-enhanced deep reinforcement learning algorithm, and a signal control knowledge distillation framework are designed, significantly improving the model's adaptability and generalization capabilities across scenarios, exhibiting good scalability and universality, and enabling it to adapt to different traffic management tasks.

[0016] To achieve the above objectives, the present invention provides the following technical solution:

[0017] Solution 1: A large-scale road network traffic management method based on a deep reinforcement learning model, comprising the following steps:

[0018] S1: Construct a multimodal road network information perception and coding system: perceive real-time multimodal road network information including urban road intersections, highway entrance ramps and emergency lanes, and generate a spatiotemporal fusion representation vector (STFR-Vector) that represents the current traffic network state by fusing spatial graph construction method and time series embedding method.

[0019] S2: Deep reinforcement learning algorithm for large model enhancement: The spatiotemporal fusion representation vector and historical state memory are concatenated as input, and the state feature is distilled using the backbone network of the pre-trained large language model to enhance the state representation. Traffic control decisions are output through a policy network with a hierarchical action space.

[0020] S3: Application of the traffic control knowledge distillation framework: Through cross-modal knowledge transfer and progressive course learning strategies, the training process of the deep reinforcement learning algorithm is guided and optimized to improve model training efficiency and generalization ability.

[0021] Furthermore, in step S1, generating the spatiotemporal fusion representation vector (STFR-Vector) specifically includes:

[0022] Spatial graph construction method: Define the road network as a topology graph , where the node set , Indicates a signal-controlled intersection; edge set , Representing the connectivity of road segments, weight matrix , Represents the weight of the edge. , The number of intersections is represented; the edge weights are calculated by fusing static topological features based on Gaussian distance kernels with dynamic flow features based on real-time queue lengths; the topological graph is processed using an improved graph attention network v2 (GATv2) to generate spatial features;

[0023] Time series embedding method: Multi-scale sliding window sampling is used to sample the time series data of each node to obtain multi-scale features, and periodic features are extracted by combining periodic feature encoder; the periodic features and multi-scale features are fused by spatiotemporal gated fusion unit (STGFU) to generate time series features;

[0024] Spatial features and temporal features are fused to obtain the final spatiotemporal fusion representation vector.

[0025] Furthermore, in step S1, the spatial graph construction method utilizes an improved graph attention network v2 (GATv2) to process the topological graph and generate spatial features. Specifically, this includes: to enhance the spatiotemporal expressiveness of the graph structure, a multi-scale adjacency matrix construction method is introduced, defining... k adjacency matrix To capture k-hop neighborhood relationships;

[0026]

[0027] in, For threshold-based The binary function; for of The power is used to calculate the relationship between nodes. The sum of the weights of the skip paths, The exponentiation operation generalizes the direct adjacency (1-hop) of nodes to... The distant neighbor relationship of a k-hop.

[0028] When dealing with large sparse road networks, a hierarchical graph construction method based on road network partitioning is adopted to construct the global topology graph. Decomposed into multiple subgraphs , This represents the number of subgraphs, where subgraphs are separated by boundary nodes. Connect and define weights across subgraphs. , Subgraph m and n Euclidean distance between the centroids Indicates the strength of cross-regional association; parameters The global scaling factor or baseline connection strength representing cross-regional association determines the overall magnitude (baseline level) of the entire Gaussian function. The connection weights within a subgraph and the connection weights across subgraphs may be in different numerical ranges. The parameters allow the weights of these two parts to be adjusted to a comparable range, preventing one from dominating the other.

[0029] In terms of dynamic traffic feature extraction, each node Associated multidimensional feature vectors :

[0030]

[0031] Among them, flow characteristics Includes the current queue length and rate of change ;

[0032] velocity characteristics Including average speed and standard deviation ;

[0033] Market share characteristics , Lane occupancy rate;

[0034] Phase characteristics The current phase state of one-hot encoded signal control for urban road intersections, highway entrance ramp control, and emergency lane opening control.

[0035] Furthermore, in step S1, the time series embedding method uses multi-scale sliding window sampling to construct a hierarchical time-aware mechanism and employs a three-level sliding window configuration (30 seconds, 5 minutes, and 15 minutes) to capture multi-granularity features of traffic flow; each window is processed by an independent temporal convolution module; and finally, the multi-scale features are formed by channel splicing.

[0036] The periodic feature encoder explicitly models the inherent periodicity of traffic flow, specifically by using a bi-periodic embedding function based on harmonic extension terms to extract periodic features.

[0037] Furthermore, in step S2, the specific method for performing state feature distillation using the backbone network of the pre-trained large language model is as follows:

[0038] A pre-trained large language model backbone network with frozen parameters is used as a feature extractor. The spatiotemporal fusion representation vector and the historical state memory are concatenated as input to obtain a preliminary enhanced representation.

[0039] By using a trainable low-rank adaptive (LoRA) adapter, the output of a large language model can be fine-tuned to suit specific traffic control tasks, thereby reducing the training data requirements while preserving the general knowledge of the large language model.

[0040] Furthermore, in step S2, the layered action space includes:

[0041] Macro-level action space: used to output continuous actions and control the macro-level flow distribution between regions;

[0042] Micro-level action space: used to output discrete actions, selecting specific signal phase combinations, ramp control rates, or emergency lane opening states for individual traffic control points.

[0043] Furthermore, in step S2, the deep reinforcement learning algorithm is trained using an improved proximal policy optimization (PPO-Clip) algorithm, and a constraint optimization loss function based on the Lagrange multiplier method is designed to enforce traffic safety constraints during the optimization process.

[0044] Furthermore, the traffic safety constraints include: minimum green light time constraint, signal phase conflict constraint, and traffic flow balance constraint.

[0045] Furthermore, in step S3, the traffic control knowledge distillation framework specifically includes:

[0046] Cross-modal knowledge transfer module: KL divergence loss is used for state representation distillation to align the feature spaces of the large language model and the traffic state encoder; mean squared error loss is used for policy distillation to constrain the policy distribution of the teacher and student models;

[0047] Progressive learning strategy: The model training process is divided into multiple stages from easy to difficult, including single intersection control stage, arterial road coordinated control stage and regional road network collaborative control stage, and the stage transition is automatically triggered according to the preset reward performance index.

[0048] Option 2: A large-scale road network traffic management system based on a deep reinforcement learning model, comprising:

[0049] The multimodal road network information perception and coding module is configured to perceive real-time multimodal road network information and generate a spatiotemporal fusion representation vector (STFR-Vector) that represents the current traffic network state by fusing spatial graph construction methods and time series embedding methods. Specifically, the generation of the spatiotemporal fusion representation vector involves: using an improved graph attention network v2 (GATv2) to process the road network spatial graph that fuses static topology and dynamic traffic features to generate spatial features; using a spatiotemporal gating fusion unit (STGFU) configured with a multi-scale sliding window to process traffic time series data to generate temporal features; and fusing the spatial features and temporal features into a spatiotemporal fusion representation vector.

[0050] The large model-enhanced deep reinforcement learning decision module is configured to receive the spatiotemporal fusion representation vector, enhance the state representation using the backbone network of a pre-trained large language model and a low-rank adaptive (LoRA) adapter, and output traffic control decisions through a policy network with a hierarchical action space.

[0051] The traffic control knowledge distillation training module is configured to train and optimize the deep reinforcement learning decision module through cross-modal knowledge transfer and progressive course learning strategies.

[0052] The beneficial effects of this invention are as follows:

[0053] 1) Significantly Enhanced Spatiotemporal Perception and Accurate Representation Capabilities: This invention utilizes a unique Spatiotemporal Fusion Representation Vector (STFR-Vector), combining Dynamic Graph Attention Network (GAT) and multi-scale temporal modeling, to accurately capture complex spatial topological dependencies and dynamic traffic flow evolution patterns across multiple time scales (seconds, minutes, and hours) in large-scale road networks. This overcomes the shortcomings of traditional methods that rely on macroscopic parameters, static regional divisions, or difficulty in handling spatial heterogeneity, resulting in a more comprehensive and accurate representation of traffic conditions and providing a high-quality input foundation for subsequent intelligent decision-making.

[0054] 2) Significantly Improved Decision Intelligence and Training Efficiency: This invention innovatively introduces a deep reinforcement learning framework enhanced with Large Language Models (LLM). By performing knowledge distillation through a frozen LLM backbone network and a trainable LoRA adapter, it effectively utilizes the general semantic understanding and reasoning capabilities inherent in the pre-trained large model, greatly reducing reliance on massive traffic interaction data and accelerating model convergence. Compared to traditional deep reinforcement learning methods that require training from scratch, this invention offers higher training efficiency and stronger policy robustness.

[0055] 3) Enhanced generalization and scalability of control strategies: Through hierarchical action space design (macro-level flow allocation + micro-level phase control) and a traffic management knowledge distillation framework, this invention can not only generate refined control actions but also effectively transfer learned knowledge. The progressive learning strategy (single point -> trunk line -> region) enables the model to gradually adapt to simple to complex scenarios, resulting in a control strategy with strong generalization performance that can be easily extended to different types and scales of traffic scenarios (urban intersections, highway ramps, etc.), solving the problem of traditional methods being "one policy for one location" and difficult to port.

[0056] 4) Effectively ensures the safety and real-time performance of traffic control: This invention integrates a safety constraint processing mechanism based on the Lagrange multiplier method into the optimization objective. This allows key safety rules such as minimum green light time and phase conflict avoidance to be incorporated as flexible constraints into the optimization process, ensuring the absolute safety of the output decision and overcoming the lack of flexibility caused by hard-coded rules in traditional methods. Simultaneously, the overall algorithm framework is designed with computational efficiency in mind, meeting the real-time control requirements of large-scale road networks (thousands of nodes), and solving the bottlenecks of excessive computational latency in traditional centralized systems and excessive communication latency in distributed systems.

[0057] In summary, this invention provides a complete end-to-end intelligent traffic management technology system, achieving technological breakthroughs in multiple levels such as perception, decision-making, training, and application, which can significantly improve the real-time performance, security, adaptability, and overall operational efficiency of large-scale road network traffic management.

[0058] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0059] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0060] Figure 1 This is a framework diagram of the large-scale road network traffic management method based on a deep reinforcement learning model, as presented in this invention. Detailed Implementation

[0061] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0062] This invention builds an environment based on either the "strategy-feedback" data of "traffic control strategy-traffic status" in a real road network or a micro-level traffic simulation of the road network, and then establishes a large-scale road network traffic control method based on a deep reinforcement learning model.

[0063] This method is based on a unified control framework for large-scale deep reinforcement learning model enhancement. It uses a spatio-temporal fusion representation vector (STFR-Vector) to fuse static topology (Gaussian distance kernel) and dynamic flow (queue correlation). A graph attention network is used to generate 512-dimensional spatial features, and a multi-scale sliding window CNN (30s / 5min / 15min) is used to extract features in the temporal dimension. Combined with periodic encoding, a spatio-temporal gated fusion unit (STGFU) is used to generate temporal features. Next, the large-scale model enhancement DRL architecture uses a frozen LLM backbone for state feature distillation. With a trainable LoRA adapter and a policy network, a hierarchical action space is designed. The macro-level continuous actions control the flow allocation in the region, and the micro-level discrete actions select the phase combination. The optimization objective adopts an improved PPO-Clip algorithm, and a Lagrange multiplier constraint optimization spatio-temporal consistency optimization loss function (STCO Loss) is designed. The key constraints include minimum green light time and phase conflict matrix. This method significantly improves the speed of spatiotemporal feature extraction in terms of spatiotemporal modeling capabilities; the graph attention network supports dynamic neighborhood perception, enabling the control policy to adapt to changes in road network topology. Meanwhile, the frozen LLM backbone retains commonsense reasoning capabilities, and LoRA fine-tuning reduces data requirements, while knowledge distillation loss ensures policy robustness.

[0064] Please see Figure 1 The method specifically includes the following parts:

[0065] 1. Multimodal road network information sensing and coding system

[0066] 1) Spatial Graph Construction Method

[0067] In the multimodal road network information perception and coding system, the spatial graph construction method is the core foundation for achieving accurate traffic state representation. Its technical essence lies in transforming the physical road network topology and dynamic traffic flow characteristics into a computable graph structure data representation. The specific construction process is as follows:

[0068] First, define the road network topology diagram. , where the node set Indicates a signal-controlled intersection, edge set It represents the connection relationship of road segments.

[0069] Weight matrix: This indicates the connection strength or influence between signal-controlled intersections (nodes) in the road network, used to construct the spatial map structure. For the first , Each signal controls the intersection weight matrix, which directly defines the intersection. Other intersections The strength of the association. In spatial graph construction, signalized intersections As a node, its neighboring nodes The influence of Weighted. The static topology weight is calculated by fusing static topology features and dynamic traffic features. Calculated using Gaussian kernel function: ,in For road section The physical length, Distance attenuation coefficient (usually taken as 1.5 times the average spacing of the road network); dynamic traffic weight. Then, it is calculated based on real-time traffic conditions: ,in express t Intersection i Queue length, express t Intersection j The length of the queue. The historical average queue length is used to standardize the queue length variation in the dynamic weights. It is the average queue length of the entire road network (all intersections) over a specific historical period, rather than the individual average of each intersection. For example, it might be the average queue length of all intersections during the same time period (such as morning rush hour) over the past few weeks or months. This design is intended to provide a global benchmark, scaling queue differences relative to historical normal levels and avoiding model instability due to excessively large or small absolute values. As a dynamic feature fusion coefficient, in its implementation, the range of queue length variation in historical data is first analyzed, and then different features are tested through ablation experiments. Impact on model performance; at the same time, ensure Accurate calculations are required to provide effective standardization; grid search and cross-validation can be used. Final edge weights. Generated via linear combination:

[0070]

[0071] To enhance the spatiotemporal expressiveness of graph structures, a multi-scale adjacency matrix construction method is introduced, defining... k adjacency matrix To capture k Skip neighborhood relationships;

[0072]

[0073] in, Represents the spatial weight matrix of k The power is used to simulate the impact of traffic on the road network. k The propagation of jumps. For threshold-based The binary function is used. The threshold τ is determined by optimizing it on the validation set using grid search or cross-validation to maximize model performance (traffic prediction accuracy). In initial experiments, τ is typically tested between 0.1 and 0.5; if the road network is dense, τ can be set higher to filter weak connections; if the road network is sparse, τ can be set lower to retain more connections. Specifically, you can start by gradually adjusting τ = 0.1 and observe whether the binarized graph structure is reasonable (ensuring each node has enough neighbors, avoiding excessive sparseness or density). In particular, when dealing with large sparse road networks, a hierarchical graph construction method based on road network partitioning is used to decompose the global graph into multiple subgraphs. ,in Indicates the first c The set of intersection nodes contained within each subgraph This represents the road segment connection relationship between these nodes, the first... c The set of edges in each subgraph, that is, the direct connecting paths between the aforementioned nodes. represents the total number of subgraphs after dividing the global road network; among which, the subgraphs are connected through boundary nodes and define the weights across subgraphs , where represents subgraph m and n is the Euclidean distance between the centroids. represents controlling the cross-region association strength. First, calculate the distance distribution between the centroids of all subgraphs, and then set = 1.0 × mean( ) as the initial value, and then fine-tune according to the model performance; meanwhile, monitor the ratio of the weights across subgraphs to the weights within subgraphs to avoid imbalance.

[0074] Boundary nodes refer to those nodes that are in subgraph but have direct connections with other subgraphs. These nodes are located at the "boundary" of the subgraphs and are responsible for information interaction between subgraphs. If a node in subgraph is connected to a node in subgraph by an edge (i.e., but , , ), then is marked as a boundary node of . The determination of boundary nodes depends on the graph partitioning algorithm. To identify boundary nodes, use metis and networkx in python.

[0075] In terms of dynamic traffic feature extraction, each node is associated with a multi-dimensional feature vector :

[0076]

[0077] Among them, the traffic feature includes the current queue length and the change rate; the speed feature includes the average speed and the standard deviation; the occupancy feature is the lane occupancy. The lane occupancy refers to the percentage of the length occupied by vehicles on a lane or a group of lanes to the total observed length at a specific time point t , that is = (total vehicle occupancy time) / (observation duration T). The phase feature is the current phase state of the one-hot encoding for signal control at urban road intersections, ramp control at highway entrances, and emergency lane opening control. is a one-hot encoded vector that indicates at the momentt control node This determines which direction of traffic movement has the right-of-way (green light or passage). One-Hot encoding is a method of converting categorical variables into binary vectors where only one bit is 1 and the rest are 0. First, all possible phase states are defined, then a position is assigned to each state to construct the vector.

[0078] 2) Time series embedding method

[0079] In time series embedding methods, the introduction of multi-scale sliding window sampling and periodic feature encoders constitutes the core technical framework for spatiotemporal feature extraction. Multi-scale sliding window sampling achieves multi-granularity feature capture of traffic flow by constructing a hierarchical time-aware mechanism and using a three-level sliding window configuration (30 seconds, 5 minutes, and 15 minutes).

[0080] For the original time series Define the sliding window function ,in Corresponding to micro, meso, and macro time scales respectively, each window is processed by an independent temporal convolution module.

[0081]

[0082] in, For the first k Features output by each window; can be It can be understood as a "feature summary" or "encoding". It is no longer the raw, lengthy, and potentially noisy time-series data (such as traffic flow per second within 30 seconds), but a small, dense vector refined by a neural network. This vector captures the most relevant and discriminative features within that time window.

[0083] Ultimately, multi-scale features are formed through channel splicing. :

[0084]

[0085] This design enables the model to simultaneously perceive instantaneous fluctuations (30-second level), short-term trends (5-minute level), and long-term patterns (15-minute level) of traffic flow.

[0086] The periodic feature encoder explicitly models the inherent periodicity of traffic flow by designing a dual-period embedding function:

[0087]

[0088] in, Indicates a point in time The periodic position encoding vector. A mathematical method that maps absolute timestamps to a high-dimensional vector space, with the aim of explicitly telling the model the phase position of the current moment in a daily cycle, a weekly cycle (and more sub-cycles extended by harmonics) in a way that is easy for a machine to understand.

[0089] (Daily cycle) and (Period) captures day / night patterns and weekday / weekend differences respectively. To improve high-frequency period resolution, harmonic extension terms are introduced:

[0090]

[0091] Corresponding to the third harmonic, this extension can effectively characterize traffic sub-modes such as the morning rush hour (approximately a 2-hour cycle). The fusion of periodic features and original features employs a gating mechanism.

[0092]

[0093] in, This represents the gating signal vector. It is a range A vector between [variables]. Each element of this vector acts as a "switch," controlling the flow of multi-scale features. The information of how many proportions to select from the periodic characteristics.

[0094] This represents the Sigmoid activation function, and its formula is: .

[0095] The weight matrix represents the gated linear transformation, which affects the concatenated eigenvectors. Perform a linear transformation to map it to a lower-dimensional space (and (With the same dimensions), it generates initial logits (logic values) for the subsequent Sigmoid function. The purpose of this matrix is ​​to learn how to generate the optimal gating signal based on the current synthesized information.

[0096] It indicates that, at the time step The periodic feature vector. That is, the periodic position encoding discussed in detail above. .

[0097] This represents the final fused temporal feature vector, which is the final output of the entire time series embedding method and periodic feature encoder module.

[0098] This represents the weight matrix for linear transformation of periodic features. Since the dimension of the original periodic feature vector may not match the dimension of the multi-scale features, Responsible for Projected onto Being in the same feature space allows for direct element-wise operations between the two.

[0099] ⊙ represents the Hadamard product. The synergistic effect of the two techniques is achieved through a spatiotemporal feature fusion module, ultimately outputting a temporal representation that combines local details and global patterns, providing robust temporal dynamic features for downstream reinforcement learning decision-making.

[0100] 3) Spatiotemporal Fusion Representation Vector (STFR-Vector)

[0101] Given a road network spatiotemporal system, STFR-Vector is defined as a joint embedding representation of spatial graph features and time series features:

[0102]

[0103] in, This represents the final spatiotemporal fusion representation vector. The fusion function can be categorized into splicing, linear transformation projection, and nonlinear activation. Features of spatial maps For time series features, The dimensions after fusion (taken as 1024 dimensions), where Features of spatial graphs Dimensions Time series features Dimensions.

[0104] Spatial graph feature generation The improved Graph Attention Network v2 (GATv2) is used for computation:

[0105] Node feature update formula:

[0106]

[0107] in, This represents vector concatenation. For the number of attention heads, As edge features, For dynamic attention coefficients, Indicates the first The trainable weight matrix corresponding to each attention head is a linear transformation matrix, the purpose of which is to transform the concatenated high-order features. Project it into a new feature space that is better suited for computational attention.

[0108] Represents a node The initial node feature vector. Represents a node The set of neighboring nodes.

[0109] Attention coefficient calculation:

[0110]

[0111] in, The learnable parameter matrix; Input node features and Dimensions. The dimension of the key and query vector for the attention mechanism is a hyperparameter, typically set to be greater than the dimension of the key and query vector. Small, used to scale the dot product and prevent gradient vanishing.

[0112] Time feature generation Calculated using a space-time gated fusion unit (STGFU):

[0113] Bidirectional GRU:

[0114]

[0115]

[0116]

[0117] in, For the forward GRU at time step The hidden state. It encodes the time series from its start to the current time. Forward information. For backward GRU at time step The hidden state. It encodes the time from the end of the time series to the current time. The back information. This represents the final hidden state after layer normalization, combining forward and backward information. It is a time series... A comprehensive representation of. These are the parameters of the GRU network, including weights and biases. For forward and backward GRUs, the parameters are independent. These are the parameters of the GRU network, including weights and biases. For forward and backward GRUs, the parameters are independent. , These are learnable weight matrices used to linearly combine the forward and backward hidden states. Their dimensions typically match the dimensions of the hidden states.

[0118] Enhanced time attention:

[0119]

[0120] in, For the time feature vector, the hidden states at all time steps are used. application The result is obtained by weighted summation following an attention mechanism. It captures globally important features of the time series, highlighting the impact of key time points. Here... The hidden state is applied to each time step, and then a weighted sum is performed, therefore It is a vector of fixed dimensions.

[0121] Feature fusion and standardization:

[0122] Fusion formula:

[0123]

[0124] in, ; ; To initially fuse feature vectors, spatial features and time characteristics It is obtained by concatenating and then performing a linear transformation. It is an intermediate representation of STFR-Vector. This is the projection weight matrix, used to weight the concatenated features (dimensions). Mapping to the target dimension . This is the bias vector, used to adjust the offset of the fusion result.

[0125] Layer normalization:

[0126]

[0127] in, For the mean and standard deviation, These are learnable parameters;

[0128] The mathematical property of scale invariance is:

[0129]

[0130] in, The input feature matrix includes the original features for all nodes and time points. It represents the input space of the model.

[0131] Here, is the Lipschitz constant. This inequality shows that the output variation of the STFR-Vector is constrained by the input variation, with the output norm not exceeding 1.25 times the input norm. This ensures the model's robustness to input perturbations and aids in generalization. Differential properties:

[0132]

[0133] in, Weight matrix The former Rows correspond to spatial features The part. Because of the splicing operation. middle, front Vis is ,so yes A submatrix with dimension .

[0134] Feature decoupling (orthogonality constraint):

[0135]

[0136] computational complexity (Linearly scalable).

[0137] 2. Deep reinforcement learning algorithms for large model enhancement

[0138] 1) State Representation Enhancement Module

[0139] Large Language Model (LLM) feature distillation enhances the semantic representation of traffic states through knowledge transfer. Its core principle can be described as: combining the spatiotemporal fusion representation vector (STFR-Vector) with historical state memory. After concatenation, the pre-trained large language model with frozen parameters is input to obtain preliminary enhanced representations:

[0140]

[0141] in, This represents the initial enhanced state representation. It combines the spatiotemporal fusion representation vector (STFR-Vector) with historical state memory. After concatenation, the pre-trained LLM with frozen parameters is input, and then the vector obtained by layer normalization (LayerNorm) has a dimension of [missing information]. . This represents a vector concatenation operation. Achieve feature standardization. This represents a pre-trained large language model with frozen parameters. During training, the model's weights are not updated and are only used for forward propagation to extract general semantic features. Indicates at time step The environmental conditions, specifically traffic condition information. Historical state memory. This refers to the hidden state or internal representation of the model at the previous time step t-1. It is a vector that encapsulates all the historical traffic state information that the model has "remembered" up to time t-1. The concatenation method involves joining these two vectors end-to-end along the feature dimension to form a longer vector, mathematically represented as... The new vector after splicing has a dimension of 1024.

[0142] To maintain the general knowledge of the large model while adapting to traffic control tasks, a low-rank adaptive (LoRA) technique is used to fine-tune the LLM output, with the parameter update format as follows: ,in, This represents the amount of parameter update when fine-tuning the LLM output using the Low Rank Adaptive (LoRA) technique. ∈ and For trainable low-rank matrices (default rank) ), original parameters Keep frozen so that the final state is represented While preserving the general semantic understanding capabilities of the language model, a dynamic gating mechanism is used. To achieve adaptive enhancement of features in the transportation domain, forming a hybrid state representation with multi-scale perception capabilities. , where ⊙ represents element-wise multiplication. It is the sigmoid activation function. This represents the weight matrix used to transform the STFR-Vector in the dynamic gating mechanism. The gating mechanism calculates and then mixes the state representations, thus... Projected onto The same feature space. This process uses KL divergence loss. To achieve knowledge distillation, in which, In KL divergence loss, represents the total number of time steps, i.e., the length of the training sequence (the number of steps in an episode). This represents the Kullback-Leibler divergence, used to measure the difference between two probability distributions. Here, Calculate the KL divergence between the LLM output distribution and the DRL model output distribution. This represents a Deep Reinforcement Learning (DRL) model (i.e., the currently trained model). In knowledge distillation, the goal of the DRL model is to mimic the output distribution of an LLM model, supplemented by a feature alignment loss. To maintain consistency in the potential space, where, In feature alignment loss In this context, is a trade-off hyperparameter used to control the importance of feature alignment loss. and These represent the latent space feature representations of the LLM and DRL models, respectively. These are the output feature vectors of the intermediate layers of the model. This represents the Frobenius norm. For a matrix... Its Frobenius norm is:

[0143]

[0144] 2) Hybrid Strategy Optimization Algorithm

[0145] The objective function is PPO-Clip, which is enhanced by STFR-Vector:

[0146]

[0147] in, This represents the pruning objective function of the PPO algorithm, used to optimize the policy parameters. The goal is to maximize expected reward while limiting the magnitude of policy updates. Indicates the desired operation, based on time steps. The empirical samples (from the empirical playback buffer). t Indicates the time step index. This represents a clipping function that restricts values ​​to a range. Internal. Used to prevent importance sampling ratio. Excessive size leads to unstable strategy updates.

[0148] Importance sampling ratio ,in, The parameter is The policy function outputs the probability distribution of actions. The parameters represent the old strategy, the strategy used during the sampling action (fixed from previous strategy versions). This indicates the action selected at time step t. : Indicates the state at time step t.

[0149] This is the clipping threshold.

[0150] Advantage estimation Calculated using STFR-enhanced GAE:

[0151]

[0152] in, , is the bias-variance tradeoff coefficient; In advantage estimation, the discount factor ( ), used to calculate the present value of future rewards. In the summation of the advantage estimation, represents the index variable, from 0 to Tt, used to calculate the contribution of future time steps. The parameter is The value function is used to estimate the value of a state. Here, the value estimation is based on enhanced state representation. Hierarchical action space design:

[0153] Macroscopic layer action space: outputs continuous action vectors The control area traffic allocation is constrained by the Tanh activation function, and its policy distribution is as follows:

[0154]

[0155] in, This represents the macro-level strategy and outputs the probability distribution of macro-level actions. K This represents the number of dimensions of macro-level actions, i.e., the number of actions related to regional flow allocation. Represents a normal distribution with a mean of 1 / 2. The variance is Microscopic layer action space: Output discrete actions Select the signal phase combination and use the Gumbel-Softmax approximation:

[0156]

[0157] in, This represents the micro-level strategy, outputting the probability distribution of discrete actions (signal phase combinations). Gumbel-Softmax is used to approximate discrete sampling. This represents the Gumbel-Softmax function, used for approximating sampling of discrete actions, where... It is a temperature parameter (which controls the smoothness of the distribution). This represents a multilayer perceptron, used to process input states and output action probabilities. This is the temperature coefficient.

[0158] 3) Safety constraint handling

[0159] Construct the Lagrange relaxation optimization objective:

[0160]

[0161] in, This represents an optimization objective function with a penalty term, combining the PPO objective with penalties for constraint violations. Represents the Lagrange multiplier, used to weigh the first... The penalty strength for each constraint violation. These multipliers are updated via gradient ascent. Indicates the first A constraint function measures the degree to which the state-action pair (s,a) violates the constraints. Indicates the first The threshold of a constraint, i.e., the maximum allowed violation value.

[0162] Key constraints include:

[0163] Minimum green light time constraint: ,in, This indicates an indicator function (exponential function) that returns 1 if the condition is true, and 0 otherwise. This indicates the current green light duration (in seconds), and is a status. Part of it.

[0164] Phase conflict constraint: ,in, The phase conflict matrix is ​​a binary matrix where each element represents the conflict relationship between signal phases (if the phases are in conflict). and If there is a conflict, the matrix element is 1, which is used to check whether the action has selected a conflict phase. This indicates microscopic-level action, i.e., the selected combination of signal phases. This represents the L1 norm, which is the sum of the absolute values ​​of the vector elements and is used to calculate the degree of violation of conflicting constraints.

[0165] Flow balancing constraints: ,in, This represents the k-th component of the macro-level action, i.e., the regional flow allocation value (continuous value). This represents the L2 norm, the square root of the sum of the squares of the vector elements, used to calculate the degree of violation of flow balance constraints. Lagrange multipliers. Update via gradient ascent:

[0166]

[0167] 4) Training Process

[0168] A double-buffering mechanism is used for alternating updates: the policy network is updated by performing minibatch SGD every 128 time steps; the value network is updated using Huber loss; and a constraint threshold is applied. Dynamic adjustment:

[0169]

[0170] in, This is the smoothing coefficient.

[0171] 3. Traffic Control Knowledge Distillation Framework

[0172] 1) Cross-modal knowledge transfer module

[0173] State characterization distillation employs the feature space of a KL divergence-aligned large language model (LLM) and a signal-controlled encoder:

[0174]

[0175] Temperature coefficient Dynamic adjustment:

[0176]

[0177]

[0178] Policy distillation constrains the distribution of teacher and student policies through mean squared error:

[0179]

[0180] Employing reparameterization techniques to ensure gradient differentiability:

[0181]

[0182] 2) Progressive learning strategy

[0183] The training process is divided into three stages, and the parameter configurations for each stage are shown in Table 1:

[0184] Table 1 Parameter Configuration for Each Stage

[0185]

[0186] The phase transition conditions are determined by the following indicators:

[0187]

[0188]

[0189] in, These represent the mean and standard deviation of the rewards for the current stage. This represents the expected value of the reward (average reward), which is the average cumulative reward that the model obtains in the current stage. The variance of the reward represents the degree of fluctuation in the reward and measures the stability of the strategy.

[0190] 3) Dynamic weight adjustment mechanism

[0191] Knowledge distillation loss weight Adaptive adjustment as training progresses:

[0192]

[0193] in, To increase the number of training steps Knowledge distillation loss weights at different times. This is the minimum value of the knowledge distillation loss weight, used to reduce the impact of distillation in the later stages of training. This represents the maximum value of the knowledge distillation loss weight, used to emphasize distillation in the early stages of training.

[0194] Constraints:

[0195]

[0196] in, This is the total loss function, used to optimize model parameters. To reinforce the learning loss function.

[0197] Simulation verification:

[0198] Testing and validation on real datasets revealed that the key parameter configurations are shown in Table 2 below:

[0199] Table 2 Key Parameter Configuration

[0200]

[0201] Table 3 Comparison of performance indicators of the core confluence zone

[0202]

[0203] Table 4 Comparison of Overall Road Section Performance Indicators

[0204]

[0205] In Tables 3 and 4, the no-control method is the baseline scenario; the PER-DDQN variable rate limiting control method uses a deep dual-Q network with priority experience replay for variable rate limiting control, abbreviated as PER-DDQN-VSL; the DDPG dynamic variable rate limiting control method uses a deep deterministic policy gradient for dynamic variable rate limiting control, abbreviated as DDPG-DVSL; the PER-DDQN dynamic variable rate limiting control method is the method of this invention, which combines PER-DDQN and dynamic variable rate limiting control, abbreviated as PER-DDQN-DVSL.

[0206] The data in Tables 3 and 4 clearly show that, in all test scenarios, the method of this invention achieved the best performance in reducing travel time and increasing average speed, thus verifying its effectiveness.

[0207] In summary, this invention proposes an intelligent traffic management method that integrates multimodal road network perception with large-model-enhanced deep reinforcement learning decision-making. Traffic management encompasses signal control at urban road intersections, highway entrance ramp control, and emergency lane opening control. Based on this, a complete technical system is constructed, including spatiotemporal fusion representation encoding, hierarchical policy optimization, and knowledge distillation collaborative training, thereby significantly improving the real-time performance, safety, and generalization ability of large-scale road network traffic management. The multimodal road network information perception encoding system achieves high-precision spatiotemporal representation of traffic states through dynamic graph attention mechanisms and multi-scale temporal modeling; the large-model-enhanced deep reinforcement learning algorithm utilizes the semantic understanding capabilities of pre-trained language models, combined with hierarchical action space design and safety constraint optimization, to improve decision-making quality in complex scenarios; the traffic management knowledge distillation framework achieves efficient transfer of traffic domain knowledge through cross-modal alignment loss and progressive course learning. This solution improves the generalization performance and accuracy of control strategies while ensuring real-time response speed.

[0208] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for large-scale road network traffic management based on a deep reinforcement learning model, characterized in that, The method includes the following steps: S1: Construct a multimodal road network information perception and coding system: perceive real-time multimodal road network information including urban road intersections, highway entrance ramps and emergency lanes, and generate a spatiotemporal fusion representation vector representing the current traffic network state by fusing spatial graph construction methods and time series embedding methods; S2: Deep reinforcement learning algorithm for large model enhancement: The spatiotemporal fusion representation vector and historical state memory are concatenated as input, and the state feature is distilled using the backbone network of the pre-trained large language model to enhance the state representation. Traffic control decisions are output through a policy network with a hierarchical action space. The specific method for performing state feature distillation using the backbone network of a pre-trained large language model is as follows: A pre-trained large language model backbone network with frozen parameters is used as a feature extractor. The spatiotemporal fusion representation vector and historical state memory are concatenated as input to obtain a preliminary enhanced representation. A trainable low-rank adaptive adapter is used to fine-tune the output of the large language model to adapt to specific traffic control tasks. S3: Application of the traffic control knowledge distillation framework: Through cross-modal knowledge transfer and progressive course learning strategies, the training process of the deep reinforcement learning algorithm is guided and optimized to improve model training efficiency and generalization ability.

2. The large-scale road network traffic control method according to claim 1, characterized in that, In step S1, generating the spatiotemporal fusion representation vector specifically includes: Spatial graph construction method: Define the road network as a topology graph , where the node set , Indicates a signal-controlled intersection; edge set , Representing the connectivity of road segments, weight matrix , Represents the weight of the edge. , The number of intersections is represented; the edge weights are calculated by fusing static topological features based on Gaussian distance kernels with dynamic flow features based on real-time queue lengths; the topological graph is processed using an improved graph attention network v2 to generate spatial features; Time series embedding method: Multi-scale sliding window sampling is used to sample the time series data of each node to obtain multi-scale features, and periodic features are extracted by combining periodic feature encoder; periodic features and multi-scale features are fused by spatiotemporal gating fusion unit to generate time series features; Spatial features and temporal features are fused to obtain the final spatiotemporal fusion representation vector.

3. The large-scale road network traffic control method according to claim 2, characterized in that, In step S1, the spatial graph construction method utilizes an improved graph attention network v2 to process the topological graph and generate spatial features. Specifically, this includes: introducing a multi-scale adjacency matrix construction method and defining... k adjacency matrix To capture k-hop neighborhood relationships; in, For threshold-based The binary function; for of The power is used to calculate the relationship between nodes. The sum of the weights of the skip paths, The exponentiation operation extends the direct adjacency relationship of nodes to... Jumping to distant neighboring regions; When dealing with large sparse road networks, a hierarchical graph construction method based on road network partitioning is adopted to construct the global topology graph. Decomposed into multiple subgraphs ,in Indicates the first c The set of intersection nodes contained within each subgraph Indicates the first c The set of edges of a subgraph This represents the number of subgraphs, where subgraphs are separated by boundary nodes. Connect and define weights across subgraphs. , Subgraph m and n Euclidean distance between the centroids Indicates the strength of cross-regional association; parameters Indicates the global scaling factor or baseline connection strength for cross-region associations; In terms of dynamic traffic feature extraction, each node Associated multidimensional feature vectors : Among them, flow characteristics Includes the current queue length and rate of change ; velocity characteristics Including average speed and standard deviation ; Market share characteristics , Lane occupancy rate; Phase characteristics The current phase state of one-hot encoded signal control for urban road intersections, highway entrance ramp control, and emergency lane opening control.

4. The large-scale road network traffic control method according to claim 1, characterized in that, In step S1, the time series embedding method uses multi-scale sliding window sampling to capture multi-granularity features of traffic flow by constructing a hierarchical time-aware mechanism and adopting a three-level sliding window configuration; each window is processed by an independent temporal convolution module; finally, the multi-scale features are formed by channel splicing. The periodic feature encoder explicitly models the inherent periodicity of traffic flow, specifically by using a bi-periodic embedding function based on harmonic extension terms to extract periodic features.

5. The large-scale road network traffic control method according to claim 1, characterized in that, In step S2, the hierarchical action space includes: Macro-level action space: used to output continuous actions and control the macro-level flow distribution between regions; Micro-level action space: used to output discrete actions, selecting specific signal phase combinations, ramp control rates, or emergency lane opening states for individual traffic control points.

6. The large-scale road network traffic control method according to claim 1, characterized in that, In step S2, the deep reinforcement learning algorithm is trained using an improved proximal policy optimization algorithm, and a constraint optimization loss function based on the Lagrange multiplier method is designed to enforce traffic safety constraints during the optimization process.

7. The large-scale road network traffic control method according to claim 6, characterized in that, The traffic safety constraints include: minimum green light time constraint, signal phase conflict constraint, and traffic flow balance constraint.

8. The large-scale road network traffic control method according to claim 1, characterized in that, In step S3, the traffic control knowledge distillation framework specifically includes: Cross-modal knowledge transfer module: KL divergence loss is used for state representation distillation to align the feature spaces of the large language model and the traffic state encoder; mean squared error loss is used for policy distillation to constrain the policy distribution of the teacher and student models; Progressive learning strategy: The model training process is divided into multiple stages from easy to difficult, including single intersection control stage, arterial road coordinated control stage and regional road network collaborative control stage, and the stage transition is automatically triggered according to the preset reward performance index.

9. A large-scale road network traffic control system applicable to the method described in any one of claims 1-2 and 4-8, characterized in that, The system includes: The multimodal road network information perception and coding module is configured to perceive real-time multimodal road network information and generate a spatiotemporal fusion representation vector that represents the current traffic network state by fusing spatial graph construction methods and time series embedding methods. Specifically, the generation of the spatiotemporal fusion representation vector involves: using an improved graph attention network v2 to process the road network spatial graph that fuses static topology and dynamic traffic features to generate spatial features; using a spatiotemporal gating fusion unit configured with a multi-scale sliding window to process traffic time series data to generate time series features; and fusing the spatial features and time series features into a spatiotemporal fusion representation vector. In the spatial graph construction method, the road network is defined as a topology graph. , where the node set , Indicates a signal-controlled intersection; edge set , Representing the connectivity of road segments, weight matrix , Represents the weight of the edge. , Indicates the number of intersections; This paper utilizes an improved graph attention network v2 to process topological graphs and generate spatial features. Specifically, it introduces a multi-scale adjacency matrix construction method and defines... k adjacency matrix To capture k-hop neighborhood relationships; in, For threshold-based The binary function; for of The power is used to calculate the relationship between nodes. The sum of the weights of the skip paths, The exponentiation operation extends the direct adjacency relationship of nodes to... Jumping to distant neighboring regions; When dealing with large sparse road networks, a hierarchical graph construction method based on road network partitioning is adopted to construct the global topology graph. Decomposed into multiple subgraphs ,in Indicates the first c The set of intersection nodes contained within each subgraph Indicates the first c The set of edges of a subgraph This represents the number of subgraphs, where subgraphs are separated by boundary nodes. Connect and define weights across subgraphs. , Subgraph m and n Euclidean distance between the centroids Indicates the strength of cross-regional association; parameters Indicates the global scaling factor or baseline connection strength for cross-region associations; In terms of dynamic traffic feature extraction, each node Associated multidimensional feature vectors : Among them, flow characteristics Includes the current queue length and rate of change ; velocity characteristics Including average speed and standard deviation ; Market share characteristics , Lane occupancy rate; Phase characteristics The current phase state of one-hot encoding for signal control at urban road intersections, highway entrance ramp control, and emergency lane opening control; The large model-enhanced deep reinforcement learning decision module is configured to receive the spatiotemporal fusion representation vector, enhance the state representation using the backbone network of a pre-trained large language model and a low-rank adaptive adapter, and output traffic control decisions through a policy network with a hierarchical action space. The traffic control knowledge distillation training module is configured to train and optimize the deep reinforcement learning decision module through cross-modal knowledge transfer and progressive course learning strategies.

Images