Multi-intersection traffic signal cooperative control method driven by cross attention neural network

Abstract

The invention discloses a multi-intersection traffic signal cooperative control method driven by a cross attention neural network, and the method employs a local cooperative Transform architecture, integrates a decision converter and a shared memory mechanism, and achieves the efficient modeling of a space-time dependence relation of a multi-intersection traffic state. The method comprises the following steps: firstly, through a memory head module, extracting a hidden state of each agent in a sequence modeling process, and updating global shared memory for supporting information interaction and strategy collaboration among multiple agents; and then, a cross attention module is adopted to carry out cross calculation on the local representation and the shared memory of each agent, so that dynamic perception and efficient modeling of the global state of the traffic system are realized. A backbone network of the model is based on Transform, and the understanding ability of time and space traffic characteristics is enhanced through position coding, self-attention and cross attention mechanisms. In the fine tuning stage, only the inserted Adapter module and the output layer are subjected to parameter updating.

Description

Technical Field

[0001] The present invention relates to the fields of traffic signal control and reinforcement learning, and in particular to an intelligent traffic signal control method based on deep reinforcement learning. Background Art

[0002] Traffic signal control is a key component of intelligent transportation systems, playing a crucial role in alleviating urban traffic congestion, improving road efficiency, and reducing carbon emissions. However, traffic signal control remains a challenging task, as urban traffic flow is influenced by multiple factors, including road structure, driving habits, and weather changes, and exhibits a high degree of spatiotemporal dynamics. Traditional traffic signal control methods primarily include fixed-time sequence-based control methods, adaptive control methods, and optimization control methods. For example, methods such as the Webster method, the Sydney Coordinated Adaptive Traffic System (SCATS), and the Split Cycle Offset Optimization Technique (SCOOT) rely on pre-defined traffic flow patterns and manually designed signal control schemes. However, these methods have limitations in coping with complex and dynamically changing traffic environments, making it difficult to adapt to unexpected traffic conditions and optimize global traffic flow.

[0003] In recent years, with the development of reinforcement learning and deep reinforcement learning technologies, reinforcement learning-based traffic signal control methods have gradually attracted the attention of researchers. Reinforcement learning methods can learn optimal signal control strategies through interaction with the environment, without the need for a pre-defined traffic model. They can adaptively adjust traffic signals to optimize key indicators such as vehicle waiting time, queue length, and road network throughput. Deep reinforcement learning further enhances reinforcement learning's modeling capabilities in complex environments, enabling intelligent traffic signal control systems to make autonomous decisions using high-dimensional traffic data (such as video streams and vehicle trajectories), learn nonlinear spatiotemporal features, and improve the adaptability and robustness of control strategies.

[0004] A traffic signal control method based on deep reinforcement learning (DL) aims to combine the adaptive learning capabilities of RL with the high-dimensional data processing capabilities of DL to address the shortcomings of traditional methods in complex traffic environments. By leveraging technologies such as Multi-Agent Reinforcement Learning (MARL) and Graph Neural Networks (GNNs), it can further enhance the coordination between different traffic lights and optimize global traffic flow. This method has broad application prospects in intelligent traffic management, smart city development, and green mobility, providing a new solution for improving the intelligence of transportation systems and optimizing urban transportation. Summary of the Invention

[0005] The present invention is an intelligent traffic signal control method based on deep reinforcement learning, which is applied to intelligent transportation systems.

[0006] Adaptively adjusting traffic signals to minimize overall traffic delay has always been a challenging task. Traditional approaches rely on pre-defined traffic flow patterns and manually designed signal control schemes, which present limitations. To overcome these limitations, recent research has focused on collaborative traffic signal control methods based on deep reinforcement learning.

[0007] This paper proposes a local collaborative Transformer module that learns from a well-trained teacher model, enabling a lightweight controller to run efficiently on devices with limited computational resources.

[0008] Secondly, in order to improve the applicability of the local collaborative Transformer module in complex traffic signal control problems, the present invention further combines offline reinforcement learning (Offline RL) and online fine-tuning strategies.

[0009] In the offline phase, the local collaborative Transformer module establishes a preliminary signal control strategy by learning from historical traffic data, thereby reducing the demand for environmental interaction data.

[0010] In the online stage, the local collaborative Transformer module only needs to adjust the parameters of the adapter module without completely fine-tuning the entire model, which significantly improves the computational efficiency of the model.

[0011] Finally, through the shared memory mechanism, multiple agents can share global information in the decision-making process, thereby improving collaboration efficiency and reducing conflicts. This method has broad application prospects in the fields of intelligent traffic management, urban traffic optimization, and dynamic signal control, and provides a new solution for improving the intelligence level of the transportation system and optimizing urban road traffic.

[0012] The present invention's intelligent traffic signal control method based on deep reinforcement learning innovates and improves upon traditional methods. By introducing a local collaborative Transformer structure and a shared memory mechanism, the present invention can comprehensively utilize historical traffic status and global traffic information to enhance the decision-making capability and adaptability of signal control. The present invention can better capture traffic flow changes, congestion patterns, and signal optimization strategies, and extract key spatiotemporal features. By combining the shared memory mechanism with self-attention and cross-attention mechanisms, the spatiotemporal dynamics of traffic flow can be more efficiently modeled.

[0013] The intelligent traffic signal control method of the present invention has broad application prospects in the fields of intelligent traffic management, urban road optimization and dynamic signal control, and has brought important impetus to the research and application of related fields.

[0014] This paper divides the entire deep reinforcement learning training process into four stages: offline data collection, teacher model training, knowledge distillation, and online fine-tuning. This phased training process aims to improve the model's learning efficiency, generalization ability, and practical deployment effect.

[0015] During the offline data collection phase, historical traffic data is used to model intersection traffic flow patterns, extract key features (such as traffic volume, speed, queue length, and other traffic trajectory data), and construct a training dataset to ensure that the model can learn effective signal control strategies from real-world environments.

[0016] During the teacher model training phase, a deep reinforcement learning (DRL) teacher model is trained based on complete historical data, enabling it to learn the optimal signal control strategy to maximize traffic flow efficiency. During training, model parameters and reward functions are optimized to improve the rationality and generalization of the strategy.

[0017] During the knowledge distillation phase, the knowledge from the teacher model is transferred to a lightweight student model, enabling it to operate in a computationally constrained environment while maintaining efficient signal control capabilities. This process reduces model complexity and improves deployment efficiency.

[0018] During the online fine-tuning phase, the student model is adaptively optimized based on a small amount of real-time interaction data to adapt to the dynamic traffic environment. By fine-tuning some parameters, the model maintains stable and efficient control capabilities under different traffic patterns.

[0019] After each round of model training, the model interacts with the environment. The key metric for evaluating model performance is the latency of all vehicles. By evaluating the average latency of vehicles in different traffic scenarios, we can determine the effectiveness of the model's optimization in real-world applications and further adjust parameters to ensure stability and efficiency in complex traffic environments.

[0020] This paper presents an intelligent traffic signal control method based on deep reinforcement learning. This method proposes a local collaborative Transformer module that integrates a decision transformer with a shared memory mechanism to efficiently model the spatiotemporal dependencies of traffic states at multiple intersections.

[0021] First, by introducing the memory head module, the hidden state of each agent in the sequence modeling process is extracted, and the global shared memory is updated to support information interaction and strategy coordination among multiple agents.

[0022] Subsequently, the Cross-Attention module is used to cross-calculate the local representation of each agent with the shared memory to achieve dynamic perception and efficient modeling of the global state of the traffic system.

[0023] In the model architecture, the backbone network is built on the Transformer architecture, leveraging positional encoding, self-attention, and cross-attention mechanisms to enhance its understanding of temporal and spatial traffic characteristics. During model fine-tuning, only the parameters of the inserted adapter module and output layer need to be updated, eliminating the need to adjust the backbone model. This significantly reduces the computational overhead required for online training.

[0024] During the evaluation process, the model interacts with the SUMO simulation environment, measures the control effect through indicators such as average delay, and provides real-time feedback on the current traffic status to guide signal decisions.

[0025] In practical applications, this paper adopts a three-stage training process of "offline pre-training + knowledge distillation + online fine-tuning":

[0026] First, the teacher model is trained offline using historical traffic trajectory data to build a high-performance policy network;

[0027] Subsequently, through knowledge distillation technology, the strategic capabilities of the teacher model are transferred to the lightweight student model to improve deployment efficiency;

[0028] Finally, before deployment or during actual operation, the Adapter module in the model is fine-tuned through limited online interaction data to adapt to real-time traffic dynamics, without the need for large-scale online training data or backbone structure updates.

[0029] The present invention can be flexibly applied in traffic networks of different scales and complexities, has excellent generalization ability and stability, can effectively improve the coordinated signal control effect of multiple intersections in cities, and provide an efficient and scalable solution for intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a schematic diagram of the multi-head attention module;

[0031] Figure 2 is a schematic diagram of the cross attention module;

[0032] Figure 3 This is a schematic diagram of the Transformer Block architecture;

[0033] Figure 4 This is a schematic diagram of the overall architecture of the local collaborative Transformer;

[0034] Figure 5 It is a flow chart of a method for coordinated control of traffic signals at multiple intersections according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below and illustrated with reference to the accompanying drawings. It should be understood that all other embodiments that can be obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] The present invention provides a traffic signal collaborative control method based on a deep reinforcement learning model.

[0037] refer to Figures 1 to 4 The implementation steps of the cross-attention neural network driven multi-intersection traffic signal coordinated control method of the present invention are as follows:

[0038] Step 1. To meet the simulation requirements for reinforcement learning training of traffic signal control, a high-fidelity simulation environment based on real traffic data was constructed. This environment uses SUMO as a traffic simulation tool and supports a variety of traffic signal control scenarios, including synthetic and real-world scenarios, including single-intersection and multi-intersection scenarios.

[0039] Step 2. The local collaborative Transformer can learn (traffic light control) strategies directly from offline data. To support its strategy training, a variety of traffic signal control strategies are used to generate offline data, including rule-based methods, single-agent reinforcement learning methods, and multi-agent reinforcement learning methods to generate trajectory data. in It represents the queue length, number of approaching vehicles, and cumulative waiting time data of parked vehicles at a time step i; Indicates the signal light phase to be selected at the next moment; It represents the total reward from the current time step to the end of the trajectory; Indicates the number of past time slices to support the strategy training of the local collaborative Transformer module in different scenarios.

[0040] Step 3. Figure 4 As shown in Figure 2, the overall architecture design of the local collaborative Transformer combines local trajectory modeling with a local neighbor shared memory mechanism. Specifically:

[0041] Single agent The input is its trajectory data and its local neighbors' shared state ,in represents the high-order representation of the neighbors to their own trajectory data, t-1 represents the number of time slices in the previous time step. In a multi-agent environment, the number of neighbors of different agents varies, resulting in the hidden state collected from the neighbors. Inconsistent dimensions. Use the Memory Head module to uniformly expand or compress to the same hidden dimension. , ensuring the consistency of the shared memory dimensions. The formula is as follows:

[0042]

[0043] in It is the linear mapping matrix in the Memory Head, which is processed by 4 layers of Transformer Block. Each Block contains two attention modules: self-attention and cross-attention. Each TransformerBlock contains a self-attention module and a cross-attention module. The self-attention module is used to process the agent A. i Self-trajectory data , the cross attention module is used to process the neighbor agent A 1:n The hidden state The hidden states of all neighboring agents before the cross attention is passed to the agent At time step Action decision The output of the shared memory with local neighbors will be comprehensively considered.

[0044] Step 4. Add the offline dataset obtained in step 2 to include the trajectory sequence , these sequences are used to train the teacher model of the local collaborative Transformer, such as Figure 3 As shown, in each Transformer Block, the input of each agent is calculated by multi-head attention to obtain a representation vector that generates context-awareness. ,in represents a high-level representation of its own trajectory data, Sharing memories with neighbors As input, it undergoes cross-attention, then passes through a feedforward network with normalization and residual connections, and the final output serves as a deep semantic representation for downstream policy learning tasks.

[0045] Here is a detailed introduction to the Multi-Head Attention module and the Cross-Attention module in the Transformer Block structure.

[0046] Process 1: Figure 1 The multi-head attention module shown, the trajectory data extracted from the offline dataset Embedding is required to embed these different modal data into a unified high-dimensional representation space, and the converted tokens are stacked according to the trajectory time step. ,in Representation tensor after trajectory embedding , represents the number of time steps, Represents the unified high-dimensional space dimension mapped to, and the embedded representation tensor As the input of the multi-head attention module to extract its global context representation, and enter the residual and normalization module, the final output is represented as The query input for Cross-Attention is as follows:

[0047]

[0048] Process 2: Figure 2 The cross attention module shown uses the output of multi-head attention in process 1 Sharing memories with neighbors As input through cross-attention, all agents can share the key information of neighbors in a partially observable environment. When calculating Cross-Attention, the output of each agent's multi-head attention As a query, neighbors share memory As the key and value, the attention fusion vector is calculated by the neighbor shared memory After the feedforward network and residual and normalized superposition transformation, the final output is a shape of Feature Sequence , the formula is as follows:

[0049]

[0050] in is the key vector dimension in Attention, .

[0051] Step 5. The teacher model trained in step 4 is used in the offline track The output generated above and student models , are normalized by the temperature coefficient T to obtain the soft target probability distribution t and the student output distribution s. On this basis, the Kullback-Leibler divergence is used as the distillation loss function , measure the distribution difference between student strategy and teacher strategy, introduce negative log-likelihood supervision loss , the final training target is a weighted combination of the two. The formula is as follows:

[0052]

[0053] in Exponential function.

[0054] Step 6. After completing the knowledge distillation in step 5, the lightweight student model obtained by the local collaborative Transformer module is used as the basic network structure and fine-tuned online before actual deployment. The fine-tuning stage adopts the efficient parameter fine-tuning method based on the Adapter module, which only updates the Adapter module and output layer after each layer of Transformer Block, keeping the pre-trained backbone parameters unchanged. First, the student model is set in the current environment with the strategy Interact with the traffic environment to sample new complete trajectories , and calculate the corresponding true return sequence to obtain a new RTG (Return-to-Go) label. Then, replace this trajectory with the trajectory with the smallest return in the offline dataset to construct an updated online training set , randomly sample subtrajectories from The compact COMPACTER++ Adapter module is inserted after the feedforward layer of each Transformer Block as training input for the online fine-tuning phase. This module consists of a down-projection layer using low-rank hypercomplex parameterized multiplication (LPHM), a nonlinear activation (GeLU), and an up-projection layer. Gradient updates are performed only on the Adapter module and the output action mapping layer.

[0055] Step 7. After completing the online fine-tuning in Step 6, the local collaborative Transformer module fixes the updated model parameters for the current iteration and continuously interacts with the traffic simulation environment (SUMO) using only the current policy to evaluate the model's decision-making performance under real-world traffic dynamics. During this interaction, the system receives state input from the environment at each time step (such as queue lengths, wait times, and signal phases at each intersection) and feeds this into the policy model. Through forward propagation through the Transformer backbone and the Adapter module, the policy model generates the action selection for each traffic light, i.e., the control signal phase decision at the current moment. The generated action is transmitted to the simulation environment through an interface, driving the next step in the evolution of the traffic system state. Corresponding rewards and environmental feedback are generated, including vehicle traffic conditions, updated state observations, and overall road network operational indicators. At the end of the simulation episode, the system saves the travel information of each vehicle in the traffic network.

[0056] Step 8. Select the optimization algorithm: Use the Adam optimizer to update the model parameters. During the training process, set different iterations, batch sizes, and temperature factors for the pre-training stage (teacher model), distillation stage (student model), and online fine-tuning stage (Adapter module) to control the softmax smoothness. Adjust the loss weight parameters, including the distillation loss weight α and the entropy term weight, through experience or random search. and RTG (Return-to-Go) scaling factor.

[0057] Step 9. Train the model: The training process consists of three stages: (1) Pre-training the teacher model using offline trajectories to optimize policy prediction capabilities; (2) Using a knowledge distillation strategy, the student model is trained based on the soft labels of the teacher model; (3) The student model is fine-tuned online, updating only the Adapter module in the Transformer to adapt to the target environment. During training, the model generates actions through forward propagation, calculates the loss, and updates the parameters through backpropagation and the optimizer until the set number of training steps or the stopping condition is reached.

[0058] Step 10. Performance Evaluation: Run multiple rounds of tests using a standard simulation environment (such as SUMO) to evaluate the model's performance in different scenarios based on traffic control metrics such as average delay. After each round of online fine-tuning, the model undergoes a complete evaluation to determine its adaptability and convergence performance.

[0059] Step 11. Model Testing and Deployment: After obtaining the optimal model parameters, the local collaborative Transformer module can be deployed in a real-world traffic control system. Thanks to its lightweight design and low computational requirements, the fine-tuned model can be directly deployed on edge devices or city traffic signal controllers. Depending on actual needs, the model is deployed in a production environment, whether on-premises, on a server, or on a cloud platform. Considerations regarding model performance, reliability, and security should be addressed during deployment.

[0060] The present invention is further described as follows:

[0061] In step 3, a hidden state representation mechanism and shared memory structure are introduced to enhance the model's ability to model temporal information between multiple agents and global traffic conditions. During the offline data training phase, each agent processes its local trajectory sequence using the Transformer's multi-head self-attention mechanism, extracting temporal features representing its current state, action, and target reward (RTG). A hidden state vector is generated at each time step. This hidden state not only encodes the agent's historical observation information but is also further converted into the agent's local memory representation via a memory projection module (memory head). The local memories of all agents are aggregated into a global shared memory structure (SharedMemory) to support subsequent information collaboration and state sharing among multiple agents. This mechanism allows each agent to maintain its individual behavior while simultaneously perceiving the global state of the entire traffic network, thereby improving collaborative control in multi-intersection environments.

[0062] In step 4, the Cross-Attention mechanism is introduced to enable each agent to globally perceive and respond to shared memory. At each decision, each agent uses its own hidden state as a query vector (Query), while the shared memory from other agents is input into the Cross-Attention layer as a key / value pair. Through this mechanism, agents can dynamically adjust their attention to information at other intersections, thereby indirectly obtaining traffic information from neighboring or even remote intersections while only being able to observe local states, enhancing the adaptability and coordination of the strategy. The introduction of cross-attention significantly improves the model's decision-making ability in partially observable multi-agent scenarios, enabling it to effectively capture cross-intersection traffic dependencies and the need for coordinated signal adjustments, ultimately achieving the goals of reducing overall vehicle delays and improving road traffic efficiency.

[0063] In summary, by introducing hidden state representation and shared memory mechanisms into the multi-agent architecture and incorporating the Cross-Attention module into the decision-making process, the local collaborative Transformer module enables global modeling and dynamic perception of traffic conditions at multiple intersections. Compared to traditional control methods based on local observations, this mechanism not only significantly enhances the information coordination capabilities between multi-agents but also improves the accuracy of modeling temporal dependencies and intersection correlations in complex traffic environments, thereby providing a more efficient, stable, and generalizable control strategy for intelligent traffic signal control tasks.

[0064] In step 3, a hidden state representation mechanism and a shared memory structure module are introduced to enhance the modeling and memory updating capabilities of temporal features in multi-agent traffic signal control. In steps 5 and 6, the local collaborative Transformer module adopts the base Transformer structure, combined with a multi-head attention mechanism and a memory head module, to model the historical state, action, and reward sequence of each agent in the traffic environment. The introduction of the memory head enables each agent to generate a compressed representation of its current hidden state at each time step when processing local observations and write it to the global shared memory, providing a foundation for subsequent information collaboration among multiple agents. This mechanism strengthens the model's ability to represent the evolution of traffic states, enabling it to extract key state evolution patterns and perform policy optimization.

[0065] The local collaborative Transformer module further enhances state perception and decision-making capabilities in multi-intersection coordinated control by integrating cross-attention and shared memory mechanisms. The cross-attention module uses the shared memory output by the memoryhead as the key / value pair and the current hidden state of each agent as the query. It dynamically calculates the correlation between the global traffic state and the current decision, thereby guiding the model to automatically focus on the states of upstream and downstream intersections closely related to the current intersection. This mechanism effectively integrates local and global information flows, enabling agents to maintain a global perspective even in partially observable environments, improving state perception and coordination efficiency, and enhancing the understanding and modeling of the spatiotemporal dynamics of traffic systems.

[0066] Finally, in step 7, the local collaborative Transformer module embeds the memory head and adapter modules into each layer of the Transformer Block, constructing a spatiotemporal modeling network structure with online adaptability. During the training phase, the model uses offline data to initialize the policy, trains a lightweight student model through knowledge distillation, and updates the adapter module parameters using a small amount of online interaction data during the deployment phase, allowing it to quickly adapt to changes in the actual traffic environment. In each decision, the model generates traffic signal control actions based on the input state, shared memory, and the current policy network, driving the evolution of the traffic system. Combined with an appropriate loss function and optimization algorithm, the model demonstrates good performance and generalization capabilities during training and inference.

[0067] In summary, the local collaborative Transformer module, by introducing a memory head and shared memory mechanism, combines the temporal modeling capabilities of the Transformer with the global perception capabilities of Cross-Attention, demonstrating remarkable innovation and practicality in multi-agent traffic signal control tasks. This approach not only enhances the collaborative capabilities between agents but also optimizes the overall operational performance of the traffic system while maintaining computational efficiency, providing strong technical support for the practical deployment of intelligent transportation systems.

[0068] refer to Figure 5 , this embodiment includes the following steps:

[0069] Step 101. Collect and construct an offline trajectory dataset (DTRL) for multi-intersection traffic signal control based on the SUMO simulation platform. The dataset contains a sequence of states (including queue length, wait time, number of approaching vehicles, etc.) at each time step, along with corresponding actions and rewards. The trajectory data is organized in chronological order, processed for missing values, and normalized by calculating the mean and standard deviation of each signal state.

[0070] Step 102. Following the model design process (corresponding to steps 1 to 6 above), a complete local collaborative Transformer module was developed using the Python programming language and the PyTorch framework. It includes a teacher model (Teacher), a student model (Student), and an adapter module (Adapter).

[0071] The model uses a GPT2-based Decision Transformer architecture and customizes the input embedding and action prediction structures for TSC (Time Series Classification) scenarios. Furthermore, the COMPACTER++ Adapter module is introduced to enable efficient online fine-tuning and reduce the number of model parameters.

[0072] Step 103. To train the model and evaluate its traffic control performance, a composite loss function is defined.

[0073] When training the teacher model, a negative log-likelihood loss (Cross-Entropy Loss) based on action prediction is used, and a policy entropy term is introduced for exploration regularization. The student model uses a joint loss function in the distillation stage:

[0074]

[0075] in, The distillation loss (KL divergence form) calculated by the soft target generated by the teacher model, is the negative log-likelihood loss of the student model to the ground truth.

[0076] Step 104. The Adam optimizer is used during model training, combined with a learning rate scheduler to adapt to the convergence speed and stability requirements of different stages (teacher pre-training, student distillation, and online fine-tuning). In the parameter space search, the training effect is improved by adjusting temperature parameters (such as the Softmax temperature T and the policy entropy weight λ) and hyperparameters such as the learning rate and batch size.

[0077] Step 105. During training, the model samples subtrajectories from offline trajectories and performs multiple rounds of forward and backward propagation updates to gradually optimize the policy network parameters. During the online fine-tuning phase, only the adapter module and output layer weights are updated, while the main Transformer remains frozen. This enables efficient online learning and rapid adaptation, improving generalization to new traffic conditions.

[0078] Step 106. After each round of training, use the evaluation environment to simulate the current model and calculate the average latency as the core performance indicator to evaluate its control effect. If the model verification performance does not meet expectations, start the hyperparameter tuning process (Step 107); if multiple rounds of tuning still cannot meet the requirements, proceed to the model structure adjustment process (Step 108).

[0079] Step 107. Hyperparameter tuning uses random search to cover learning rate, distillation loss weight α, RTG scaling factor ( 、 ), Adapter module structure, etc. Different parameter combinations have a significant impact on the model's convergence speed and final performance. Reasonable configuration can significantly reduce training time and improve control effectiveness. After tuning, return to step 105 for retraining.

[0080] Step 108. If the model performance still does not meet the requirements, adjust structural parameters such as the number of Transformer layers, the number of attention heads, and the embedding dimension, or replace the adapter module (for example, switching from COMPACTER++ to Houlsby or LoRA). You can also try modifying the reward function form, state input method, or introducing additional neighbor information. After completing the structural modification, re-enter step 102 to reconstruct and train the model.

[0081] Step 109. After the model passes evaluation, further analyze its computing resource consumption. For time complexity, measure the time cost of training and inference; for space complexity, measure the model parameter size and memory usage. Compare the number of parameters and training time of different models (e.g., Teacher, Student, and Adapter) in a table to clarify the deployment advantages of the local collaborative Transformer module and the Adapter model in embedded devices or edge computing environments.

[0082] The embodiment described above is merely a preferred embodiment of the present invention and does not limit the present invention in any way. Other variations and modifications are possible without departing from the technical solutions described in the claims. Such variations and modifications still fall within the scope of the present invention.

Claims

1. A cross-attention neural network driven multi-intersection traffic signal coordinated control method, characterized by In the intelligent transportation system, the phase control of each intersection signal light is implemented by a local control model. The local control model uses local traffic trajectory data as input to obtain a local phase control strategy. A teacher model is deployed on the server side of the intelligent transportation system, and a student model is deployed on each local controller as the control model. The teacher model, each student model, and the adaptation module together form a local collaborative Transformer architecture. The teacher model is first trained offline using historical traffic data. Then, knowledge distillation is used to transfer the teacher model's knowledge to the student model. Finally, during the online fine-tuning phase of the student model, the adapter module is fine-tuned to adapt to the dynamic traffic environment on site. The multi-intersection traffic signal coordinated control method comprises the following steps: Step 101. Data preparation Collect and construct an offline trajectory dataset for traffic signal control at multiple intersections; The dataset contains the state sequence, action, and reward information corresponding to each time step; the trajectory data is organized in chronological order, and has been processed with missing values ​​and state normalization; Step 102. Build the model The teacher model consists of a decision transformer and a Transformer-based backbone network. Offline trajectory data is organized in chronological order. After being processed by the decision transformer, the backbone network determines the future signal phase. The backbone network consists of multiple layers of Transformer Blocks, each of which contains a multi-head attention module and a cross-attention module. In a distributed environment with multiple teacher models, the multi-head attention module is used to process the teacher model's own trajectory data, while the cross-attention module is used to process the hidden states of neighboring teacher models. Step 103. Define the loss function The loss function is a composite loss function; When training the teacher model, the negative log-likelihood loss Cross-Entropy Loss based on action prediction is adopted, and the policy entropy term is introduced; The student model uses a joint loss function in the distillation stage: ,in, is the distillation loss calculated by the soft target generated by the teacher model, is the negative log-likelihood loss of the student model to the ground truth; α is the distillation loss weight, and β is the negative log-likelihood loss weight; Step 104. Select the optimization function The Adam optimizer is used in the model training process, combined with a learning rate scheduler to meet the convergence speed and stability requirements of the teacher pre-training, student distillation, and online fine-tuning stages. In parameter space search, improve training results by adjusting parameters; Step 105. Train the model During the offline training of the teacher model, the teacher model samples sub-trajectories from the offline trajectory, performs multiple rounds of forward propagation and back-propagation updates, and gradually optimizes the parameters of the policy network; During the online fine-tuning phase of the student model, only the Adapter module and output layer weights are updated, while the Transformer body remains frozen; Step 106. Performance Evaluation After completing each round of training, the current model is simulated and tested using the evaluation environment. The average delay of the vehicle in different traffic scenarios is calculated as a performance indicator to evaluate the control effect of the control model.

2. The method for coordinated control of multi-intersection traffic signals driven by a cross-attention neural network according to claim 1 is characterized in that The teacher model is an agent, in a multi-agent environment: During the offline data training phase, each agent processes its own local trajectory sequence through the Transformer's multi-head self-attention mechanism, extracting temporal features representing the current state, action, and target reward, and generating a hidden state vector at each time step. This hidden state encodes the agent's historical observation information and is further converted into the agent's local memory representation through the memory projection module memoryhead. The local memories of all agents will be aggregated to form a global shared memory structure called Shared Memory to support information collaboration and state sharing among multiple agents.

3. The method for coordinated control of multi-intersection traffic signals driven by a cross-attention neural network according to claim 1 is characterized in that In step 101, the state sequence includes queue length, waiting time, and number of approaching vehicles; Offline trajectory data is organized in chronological order and undergoes missing value processing and state normalization. The normalization method is to calculate the mean and standard deviation of the corresponding state of each traffic light.

4. The cross-attention neural network driven multi-intersection traffic signal coordinated control method according to claim 1 is characterized by: In step 104, in the parameter space search, the training effect is improved by adjusting the temperature parameter and the learning rate and batch size hyperparameters; the temperature parameter includes the Softmax temperature T and / or the weight λ of the policy entropy.

5. The method for coordinated control of multi-intersection traffic signals driven by a cross-attention neural network according to claim 1 is characterized in that In step 106, If the control effect meets expectations, proceed to step 109 to evaluate the resource consumption of the model; If the control effect does not meet expectations, then proceed to step 107 to perform hyperparameter tuning. If the control effect still does not meet expectations after multiple rounds of hyperparameter tuning, then proceed to step 108 to adjust the model structure. Step 107. Adjust hyperparameters Hyperparameter tuning uses random search, including learning rate, distillation loss weight α, RTG scaling factor, and Adapter module structure; Different parameter combinations have a significant impact on the model's convergence speed and final performance. Reasonable configuration can significantly reduce training time and improve control effects. After tuning, return to step 105 for retraining; Step 108. Improve the model structure Ways to improve the model structure include: Adjust the parameters of the Transformer layers, number of attention heads, and / or embedding dimensions, or Replace the adapter module, or Modify the reward function form, state input method and / or introduce additional neighbor information; After the structure is modified, re-enter step 102; Step 109. Resource consumption assessment In terms of time complexity, it measures the time cost required for training and inference; In terms of space complexity, it measures the model parameter size and memory usage; By comparing the number of parameters and training time of different models, the deployment advantages of the local collaborative Transformer module and the Adapter model in embedded devices or edge computing environments are clarified.

6. The method for coordinated control of multi-intersection traffic signals driven by a cross-attention neural network according to claim 2 is characterized in that Offline trajectory data The status It represents the queue length, number of approaching vehicles, and cumulative waiting time of parked vehicles at a time step i; action Indicates the signal light phase to be selected at the next moment; reward It represents the total reward from the current time step to the end of the trajectory; Indicates the number of time slices that have passed.

7. The method for coordinated control of multi-intersection traffic signals driven by a cross-attention neural network according to claim 6 is characterized in that In a multi-agent environment, a single agent The input is its trajectory data and its local neighbors' shared state ,in It represents the high-order representation of the neighbor's own trajectory data, and t-1 represents the time slice number of the previous time step; Hidden states collected by a single agent from its neighbors The dimensions are uniformly expanded or compressed to the same hidden dimension through the Memory Head module , so that the dimensions of shared memory are consistent, the formula is: ,in is the linear mapping matrix in Memory Head; Processed by multiple layers of Transformer Block; The multi-head attention module in each layer of Transformer Block processes agent A i Self-trajectory data , the cross attention module is used to process the neighbor agent A 1:n The hidden state ; The hidden states of all neighboring agents are fed into the cross-attention module before the agent At time step Action decision Comprehensively consider its historical trajectory representation and the output of the shared memory with local neighbors; In each layer of Transformer Block, the input of each agent is calculated by the multi-head attention module to generate a context-aware representation vector ,in A high-level representation of its own trajectory data; Sharing memories with neighbors As input, it passes through the cross-attention module, then through the feedforward network with normalization and residual connection, and the final output serves as the deep semantic representation of the downstream strategy learning task.

8. The method for coordinated control of multi-intersection traffic signals driven by a cross-attention neural network according to claim 7 is characterized in that In the multi-head attention module, trajectory data First, perform the embedding operation to embed the input data of different modalities into a unified high-dimensional representation space; The converted tokens are stacked according to the trajectory time step. ,in Representation tensor after trajectory embedding , represents the number of time steps, Represents the unified high-dimensional space dimension mapped to; Embedded representation tensor As the input of the multi-head attention module to extract The global context is represented and enters the residual and normalization modules, and the final output is represented as , used for the query input of the cross attention module, the formula is: ， In the cross attention module, the output of the multi-head attention module is used Sharing memories with neighbors As input, after calculation by the cross attention module, all agents share the key information of their neighbors in a partially observable environment. When the cross attention module is calculated, the output of the multi-head attention module of each agent As a query, the neighbors share memory As the key Key and value Value, the attention fusion vector is calculated by the neighbor shared memory ; After the feedforward network and residual and normalized superposition transformation, the final output is a shape of Feature Sequence , the formula is: ,in is the key vector dimension in Attention, .

9. The cross-attention neural network driven multi-intersection traffic signal coordinated control method according to claim 8 is characterized by: In knowledge distillation, the trained teacher model is used in the offline track The output generated above And the output of the student model After normalization with temperature coefficient T, the soft target probability distribution t and student output distribution s are obtained; Then, Kullback-Leibler divergence is used as the distillation loss function , measure the distribution difference between student strategy and teacher strategy, introduce negative log-likelihood supervision loss , the final training target is a weighted combination of the two; the formula is: in is an exponential function.

10. The cross-attention neural network driven multi-intersection traffic signal coordinated control method according to claim 9 is characterized in that After knowledge distillation is completed, the student model is used as the basic network structure and fine-tuned online before actual deployment; The fine-tuning phase uses a parameter fine-tuning method based on the Adapter module. Only the Adapter module and output layer after each Transformer Block are updated, keeping the pre-trained backbone parameters unchanged. The steps are as follows: First, the student model is strategically placed in the current environment. Interact with the traffic environment to sample new complete trajectories , and calculate the corresponding real return sequence to obtain a new RTG (Return-to-Go) label; Next, the trajectory Replace it with the trajectory with the smallest return in the offline dataset to construct an updated online training set , randomly sample subtrajectories from , used as training input for the online fine-tuning phase; The Adapter module is inserted after the feed-forward layer of each Transformer Block.

Images