Cross-attention neural network-driven multi-intersection traffic signal cooperative control method

Abstract

A cross-attention neural network-driven collaborative traffic signal control method for multiple intersections employs a local collaborative Transformer architecture, integrating a decision transformer and a shared memory mechanism to efficiently model the spatiotemporal dependencies of traffic states at multiple intersections. First, a memory head module extracts the hidden states of each agent during sequence modeling and updates the global shared memory to support information interaction and policy collaboration among multiple agents. Then, a cross-attention module cross-computes the local representations of each agent with the shared memory, achieving dynamic perception and efficient modeling of the global state of the traffic system. The backbone network of the model is based on Transformer, enhancing its understanding of temporal and spatial traffic characteristics through position encoding, self-attention, and cross-attention mechanisms. During the fine-tuning phase, only the inserted Adapter module and the output layer are updated.

Description

Technical Field

[0001] This invention relates to the fields of traffic signal control and reinforcement learning, and specifically to an intelligent traffic signal control method based on deep reinforcement learning. Background Technology

[0002] Traffic signal control is a crucial component of intelligent transportation systems, playing a vital role in alleviating urban traffic congestion, improving road efficiency, and reducing carbon emissions. However, traffic signal control remains a challenging task because urban traffic flow is influenced by various factors such as road structure, driving habits, and weather changes, exhibiting a high degree of spatiotemporal dynamics. Traditional traffic signal control methods mainly include fixed-time control methods, adaptive control methods, and optimization control methods. For example, methods such as the Webster method, SCATS (Sydney Coordinated Adaptive Traffic System), and SCOOT (Split Cycle Offset Optimization Technique) rely on preset traffic flow patterns and manually designed signal control schemes. However, these methods have limitations in dealing with complex and dynamically changing traffic environments, struggling to adapt to sudden traffic situations and optimize global traffic flow.

[0003] In recent years, with the development of reinforcement learning and deep reinforcement learning technologies, traffic signal control methods based on reinforcement learning have gradually attracted researchers' attention. Reinforcement learning methods can learn optimal signal control strategies through interaction with the environment, without the need for pre-set traffic models. They can adaptively adjust traffic signals to optimize key indicators such as vehicle waiting time, queue length, and network throughput. Deep reinforcement learning further enhances the modeling capabilities of reinforcement learning in complex environments, enabling intelligent traffic signal control systems to make autonomous decisions using high-dimensional traffic data (such as video streams and vehicle trajectories) and learn nonlinear spatiotemporal characteristics, thereby improving the adaptability and robustness of control strategies.

[0004] The proposed traffic signal control method based on deep reinforcement learning aims to combine the adaptive learning capabilities of reinforcement learning with the high-dimensional data processing capabilities of deep learning to improve the shortcomings of traditional methods in complex traffic environments. By employing techniques such as Multi-Agent Reinforcement Learning (MARL) and Graph Neural Networks (GNNs), the cooperation between different traffic lights can be further enhanced, optimizing global traffic flow. This type of method has broad application prospects in intelligent traffic management, smart city construction, and green mobility, providing new solutions for improving the intelligence level of transportation systems and optimizing urban traffic. Summary of the Invention

[0005] This invention is an intelligent traffic signal control method based on deep reinforcement learning, which is applied to intelligent transportation systems.

[0006] Adaptively adjusting traffic signals to minimize overall traffic flow delays remains a challenging task in current technologies. Traditional solutions rely on preset traffic flow patterns and manually designed signal control schemes, which have limitations. To overcome these limitations, recent research trends have focused on traffic signal cooperative control methods based on deep reinforcement learning.

[0007] This invention proposes a locally cooperative Transformer module. By learning from a well-trained teacher model, a lightweight controller can operate efficiently on computationally limited devices.

[0008] Secondly, in order to improve the applicability of the local cooperative Transformer module in complex traffic signal control problems, this invention further combines offline reinforcement learning (Offline RL) and online fine-tuning strategies.

[0009] During the offline phase, the local collaborative Transformer module learns from historical traffic data to establish a preliminary signal control strategy, thereby reducing the need for environmental interaction data.

[0010] During the online phase, the local collaborative Transformer module only needs to adjust the parameters of the adapter module, without having to fine-tune the entire model, which significantly improves the computational efficiency of the model.

[0011] Finally, through the shared memory mechanism, multiple agents can share global information during the decision-making process, thereby improving collaboration efficiency and reducing conflicts. This method has broad application prospects in fields such as intelligent traffic management, urban traffic optimization, and dynamic signal control, providing a new solution for improving the intelligence level of traffic systems and optimizing urban road traffic.

[0012] This invention presents an intelligent traffic signal control method based on deep reinforcement learning, which innovates and improves upon traditional methods. By introducing a local cooperative Transformer structure and a shared memory mechanism, this invention can comprehensively utilize historical traffic conditions and global traffic information, enhancing the decision-making ability and adaptability of signal control. This invention can better capture traffic flow changes, congestion patterns, and signal optimization strategies, and extract key spatiotemporal features. Through the combination of the shared memory mechanism with self-attention and cross-attention mechanisms, the spatiotemporal dynamic changes of traffic flow can be modeled more efficiently.

[0013] The intelligent traffic signal control method of the present invention has broad application prospects in fields such as intelligent traffic management, urban road optimization and dynamic signal control, and has brought important impetus to the research and application in related fields.

[0014] This invention divides the entire deep reinforcement learning training process into four stages: offline data collection, teacher model training, knowledge distillation, and online fine-tuning. This phased training process aims to improve the model's learning efficiency, generalization ability, and practical deployment performance.

[0015] During the offline data collection phase, historical traffic data is used to model the traffic flow patterns at intersections and extract key features (such as traffic flow, vehicle speed, queue length, and other traffic trajectory data) to build a training dataset, ensuring that the model can learn effective signal control strategies from the real environment.

[0016] During the teacher model training phase, a deep reinforcement learning (DRL) teacher model is trained based on complete historical data to learn the optimal signal control strategy to maximize traffic flow efficiency. During training, the rationality and generalization ability of the strategy are improved by optimizing model parameters and the reward function.

[0017] In the knowledge distillation phase, knowledge from the teacher model is transferred to the lightweight student model, enabling it to operate in computationally limited environments while maintaining efficient signal control capabilities. This process reduces model complexity and improves deployment efficiency.

[0018] During the online fine-tuning phase, the student model undergoes adaptive optimization based on a small amount of real-time interactive data to adapt to the dynamic traffic environment. By fine-tuning some parameters, the model is ensured to maintain stable and efficient control capabilities under different traffic patterns.

[0019] After each round of model training, the model interacts with the environment. The key metric for evaluating model performance is the latency of all vehicles. By assessing the average latency of vehicles under different traffic scenarios, we can determine the optimization effect of the model in practical applications and further adjust the parameters to ensure its stability and efficiency in complex traffic environments.

[0020] This invention discloses an intelligent traffic signal control method based on deep reinforcement learning. The method proposes a local cooperative Transformer module, which integrates a decision transformer and a shared memory mechanism to efficiently model the spatiotemporal dependencies of traffic states at multiple intersections.

[0021] First, by introducing the memory head module, the hidden state of each agent in the sequence modeling process is extracted and the global shared memory is updated to support information interaction and policy coordination among multiple agents.

[0022] Subsequently, a Cross-Attention module is used to cross-compute the local representation of each agent with the shared memory, thereby achieving dynamic perception and efficient modeling of the global state of the traffic system.

[0023] In the model structure, the backbone network is built based on Transformer, and enhances the understanding of temporal and spatial traffic features through positional encoding, self-attention, and cross-attention mechanisms. During the model fine-tuning stage, only the parameters of the inserted Adapter module and output layer need to be updated, without adjusting the backbone model, which significantly reduces the computational overhead required for online training.

[0024] During the evaluation process, the model interacts with the SUMO simulation environment, measures the control effect through indicators such as average delay, and provides real-time feedback on the current traffic status to guide signal decisions.

[0025] In practical applications, this invention adopts a three-stage training process of "offline pre-training + knowledge distillation + online fine-tuning":

[0026] First, the teacher model is trained offline using historical traffic trajectory data to build a high-performance policy network;

[0027] Subsequently, through knowledge distillation technology, the strategic capabilities of the teacher model are transferred to the lightweight student model to improve deployment efficiency;

[0028] Finally, before deployment or during actual operation, the Adapter module in the model is fine-tuned using limited online interactive data to adapt to real-time traffic dynamics without the need for large-scale online training data or backbone structure updates.

[0029] This invention can be flexibly applied in traffic networks of varying sizes and complexities, possesses excellent generalization ability and stability, and can effectively improve the collaborative signal control effect of multiple intersections in cities, providing an efficient and scalable solution for intelligent transportation systems. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of a multi-head attention module;

[0031] Figure 2 This is a schematic diagram of the cross-attention module;

[0032] Figure 3 This is a schematic diagram of the Transformer Block architecture;

[0033] Figure 4 This is a schematic diagram of the overall architecture of the locally collaborative Transformer;

[0034] Figure 5 This is a flowchart of a multi-intersection traffic signal coordinated control method according to an embodiment of the present invention. Detailed Implementation

[0035] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below, and will be illustrated with reference to the accompanying drawings. It should be understood that all other embodiments that can be obtained by those skilled in the art without inventive effort are within the protection scope of the present invention.

[0036] This invention presents a traffic signal cooperative control method based on a deep reinforcement learning model.

[0037] refer to Figures 1-4 The implementation steps of the cross-attention neural network-driven multi-intersection traffic signal cooperative control method of the present invention are as follows:

[0038] Step 1. To meet the requirements of traffic signal control reinforcement learning training for the simulation environment, a high-fidelity simulation environment based on real traffic data was constructed. This environment uses SUMO as the traffic simulation tool and supports a variety of different traffic signal control scenarios, covering synthetic and real scenarios, including single-intersection scenarios and multi-intersection scenarios.

[0039] Step 2. The Local Cooperative Transformer can directly learn (traffic light control) strategies from offline data. To support its strategy training, various traffic signal control strategies are used to generate offline data, including rule-based methods, single-agent reinforcement learning-based methods, and multi-agent reinforcement learning methods, to generate trajectory data. in This represents the queue length, number of approaching vehicles, and cumulative waiting time of parked vehicles at time step i. This indicates the traffic light phase to be selected at the next moment; This represents the total reward from the current time step to the end of the trajectory; This indicates the number of past time slices, to support policy training of the local collaborative Transformer module in different scenarios.

[0040] Step 3. (As shown) Figure 4 As shown, the overall architecture design of the Local Cooperative Transformer combines local trajectory modeling with a local neighbor shared memory mechanism. Specifically:

[0041] Single intelligent agent The input is its trajectory data and its local neighbor shared state ,in This represents the higher-order representation of a neighbor's trajectory data, where t-1 represents the time slice number of the previous time step. In a multi-agent environment, the number of neighbors varies among different agents, leading to differences in the hidden states collected from neighbors. Inconsistent dimensions. This can be addressed by using the Memory Head module to uniformly expand or compress to the same hidden dimension. This ensures consistency in the dimensions of shared memory. The formula is shown below:

[0042]

[0043] in The linear mapping matrix in the Memory Head is processed through four Transformer Blocks. Each Block contains two attention modules: self-attention and cross-attention. The self-attention module is used to process agent A. i Self-trajectory data The cross-attention module is used to process neighbor agent A. 1:n Hidden state The hidden states of all neighboring agents are considered before cross-attention is received. At time step Action decision The output will take into account both its historical trajectory representation and the shared memory of its local neighbors.

[0044] Step 4. The offline dataset obtained in Step 2 contains trajectory sequences. These sequences were used to train a teacher model for a local collaborative Transformer, such as Figure 3 As shown, in each Transformer Block, the input of each agent is processed by multi-head attention to obtain a context-aware representation vector. ,in This represents a higher-order representation of its own trajectory data. Sharing memories with neighbors The input is processed through cross-attention, then through a feedforward network connected to normalization and residuals, and the final output serves as the deep semantic representation for the downstream policy learning task.

[0045] This section details the multi-head attention module and cross-attention module in the Transformer Block structure.

[0046] Process 1: As Figure 1 The multi-head attention module shown here extracts trajectory data from an offline dataset. Embedding is required to embed these different modalities of data into a unified high-dimensional representation space. The transformed tokens are then stacked according to the trajectory time steps. ,in Representation tensor after trajectory embedding , Indicates the number of time steps. The representation tensor after embedding represents the unified high-dimensional space dimension mapped to. As input to the multi-head attention module to extract its global context representation, and then fed into the residual and normalization module, the final output representation is... The query input used for Cross-Attention is shown in the following formula:

[0047]

[0048] Process 2: As Figure 2 The cross-attention module shown utilizes the output of multi-head attention from process 1. Sharing memories with neighbors As input, cross-attention allows all agents to share key information about their neighbors in a partially observable environment. During cross-attention computation, the output of each agent's multi-head attention... As a query, neighbors share memory. The attention fusion vector, calculated using shared neighbor memory, serves as both key and value. After a superposition transformation involving a feedforward network and residual normalization, the final output is a shape of... Feature sequence The formula is shown below:

[0049]

[0050] in The dimension of the key vector in Attention. .

[0051] Step 5. Analyze the teacher model trained in Step 4 offline. The output generated above and student model After normalization by the temperature coefficient T, the soft target probability distribution t and the student output distribution s are obtained. Based on this, the Kullback-Leibler divergence is used as the distillation loss function. To measure the distributional difference between student and teacher strategies, a negative log-likelihood supervision loss is introduced. The ultimate training objective is a weighted combination of the two. The formula is shown below:

[0052]

[0053] in Exponential function.

[0054] Step 6. After completing the knowledge distillation in Step 5, the lightweight student model obtained from the locally collaborative Transformer module is used as the basic network structure and fine-tuned online before actual deployment. The fine-tuning phase employs an efficient parameter fine-tuning method based on the Adapter module, updating only the Adapter module and output layer after each Transformer Block, while keeping the pre-trained backbone parameters unchanged. First, the student model is deployed in the current environment with a strategy... Interact with the traffic environment to sample new complete trajectories The corresponding true return sequence is calculated to obtain a new RTG (Return-to-Go) label. Then, this trajectory is replaced with the trajectory with the smallest return in the offline dataset to construct an updated online training set. Randomly sample sub-trajectories from them The training input is used for the online fine-tuning phase. After the feedforward layer of each Transformer Block, a compact COMPACTER++ Adapter module is inserted. This module consists of a downprojection layer with low-rank hypercomplex parameterized multiplication (LPHM), a nonlinear activation (GeLU), and an upprojection layer, and performs gradient updates only on the Adapter module and the output action map layer.

[0055] Step 7. After completing the online fine-tuning in Step 6, the Local Cooperative Transformer module fixes the model parameters updated in the current iteration and continuously interacts with the traffic simulation environment (SUMO) using only the current policy to evaluate the model's decision-making performance in real traffic dynamics. During the interaction, the system receives state input from the environment at each time step (such as queue length, waiting time, signal phase, etc. at each intersection) and inputs it into the policy model. Through forward propagation between the Transformer backbone and the Adapter module, it generates the action selection for each traffic light, i.e., the control signal phase decision at the current moment. The generated actions are transmitted to the simulation environment through the interface, driving the traffic system state to evolve to the next step, while generating corresponding rewards and environmental feedback, including vehicle traffic conditions, updated state observations, and overall road network operation indicators. After the entire simulation episode ends, the system will save the travel information of each vehicle in the traffic network.

[0056] Step 8. Select Optimization Algorithm: Update the model parameters using the Adam optimizer. During training, different iteration counts, batch sizes, and temperature factors are set for the pre-training phase (teacher model), distillation phase (student model), and online fine-tuning phase (Adapter module) to control softmax smoothness. Loss weight parameters, including distillation loss weight α and entropy term weight, are adjusted empirically or through random search. And the RTG (Return-to-Go) scaling factor.

[0057] Step 9. Training the Model: The training process includes the following three stages: (1) Pre-training the teacher model using offline trajectories to optimize policy prediction capabilities; (2) Using a knowledge distillation strategy, training the student model based on the soft labels of the teacher model; (3) Fine-tuning the student model online, updating only the Adapter module in the Transformer to adapt to the target environment. During training, the model generates actions through forward propagation, calculates the loss, and updates parameters through backpropagation and the optimizer until the set number of training steps or stopping condition is reached.

[0058] Step 10. Performance Evaluation: Run multiple rounds of tests using a standard simulation environment (such as SUMO) to evaluate the model's performance in different scenarios based on traffic control metrics such as average delay. After each round of online fine-tuning, the model undergoes a complete evaluation to determine its adaptability and convergence performance.

[0059] Step 11. Model Testing and Deployment: After obtaining the optimal model parameters, the Local Cooperative Transformer module can be deployed in a real traffic control system. Thanks to its lightweight design and low computational requirements, the fine-tuned model can be directly deployed on edge devices or city traffic signal controllers. Depending on actual needs, the model can be deployed to a production environment, such as a local environment, server, or cloud platform. During deployment, factors such as model performance, reliability, and security need to be considered.

[0060] The present invention is further described below:

[0061] Step 3 introduces a hidden state representation mechanism and a shared memory structure to enhance the model's ability to model temporal information among multiple agents and the global traffic state. During offline data training, each agent processes local trajectory sequences through the Transformer's multi-head self-attention mechanism, extracting temporal features representing the current state, action, and target reward (RTG), and generating a hidden state vector at each time step. This hidden state not only encodes the agent's historical observation information but is also further converted into the agent's local memory representation through a memory projection module. The local memories of all agents are aggregated to form a global shared memory structure to support subsequent information collaboration and state sharing among multiple agents. This mechanism allows each agent to perceive the global situation of the entire traffic network while retaining its individual behavioral patterns, thereby improving the collaborative control effect in multi-intersection environments.

[0062] In step 4, a Cross-Attention mechanism is introduced to enable agents to globally perceive and respond to shared memories. During each decision-making process, each agent uses its own hidden state as a query vector, while the shared memories from other agents are input as key / value pairs into the Cross-Attention layer. Through this mechanism, agents can dynamically adjust their attention to information from other intersections, thereby indirectly acquiring traffic information from neighboring or even distant intersections while only observing local states, enhancing the adaptability and coordination of the strategy. The introduction of Cross-Attention significantly improves the model's decision-making ability in partially observable scenarios involving multiple agents, enabling it to effectively capture cross-intersection traffic dependencies and collaborative signal adjustment needs, ultimately achieving the goals of reducing overall vehicle latency and improving road traffic efficiency.

[0063] In summary, by introducing hidden state representation and shared memory mechanisms into the multi-agent structure, and combining the Cross-Attention module with the decision-making process, the local collaborative Transformer module can achieve global modeling and dynamic perception of traffic states at multiple intersections. Compared with traditional control methods based on local observation, this mechanism not only significantly enhances the information collaboration capabilities among multiple agents, but also improves the modeling accuracy of temporal dependencies and intersection correlations in complex traffic environments. This results in a more efficient, stable, and generalizable control strategy for intelligent traffic signal control tasks.

[0064] Step 3 introduces a hidden state representation mechanism and a shared memory structure module to enhance the modeling and memory updating capabilities of temporal features in multi-agent traffic signal control. In steps 5 and 6, the local cooperative Transformer module adopts a base Transformer structure, combined with a multi-head attention mechanism and a memory head module, to model the historical states, actions, and reward sequences of each agent in the traffic environment. The introduction of the memory head enables each agent to generate a compressed representation of its current hidden state at each time step when processing local observations and write it into the global shared memory, providing a foundation for subsequent information collaboration among multiple agents. This mechanism strengthens the model's ability to represent the evolution of traffic states, enabling it to extract key state evolution patterns and optimize policies.

[0065] The Local Cooperative Transformer module further enhances state perception and decision-making capabilities under multi-intersection cooperative control by integrating Cross-Attention and shared memory mechanisms. The Cross-Attention module uses the shared memory output by the memoryhead as the key / value pair and the current hidden state of each agent as the query. It dynamically calculates the correlation between the global traffic state and the current decision, thereby guiding the model to automatically focus on the states of upstream and downstream intersections closely related to the current intersection. This mechanism effectively integrates local and global information flows, enabling the agent to maintain a global perspective even in partially observable environments, improving state perception and coordination efficiency, and strengthening the understanding and modeling of the spatiotemporal dynamics of the traffic system.

[0066] Finally, in step 7, the local collaborative Transformer module constructs a spatiotemporal modeling network structure with online adaptability by embedding the memory head and Adapter module into each Transformer Block. During the training phase, the model uses offline data for policy initialization, trains a lightweight student model through knowledge distillation, and updates the Adapter module parameters with a small amount of online interaction data during the deployment phase, thus quickly adapting to changes in the real traffic environment. In each decision, the model generates traffic signal control actions based on the input state, shared memory, and the current policy network, driving the traffic system forward. With appropriate loss functions and optimization algorithms, the model exhibits good performance and generalization ability during training and inference.

[0067] In summary, the local collaborative Transformer module, by introducing a memory head and a shared memory mechanism, combined with the temporal modeling capabilities of Transformer and the global perception capabilities of Cross-Attention, demonstrates outstanding innovation and practicality in multi-agent traffic signal control tasks. This method not only enhances the collaborative capabilities among agents but also optimizes the overall operational performance of the traffic system while maintaining computational efficiency, providing strong technical support for the practical deployment of intelligent transportation systems.

[0068] refer to Figure 5 This embodiment includes the following steps:

[0069] Step 101. Collect and construct an offline trajectory dataset (DTRL) for multi-intersection traffic signal control based on the SUMO simulation platform. The dataset contains the state sequence for each time step (including queue length, waiting time, number of approaching vehicles, etc.), corresponding actions, and reward information. The trajectory data is organized in chronological order and undergoes missing value processing and state normalization. The normalization method is to calculate the mean and standard deviation of the state corresponding to each traffic light.

[0070] Step 102. Following the model design process (corresponding to steps 1 to 6 above), a complete local collaborative Transformer module was developed using the Python programming language and the PyTorch framework. It includes a teacher model, a student model, and an adapter module.

[0071] The model architecture adopts a GPT2-based decision transformer, and the input embedding and action prediction structures are customized for the TSC (Time Series Classification) scenario. Simultaneously, the COMPACTER++ Adapter module is introduced to achieve efficient online fine-tuning capabilities and reduce the number of model parameters.

[0072] Step 103. To train the model and evaluate its traffic control performance, a composite loss function is defined.

[0073] The teacher model employs a negative log-likelihood loss (Cross-Entropy Loss) based on action prediction, and introduces a policy entropy term for exploratory regularization. The student model uses a joint loss function during the distillation phase.

[0074]

[0075] in, Distillation loss (KL divergence form) calculated from the soft target generated by the teacher model. The negative log-likelihood loss of the student model to the ground truth.

[0076] Step 104. The Adam optimizer was used during model training, combined with a learning rate scheduler, to adapt to the convergence speed and stability requirements of different stages (teacher pre-training, student distillation, and online fine-tuning). In the parameter space search, temperature parameters (such as the temperature T in Softmax and the weight λ of the policy entropy) and hyperparameters such as learning rate and batch size were adjusted to improve training performance.

[0077] Step 105. During training, the model samples sub-trajectories from offline trajectories, performs multiple rounds of forward and backward propagation updates, and gradually optimizes the parameters of the policy network. In the online fine-tuning phase, only the Adapter module and output layer weights are updated, while the Transformer body remains frozen to achieve efficient online learning and rapid adaptation, improving the generalization ability to new traffic states.

[0078] Step 106. After each round of training, use the evaluation environment to simulate and test the current model, calculate the average latency as the core performance indicator, and evaluate its control effect. If the model validation performance does not meet expectations, initiate the hyperparameter tuning process (Step 107); if multiple rounds of tuning still fail to meet the requirements, proceed to model structure adjustment (Step 108).

[0079] Step 107. Hyperparameter tuning uses a random search method, covering the learning rate, distillation loss weight α, and RTG scaling factor ( , The model includes modules such as the Adapter module. Different parameter combinations have a significant impact on the model's convergence speed and final performance; a reasonable configuration can significantly reduce training time and improve control performance. After optimization, return to step 105 for retraining.

[0080] Step 108. If the model performance still does not meet the requirements, you can adjust structural parameters such as the number of Transformer layers, the number of attention heads, and the embedding dimension, or replace the adaptation module (such as replacing COMPACTER++ with Houlsby, LoRA, etc.). You can also try modifying the reward function form, the state input method, or introducing additional neighbor information. After completing the structural modifications, re-enter step 102 to reconstruct and train the model.

[0081] Step 109. After the model passes evaluation, further analyze its computational resource consumption. In terms of time complexity, measure the time cost required for training and inference; in terms of space complexity, measure the model parameter size and memory usage. Compare the number of parameters and training time of different models (such as Teacher, Student, and Adapter) using a table to clarify the deployment advantages of the local collaborative Transformer module and Adapter model in embedded devices or edge computing environments.

[0082] The embodiments described above are merely preferred embodiments of the present invention and do not constitute any limitation on the invention. Other variations and modifications may exist without departing from the scope of the claims. These variations and modifications still fall within the scope of the present invention.

Claims

1. A multi-intersection traffic signal cooperative control method driven by a cross-attention neural network, characterized in that... In intelligent transportation systems, the phase control of traffic lights at each intersection is implemented by a local control model. The local control model takes local traffic trajectory data as input to obtain the local phase control strategy. A teacher model is deployed on the server side of the intelligent transportation system, and a student model is deployed as a control model on each local controller. The teacher model, each student model, and the adapter module together constitute a local collaborative Transformer architecture. First, the teacher model is trained offline using historical traffic data. Then, through knowledge distillation, the knowledge of the teacher model is transferred to the student model. Finally, in the online fine-tuning stage of the student model, the adapter module is fine-tuned to adapt to the dynamic traffic environment on site. The multi-intersection traffic signal coordinated control method includes the following steps: Step 101. Data Preparation Collect and construct an offline trajectory dataset for traffic signal control at multiple intersections; The dataset contains the state sequence, action, and corresponding reward information for each time step; the trajectory data is organized in chronological order and has undergone missing value processing and state normalization. Step 102. Building the Model The teacher model includes a decision transformer and a Transformer-based backbone network; offline trajectory data is organized in chronological order, and after being processed by the decision transformer, the offline trajectory data is then processed by the backbone network to determine the future traffic light phase; The backbone network consists of multiple Transformer Blocks, each containing a multi-head attention module and a cross-attention module. In a distributed environment with multiple teacher models, the multi-head attention module is used to process the trajectory data of the teacher model itself, and the cross-attention module is used to process the hidden states of the neighboring teacher models. Step 103. Define the loss function The loss function is a composite loss function; When training the teacher model, a negative log-likelihood loss based on action prediction (Cross-Entropy Loss) is used, and a policy entropy term is introduced. The student model uses a joint loss function during the distillation phase: ， in, The distillation loss is calculated from the soft target generated by the teacher model. α is the negative log-likelihood loss of the student model to the ground truth; α is the weight of the distillation loss and β is the weight of the negative log-likelihood loss. Step 104. Select the optimization function The Adam optimizer was used during model training, combined with a learning rate scheduler, to meet the requirements of convergence speed and stability during teacher pre-training, student distillation, and online fine-tuning stages. In parameter space search, training performance is improved by adjusting parameters; Step 105. Train the model During the offline training of the teacher model, the teacher model samples sub-trajectories from the offline trajectory, performs multiple rounds of forward and backward propagation updates, and gradually optimizes the parameters of the policy network. During the online fine-tuning phase of the student model, only the Adapter module and the output layer weights are updated, while the Transformer body remains frozen. Step 106. Performance Evaluation After each round of training, the current model is simulated and tested using an evaluation environment. The average delay of the vehicle under different traffic scenarios is calculated as a performance indicator to evaluate the control effect of the control model. In step 106, If the control effect meets expectations, proceed to step 109 to evaluate the model's resource consumption. If the control effect does not meet expectations, proceed to step 107 for hyperparameter tuning; if the control effect still does not meet expectations after multiple rounds of hyperparameter tuning, proceed to step 108 for model structure adjustment. Step 107. Adjust hyperparameters Hyperparameter tuning employs a random search approach, including learning rate, distillation loss weight α, RTG scaling factor, and Adapter module structure; Different parameter combinations have a significant impact on the model's convergence speed and final performance. A reasonable configuration can significantly reduce training time and improve control performance. After optimization, return to step 105 for retraining. Step 108. Improve the model structure Ways to improve the model structure include: Adjust the parameters for the number of Transformer layers, the number of attention heads, and / or the embedding dimension, or Replace the adapter module, or Modify the form of the reward function, the state input method, and / or introduce additional neighbor information; After the structure is modified, proceed to step 102 again; Step 109. Resource Consumption Assessment In terms of time complexity, it measures the time cost required for training and inference; In terms of space complexity, it measures the size of the model parameters and memory usage; By comparing the number of parameters and training time of different models, the deployment advantages of the local collaborative Transformer module and Adapter model in embedded devices or edge computing environments are clarified.

2. The multi-intersection traffic signal cooperative control method driven by a cross-attention neural network according to claim 1, characterized in that... The teacher model is an agent, and in a multi-agent environment: During the offline data training phase, each agent processes its own local trajectory sequence through the Transformer's multi-head self-attention mechanism, extracts temporal features representing the current state, action, and target reward, and generates a hidden state vector at each time step. This hidden state encodes the agent's historical observation information and is further converted into the agent's local memory representation through the memory projection module memoryhead. The local memories of all agents will be aggregated into a global shared memory structure to support information collaboration and state sharing among multiple agents.

3. The multi-intersection traffic signal cooperative control method driven by cross-attention neural network according to claim 1, characterized in that: In step 101, the state sequence includes queue length, waiting time, and number of approaching vehicles; The offline trajectory data is organized in chronological order and undergoes missing value processing and state normalization. The normalization process involves calculating the mean and standard deviation of the state corresponding to each traffic light.

4. The multi-intersection traffic signal cooperative control method driven by cross-attention neural network according to claim 1, characterized in that: In step 104, during the parameter space search, the training effect is improved by adjusting the temperature parameter, learning rate, and batch size hyperparameter; the temperature parameter includes the temperature coefficient δ of Softmax and / or the weight λ of policy entropy.

5. The multi-intersection traffic signal cooperative control method driven by cross-attention neural network according to claim 2, characterized in that... Offline trajectory data in state This represents the queue length, number of approaching vehicles, and cumulative waiting time of parked vehicles at time step i. action This indicates the traffic light phase to be selected at the next moment; award This represents the total reward from the current time step to the end of the trajectory; This indicates the number of time steps.

6. The multi-intersection traffic signal cooperative control method driven by cross-attention neural network according to claim 5, characterized in that... In a multi-agent environment, a single agent The input is its trajectory data and its local neighbor shared state ,in This represents a higher-order representation of the neighbor's trajectory data for itself. Indicates the time slice number of the previous time step; Hidden state collected by a single agent from its neighbors Dimensions are uniformly expanded or compressed to the same hidden dimension through the Memory Head module. To ensure consistency in the dimensions of shared memory, the formula is: ， in This is a linear mapping matrix in the Memory Head; Processed through multiple layers of Transformer Blocks; The multi-head attention module in each Transformer Block processes agent A. i Self-trajectory data The cross-attention module is used to process neighboring agents. Hidden state ; The hidden states of all neighboring agents are passed to the cross-attention module before being entered into the agent's state. At time step Action decision Taking into account both its historical trajectory representation and the output that shares memory with its local neighbors; In each Transformer Block, the input of each agent is processed by a multi-head attention module to obtain a context-aware representation vector. ,in A higher-order representation of its own trajectory data; Sharing memories with neighbors The input passes through a cross-attention module, then through a feedforward network connected to normalization and residuals, and the final output serves as the deep semantic representation for the downstream policy learning task.

7. The multi-intersection traffic signal cooperative control method driven by cross-attention neural network according to claim 6, characterized in that... In the multi-head attention module, trajectory data First, an embedding operation is performed to embed input data from different modalities into a unified high-dimensional representation space; The converted tokens are stacked according to the trajectory time steps. ,in, Representation tensor after trajectory embedding , Indicates the number of time steps. This represents the dimension of the unified high-dimensional space to which the mapping is performed; Embedded representation tensor As input to the multi-head attention module, to extract The global context is represented and then processed by the residual and normalization modules. The final output is represented as follows. The query input for the cross-attention module is: ， ， Am , ， In the cross-attention module, the output of the multi-head attention module is utilized. Sharing memories with neighbors As input, after calculation by the cross-attention module, all agents share key information about their neighbors in a partially observable environment. During the cross-attention module's calculation, the output of each agent's multi-head attention module is... As a query, while neighbors share memory. The attention fusion vector, calculated using neighbor shared memory, serves as both the key and value. After the feedforward network and residual normalization superposition transformation, the final output is a shape of Feature sequence , The formula is: ， ， ) ， in is the dimension of the key vector in Attention, and LayerNorm is the residual connection and layer normalization.

8. The multi-intersection traffic signal cooperative control method driven by cross-attention neural network according to claim 7, characterized in that... In knowledge distillation, the trained teacher model is used in offline trajectories. The output generated above and the output of the student model After normalization by the temperature coefficient δ, the soft target probability distribution φ and the student output distribution μ are obtained. Then, the Kullback-Leibler divergence was used as the distillation loss function. To measure the distributional difference between student and teacher strategies, negative log-likelihood supervision loss is introduced. The ultimate training objective is a weighted combination of the two; the formula is: ， ， ， ， in It is an exponential function.

9. The multi-intersection traffic signal cooperative control method driven by a cross-attention neural network according to claim 8, characterized in that: After knowledge distillation is completed, the student model is used as the basic network structure, and online fine-tuning is performed before actual deployment. The fine-tuning phase employs a parameter fine-tuning method based on the Adapter module, updating only the Adapter module after each Transformer Block and the output layer, while keeping the pre-trained backbone parameters unchanged. The steps are as follows: First, the student model is placed in the current environment with a strategy. Interact with the traffic environment to sample new complete trajectories And calculate the corresponding real return sequence to obtain the new Return-to-Go RTG label; Next, the trajectory Replace the trajectory with the one with the smallest return in the offline dataset to construct an updated online training set. Randomly sample sub-trajectories from them , used as training input during the online fine-tuning phase; The Adapter module is inserted after the feedforward layer of each Transformer Block.

Images