Traffic signal coordination control method based on deformable attention

Abstract

The application discloses a traffic signal cooperative control method based on a deformable attention, and steps are as follows: firstly, a shared Decision Transformer backbone network is used to time-series encode historical trajectories of each intersection to obtain a local hidden state sequence; on this basis, a local three-dimensional space-time candidate node set composed of multi-hop candidate intersections and time lag layers is constructed for each decision intersection; further, a fixed number of three-dimensional space-time reference anchor points are constructed, and a restricted offset is generated by using the local hidden state of the current intersection, so that the reference anchor points can be adaptively moved in the three-dimensional space-time sampling field; subsequently, on the local three-dimensional space-time graph composed of the multi-hop candidate intersections and the time lag layers, a kNN / RBF weight is used for differentiable weighted interpolation to obtain a fixed number of three-dimensional space-time sampling cooperative representations; finally, the sampling cooperative representations are fused through a cross-attention module and an attention gating mechanism, and a traffic signal phase action of the current intersection is output by an action prediction head.

Description

Technical Field

[0001] This invention relates to the fields of intelligent traffic signal control, multi-agent reinforcement learning, offline reinforcement learning, knowledge distillation, and deep sequence decision making, and specifically to a traffic signal cooperative control method based on deformable attention.

[0002] This method is applicable to multi-intersection, regional road network, urban road network, and traffic signal cooperative control scenarios in intelligent transportation systems; it can also be used as a local signal control method in single-intersection scenarios. It can generate traffic signal phase control actions based on the historical traffic conditions of each intersection, historical signal control actions, traffic operation feedback or target reports, and the spatiotemporal states of multi-hop neighbors. Background Technology

[0003] Traffic signal control is a core component of intelligent transportation systems, playing a crucial role in alleviating urban traffic congestion, reducing vehicle delays, improving road efficiency, and reducing energy consumption. As urban road networks continue to expand, traffic flow exhibits significant non-stationarity, periodicity, suddenness, and spatial propagation. Signal control decisions at a single intersection often affect multiple surrounding intersections through vehicle queuing, overflow, green wave coordination, and phase switching. Therefore, in complex urban road networks, relying solely on local observations at a single intersection for independent control is insufficient to fully optimize the overall operational efficiency of the road network.

[0004] Traditional traffic signal control methods mainly include fixed-time control, adaptive control, and optimization control. For example, fixed-time control methods rely on manual experience or historical traffic statistics to set the cycle, green ratio, and phase difference; adaptive control methods adjust the signal scheme based on real-time traffic conditions collected by detectors; and optimization control methods obtain the control strategy by establishing a traffic flow model and solving for the optimization objective. These methods are effective under regular road networks and relatively stable traffic demand, but when faced with multi-intersection coupling, drastic changes in traffic demand, sudden congestion, and irregular topologies, they typically require extensive manual design and complex parameter adjustments, making it difficult to achieve efficient global coordinated control.

[0005] In recent years, reinforcement learning and deep reinforcement learning techniques have been widely applied to traffic signal control tasks. Reinforcement learning-based methods can learn signal control strategies based on traffic operation states, signal control actions, and feedback on control effects, without relying entirely on pre-set traffic models. Deep reinforcement learning further utilizes neural networks to learn representations of high-dimensional traffic states, improving modeling capabilities in complex scenarios. To adapt to multi-intersection traffic signal control tasks, methods such as multi-agent reinforcement learning, graph neural networks, attention mechanisms, and decision transformers have been used to model the cooperative relationships between intersections, achieving progress in metrics such as average delay, vehicle waiting time, and throughput.

[0006] However, existing multi-agent traffic signal control methods still have shortcomings. On the one hand, some methods mainly utilize direct neighbor information, making it difficult to capture the propagation relationship of traffic congestion between multi-hop intersections. On the other hand, the number and topological structure of neighbors vary between different intersections, and directly collecting the hidden states of neighbors can easily lead to inconsistent input dimensions or require complex filling, truncation, and graph aggregation operations. Furthermore, explicitly constructing complex semantic relevance scores, topological feature encoding, and selector modules for each candidate intersection can easily introduce many additional parameters, increasing the training difficulty and deployment cost.

[0007] Therefore, a new traffic signal cooperative control method is needed that, while retaining the ability of Decision Transformer to model historical trajectory sequences, can fuse multi-hop intersections and their time-lag state information in a fixed-dimensional, lightweight, and differentiable manner, thereby improving the cooperative control capability of multi-agents, reducing training complexity, and enhancing the safety and robustness of actual deployment. Summary of the Invention

[0008] This invention relates to a traffic signal cooperative control method based on deformable attention, which is applicable to traffic signal cooperative control scenarios in intelligent transportation systems, such as multi-intersection, regional road network, and urban road network, and can be extended to single-intersection signal control scenarios.

[0009] I. The design concept of this invention is as follows:

[0010] First, a shared Decision Transformer (DT) backbone network is used to perform time-series modeling of the historical traffic states, historical signal control actions, traffic operation feedback quantities, and target reward sequences of each intersection to obtain the local hidden state representation of the current intersection, and a lightweight time-lag memory is extracted from the hidden state sequences of each intersection.

[0011] Subsequently, a local three-dimensional spatiotemporal candidate node set is constructed based on multi-hop candidate intersections and time lag layers, and lightweight spatiotemporal coordinates are established for node-time lag pairs, thereby unifying the spatial multi-hop propagation relationship and time lag state in the same three-dimensional spatiotemporal sampling field.

[0012] In the model structure, this invention constructs a fixed number of three-dimensional spatiotemporal reference anchors to characterize the local time memory region of the current decision intersection, the one-hop neighbor propagation region of the current time layer, the multi-hop propagation region of the previous time layer, and the long-range propagation region of an earlier time layer. For each reference anchor, a shared offset generation network is used to input the local hidden state representation of the current intersection to generate a three-dimensional spatiotemporal offset with limited amplitude, enabling the reference anchor to adaptively move in the three-dimensional spatiotemporal sampling field according to the current traffic state.

[0013] To address the characteristic of traffic networks typically exhibiting irregular graph structures, this invention further employs a weighted fusion of the hidden states of neighboring node-time-delay pairs on a local 3D spatiotemporal graph constructed from the traffic network topology and time-lag layers. This is achieved by combining nearest neighbor search with a differentiable interpolation method based on radial basis function distance weights. This yields a fixed number of 3D spatiotemporal sampled collaborative representations. Subsequently, using the current intersection's hidden representation as the query and the sampled collaborative representations as the key and value, a cross-attention module is used to read multi-hop collaborative information. An attention gating mechanism dynamically assigns importance weights to different sampling slots, ultimately generating a contextual representation for traffic signal phase action prediction. This structure avoids the input dimension inconsistency caused by differences in the number of neighbors at different intersections and reduces the additional parameters introduced by explicit semantic relevance scoring, complex topological feature encoding, and candidate selector modules.

[0014] II. In practical applications, this invention employs a three-stage training process: offline pre-training, knowledge distillation, and online fine-tuning.

[0015] First, a teacher model containing a local 3D spatiotemporal map differentiable sampling module is trained using historical traffic control trajectory data to learn multi-intersection traffic signal collaborative control strategies.

[0016] Subsequently, the policy distribution of the teacher model is transferred to the lightweight student model through knowledge distillation, and the actual signal phase action supervision loss is combined to maintain the student model's ability to fit historical control actions and signal control capabilities.

[0017] Finally, during the online operation or trial operation of the target road network, a small number of online traffic control trajectories consisting of real-time traffic conditions, actual signal phase control actions, and traffic operation feedback information are used to fine-tune the Adapter module, action output layer, and related parameters of lightweight collaborative fusion in the student model to adapt to real-time traffic flow changes.

[0018] III. During the assessment and deployment process:

[0019] By receiving real-time traffic conditions from traffic detection equipment, roadside sensing equipment, signal controllers, or traffic management platforms, and generating corresponding signal phase control commands for intersections;

[0020] The control effect is evaluated by indicators such as average latency, average waiting time, average travel time and throughput. It can also be combined with a safety fallback mechanism to automatically fall back to the direct neighbor cooperative control mode when multi-hop information is missing, communication is abnormal or the multi-hop cooperative strategy does not meet the preset performance conditions.

[0021] This invention enhances the spatiotemporal coordination and environmental adaptability among multiple agents while ensuring computational efficiency and lightweight deployment, thereby reducing vehicle delays and congestion. It provides a lightweight, stable, and scalable technical solution for multi-intersection traffic signal collaborative control in intelligent transportation systems.

[0022] The present invention provides a traffic signal cooperative control method based on deformable attention, which addresses the problems in existing technologies such as strong reliance on direct neighbors in multi-intersection traffic signal cooperative control, insufficient utilization of multi-hop traffic propagation information, inconsistent input dimensions due to differences in the number of neighbors at different intersections, complex parameters of cooperative modules, and insufficient security in actual deployment.

[0023] The traffic signal cooperative control method of this invention uses a shared Decision Transformer backbone network to temporally encode the historical trajectory of each intersection to obtain a local hidden state sequence. Based on this, a local three-dimensional spatiotemporal candidate node set composed of multi-hop candidate intersections and a time lag layer is constructed for each decision intersection. A fixed number of three-dimensional spatiotemporal reference anchor points are further constructed, and a restricted offset is generated using the current local hidden state of the intersection, enabling the reference anchor points to adaptively move in the three-dimensional spatiotemporal sampling field. Subsequently, the method is applied to the local three-dimensional spatiotemporal map (which is composed of multi-hop candidate intersections and a time lag layer). The weights are differentiable weighted interpolation to obtain a fixed number of three-dimensional spatiotemporal sampled collaborative representations; finally, the sampled collaborative representations are fused through a cross-attention module and an attention gating mechanism, and the traffic signal phase action of the current intersection is output by the action prediction head.

[0024] This invention comprises three stages: pre-training, knowledge distillation, and online fine-tuning, and can be combined with model evaluation, hyperparameter tuning, online replay pool updates, and a safe rollback mechanism to complete practical deployment.

[0025] The traffic signal cooperative control method of the present invention specifically includes the following steps:

[0026] Step 1. Obtain basic traffic signal control data for the target road network. The basic data includes the topological connection relationship of each controlled intersection in the target road network, lane structure, signal phase, intersection spatial coordinates, vehicle operating status collected by traffic detection equipment, and historical signal control records.

[0027] At each traffic control time step, the traffic status, current signal phase, historical control actions, and traffic operation feedback information of each intersection are acquired to construct a multi-intersection traffic signal collaborative control problem. The traffic status includes queue length, lane occupancy rate, vehicle waiting time, and number of approaching vehicles; the traffic operation feedback information includes vehicle traffic conditions, waiting time, delay time, and road network efficiency.

[0028] Step 2. In the pre-training phase, an initial traffic signal control strategy is learned based on historical traffic control trajectory data. This method constructs a historical traffic control trajectory dataset based on historical traffic operation records of the target road network. This dataset consists of the states, signal control actions, traffic operation feedback quantities, and target rewards of multiple controlled intersections at consecutive traffic control time steps. The historical traffic operation records are derived from traffic detectors, roadside sensing devices, signal controller logs, traffic management platform records, or vehicle-to-infrastructure (V2I) system data. For any intersection, its historical trajectory is represented as a sequence of target rewards (Return-to-Go, RTG), traffic state observations, and signal control actions at multiple consecutive time steps, used for temporal modeling of the subsequent decision transformer backbone network.

[0029]

[0030] in Indicates intersection At time step of Target return or future cumulative return; Indicates intersection At time step Traffic condition monitoring typically includes the queue length of each approach lane at the current intersection, the number of approaching vehicles in each approach lane, and the waiting time data for vehicles in each lane. Indicates intersection At time step The traffic signal control actions executed, and the selection of the traffic light phase at the next moment.

[0031] Step 3. For each intersection Historical trajectory The hidden state sequence is obtained by encoding through the SharedDecision Transformer backbone network.

[0032]

[0033] Current intersection The local representation is denoted as In order to construct a three-dimensional spatiotemporal sampling field, from each intersection Extracting lightweight time-lag memory from hidden state sequences This serves as the content feature in the subsequent set of 3D spatiotemporal candidate nodes. In a preferred embodiment, the hidden state representations of the current time, the previous time, and the time before that are extracted to construct a time lag memory. Generally, the time lag can be used... .

[0034] Step 4. For the current decision intersection structure Multi-hop candidate set

[0035]

[0036] in For traffic network map Zhongcong intersection At the intersection The shortest graph distance; This is the maximum multi-hop search radius. In a preferred embodiment, the maximum multi-hop search radius is... Set to 3, then the collection contains Candidates, when the moved reference anchor point is primarily sampled, fall on When candidate nodes are selected, the method can degenerate into a direct neighbor cooperative control mode.

[0037] Step 5. Construct a local 3D spatiotemporal candidate node set based on the multi-hop candidate set and the time lag layer.

[0038]

[0039] in This is a time-delay set. Each node-delay pair Content characteristics are .

[0040] Step 6. For each node-delay pair Constructing lightweight spacetime coordinates

[0041]

[0042] in Indicates candidate intersections Relative to the current intersection Spatial coordinates; Represents the normalized time layer coordinates. Prioritize using the relative positions normalized from the actual road network coordinates. This applies when the actual two-dimensional coordinates of each intersection can be obtained, where Indicates the current decision-making crossroads The actual coordinates on the map; Indicates candidate intersections The actual coordinates on the map; This indicates the lateral displacement of the candidate intersection relative to the current intersection. This indicates the longitudinal displacement of the candidate intersection relative to the current intersection. Normalization is used to prevent coordinate values ​​from becoming too large. Through normalization, spatial coordinates and temporal coordinates are kept within a comparable scale, preventing spatial distance from having an excessive influence on the temporal dimension in subsequent distance calculations. If the actual road network coordinates are unavailable, topological coordinates based on graph distance are used. ,use Distance, describing candidate intersections Distance from the current intersection How far is it, applicable when there are no real map coordinates. Step 6 provides coordinates for step 8, and the lightweight spatiotemporal coordinates are used to provide a unified three-dimensional spatiotemporal position representation for subsequent anchor point offsets and differentiable interpolation sampling.

[0043] Step 7. Construct four three-dimensional spacetime reference anchor points. ,in The four three-dimensional spatiotemporal reference anchor points are initialized as follows: Current intersection exist Spatiotemporal position of a moment Current moment Decision Crossroads Direct neighbor propagation reference location , The intersection is Reference position for propagation of time Reference position for the propagation of candidates at earlier time layers , is used to describe the local spatiotemporal propagation reference location around the current intersection.

[0044] Step 8. For each anchor point Utilize the local hidden state of the current intersection Input sharing Generate restricted offset

[0045]

[0046] in This is the offset scale factor. Used to limit the offset magnitude : It is a three-dimensional spacetime coordinate dimension, for example ,correspond The offset output by the offset generation network can be divided into offsets corresponding to four anchor slots. Then calculate the reference anchor point after the movement.

[0047] Step 9. For each moved reference anchor point In the three-dimensional spatiotemporal candidate node set Find the nearest Node-delay pair In the current method Since the traffic network is not a regular image grid, it is constructed from the traffic network topology and time lag layers, using... On a local 3D spatiotemporal graph of a set of nodes, using Differentiable sampling is performed to obtain the interpolation weights of the offset anchor point with respect to the nearest true candidate.

[0048]

[0049] Among them, the molecule is the current candidate. The distance weighting requires that the denominator include all nearby candidates. The distance weights are summed up and used for normalization; Indicates the first One reference anchor point; Indicates candidate nodes Coordinates in the three-dimensional spatiotemporal sampling space; This represents another candidate intersection index in the normalized denominator; Indicates the closest to the offset anchor point One real-world spacetime candidate; This represents the squared Euclidean distance between the offset anchor point and the candidate spatiotemporal node; Indicates the attenuation scale of sampling temperature or distance. The smaller the value, the closer the sampling result is to "hard selection of the nearest node". The larger the value, the smoother the weight distribution of neighboring candidate nodes. Differentiable weighted interpolation of neighboring candidate hidden states is performed using distance weights based on radial basis functions to obtain the... Sampling collaborative representation of reference anchor points

[0050]

[0051] Step 10. (As shown) Figure 3 As shown, the current intersection is hidden. As The four samples are represented together. As and It integrates a cross-attention module for collaborative information fusion and... The mechanism dynamically calculates the importance of each sampling slot to the current intersection decision, and obtains the final context representation. .

[0052] Step 11. Input action prediction head, for the first At each time step, the action prediction head first obtains the action through a linear mapping. :

[0053] ,

[0054] in Indicates the number of available traffic phases at the current intersection; This represents the action prediction head weight matrix; It is the final context representation; This indicates head bias in action prediction. Then, through... Get the current intersection At time step Select phase probability

[0055]

[0056] This yields the probability distribution of actions at the current decision intersection at the corresponding time step. And the action probability of each candidate signal phase satisfies During reasoning, through Generate the current intersection exist Traffic signal phase actions at any given time, among which This represents the last time step of the current input window; under the conditions of satisfying traffic safety constraints and signal phase switching constraints, it can be determined based on the action probability distribution. Generate the current intersection exist The system tracks the traffic signal phase actions at any given time and transmits them via the traffic signal control interface to the corresponding intersection's signal controller or traffic signal control system to generate or update the signal light phase control commands for the current control time step.

[0057] Step 12. In the pre-training and online fine-tuning phases, the multi-intersection cooperative control model is trained using active action loss and offset constraint loss. (Current intersection...) The active loss is as follows:

[0058]

[0059] In all valid Calculate the actual signal phase action at the position And average the effective positions, plus separately calculated The average is then taken, and the negative sign is used as the loss to be minimized. express Effective Location set; express Which sample in the middle; Indicates the time step position within a sample window; This indicates the phase actions actually performed in the historical traffic control trajectory; This represents the distribution of actions predicted by the model; Represents the entropy regularity coefficient; Indicates all Number of locations, including Location; This represents the action distribution entropy. The offset constraint loss is used to limit the offset amplitude of the reference anchor point, preventing the moved reference anchor point from deviating from the effective three-dimensional spatiotemporal sampling area. (Current intersection) The offset constraint loss is as follows:

[0060]

[0061] in, This indicates the number of anchor points at each time step; Indicates the current intersection , The Middle Individual samples, time steps , No. The offset vector of each anchor point In this method, Represents three-dimensional spatiotemporal sampling coordinates; Indicates the first The squared norm of the anchor point offset vector. Adding the active action loss to the offset constraint loss yields the total loss used to train the model in this method. Backpropagation is performed based on the total loss, and the trainable parameters in the multi-way cooperative control model are updated.

[0062] The multi-way collaborative control model includes a multi-head attention module, a cross-attention module, and an attention gate mechanism.

[0063] Multi-Head Attention Module:

[0064] like Figure 1 As shown, due to trajectory data The raw data in Since their types and dimensions are different, embedding is needed to map them to the same hidden dimension. To characterize the temporal location information in the historical trajectory sequence, for each time step The corresponding input By incorporating time-position embedding, therefore for lengths of... The historical trajectory is stacked into a sequence according to time steps. ,in Indicates intersection At the present moment Transformer input sequence, express quantity, Indicate each The hidden dimension. The stacked sequence is input into... In a layered Transformer block, each layer of Transformer contains An attention head, initial layer input Then the first Input of the Transformer layer Transformer blocks typically normalize the input first, and multi-head attention projects the input linearly into... For the first Size, during the calculation of attention score, will be included Because it uses historical trajectory modeling, it is used to prevent errors in earlier time steps. Accessing subsequent time steps during attention computation Then through Obtain the attention weights; then weight the results using the attention weights. This yields a single-head output. The outputs of all attention heads are concatenated, and then the output projection matrix is ​​used to obtain... Module output. The output, after being processed by residual connections and layer normalization, enters the feedforward network sublayer to obtain the output of the corresponding Transformer Block. This is because residual connections allow the model to retain original information while learning incremental updates. Then, after... Feedforward network, to obtain the first The final output of the Transformer block. Layers of Transformer blocks are stacked vertically for computation, and finally... Use actions Previous Predicting actions, which means using the state position corresponding to... To predict actions, and thus ultimately obtain

[0065] The complete process formula is as follows:

[0066]

[0067] After embedding:

[0068]

[0069] Add time and location embedding:

[0070]

[0071] Stacked into a sequence:

[0072]

[0073] Input into a multi-level Transformer:

[0074]

[0075]

[0076]

[0077] in The interior is:

[0078]

[0079]

[0080]

[0081] in Indicating causal attention , This indicates the output projection matrix.

[0082] Finally, take the corresponding time step. :

[0083]

[0084] get:

[0085]

[0086] Cross-Attention module:

[0087] The cross-attention module is used to enable the current decision intersection to read external collaborative information from the three-dimensional spatiotemporal sampled collaborative representation. For example... Figure 2 As shown, the current intersection is hidden. and using steps 1 to 9 to obtain This serves as the input to the Cross-Attention module. During the Cross-Attention module's computation, the current intersection... As ,get Four-sample co-representation As and ,get and Then calculate cross-attention to obtain Finally, by performing residual connections and normalization, the current intersection is obtained. With the A candidate fusion representation after cross-attention of reference anchors:

[0088] ,

[0089] Attention gate:

[0090] The attention gating mechanism is used to adaptively determine the importance weights of multiple candidate fused representations to the current signal control decision based on the current hidden state representation of the decision intersection. For example... Figure 2 As shown, the current intersection is hidden. And the candidate fusion representation obtained using the cross-attention module. This serves as the input to the attention gate. During the attention gate calculation, the current intersection... As ,get Four candidate fusion representations As and ,get and Then calculate the current intersection for the first... Candidate fusion heads Fraction:

[0091]

[0092] Then we obtain the context representation:

[0093]

[0094] Finally, residual connections and layer normalization are performed to obtain the final context representation from step 10:

[0095]

[0096] Step 13. Using the multi-hop collaborative teacher model trained in Step 12 as the teacher network and the locally collaborative Transformer model with a smaller parameter size as the student network, train the lightweight student model using knowledge distillation. Specifically, sub-trajectories from the historical traffic control trajectory dataset are input into the teacher and student models, and the teacher model outputs the teacher's actions. The student model outputs student actions. After both were normalized by temperature coefficient (T), the teacher's result was obtained. probability distribution Student strategy distribution .use Divergence measures the distributional difference between student and teacher strategies, and is used to obtain the knowledge distillation loss. At the same time, to ensure that the student model learns not only from the teacher model... Furthermore, it can maintain the supervised fitting ability of actual signal phase actions in historical traffic control trajectories, and introduces a Decision Transformer supervised loss based on actual signal phase action labels. Ultimately, the distillation training objective for the student model is a weighted sum of the two.

[0097] The complete formula is shown below:

[0098] teacher probability distribution Student strategy distribution :

[0099]

[0100] in, Indicates the current intersection A set of actions; This represents a temporary summation variable within the action set, used only to sum the current action in the numerator. Distinguishing them does not imply any new or special actions; It indicates the time step position in the trajectory sequence, not the entire trajectory itself; This indicates the teacher model's response to actions. Unnormalized scoring; Represents the student model's action Unnormalized scoring; Represents the distillation temperature coefficient, used for control. The smoothness of the distribution.

[0101] Knowledge distillation loss :

[0102]

[0103] The smaller the value, the closer the student model's output distribution is to the teacher model's output distribution. Indicates the first All valid intersections Location set; Indicates the number of valid positions; This indicates the distribution of teachers and students in the action. Differences in the above.

[0104] Monitoring losses :

[0105]

[0106] Its purpose is to enable student models to produce higher probabilities of actual signal phase actions based on historical traffic control data. Among these, Labels representing actual actions recorded in historical traffic signal control records; This represents the student model's predicted probability of the actual signal phase action.

[0107] Final distillation training objective :

[0108]

[0109] Step 14. After completing the knowledge distillation in Step 13, the trained lightweight student model is used as the base network structure for the online fine-tuning phase. The goal of the online fine-tuning phase is to further adapt the model to the real-time traffic flow distribution of the current road network in the target road network, while reducing the perturbation of the pre-trained knowledge representation during online fine-tuning. First, the student model is set to its current policy... For the online control strategy of the target road network, the intelligent transportation system receives the real-time traffic status of each controlled intersection during the actual operation or online trial operation of the target road network, and outputs corresponding signal phase control actions or control suggestions; subsequently, based on the vehicle operation feedback information returned by traffic detection equipment, roadside sensing equipment, signal controllers or traffic management platforms, a new online traffic control trajectory is formed. The system calculates online control effectiveness evaluation metrics and target return-to-go based on vehicle delay, waiting time, queue length, traffic efficiency, or combinations thereof. Subsequently, newly acquired trajectories are added to the online playback dataset. When the playback dataset size exceeds a preset threshold, a trajectory replacement strategy is employed to retain higher-quality or more recent trajectories, thereby constructing an online training set. During the online fine-tuning process, from Random sampling sub-trajectory As input to the student model, to reduce the computational cost and overfitting risk of online fine-tuning, in a preferred embodiment, a compact Adapter module is inserted after each block of the Transformer backbone network. Only the parameters related to the Adapter module, action output layer, and lightweight co-fusion are updated, while most of the pre-trained backbone parameters are frozen. The Adapter module can adopt a COMPACTER++ structure, which consists of a downprojection using low-rank hypercomplex parameterized multiplication (LPHM), a nonlinear activation function (GeLU), and an upprojection. The loss function in the online fine-tuning stage is consistent with the total loss function in step 12. .

[0110] Step 15. After completing the online fine-tuning in Step 14, fix the model parameters updated in the current iteration, switch the model to the evaluation state, and continuously evaluate the traffic operation data of the target road network within a preset evaluation period to assess the model's decision-making performance in real traffic dynamics. At each control time step, traffic detection equipment, roadside sensing equipment, or the traffic management platform provides the system with the current observation status of each controlled intersection. For any intersection... The system constructs historical trajectory input based on the current state, historical actions, historical traffic operation feedback, and return-to-go, and obtains the local hidden state representation of the current intersection through the Transformer backbone. The subsequent forward inference process of the model is consistent with the historical trajectory encoding, 3D spatiotemporal candidate construction, anchor point offset sampling, collaborative information fusion, and action prediction processes in steps 3 to 11. The system combines the signal phase actions of all controlled intersections into a joint control action and transmits it to the corresponding signal controller or traffic signal control system equipment to drive the target road network to execute the corresponding signal control scheme in the next control time step. Traffic detection equipment, roadside sensing equipment, signal controllers, or traffic management platforms acquire traffic status and traffic operation feedback information for the next control time step. The traffic operation feedback information includes vehicle traffic conditions, queuing conditions, waiting time, travel time, delay time, and overall road network operation indicators. The system can calculate the traffic control effect evaluation quantity or target return based on the above feedback information. After a preset evaluation cycle, the system saves the vehicle travel information and overall evaluation results in the traffic network for subsequent performance analysis.

[0111] Step 16. Select the optimization algorithm and training hyperparameters. An adaptive optimization algorithm is used to update the model parameters during training. In one implementation, the main model parameters are... or Adaptive optimizer, temperature parameter adopted The optimizer is updated separately; in another implementation, it can also be updated uniformly. The optimizer updates the trainable parameters. Different training iterations, batch size, learning rate, weight decay coefficient, temperature coefficient, and reward scaling coefficient are set for the three stages: teacher model pre-training, student model distillation, and online fine-tuning. The temperature coefficient is set during the distillation stage. Used to control teachers The loss weights are set according to the smoothness of the student output distribution. and The proportions of knowledge distillation loss and actual signal phase action supervision loss in the total loss are controlled separately; the online fine-tuning stage is set... , , , , and Hyperparameters such as return-to-go scaling factor, online sampling reward scaling factor, evaluation reward scaling factor, online replay pool size, and evaluation interval can also be set during training. These hyperparameters can be determined empirically, through grid search, or random search to achieve better training stability and traffic control performance.

[0112] Step 17. Training the model. The entire training process consists of three stages: (1) Pre-training the teacher model using historical traffic control trajectories. This stage retains the Decision Transformer backbone structure and introduces a local 3D spatiotemporal graph differentiable module, enabling the model to construct multi-hop candidate information during synchronous multi-agent training. During training, the model trains each valid historical trajectory. (1) Predict signal phase actions and use the actual signal phase actions in historical traffic control trajectories as the supervision target. (2) Train the student model using a knowledge distillation strategy. The teacher model is generated based on historical traffic control trajectories. The student model simultaneously learns the actual signal phase action labels of the teacher output distribution and historical trajectories. (3) Online fine-tuning of the student model. During the online operation or trial operation of the target road network, the model receives real-time traffic status and forms new online traffic control trajectories based on traffic operation feedback. These trajectories are added to the online training set or replay pool, and sub-trajectories are sampled from them for training. In the online fine-tuning stage, only the Adapter module, action output layer, and lightweight collaborative parameters can be updated to reduce online training overhead. Alternatively, more parameters can be opened for joint fine-tuning according to deployment requirements. In each stage, the model generates action distributions through forward propagation, calculates the corresponding loss, obtains gradients through backpropagation, and the optimizer updates trainable parameters until the preset number of training steps is reached or the stopping condition is met.

[0113] Step 18. Performance Evaluation. After model training, the model is evaluated multiple times based on historical playback data, online trial operation data, or actual operation data of the target road network, and its performance is assessed using traffic operation indicators. Evaluation indicators include average latency, average waiting time, average travel time, vehicle throughput efficiency, and overall road network throughput. The primary evaluation indicator is average latency, which can be expressed as... ,in This indicates the number of vehicles that completed the trip. Indicates vehicle The average delay in the traffic network is used to characterize the additional congestion of vehicles within the network. During the evaluation process, interpretability information from the multi-hop sampling module can also be recorded, including the main real intersections sampled by the offset anchor points, the corresponding time lag layers, sampling weight entropy, offset norm, and the proportion of direct neighbor slots, to analyze the model's utilization of multi-hop spatiotemporal collaborative information.

[0114] Step 19. Model Testing and Deployment. After obtaining the optimal model parameters, the trained lightweight student model is used as the final deployment model. During deployment, the system first pre-constructs intersection adjacency relationships, multi-hop candidate sets, and candidate spatiotemporal coordinates based on the topology of the target road network; then, at each traffic control time step, it receives the status of each intersection from traffic detectors, roadside sensing devices, signal controllers, traffic management platforms, or vehicle-road cooperative system interfaces, including queue length, waiting time, vehicle arrival status, lane occupancy, and current signal phase. For any intersection... The deployment system is based on the most recent The historical states, actions, and traffic operation feedbacks at each time step, along with the return-to-go model input, are used to obtain a local hidden state representation through forward propagation of a lightweight student model. This is then used to generate a final fused representation based on 3D spatiotemporal multi-hop sampling information. The action prediction head outputs the current phase action based on this representation, and the traffic controller converts this action into actual traffic light phase control commands. In actual deployment, the system can be configured with a safety fallback mechanism: when multi-hop candidate information is missing, neighbor communication is unavailable, sampling weights are abnormal, or the multi-hop cooperative control strategy fails to meet preset performance improvement conditions within the recent evaluation period, it automatically falls back to the direct neighbor cooperative control mode; when the multi-hop cooperative strategy is consistently superior to the direct neighbor strategy, multi-hop cooperative inference is then enabled. Through this mechanism, the model's ability to model complex traffic propagation relationships and its online adaptability can be improved while ensuring traffic control safety and real-time performance.

[0115] In steps 2 and 3, the decision transformer backbone network performs time-series modeling of the historical state, actions, and target reward sequence of each intersection, and outputs a hidden state sequence. The time lag memory It is used only as a feature body of the three-dimensional spatiotemporal sampling field and does not participate in explicit semantic relevance scoring, thereby reducing additional parameters and avoiding duplication with the historical semantic encoding of the backbone network.

[0116] In steps 4 and 5, the local three-dimensional spatiotemporal candidate node set It consists of a multi-hop intersection candidate and a time lag layer, with each candidate consisting of intersection nodes. and its corresponding time lag layer Together, they determine and form node-time delay pairs. This enables the model to simultaneously perceive intersections with different spatial propagation distances and historical states with different propagation stages in time.

[0117] In step 6, if the actual road network coordinates can be obtained, then ,in , For the current intersection The actual coordinates, , Candidate intersections The actual coordinates, , For normalized scaling; if the actual road network coordinates cannot be obtained, then The spatial position of the candidate intersection relative to the current intersection is represented by the normalized graph distance.

[0118] In step 7, the four three-dimensional spatiotemporal reference anchor points include: local time memory anchor points. ; Current layer one-hop neighbor anchor point ; Upper-level two-hop propagation anchor point Earlier-stage long-range propagation anchor points ;in This indicates that the corresponding candidate spatiotemporal coordinates are subjected to average pooling, max pooling, or weighted pooling.

[0119] In step 8, the Share parameters across the four anchor slots and use the current intersection hiding state. Generate as input dimensional offset, where The spatiotemporal coordinate dimension; the The output is through Function and offset scale factor Amplitude constraints are applied to prevent the reference anchor point from shifting into an invalid spatiotemporal region during training.

[0120] In step 9, the differentiable weighted interpolation adopts... Form, weight Based on the reference anchor point after movement With candidate spatiotemporal coordinates The squared Euclidean distance between them is calculated, with closer distances resulting in higher weights, and the sampled collaborative representation is obtained through neighborhood normalization. .

[0121] In step 10, the cross-attention module uses the current intersection hidden state. Mapping to obtain the query vector The key vector is obtained by mapping using four sampled collaborative representations. Sum value vector Four fusion slots are obtained through attention calculation, residual connection, and layer normalization; The mechanism further dynamically allocates weights based on the current hidden state of the intersection and the fusion slots, resulting in... .

[0122] In step 12, the training loss includes the traffic signal control active action loss and the offset constraint loss. ,in Used to constrain the offset of the reference anchor point; This includes one or more of the following: action prediction loss, teacher model training loss, knowledge distillation loss, or online fine-tuning loss.

[0123] In the knowledge distillation stage, the output distributions generated by the teacher model on historical traffic control trajectories and the student model output distributions are normalized using temperature coefficients, and then... A weighted combination of divergence and negative log-likelihood supervision loss is used as the training objective for the student model to obtain a lightweight controller.

[0124] During the online fine-tuning phase, insert after the Transformer Block of the student model. Module, and only update The parameters of the module and the output action mapping layer remain unchanged from the pre-trained backbone parameters, so as to adapt the target road network using a small amount of online traffic control trajectory data consisting of real-time traffic conditions, actual signal control actions and traffic operation feedback information.

[0125] The model evaluation and deployment process includes: conducting multiple rounds of evaluation based on historical playback data, online trial operation data, or actual operation data of the target road network, and calculating traffic operation indicators such as average delay, average waiting time, average travel time, and throughput; when the model performance meets the preset requirements, the obtained model is deployed on a local server, cloud platform, edge device, or urban traffic signal controller for real-time generation of multi-intersection traffic signal control actions.

[0126] In summary, this invention introduces a three-dimensional spatiotemporal reference anchor mechanism, an anchor point offset learning mechanism, and a mechanism based on... This method employs a local 3D spatiotemporal graph differentiable interpolation sampling mechanism, combined with the Decision Transformer's temporal modeling capabilities for historical trajectory sequences, Cross-Attention's ability to fuse multi-intersection collaborative information, and AttentionGate's adaptive selection capability for the importance of externally sampled information. This enhances the collaborative modeling capability and engineering applicability in multi-agent traffic signal control tasks. The method can uniformly map multi-hop candidate intersections and their time-lag states under different scales and topologies to a fixed number of collaborative representations, avoiding inconsistencies in input dimensions due to differences in the number of neighbors, while preserving multi-hop traffic propagation information and spatiotemporal dynamic correlation features. Furthermore, by combining teacher model training, knowledge distillation, and online fine-tuning processes, this method enhances the information collaboration capabilities and environmental adaptability among traffic agents while ensuring model computational efficiency and lightweight deployment. This effectively optimizes the overall operation of the traffic system, reduces vehicle delays and congestion, and provides technical support for the deployment of intelligent transportation systems in real-world multi-intersection scenarios.

[0127] Compared with the prior art, the main advantages of the present invention are:

[0128] First, this invention constructs three-dimensional spatiotemporal reference anchor points and node-time delay pairs to unify spatial multi-hop intersections and time-delay states into the same local sampling field, enabling the model to perceive the propagation relationship of traffic congestion between different intersections and different time layers.

[0129] Second, the present invention uses a fixed number of reference anchor points and sampling collaborative representation to avoid the problem of inconsistent input dimensions caused by differences in the number of neighbors at different intersections.

[0130] Third, this invention replaces the explicit complex candidate selector with local three-dimensional spatiotemporal graph differentiable interpolation sampling based on kNN / RBF, which reduces additional parameters and improves training stability and deployment efficiency.

[0131] Fourth, this invention combines Cross-Attention and Attention Gate to enable the current intersection to adaptively fuse information from different sampling slots.

[0132] Fifth, this invention combines knowledge distillation, online fine-tuning, and a safe rollback mechanism to improve the collaborative capability, adaptability, and engineering deployability of multi-agent traffic signal control systems while ensuring the model's lightweight nature and real-time performance. Attached Figure Description

[0133] Figure 1 This is a diagram of the Multi-Head Attention module model.

[0134] Figure 2 The diagram shows the Cross-Attention module and the Attention gate model.

[0135] Figure 3 Output the overall module model diagram;

[0136] Figure 4 This is a complete input-to-output model diagram;

[0137] Figure 5 This is a flowchart illustrating the specific implementation steps of the method. Detailed Implementation

[0138] To better understand the technical solution of the present invention, the present invention will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the following embodiments are only for explaining the present invention and are not intended to limit the scope of protection of the present invention. Equivalent substitutions, parameter adjustments, module combinations, or structural modifications made by those skilled in the art without departing from the concept of the present invention should all fall within the scope of protection of the present invention.

[0139] This invention provides a traffic signal cooperative control method based on deformable attention. This method is designed for traffic control tasks at multiple intersections. It uses historical traffic conditions, historical signal control actions, traffic operation feedback quantities, and target reward sequences provided by the system as model inputs, and the traffic signal phase at the next time step for each intersection as the model output.

[0140] like Figure 5 As shown, the specific implementation process of this method is as follows:

[0141] Step 101. Obtain basic traffic signal control data for the target road network. Based on the target road network, determine the roads, lanes, intersections, signal phases, detector deployment locations, intersection topological connections, and control cycles. Obtain data such as vehicle flow, queue length, lane occupancy, vehicle arrival status, vehicle waiting time, and historical signal control records from traffic detectors, roadside sensing devices, signal controllers, or traffic management platforms to construct a multi-intersection traffic signal collaborative control system.

[0142] Step 102. Obtain historical traffic trajectory data. Based on traffic detection data, signal control logs, and traffic operation evaluation records generated during the historical operation of the target road network, extract the traffic state, actual signal phase actions, control effect evaluation values, and traffic state for the next time step at each controlled intersection in continuous control time steps. Calculate the target return-to-go based on vehicle delay, waiting time, queue length, traffic efficiency, or a combination thereof. Then, process the state features by handling missing values, normalizing, and windowing to obtain historical traffic control sub-trajectory samples suitable for training the Decision Transformer.

[0143] Step 103. Construct a shared Decision Transformer backbone network. For each intersection, the return-to-go, traffic state, and historical actions are mapped to a unified hidden dimension through embedding layers, and time-location embeddings are added. Tokens from multiple time steps are stacked in chronological order and input into multiple Transformer Blocks. Each TransformerBlock contains causal multi-head self-attention, a feedforward network, residual connections, and layer normalization to model the temporal dependencies in the current intersection's own historical trajectory.

[0144] Step 104. Extract the time-lag memory. At each training or inference time step, a snapshot of the hidden state sequence of each intersection is saved, and the hidden states at the current time and several historical time steps are extracted to form a lightweight time-lag memory. This memory serves as the content feature in the 3D spatiotemporal sampling field, which is then read by the collaborative sampling module of other intersections.

[0145] Step 105. Construct a multi-hop candidate set. Based on the adjacency relationships of the traffic network, use breadth-first search, shortest path search, or a pre-computed topology table to construct a multi-hop candidate set with a radius not exceeding R for each decision intersection. The candidate set can include direct neighbors, two-hop neighbors, and three-hop neighbors, and different search radii can be set according to the actual network size.

[0146] Step 106. Construct a local 3D spatiotemporal candidate node set and lightweight coordinates. For each candidate intersection, combine it with each time layer in the time lag set to form a node-time lag pair, and assign 3D spatiotemporal coordinates to each node-time lag pair. If real map coordinates exist, use the normalized 2D coordinates and normalized time layer coordinates of the candidate intersection relative to the current intersection; if real map coordinates do not exist, use graph distance-based topological coordinates as the spatial location description.

[0147] Step 107. Initialize four 3D spatiotemporal reference anchor points. Construct four reference anchor points based on the current intersection's local location, the one-hop neighbor region, the two-hop propagation region, and the propagation region at a more distant time layer. Each reference anchor point can be obtained by average pooling, max pooling, or weighted pooling of the corresponding candidate spatiotemporal coordinates. When the candidate region corresponding to a certain anchor point is empty, a fallback initialization can be performed using direct neighbors, the current intersection's own historical state, or the nearest available candidate region.

[0148] Step 108. Generate anchor point offsets. Input the current local hidden state of the intersection into the shared offset network, and output the 3D offsets of the four anchor points. The offsets are then processed... The function and offset scale factor constraints are then superimposed onto the initial reference anchor point to obtain the moved reference anchor point. The offset network shares parameters among the four anchor point slots, which reduces model complexity and makes the anchor point movement primarily driven by the current intersection traffic state.

[0149] Step 109. Perform local 3D spatiotemporal graph differentiable interpolation sampling. For each moved reference anchor point, find the k closest candidates in the candidate node-delay pair set of the local 3D spatiotemporal graph, and calculate the distance based on the 3D spatiotemporal coordinates. Weights. Then, normalized weights are used to perform a weighted summation of the candidate hidden states to obtain the sampled collaborative representation corresponding to the anchor point. This process is differentiable, therefore the offset network, fusion module, and backbone network can be jointly optimized through end-to-end training.

[0150] Step 110. Perform collaborative information fusion. Use the current hidden state of the intersection as... The four sampled co-representations are used as and The Cross-Attention module is input to obtain four candidate fused representations. Then, the Attention Gate module is used to calculate the importance weights of the four fused slots. The weighted fusion result is then residually connected to the current intersection hidden state and normalized to obtain the final context representation.

[0151] Step 111. Generate traffic signal phase actions. Input the final context representation into the action prediction head, and perform linear mapping and... Normalization is performed to obtain the probability distribution of each candidate phase. During the training phase, the negative log-likelihood loss is calculated based on the actual signal phase action labels. During the inference phase, the phase with the highest probability can be selected as the deterministic control action, or, under the conditions of satisfying traffic safety constraints and signal phase switching constraints, candidate control actions or control suggestions can be generated based on the probability distribution for online trial operation or restricted policy updates.

[0152] Step 112. Train the teacher model. Pre-train the teacher model, which includes a local 3D spatiotemporal map differentiable sampling module, using historical traffic control trajectory data. The training objectives include active action loss and offset constraint loss. The active action loss is used to improve the model's ability to fit the actual signal phase action, while the offset constraint loss is used to prevent excessive offset of anchor points. During training, the Decision Transformer backbone parameters are shared across intersections and updated jointly within a synchronous multi-agent training framework.

[0153] Step 113. Train the student model. Use the pre-trained teacher model as the teacher network and the lightweight local collaborative Transformer model as the student network. Both the teacher and student models output action logits for the same historical traffic control sub-trajectory, and the probability distribution is obtained after normalization by the temperature coefficient. The student model simultaneously minimizes... Distillation loss and actual signal phase action supervision loss are used to reduce the number of model parameters while preserving the control capability of the teacher model as much as possible.

[0154] Step 114. Perform online fine-tuning. During the online trial or actual deployment of the target road network, the distilled student model is applied. The system continuously receives real-time traffic detection data and generates signal phase control actions or control suggestions. Based on the actual executed signal phase control actions and their corresponding traffic operation feedback information, a new online traffic control trajectory is formed and added to the online playback pool. During the online fine-tuning phase, most Transformer backbone parameters can be frozen, and only the Adapter module, action prediction head, and lightweight collaborative fusion parameters are updated. The Adapter module can use COMPACTER++, LoRA, Houlsby Adapter, or other parameter-efficient fine-tuning structures.

[0155] Step 115. Set the optimizer and hyperparameters. During the teacher pre-training, student distillation, and online fine-tuning phases, the learning rate, batch size, number of training steps, temperature coefficient, distillation loss weight, supervised loss weight, offset constraint weight, entropy regularization weight, history window length, online replay pool capacity, and evaluation interval can be set respectively. These hyperparameters can be determined empirically, through grid search, or random search.

[0156] Step 116. Perform model evaluation. After training, conduct continuous operational evaluations within multiple preset evaluation periods in the traffic signal control system, and statistically analyze indicators such as average delay, average waiting time, average travel time, vehicle throughput efficiency, and throughput. The evaluation should use the same historical playback data, the same evaluation period, the same traffic demand conditions, the same road network topology, and the same signal control constraints to ensure fair comparisons between different models or different inference modes.

[0157] Step 117. Record interpretability and safety metrics. During the evaluation process, information such as candidate intersections, candidate time layers, kNN / RBF weights, weight entropy, offset norm, direct neighbor sampling ratio, and multi-hop candidate sampling ratio sampled at each anchor point can be recorded to determine whether the model truly utilizes multi-hop spatiotemporal collaborative information and to analyze the causes of abnormal sampling or policy degradation.

[0158] Step 118. Perform a safety rollback and inference strategy selection. The deployed system can simultaneously retain the direct neighbor collaborative branch and the 3D spatiotemporal multi-hop sampling branch. When multi-hop candidate information is missing, communication is abnormal, sampling weight distribution is abnormal, or recent evaluation results show that the multi-hop collaborative control strategy has not met the preset performance improvement conditions, the system automatically rolls back to the direct neighbor collaborative control mode; when the multi-hop collaborative branch continuously meets the preset performance improvement conditions relative to the direct neighbor collaborative branch under fixed evaluation conditions, the system then enables multi-hop collaborative inference.

[0159] Step 119. Complete model deployment. After obtaining model parameters that meet performance requirements, deploy the lightweight student model to a local server, cloud platform, edge computing device, or urban traffic signal controller. The deployment system constructs model inputs based on real-time traffic detection data, completes three-dimensional spatiotemporal collaborative sampling and motion prediction, and converts the output motions into traffic light phase control commands, thereby achieving real-time collaborative control of traffic signals at multiple intersections.

[0160] In the above embodiments, the preferred number of three-dimensional spatiotemporal reference anchor points is four, but the present invention is not limited thereto. Without changing the core idea of ​​the present invention, the number of reference anchor points can be adjusted to other numbers based on the road network scale, computing resources, and control accuracy requirements; time lag set, maximum multi-hop search radius, Number of neighbors and Temperature parameters can also be adjusted based on traffic propagation speed, signal control cycle, and road network topology.

[0161] In the above embodiments, the offset network preferably uses only the current local hidden state of the intersection as input to reduce additional parameters and avoid introducing complex candidate scoring branches. However, in other embodiments, anchor point slot embedding, intersection type embedding, or lightweight topology hints can be introduced without departing from the concept of the present invention to enhance the distinguishability between different anchor point slots.

[0162] In the above embodiments, the differentiable interpolation on the local 3D spatiotemporal map preferably adopts the kNN / RBF form. This method is suitable for situations where the traffic network is not a regular image grid, and it enables continuous, differentiable, and interpretable weighted sampling among real candidate nodes. In other embodiments, similar differentiable interpolation can also be achieved using other monotonic distance decay functions or normalized weighting functions.

[0163] In summary, this invention improves the collaborative modeling capability and engineering applicability in multi-agent traffic signal control tasks by employing a three-dimensional spatiotemporal reference anchor mechanism, an anchor offset learning mechanism, and a local three-dimensional spatiotemporal graph differentiable interpolation sampling mechanism based on kNN / RBF, combined with the Decision Transformer's ability to model historical trajectory sequences in a temporal manner, the Cross-Attention's ability to fuse multi-intersection collaborative information, and the Attention Gate's ability to adaptively select the importance of external sampled information.

[0164] This invention can uniformly map multi-hop candidate intersections and their time-lag states under different scales and topologies into a fixed number of collaborative representations, avoiding the problem of inconsistent input dimensions caused by differences in the number of neighbors, while preserving multi-hop traffic propagation information and spatiotemporal dynamic correlation features.

[0165] Furthermore, this invention combines teacher model training, knowledge distillation, and online fine-tuning processes to enhance the information collaboration and environmental adaptability among various traffic intelligent agents while ensuring model computational efficiency and lightweight deployment. This effectively optimizes the overall operation of the intelligent transportation system, reduces vehicle delays and congestion, and provides technical support for the deployment of intelligent transportation systems in real multi-intersection scenarios.

[0166] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, or combinations made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A traffic signal cooperative control method based on deformable attention, wherein a control model is deployed in a local server, cloud platform, edge computing device or urban traffic signal controller in an intelligent transportation system; the deployed model outputs action prediction based on real-time traffic detection data, and converts the output action into traffic light phase control commands, thereby realizing real-time cooperative control of traffic signals at multiple intersections; Its characteristics are The deployment model is trained using knowledge distillation; the teacher mode is a multi-way collaborative control model, which includes a multi-head attention module, a cross-attention module, and an attention gate mechanism. The steps to obtain the deployment model include: Step 1. Obtain basic traffic signal control data for the target road network; Step 2. Construct historical traffic control trajectory data; The historical traffic control trajectory dataset is constructed based on the historical traffic operation records of the target road network. It consists of the status, signal control actions, traffic operation feedback quantities, and target reward RTG of multiple controlled intersections at continuous traffic control time steps. The historical trajectory of any intersection is represented as a sequence of target reports, traffic state observations, and signal control actions at multiple consecutive time steps, as follows: ， In the formula, Indicates intersection The historical trajectory; Indicates intersection At time step RTG; Indicates intersection At time step Traffic condition monitoring includes the queue length of each approach lane at the current intersection, the number of approaching vehicles in each approach lane, and the waiting time data for vehicles in each lane; Indicates intersection At time step The traffic signal control actions executed, and the selection of the traffic light phase at the next moment; Step 3. For The intersection is obtained through encoding using the shared decision transformer (DT) backbone network. Hidden state sequence , represented as: ， Current intersection The local representation is denoted as ; To construct a three-dimensional spatiotemporal sampling field, from Extracting lightweight time lag memory , as content features in the subsequent three-dimensional spatiotemporal candidate node set; Step 4. For the current decision intersection structure The multi-hop candidate set is represented as: ， in, For traffic network map Zhongcong intersection At the intersection The shortest graph distance; The maximum multi-hop search radius; Step 5. Construct a local 3D spatiotemporal candidate node set based on the multi-hop candidate set and the time lag layer, represented as: ， in, For time lag sets; each node-time lag pair Content characteristics are ; Step 6. For each node-delay pair Construct lightweight spacetime coordinates, represented as: ， in, Indicates candidate intersections Relative to the current intersection Spatial coordinates; Represents the normalized time layer coordinates; Step 7. Construct four three-dimensional spacetime reference anchor points. ,in The initialization is as follows: Current intersection exist Spatiotemporal position of a moment , Current moment Decision Crossroads Direct neighbor propagation reference location , The intersection is Reference position for propagation of time , Reference position for the propagation of more distant candidates in earlier time layers , used to describe the local spatiotemporal propagation reference location around the current intersection; This indicates that the corresponding candidate spatiotemporal coordinates are subjected to average pooling, max pooling, or weighted pooling. Step 8. For each anchor point Utilize the local hidden state of the current intersection Input shared offset network Generate a restricted offset, represented as: ， in, This is the offset scale factor. Used to limit the offset magnitude : It is a three-dimensional spacetime coordinate dimension; The restricted offset is divided into offsets corresponding to 4 anchor point slots. Then calculate the reference anchor point after the movement. , ; Step 9. For each moved reference anchor point In the three-dimensional spatiotemporal candidate node set Find the nearest Node-delay pair , ; Constructed from traffic network topology and time lag layer, with On a local 3D spatiotemporal graph of a set of nodes, using Differentiable sampling is performed in a specific form. The interpolation weights of the offset anchor points relative to neighboring true candidates are expressed as: ， Among them, molecules are the current candidates. Distance weights; the denominator is the sum of all nearby candidates. The distance weights are summed up and used for normalization; Indicates the first One reference anchor point; Indicates candidate nodes Coordinates in the three-dimensional spatiotemporal sampling space; This represents another candidate intersection index in the normalized denominator; Indicates distance offset from anchor point Recent One real-world spacetime candidate; Indicates the offset anchor point The squared Euclidean distance between the candidate spatiotemporal nodes; Indicates the sampling temperature; The smaller the value, the closer the sampling result is to "hard selection of the nearest node"; The larger the value, the smoother the weight distribution of neighboring candidate nodes; By using kNN / RBF with distance weights based on radial basis functions to perform differentiable weighted interpolation on neighboring candidate hidden states, the th... The collaborative representation of the sampling at each reference anchor point is as follows: ； Step 10. Hide the current intersection. As The four samples are represented together. As and The system integrates a Cross-Attention module for collaborative information fusion and uses an Attention gate mechanism to dynamically calculate the importance of each sampling slot for the current intersection decision, thus obtaining the final context representation. ; Step 11. Input action prediction head; For the At each time step, the action prediction head first obtains the action through a linear mapping. : ， ， in, Indicates the number of available traffic phases at the current intersection; This represents the action prediction head weight matrix; It is the final context representation; This indicates head bias in motion prediction; Then, through Get the current intersection At time step Select phase The probability is expressed as: ， This yields the probability distribution of actions at the current decision intersection at the corresponding time step. And the action probability of each candidate signal phase satisfies ; During reasoning, through Generate the current intersection exist Traffic signal phase actions at any given time, among which This indicates the last time step in the current input window; Under the conditions of satisfying traffic safety constraints and signal phase switching constraints, based on the action probability distribution Generate the current intersection exist Traffic signal phase actions at any given time; And transmits the signal control instructions to the signal controller or traffic signal control system at the corresponding intersection through the traffic signal control interface to generate or update the signal light phase control instructions for the current control time step; Step 12. Train the multi-way collaborative control model using active action loss and offset constraint loss; Current intersection The active loss is: ， In all valid Calculate the logarithmic probability of the actual signal phase action at the location. And take the average of the effective positions, plus the separately calculated... The average is then taken, and the negative sign is used as the loss to be minimized. in, express Effective Location set; express The Middle One sample; Indicates the time step position within a sample window; This indicates the phase actions actually performed in the historical traffic control trajectory; This represents the distribution of actions predicted by the model; Represents the entropy regularity coefficient; Indicates all Number of locations, including Location; Represents the entropy of action distribution; Current intersection The offset constraint loss is: ， in, This indicates the number of anchor points at each time step; Indicates the current intersection , The Middle Individual samples, time steps , No. Offset vector of anchor points , Represents three-dimensional spatiotemporal sampling coordinates; Indicates the first The squared norm of the anchor point offset vector. ; Adding the active action loss to the offset constraint loss yields the total loss used to train the model. ; yes Weight parameters; It is one or more of the following: action prediction loss, teacher model training loss, knowledge distillation loss, or online fine-tuning loss; Backpropagation is performed based on the total loss, and the trainable parameters in the multi-way cooperative control model are updated. Step 13. Using knowledge distillation, the multi-way collaborative control model trained in Step 12 is used as the teacher model to train a local collaborative Transformer model with a small parameter scale, and this model is used as the student model. Step 14. Fine-tune and evaluate the student model online: First, the network structure of the student model trained in step 13 is used as the basic network structure for the online fine-tuning stage, and online fine-tuning is performed; then, the model parameters of the current student model are fixed, and the student model is evaluated.

2. The traffic signal cooperative control method according to claim 1, characterized in that: The multi-way collaborative control model is a multi-hop collaborative teacher model, which includes: a multi-head attention module, a cross-attention module, and an attention gating mechanism; The multi-head attention module: The workflow is as follows: Trajectory data The raw data in First, perform embedding processing to map to the same hidden dimension. ; at each time step The corresponding input Add time and position embedding; for lengths of The historical trajectory is stacked into a sequence according to time steps. ,in Indicates intersection At the present moment Transformer input sequence, express quantity, Indicate each The hidden dimension; Will Input until available In a layered Transformer block, each layer of Transformer block contains An attention head, initial layer input Then the first Input of the Transformer block ; The Transformer block first normalizes the input, then uses multi-head attention to linearly project the input into a vector shape. ; For the For each attention head, in the process of calculating the attention score, a causal mask is first added. Then, through Obtain the attention weights; then, weight the results using the attention weights. , obtained the Single-head output of an attention head; The outputs of all attention heads are concatenated, and then the output of the Multi-Head Self-Attention Module (MHSA) is obtained through the output projection matrix. The MHSA output, after residual connection and layer normalization, enters the feedforward network sublayer to obtain the output of the corresponding Transformer Block. Finally, it passes through a multilayer perceptron. Feedforward network, to obtain the first The final output of the Transformerblock layer; have The Transformer blocks are stacked vertically in sequence for computation, and finally obtained ; Use actions Previous As the hidden state corresponding to the usage state position To predict actions, and ultimately obtain ; The complete process formula is as follows: ， After embedding: ， Add time and location embedding: ， Stacked into a sequence: ， Input into a multi-level Transformer: ， ， ， in The interior is: ， ， ， in, Indicating causal attention , Indicates the output projection matrix; Finally, take the corresponding time step. : ， get: ； The cross-attention module: The workflow for enabling the current decision intersection to read external collaborative information from the 3D spatiotemporal sampled collaborative representation is as follows: Hide the current intersection and the result obtained in step 9 As input to the cross-attention module; during the cross-attention module's computation, First, at the current intersection As ,get Four-sample co-representation As and ,get and ; Then, calculate the cross-attention and obtain... ; Finally, by performing residual connections and normalization, the current intersection is obtained. With the A candidate fusion representation calculated using cross-attention for each reference anchor point: ， ； Until all candidate fusion representations are obtained ; The attention gating mechanism: This is used to adaptively determine the importance weights of multiple candidate fused representations to the current signal control decision based on the current hidden state representation of the decision intersection. The workflow is as follows: Hide the current intersection and the cross-attention module obtained As input to the attention gating mechanism; during the computation of the attention gating mechanism, First, at the current intersection As ,get Four candidate fusion representations As and ,get and ; Then, calculate the current intersection for the first... Candidate fusion heads Fraction: ， Next, we obtain the context representation: ， Finally, residual connections and layer normalization are performed to obtain the final context representation from step 10: 。 3. The traffic signal cooperative control method according to claim 2, characterized in that, in step 13, the method for training the student model is as follows: Sub-trajectories from the historical traffic control trajectory dataset are input into the teacher and student models, and the teacher model outputs the teacher's actions. The student model outputs student actions. After both were normalized to their temperature coefficient T, the teacher's results were obtained. probability distribution Student strategy distribution ; Then, adopt Divergence measures the distributional difference between student and teacher strategies, and yields the knowledge distillation loss. Simultaneously, a Decision Transformer supervision loss based on actual signal phase action labels is introduced. Ultimately, the distillation training objective for the student model is a weighted sum of the two, as shown in the complete formula below: ， in, Indicates the current intersection A set of actions; This represents a temporary summation variable within the action set, used to sum the current action in the numerator. Distinguishing them does not imply any new or special actions; It represents the time step position in the trajectory sequence, not the entire trajectory itself; This indicates the teacher model's response to actions. Unnormalized scoring; Represents the student model's action Unnormalized scoring; Indicates the distillation temperature coefficient; Knowledge distillation loss : , The smaller the value, the closer the student model's output distribution is to the teacher model's output distribution; in, Indicates the first All valid intersections Location set; Indicates the number of valid positions; This indicates the distribution of teachers and students in the action. Differences in; Monitoring losses : Its purpose is to enable student models to give higher probabilities of actual signal phase actions in historical traffic control data. in, Labels representing actual actions recorded in historical traffic signal control records; This represents the student model's predicted probability of the actual signal phase action; Final distillation training objective : , and These are the weighting coefficients for knowledge distillation loss and supervision loss, respectively.

4. The traffic signal cooperative control method according to claim 1, characterized in that: In step 14, The steps for online fine-tuning of the student model include: First, let the student model have its current policy. As an online control strategy for the target road network, the intelligent transportation system receives the real-time traffic status of each controlled intersection during the actual operation or online trial operation of the target road network, and outputs corresponding signal phase control actions or control suggestions. Subsequently, based on the returned vehicle operation feedback information, a new online traffic control trajectory is generated. And obtain online control effectiveness evaluation data and target returns. ; Next, the newly acquired trajectories are added to the online playback dataset; if the playback dataset size exceeds a preset threshold, only high-quality or recent trajectories are retained to construct the online training set. ; During online fine-tuning, from Random sampling sub-trajectory As input to the student model; The evaluation steps for the student model include: At each control time step, the current observation status of each controlled intersection is collected; For any intersection Intelligent transportation systems, based on current status, historical actions, historical traffic operation feedback, and... Construct historical trajectory input and obtain the local hidden state representation of the current intersection through the Transformer backbone. The subsequent forward inference process of the model is consistent with steps 3 to 11; The intelligent transportation system (ITS) combines the signal phase actions of all controlled intersections into a joint control action and transmits it to the corresponding signal controller or traffic signal control system to drive the target road network to execute the corresponding signal control scheme in the next control time step. Traffic detection equipment, roadside sensing equipment, signal controllers, or traffic management platforms acquire traffic status and traffic operation feedback information for the next control time step. The traffic operation feedback information includes vehicle traffic conditions, queuing conditions, waiting time, travel time, delay time, and overall road network operation indicators. The ITS calculates the traffic control effectiveness evaluation quantity or target return based on the traffic operation feedback information. After a preset evaluation cycle, the ITS saves the vehicle travel information in the traffic network and the overall evaluation results for subsequent performance analysis.

5. The traffic signal cooperative control method according to claim 1, characterized in that: The model parameters for the teacher and student models are adopted. or Adaptive optimizer; temperature parameters are adopted The optimizer is updated separately; alternatively, both model parameters and temperature parameters are updated. The optimizer updates the trainable parameters.

6. The traffic signal cooperative control method according to claim 1, Its key feature is that during the pre-training phase of the teacher model, historical traffic control trajectories are used to pre-train the teacher model; this phase retains the core structure of the Decision Transformer and introduces a local 3D spatiotemporal graph differentiable sampling module; during training, the teacher model processes each valid historical window... Predict signal phase actions and use the actual signal phase actions in historical traffic control trajectories as the monitoring target; During the student model distillation stage, the teacher model is generated based on historical traffic control trajectories. The student model simultaneously learns the teacher's output distribution and the actual signal phase action labels of the historical trajectory; During the online fine-tuning phase of the student model, the student model receives real-time traffic status during the online operation or trial operation of the target road network, and forms new online traffic control trajectories based on traffic operation feedback. These trajectories are then added to the online training set or replay pool, and sub-trajectories are sampled from them for training. During the online fine-tuning phase, only the Adapter module, action output layer, and lightweight collaboration-related parameters are updated, or more parameters can be opened for joint fine-tuning according to deployment requirements; In each stage, the model generates the action distribution through forward propagation, calculates the corresponding loss, obtains the gradient through backpropagation, and updates the trainable parameters by the optimizer until the preset number of training steps is reached or the stopping condition is met. Effectiveness evaluation: After the model is trained, it is evaluated in multiple rounds based on historical playback data, online trial operation data or actual operation data of the target road network, and the model performance is evaluated by traffic operation indicators. Evaluation metrics include average latency, expressed as ,in This indicates the number of vehicles that completed the trip. Indicates vehicle In a transportation network, the average delay is used to characterize the additional congestion of vehicles within the network. During the evaluation process, interpretability information of the multi-hop sampling module is recorded, including the real intersections sampled by the offset anchor points, the corresponding time lag layers, the sampling weight entropy, the offset norm, and the proportion of direct neighbor slots, in order to analyze the model's utilization of multi-hop spatiotemporal collaborative information.

7. The traffic signal cooperative control method according to claim 1, characterized in that... After obtaining the optimal model parameters, the trained student model will be used as the final deployment model; during deployment, First, based on the topology of the target road network, the intersection adjacency relationship, the multi-hop candidate set, and the candidate spatiotemporal coordinates are pre-constructed; Then, at each traffic control time step, the status of each intersection is received from traffic detectors, roadside sensing devices, signal controllers, traffic management platforms, or vehicle-road cooperative system interfaces; the intersection status includes queue length, waiting time, vehicle arrival status, lane occupancy rate, and current signal phase information; For any intersection First, according to recent Historical state, historical actions, and historical traffic operation feedback at each time step Construct model input; The local hidden state representation is obtained through forward propagation of the student model, and the final fused representation is generated based on the 3D spatiotemporal multi-hop sampling information. The action prediction head outputs the phase action at the current moment based on the final fused representation, and the traffic controller converts the action into actual traffic light phase control commands; During deployment, the intelligent transportation system is equipped with a safety fallback mechanism: When multi-hop candidate information is missing, neighbor communication is unavailable, sampling weights are abnormal, or the multi-hop cooperative control strategy fails to meet the preset performance improvement conditions in the recent evaluation period, it will automatically fall back to the direct neighbor cooperative control mode. Multi-hop collaborative reasoning is only enabled when the multi-hop collaborative strategy consistently outperforms the direct neighbor strategy.

8. The traffic signal cooperative control method according to claim 1, characterized in that: In step 1, the basic data includes the topological connection relationship of each controlled intersection in the target road network, lane structure, signal phase, intersection spatial coordinates, vehicle operating status collected by traffic detection equipment, and historical signal control records. At each traffic control time step, acquire the traffic status, current signal phase, historical control actions, and traffic operation feedback information for each intersection at the current time step; among which: Traffic conditions include queue length, lane occupancy, vehicle waiting time, and number of approaching vehicles; Traffic operation feedback information includes vehicle traffic status, waiting time, delay time, and road network efficiency.

9. The traffic signal cooperative control method according to claim 1, characterized in that: In step 3, the hidden state representations of the current time, the previous time, and the time before that are extracted to construct a time-lag memory; for general cases, the time lag is used... ; In step 4, the maximum multi-hop search radius Set to 3, then the collection contains Candidates, when the moved reference anchor point is primarily sampled, fall on When candidate nodes are selected, the method degenerates into a direct neighbor cooperative control mode.

10. The traffic signal cooperative control method according to claim 1, characterized in that, In step 6, a) If the actual road network coordinates are obtained, the normalized relative positions of the actual road network coordinates should be used first, represented as: This applies when the actual two-dimensional coordinates of each intersection are obtained, where, Indicates the current decision-making crossroads The actual coordinates on the map; Indicates candidate intersections The actual coordinates on the map; This indicates the lateral displacement of the candidate intersection relative to the current intersection. This indicates the longitudinal displacement of the candidate intersection relative to the current intersection. Indicates the normalization scale; b) If the true road network coordinates are unavailable, use topological coordinates based on graph distance. Distance to describe candidate intersections Distance from the current intersection How far; the topological coordinates based on graph distance are represented as: .

Images