A traffic signal control method and device based on space-time attention and safety constraints

Abstract

The application relates to a traffic signal control method based on space-time attention and safety constraints, comprising the following steps: acquiring real-time traffic state data of a traffic intersection and constructing a state vector; constructing a space-time graph convolution network, predicting traffic flow and obtaining a future traffic flow prediction value; using a space-time attention gate mechanism with conservative cold start to dynamically weight the confidence of the future traffic flow prediction value, generating an enhanced state representation; outputting an optimal signal phase action; performing legality verification and logic correction on the optimal signal phase action to output a safe execution signal phase action; constructing a composite reward function to calculate a multi-objective composite reward value, and using the composite reward value to update the weight parameters of a double-input deep Q network, realizing real-time closed-loop adaptive control of a road network signal. The application improves the physical executability, training stability, forward-looking response capability and control adaptability in complex traffic scenarios of the traffic signal control strategy.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a traffic signal control method and device based on spatiotemporal attention and safety constraints. Background Technology

[0002] With the continuous advancement of urbanization, the scale of urban road traffic flow is constantly increasing, leading to increasingly prominent problems such as traffic congestion, declining intersection efficiency, and insufficient priority vehicle passage. Traffic signal control, as a crucial component of urban traffic management, directly impacts vehicle delays, queue lengths, road network operational efficiency, and traffic safety. Existing traffic signal control methods mainly include fixed-time control, inductive control, and adaptive control. Fixed-time control primarily sets signal timing parameters based on historical traffic statistics, making it difficult to adapt to real-time fluctuations in traffic flow. Inductive control can dynamically adjust based on local detection data, but it typically focuses on single intersections or localized areas, struggling to address global coordination in complex road networks. With the development of artificial intelligence technology, traffic signal control methods based on deep reinforcement learning are gaining attention. These methods can learn control strategies through interaction with the traffic environment, exhibiting strong adaptability in complex dynamic scenarios.

[0003] However, existing traffic signal control methods based on deep reinforcement learning still have the following shortcomings in practical applications: First, existing methods usually make reactive decisions based on the current traffic state, failing to fully utilize future traffic flow evolution trends. Even when predictive models are introduced, they often lack dynamic adjustment mechanisms for the reliability of prediction results, easily leading to prediction noise directly interfering with control decisions. Second, the control actions output by reinforcement learning agents in existing methods usually lack explicit physical safety constraints, easily generating actions that do not conform to the signal controller's execution rules during the training and exploration phase, affecting the stability of algorithm training and the feasibility of subsequent engineering deployment. Third, the reward functions of existing methods are mostly constructed around local indicators such as the current queue length and waiting time, lacking forward-looking guidance for traffic flow and failing to meet the needs of green wave coordination, overflow suppression, and priority vehicle passage. Fourth, in complex scenarios such as high-saturation traffic flow, sudden disturbances, or priority vehicle intervention, existing methods often lack targeted handling mechanisms for extreme states, resulting in insufficient robustness and adaptability of the control strategy.

[0004] Therefore, there is an urgent need to provide a traffic signal control method that can comprehensively utilize future traffic flow prediction information, meet traffic signal execution safety constraints, and balance control efficiency with adaptability to complex scenarios, so as to improve the real-time performance, executability and comprehensive control performance of traffic signal control in complex road network environments. Summary of the Invention

[0005] To address the problems of insufficient utilization of predictive features, lack of physical safety constraints, insufficient foresight of reward mechanisms, and insufficient robustness of control in high-saturation traffic flow scenarios in existing traffic signal control methods, this invention provides a deep reinforcement learning-based traffic signal control method and device based on spatiotemporal attention and safety constraints. This method improves the responsiveness of traffic signal control strategies to complex dynamic traffic flows by combining traffic state acquisition, future traffic flow prediction, spatiotemporal attention weighting, conservative cold-start gating fusion, dual-input deep Q-network decision-making, finite state machine action projection, and composite reward closed-loop optimization. At the same time, it enhances the physical executability of output actions and the control adaptability in high-saturation traffic scenarios.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a traffic signal control method based on spatiotemporal attention and safety constraints, the method comprising the following sequential steps:

[0007] (1) Obtain real-time traffic status data at traffic intersections and construct state vectors based on the real-time traffic status data. ;

[0008] (2) Construct a spatiotemporal graph convolutional network based on real-time traffic state data and state vectors. Constructing tensors , tensor Input a spatiotemporal graph convolutional network to predict traffic flow and obtain future traffic volume forecasts. ;

[0009] (3) Using a spatiotemporal attention gating mechanism with conservative cold start, the future traffic flow prediction value is obtained. Dynamic confidence weighting is applied to generate enhanced state representations. ;

[0010] (4) Transfer the state vector and enhanced state representation Input a dual-input deep Q-network and output the optimal signal phase action. ;

[0011] (5) Through the action projection layer based on finite state machine, the action is performed at the optimal signal phase. Before issuing the execution order, the optimal signal phase is activated. Perform legality checks and logical corrections, and output a safe execution signal phase action;

[0012] (6) Construct a composite reward function Calculate the multi-objective composite reward value, and use the multi-objective composite reward value to update the weight parameters of the dual-input deep Q network, so that the phase action of the signal output by the model is gradually optimized.

[0013] (7) Repeat steps (1) to (6) to achieve real-time closed-loop adaptive control of road network signals.

[0014] Step (1) specifically refers to: using a preset time granularity Collect real-time status data of the traffic network and divide the traffic network into... Each intersection node is constructed. state vector The dimension of the state vector is State vector This includes the lane queue length, average vehicle waiting time, vehicle arrival rate, current phase duration, average vehicle speed, and intersection queue overflow coefficient for phase A, as well as the lane queue length, average vehicle waiting time, vehicle arrival rate, current phase duration, average vehicle speed, and intersection queue overflow coefficient for phase B.

[0015] 3. The traffic signal control method based on spatiotemporal attention and safety constraints according to claim 1, characterized in that: step (2) specifically refers to: the spatiotemporal graph convolutional network includes two stacked spatiotemporal convolutional blocks, namely a first spatiotemporal convolutional block and a second spatiotemporal convolutional block, the first spatiotemporal convolutional block and the second spatiotemporal convolutional block have the same structure, the first spatiotemporal convolutional block includes a first temporal convolutional layer, a first spatial graph convolutional layer and a second temporal convolutional layer; the second spatiotemporal convolutional block includes a third temporal convolutional layer, a second spatial graph convolutional layer and a fourth temporal convolutional layer;

[0016] Based on real-time traffic status data and state vectors Constructing a four-dimensional tensor ,in, For batch size, For historical steps, This represents the number of intersection nodes. The input feature dimension has a value of 2, representing the vehicle arrival rate at phases A and B. From the state vector Extracted from;

[0017] The processing method for the first spatiotemporal convolutional block is as follows:

[0018] The first convolutional layer uses a two-dimensional convolutional kernel. tensor Perform time-dimension convolution, converting the input feature dimension Map from 2 to Output the first intermediate tensor Capture the local temporal correlation of traffic within a single node;

[0019] The first spatial graph convolutional layer receives the first intermediate tensor. Based on the preset road network topology adjacency matrix ,use Chebyshev polynomials are used to perform spectral domain graph convolution to extract the spatial coupling features between the current node and its multi-level neighboring nodes in the road network, i.e., the spatial coupling feature tensor. The dimension of the spatial coupling feature remains 64;

[0020] The second time convolutional layer uses a two-dimensional convolutional kernel. Spatial Coupled Feature Tensor Perform a time-axis sliding convolution again to further compress temporal redundancy and enhance the nonlinear representation of cross-dimensional features, outputting the first feature tensor. ;

[0021] The processing method for the second spatiotemporal convolutional block is as follows:

[0022] The third convolutional layer uses a two-dimensional convolutional kernel. For the first characteristic tensor Perform convolutions along the time dimension while maintaining the feature dimension as 1. Output the second intermediate tensor This further captures the local temporal correlation of traffic within a single node;

[0023] The second spatial graph convolutional layer receives the second intermediate tensor. Based on the preset road network topology adjacency matrix ,use Chebyshev polynomials are used to perform spectral domain graph convolution; deep mining of cross-node propagation paths and diffusion effects of congestion in road networks under time lag is performed, and a third intermediate tensor is output. The third intermediate tensor The dimension remains 64;

[0024] The fourth temporal convolutional layer uses a two-dimensional convolutional kernel. For the third intermediate tensor Perform the time-axis sliding convolution again to change the dimension of the feature channels from... Dimensional expansion to Output the second feature tensor This further reduces time redundancy and enhances the nonlinear expression of cross-dimensional features, enabling abstract semantic representation of deep features;

[0025] The second characteristic tensor The data is flattened and input into a fully connected projection layer. Through linear and nonlinear mapping, the high-dimensional spatiotemporal features are restored to the target dimension, and the predicted future traffic flow is output. ,in, .

[0026] Step (3) specifically includes the following steps in sequence:

[0027] (3a) Perform time-attention sharpening: for future traffic flow forecasts Along the spatial node dimension and input feature dimension Global average pooling is performed to obtain a four-dimensional tensor. This four-dimensional tensor is then reduced to a two-dimensional matrix reflecting the intensity of flow evolution over time. Subsequently, a multilayer perceptron with a single hidden layer is used to map the features of each prediction time step to the original time score. ,in, , Implement a temperature sharpening strategy: introduce a time-temperature coefficient. Original time score Scaling is performed, and the temporal attention weights at different time steps within the prediction window are calculated using the Softmax function. :

[0028]

[0029] In the formula, The value is 0.35; Index for summation at time steps; For the first The raw score corresponding to each prediction time step;

[0030] (3b) Spatial attention weighting: For future traffic flow forecasts along and feature dimensions Global average pooling is performed to extract spatial activity features of each intersection node; a multilayer perceptron is then used to map the spatial activity features to the raw spatial scores of each node. , , The number of intersection nodes is determined; a temperature sharpening strategy is implemented, introducing a spatial temperature coefficient. Scoring of the original space Scaling is performed, and the spatial attention weights of each intersection node are calculated using the Softmax function. :

[0031]

[0032] In the formula, The value is 0.35; Sum index for nodes; For the first The original spatial score corresponding to each intersection node;

[0033] (3c) Conservative initialization gating fusion: utilizing temporal attention weights Spatial attention weights Forecast of future traffic flow The aggregated features are obtained by performing a weighted summation. Mapped to an embedding vector by a projection layer Introducing a gating unit ,in, It is the Sigmoid activation function. The learnable weight matrix for the gated unit. For gated bias term and Initialize to a negative value, taking the value -1.2, so that the gated output is less than 0.5 in the early stage of training;

[0034] (3d) Generate enhanced state representation To achieve conservative cold start and adaptive fusion of predicted features:

[0035] ;

[0036] in, This indicates element-wise multiplication.

[0037] Step (4) specifically includes the following steps in sequence:

[0038] (4a) Improve the deep Q network to obtain a dual-input deep Q network: replace the single input layer of the deep Q network with a dual-input layer, which includes a first input layer and a second input layer, and the input of the first input layer is the state vector. The input to the second input layer is the augmented state representation. The first input layer is followed by a first fully connected layer and a second fully connected layer with hidden dimensions of 128 and 64, respectively. The output of the first fully connected layer is the feature. The output of the second fully connected layer is the feature. The second input layer is followed by a third fully connected layer and a fourth fully connected layer with hidden dimensions of 128 and 64, respectively. The output of the third fully connected layer is the feature. The output of the fourth fully connected layer is the feature. ;

[0039] After the second and fourth fully connected layers and before the hidden layers of the deep Q-network, a feature fusion layer is added. This feature fusion layer processes the features... Perform dimensional alignment, weighted fusion, and noise reduction enhancement to output fused features. ;

[0040] (4b) Fusion characteristics The Q-value vectors of each action are output through a linear output layer. ,in, These are the learnable weight parameters for a two-input deep Q-network; This is the real-time state vector of the current traffic intersection. Traffic signal phase operation;

[0041] (4c) Adopt The strategy involves selecting actions within a phase selection set {0, 1}, where 0 corresponds to phase A (green light) and 1 corresponds to phase B (green light). When the random number is less than the exploration rate... Actions are randomly selected at any time, when the random number is greater than or equal to the exploration rate. When the Q value is maximized, the action with the highest Q value is selected as the optimal signal phase action. :

[0042] ;

[0043] In the formula, Indicates that The action with the highest value ;

[0044] Exploration rate The initial value is 1.0, which decays to a minimum value of 0.02 with an attenuation coefficient of 0.998.

[0045] Step (5) refers to: maintaining the finite state machine for each intersection node, wherein the state variables of the finite state machine include the current green light phase, green light start time, whether it is in the yellow light period, and yellow light end time; and performing multi-level safety constraint judgments on the phase action output by the dual-input deep Q network based on the state variables of the finite state machine, wherein the multi-level safety constraint judgments include:

[0046] (5a) Yellow light transition constraint: When the current state is in the yellow light period, the current yellow light phase is maintained until the simulation time reaches the yellow light end time;

[0047] (5b) Maximum green light time constraint: When the current green light duration reaches the maximum green light time threshold, the phase action is forcibly switched to the yellow light transition process;

[0048] (5c) Minimum green light time constraint: When the current green light duration does not reach the minimum green light time threshold and the phase action is inconsistent with the current green light phase, the current green light phase remains unchanged;

[0049] (5d) Normal phase switching: When the yellow light over-constraint, the maximum green light time constraint, and the minimum green light time constraint are all satisfied and the phase action is inconsistent with the current green light phase, the yellow light phase command is output first, and the target green light phase is switched after the yellow light ends, forming an executable signal phase action.

[0050] In step (6), the composite reward function for:

[0051] ;

[0052] in, Rewards for efficiency For non-linear safety penalties, To predict consistency-oriented rewards, i.e., positive rewards. Rewards for priority vehicles;

[0053] Efficiency Rewards The formula is:

[0054] ;

[0055] in, All are weighting coefficients; This represents the sum of the queue lengths for all lanes at the intersection. The average waiting time for all vehicles; Represents the arithmetic mean function;

[0056] When any lane queue length Exceeding the overflow threshold At that time, apply nonlinear security penalties By using a forced model to prioritize traffic flow at intersections with high spillover risk, road network deadlock can be mitigated.

[0057] (1);

[0058] (2);

[0059] In the formula, This is the overflow penalty coefficient. For exponential scaling factor, The coefficient is a power factor; Equation (1) is applicable to extreme congestion scenarios, when the lane queue length is... When approaching the physical capacity limit of the intersection, choose equation (1); equation (2) is suitable for normal congestion scenarios, when the queue length of each lane at the intersection is... Exceeding the preset threshold However, it has not yet reached a state where vehicles behind cannot enter at all. Choose equation (2); calculate the phase difference of the future predicted flow. Difference between current queue pressure :

[0060] ;

[0061] ;

[0062] In the formula, , These represent the total number of vehicles arriving in phases A and B respectively within the future forecast period; , These are the real-time total queue lengths for the lanes corresponding to phase A and phase B at the current time, respectively.

[0063] like and If the confidence threshold is exceeded, a positive reward is given. Otherwise, give a negative reward or zero reward, or a positive reward. The formula is:

[0064] ;

[0065] in, The confidence coefficient is calculated based on the predicted total flow and variance. As a guide for reward weighting;

[0066] A priority vehicle reward system will be introduced for emergency vehicles and public transport priority vehicles. :

[0067] ;

[0068] in, This represents the number of emergency vehicles passing through during the current control period. This is the total cumulative waiting time for all emergency vehicles within the intersection; This represents the total number of emergency vehicles queuing at the intersection. This indicates a congestion signal indicating the presence of an emergency vehicle that has not received a valid clearance phase. This is the traffic efficiency gain coefficient; For special vehicle delay sensitivity coefficient; Clear the weight coefficients for the path; This is the phase conflict suppression coefficient.

[0069] Another object of the present invention is to provide an electronic device comprising:

[0070] Processor; and

[0071] A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the traffic signal control method based on spatiotemporal attention and safety constraints as described above.

[0072] The present invention also provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the traffic signal control method based on spatiotemporal attention and safety constraints as described above.

[0073] As can be seen from the above technical solution, the beneficial effects of the present invention are as follows: First, the present invention achieves hard safety of traffic control actions: by introducing an action projection layer based on a finite state machine, the desired action output by the dual-input deep Q network is forcibly mapped to a safe action that conforms to physical constraints. The present invention no longer relies on soft rewards and penalties, but completely eliminates violations such as short green lights and missing yellow lights through hard logic interception, ensuring the safe deployment capability of the system in a real road network environment; Second, it innovatively proposes a conservative cold start gating mechanism, which significantly improves the model stability: by adjusting the bias term of the gating unit... Initializing to negative values ​​artificially suppresses the pass rate of unreliable predictions in the early stages of training. This phased evolutionary strategy of first learning real-time traffic conditions and then fusing prediction data improves the convergence speed and initial stability of the reinforcement learning algorithm by more than 20% compared to traditional methods. Third, it possesses forward-looking intelligent green wave coordination capabilities: This invention utilizes a spatiotemporal graph convolutional network to capture traffic trends over the next 15 minutes and guides the agent to make decisions that align with future traffic flow trends through prediction consistency rewards. Compared to traditional methods, this invention can predict the arrival time of convoys at intersections and spontaneously form dynamic green wave belts, significantly reducing the number of times vehicles stop on main roads. Fourth, it introduces a temperature-sharpened multi-dimensional spatiotemporal attention mechanism to achieve precise perception: It can accurately pinpoint key peak times and congestion source intersections from complex road network data, avoiding the information overload and feature dilution problems common in multi-intersection control. Fifth, it provides extreme robustness in extreme scenarios and responsiveness for special vehicles: This invention has stronger anti-deadlock capabilities when facing oversaturated traffic flow, and its proprietary priority vehicle reward... This ensures that ambulances, fire trucks, and other special vehicles can obtain the highest priority phase release rights under any circumstances, reflecting the system's extremely high level of humanized and intelligent management. Attached Figure Description

[0074] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0075] like Figure 1 As shown, a traffic signal control method based on spatiotemporal attention and safety constraints includes the following sequential steps:

[0076] (1) Obtain real-time traffic status data at traffic intersections and construct state vectors based on the real-time traffic status data. ;

[0077] (2) Construct a spatiotemporal graph convolutional network based on real-time traffic state data and state vectors. Constructing tensors , tensor Input a spatiotemporal graph convolutional network to predict traffic flow and obtain future traffic volume forecasts. ;

[0078] (3) Using a spatiotemporal attention gating mechanism with conservative cold start, the future traffic flow prediction value is obtained. Dynamic confidence weighting is applied to generate enhanced state representations. ;

[0079] (4) Transfer the state vector and enhanced state representation Input a dual-input deep Q-network and output the optimal signal phase action. ;

[0080] (5) Through the action projection layer based on finite state machine, the action is performed at the optimal signal phase. Before issuing the execution order, the optimal signal phase is activated. Perform legality checks and logical corrections, and output a safe execution signal phase action;

[0081] (6) Construct a composite reward function Calculate the multi-objective composite reward value, and use the multi-objective composite reward value to update the weight parameters of the dual-input deep Q network, so that the phase action of the signal output by the model is gradually optimized.

[0082] (7) Repeat steps (1) to (6) to achieve real-time closed-loop adaptive control of road network signals.

[0083] Step (1) specifically refers to: using a preset time granularity Collect real-time status data of the traffic network and divide the traffic network into... Each intersection node is constructed. state vector The dimension of the state vector is State vector This includes the lane queue length, average vehicle waiting time, vehicle arrival rate, current phase duration, average vehicle speed, and intersection queue overflow coefficient for phase A, as well as the lane queue length, average vehicle waiting time, vehicle arrival rate, current phase duration, average vehicle speed, and intersection queue overflow coefficient for phase B. The preferred time is 60 seconds. Nine intersections in the road network adopt bidirectional green wave control with phase difference coordination. The traffic lights at each intersection switch asynchronously, and traffic is released sequentially in the west → east and north → south directions to improve the rationality and authenticity of the green wave.

[0084] Step (2) specifically refers to: the spatiotemporal graph convolutional network includes two stacked spatiotemporal convolutional blocks, namely the first spatiotemporal convolutional block and the second spatiotemporal convolutional block. The first spatiotemporal convolutional block and the second spatiotemporal convolutional block have the same structure. The first spatiotemporal convolutional block includes a first temporal convolutional layer, a first spatial graph convolutional layer and a second temporal convolutional layer; the second spatiotemporal convolutional block includes a third temporal convolutional layer, a second spatial graph convolutional layer and a fourth temporal convolutional layer.

[0085] Based on real-time traffic status data and state vectors Constructing a four-dimensional tensor ,in, For batch size, For historical steps, This represents the number of intersection nodes. The input feature dimension has a value of 2, representing the vehicle arrival rate at phases A and B. From the state vector Extracted from;

[0086] The processing method for the first spatiotemporal convolutional block is as follows:

[0087] The first convolutional layer uses a two-dimensional convolutional kernel. tensor Perform time-dimension convolution, converting the input feature dimension Map from 2 to Output the first intermediate tensor Capture the local temporal correlation of traffic within a single node; ;

[0088] The first spatial graph convolutional layer receives the first intermediate tensor. Based on the preset road network topology adjacency matrix ,use Chebyshev polynomials are used to perform spectral domain graph convolution to extract the spatial coupling features between the current node and its multi-level neighboring nodes in the road network, i.e., the spatial coupling feature tensor. The dimension of the spatial coupling feature remains 64;

[0089] The second time convolutional layer uses a two-dimensional convolutional kernel. Spatial Coupled Feature Tensor Perform a time-axis sliding convolution again to further compress temporal redundancy and enhance the nonlinear representation of cross-dimensional features, outputting the first feature tensor. ;

[0090] The processing method for the second spatiotemporal convolutional block is as follows:

[0091] The third convolutional layer uses a two-dimensional convolutional kernel. For the first feature tensor Perform convolutions along the time dimension while maintaining the feature dimension as 1. Output the second intermediate tensor This further captures the local temporal correlation of traffic within a single node;

[0092] The second spatial graph convolutional layer receives the second intermediate tensor. Based on the preset road network topology adjacency matrix ,use Chebyshev polynomials are used to perform spectral domain graph convolution; deep mining of cross-node propagation paths and diffusion effects of congestion in road networks under time lag is performed, and a third intermediate tensor is output. The third intermediate tensor The dimension remains 64;

[0093] The fourth temporal convolutional layer uses a two-dimensional convolutional kernel. For the third intermediate tensor Perform the time-axis sliding convolution again to change the dimension of the feature channels from... Dimensional expansion to Output the second feature tensor This further reduces time redundancy and enhances the nonlinear expression of cross-dimensional features, enabling abstract semantic representation of deep features;

[0094] The second characteristic tensor The data is flattened and input into a fully connected projection layer. Through linear and nonlinear mapping, the high-dimensional spatiotemporal features are restored to the target dimension, and the predicted future traffic flow is output. ,in, It will cover the next 15 minutes.

[0095] Step (3) specifically includes the following steps in sequence:

[0096] (3a) Perform time-attention sharpening: for future traffic flow forecasts Along the spatial node dimension and input feature dimension Global average pooling is performed to obtain a four-dimensional tensor. This four-dimensional tensor is then reduced to a two-dimensional matrix reflecting the intensity of flow evolution over time. Subsequently, a multilayer perceptron with a single hidden layer is used to map the features of each prediction time step to the original time score. ,in, , Implement a temperature sharpening strategy: introduce a time-temperature coefficient. Original time score Scaling is performed, and the temporal attention weights at different time steps within the prediction window are calculated using the Softmax function. :

[0097]

[0098] In the formula, The value is 0.35; Index for summation at time steps; For the first The original score corresponding to each prediction time step;

[0099] (3b) Spatial attention weighting: For future traffic flow forecasts along and feature dimensions Global average pooling is performed to extract spatial activity features of each intersection node; a multilayer perceptron is then used to map the spatial activity features to the raw spatial scores of each node. , , The number of intersection nodes is determined; a temperature sharpening strategy is implemented, introducing a spatial temperature coefficient. Scoring of the original space Scaling is performed, and the spatial attention weights of each intersection node are calculated using the Softmax function. :

[0100]

[0101] In the formula, The value is 0.35; Sum index for nodes; For the first The original spatial score corresponding to each intersection node;

[0102] (3c) Conservative initialization gating fusion: utilizing temporal attention weights Spatial attention weights Forecast of future traffic flow The aggregated features are obtained by performing a weighted summation. Mapped to an embedding vector by a projection layer Introducing a gating unit ,in, It is the Sigmoid activation function. The learnable weight matrix for the gated unit. For gated bias term and Initialize to a negative value, taking the value -1.2, so that the gated output is less than 0.5 in the early stage of training;

[0103] (3d) Generate enhanced state representation To achieve conservative cold start and adaptive fusion of predicted features:

[0104] ;

[0105] in, This indicates element-wise multiplication.

[0106] Step (4) specifically includes the following steps in sequence:

[0107] (4a) Improve the deep Q network to obtain a dual-input deep Q network: replace the single input layer of the deep Q network with a dual-input layer, which includes a first input layer and a second input layer, and the input of the first input layer is the state vector. The input to the second input layer is the augmented state representation. The first input layer is followed by a first fully connected layer and a second fully connected layer with hidden dimensions of 128 and 64, respectively. The output of the first fully connected layer is the feature. The output of the second fully connected layer is the feature. The second input layer is followed by a third fully connected layer and a fourth fully connected layer with hidden dimensions of 128 and 64, respectively. The output of the third fully connected layer is the feature. The output of the fourth fully connected layer is the feature. ;

[0108] After the second and fourth fully connected layers and before the hidden layers of the deep Q-network, a feature fusion layer is added. This feature fusion layer processes the features... Perform dimensional alignment, weighted fusion, and noise reduction enhancement to output fused features. ;

[0109] (4b) Fusion characteristics The Q-value vectors of each action are output through a linear output layer. ,in, These are the learnable weight parameters for a two-input deep Q-network; This is the real-time state vector of the current traffic intersection. Traffic signal phase operation;

[0110] (4c) Adopt The strategy involves selecting actions within a phase selection set {0, 1}, where 0 corresponds to phase A (green light) and 1 corresponds to phase B (green light). When the random number is less than the exploration rate... Actions are randomly selected at any time, when the random number is greater than or equal to the exploration rate. When the Q value is maximized, the action with the highest Q value is selected as the optimal signal phase action. :

[0111] ;

[0112] In the formula, Indicates that The action with the highest value ;

[0113] Exploration rate The initial value is 1.0, which decays to a minimum value of 0.02 with an attenuation coefficient of 0.998.

[0114] Step (5) refers to: maintaining the finite state machine for each intersection node, wherein the state variables of the finite state machine include the current green light phase, green light start time, whether it is in the yellow light period, and yellow light end time; and performing multi-level safety constraint judgments on the phase action output by the dual-input deep Q network based on the state variables of the finite state machine, wherein the multi-level safety constraint judgments include:

[0115] (5a) Yellow light transition constraint: When the current state is in the yellow light period, the current yellow light phase is maintained until the simulation time reaches the yellow light end time;

[0116] (5b) Maximum green light time constraint: When the current green light duration reaches the maximum green light time threshold, the phase action is forcibly switched to the yellow light transition process;

[0117] (5c) Minimum green light time constraint: When the current green light duration does not reach the minimum green light time threshold and the phase action is inconsistent with the current green light phase, the current green light phase remains unchanged;

[0118] (5d) Normal phase switching: When the yellow light over-constraint, the maximum green light time constraint, and the minimum green light time constraint are all satisfied and the phase action is inconsistent with the current green light phase, the yellow light phase command is output first, and the target green light phase is switched after the yellow light ends, forming an executable signal phase action.

[0119] In step (6), the composite reward function for:

[0120] ;

[0121] in, As an efficiency reward, it is used to characterize the total length of lane queues at intersections and the average waiting time for vehicles; This is a non-linear safety penalty used to impose a penalty on high overflow risk states when the queue length of any lane exceeds the overflow threshold. For prediction consistency-oriented rewards, rewards or penalties are given based on the consistency relationship between the future predicted flow phase difference and the current queue pressure difference; The reward for priority vehicles is calculated using a weighted average based on the number of priority vehicles passing through, their cumulative waiting time, queue length, and congestion status.

[0122] Efficiency Rewards The formula is:

[0123] ;

[0124] in, All are weighting coefficients; This represents the sum of the queue lengths for all lanes at the intersection. The average waiting time for all vehicles; Represents the arithmetic mean function;

[0125] When any lane queue length Exceeding the overflow threshold At that time, apply nonlinear security penalties By using a forced model to prioritize traffic flow at intersections with high spillover risk, road network deadlock can be mitigated.

[0126] (1);

[0127] (2);

[0128] In the formula, This is the overflow penalty coefficient. For exponential scaling factor, The coefficient is a power factor; Equation (1) is applicable to extreme congestion scenarios, when the lane queue length is... When approaching the physical capacity limit of the intersection, choose equation (1); equation (2) is suitable for normal congestion scenarios, when the queue length of each lane at the intersection is... Exceeding the preset threshold However, it has not yet reached a state where vehicles behind cannot enter at all. Choose equation (2); calculate the phase difference of the future predicted flow. Difference between current queue pressure :

[0129] ;

[0130] ;

[0131] In the formula, , These represent the total number of vehicles arriving in phases A and B respectively within the future forecast period; , These are the real-time total queue lengths for the lanes corresponding to phase A and phase B at the current time, respectively.

[0132] like and If the confidence threshold is exceeded, a positive reward is given. Otherwise, give a negative reward or zero reward, or a positive reward. The formula is:

[0133] ;

[0134] in, The confidence coefficient is calculated based on the predicted total flow and variance. As a guide for reward weighting;

[0135] A priority vehicle reward system will be introduced for emergency vehicles and public transport priority vehicles. :

[0136] ;

[0137] in, This represents the number of emergency vehicles passing through during the current control period. This is the total cumulative waiting time for all emergency vehicles within the intersection; This represents the total number of emergency vehicles queuing at the intersection. This indicates a congestion signal indicating the presence of an emergency vehicle that has not received a valid clearance phase. This is the traffic efficiency gain coefficient; For special vehicle delay sensitivity coefficient; Clear the weight coefficients for the path; This is the phase conflict suppression coefficient.

[0138] Priority vehicle rewards This invention enables high-priority vehicles to pass through while ensuring overall traffic efficiency and safety.

[0139] In summary, this invention achieves hard safety for traffic control actions: it no longer relies on soft rewards and penalties, but instead uses hard logic to completely eliminate violations such as short green lights and missing yellow lights, ensuring the system's safe deployment capability in real road network environments; it also innovatively proposes a conservative cold-start gating mechanism, significantly improving model stability: by adjusting the gating unit bias term... Initializing to negative values ​​artificially suppresses the pass rate of unreliable predictions in the early stages of training. This phased evolutionary strategy of first learning real-time traffic conditions and then fusing prediction data improves the convergence speed and initial stability of the reinforcement learning algorithm by more than 20% compared to traditional methods. It possesses forward-looking intelligent green wave coordination capabilities: this invention utilizes a spatiotemporal graph convolutional network to capture traffic trends over the next 15 minutes and guides the agent to make decisions that align with future traffic flow trends through prediction consistency rewards. Compared to traditional methods, this invention can predict the arrival time of convoys at intersections and spontaneously form dynamic green wave bands, significantly reducing the number of times vehicles stop on main roads. It introduces a temperature-sharpened multi-dimensional spatiotemporal attention mechanism to achieve precise perception: it can accurately pinpoint key peak times and congestion source intersections from complex road network data, avoiding the information overload and feature dilution problems common in multi-intersection control. It provides extreme robustness in extreme scenarios and responsiveness for special vehicles: this invention has stronger anti-deadlock capabilities when facing oversaturated traffic flow, and its proprietary priority vehicle reward... This ensures that ambulances, fire trucks, and other special vehicles can obtain the highest priority phase release rights under any circumstances, reflecting the system's extremely high level of humanized and intelligent management.

[0140] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A traffic signal control method based on spatiotemporal attention and safety constraints, characterized in that: The method includes the following steps in sequence: (1) Obtain real-time traffic status data at traffic intersections and construct state vectors based on the real-time traffic status data. ; (2) Construct a spatiotemporal graph convolutional network based on real-time traffic state data and state vectors. Constructing tensors , tensor Input a spatiotemporal graph convolutional network to predict traffic flow and obtain future traffic volume forecasts. ; (3) Using a spatiotemporal attention gating mechanism with conservative cold start, the future traffic flow prediction value is obtained. Dynamic confidence weighting is applied to generate enhanced state representations. ; (4) Transfer the state vector and enhanced state representation Input a dual-input deep Q-network and output the optimal signal phase action. ; (5) Through the action projection layer based on finite state machine, the action is performed at the optimal signal phase. Before issuing the execution order, the optimal signal phase is activated. Perform legality checks and logical corrections, and output a safe execution signal phase action; (6) Construct a composite reward function Calculate the multi-objective composite reward value, and use the multi-objective composite reward value to update the weight parameters of the dual-input deep Q network, so that the phase action of the signal output by the model is gradually optimized. (7) Repeat steps (1) to (6) to achieve real-time closed-loop adaptive control of the road network signal; In step (6), the composite reward function for: ； in, Rewards for efficiency For non-linear safety penalties, To predict consistency-oriented rewards, i.e., positive rewards. Rewards for priority vehicles; Efficiency Rewards The formula is: ； in, All are weighting coefficients; This represents the sum of the queue lengths for all lanes at the intersection. The average waiting time for all vehicles; Represents the arithmetic mean function; When any lane queue length Exceeding the overflow threshold At that time, apply nonlinear security penalties By using a forced model to prioritize traffic flow at intersections with high spillover risk, road network deadlock can be mitigated. （1）； （2）； In the formula, This is the overflow penalty coefficient. For exponential scaling factor, The coefficient is a power factor; Equation (1) is applicable to extreme congestion scenarios, when the lane queue length is... When approaching the physical capacity limit of the intersection, choose equation (1); equation (2) is suitable for normal congestion scenarios, when the queue length of each lane at the intersection is... Exceeding the preset threshold However, it has not yet reached a state where vehicles behind cannot enter at all. Choose equation (2); calculate the phase difference of the future predicted flow. Difference between current queue pressure : ； ； In the formula, , These represent the total number of vehicles arriving in phases A and B respectively within the future forecast period; , These are the real-time total queue lengths for the lanes corresponding to phase A and phase B at the current time, respectively. like and If the confidence threshold is exceeded, a positive reward is given. Otherwise, give a negative reward or zero reward, or a positive reward. The formula is: ； in, The confidence coefficient is calculated based on the predicted total flow and variance. Assigning reward weights accordingly; A priority vehicle reward system will be introduced for emergency vehicles and public transport priority vehicles. : ； in, This represents the number of emergency vehicles passing through during the current control period. This is the total cumulative waiting time for all emergency vehicles within the intersection; This represents the total number of emergency vehicles queuing at the intersection. This indicates a congestion signal indicating the presence of an emergency vehicle that has not received a valid clearance phase. This is the traffic efficiency gain coefficient; For special vehicle delay sensitivity coefficient; Clear the weight coefficients for the path; This is the phase conflict suppression coefficient.

2. The traffic signal control method based on spatiotemporal attention and safety constraints according to claim 1, characterized in that: Step (1) specifically refers to: using a preset time granularity Collect real-time status data of the traffic network and divide the traffic network into... Each intersection node is constructed. state vector The dimension of the state vector is State vector This includes the lane queue length, average vehicle waiting time, vehicle arrival rate, current phase duration, average vehicle speed, and intersection queue overflow coefficient for phase A, as well as the lane queue length, average vehicle waiting time, vehicle arrival rate, current phase duration, average vehicle speed, and intersection queue overflow coefficient for phase B.

3. The traffic signal control method based on spatiotemporal attention and safety constraints according to claim 1, characterized in that: Step (2) specifically refers to: the spatiotemporal graph convolutional network includes two stacked spatiotemporal convolutional blocks, namely the first spatiotemporal convolutional block and the second spatiotemporal convolutional block. The first spatiotemporal convolutional block and the second spatiotemporal convolutional block have the same structure. The first spatiotemporal convolutional block includes a first temporal convolutional layer, a first spatial graph convolutional layer and a second temporal convolutional layer; the second spatiotemporal convolutional block includes a third temporal convolutional layer, a second spatial graph convolutional layer and a fourth temporal convolutional layer. Based on real-time traffic status data and state vectors Constructing a four-dimensional tensor ,in, For batch size, For historical steps, This represents the number of intersection nodes. The input feature dimension has a value of 2, representing the vehicle arrival rate at phases A and B. From the state vector Extracted from; The processing method for the first spatiotemporal convolutional block is as follows: The first convolutional layer uses a two-dimensional convolutional kernel. tensor Perform time-dimension convolution, converting the input feature dimension Map from 2 to Output the first intermediate tensor Capture the local temporal correlation of traffic within a single node; The first spatial graph convolutional layer receives the first intermediate tensor. Based on the preset road network topology adjacency matrix ,use Chebyshev polynomials are used to perform spectral domain graph convolution to extract the spatial coupling features between the current node and its multi-level neighboring nodes in the road network, i.e., the spatial coupling feature tensor. The dimension of the spatial coupling feature remains 64; The second time convolutional layer uses a two-dimensional convolutional kernel. Spatial Coupled Feature Tensor Perform a time-axis sliding convolution again to further compress temporal redundancy and enhance the nonlinear representation of cross-dimensional features, outputting the first feature tensor. ; The processing method for the second spatiotemporal convolutional block is as follows: The third convolutional layer uses a two-dimensional convolutional kernel. For the first characteristic tensor Perform convolutions along the time dimension while maintaining the feature dimension as 1. Output the second intermediate tensor This further captures the local temporal correlation of traffic within a single node; The second spatial graph convolutional layer receives the second intermediate tensor. Based on the preset road network topology adjacency matrix ,use Chebyshev polynomials are used to perform spectral domain graph convolution; deep mining of cross-node propagation paths and diffusion effects of congestion in road networks under time lag, outputting a third intermediate tensor. The third intermediate tensor The dimension remains 64; The fourth temporal convolutional layer uses a two-dimensional convolutional kernel. For the third intermediate tensor Perform the time-axis sliding convolution again to change the dimension of the feature channels from... Dimensional expansion to Output the second feature tensor This further reduces time redundancy and enhances the nonlinear expression of cross-dimensional features, enabling abstract semantic representation of deep features; The second characteristic tensor The data is flattened and input into a fully connected projection layer. Through linear and nonlinear mapping, the high-dimensional spatiotemporal features are restored to the target dimension, and the predicted future traffic flow is output. ,in, .

4. The traffic signal control method based on spatiotemporal attention and safety constraints according to claim 1, characterized in that: Step (3) specifically includes the following steps in sequence: (3a) Perform time-attention sharpening: for future traffic flow forecasts Along the spatial node dimension and input feature dimension Global average pooling is performed to obtain a four-dimensional tensor. This four-dimensional tensor is then reduced to a two-dimensional matrix reflecting the intensity of flow evolution over time. Subsequently, a multilayer perceptron with a single hidden layer is used to map the features of each prediction time step to the original time score. ,in, , Implement a temperature sharpening strategy: introduce a time-temperature coefficient. Original time score Scaling is performed, and the temporal attention weights at different time steps within the prediction window are calculated using the Softmax function. : ； In the formula, The value is 0.35; Index for summation at time steps; For the first The original score corresponding to each prediction time step; (3b) Spatial attention weighting: For future traffic flow forecasts along and feature dimensions Global average pooling is performed to extract spatial activity features of each intersection node; a multilayer perceptron is then used to map the spatial activity features to the raw spatial scores of each node. , , The number of intersection nodes is determined; a temperature sharpening strategy is implemented, introducing a spatial temperature coefficient. Scoring of the original space Scaling is performed, and the spatial attention weights of each intersection node are calculated using the Softmax function. : ； In the formula, The value is 0.35; Sum index for nodes; For the first The original spatial score corresponding to each intersection node; (3c) Conservative initialization gating fusion: utilizing temporal attention weights Spatial attention weights Forecast of future traffic flow The aggregated features are obtained by performing a weighted summation. Mapped to an embedding vector by a projection layer Introducing a gating unit ,in, It is the Sigmoid activation function. The learnable weight matrix for the gated unit. For gated bias term and Initialize to a negative value, taking the value -1.2, so that the gated output is less than 0.5 in the early stage of training; (3d) Generate enhanced state representation To achieve conservative cold start and adaptive fusion of predicted features: ； in, This indicates element-wise multiplication.

5. The traffic signal control method based on spatiotemporal attention and safety constraints according to claim 1, characterized in that: Step (4) specifically includes the following steps in sequence: (4a) Improve the deep Q network to obtain a dual-input deep Q network: replace the single input layer of the deep Q network with a dual-input layer, which includes a first input layer and a second input layer, and the input of the first input layer is the state vector. The input to the second input layer is the augmented state representation. The first input layer is followed by a first fully connected layer and a second fully connected layer with hidden dimensions of 128 and 64, respectively. The output of the first fully connected layer is the feature. The output of the second fully connected layer is the feature. The second input layer is followed by a third fully connected layer and a fourth fully connected layer with hidden dimensions of 128 and 64, respectively. The output of the third fully connected layer is the feature. The output of the fourth fully connected layer is the feature. ; After the second and fourth fully connected layers and before the hidden layers of the deep Q-network, a feature fusion layer is added. This feature fusion layer processes the features... Perform dimensional alignment, weighted fusion, and noise reduction enhancement to output fused features. ; (4b) Fusion characteristics The Q-value vectors of each action are output through a linear output layer. ,in, These are the learnable weight parameters for a two-input deep Q-network; This is the real-time state vector of the current traffic intersection. Traffic signal phase operation; (4c) Adopt The strategy selects actions, and the action space is the phase selection set {0,1}, where 0 corresponds to phase A (green light) and 1 corresponds to phase B (green light). When the random number is less than the exploration rate... Actions are randomly selected when the random number is greater than or equal to the exploration rate. When the Q value is maximized, the action with the highest Q value is selected as the optimal signal phase action. : ； In the formula, Indicates that The action with the highest value ; Exploration rate The initial value is 1.0, which decays to a minimum value of 0.02 with an attenuation coefficient of 0.

998.

6. The traffic signal control method based on spatiotemporal attention and safety constraints according to claim 1, characterized in that: Step (5) refers to maintaining the finite state machine of each intersection node. The state variables of the finite state machine include the current green light phase, the green light start time, whether it is in the yellow light period, and the yellow light end time. Based on the state variables of the finite state machine, a multi-level safety constraint judgment is sequentially performed on the phase action of the output of the dual-input deep Q network. The multi-level safety constraint judgment includes: (5a) Yellow light transition constraint: When the current state is in the yellow light period, the current yellow light phase is maintained until the simulation time reaches the yellow light end time; (5b) Maximum green light time constraint: When the current green light duration reaches the maximum green light time threshold, the phase action is forcibly switched to the yellow light transition process; (5c) Minimum green light time constraint: When the current green light duration does not reach the minimum green light time threshold and the phase action is inconsistent with the current green light phase, the current green light phase remains unchanged; (5d) Normal phase switching: When the yellow light over-constraint, the maximum green light time constraint, and the minimum green light time constraint are all satisfied and the phase action is inconsistent with the current green light phase, the yellow light phase command is output first, and the target green light phase is switched after the yellow light ends, forming an executable signal phase action.

7. An electronic device, comprising: processor; as well as A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the traffic signal control method based on spatiotemporal attention and safety constraints as described in any one of claims 1-6.

8. A computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the traffic signal control method based on spatiotemporal attention and safety constraints as described in any one of claims 1-6.

Images