Traffic signal control method and system integrating extended Kalman filtering and attention enhancement
By integrating the extended Kalman filter with the attention-enhanced traffic signal control method, the problem of insufficient state perception of deep reinforcement learning in traffic signal control is solved, more efficient traffic flow prediction and signal light decision-making are achieved, and the intelligence and operational efficiency of intersections are improved.
Patent Information
- Application Number
- CN202510877641.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-27
- Publication Date
- 2025-09-19
AI Technical Summary
Existing traffic signal control methods based on deep reinforcement learning have limitations in state representation, insufficient perception of future traffic trends, and low efficiency in integrating predicted information with current status. These problems lead to insufficient foresight and robustness in decision-making, affecting urban traffic congestion and the level of intersection intelligence.
A traffic signal control method that integrates extended Kalman filtering and attention enhancement uses extended Kalman filtering to predict future traffic flow states, and uses an attention enhancement network to combine environmental states and predicted traffic flow. Combined with a deep Q network, it optimizes traffic light phase decisions to achieve more efficient state perception and forward-looking decision-making.
It improves the integrity and foresight of traffic signal control state expression, improves intersection operation efficiency and intelligence level, and alleviates urban traffic congestion.
Smart Images
Figure CN120673595A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of traffic signal control, and in particular to a traffic signal control method and system integrating extended Kalman filtering and attention enhancement. Background Art
[0002] Intersections are key nodes in urban transportation networks, and their operational efficiency directly impacts overall traffic flow. In recent years, deep reinforcement learning (DRL) has brought new developments to traffic signal control, and its autonomous learning capabilities have shown great potential. However, current DRL-based methods still suffer from limitations in state representation, insufficient perception of future traffic trends, and inefficient fusion of predicted information with current states in practical applications. This limits the foresight and robustness of their decisions, while also placing higher demands on the learning efficiency and stability of the algorithm. Therefore, developing a new adaptive traffic signal control method that can effectively integrate multi-source information, enhance state perception, and achieve forward-looking decision-making is of urgent need and significant value for alleviating urban traffic congestion and improving the intelligence level of intersections. Summary of the Invention
[0003] The present invention aims to solve the technical problems existing in the related art at least to a certain extent.
[0004] The purpose of this invention is to provide a new adaptive traffic signal control method that can effectively integrate multi-source information, enhance state perception and achieve forward-looking decision-making. It is of urgent need and important value for alleviating urban traffic congestion and improving the intelligence level of intersections.
[0005] In order to achieve the above-mentioned objectives, the present invention provides a traffic signal control method integrating extended Kalman filtering and attention enhancement, comprising the following steps:
[0006] S1. Obtain the current environment state of the intersection and construct a preliminary environment state vector;
[0007] S2. Based on historical traffic flow data and current observed vehicle flow, the extended Kalman filter prediction module is used to predict the arrival flow of each import lane in the next time window;
[0008] S3. Combining the current environment state with the predicted arrival traffic through an attention enhancement network to form an enhanced state vector;
[0009] S4. Based on the enhanced state vector, use the deep Q network to output the Q value corresponding to each optional action, and select the action to control the phase of the traffic light;
[0010] S5. Update the parameters of the deep Q network based on the reward obtained after executing the action and the environment state at the next moment to optimize the traffic signal control strategy.
[0011] Preferably, the environmental state vector construction formula in step S1 is:
[0012]
[0013] in, is the signal phase at time t, is the number of vehicles in the queue at entrance lane i at time t.
[0014] Preferably, the specific steps of step S2 are:
[0015] S21. Constructing Discrete-Time Dynamic Systems
[0016]
[0017] in, is the process noise, and the covariance is ; is the measurement noise, and the covariance is ; represents the flow observation value at the current time window k, is the mapping function from state to observation space, Defined as the system state vector:
[0018]
[0019] in Estimate the flow rate for the current time window, is the flow rate change rate, is the flow acceleration;
[0020] To describe the state from time arrive The state transition model of the evolution process is defined by the following set of equations:
[0021]
[0022] in, is the time step, Depends on current traffic The nonlinear adjustment factor is as follows:
[0023]
[0024] in, , are high and low flow thresholds, respectively. , is the corresponding adjustment coefficient, is the scaling factor;
[0025] S22, based on the optimal state estimation obtained at the previous moment , calculate the state prior through the state transfer function: ;
[0026] S23, calculate the state transfer function The Jacobian matrix at :
[0027] ;
[0028] S24, based on the state transfer function at the estimated point The Jacobian matrix at , use the extended Kalman filter to linearize the state covariance propagation prediction and calculate the prediction covariance matrix :
[0029]
[0030] in, Represent the propagation process of historical state uncertainty, that is, model the impact of existing errors as the state evolves; Represents the prior process noise introduced into the model to compensate for the model's approximation error to the real system;
[0031] S25, based on the state estimation , by measuring the function Calculate the expected observation value under the system prediction :
[0032] ;
[0033] Get the current time window Actual flow observation value at time , which is compared with the expected observation value to obtain the measurement residual :
[0034] ;
[0035] S26. Calculate predicted measurement values The Jacobian matrix of :
[0036] ;
[0037] S27. Calculate Kalman gain
[0038]
[0039] The residual covariance , measures the total uncertainty in the predicted measurement value, is the covariance of the predicted state, is the measurement noise covariance;
[0040] S28. Use Kalman gain to correct state prediction: , get the optimal posterior state estimate at the current moment, which serves as the basis for the next round of predictions, and updates the covariance at the same time:
[0041] .
[0042] Preferably, in step S3, the current environment state is combined with the predicted arrival flow through the attention enhancement network to form an enhanced state vector, specifically:
[0043] S31. Using state embedding network and predict embedding network Feature extraction is performed on the current environmental state and the predicted arrival flow to obtain the state feature vector and the predicted feature vector ;
[0044] S32, based on the state feature vector , the attention score is calculated through the attention enhancement network, and the attention weight is obtained by normalizing it with the Sigmoid function :
[0045]
[0046] The first layer activation function uses the ReLU function. and They are the first layer weights and biases of the attention enhancement network, and the second layer activation function Using the Sigmoid function, and They are the weights and bias of the second layer of the attention enhancement network;
[0047] S33, based on the attention weight , for the predicted feature vector Perform weighted calculation to obtain weighted prediction features :
[0048] ;
[0049] S34, the state feature vector and the weighted prediction feature vector Perform concat processing to generate an enhanced state vector :
[0050] .
[0051] As an example, the state feature vector in step S31 is and the predicted feature vector The extraction formulas are:
[0052]
[0053]
[0054] The activation function uses the ReLU function to strengthen the expression of important features. and is the weight and bias of the first layer of the state embedding network, and To predict the weights and bias of the first layer of the embedding network.
[0055] Preferably, in step S4, based on the enhanced state vector, a deep Q network is used to output the Q value corresponding to each optional action, and an action is selected to control the phase of the traffic light, specifically:
[0056] S41, based on enhanced state vector , use the deep Q network to calculate the Q value corresponding to each action a;
[0057] S42, using the ε-greedy strategy to select actions , the specific rules are:
[0058]
[0059] in, is the exploration rate, which decays linearly with training iterations, Represents the action space Random selection with medium probability;
[0060] S43, through the signal interface to move Converted into phase control instructions.
[0061] Preferably, in step S5, the parameters of the deep Q network are updated according to the reward observed after executing the action and the environment state at the next moment to optimize the traffic signal control strategy, specifically:
[0062] S51, according to the execution action , and receive corresponding rewards , used to quantify actions The improvement effect on traffic operation is as follows:
[0063]
[0064] in, is the number of queued vehicles at the entrance lane i at time t; n is the number of entrance lanes; is the queue length weight factor, which is used to penalize the current high queue state; is the queue trend weight factor, which rewards the downward trend of queue length;
[0065] S52, enter the next moment environment state , and in this state, the next traffic flow prediction is carried out to obtain ;
[0066] S53: The historical interaction data obtained in the above steps is stored in an experience replay pool;
[0067] S54, using uniform random sampling method to extract from the experience pool Samples are taken as a batch, and the TD target value of each sample in the batch is calculated using the deep Q network:
[0068]
[0069] in, is the discount factor for future rewards, Represents the Q value output of the target network;
[0070] S55. Estimate the Q value of the current state-action pair using the current online network:
[0071]
[0072] Calculate the time difference error between the estimated Q value and the TD target value:
[0073] ;
[0074] S56. Calculate the mean square of the TD error of all samples in the batch and construct the loss function :
[0075] ;
[0076] S57, use Adam optimizer to optimize online network parameters Perform back propagation update, and the parameter update rule is:
[0077]
[0078] in, is the learning rate, yes right gradient;
[0079] S58, parameters based on the current online network , regularly adjust the parameters of the deep Q network to update.
[0080] Preferably, the reward The calculation formula is:
[0081]
[0082] in, is the average number of vehicles in the queue, is the average number of vehicles in the queue at the next moment, is the average number of vehicles in the queue at the next moment , and is the preset adjustment factor.
[0083] As an advantage, the experience pool capacity is , the storage format is six-tuple ,in, represents the environmental state vector at time t; Represents the predicted state vector at time t+1; represents the action chosen at time t; Represents the action performed at time t After the reward; Represents the environment state vector at time t+1.
[0084] In another aspect, the present invention provides a traffic signal control system integrating extended Kalman filtering and attention enhancement, comprising:
[0085] The environment module is used to obtain the current environment state of the intersection and construct a preliminary environment state vector;
[0086] Prediction module, with built-in extended Kalman filter prediction module, is used to predict the arrival flow of each import lane in the next time window;
[0087] Attention enhancement module, with a built-in attention enhancement network, is used to combine the current environment state with the predicted arrival traffic to form an enhanced state vector;
[0088] The decision module has a built-in deep Q-network that calculates the Q value corresponding to each optional action, selects the action to control the traffic light phase, and updates the parameters of the deep Q-network based on the reward observed after executing the action and the next environmental state to optimize the traffic signal control strategy.
[0089] Beneficial effects: The present invention integrates the real-time estimation information of future traffic flow status by the extended Kalman filter, designs an attention enhancement module, and designs a more efficient state combination method, thereby improving the integrity and foresight of state expression, thereby improving the control effect of the final strategy. BRIEF DESCRIPTION OF THE DRAWINGS
[0090] Figure 1 This is a network structure diagram of the traffic signal control method integrating extended Kalman filtering and attention enhancement according to the present invention;
[0091] Figure 2 Schematic diagram of the process of Kalman filter prediction in the present invention;
[0092] Figure 3 Schematic diagram of the internal flow of the attention enhancement module in the present invention. DETAILED DESCRIPTION
[0093] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the embodiments described are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0094] The following combination Figure 1-Figure 3 The present invention describes a traffic signal control method that integrates extended Kalman filtering and attention enhancement.
[0095] Example 1: Figure 1 As shown, this embodiment provides a traffic signal control method integrating extended Kalman filtering and attention enhancement, comprising the following steps:
[0096] Step 1: Get the current environment state of the intersection and construct the current state vector The current environmental state includes the current signal phase and the number of vehicles in queue at each entrance The current signal phase communicates with the physical traffic signal controller or integrated traffic management platform deployed at the intersection to obtain the precise phase information of the target intersection signal light. The number of queued vehicles at each entrance lane is obtained through video detectors deployed at the intersection.
[0097] Step 2, such as Figure 2As shown in Figure 1, an independent extended Kalman filter prediction module is set in each lane, and the extended Kalman filter prediction module is used to predict the arrival flow of each import lane in the next time window. The specific steps are as follows:
[0098] Step 2.1, first build a discrete-time dynamic system
[0099] in, is the process noise, which obeys the Gaussian distribution with mean 0 and covariance matrix Q. is the measurement noise, the covariance matrix is R, is the system state vector defined, Estimate the flow rate for the current time window, is the flow rate change rate, is the flow acceleration, represents the flow observation value at the current time window k, is the mapping function from state to observation space;
[0100] Step 2.2, State Transition Model Its specific form is defined by the following set of equations:
[0101]
[0102] in, is the time step, Depends on current traffic The nonlinear adjustment factor is as follows:
[0103]
[0104] in, , are high and low flow thresholds, respectively. , is the corresponding adjustment coefficient, is the scaling factor, which is adjusted according to the different traffic flows in each lane;
[0105] Step 2.3, based on the optimal estimate obtained at the previous moment , calculate the state prior through the state transfer function: ;
[0106] Step 2.4 Calculate the state transfer function in The Jacobian matrix at :
[0107]
[0108] By locally linearizing the system, this Jacobian matrix approximately describes how small changes in state propagate;
[0109] Step 2.5, based on the state transfer function obtained in step 2.3, estimate the point The Jacobian matrix at , linearize the propagation prediction of the state covariance and calculate the prediction covariance matrix The process uses the covariance prediction formula of the extended Kalman filter: In this expression, the term Represent the propagation process of historical state uncertainty, that is, model the impact of existing errors as the state evolves; It represents the prior process noise introduced in the model due to simplified or unmodeled factors, which is used to compensate for the approximation error of the model to the real system;
[0110] Step 2.6: The current time window is directly measured but noisily by the traffic detectors deployed at each entrance lane. Actual flow observation value at time ;
[0111] At the same time, according to the state estimate obtained in the previous prediction step , by measuring the function Calculate the expected observation value under the system prediction ,Right now:
[0112]
[0113] Compare the observed values to the predicted values to get the measurement residuals , and its calculation formula is:
[0114]
[0115] This residual measures the difference between the system's predicted state and the actual observation, and serves as an important basis for subsequent updates of the state estimate and covariance matrix;
[0116] In step 2.7, the relationship between the predicted measurement value and the system state is: , since the measurement function For the state vector The partial derivative of is a constant, and its Jacobian matrix The calculation is as follows:
[0117]
[0118] Step 2.8, calculate the Kalman gain , which is used to balance the trust in the model prediction value and the trust in the new measurement value, where the residual covariance A measure of the total uncertainty in the predicted measurement value, given by the covariance of the predicted state Covariance with measurement noise Joint decision-making;
[0119] Step 2.9, use the Kalman gain to correct the state prediction: , we get the optimal posterior state estimate at the current moment, which will then serve as the basis for the next round of predictions and update the covariance: , forming a closed-loop iterative process of "prediction-measurement-correction" to ensure the real-time and accuracy of traffic prediction.
[0120] Step 3, such as Figure 3 As shown, the current environment state and the predicted arrival flow are combined through the attention enhancement network to form an enhanced state vector. The specific steps are:
[0121] Step 3.1: The original data dimensions of the current environment state and the predicted traffic are different, and they need to be mapped to a unified dimensional space through feature embedding;
[0122] Step 3.2, Current Environment Status , through the state embedding network Map it to dimensional feature space:
[0123]
[0124] The activation function uses the ReLU function to strengthen the expression of important features. and Embed the weights and biases of the first layer of the state embedding network;
[0125] Step 3.3, predict traffic , by predicting the embedding network Map it to dimensional feature space:
[0126]
[0127] Get the feature vector of uniform dimension, where and To predict the weights and biases of the first layer of the embedding network;
[0128] Step 3.4: embed the features of the current environment state Input the first layer of the attention enhancement network and extract the hidden layer features:
[0129]
[0130] Among them, the first layer activation function adopts the ReLU function. and They are the weights and bias of the first layer of the attention enhancement network;
[0131] Step 3.5, the hidden layer features , input the second layer of the attention enhancement network, and output the attention weight between 0-1 :
[0132]
[0133] in is the Sigmoid function, ensuring the output attention weight , and They are the weights and bias of the second layer of the attention enhancement network;
[0134] Step 3.6, attention weight is a scalar, the embedding feature of the predicted traffic Weighted to highlight the importance of the predicted information in the current state:
[0135]
[0136] Step 3.7, embed the current state features and weighted prediction features Concatenate along the feature dimension to form an enhanced state vector containing prediction information , and send it to subsequent calculations.
[0137] Step 4: Input the enhanced state vector into a deep Q-network (DQN). The DQN outputs the Q value corresponding to each optional action and selects an action based on a preset strategy to control the phase of the traffic light. The specific steps are as follows:
[0138] Step 4.1, the state vector is enhanced As the input of the Q network, it outputs the Q value corresponding to each action a;
[0139] Step 4.2, select actions using the ε-greedy strategy , the specific rules are:
[0140]
[0141] in is the exploration rate, which decays linearly with training iterations, Represents the action space Random selection with medium probability;
[0142] Step 4.3, Select Action Finally, the action is converted into phase control instructions through the signal machine interface;
[0143] Step 5: This step updates the DQN network parameters by combining the observed reward after executing the action and the next moment's environment state with the experience replay mechanism and dual network structure to achieve iterative optimization of the traffic signal control strategy. The specific steps are as follows:
[0144] Step 5.1, when executing the action Then, we get the corresponding reward function , used to quantify actions The improvement effect on traffic operation is as follows:
[0145]
[0146] is the number of queued vehicles at the entrance lane i at time t, which is collected in real time by the intersection camera detector;
[0147] n is the number of inlet channels;
[0148] is the queue length weight factor, which is used to penalize the current high queue state;
[0149] is the queue trend weight factor, which rewards the downward trend of queue length;
[0150] Step 5.2, enter the next moment environment state , and in this state, the next traffic flow prediction is carried out to obtain ;
[0151] Step 5.3: The historical interaction data obtained in the above steps is stored in the experience replay pool. The experience pool capacity is set to , the storage format is six-tuple For subsequent network training, where:
[0152] represents the environmental state vector at time t;
[0153] Represents the predicted state vector at time t+1;
[0154] represents the action chosen at time t;
[0155] Represents the action performed at time t After the reward;
[0156] Represents the environmental state vector at time t+1;
[0157] Represents the predicted state vector at time t+1;
[0158] Step 5.4: After each action is executed, the current moment reward storage interaction data is obtained and a network training is triggered. The uniform random sampling method is used to extract the experience pool. Samples are considered as a batch. The sampling format of each batch is:
[0159] ;
[0160] In step 5.5, the target Q network is used to calculate the TD target value for each sample in the batch, namely:
[0161]
[0162] in is the discount factor for future rewards, Represents the Q value output of the target network;
[0163] Step 5.6, then, uses the current online network to estimate the Q value of the current state-action pair:
[0164]
[0165] Calculate the time-difference error between this estimate and the TD target value:
[0166] ;
[0167] Step 5.7: Calculate the mean square of the TD errors of all samples in a batch and construct the loss function :
[0168] ;
[0169] Step 5.8, use Adam optimizer to optimize the online network parameters Perform back propagation update, and the parameter update rule is:
[0170]
[0171] in is the learning rate, yes right gradient;
[0172] Step 5.9, in each training After that, the target network is updated and the parameters of the current online network are updated. Copy the parameters assigned to the target network .
[0173] In this embodiment, cumulative rewards, average driving speed, and average one-way waiting time are used as evaluation indicators. Three algorithms including the algorithm of this embodiment are verified on the hangzhou_1x1_bc-tyc_18041607_1h dataset using SUMO simulation software. The results are shown in Table 1:
[0174] Table 1
[0175]
[0176] As can be seen from Table 1, the algorithm of this embodiment performs best for the trained model in the data set. By selecting Kalman filtering for traffic prediction and combining the attention enhancement mechanism to dynamically adjust the weight of the prediction information, the intelligent agent can make better decisions.
[0177] Example 2: The present invention provides a traffic signal control system integrating extended Kalman filtering and attention enhancement, comprising:
[0178] The environment module is used to obtain the current environment state of the intersection and construct a preliminary environment state vector;
[0179] Prediction module, with built-in extended Kalman filter prediction module, is used to predict the arrival flow of each import lane in the next time window;
[0180] Attention enhancement module, with a built-in attention enhancement network, is used to combine the current environment state with the predicted arrival traffic to form an enhanced state vector;
[0181] The decision module has a built-in deep Q-network that calculates the Q value corresponding to each optional action, selects the action to control the traffic light phase, and updates the parameters of the deep Q-network based on the reward observed after executing the action and the next environmental state to optimize the traffic signal control strategy.
[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A traffic signal control method integrating extended Kalman filtering and attention enhancement, characterized in that: The steps include: S1. Obtain the current environment state of the intersection and construct a preliminary environment state vector; S2. Based on historical traffic flow data and current observed vehicle flow, the extended Kalman filter prediction module is used to predict the arrival flow of each import lane in the next time window; S3. Combining the current environment state with the predicted arrival traffic through an attention enhancement network to form an enhanced state vector; S4. Based on the enhanced state vector, use the deep Q network to output the Q value corresponding to each optional action, and select the action to control the phase of the traffic light; S5. Update the parameters of the deep Q network based on the reward obtained after executing the action and the environment state at the next moment to optimize the traffic signal control strategy.
2. The traffic signal control method integrating extended Kalman filtering and attention enhancement according to claim 1 is characterized in that: The formula for constructing the environment state vector in step S1 is: ； in, is the signal phase at time t, is the number of vehicles in the queue at entrance lane i at time t.
3. The traffic signal control method integrating extended Kalman filtering and attention enhancement according to claim 2 is characterized in that: The specific steps of step S2 are: S21. Constructing Discrete-Time Dynamic Systems ； in, is the process noise, and the covariance is ; is the measurement noise, and the covariance is ; represents the flow observation value at the current time window k, is the mapping function from state to observation space, Defined as the system state vector: ； in Estimate the flow rate for the current time window, is the flow rate change rate, is the flow acceleration; To describe the state from time arrive The state transition model of the evolution process is defined by the following set of equations: ； in, is the time step, Depends on current traffic The nonlinear adjustment factor is as follows: ； in, , are high and low flow thresholds, respectively. , is the corresponding adjustment coefficient, is the scaling factor; S22, based on the optimal state estimation obtained at the previous moment , calculate the state prior through the state transfer function: ; S23, calculate the state transfer function The Jacobian matrix at : ； S24, based on the state transfer function at the estimated point The Jacobian matrix at , use the extended Kalman filter to linearize the state covariance propagation prediction and calculate the prediction covariance matrix : ； in, Represent the propagation process of historical state uncertainty, that is, model the impact of existing errors as the state evolves; Represents the prior process noise introduced into the model to compensate for the model's approximation error to the real system; S25, based on the state estimation , by measuring the function Calculate the expected observation value under the system prediction : ； Get the current time window Actual flow observation value at time , which is compared with the expected observation value to obtain the measurement residual : ； S26. Calculate predicted measurement values The Jacobian matrix of : ； S27. Calculate Kalman gain ； The residual covariance , measures the total uncertainty in the predicted measurement value, is the covariance of the predicted state, is the measurement noise covariance; S28. Use Kalman gain to correct state prediction: , get the optimal posterior state estimate at the current moment, which serves as the basis for the next round of predictions, and updates the covariance at the same time: 。 4. The traffic signal control method integrating extended Kalman filtering and attention enhancement according to claim 3 is characterized in that: In step S3, the current environment state is combined with the predicted arrival traffic through the attention enhancement network to form an enhanced state vector, specifically: S31. Using state embedding network and predict embedding network Feature extraction is performed on the current environmental state and the predicted arrival flow to obtain the state feature vector and the predicted feature vector ; S32, based on the state feature vector , the attention score is calculated through the attention enhancement network, and the attention weight is obtained by normalizing it with the Sigmoid function : ； The first layer activation function uses the ReLU function. and They are the first layer weights and biases of the attention enhancement network, and the second layer activation function Using the Sigmoid function, and They are the weights and bias of the second layer of the attention enhancement network; S33, based on the attention weight , for the predicted feature vector Perform weighted calculation to obtain weighted prediction features : ； S34, the state feature vector and the weighted prediction feature vector Perform concat processing to generate an enhanced state vector : 。 5. The traffic signal control method integrating extended Kalman filtering and attention enhancement according to claim 4 is characterized in that: The state feature vector in step S31 and the predicted feature vector The extraction formulas are: ； ； The activation function uses the ReLU function to strengthen the expression of important features. and is the weight and bias of the first layer of the state embedding network, and Predict the weights and biases of the first layer of the embedding network.
6. The traffic signal control method integrating extended Kalman filtering and attention enhancement according to claim 5 is characterized in that: In step S4, based on the enhanced state vector, a deep Q network is used to output the Q value corresponding to each optional action, and an action is selected to control the phase of the traffic light, specifically: S41, based on enhanced state vector , use the deep Q network to calculate the Q value corresponding to each action a; S42, using the ε-greedy strategy to select actions , the specific rules are: ； in, is the exploration rate, which decays linearly with training iterations, Represents the action space Random selection with medium probability; S43, through the signal interface to move Converted into phase control instructions.
7. The traffic signal control method integrating extended Kalman filtering and attention enhancement according to claim 6 is characterized in that: In step S5, the parameters of the deep Q network are updated according to the reward observed after executing the action and the environment state at the next moment to optimize the traffic signal control strategy, specifically: S51, according to the execution action , and receive corresponding rewards , used to quantify actions The improvement effect on traffic operation is in the following forms: ； in, is the number of queued vehicles at the entrance lane i at time t; n is the number of entrance lanes; is the queue length weight factor, which is used to penalize the current high queue state; is the queue trend weight factor, which rewards the downward trend of queue length; S52, enter the next moment environment state , and in this state, the next traffic flow prediction is carried out to obtain ; S53: The historical interaction data obtained in the above steps is stored in an experience replay pool; S54, using uniform random sampling method to extract from the experience pool Samples are taken as a batch, and the TD target value of each sample in the batch is calculated using the deep Q network: ； in, is the discount factor for future rewards, Represents the Q value output of the target network; S55. Estimate the Q value of the current state-action pair using the current online network: ； Calculate the time difference error between the estimated Q value and the TD target value: ； S56. Calculate the mean square of the TD error of all samples in the batch and construct the loss function : ； S57, use Adam optimizer to optimize online network parameters Perform back propagation update, and the parameter update rule is: ； in, is the learning rate, yes right gradient; S58, parameters based on the current online network , regularly adjust the parameters of the deep Q network to update.
8. The traffic signal control method integrating extended Kalman filtering and attention enhancement according to claim 7 is characterized in that: The reward The calculation formula is: ； in, is the average number of vehicles in the queue, is the average number of vehicles in the queue at the next moment, is the average number of vehicles in the queue at the next moment , and is the preset adjustment factor.
9. The traffic signal control method integrating extended Kalman filtering and attention enhancement according to claim 8 is characterized in that: The experience pool capacity is , the storage format is six-tuple ,in, represents the environmental state vector at time t; Represents the predicted state vector at time t+1; represents the action chosen at time t; Represents the action performed at time t After the reward; Represents the environment state vector at time t+1.
10. A traffic signal control method integrating extended Kalman filtering and attention enhancement, for implementing the method according to any one of claims 1 to 9, characterized in that: include: The environment module is used to obtain the current environment state of the intersection and construct a preliminary environment state vector; Prediction module, with built-in extended Kalman filter prediction module, is used to predict the arrival flow of each import lane in the next time window; Attention enhancement module, with a built-in attention enhancement network, is used to combine the current environment state with the predicted arrival traffic to form an enhanced state vector; The decision module has a built-in deep Q-network that calculates the Q value corresponding to each optional action, selects the action to control the traffic light phase, and updates the parameters of the deep Q-network based on the reward observed after executing the action and the next environmental state to optimize the traffic signal control strategy.