Traffic flow prediction and signal optimization method based on deep learning and reinforcement learning

Abstract

The invention provides a traffic flow prediction and signal control optimization method based on a space-time diagram convolutional network (STGCN), a long short-term memory network (LSTM) and a near-end strategy optimization (PPO) algorithm, and is suitable for intelligent traffic management in a smart city. According to the method, through combination of deep learning and reinforcement learning, urban traffic flow is accurately predicted, traffic signal lamp timing is optimized, urban traffic congestion is reduced, and road utilization efficiency is improved. The method comprises the following steps: firstly, capturing spatial dependence among different road sections through an STGCN model, and processing time dynamic change of traffic flow by using an LSTM model; and then, optimizing signal lamp timing based on a PPO algorithm, and dynamically adjusting a signal time sequence, thereby reducing the vehicle waiting time to the maximum extent and improving the traffic efficiency of the whole traffic flow. In addition, the system integrates external factors such as weather, holidays and festivals, and the adaptive capacity of the model in a complex traffic environment is enhanced. Experiments show that by using the method of the invention, the accuracy of traffic flow prediction is significantly improved, the average waiting time of vehicles is reduced, the traffic volume is increased, and the efficiency of a traffic system is effectively improved. The system is suitable for a large-scale urban traffic network, has good expansibility and real-time performance, and provides powerful technical support for intelligent urban traffic management.

Description

Technical Field

[0001] The present invention relates to intelligent traffic management technology, and in particular to a traffic flow prediction and signal optimization method based on deep learning and reinforcement learning, which can be used for traffic flow control and optimization in smart cities. Background Art

[0002] With the acceleration of urbanization, traffic congestion has become one of the main problems facing modern cities. Traditional traffic management systems usually rely on fixed-time control or rule-based adaptive signal control schemes, which have great limitations in coping with changing traffic conditions, especially in highly dynamic and complex traffic flow environments, which can easily lead to serious traffic congestion and waste of resources.

[0003] In recent years, intelligent traffic management systems based on deep learning and reinforcement learning have shown great application potential. Deep learning models, such as long short-term memory networks (LSTM) and spatiotemporal graph convolutional networks (STGCN), can effectively capture the spatiotemporal characteristics of traffic flow and improve the accuracy of traffic flow prediction. Reinforcement learning, especially the proximal policy optimization (PPO) algorithm, can optimize traffic signal control through continuous learning and adaptation, thereby dynamically adjusting signal timing and reducing traffic delays. Summary of the invention

[0004] The present invention proposes a traffic flow prediction and signal optimization method combining deep learning and reinforcement learning, which predicts traffic flow by combining a spatiotemporal graph convolutional network and a long short-term memory network, and uses a proximal strategy optimization algorithm to dynamically control traffic signals to achieve real-time traffic optimization. The method of the present invention can effectively reduce traffic congestion, improve road utilization, and reduce carbon emissions.

[0005] Specifically, the innovative features of the present invention include:

[0006] 1. Capturing spatiotemporal dependencies: Using a hybrid model combining STGCN and LSTM, we can accurately predict urban traffic flow and capture the spatiotemporal dependencies in the traffic network.

[0007] 2. Signal control optimization: Use the PPO algorithm to dynamically optimize the timing of traffic lights, reduce vehicle waiting time, and improve overall traffic flow efficiency.

[0008] 3. Integration of external factors: Introduce external factors such as weather and holidays into the model to enhance the system’s ability to adapt to real and complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] 1. Figure 1It is a schematic diagram of the system architecture of the present invention, including a data acquisition module, a data processing module, a prediction module, a signal control module and an execution module.

[0010] 2. Figure 2 Schematic diagram of the network structure of the traffic flow prediction model (STGCN-LSTM).

[0011] 3. Figure 3 Traffic signal control flow chart of PPO reinforcement learning algorithm.

[0012] 4. Figure 4 Performance comparison of STGCN-LSTM, ARIMA and Transformer models

[0013] 5. Figure 5 Comparison of actual and predicted traffic flow of different models

[0014] 6. Figure 6 To compare the performance of different traffic signal control methods

[0015] 7. Figure 7 The reward progress and rolling average of rewards during PPO training

[0016] 8. Figure 8 The traffic delay distribution and the time-varying traffic flow

[0017] 9. Fig. 9 Comparison of MAE and RMSE between commercial and residential areas with external factors included

[0018] 10. Fig.10 Traffic flow prediction for commercial and residential areas

[0019] 11. Fig.11 Comparative analysis of execution time and computational cost for traffic prediction models

[0020] 12. Fig.12 is the prediction error of the model under different city sizes

[0021] 13. Fig.13 Traffic prediction error and travel time analysis for different models

[0022] 14. Fig.14 Comparison of travel time, waiting time and vehicle speed under different control methods DETAILED DESCRIPTION

[0023] 1. Overall Architecture

[0024] (I) The method of the present invention consists of two parts: a traffic flow prediction module and a traffic signal control optimization module.

[0025] 1. Traffic flow prediction module: The STGCN-LSTM model is used to capture the spatiotemporal dependencies in the traffic network and provide accurate traffic flow prediction.

[0026] 2. Traffic signal control optimization module: Using the PPO reinforcement learning algorithm, the signal timing is dynamically adjusted according to the real-time traffic flow prediction results to optimize the traffic capacity of the intersection.

[0027] (II) The overall framework of the system Figure 1 As shown, it includes the following main steps:

[0028] 1. Data collection and preprocessing: Use traffic sensors, cameras and other equipment to collect traffic flow data, vehicle speed, occupancy rate and other information on various road sections in the city, and clean and normalize these data.

[0029] 2. Traffic flow prediction: Use the STGCN-LSTM model to analyze the preprocessed data and obtain the prediction results of future traffic flow.

[0030] 3. Signal control optimization: Based on the predicted traffic flow, the PPO algorithm dynamically selects the optimal signal control strategy to achieve optimal scheduling of traffic flow.

[0031] 4. Signal execution: Through the execution module, the timing of traffic lights is adjusted in real time to control traffic flow.

[0032] 2. Traffic Flow Prediction Module

[0033] 1. Spatio-temporal Graph Convolutional Network (STGCN): This network is used to capture the spatial dependencies between different road nodes and model the traffic network as a graph structure, where nodes represent different road sections and edges represent the connections between adjacent road sections. Through graph convolution operations, the spatial features between roads can be effectively extracted.

[0034] 2. Long Short-Term Memory Network (LSTM): The spatial features output by STGCN are then input into LSTM to capture the temporal dynamics of traffic flow. LSTM is able to handle both long-term and short-term temporal dependencies, thereby improving the accuracy of traffic flow prediction.

[0035] 3. Signal Control Optimization Module

[0036] 1. Reinforcement learning framework: The traffic signal control problem is modeled as a Markov decision process (MDP), including state space, action space and reward function.

[0037] (1) State space: includes information such as the current traffic flow at the intersection, lane occupancy rate, and vehicle waiting time.

[0038] (2) Action space: defined as the choice of different signal timing schemes, including extending the red light, green light, yellow light, etc.

[0039] (3) Reward function: The reward function is set to minimize vehicle waiting time and improve traffic efficiency. Specifically, it is the negative number of vehicle waiting time. The larger the reward value, the smoother the traffic flow.

[0040] 2. PPO algorithm: PPO algorithm is used for signal optimization. PPO ensures the stability of learning by limiting the step size of strategy update. Compared with traditional Q-learning, PPO shows better results in large-scale and complex traffic control tasks. The algorithm learns the optimal signal control strategy by interacting with the simulation environment, which simulates real traffic conditions, including vehicle flow, lane occupancy and signal light changes.

[0041] 4. Integration of External Factors

[0042] In order to improve the performance of the system in complex traffic environments, the method of the present invention introduces external factors as additional inputs, including weather conditions, holidays and emergencies. These external factors are quantified and used as additional input features to participate in the training of the model and the decision-making of signal control. By considering external factors, the system shows higher robustness and adaptability in traffic flow prediction and signal control.

[0043] V. Experiment and Verification

[0044] The system was tested using the METR-LA traffic data set from Los Angeles. The experimental results show that the system of the present invention has significant improvements in mean absolute error (MAE), root mean square error (RMSE), vehicle waiting time and traffic volume compared with the traditional ARIMA model and rule-based signal control method. Specifically:

[0045] 1. When using STGCN-LSTM for traffic flow prediction, the prediction error is reduced by more than 20% compared with traditional methods.

[0046] 2. Using PPO to optimize signal control, vehicle waiting time is reduced by 30% and traffic volume is increased by 15% compared to the fixed time solution.

[0047] Scope of protection of claims

[0048] The protection scope of the present invention includes but is not limited to the above-mentioned embodiments. Any modification, replacement, combination or equivalent conversion within the scope of the present invention shall be included in the scope of the claims of the present invention.

[0049] in conclusion

[0050] By combining deep learning and reinforcement learning, this paper proposes a method for smart city traffic flow prediction and signal optimization, which can effectively meet the traffic management needs of modern cities. By adopting STGCN-LSTM and PPO algorithms, this paper improves the accuracy of traffic flow prediction and the dynamic adaptability of signal control, reduces traffic congestion and improves road utilization efficiency.

Claims

1. A traffic flow prediction and signal optimization method based on deep learning and reinforcement learning, characterized by: (1) It includes a traffic flow prediction module and a traffic signal control optimization module; (2) The traffic flow prediction module uses a hybrid model that combines the Spatio-Temporal Graph Convolutional Network (STGCN) with the Long Short-Term Memory Network (LSTM) to capture the spatiotemporal dependencies in traffic flow. (3) The signal control optimization module adopts the proximal policy optimization (PPO) algorithm to dynamically adjust the duration of traffic lights through reinforcement learning and optimize the traffic signal timing of multiple intersections, thereby reducing vehicle waiting time and improving the overall traffic efficiency.

2. The method of claim 1, wherein: The traffic flow prediction module captures the spatial dependencies between different roads by constructing a graph structure of the traffic network and captures the dynamic changes in time through an LSTM network to provide accurate traffic flow prediction.

3. The method according to claim 1 or 2, wherein: The traffic flow prediction model further incorporates external factors, including weather conditions, holidays and emergencies, to improve the model's adaptability to complex traffic environments.

4. The method of claim 1, wherein: The traffic signal control optimization module optimizes the traffic signal timing through the PPO algorithm. The state space includes information such as lane occupancy, vehicle waiting time, and the number of vehicles entering the lane. The action space includes the time adjustment of red, green, and yellow lights. The reward function is set to minimize the average waiting time of vehicles and maximize the vehicle throughput.

5. The method of claim 4, wherein: The reward function further includes parameter adjustments for increasing vehicle speed and reducing fuel consumption to optimize the environmental friendliness of the transportation system.

6. The method of claim 1, wherein: The traffic signal control optimization module trains the PPO algorithm by constructing a simulated traffic environment. The simulated environment includes multiple intersections, vehicle flows, lane occupancy rates, and traffic signals. The PPO algorithm learns the optimal traffic signal control strategy by interacting with the environment.

7. The method of claim 6, wherein: The PPO algorithm uses Multi-Agent Reinforcement Learning (MARL) in the learning process to achieve signal coordination between multiple intersections, thereby improving the traffic flow capacity of large-scale urban networks.

8. The method of claim 1, wherein: The algorithm of the traffic signal control optimization module can be implemented by combining edge computing and cloud computing to ensure real-time response capabilities in large-scale deployment.

9. An intelligent traffic management system for implementing the method according to any one of claims 1 to 8, characterized in that: It includes a data acquisition module, a data processing module, a prediction and control module, and an execution module connected to the traffic signal light; The data collection module collects real-time traffic data and external factor information of different sections in the urban traffic network; The data processing module pre-processes the collected data and inputs it into the prediction and control module; The prediction and control module uses the STGCN-LSTM model to predict traffic flow and optimizes the control strategy of traffic lights through the PPO algorithm.

10. The system of claim 9, wherein: The execution module is connected to the traffic lights and can execute the optimized traffic light timing in real time to reduce traffic congestion, improve traffic efficiency and reduce carbon emissions. The above claims describe an intelligent traffic flow prediction and signal optimization method and system based on STGCN, LSTM and PPO algorithms, emphasizing the ability to capture spatiotemporal dependencies, intelligent optimization of signal control, and adaptability to external environmental factors. Such a system is particularly suitable for traffic management in smart cities.

Images