Traffic flow prediction method based on artificial intelligence

Abstract

The invention provides a traffic flow prediction method based on artificial intelligence, and the method comprises the steps: extracting the multi-granularity features of a time dimension and a space dimension based on cleaned historical traffic data, obtaining the traffic flow statistical features under different time scales, calculating the spatial correlation between road nodes and road segments according to a road network topology structure, and carrying out the calculation of the spatial correlation between the road nodes and the road segments. Constructing a spatial weight matrix; the heterogeneous data of weather, holidays and the like are quantified into numeric features or category features, the numeric features or category features are fused with the traffic flow features, multi-view and multi-mode feature representation is obtained, and the adaptability of the model to external factor changes is enhanced; a long-short-term memory network is adopted to model the spatial-temporal characteristics, a long-term dependency relationship and a short-term fluctuation rule of traffic flow are adaptively captured through a gating mechanism, and meanwhile, an attention mechanism is introduced to dynamically adjust importance weights of different spatial-temporal positions, so that the attention of the model to a key area is improved.

Description

Technical Field

[0001] The invention belongs to the field of traffic technology, and in particular relates to a traffic flow prediction method based on artificial intelligence. Background Art

[0002] A key technical issue in traffic flow prediction is how to extract effective features and patterns from massive historical traffic data. Traditional traffic flow prediction methods mainly rely on time series analysis and statistical learning, but in the face of increasingly complex traffic scenarios and growing data scale, these methods often find it difficult to capture the inherent laws and driving factors of traffic flow changes. Therefore, it is urgent to study traffic flow prediction methods based on artificial intelligence to fully tap the value of traffic big data.

[0003] However, it is not easy to apply artificial intelligence technology to traffic flow prediction. First of all, traffic flow data has obvious spatiotemporal correlation, and it is crucial to effectively integrate the characteristics of time dimension and space dimension in the modeling process. Secondly, traffic flow is affected by many factors, such as weather conditions, holidays, road construction, etc. How to unify and integrate these heterogeneous data is a major challenge. Furthermore, the quality of traffic flow data varies, and there are problems such as noise, outliers and missing values, which require appropriate data cleaning and preprocessing. Finally, the traffic flow prediction model needs to have a certain degree of interpretability so that the traffic management department can assist in decision-making and analysis, which puts higher requirements on the design and optimization of the model. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a traffic flow prediction method based on artificial intelligence to solve the problems existing in the above prior art.

[0005] To achieve the above object, the present invention provides a traffic flow prediction method based on artificial intelligence, comprising the following steps:

[0006] Collect historical traffic data and perform cleaning and preprocessing to obtain preprocessed data;

[0007] Based on the preprocessed data, extracting traffic flow statistical characteristics at different time scales, and calculating the spatial correlation between road nodes and road sections according to the road network topology structure;

[0008] Quantifying heterogeneous data into numerical features and fusing them with the traffic flow statistical features to obtain a comprehensive feature representation;

[0009] Based on the comprehensive feature representation and spatial correlation, a traffic flow prediction model is constructed through a long short-term memory network and a graph convolutional network;

[0010] The traffic flow prediction model is used to predict future traffic flow and generate prediction information, which is then visualized.

[0011] Preferably, the method for obtaining preprocessed data comprises:

[0012] A big data processing framework is selected to build a data processing environment, and distributed storage and parallel computing of historical traffic data are carried out. Noise and outliers are identified and eliminated through anomaly detection algorithms, and data interpolation methods are used to fill in missing values ​​to obtain preprocessed data.

[0013] Preferably, the method for acquiring traffic flow statistical characteristics includes:

[0014] The pre-processed data is processed by using spatiotemporal data fusion technology, and the mean, variance and peak value of traffic flow at each time granularity are counted as traffic flow statistical features according to the time dimension, such as hour, day and week.

[0015] Preferably, the method for calculating the spatial correlation includes:

[0016] Graph theory algorithms are used to calculate the spatial correlation between road nodes and sections, and to obtain an adjacency matrix that reflects the spatial dependency between nodes and sections.

[0017] According to the adjacency matrix, a spatial autocorrelation analysis method is used to calculate the spatial correlation of nodes and sections in the road network.

[0018] Preferably, the method for obtaining the comprehensive feature representation comprises:

[0019] The heterogeneous data includes weather and holiday heterogeneous data. Numerical features of temperature and precipitation are extracted from weather data, and holiday data are converted into categorical features using one-hot encoding.

[0020] By using data fusion technology, the numerical features and categorical features are combined with the traffic flow statistical features to obtain a comprehensive feature representation.

[0021] Preferably, the method for constructing a traffic flow prediction model comprises:

[0022] The first model is obtained by training the long short-term memory network through comprehensive feature representation and spatial correlation;

[0023] The spatial correlation is modeled through a graph convolutional network to obtain a second model;

[0024] The model integration technology is used to integrate the first model and the second model to obtain the traffic flow prediction model.

[0025] Preferably, an attention mechanism is introduced when constructing the first model to dynamically adjust the importance weights of different spatiotemporal positions.

[0026] Preferably, in the model prediction stage, a distributed computing framework is used for parallel real-time prediction. Through data sharding and model parallel technology, traffic data and traffic flow prediction models are divided into multiple subtasks and assigned to different computing nodes for simultaneous calculation.

[0027] Compared with the prior art, the present invention has the following advantages and technical effects:

[0028] The present invention discloses a traffic flow prediction method based on big data and deep learning. The method first cleans and preprocesses massive historical traffic data, extracts multi-granularity spatiotemporal features, and integrates heterogeneous data such as weather and holidays. Then, a deep learning model combining a long short-term memory network and a graph convolutional network is used to model the spatiotemporal evolution of traffic flow. The model captures long-term and short-term dependencies through a gating mechanism and an attention mechanism, uses a graph convolutional network to model the topological structure of a road network, and enhances spatial perception. Finally, a distributed computing framework is used to achieve parallel training and real-time prediction of the model to improve efficiency. The present invention can effectively process large-scale traffic data, accurately capture the spatiotemporal evolution of traffic flow, achieve quasi-real-time and high-precision traffic flow prediction, and provide strong support for intelligent traffic management and decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0030] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0032] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Embodiment 1

[0034] like Figure 1 As shown, this embodiment provides a traffic flow prediction method based on artificial intelligence, comprising the following steps:

[0035] Collect historical traffic data and perform cleaning and preprocessing to obtain preprocessed data;

[0036] Based on the preprocessed data, extracting traffic flow statistical characteristics at different time scales, and calculating the spatial correlation between road nodes and road sections according to the road network topology structure;

[0037] Quantifying heterogeneous data into numerical features and fusing them with the traffic flow statistical features to obtain a comprehensive feature representation;

[0038] Based on the comprehensive feature representation and spatial correlation, a traffic flow prediction model is constructed through a long short-term memory network and a graph convolutional network;

[0039] The traffic flow prediction model is used to predict future traffic flow and generate prediction information, which is then visualized.

[0040] The specific steps include:

[0041] S101. Collect massive amounts of historical traffic data, clean and pre-process them using a big data processing framework, identify and remove noise and outliers through anomaly detection algorithms, and use data interpolation methods to fill in missing values, thereby improving the data quality and laying the foundation for subsequent feature extraction and model training.

[0042] Specifically, when building a big data processing environment, you can choose HDFS in the Hadoop ecosystem as a distributed storage system and use MapReduce for parallel computing. For example, you can partition massive historical traffic data by date and region and store it on different data nodes of HDFS to achieve distributed storage of data. In the data exploration and analysis stage, you can use statistical methods to perform descriptive analysis on the data, such as calculating the mean, variance, quantile, etc. of the data to identify the characteristics of data distribution. At the same time, you can use visualization methods such as box plots to intuitively find outliers and outliers in the data. In terms of anomaly detection, you can use distance-based anomaly detection algorithms, such as Euclidean distance or Mahalanobis distance, to calculate the distance between each data point and other data points, and mark data points whose distance exceeds a certain threshold (such as 3 times the standard deviation) as outliers. For missing value processing, you can use the nearest neighbor interpolation method to estimate the missing value using the average value of the k nearest neighbor data points around the missing value. Through methods such as cross-validation, you can optimize the selection of k values ​​and improve the accuracy of interpolation.

[0043] S102: Based on the cleaned traffic data, extract multi-granularity features in the time dimension and the space dimension, obtain traffic flow statistical features at different time scales, and calculate the spatial correlation between road nodes and road sections according to the road network topology.

[0044] Specifically, in the time dimension, traffic statistics can be performed at granularities such as 5 minutes, 1 hour, and 1 day, and the mean, variance, peak, and other indicators of each time period can be calculated. For example, the average traffic flow during the morning peak period of 7:00-9:00 is 500 vehicles / hour, and the average traffic flow during the evening peak period of 17:00-19:00 is 600 vehicles / hour, which is much higher than the 200 vehicles / hour during normal periods. In the spatial dimension, graph theory algorithms can be used to calculate the adjacency matrix of the road network and quantitatively analyze the strength of the association between nodes and road sections. For example, the PageRank algorithm finds that the connectivity of roads around a certain business district is very high, and the traffic flow is obviously linked. The congestion trend of the road section can be judged according to the traffic statistics. For example, during the peak hours on weekdays, if the average traffic flow is more than 50% higher than usual and lasts for more than 2 hours, it can be determined as a high-congestion section. The propagation effect of congestion is revealed through spatial correlation analysis. For example, in the adjacency matrix, the correlation coefficient of two road sections reaches 8, indicating that the traffic flow between them is greatly affected. Once an accident occurs on a road section, the adjacent road sections will also be affected, and the traffic pressure will increase sharply. The data mining algorithm is used to comprehensively identify the spatiotemporal pattern of traffic flow, set an abnormal threshold based on the mean and variance, and perform online detection on real-time data. If the traffic flow exceeds 2 standard deviations from the mean in three consecutive time windows, it is determined to be an abnormal event and trigger an early warning. At the same time, historical data is mined to find the abnormal association pattern and evolution law, and the cause is analyzed.

[0045] S103, quantifying the heterogeneous data such as weather and holidays into numerical features or categorical features, and fusing them with the above-mentioned traffic flow features to obtain multi-view and multi-modal feature representation, thereby enhancing the adaptability of the model to changes in external factors.

[0046] Specifically, heterogeneous data including weather and holidays are obtained, such as temperature, precipitation, holiday type, etc. For weather data, numerical features such as temperature and precipitation are extracted. For holiday data, unique hot encoding is used to convert them into categorical features. Traffic flow data for the past year is obtained, and features such as flow mean, peak value, and rate of change are extracted. Using data fusion technology, weather features, holiday features, and traffic flow features are spliced ​​together to construct a comprehensive feature matrix including dimensions such as temperature, precipitation, holiday type, and flow mean.

[0047] S104. Use long short-term memory network to model the above-mentioned spatiotemporal characteristics, and adaptively capture the long-term dependency and short-term fluctuation of traffic flow through the gating mechanism. At the same time, introduce the attention mechanism to dynamically adjust the importance weights of different spatiotemporal positions to improve the model's attention to key areas.

[0048] Specifically, the spatiotemporal features are used as the input of the long short-term memory network. Through the forget gate, input gate and output gate in the long short-term memory network, the long-term dependencies and short-term fluctuations in the traffic flow data are adaptively captured. The attention mechanism is introduced in the hidden layer of the long short-term memory network, and the importance weights of different spatiotemporal positions are dynamically adjusted through the attention weight matrix. According to the attention weight matrix, the attention score of each spatiotemporal position is calculated to obtain the attention distribution of the key area. The attention distribution is used as a mask and multiplied with the hidden state of the long short-term memory network to enhance the model's attention to the key area. The hidden state after attention enhancement is input into the fully connected layer to generate the traffic flow prediction value of each area in the future. The mean square error and other evaluation indicators are used to evaluate the accuracy of the prediction results, and the back propagation algorithm is used to optimize the model parameters to improve the prediction accuracy.

[0049] In this embodiment, the feature matrix is ​​divided into several time slices in chronological order, and each time slice contains data from the past 24 hours as the input of the long short-term memory network. A long short-term memory network with three hidden layers is built, each hidden layer contains 128 neurons, and the forget gate, input gate and output gate are realized through the sigmoid activation function to capture the long-term trend and short-term fluctuation of traffic flow data. The attention mechanism is introduced in the second hidden layer, and the hidden state is converted into an attention weight matrix using the softmax function. The attention score of each spatiotemporal position is calculated according to the weight matrix, and the key sections and time periods are highlighted. The attention distribution is multiplied element by element with the hidden state to obtain the attention-enhanced hidden state. The enhanced hidden state is passed into a fully connected layer containing 256 neurons, and the activation function is ReLU, which outputs the traffic flow forecast value of each monitoring point in the next 1 hour. The mean square error is used as the loss function, the Adam optimization algorithm is adopted, the learning rate is set to 0.01, the batch size is 64, and 500 epochs are trained. The model performance was evaluated on the test set, with an average absolute percentage error of 2%, which is better than the traditional autoregressive model and support vector machine regression model. This proves that the introduction of the attention mechanism can effectively improve the prediction accuracy of the long short-term memory network for traffic flow.

[0050] S105. Based on the above-mentioned long short-term memory network, a graph convolutional network is introduced to model spatial correlation. The road network topology is described by the adjacency matrix. The convolution operation is used to aggregate the feature information of adjacent nodes to capture the evolution law of traffic flow in the local road network and enhance the spatial perception ability of the model.

[0051] Specifically, in order to construct the adjacency matrix of the road network topology, the NetworkX library can be used. First, the road network is abstracted into an undirected graph, where intersections are represented as nodes and road segments are represented as edges. Then, the adjacency matrix is ​​generated using the adjacency_matrix function of NetworkX. The matrix is ​​an N×N two-dimensional array, where N is the number of nodes. The element A[i][j]=1 in the matrix indicates that there is a connection between node i and node j, and A[i][j]=0 indicates that there is no connection. Next, the characteristic information of each node in the road network is obtained, including traffic flow, vehicle speed and other traffic status data. Traffic data can be collected regularly by installing sensors at each intersection, and the data is transmitted to a central server for processing. For each node, its historical traffic flow data is extracted, and a time series data set is constructed as the input of the graph convolutional network. In the graph convolution operation, for each node, its adjacent nodes are determined by the adjacency matrix, and the features of the adjacent nodes are aggregated using a weighted summation method. For example, the graph convolution formula approximated by Chebyshev polynomials can be used to update the feature vector x_i of node i to the weighted sum of the feature vectors of its adjacent nodes, where the weight is determined by the Chebyshev polynomial coefficients. The aggregated node feature vector can capture the evolution law of traffic flow in the local road network. The aggregated node features are input into the long short-term memory network, and the time series data is modeled through the gating mechanism. The long short-term memory network contains input gates, forget gates, and output gates. By adjusting the weights of the gates, the long-term dependencies and time evolution laws of traffic flow can be adaptively captured. The results of graph convolution are introduced into the hidden state of the long short-term memory network to fuse the spatial correlation features with the time series features, thereby enhancing the modeling ability of the model for complex traffic scenes. The node feature vector obtained by graph convolution can be fused with the hidden state of the long short-term memory network by splicing or element-wise multiplication to obtain the fused spatiotemporal feature representation. By stacking multiple layers of graph convolution and long short-term memory networks, high-level spatiotemporal features are gradually extracted. For example, a 3-layer graph convolution and a 2-layer long short-term memory network can be used, with the output of each layer as the input of the next layer, to gradually abstract the evolution law of traffic flow in the local road network into a global spatiotemporal dependency, thereby enhancing the model's ability to describe the evolution law of traffic flow. Finally, based on the fused spatiotemporal features, the fully connected layer is used to map them to the output space to obtain the traffic flow prediction results for each future time step. For example, a fully connected layer with 128 neurons can be used to map the spatiotemporal features into traffic flow prediction values ​​every 5 minutes in the next hour to complete the spatiotemporal prediction task of traffic flow.

[0052] S106. During the model training and reasoning phase, a distributed computing framework is used to implement parallel model training and real-time prediction. Through data sharding and model parallel technology, the above-mentioned large-scale traffic data and complex models are divided into multiple subtasks and assigned to different computing nodes for simultaneous processing, thereby improving the efficiency of model training and prediction and achieving quasi-real-time traffic flow prediction.

[0053] In this embodiment, the LSTM and GCN models are converted to TensorFlow Serving format and deployed on 10 model servers. Model compression technology is used to quantize model parameters into 8-bit integers to reduce the model size and accelerate the reasoning process. Through the load balancing strategy, the reasoning requests are distributed to different model servers. Using multi-model parallel technology, the processing of a single request is completed within 5ms to meet the latency requirements of real-time prediction. At the same time, a model integration strategy is introduced to train multiple LSTM and GCN models with different hyperparameters. The prediction results of each model are combined by weighted averaging to improve the overall prediction accuracy, achieving an accuracy rate of more than 90% in complex traffic scenarios.

[0054] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A traffic flow prediction method based on artificial intelligence, characterized in that: The following steps are involved: Collect historical traffic data and perform cleaning and preprocessing to obtain preprocessed data; Based on the preprocessed data, extracting traffic flow statistical characteristics at different time scales, and calculating the spatial correlation between road nodes and road sections according to the road network topology structure; Quantifying heterogeneous data into numerical features and fusing them with the traffic flow statistical features to obtain a comprehensive feature representation; Based on the comprehensive feature representation and spatial correlation, a traffic flow prediction model is constructed through a long short-term memory network and a graph convolutional network; The traffic flow prediction model is used to predict future traffic flow and generate prediction information, which is then visualized.

2. The traffic flow prediction method based on artificial intelligence according to claim 1 is characterized in that: The method for obtaining preprocessed data comprises: A big data processing framework is selected to build a data processing environment, and distributed storage and parallel computing of historical traffic data are carried out. Noise and outliers are identified and eliminated through anomaly detection algorithms, and data interpolation methods are used to fill in missing values ​​to obtain preprocessed data.

3. The traffic flow prediction method based on artificial intelligence according to claim 1 is characterized in that: The method for acquiring the traffic flow statistical characteristics includes: The pre-processed data is processed by using spatiotemporal data fusion technology, and the mean, variance and peak value of traffic flow at each time granularity are statistically calculated in the time dimension according to hours, days and weeks as traffic flow statistical features.

4. The traffic flow prediction method based on artificial intelligence according to claim 1 is characterized in that: The calculation method of the spatial correlation includes: Graph theory algorithms are used to calculate the spatial correlation between road nodes and sections, and to obtain an adjacency matrix that reflects the spatial dependency between nodes and sections. According to the adjacency matrix, a spatial autocorrelation analysis method is used to calculate the spatial correlation of nodes and sections in the road network.

5. The traffic flow prediction method based on artificial intelligence according to claim 1 is characterized in that: The method for obtaining the comprehensive feature representation comprises: The heterogeneous data includes weather and holiday heterogeneous data. Numerical features of temperature and precipitation are extracted from weather data, and holiday data are converted into categorical features using one-hot encoding. By using data fusion technology, the numerical features and categorical features are combined with the traffic flow statistical features to obtain a comprehensive feature representation.

6. The traffic flow prediction method based on artificial intelligence according to claim 1 is characterized in that: The method for constructing a traffic flow prediction model comprises: The first model is obtained by training the long short-term memory network through comprehensive feature representation and spatial correlation; The spatial correlation is modeled through a graph convolutional network to obtain a second model; The model integration technology is used to integrate the first model and the second model to obtain the traffic flow prediction model.

7. The traffic flow prediction method based on artificial intelligence according to claim 6 is characterized in that: When constructing the first model, an attention mechanism is introduced to dynamically adjust the importance weights of different spatiotemporal positions.

8. The traffic flow prediction method based on artificial intelligence according to claim 6 is characterized in that: In the model prediction stage, a distributed computing framework is used for parallel real-time prediction. Through data sharding and model parallel technology, traffic data and traffic flow prediction models are divided into multiple subtasks and assigned to different computing nodes for simultaneous calculation.

Images