Intelligent visual analysis and early warning method for traffic congestion by integrating multi-source heterogeneous big data

Abstract

The present invention relates to an intelligent visual analysis and early warning method for traffic congestion that integrates multi-source heterogeneous big data, and belongs to the field of electronic digital data processing technology. It takes into account the impact of multiple factors on road speed, adopts the technology of multi-source heterogeneous information fusion to vectorize the factors affecting road speed, and proposes a road speed prediction model that integrates multiple source influencing factors. At the same time, it introduces the concept of speed performance index to divide urban traffic status and extract congested sections. It provides an instance-level explanation for the road speed prediction model and explores the key factors of road congestion. Integrating the above-mentioned methods, the visual analysis system designed and implemented includes several linked views and novel visualization designs, which can perceive the spatial dependencies and temporal evolution patterns of traffic congestion, and can quickly identify the causes of congestion, helping field experts to intervene as early as possible, formulate reasonable road planning, and avoid unnecessary traffic congestion.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic digital data processing, and in particular relates to an intelligent visual analysis and early warning method for traffic congestion by integrating multi-source heterogeneous big data. Background Art

[0002] On the one hand, traffic congestion will increase people's travel time and costs, which in turn affects the rapid development of the social economy; on the other hand, the probability of traffic accidents will rise sharply, road traffic order will be seriously disrupted, and people's lives will be threatened; in addition, road vehicles are in an alternating state of starting and stopping for a long time, resulting in an increase in exhaust emissions, indirectly affecting the improvement of air quality and hindering the implementation of the concept of sustainable development. Therefore, alleviating traffic congestion is urgent.

[0003] Accurately predicting urban traffic congestion is crucial for effectively alleviating urban traffic congestion and creating a comfortable and convenient travel environment. With the rapid development of next-generation artificial intelligence (AI) and big data technologies, many researchers are working to use machine learning methods to predict road speeds, providing powerful analytical and decision-making support for modern smart transportation. However, existing machine learning methods face a performance-interpretability trade-off. On the one hand, researchers are striving to build more complex models and more diverse features to improve the accuracy of model speed prediction. On the other hand, most machine learning models remain complex black boxes. Decision makers want to understand the model's decision-making basis, particularly the impact of various factors on traffic congestion from a model perspective. However, the complexity of the models and the diversity of their features pose challenges in interpreting the models. Machine learning interpretability analysis can enhance model transparency while ensuring accurate predictions and further quantify the importance of influencing factors. However, existing interpretability methods can only provide numerical explanations, making it difficult for decision makers to understand the temporal and spatial evolution of traffic congestion and the changes in influencing factors, nor can they explore the inherent relationships between influencing factors and their importance.

[0004] Therefore, at this stage, it is necessary to design an intelligent visual analysis and early warning method for traffic congestion that integrates multi-source heterogeneous big data to solve the above problems. Summary of the Invention

[0005] The present invention aims to provide an intelligent visual analysis and early warning method for traffic congestion that integrates multi-source heterogeneous big data. This method addresses the technical issues inherent in the prior art and provides interpretable analysis of traffic congestion from both a global overview and detailed exploration perspective. First, considering the impact of multiple factors on road speed, the method employs multi-source heterogeneous information fusion to vectorize each factor influencing road speed. A road speed prediction model incorporating these factors is proposed. The concept of a speed performance index is introduced to categorize urban traffic conditions and identify congested road sections. Next, a quantitative analysis method for factors influencing road speed based on SHAP interpretability is proposed, providing instance-level explanations for the road speed prediction model and identifying key factors contributing to road congestion. Integrating these methods, a visual analysis system is designed and implemented, comprising several interconnected views and a novel visualization design. This system understands the spatial dependencies and temporal evolution of traffic congestion and can quickly identify the causes of congestion, enabling domain experts to intervene early, formulate rational road plans, and avoid unnecessary congestion.

[0006] To achieve the above object, the technical solution of the present invention is:

[0007] The intelligent visual analysis and early warning method for traffic congestion that integrates multi-source heterogeneous big data includes the following steps:

[0008] S1. Considering the multi-source influencing factors of road speed, we use multi-source heterogeneous information fusion technology to vectorize each factor affecting road speed. From the perspective of multi-source influencing factors, we propose a road speed prediction model that integrates multi-source spatiotemporal data. This model can perceive urban traffic conditions, extract congested sections, and discover the spatiotemporal variation patterns of congestion, thus assisting decision makers in taking timely intervention measures to alleviate traffic congestion.

[0009] S2. Model the response relationship between influencing factors and road speed using the XGBoost model. A method for mapping multi-source influencing factors to characteristic factors is proposed. This method explores the nonlinear effects of road network structure, weather, air quality, and holiday characteristic factors on road speed, and explores the intrinsic relationship between road characteristic factors and the importance of their corresponding features.

[0010] S3. A visual analysis framework for factors affecting traffic congestion based on explainable machine learning is proposed. This framework can provide insights into traffic congestion trends through a comprehensive overview, detailed exploration, multi-view linkage, and rich interactive methods. It not only supports comparative analysis between multiple roads, but also conducts instance-level exploration of a single road. Time series views and spatial distribution views are designed to discover changes in road congestion in time and space. A multi-level visual analysis system for factors affecting traffic congestion is designed and implemented to help decision makers explore and perceive potential influencing factors of congestion.

[0011] Furthermore, step 2 includes the following sub-steps:

[0012] S21. Traffic state classification. Road speed is the most common indicator for measuring traffic conditions. The Road Speed Performance Index (RSPI) is introduced as a traffic state evaluation indicator. The predicted speeds obtained through machine learning are used to classify traffic states, thereby identifying congested roads and analyzing influencing factors. Traffic state classification compares the current road speed with the maximum allowable speed. The calculation method is shown in the following formula.

[0013]

[0014] Among them, R v Represents the speed performance index, R v ∈[0,100]; select the maximum speed in each time slice on each road as the maximum driving speed V max ; The road speed at each time slice is taken as the average travel speed v; the speed performance index of each time slice on each road is calculated, and the road congestion situation is judged based on the index classification.

[0015] S22. Quantitative analysis of factors influencing road speed: This method uses an interpretable machine learning framework to study the nonlinear effects of road attributes, weather, air quality, and holiday characteristics on road speed, and quantifies the degree of influence of these characteristics to help users analyze the factors affecting road congestion.

[0016] S23. Similarity mining of factors affecting road speed: Dimensionality reduction technology is used to project the 77-dimensional road feature importance into a two-dimensional space. The closer the distance between the scattered points, the more similar the feature importance values of the corresponding roads are.

[0017] Furthermore, in step S21:

[0018] Three thresholds of 25, 50, and 75 are set as the classification criteria for urban road traffic status. For the taxi trajectory dataset, the speed performance index of all roads in each time slice is calculated. Based on the speed results obtained by the XGBoost-based road speed prediction model, the corresponding speed performance index is calculated.

[0019] Furthermore, step S22 is specifically as follows:

[0020] The road speed attribution method uses the SHAP method to quantify the feature importance of the XGBoost model output, thereby explaining the influencing factors. SHAP is based on cooperative game theory and the idea of local explanation. It estimates the contribution of each input feature to the prediction result based on its marginal contribution, thereby explaining each instance. The input is a 77-dimensional road feature vector and a trained road speed prediction model. Using the SHAP-based feature importance quantification method, a local explanation for each instance feature is ultimately obtained. The feature explanations divided into a subspace are then accumulated through the resultMap mapping relationship to obtain the explanation of each feature factor.

[0021] SHAP's global attribution analysis interprets the overall road influencing factors by calculating the average and then performing a causal analysis. During the local interpretation process, each individual instance generates a set of Shapley values, and all instances in the dataset are combined to construct an explanation matrix. Each row of the matrix represents an instance, and each column represents the contribution of each feature. To perform a global interpretation and explore the overall contribution of the features to the model's prediction results, the average of the absolute values of each feature's Shapley values is used as a measure of the corresponding feature's importance. A larger value indicates a more important feature and a greater impact on the prediction results.

[0022] Furthermore, step S23 is specifically as follows:

[0023] We introduced a data augmentation method and designed a feature vector dimensionality reduction view. This view can intuitively display the scatter distribution of the road feature importance of any feature and support tracing the scatter points back to two-dimensional space. The "fish-shaped" scatter points represent the location where the road feature importance value of each road's 77-dimensional features is reduced in dimension using UMAP and projected into two-dimensional space. The specific steps are as follows:

[0024] (1) Selecting specific one-dimensional features to analyze the similarity of factors affecting road speed;

[0025] (2) The importance values of specific features are divided into five attribute boxes from small to large, namely "very low", "low", "medium", "high", and "very high";

[0026] (3) Select an attribute box and filter the data points within the corresponding range;

[0027] (4) Calculate the Delaunay triangulation of the filter points;

[0028] (5) Set the threshold ε: Set the two vertices and If and only if the distance between two points is greater than the selected distance threshold ε, the triangle to which the line connecting the two points belongs is deleted. For the selection of the threshold, we calculate the pairwise distance between each scattered point after UMAP dimensionality reduction, generate a distance matrix, and store it through CSR compressed sparsity. The minimum spanning tree MST is calculated to provide a default value for the filter threshold ε, and the threshold ε is obtained as follows:

[0029] ε=mst 75 +1.5·(mst 75 -mst 25 );

[0030] where mst 75 Represents the 75th percentile of the MST edge length; mst 25 Represents the 25th percentile of the MST edge length; mst 75 -mst 25 Represents the interquartile range of side lengths.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] One of the benefits of this solution is that it provides an interpretable analysis of traffic congestion from both a global overview and detailed exploration perspective. First, considering the impact of multiple factors on road speed, a multi-source heterogeneous information fusion technique is used to vectorize the various factors influencing road speed. A road speed prediction model that integrates these factors is proposed. The concept of a speed performance index is also introduced to classify urban traffic conditions and identify congested sections. Then, a quantitative analysis method for factors influencing road speed based on SHAP interpretability is proposed, providing instance-level explanations for the road speed prediction model and uncovering the key factors contributing to road congestion. Integrating these methods, a visual analysis system was designed and implemented, comprising several interconnected views and novel visualization designs. This system can perceive the spatial dependencies and temporal evolution of traffic congestion and quickly identify the causes of congestion, enabling domain experts to intervene early, formulate reasonable road plans, and avoid unnecessary traffic congestion. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is the traffic status distribution provided by the present invention.

[0034] Figure 2 Schematic diagram of the quantitative analysis framework of factors affecting road speed based on SHAP interpretability in the present invention.

[0035] Figure 3 This is a schematic diagram of how the Shapley value provided by the present invention drives the prediction results.

[0036] Figure 4Schematic diagram of the road feature importance dimensionality reduction diagram provided by the present invention: (a) LLE; (b) LPP; (c) UMAP.

[0037] Figure 5 This is a schematic diagram of the coloring situation provided by the present invention.

[0038] Figure 6 Schematic diagram of the importance distribution of road features of the road entropy provided by the present invention.

[0039] Figure 7 A comparison diagram of the road feature importance value distribution provided by the present invention: (a) road speed at the previous moment; (b) road level.

[0040] Figure 8 This is the SHAP summary diagram of road attributes provided by the present invention.

[0041] Figure 9 This is the SHAP summary diagram of the holiday features provided by the present invention.

[0042] Figure 10 SHAP summary diagram of air quality characteristics provided by the present invention.

[0043] Figure 11 SHAP summary graphs of hourly features provided by the present invention (a) 0 hours to 10 hours; (b) 11 hours to 22 hours.

[0044] Figure 12 This is a SHAP summary diagram of weather characteristics provided by the present invention. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for the purpose of explaining the present invention and are not intended to limit the present invention. That is, the embodiments described herein are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and illustrated in the drawings herein can be arranged and designed in various different configurations.

[0046] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely intended to represent selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0047] This paper proposes an intelligent visual analysis and early warning method for traffic congestion that integrates multi-source heterogeneous big data. The method includes the following steps:

[0048] S1. Considering the multi-source influencing factors of road speed, we use multi-source heterogeneous information fusion technology to vectorize each factor affecting road speed. From the perspective of multi-source influencing factors, we propose a road speed prediction model that integrates multi-source spatiotemporal data. This model can perceive urban traffic conditions, extract congested sections, and discover the spatiotemporal variation patterns of congestion, thus assisting decision makers in taking timely intervention measures to alleviate traffic congestion.

[0049] S2. Model the response relationship between influencing factors and road speed using the XGBoost model. A method for mapping multi-source influencing factors to characteristic factors is proposed. This method explores the nonlinear effects of road network structure, weather, air quality, and holiday characteristic factors on road speed, and explores the intrinsic relationship between road characteristic factors and the importance of their corresponding features.

[0050] First, we introduce the principles for classifying road traffic conditions to extract congested roads. We then focus on predicting road speeds, using a SHAP-based feature importance quantification method to explain and analyze the factors that influence road speed changes from a feature perspective.

[0051] Road speed is the most common indicator for measuring traffic conditions. The Speed Performance Index (SPI) is introduced as a traffic condition evaluation indicator. The predicted speed obtained through machine learning is used to classify traffic conditions, thereby extracting congested roads for influencing factor analysis. The basic idea of traffic condition classification is to compare the current road speed with the maximum allowed speed. The calculation method is shown in Formula 3-1. v Represents the speed performance index, R v ∈[0,100]; select the maximum speed in each time slice on each road as the maximum driving speed V max The road speed at each time slice is taken as the average travel speed v. The speed performance index of each time slice on each road is calculated, and the road congestion situation is determined based on the index classification.

[0052]

[0053] We set three thresholds of 25, 50, and 75 as the classification criteria for urban road traffic status, as shown in the following table.

[0054]

[0055] For the taxi trajectory dataset of City C in October 2021, the speed performance index of all roads in each time slice is calculated, and the distribution is plotted in a ring diagram, as shown in the following example: Figure 1As shown in the figure, severe congestion accounts for 2.03% of the traffic conditions in City C, while mild congestion accounts for 21.53%, which is consistent with the typical urban congestion distribution and indicates that this dataset is usable. Based on the speed results obtained by the XGBoost-based road speed prediction model, we calculated the corresponding speed performance index and extracted congested road sections and times in the visual analysis system, facilitating the perception of spatiotemporal changes in congestion and analyzing the factors affecting it.

[0056] Given the black-box nature of machine learning models and the complexity of multi-source spatiotemporal data, researchers are unable to perform logical reasoning on the model's decision-making results. To better understand the internal logic behind machine learning models' decisions, scholars at home and abroad are committed to studying simple, human-understandable linear algorithms as surrogate models for model decision-making. However, building explanatory models that approximate machine learning and proving their usability remain significant challenges for decision-makers who lack machine learning knowledge. This paper uses an interpretable machine learning framework to study the nonlinear effects of characteristic factors such as road attributes, weather, air quality, and holidays on road speeds, and quantifies the degree of influence of these characteristic factors to help users analyze the factors affecting road congestion.

[0057] The road speed attribution method is based on the SHAP method to quantify the importance of features in the output of the XGBoost model, thereby explaining the influencing factors. SHAP is a method proposed by Lundberg and Lee in 2017 to explain the output of the model. SHAP is based on cooperative game theory and the idea of local explanation. It estimates the contribution of each input feature to the prediction result based on its marginal contribution, thereby explaining each instance. The explanation framework for the factors affecting road speed in this experiment is as follows: Figure 2 As shown, the input is a 77-dimensional road feature vector and a trained road speed prediction model. According to the SHAP-based feature importance quantification method, a local explanation for each instance feature is finally obtained. Then, through the resultMap mapping relationship, the feature explanations divided into a subspace are accumulated to obtain the explanation of each feature factor. The following is a detailed introduction to the SHAP-based feature importance quantification method. SHAP local attribution analysis is to explain a certain instance and perform causal analysis. By training the SHAP method, any feature of each instance will generate a Shapley value, which means the contribution value to the output prediction result of the model that needs to be explained. For the j-th feature, the weighted sum of all possible feature value combinations is taken to obtain the Shapley value of the feature, as shown in Formula 3-2. Where S is the feature subset in the model, {x1,…,x m} represents the set of all input features, m is the number of all features, {x1,…,x m}\{xj} does not include {x j} all possible combinations of input features, f * (S) represents the predicted feature values in feature set S. Considering the order of these m features, there are m! permutations and combinations. If the jth feature is fixed, there are |S|! (m - |S| - 1)! combinations remaining. |S|! (m - |S| - 1)! m! is the percentage of feature combinations in subset S, which serves as the weight of subset S.

[0058]

[0059] Assume f * Represents the original prediction model that needs to be explained, and g represents the explanation model. Based on the idea of local explanation, the explanation model is used to explain the output of the original prediction model. Usually, the input x′ of the explanation model is a simplification of the original model input x, which can be converted by a function, such as x = h x (x′), and make g(z′)≈f * (h x (z′)), where z′≈x′. SHAP interprets the Shapley value as a cumulative feature attribution method, which can be used as shown in Formula 3-3, where z′∈{0,1} M Indicates whether the corresponding feature can be observed, if it is observed, it is 1, otherwise it is 0; M represents the number of simplified input features, represents the feature attribution Shapley value of the j-th feature, is a constant that explains the model and represents the predicted mean of all training samples.

[0060]

[0061] The SHAP analysis of a single sample is performed by calculating the Shapley value of each feature, and the output value of each sample obtained by the road speed prediction model is the sum of the Shapley values of all features and the average prediction output, as shown in the following example: Figure 3 As shown in the figure, we can see how each feature contributes to the prediction results. For the sample {x1=a1,x2=a2,x3=a3,x4=a4}, we calculate the Shapley value of each feature. The predicted mean of all training samples Explanation model g's prediction of the sample is explained as the The sum of the predicted mean of all training samples is summed, that is, When the sample prediction value is greater than the prediction mean When , we believe that the feature as a whole has a positive impact on the instance, making the prediction larger; when the sample prediction value is less than When , we believe that the feature as a whole has a negative impact on the instance, making the prediction smaller.

[0062] SHAP's global attribution analysis is to perform causal analysis after explaining the overall road influencing factors by calculating the average. In the local explanation process, each individual instance will generate a set of Shapley values, and all instances in the data set will be combined to construct an explanation matrix. Each row of the matrix represents an instance; each column represents the contribution of each feature. In order to conduct a global explanation and explore the overall contribution of the features to the model prediction results, the average of the absolute value of the Shapley value of each feature is used as the importance metric of the corresponding feature. The larger the value, the more important the feature is and the greater the impact on the prediction results. The specific calculation is shown in Formula 3-4. Where n represents the number of samples in the data set, Indicates the Shapley value of the jth feature of the i-th sample.

[0063]

[0064] In order to explore the similarities and differences in the degree of influence of road speed factors on the overall city, we calculated the global feature importance values of 77 influencing factors {FI1, FI2, ..., FI 77}, sorting the obtained global feature importance in descending order reveals the relationship between feature influences for attribution analysis. To better explore the overall impact of a road feature on road speed prediction, a road feature importance representation method based on the SHAP interpretability method is proposed for analyzing factors influencing road speed at the road level. The specific implementation steps are shown in Algorithm 2.

[0065]

[0066] After obtaining the global feature importance {FI1,FI2,…,FI 77} and the road feature importance matrix FIn′×m′, we use resultMap to map the relationship and accumulate the feature importance values of the influencing factors divided in the same subspace to obtain the global feature importance of the ten feature factors and the road feature importance, which is convenient for effective attribute attribution in the subsequent analysis of the influencing factors of congested roads.

[0067] Because different indicators have different dimensions and units, we need to standardize the data before analyzing the factors affecting road speed to address the different nature of the data. Common standardization methods include "normalization," "logarithmic transformation," and "Z-score standardization." Normalization linearly transforms the original data to bring the results into the real number range [0, 1]. Its advantages are simplicity and computational convenience. Therefore, we decided to use "normalization" and normalize the predicted speed, Shapley value, global, and road feature importance of each instance to [0, 1], as shown in Formula 3-5. Here, ξ represents the original true value; min(ξ) represents the minimum sample data value; max(ξ) represents the maximum sample data value; and ξ′ represents the normalized value.

[0068]

[0069] In analyzing factors influencing road speed, discovering similarities in factors across roads is crucial. This facilitates finding roads with similar congestion causes and provides a basis for users to discover roads of interest. However, humans cannot visually capture high-dimensional data; their most intuitive understanding of data is in two-dimensional space. Therefore, we use dimensionality reduction techniques to project the 77-dimensional road feature importance into two-dimensional space. The closer the distance between scattered points, the more similar the feature importance values of the corresponding roads.

[0070] Existing dimensionality reduction methods are divided into linear and nonlinear dimensionality reduction. Linear dimensionality reduction methods preserve the most important components in high-dimensional data. However, many data exhibit nonlinear relationships. Nonlinear dimensionality reduction methods can better identify topological structures and embedding patterns between data. In selecting dimensionality reduction methods, four nonlinear dimensionality reduction methods were tried: multidimensional scaling, locally linear embedding, locally preserving projection, consistent manifold approximation, and projection.

[0071] (1) The core idea of Multidimensional Scaling (MDS) is to use n points in a low-dimensional space to represent the Euclidean distance or similarity between n samples in a high-dimensional space. That is, the Euclidean distance between samples in the original high-dimensional space can remain consistent or nearly consistent in the low-dimensional space. Therefore, we first need to calculate the Euclidean distance between any two samples to construct a distance matrix. However, due to the large number of samples in the dataset of this experiment, the generated distance matrix is too large, resulting in memory overflow and inability to complete the dimensionality reduction calculation. MDS is not suitable as a dimensionality reduction method for this experiment.

[0072] (2) The core idea of Locally Linear Embedding (LLE) is to transform the global nonlinearity of the data into local linearity. Any point in the dataset is described by a combination of the weights of its neighboring nodes, and a low-dimensional embedding of the data is found by optimizing the eigenvectors, so that the weight of each point in the low-dimensional space remains unchanged from the weight of each point in the high-dimensional space. This method can better preserve the spatial structure of high-dimensional data.

[0073] (3) The core idea of Locality Preserving Projection (LPP) is to fuse the neighborhood information of each node and construct a network graph. Then, based on the Laplace principle, a transformation matrix is generated. Ultimately, this matrix maps high-dimensional data into a low-dimensional space. Therefore, LPP can optimally preserve the local neighborhood information of the data.

[0074] (4) Uniform Manifold Approximation and Projection (UMAP) performs better in some aspects of preserving the global structure of the data and is usually faster than other dimensionality reduction methods. Its algorithm is mainly divided into two parts: first, a specific weighted k-nearest neighbor graph is constructed in the high-dimensional space, and then the low-dimensional layout of the k-nearest neighbor graph is calculated.

[0075] This experiment visualizes the results of dimensionality reduction of 77-dimensional road feature importance, such as Figure 4 As shown, Figure 4 a, b, and c in the figure show the dimensionality reduction effects of LLE, LPP, and UMAP, respectively. Figure 4 a and Figure 4 The scattered points in b in the figure overlap too much and are concentrated in one area, which makes it impossible to perceive the differences between the scattered points. Figure 4 The distribution of scattered points in c in the figure is generally concentrated, while there is a certain distance between adjacent sample points, which perfectly reflects the similarity and heterogeneity of road feature importance. Therefore, this experiment uses the Unified Mapping (UMAP) dimensionality reduction technique to reduce the importance of road features. Furthermore, the majority of scattered points in the figure are located on the right side, while there are fewer scattered points on the left side. Some scattered points are clustered separately, far from the overall distribution. This provides interesting samples for users to explore similarities in factors affecting road speed.

[0076] In order to discover the relationship between each road feature and its importance distribution, a data enhancement method is introduced and a feature vector dimensionality reduction view is designed. This view can intuitively show the scattered distribution of the importance of any feature of the road feature and support tracing the scattered points back to two-dimensional space, such as Figure 6As shown. The "fish-shaped" scattered points represent the location where the road feature importance value of each road's 77-dimensional features is reduced in dimension using UMAP and projected into two-dimensional space. The drop-down menu in the upper right corner allows you to select one of the 77-dimensional features for analysis. The colored area in the figure represents the size distribution of the road feature importance value of one of the 77-dimensional features, as shown in the figure. Figure 5 As shown, its main practices are as follows:

[0077] (1) Selecting a specific one-dimensional feature to analyze the similarity of factors affecting road speed, such as road entropy;

[0078] (2) The feature importance values of road entropy are divided into five attribute boxes from small to large, namely "very low", "low", "medium", "high", and "very high";

[0079] (3) Select an attribute box and filter the data points within the corresponding range;

[0080] (4) Calculate the Delaunay triangulation of the filter points;

[0081] (5) Set the threshold ε: Set the two vertices and Connect, if and only if the distance between the two points is greater than the selected distance threshold ε, then delete the triangle to which the line connecting the two points belongs. For the selection of the threshold, we calculate the distance between each two scattered points after UMAP dimensionality reduction, generate a distance matrix, and store it through CSR compressed sparsity. Next, calculate the minimum spanning tree MST to provide a default value for the filter threshold ε, and obtain the threshold ε as shown in Formula 3-6. Where mst 75 Represents the 75th percentile of the MST edge length; mst 25 Represents the 25th percentile of the MST edge length;

[0082] mst 75 -mst 25 Represents the interquartile range of side lengths.

[0083] ε=mst 75 +1.5·(mst 75 -mst 25 ) (3-6)

[0084] (6) Get the boundary of the triangulation and find all the points in the current range that do not belong to the contour and highlight them.

[0085] Regarding the feature of “road entropy”, we Figure 6As can be seen from the figure, the distribution of road feature importance values is vertical. Features located below the fish body have extremely high importance values and a significant impact, leading to significant changes in road speeds. This indicates that these roads are significantly affected by "road entropy," potentially significantly increasing predicted speeds to alleviate congestion or significantly reducing predicted speeds to cause congestion. This view only provides a preliminary overview of the distribution of road feature influence. In our subsequently designed multi-level visual analysis system, we will be able to select these roads to explore factors influencing road speeds.

[0086] for Figure 7 The road feature importance values for the two features shown, "Previous Target Road Speed" and "Road Class," are similarly distributed horizontally. Roads at the tail of the fish have extremely high feature importance values, with a significant impact, causing significant changes in road speed. This indicates that these roads are significantly affected by the "Previous Target Road Speed" and "Road Class" features. Meanwhile, features at the head of the fish have lower feature importance values and a smaller impact on road speed, indicating that the speed of these roads is not significantly affected by the "Previous Target Road Speed" and "Road Class" features. Therefore, we believe that the "Previous Target Road Speed" and "Road Class" features have a positive correlation in their impact on road speed, which can be further explored in the visual analysis system.

[0087] The SHAP summary plot shows how changes in eigenvalues affect feature importance. The summary plot includes all instances in the dataset, with each point representing the Shapley value of a particular feature in that instance. The y-axis encodes the feature, the x-axis encodes the Shapley value, and the color encodes the magnitude of the original eigenvalue. A "viridis" color scheme is used for color mapping, with purple representing lower eigenvalues and yellow representing higher eigenvalues. Overlapping points are jittered along the y-axis to help us understand the distribution of each feature's Shapley value and the relationship between the original eigenvalue and the feature's Shapley value. Features are sorted in descending order by the mean absolute value of their Shapley values.

[0088] We analyze each influencing factor one by one according to the mapping relationship of resultMap. First, we analyze the factors affecting road speed based on the three characteristic factors of "speed", "road entropy" and "road level". Figure 8As can be seen, road attributes influence the predicted road speed. The seven influencing factors in the figure have essentially the same influence on the model's prediction results. As the characteristic value increases, the Shapley value increases, positively influencing the model's predicted value and increasing it. The characteristic for road speed is the road speed at the previous moment. We can see that the road speed at the previous moment significantly influences the road speed at the next moment, and the speed at the target road at a certain moment has a greater impact on the model's prediction results than the speed at the same moment on adjacent roads. Regarding the "road class" feature, higher road classes have larger Shapley values, resulting in higher road speeds. Roads that meet national standards have higher design speeds.

[0089] Depend on Figure 9 It can be seen that holiday characteristics have an impact on road speed prediction. The "holiday" feature value has only two inputs, "0" or "1". The yellow scatter points in the figure represent the "holiday" feature value of "1", and the purple scatter points represent the "holiday" feature value of "0". "0" means that the date to which the instance belongs is not a holiday, and "1" means that the date to which the instance belongs is a holiday. During holidays, the Shapley value is greater than zero, which has a positive effect on the model's prediction output, making the prediction result larger; during commuting, the Shapley value is less than zero, which has a negative effect on the model's prediction output, making the prediction result smaller. We can explain this as there is no commuting peak during holidays, and the Transportation Bureau of City C issued an announcement during the National Day holiday that overall transportation is safe, stable and orderly, with no safety accidents. Taxis in the airport and railway station areas achieved "zero complaints", indicating good traffic conditions and confirming the effectiveness of our explanatory model.

[0090] Depend on Figure 10 As can be seen, air quality and other features influence road speed prediction. The influence of features "O3" and "PM2.5" on the model's prediction results is quite distinct. When the "PM2.5" feature value increases, the Shapley value is greater than zero, positively impacting the model's prediction output and increasing the prediction result. However, when the "PM2.5" feature value decreases, the Shapley value decreases below zero, negatively impacting the model's prediction output and decreasing the prediction result. The impact of feature "O3" on the prediction results is exactly the opposite: larger "O3" feature values have a negative impact on the model's prediction output, while smaller "O3" feature values have a positive impact on the model's prediction output. When the "PM10" feature value is high, the yellow scattered points are located near the zero mark on the horizontal axis, indicating little impact on the model's prediction results. However, when the "PM10" value is low, it has a positive or negative impact on the model's prediction results. Features "NO2," "AQI," and "SO2" do not exhibit a clear pattern in their impact on the model's prediction results.

[0091] Depend on Figure 11 As can be seen, features such as hour have an impact on road speed prediction. Because the hour feature has 24 dimensions and memory overflow occurs during calculation, the hour feature is calculated twice. The hour feature has only two input values: "0" or "1." Yellow points in the figure represent hour feature values of "1," while purple points represent hour feature values of "0." A "0" indicates that the time of day of the instance falls outside the range of the hour feature, while a "1" indicates that the time of day of the instance falls within the range of the hour feature. The figure clearly shows that features "0 hour," "1 hour," "2 hours," "3 hours," "4 hours," and "5 hours" generally have a positive impact on the model's prediction output, increasing the prediction result. The positive impact is greatest at "2 hours." We can infer that these positively impacting hourly features occur in the early morning hours, when there are fewer vehicles on the road and the road conditions are good, preventing speed reductions due to congestion. However, over time, the positive impact weakens significantly at "5 hours," and by "6 hours" and "7 hours," the feature's impact on the model's prediction output becomes negative. Subsequent hourly features do not significantly influence the prediction results. Until the appearance of "15 hours," "16 hours," and "17 hours," the model's prediction output has a negative impact, causing the predicted result to decrease. These two time periods coincide with morning and evening rush hour traffic, during which citizens commute en masse, leading to severe urban congestion and slower road speeds. After the evening rush hour ends, traffic conditions ease, and the features "21 hours," "22 hours," and "23 hours" have a positive effect on the model's prediction output, causing the predicted result to increase.

[0092] Depend on Figure 12 As can be seen, weather and other characteristics influence road speed prediction. The representation of weather feature values is the same as for "hours" and "holidays." Yellow points indicate that the weather type for that instance corresponds to the characteristic weather type, while purple points indicate that the weather type for that instance does not. Since October 2021 in City C was early autumn, the weather was relatively stable, with no significant fluctuations in the "minimum temperature" and "maximum temperature," and thus had little impact on the predicted road speed. Furthermore, there was no severe weather, such as "blizzards," "sandstorms," or "haze," which had no impact on the model's predictions. To ensure that the model can make accurate predictions when used with new datasets and maintain good generalization, these weather features are also considered as factors affecting road speed. For this dataset, the figure shows that the feature "moderate rain" reduces the Shapley value. Negative Shapley values negatively impact the model's predictions, resulting in lower speeds. This can be understood as moderate rain causing poor road conditions, reducing urban road capacity, slowing road speeds, or causing congestion.

[0093] The main focus is determining whether a road is congested and providing a detailed explanation of the factors influencing predicted road speeds. First, a speed performance index is used to approximate road traffic conditions, which are then divided and extracted to facilitate further exploration. The SHAP model is then used to quantitatively analyze the factors influencing predicted road speeds. The 77-dimensional road feature values and a trained machine learning model are used as input to the SHAP interpretation model. The Shapely value of each instance feature, representing the degree of influence of the feature on the prediction result, is obtained for subsequent visual analysis tasks. The relationship between feature value and feature importance is analyzed using a SHAP summary plot.

[0094] S3. A visual analysis framework for factors affecting traffic congestion based on explainable machine learning is proposed. This framework can provide insights into traffic congestion trends through a comprehensive overview, detailed exploration, multi-view linkage, and rich interactive methods. It not only supports comparative analysis between multiple roads, but also conducts instance-level exploration of a single road. Time series views and spatial distribution views are designed to discover changes in road congestion in time and space. A multi-level visual analysis system for factors affecting traffic congestion is designed and implemented to help decision makers explore and perceive potential influencing factors of congestion.

[0095] The visual analysis framework mainly includes two processes: the first process is the data preprocessing part, which includes data cleaning and feature vector construction, generation of road speed prediction model, extraction of congested sections and quantification of factors affecting road speed.

[0096] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions and effects do not exceed the scope of the technical solution of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. An intelligent visual analysis and early warning method for traffic congestion that integrates multi-source heterogeneous big data is characterized by: The steps include: S1. Considering the multi-source influencing factors of road speed, we use multi-source heterogeneous information fusion technology to vectorize each factor affecting road speed. From the perspective of multi-source influencing factors, we propose a road speed prediction model that integrates multi-source spatiotemporal data. This model can perceive urban traffic conditions, extract congested sections, and discover the spatiotemporal variation patterns of congestion, thus assisting decision makers in taking timely intervention measures to alleviate traffic congestion. S2. Model the response relationship between influencing factors and road speed using the XGBoost model. A method for mapping multi-source influencing factors to characteristic factors is proposed. This method explores the nonlinear effects of road network structure, weather, air quality, and holiday characteristic factors on road speed, and explores the intrinsic relationship between road characteristic factors and the importance of their corresponding features. S3. Propose a visual analysis framework for factors affecting traffic congestion based on explainable machine learning. Design time series and spatial distribution views to identify temporal and spatial variations in road congestion. Design and implement a multi-level visual analysis system for factors affecting traffic congestion to help decision makers identify and understand potential factors affecting congestion. Step 2 includes the following sub-steps: S21. Traffic state classification. Road speed is the most common indicator for measuring traffic conditions. The Road Speed Performance Index (RSPI) is introduced as a traffic state evaluation indicator. The predicted speeds obtained through machine learning are used to classify traffic states, thereby identifying congested roads and analyzing influencing factors. Traffic state classification compares the current road speed with the maximum allowable speed. The calculation method is shown in the following formula. Among them, R v Represents the speed performance index, R v ∈[0,100]; select the maximum speed in each time slice on each road as the maximum driving speed V max The road speed at each time slice is taken as the average travel speed v; the speed performance index of each time slice on each road is calculated, and the road congestion situation is judged based on the index classification; S22. Quantitative analysis of factors influencing road speed: This method uses an interpretable machine learning framework to study the nonlinear effects of road attributes, weather, air quality, and holiday characteristics on road speed, and quantifies the degree of influence of these characteristics to help users analyze the factors affecting road congestion. S23. Similarity mining of factors affecting road speed: Using dimensionality reduction technology, the 77-dimensional road feature importance is projected into a two-dimensional space. The closer the distance between scattered points, the more similar the feature importance values of the corresponding roads are. Step S22 is specifically as follows: The road speed attribution method uses the SHAP method to quantify the feature importance of the XGBoost model output, thereby explaining the influencing factors. SHAP is based on cooperative game theory and the idea of local explanation. It estimates the contribution of each input feature to the prediction result based on its marginal contribution, thereby explaining each instance. The input is a 77-dimensional road feature vector and a trained road speed prediction model. Using the SHAP-based feature importance quantification method, a local explanation for each instance feature is ultimately obtained. The feature explanations divided into a subspace are then accumulated through the resultMap mapping relationship to obtain the explanation of each feature factor. SHAP's global attribution analysis interprets the overall road influencing factors by calculating the average and then performing a causal analysis. During the local interpretation process, each individual instance generates a set of Shapley values, and all instances in the dataset are combined to construct an explanation matrix. Each row of the matrix represents an instance, and each column represents the contribution of each feature. To perform a global interpretation and explore the overall contribution of the features to the model prediction results, the average of the absolute values of the Shapley values of each feature is used as the importance metric for the corresponding feature. The larger the value, the more important the feature is and the greater its impact on the prediction results.

2. The intelligent visual analysis and early warning method for traffic congestion by integrating multi-source heterogeneous big data according to claim 1 is characterized in that: In step S21: Three thresholds of 25, 50, and 75 are set as the classification criteria for urban road traffic status. For the taxi trajectory dataset, the speed performance index of all roads in each time slice is calculated. Based on the speed results obtained by the XGBoost-based road speed prediction model, the corresponding speed performance index is calculated.

3. The intelligent visual analysis and early warning method for traffic congestion by integrating multi-source heterogeneous big data according to claim 2 is characterized in that: Step S23 is specifically as follows: We introduced a data augmentation method and designed a feature vector dimensionality reduction view. This view can intuitively display the scatter distribution of the road feature importance of any feature and support tracing the scatter points back to two-dimensional space. The "fish-shaped" scatter points represent the position where the road feature importance value of each road's 77-dimensional features is reduced by UMAP and projected into two-dimensional space. The specific steps are as follows: (1) Selecting specific one-dimensional features to analyze the similarity of factors affecting road speed; (2) The importance values of specific features are divided into five attribute boxes from small to large, namely "very low", "low", "medium", "high", and "very high"; (3) Select an attribute box and filter the data points within the corresponding range; (4) Calculate the Delaunay triangulation of the filter points; (5) Setting the threshold ε: Connect two vertices u and v. If and only if the distance between the two points is greater than the selected distance threshold ε, delete the triangle to which the line connecting the two points belongs. For the selection of the threshold, calculate the pairwise distances between the scattered points after UMAP dimensionality reduction, generate a distance matrix, and store it through CSR compressed sparsity. Calculate the minimum spanning tree MST to provide a default value for the filter threshold ε. The threshold ε is obtained as follows: ε=mst 75 +1.5·(mst 75 -mst 25 ); where mst 75 Represents the 75th percentile of the MST edge length; mst 25 Represents the 25th percentile of the MST edge length; mst 75 -mst 25 Represents the interquartile range of side lengths.

Images