Similar traffic accident sample identification method

Abstract

This invention relates to a method for identifying similar traffic accident samples, addressing the problems of inaccurate cluster counts and high computational complexity in noisy scenarios when using clustering algorithms for similar traffic accident sample identification. The method falls under the field of traffic safety. The application includes: inputting a dataset of n accident sample points and preprocessing it; calculating the similarity of all accident sample points in the preprocessed dataset to generate a similarity matrix; predicting the optimal number of clusters k based on the eigenvectors of the similarity matrix; constructing a degree matrix corresponding to the similarity matrix; calculating and constructing a regularized Laplacian matrix based on the similarity matrix and the degree matrix; solving for the eigenvectors corresponding to the k largest eigenvalues ​​in the matrix; constructing a k-dimensional feature matrix T; treating each row of the feature matrix T as a point in a k-dimensional space, resulting in n points; and using a clustering algorithm to cluster these n points to obtain the clustering results, thus completing the identification of similar traffic accident samples.

Description

Technical Field

[0001] This application relates to a method for identifying similar traffic accident samples, which belongs to the field of traffic safety. Background Technology

[0002] Traffic accidents are influenced by a variety of risk factors, but the types of risk factors considered in model construction are always limited, unable to include all potential influencing factors, and can only include some known variables. Due to the influence of unobserved factors, the effects of variables included in the model on traffic safety may exhibit heterogeneity (or variability) across different road sections or samples; this variability is called "unobserved heterogeneity." Heterogeneity between data groups can lead to decreased model performance and inconsistent results in the analysis of the impact of included model variables on traffic safety. Therefore, before analyzing and quantifying the impact of each risk factor on the severity of accidents, clustering methods can be used to divide the traffic accident dataset into different groups that maximize inter-group heterogeneity, thereby reducing intra-group heterogeneity.

[0003] Clustering methods can group together the complex factors influencing accidents to identify different accident patterns. By grouping similar accidents into one category, we can better understand their common characteristics and underlying causes. Commonly used clustering methods for processing traffic accident data include K-means clustering, hierarchical clustering, DBSCAN clustering, and spectral clustering. K-means clustering divides data points into K clusters, maximizing inter-cluster variance and minimizing intra-cluster variance. It is suitable for processing large amounts of data but is sensitive to initial points and may require multiple runs to find the optimal result. Hierarchical clustering creates a tree-like clustering structure, which can be bottom-up (agglomerated) or top-down (split-like). Suitable for datasets with small to medium amounts of data, it can generate clustering results at different levels. The DBSCAN algorithm, based on density space clustering, does not require a preset number of clusters, can identify cluster structures of arbitrary geometric shapes, and can effectively handle noisy datasets. This characteristic makes it uniquely valuable in analyzing traffic accident data with complex distributions. Spectral clustering, a graph theory-based method, performs clustering through the Laplace matrix decomposition of graphs. It has a significant advantage over other clustering algorithms when processing traffic accident data with high dimensions or non-convex shapes, but its computational complexity is high.

[0004] Furthermore, traditional spectral clustering relies on the Eigengap method to determine the number of clusters: assuming that "the first C eigenvalues ​​are significantly greater than the subsequent eigenvalues, and C is the number of clusters". However, this logic completely fails when the data contains noise. Noise will cause the eigenvalues ​​to deviate from the ideal distribution, thus making it impossible to accurately identify the number of clusters. Summary of the Invention

[0005] To address the issues of inaccurate cluster counts and high computational complexity in noisy scenarios when using clustering algorithms to identify similar traffic accident samples, this application provides a method for identifying similar traffic accident samples.

[0006] This application provides a method for identifying similar traffic accident samples, comprising:

[0007] Step 1: Input a dataset of n accident sample points and perform data preprocessing on the accident sample points in the dataset;

[0008] Step 2: Calculate the similarity of all accident sample points in the preprocessed dataset and generate a similarity matrix S;

[0009] Step 3: Based on the feature vectors of the similarity matrix S, predict the optimal number of clusters k;

[0010] Step 4: Construct the degree matrix D corresponding to the similarity matrix S;

[0011] Step 5: Calculate and construct the regularized Laplacian matrix based on the similarity matrix S and the degree matrix D. Solve for the Laplace matrix Construct the eigenvectors corresponding to the first k largest eigenvalues ​​in the set. Given a feature matrix T, each row of the feature matrix T is considered as a point in a k-dimensional space, resulting in a total of n points. A clustering algorithm is used to cluster these n points to obtain the clustering results, thus completing the identification of similar traffic accident samples.

[0012] Preferably, step 2 includes:

[0013] Step 21: Set the initial values ​​for the number of clusters k and the number of iterations t, where the initial value of k is 2 and the initial value of t is 1; perform eigenvalue decomposition on the similarity matrix S to obtain n eigenvalues ​​and eigenvectors of the n eigenvalues.

[0014] Step 22: Select the eigenvectors corresponding to the first k largest eigenvalues ​​to form an eigenvector matrix. Step 23, obtain , Represents the rotation matrix;

[0015] Step 24: Calculate the cost function :

[0016]

[0017] in, yes The largest element in the i-th row. express The element in the i-th row and j-th column;

[0018] Step 25, if If the iteration stops, then the current k is the predicted optimal number of clusters; otherwise, proceed to step 26.

[0019] Step 26: Adjust the rotation matrix using gradient descent. Select the eigenvectors corresponding to the first k+1 largest eigenvalues. ,renew , , Proceed to step 23;

[0020] in, Indicates the learning rate. express The gradient.

[0021] Preferably, in the iterative process, step 5 alternates between different clustering algorithms to cluster the n points, obtaining clustering results. After each clustering, the silhouette coefficient of the clustering results is recalculated. :

[0022]

[0023] in, This represents the silhouette coefficient of sample i. Let i be the average distance between sample point i and other samples in its cluster. Let i be the average distance between sample point i and all samples in the cluster that is closest to it in all other clusters;

[0024] If the new silhouette coefficient is higher than that in the previous iteration, then update the cluster labels and the current silhouette coefficient. Otherwise, keep the original cluster labels and silhouette coefficients unchanged and continue to the next iteration, until the maximum number of iterations is reached.

[0025] The beneficial effects of this application are that it determines the number of clusters through the feature vector structure. The interference of noise on the feature vector is much smaller than that on the feature value. Combined with gradient descent optimization, the bias caused by noise can be further corrected. Finally, the accuracy of cluster number prediction is significantly higher than that of traditional methods. This application also proposes improvements to the classical spectral clustering method to address its shortcomings. It constructs a traffic accident sample set partitioning method based on an improved nearest neighbor density adjustment adaptive spectral clustering algorithm, and uses multiple clustering methods to alternately optimize the clustering results to improve its clustering effect. Attached Figure Description

[0026] Figure 1 The flowchart of the clustering method in this application;

[0027] Figure 2 This shows the iterative changes in the contour coefficients of small vehicle accidents.

[0028] Figure 3 This shows the iterative changes in the profile coefficients of large vehicle accidents. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0030] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0031] The present application will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the application.

[0032] The method for identifying similar traffic accident samples in this application includes:

[0033] Step 1: Input a dataset P containing n accident sample points, and perform data preprocessing on the accident sample points in dataset P:

[0034] Step 2: Traverse all accident sample points in the preprocessed dataset P, calculate the similarity between accident sample points, and generate a similarity matrix S:

[0035] Step 3: Based on the feature vectors of the similarity matrix S, predict the optimal number of clusters k;

[0036] Step 4: Construct the degree matrix D corresponding to the similarity matrix S;

[0037] Step 5: Calculate and construct the regularized Laplacian matrix based on the similarity matrix S and the degree matrix D. Solve for the Laplace matrix Construct the eigenvectors corresponding to the first k largest eigenvalues ​​in the set. Given a feature matrix T, each row of the feature matrix T is considered as a point in a k-dimensional space, resulting in a total of n points. A clustering algorithm is used to cluster these n points to obtain the clustering results, thus completing the identification of similar traffic accident samples.

[0038] For step 1, before clustering begins, the data needs to be dedimensionalized. The purpose is to eliminate the perturbations to the model caused by different magnitudes of risk factors by transforming the data to a common scale, thereby improving the model's performance and the accuracy of distance measurement. Common dedimensionalization methods include: min-max normalization, Z-score standardization, maximum absolute value scaling, and mean normalization. This application uses min-max normalization, whose formula is shown in equation (1). Through standardization, all risk factors are scaled proportionally to a specific range, which is usually from 0 to 1.

[0039] (1)

[0040] Regarding step 2, in traditional spectral clustering algorithms, the parameters of the Gaussian kernel function are highly sensitive to clustering performance; improper parameter selection will severely affect the accuracy of the clustering results. Especially when dealing with unevenly distributed data density, relying solely on Euclidean distance to assess the similarity between samples often ignores the potential impact of local neighborhood density on clustering performance. To alleviate this problem, Zelnik-Manor et al. proposed the Self-tuning algorithm, defining the Gaussian kernel parameter as the data point p... i The method uses the Euclidean distance to its Kth nearest neighbor, thus avoiding the drawbacks of manually setting parameters and introducing some local density information. However, the single nearest neighbor selected in this method is highly susceptible to noise disturbances, affecting parameter stability.

[0041] This application proposes an improved strategy based on the Self-tuning method: by calculating data point p i The Gaussian kernel parameters are determined by the average Euclidean distance of its K nearest neighbors. This method integrates distance information from multiple neighboring points, which can more effectively mitigate the impact of noise, and improves the algorithm's adaptability by flexibly setting the W value. The similarity of the accident sample points is:

[0042] (2)

[0043] (3)

[0044] (4)

[0045] in, Indicates accident sample points and accident sample points The Euclidean distance between them Indicates accident sample points The number of nearest neighbors, the Gaussian kernel parameter is based on the number of nearest neighbors of a data point. The distance is calculated using the Euclidean distance between the nearest neighbors, which to some extent avoids the complexity and subjectivity caused by manual settings.

[0046] Indicates accident sample points The average Euclidean distance to its W-th nearest neighbor. It can be used as a measure of the density of local neighborhood points, therefore That is, the accident sample point and accident sample points Differences in neighborhood density;

[0047] This represents the maximum density difference among all samples. The Gaussian kernel function defined by this strategy is based on the density of the neighboring regions of the data points. Adjusting the similarity and density difference between data points pass After normalization, it serves as a moderating factor, achieving a negative correlation mapping between density difference and similarity: the greater the density difference, the higher the weight of the moderating factor, and the lower the similarity. The similarity decreases accordingly, and vice versa. Regarding step 3, determining the number of clusters in the dataset is a crucial issue in spectral clustering analysis. Existing manual annotation methods are not only time-consuming but may also lead to inconsistencies in results due to subjective judgment. This section proposes an automated method based on feature vector structure for predicting the number of clusters, aiming to improve the efficiency and accuracy of the clustering process.

[0048] In existing spectral clustering, eigenvalue analysis of the similarity matrix S is used to estimate the number of clusters C. However, this method encounters problems when clustering under noisy conditions. Assume the largest eigenvalue of S is... The multiplicity is C. However, under the influence of noise, the eigenvalue will deviate from 1.

[0049] Here we consider the eigenvectors of the similarity matrix S. , corresponding to each cluster c. Ideally, S is a block diagonal matrix, whose eigenvectors can be represented as... ,in This is the feature vector corresponding to cluster c, with the remaining positions set to 0.

[0050] Define a cost function to measure the alignment between the feature vector set and the normalized coordinate system:

[0051] Using gradient descent to minimize the cost function and find the optimal rotation matrix, in a preferred embodiment, step 3 specifically includes:

[0052] Step 31: Set the initial values ​​for the number of clusters k and the number of iterations t, where the initial value of k is 2 and the initial value of t is 1; perform eigenvalue decomposition on the similarity matrix S to obtain n eigenvalues ​​and eigenvectors of the n eigenvalues.

[0053] Step 32: Select the eigenvectors corresponding to the first k largest eigenvalues ​​to form an eigenvector matrix. ;

[0054] Initial alignment: Start with the feature axes x1 and x2 corresponding to the first two largest eigenvalues ​​of the similarity matrix S;

[0055] Step 33, obtain , Represents the rotation matrix;

[0056] Initialize the rotation matrix and compute the initial cost function;

[0057] Step 34: Calculate the cost function :

[0058] (5)

[0059] in, yes The largest element in the i-th row. express The element in the i-th row and j-th column;

[0060] Step 35, if If the iteration stopping condition is met, then the current k is the predicted optimal number of clusters; otherwise, proceed to step 36; if The iteration stops when the acceptable minimum value or the preset number of iterations is reached.

[0061] Step 36: Adjust the rotation matrix using gradient descent. The gradual increase of eigenvectors: gradually increasing the next largest eigenvector. Add to the currently aligned feature vector set and update , , Proceed to step 33;

[0062] in, Indicates the learning rate. express The gradient.

[0063] Until all eigenvectors have been considered. The choice is such that... The smallest k is used as the final number of clusters.

[0064] Regarding step 5, the Laplace matrix of this application :

[0065] (6)

[0066] in, Represents the identity matrix.

[0067] Solving the Laplace matrix The first k largest eigenvalues ​​and their corresponding eigenvectors The feature vector s is normalized and constructed. 3D feature matrix Each row of matrix T is considered as a point in k-dimensional space, resulting in n points. A clustering algorithm is then used to cluster these points. The output should be the accident cluster classification results. The i-th row of the feature matrix T is divided into... When clustered, it represents the corresponding accident sample point p. i belong .

[0068] Furthermore, while this application improves the clustering effect, the size of its silhouette coefficient is still relatively limited, which may affect the subsequent model building. Therefore, based on the clustering method in this application, four clustering methods—GMM, K-means, hierarchical clustering, and MeanShift—are used to alternately optimize the results. Through this alternating optimization strategy, the clustering quality can be gradually improved based on the initial clustering results, leveraging the complementarity of different algorithms to ultimately obtain more reasonable and accurate clustering results.

[0069] After the initial clustering is performed, initial cluster labels and silhouette coefficients are obtained.

[0070] Set the maximum number of iterations; here it is set to 30.

[0071] In each iteration, different clustering algorithms are used alternately to optimize the current clustering result. Gaussian Mixture Model (GMM) is used for optimization in iterations 0, 4, 8, ...; K-means clustering is used for optimization in iterations 1, 5, 9, ...; Hierarchical Clustering is used for optimization in iterations 2, 6, 10, ...; and MeanShift is used for optimization in iterations 3, 7, 11, ...

[0072] After each clustering, the silhouette coefficient of the clustering results is recalculated. :

[0073] (7)

[0074] in, This represents the silhouette coefficient of sample i. is the average distance between sample point i and other samples in its cluster, which reflects the density of the sample in the cluster; The average distance between sample point i and all samples in the cluster that is closest to it in all other clusters reflects the degree of separation between the sample and its nearest outer cluster.

[0075] If the new silhouette coefficient is higher than that in the previous iteration, then update the cluster labels and the current silhouette coefficient. Otherwise, keep the original cluster labels and silhouette coefficients unchanged and continue to the next iteration, until the maximum number of iterations is reached.

[0076] Before clustering begins, the data needs to be dedimensionalized. This process aims to eliminate the perturbations to the model caused by risk factors of varying magnitudes by transforming the data to a common scale, thereby improving model performance and the accuracy of distance metrics. This paper uses min-maximum normalization, which, through standardization, scales all risk factors proportionally to a specific range, typically from 0 to 1.

[0077] The clustering method based on this application was programmed using a Python compiler. First, the number of clusters for 7133 small vehicle accidents and 1110 large vehicle accidents was predicted based on feature vectors. The results are shown in Table 1. The table shows that for small vehicle accidents, the cost function is minimized to 192.99 when there are 8 clusters. However, since too many clusters would make subsequent work cumbersome, the next lowest cost function was chosen, resulting in 3 clusters for small vehicle accidents. For large vehicle accidents, the iteration stopped when there were 3 clusters and the cost function value was 160.05, meaning that the cost function is minimized when there are 3 clusters. Therefore, the number of clusters for large vehicle accidents is also 3.

[0078] Table 1. Prediction results of cluster number

[0079] After selecting the number of clusters, the clustering method of this application was used to perform cluster analysis on samples of large vehicle accidents and small vehicle accidents. The number of samples in each cluster is shown in Table 2. To verify the effectiveness of the clustering method of this application, this study used the silhouette coefficient as a quantitative evaluation index to objectively evaluate the clustering results.

[0080] Profile coefficient The value of is in the range of [-1, 1]. The closer it is to 1, the better the sample point is clustered, and the distance between the sample point and the points inside the same cluster is much smaller than the distance to the nearest outer cluster. The closer it is to 0, the more likely the sample point is on the boundary between two clusters, and the clustering effect is not obvious.

[0081] Table 2 Sample size for each cluster

[0082]

[0083] The clustering method of this application is compared and analyzed with the classic spectral clustering model, and the results are shown in Table 3. As can be seen from the table, the clustering method of this application exhibits a higher silhouette coefficient value compared with the classic spectral clustering method, indicating that the clustering performance of this method is significantly better than that of traditional algorithms, and the risk factors within the clusters have higher homogeneity.

[0084] Table 3. Silhouette coefficients of various clustering algorithms

[0085]

[0086] As shown in Table 3, although the clustering method in this application improves the clustering effect, the size of its silhouette coefficient is still relatively limited, which may affect the subsequent model building. Therefore, based on the clustering method in this application, four clustering methods—GMM, K-means, hierarchical clustering, and MeanShift—are used to alternately optimize the results. Through this alternating optimization strategy, the clustering quality can be gradually improved based on the initial clustering results, leveraging the complementarity of different algorithms to ultimately obtain more reasonable and accurate clustering results.

[0087] Clustering results of accident samples of large and small vehicles are shown below Figure 2 and Figure 3 As the number of iterations increased, the silhouette coefficient gradually converged to 0.36 and 0.33. The clustering method of this application, which underwent alternating optimization, was compared and analyzed with the K-means clustering algorithm and the clustering method of this application. The results are shown in Table 4. The results show that alternating optimization of the clustering algorithm of this application improved the silhouette coefficient compared to the previous method, indicating that alternating optimization helps to improve the classification performance of the clustering model.

[0088] Table 4. Silhouette coefficients of various clustering algorithms

[0089]

[0090] Using the clustering results after alternating optimization as the final result, the risk factors with a high frequency proportion in each cluster were extracted as cluster features. The cluster feature descriptions are shown in Table 5. As can be seen from the table, the main characteristics of small vehicle accidents are time features (week, month) and weather features; while the main characteristics of large vehicle accidents are weather features, time features (month), and whether the driver is familiar with the road.

[0091] Table 5. Description of Cluster Feature Variables

[0092]

[0093] The optimization strategy in this application significantly improves the clustering accuracy of traffic accident samples. It automatically determines the optimal number of clusters by combining feature vector incremental search, eliminating bias from human intervention. It introduces a local density difference factor to dynamically adjust similarity calculation, solving the misclustering problem in density-imbalanced datasets. It achieves Gaussian kernel parameter adaptation by dynamically setting the Gaussian kernel parameter through the average distance of nearest neighbors. It constructs an alternating optimization framework, using GMM, K-means, hierarchical clustering, and MeanShift to iteratively optimize the initial clustering results. Finally, it accurately extracts high-risk scenario features such as "weekdays / bad weather / July-September" (1665 accidents) for small vehicles and "sunny days / January-March / drivers unfamiliar with roads" (311 accidents) for large vehicles, providing theoretical support for subsequent traffic safety accident analysis and traffic planning optimization.

[0094] While this application has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of this application. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of this application as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A method for identifying similar traffic accident samples, characterized in that, include: Step 1: Input a dataset of n accident sample points and perform data preprocessing on the accident sample points in the dataset; Step 2: Calculate the similarity of all accident sample points in the preprocessed dataset and generate a similarity matrix S; Step 3: Based on the feature vectors of the similarity matrix S, predict the optimal number of clusters k; Step 4: Construct the degree matrix D corresponding to the similarity matrix S; Step 5: Calculate and construct the regularized Laplacian matrix based on the similarity matrix S and the degree matrix D. Solve for the Laplace matrix Construct the eigenvectors corresponding to the first k largest eigenvalues ​​in the set. The feature matrix T is used to treat each row of the feature matrix T as a point in a k-dimensional space, with a total of n points. A clustering algorithm is used to cluster the n points to obtain the clustering results, thus completing the identification of similar traffic accident samples. During the iteration process, step 5 alternates between different clustering algorithms to cluster the n points, obtaining clustering results. After each clustering, the silhouette coefficient of the clustering results is recalculated. : in, This represents the silhouette coefficient of sample i. Let i be the average distance between sample point i and other samples in its cluster. Let i be the average distance between sample point i and all samples in the cluster that is closest to it in all other clusters; If the new silhouette coefficient is higher than the silhouette coefficient in the previous iteration, then update the cluster label and the current silhouette coefficient; otherwise, keep the original cluster label and silhouette coefficient unchanged and continue to the next iteration until the maximum number of iterations is reached. Different clustering algorithms include GMM clustering algorithm, K-means clustering algorithm, hierarchical clustering algorithm and MeanShift clustering algorithm; The similarity of the accident sample points is: in, Indicates accident sample points and accident sample points The Euclidean distance between them Indicates accident sample points The number of nearest neighbors, Indicates accident sample points The average of the Euclidean distances to its W nearest neighbors. Indicates accident sample points The average of the Euclidean distances to its W nearest neighbors. This represents the maximum value of the density difference among all samples; ； 。 2. The method for identifying similar traffic accident samples according to claim 1, characterized in that, Step 2 includes: Step 21: Set the initial values ​​for the number of clusters k and the number of iterations t, where the initial value of k is 2 and the initial value of t is 1; perform eigenvalue decomposition on the similarity matrix S to obtain n eigenvalues ​​and eigenvectors of the n eigenvalues. Step 22: Select the eigenvectors corresponding to the first k largest eigenvalues ​​to form an eigenvector matrix. Step 23, obtain , Represents the rotation matrix; Step 24: Calculate the cost function : in, yes The largest element in the i-th row. express The element in the i-th row and j-th column; Step 25, if If the iteration stopping condition is met, then the current k is the predicted optimal number of clusters; otherwise, proceed to step 26. Step 26: Adjust the rotation matrix using gradient descent. The adjusted rotation matrix is ​​shown below. Select the eigenvectors corresponding to the first k+1 largest eigenvalues. ,renew , , Proceed to step 23; in, Indicates the learning rate. express The gradient.

3. The method for identifying similar traffic accident samples according to claim 1, characterized in that, Laplace matrix : in, Represents the identity matrix.

4. The method for identifying similar traffic accident samples according to claim 1, characterized in that, In step 1, a dataset of n accident sample points is input, and the accident sample points in the dataset are subjected to min-max normalization to complete the preprocessing.

5. A computer-readable storage device storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the similar traffic accident sample identification method as described in any one of claims 1 to 4.

6. A device for identifying similar traffic accident samples, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the similar traffic accident sample identification method as described in any one of claims 1 to 4.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the similar traffic accident sample identification method as described in any one of claims 1 to 4.

Images