Traffic accident scene clustering method based on K-means algorithm

Abstract

The invention relates to the technical field of vehicle safety, in particular to a traffic accident scene clustering method based on a K-means algorithm, and the method comprises the following steps: screening accident data fields from an accident database, and carrying out the data cleaning and recoding; performing importance ranking on the data variables based on a Bayesian optimization random forest model, and selecting the data variables with high importance; performing correlation analysis on the selected data variables through Cramer's V test, and screening to obtain characteristic variables; according to the feature variables obtained by screening, jointly determining the optimal clustering number of the K-means algorithm by adopting a contour coefficient and cluster minimum sample number method; determining an initial clustering center according to the determined optimal clustering value; k-means clustering is carried out, and the boundary condition of each scene is determined. According to the traffic accident scene clustering method based on the K-means algorithm provided by the invention, the efficiency of determining the boundary of the test scene in a large amount of complex data can be improved by adaptively selecting the characteristic parameters.

Description

Technical Field

[0001] The invention relates to the technical field of traffic safety, and in particular to a traffic accident scene clustering method based on a K-means algorithm. Background Art

[0002] With the acceleration of urbanization and the sharp increase in the number of cars, the incidence of traffic accidents is also rising. Traditional vehicle safety performance testing methods mainly rely on preset standard scenarios, which often cannot cover all possible types of traffic accidents and complex traffic environments. Existing test scenarios are too simple to fully reflect the diversity and complexity of real life, resulting in a large gap between the performance of vehicles in real accident scenarios and test results.

[0003] At present, there are two main methods for determining vehicle active safety test scenarios. One is the traditional statistical method, which uses probability distribution analysis, extreme value analysis, mode analysis and other methods combined with the knowledge of domain experts to determine the scene boundary conditions. Although this method can reflect the characteristics of real traffic accidents to a certain extent, it has problems such as huge workload and reliance on expert experience, and it is difficult to fully cover real accident data. Another method is to use machine learning technology to determine the boundary conditions of the test scenario through methods such as feature selection algorithms, logistic regression, decision trees and support vector machines. Machine learning methods have advantages in processing large-scale data and automatically identifying key features, but the interpretability of their models is poor, and the results are difficult to intuitively understand or explain, which may introduce uncertainty and confusion when determining the test scenario. Therefore, it is urgent to propose a method for efficiently determining the boundaries of test scenarios in large amounts of complex data. Summary of the invention

[0004] The purpose of the present invention is to provide a traffic accident scene clustering method based on the K-means algorithm, which can improve the efficiency of determining the boundaries of test scenes.

[0005] The basic scheme provided by the present invention is a traffic accident scene clustering method based on the K-means algorithm, comprising the following steps: screening accident data fields from an accident database, and performing data cleaning and recoding; ranking data variables by importance based on a random forest model optimized by Bayes, and selecting data variables with high importance; performing correlation analysis on the selected data variables by Cramer's V test, and screening characteristic variables; based on the characteristic variables obtained by screening, jointly determining the optimal cluster number of the K-means algorithm by using the method of silhouette coefficient and minimum cluster sample number; determining the initial cluster center according to the determined optimal cluster value; and performing K-means clustering to determine the boundary conditions of each scene.

[0006] Furthermore, the data recoding step includes one-hot encoding of categorical variables and normalization of continuous variables.

[0007] Furthermore, it is characterized in that the parameters of the random forest model based on Bayesian optimization include the number of base learners, whether to perform sampling with replacement, out-of-bag data evaluation and the minimum number of samples that can be divided into nodes.

[0008] Furthermore, it is characterized in that the Cramer's V test calculates the correlation coefficient between variables. When the correlation coefficient V>0.4, the data variables with strong correlation and low importance are eliminated. The Cramer's V coefficient calculation formula is as follows:

[0009] Among them, χ 2 is the chi-square statistic, N is the sample size, c is the number of columns, and r is the number of rows.

[0010] Furthermore, the silhouette coefficient is used to evaluate the clustering effect, and the silhouette coefficient calculation formula is as follows:

[0011]

[0012] Among them, a(i) is the average distance between sample i and other samples in the same cluster, and b(i) is the minimum value of the average distance between sample i and all samples in other clusters.

[0013] Furthermore, according to the determined number of clusters, the initial cluster center is determined by hierarchical clustering, and the calculation formula is as follows:

[0014]

[0015] Among them, C i and C j are the two clusters to be compared, n i and n j are their sample numbers respectively, and d(x, y) is the distance between sample x and sample y.

[0016] Furthermore, the K-means clustering formula is as follows:

[0017] C i =argmin k ||x (i) -d k || 2

[0018] Among them, x (i) For each data point and all cluster centers d k , and thus assign it to the cluster corresponding to the nearest cluster center according to the distance.

[0019] Furthermore, by selecting the mode as the boundary of each class, if the conditions do not match the real accident data, the submode is used to determine the boundary conditions.

[0020] The principles and beneficial effects of the present invention are:

[0021] 1. Improve the comprehensiveness and accuracy of test scenarios: Through the clustering method based on the K-means algorithm, key data variables are adaptively identified and selected to construct a variety of traffic accident test scenarios, making the test scenarios closer to real accident situations, thereby improving the comprehensiveness and accuracy of vehicle active safety performance testing.

[0022] 2. Save computing resources and improve operation efficiency: By adaptively screening feature variables based on the random forest model and correlation analysis based on Bayesian optimization, unnecessary over-segmentation is effectively avoided, computing resources are saved, and the operation efficiency of the algorithm is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of a traffic accident scene clustering method based on K-means algorithm;

[0024] Figure 2 This is a random forest model diagram for a traffic accident scene clustering method based on the K-means algorithm. DETAILED DESCRIPTION

[0025] The following is further described in detail through specific implementation methods:

[0026] Embodiment 1

[0027] like Figure 1 A traffic accident scene clustering method based on the K-means algorithm is shown, comprising the following steps: selecting accident data fields from an accident database, and performing data cleaning and recoding; sorting the data variables by importance based on a random forest model optimized by Bayesian, and selecting data variables with high importance; performing correlation analysis on the selected data variables through a Cramer's V test, and screening characteristic variables; determining the optimal number of clusters of the K-means algorithm based on the screened characteristic variables using the silhouette coefficient and the minimum number of cluster samples; and performing K-means clustering to determine the boundary conditions of each scene.

[0028] According to the requirements, N accident fields of accident data are screened out from the accident database. In this embodiment, N accident fields containing different information are extracted from a traffic accident database, including the time, location, weather conditions, vehicle type, driver age, etc. of the accident.

[0029] Clean the accident data that has been screened out. The data cleaning process includes handling missing values and outliers in the data. For missing values, interpolation or deletion methods can be used, and for outliers, they are identified and removed through statistical analysis. In addition, redundant information in the data needs to be removed to ensure that each data record is independent and valid. At the same time, handle the inconsistencies and incomplete information in the data to ensure the integrity and consistency of the data.

[0030] Recode the categorical variables and continuous variables in the data. For categorical variables (such as weather conditions, road types, etc.), the one-hot encoding method is adopted to convert each categorical variable into binary encoding. Specifically, if the value range of a certain categorical variable is {1, 2,..., n}, it is encoded into n binary variables, each variable corresponding to a category, marked as 1 at the i-th position, and marked as 0 at the remaining positions. For continuous variables (such as accident time, number of injured people, etc.), first segment them into categorical variables and then perform one-hot encoding, or directly perform normalization processing to standardize the data for subsequent clustering analysis.

[0031] Use the random forest model based on Bayesian optimization to rank the importance of N variables in the data, and select T data variables with high importance. Among them, the specific principle of the random forest model based on Bayesian optimization is as follows: According to the clustering purpose, select a target data variable from the recoded data, and all the remaining data variables will be used to rank the importance of the target data variable through the random forest model based on Bayesian optimization. The data variables ranked靠前 are used as alternative clustering data variables, and the data variables ranked靠后 cannot be used as variables for clustering analysis. The processing method is to remove them as supplementary information for clustering analysis. The parameters of Bayesian optimization include the number of base learners, whether to perform sampling with replacement, whether to evaluate the model using out-of-bag data, the minimum number of samples that can be divided at a node, the minimum number of samples contained in a leaf node, the number of features considered when splitting a node, and the maximum depth of CART. The random forest model is as Figure 2 shown.

[0032] Use Cramer’s V test to perform correlation analysis on the obtained T data variables. When V > 0.4, remove the data variables with lower importance, so as to adaptively obtain P feature variables (P < T) to make the clustering effect more prominent. The calculation formula of the Cramer’s V coefficient is as follows:

[0033] Among them, χ 2 is the chi-square statistic, N is the sample size, c is the number of columns, and r is the number of rows.

[0034] If the coefficient V>0.4, it indicates that the two data variables have a strong correlation, and the data variable with higher importance is automatically selected as the feature variable. Finally, if the number of feature variables is small and the clustering effect is affected, the data variables can be replaced according to the importance ranking as described above.

[0035] According to the obtained characteristic variable P, the silhouette coefficient and the minimum number of cluster samples are used to jointly determine the optimal clustering K value.

[0036] First, calculate the sample single silhouette coefficient s( i ), the specific calculation formula is:

[0037]

[0038] Among them, a(i) is the average distance between sample i and other samples in the same cluster, and b(i) is the minimum value of the average distance between sample i and all samples in other clusters.

[0039] Secondly, the average silhouette coefficient SC is calculated according to S(i). The specific calculation formula is:

[0040]

[0041] Where n represents the number of samples in each category.

[0042] In this embodiment, the minimum number of cluster samples is set to a threshold value M according to the sample size, where M = 2% N. When the silhouette coefficient SC reaches the maximum value SCmax, if the minimum number of cluster samples Q is very small, this class is not representative. Therefore, when taking the maximum silhouette coefficient SCmax, if Q>M is established, the clustering K value is determined. Otherwise, the current maximum average silhouette coefficient is removed, and the above steps are repeated until Q>M to determine the K value.

[0043] Perform hierarchical clustering based on the determined K value to determine the initial cluster center D, where D = d1, d2, ..., d k , the calculation formula is as follows:

[0044]

[0045] Among them, C i and C j are the two clusters to be compared, n i and n j are their sample numbers respectively, and d(x, y) is the distance between sample x and sample y.

[0046] K-means clustering is performed based on the determined characteristic variable P, the determined K value, and the determined initial cluster center D to divide the existing data N into L categories.

[0047] The K-means clustering formula is as follows:

[0048] C i =argmin k ||x (i) -d k || 2

[0049] Among them, x (i) For each data point and all cluster centers d k The distance between the data points is calculated, and the data points are assigned to the cluster corresponding to the nearest cluster center according to the distance. For each cluster, the mean of all data points assigned to the cluster is used as the new cluster center. Repeat the data point assignment and cluster center update until the cluster center no longer changes.

[0050] This embodiment also includes adjusting the boundaries of L categories of scenes and performing weighted proportion statistics on the boundaries of the L scenes.

[0051] After the cluster analysis is completed, data mining is performed on each category, and the mode is selected as the boundary of each category. Finally, it is checked whether the selected test scenario boundary is consistent with the real accident data. If some conditions do not conform to the real accident, the sub-mode is used to determine the boundary conditions.

[0052] Finally, by performing weighted statistics on the L scene boundaries, the coverage of the accident data is calculated to ensure that the test scenario can fully cover the actual traffic accident data.

[0053] The above are only embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the scope of protection of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A traffic accident scene clustering method based on K-means algorithm, characterized in that: The following steps are involved: The accident data fields were screened from the accident database, and the data were cleaned and recoded. The importance of multiple variables of the data was ranked based on the random forest model optimized by Bayesian, and data variables with high importance were selected. The correlation analysis of the selected data variables was performed through Cramer's V test to screen out characteristic variables. Based on the screened characteristic variables, the silhouette coefficient and the minimum number of cluster samples were used to jointly determine the optimal number of clusters of the K-means algorithm. The initial cluster center was determined based on the determined optimal cluster value. K-means clustering was performed to determine the boundary conditions of each scene.

2. The traffic accident scene clustering method based on K-means algorithm according to claim 1, characterized in that: The data recoding step includes one-hot encoding of categorical variables and normalization of continuous variables.

3. The traffic accident scene clustering method based on K-means algorithm according to claim 2 is characterized in that: The parameters of the random forest model based on Bayesian optimization include the number of base learners, whether to perform sampling with replacement, out-of-bag data evaluation, and the minimum number of samples that a node can be divided into.

4. The traffic accident scene clustering method based on K-means algorithm according to claim 3 is characterized in that: In the calculation of the correlation coefficient between variables by Cramer's V test, when the correlation coefficient V>0.4, the data variables with strong correlation and low importance are eliminated, wherein the coefficient calculation formula of Cramer's V is as follows: Among them, 2 is the chi-square statistic, N is the sample size, c is the number of columns, and r is the number of rows.

5. The traffic accident scene clustering method based on K-means algorithm according to claim 4 is characterized in that: The calculation formula of the silhouette coefficient is as follows: Among them, a(i) is the average distance between sample i and other samples in the same cluster, and b(i) is the minimum value of the average distance between sample i and all samples in other clusters.

6. The traffic accident scene clustering method based on K-means algorithm according to claim 1, characterized in that: The initial cluster center is determined by performing hierarchical clustering according to the determined optimal cluster value, and the calculation formula is as follows: Among them, C i and C j are the two clusters to be compared, n i and n j are their sample numbers respectively, and d(x,y) is the distance between sample x and sample y.

7. The traffic accident scene clustering method based on K-means algorithm according to claim 1, characterized in that: The K-means clustering formula is as follows: C i =argmin k ||x (i) -d k || 2 Among them, x (i) For each data point and all cluster centers d k The distance is assigned to the cluster corresponding to the nearest cluster center according to the distance.

8. The traffic accident scene clustering method based on K-means algorithm according to claim 1, characterized in that: It also includes that after the cluster analysis is completed, the mode is selected as the boundary of each cluster. If the conditions do not meet the real accident data, the sub-mode is used to determine the boundary conditions.

Images