Railway accident type prediction method based on KNN and AdaBoost
By filling missing data with the KNN algorithm and combining it with the AdaBoost ensemble learning method, the problems of missing data and class imbalance in railway accident type prediction are solved, and the accuracy and performance of the prediction are improved.
Patent Information
- Application Number
- CN202111550459.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-17
- Publication Date
- 2025-09-16
- Estimated Expiration
- 2041-12-17
AI Technical Summary
The accuracy of railway accident type prediction in existing technologies is not high, mainly because the problems of missing data and class imbalance have not been effectively solved.
The KNN algorithm is used to fill in the missing data, and combined with the AdaBoost ensemble learning method, multiple decision tree weak classifiers are used for training to establish a railway accident type classifier. The prediction performance is improved through data preprocessing and algorithm optimization.
The accuracy, precision, recall rate and F1-score performance of railway accident type prediction have been improved, effectively solving the problems of missing data and class imbalance, and improving the accuracy of prediction.
Smart Images
Figure CN114444765B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of machine learning, and in particular relates to a railway accident type prediction method based on KNN (K-Nearest Neighbor) and AdaBoost. Background Art
[0002] The rapid development of the railway industry plays a vital role in driving national economic development. At the same time, railway safety issues are receiving significant attention. Railway accident type prediction utilizes historical accident records to estimate and predict future accidents, extracting useful information from historical data and establishing a comprehensive accident prediction and early warning mechanism. This is crucial for promoting the informatization of the railway industry, improving transportation efficiency, and preventing safety hazards.
[0003] However, historical railway accident data often contains missing data. When mining historical data and building predictive models, it cannot be directly used; instead, appropriate data infill techniques are required. Railway accident type prediction is inherently a multi-classification problem, and historical railway data exhibits a significant imbalance in accident classes, with derailments and collisions dominating, while explosions, fires, and other accident types are relatively rare. Failure to address this class imbalance in the raw data will result in inaccurate accident type predictions. Summary of the Invention
[0004] The purpose of the present invention is to provide a railway accident type prediction method based on KNN and AdaBoost to solve the problem of low accident type prediction accuracy in the prior art.
[0005] The technical solution adopted in the present invention is:
[0006] The railway accident type prediction method based on KNN and AdaBoost includes the following steps:
[0007] Step 1: perform sparsity analysis on the original railway accident history dataset and delete some redundant attributes according to the sparsity threshold;
[0008] Step 2: hard-code the character attributes in the railway accident history dataset processed in Step 1 to convert the character attributes into numeric attributes;
[0009] Step 3: Perform data filling based on the KNN algorithm on the railway accident history dataset processed in step 2 to fill in the missing values in the original data;
[0010] Step 4: normalize the railway accident history dataset processed in step 3 and randomly divide it into training and test sets according to a certain ratio;
[0011] Step 5: Randomly divide the railway accident history dataset normalized in step 4 into a training set and a test set. Use the AdaBoost algorithm to train the classifier on the training set data and verify the classifier performance on the test set.
[0012] The present invention is also characterized in that:
[0013] In step 3, the KNN algorithm is used to fill in the missing values of the railway accident data. By calculating the distance between the missing sample and other complete samples, the first k closest complete samples are taken and the weighted average is used as the value after the missing sample is filled.
[0014] The specific steps of step 3 are:
[0015] Step 3.1: For each accident record R containing missing values i , calculate R i The distance d between samples of all complete accident records is sorted using the KNN algorithm, and the first k sample records with the closest distance are retained as the basis for filling the current missing sample. A complete accident record represents a record with no missing values, and the distance d between samples is calculated as shown in formula (1).
[0016]
[0017] where R i and R i They represent the i-th record and the j-th record respectively. In this formula, the record on the left is a record with missing values by default, and the record on the right is a complete record. i To the complete record R j The distance d(i, j) between samples is shown in formula (1), which represents the similarity of two records in the sample space. The smaller the distance, the closer the two are.
[0018] In step 3.2, the missing records are filled in based on the first k accident records closest to the record with missing values in the sample space. The formula for calculating the current missing record from k accident records is shown in formula (2).
[0019]
[0020] where R i is the i-th accident record with missing values, R j is the jth distance from R in the sample space i The most recent record, as shown in formula (2), is calculated as above to obtain the R after missing value filling. i .
[0021] In step 5, the AdaBoost ensemble learning method is used to establish a railway accident type classifier, including using M identical decision tree weak classifiers as weak learners and obtaining a final software defect strong classifier through weighted averaging;
[0022] The training process of step 5 is as follows:
[0023] In step 5.1, the resampled training set in step 4 is divided into 10 parts through a 10-fold cross-validation. The entire training data set is divided into 10 parts, each of which will be used as a validation set. During training, 9 parts of the data are used for training, and the remaining part is used as a validation set to test the classifier performance. This process is repeated 10 times, and the weighted average is used as the final performance indicator of the classifier. At the end of the training process, the trained railway accident type classifier C is obtained. The process of using the AdaBoost learning method to train the final classifier is shown in formulas (4), (5), and (6):
[0024]
[0025]
[0026]
[0027] Among them, α m is the weight of the mth decision tree weak classifier, C m (x) is the mth decision tree weak classifier, C(x) is the strong classifier obtained by ensemble learning, and the sign function is used to take the positive and negative weighted results of the m weak classifiers, e m The error rate of the mth decision tree weak classifier.
[0028] In step 5.2, use the strong classifier C trained in step 5.1 and perform a performance test on the test set using the precision, accuracy, recall, and F1-score performance indicators.
[0029] The beneficial effects of the present invention are:
[0030] 1) The method of the present invention proposes a railway accident type prediction method based on KNN and AdaBoost, which uses the KNN algorithm to fill in missing data, solving the problem of missing original data. At the same time, it takes advantage of the ensemble learning AdaBoost method and uses multiple weak learners for learning to improve the final railway accident type prediction performance, thereby improving the final accident type prediction accuracy, precision, recall rate and F1-score performance.
[0031] 2) The method of the present invention starts with the original railway accident history data. At the data level, redundant attributes are deleted, data is padded, character data is encoded, and normalization operations are performed, effectively preprocessing the original data. At the algorithm level, the ensemble learning method AdaBoost is used, and multiple weak learners are used for learning, effectively solving the problem of poor classification performance caused by class imbalance in the original data. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 This is an overall flow chart of the railway accident type prediction method based on KNN and AdaBoost of the present invention;
[0033] Figure 2 A pie chart of accident types of a railway accident history dataset used in the railway accident type prediction method based on KNN and AdaBoost of the present invention;
[0034] Figure 3 This is the confusion matrix of the accident type classification results of the railway accident type prediction method based on KNN and AdaBoost on the test set of the present invention. DETAILED DESCRIPTION
[0035] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0036] The present invention is based on the railway accident type prediction method of KNN and AdaBoost, such as Figures 1 to 3 , including the following steps:
[0037] Step 1: perform sparsity analysis on the original railway accident history dataset and delete some redundant attributes according to the set sparsity threshold;
[0038] Step 2: hard-code the character attributes in the railway accident history dataset processed in Step 1 to convert the character attributes into numeric attributes;
[0039] Step 3: Perform data filling based on the KNN algorithm on the railway accident history dataset processed in step 2 to fill in the missing values in the original data;
[0040] Step 4: normalize the railway accident history dataset processed in step 3 and randomly divide the normalized data into a training set and a test set;
[0041] In step 5, the classifier is trained on the training set data using the AdaBoost algorithm, and the classifier performance is verified on the test set.
[0042] In step 1, the railway accident historical dataset used comes from the railway equipment accident data published by the Federal Railroad Administration of the United States, specifically historical data from 2016 to 2018. This dataset includes accident type, specific location and time of the accident, railway number, vehicle information, personnel information, etc., and contains more than 7,000 accident records. The accident types are divided into five major categories, namely derailment, collision, intersection, obstacle, and fire, and 11 minor categories. Detailed descriptions of the accident types can be found in [1]. Figure 2 .
[0043] Because the original data contains a large amount of missing data and redundant attributes, in step 1, the redundant attributes in the original data are first deleted. Based on the data sparsity of the original data attributes, a sparsity threshold is set, and all attributes with sparsity greater than the threshold are deleted, resulting in a railway accident dataset with redundant attributes removed.
[0044] In step 2, since there are a large number of descriptive attributes in the historical data of railway accidents, they cannot be directly passed to the model for learning. They need to be encoded accordingly and converted into numerical data. In this invention, a hard-coding method is used to convert unique and limited character attributes such as region and railway number into unique corresponding numerical values, and the text description attributes of the accident are directly deleted.
[0045] In step 3, there is a serious data missing phenomenon in the original railway accident data, and a large number of accident record attributes have missing values. The present invention adopts a data completion algorithm based on the KNN algorithm. The specific steps are as follows:
[0046] Step 3.1: For each accident record R containing missing values i , calculate R i The distance d between samples of all complete accident records is sorted using the KNN algorithm, and the first k sample records with the closest distance are retained as the basis for filling the current missing sample. A complete accident record represents a record with no missing values, and the distance d between samples is calculated as shown in formula (1).
[0047]
[0048] where R i and R i They represent the i-th record and the j-th record respectively. In this formula, the record on the left is a record with missing values by default, and the record on the right is a complete record. i To the complete record R j The distance d(i, j) between samples is shown in formula (1), which represents the similarity of two records in the sample space. The smaller the distance, the closer the two are.
[0049] In step 3.2, the missing records are filled in based on the first k accident records closest to the record with missing values in the sample space. The formula for calculating the current missing record from k accident records is shown in formula (2).
[0050]
[0051] where R i is the i-th accident record with missing values, R j is the jth distance from R in the sample space i The most recent record, as shown in formula (2), is calculated as above to obtain the R after missing value filling. i .
[0052] Step 4: Normalize the railway accident dataset processed in Step 3 to facilitate model training and convergence. The normalized data is then randomly divided into a training set and a test set according to a certain ratio. The data normalization formula is shown in Equation (3).
[0053]
[0054] where R a represents the railway accident data set after processing in steps 1 to 3, as described in formula (3), mean(R a ) and std(R a ) represent the calculation of R in column-major order. a Therefore, after the operation of formula (3), the railway accident data set normalized by column is obtained. The distribution of the data conforms to the normal distribution with a mean of 0 and a standard deviation of 1.
[0055] In step 5, the training set and test set randomly divided in step 4 are used to build and test the model.
[0056] The specific operations of step 5 are as follows:
[0057] A railway accident type classifier was established using the AdaBoost ensemble learning method. M identical decision tree weak classifiers were used as weak learners, and the final software defect strong classifier was obtained through weighted averaging.
[0058] The specific training process is:
[0059] In step 5.1, the resampled training set in step 4 is divided into 10 parts through a 10-fold cross-validation. The entire training data set is divided into 10 parts, each of which will be used as a validation set. During training, 9 parts of the data are used for training, and the remaining part is used as a validation set to test the classifier performance. This process is repeated 10 times, and the weighted average is used as the final performance indicator of the classifier. At the end of the training process, the trained railway accident type classifier C is obtained. The process of using the AdaBoost learning method to train the final classifier is shown in formulas (4), (5), and (6):
[0060]
[0061]
[0062]
[0063] Among them, α m is the weight of the mth decision tree weak classifier, C m (x) is the mth decision tree weak classifier, C(x) is the strong classifier obtained by ensemble learning, and the sign function is used to take the positive and negative weighted results of the m weak classifiers, e m The error rate of the mth decision tree weak classifier.
[0064] In step 5.2, use the strong classifier C trained in step 5.1 and perform a performance test on the test set using the precision, accuracy, recall, and F1-score performance indicators.
[0065] The method proposed in the present invention deletes sparse attribute columns, encodes character data, fills in data, and normalizes the original railway accident history data at the data level. At the algorithm level, it uses the ensemble learning method AdaBoost and adopts a ten-fold cross-validation method during training to ensure the randomness and rationality of the training process, ultimately improving the performance of railway accident type prediction.
[0066] Example 1
[0067] Execute step 1. The railway accident data used in this embodiment comes from the railway equipment accident data publicly available from the Federal Railroad Administration of the United States. Data from 2016 to 2018 is used, including more than 7,000 railway accident records, 11 types of accidents, which can be further divided into five categories. The original data contains a total of 144 attributes, including accident type, specific location of the accident, time, railway number, vehicle information, personnel information, etc. For detailed data descriptions, see Tables 1 and 2.
[0068] Table 1 Description of the original railway accident dataset
[0069]
[0070] Table 2 Some attributes of railway accident raw data
[0071]
[0072] Since there are many data gaps in the original data, when the missing data accounts for more than 85% of the total data, conventional data completion methods cannot reasonably complete the data. Therefore, this embodiment uses 85% as the threshold for attribute sparsity and deletes all attributes with sparsity greater than this threshold. After screening in step 1, the number of attributes of the railway accident data is deleted from 144 to 80.
[0073] Execute step 2 to hard-code some character-type attributes in the railway accident data and convert them into numeric data. In this example, the 15 attributes narr1 through narr15 in the original data are directly deleted and not used as features for accident prediction. Instead, the limited character-type attributes, such as station, railroad, and typspd, are hard-coded, uniquely mapping each character to a numeric value to ensure consistency between these attribute types and other numeric attributes.
[0074] Execute step 3 to perform data filling on the railway accident data processed in steps 1 and 2. In this embodiment, a KNN-based data filling method is used. For each missing sample, the inter-sample distance between it and all other non-missing samples is calculated. The obtained inter-sample distances are sorted, and the weighted average of the top k closest samples is taken to fill the missing sample. In this embodiment, k is set to 5, and the weights of the five non-missing samples are assumed to be the same.
[0075] Execute step 4 to normalize the railway accident data processed in step 3. Normalize the data by column, using the attribute columns as the primary order. Split the normalized data into training and test sets in an 8:2 ratio, randomly sorting the samples before splitting.
[0076] Execute step 5. In this embodiment, the weak classifier set by the ensemble learning method AdaBoost is a decision tree, and the number of weak classifiers M is 200. 200 decision tree weak classifiers are trained on the training set data. After the classifier weights are updated, a strong classifier of railway accident type is finally obtained by weighting. This strong classifier is used to evaluate the method performance on the test set.
[0077] Since railway accident data contains multiple accident types, the classification results are considered as multiple binary classification problems. As with the binary classification problem, the four evaluation indicators of accuracy, precision, recall, and F1-score are used to test the performance of the classifier on the test set. In order to calculate these four indicators, the concept of confusion matrix is introduced. The confusion matrix of the binary classification results is shown in Table 3. The confusion matrix of the classification results of the method proposed in this invention on the test set is shown in Figure 3 .
[0078] Table 3 Binary classification confusion matrix
[0079]
[0080] According to the true category of the sample and the results of the model prediction, it can be divided into four cases: true positive (TP), false negative (FN), false positive (FP) and true negative (TN). According to the actual distribution of these four cases, the following four evaluation indicators can be calculated:
[0081] The calculation formula of accuracy is shown in formula (7).
[0082]
[0083] The calculation formula of precision is shown in formula (8).
[0084]
[0085] The calculation formula of recall is shown in formula (9).
[0086]
[0087] The calculation formula of F1-score is shown in formula (10).
[0088]
[0089] Wherein, L represents the total number of accident types, which is 11 in this embodiment; N i represents the number of samples with accident type i, N represents the total number of samples; TP i Indicates the number of samples correctly predicted as the i-th type of accident; FP i Indicates the number of samples that are incorrectly predicted as the i-th type of accident; TN i Indicates the number of samples that are correctly predicted not to be the i-th type of accident; FNi Represents the number of samples that are incorrectly predicted to be not the i-th accident type.
[0090] According to the four evaluation indicators described in formulas (7) to (10), the performance of the classifier on the test set is evaluated. The confusion matrix of the classification results of the method proposed in this invention on the test set is Figure 3 The horizontal axis represents the predicted accident category, the vertical axis represents the actual accident category, the diagonal column represents the number of TP and TN in each category, the upper right corner of the diagonal represents the number of FN, and the lower right corner of the diagonal represents the number of FP. Figure 3 It can be seen that the values on the diagonal account for the largest proportion of the sum of all values in each row, indicating that the proposed method performs well on the test set.
Claims
1. A railway accident type prediction method based on KNN and AdaBoost, characterized by: The following steps are involved: Step 1: Perform sparsity analysis on the original railway accident history dataset and delete some redundant attributes according to the sparsity threshold. The sparsity threshold is 85%. Step 2: hard-code the character attributes in the railway accident history dataset processed in Step 1 to convert the character attributes into numeric attributes; Step 3: Perform data filling based on the KNN algorithm on the railway accident history dataset processed in step 2 to fill in the missing values in the original data; Step 4: normalize the railway accident history dataset processed in step 3 and randomly divide it into training and test sets according to a certain ratio; Step 5: randomly divide the railway accident history dataset normalized in step 4 into a training set and a test set, use the AdaBoost algorithm to train the classifier on the training set data, and verify the classifier performance on the test set; In step 3, the KNN algorithm is used to fill in the missing values of the railway accident data. By calculating the distance between the missing sample and other complete samples, the first k closest complete samples are taken and the weighted average is used as the value after the missing sample is filled in. The specific steps of step 3 are: Step 3.1: For each accident record containing missing values ,calculate Inter-sample distance to all complete accident records d , use the KNN algorithm to sort the distances between the samples and retain the closest distance k Sample records as the basis for filling the current missing samples; The complete accident record represents a record without missing values, and the distance between samples d The calculation method is shown in formula (1): (1) in and Respectively represent i Records and j records. In this formula, the record on the left is a record with missing values, and the record on the right is a complete record. To the full record The distance between samples As shown in formula (1), it represents the similarity of two records in the sample space. The smaller the distance, the closer the two are. Step 3.2, based on the nearest predecessor in the sample space with missing value records k accident records, and fill in the missing records; k The formula for calculating the current missing records is shown in formula (2): (2) in is the i-th accident record with missing values, For the j The distance in the sample space is The most recent record, as shown in formula (2), is obtained after the missing value is filled. ; The data normalization formula in step 4 is formula (3), (3) in represents the railway accident dataset after processing in steps 1 to 3. and Respectively represent calculations in column-major order The mean and standard deviation of the railway accident data set are obtained by the operation of formula (3) after column normalization. The distribution of the data conforms to the normal distribution with a mean of 0 and a standard deviation of 1. In step 5, the AdaBoost ensemble learning method is used to establish a railway accident type classifier, including using M identical decision tree weak classifiers as weak learners and obtaining a final software defect strong classifier through weighted averaging; The training process of step 5 is specifically as follows: Step 5.1: Perform a ten-fold cross-validation on the resampled training set in step 4. The entire training data set is divided into 10 parts, each of which will be used as a validation set. During training, 9 parts of the data are used for training, and the remaining part is used as a validation set to test the classifier performance. Repeat this process 10 times, and the weighted average is used as the final performance indicator of the classifier. When the training process is completed, the trained railway accident type classifier is obtained. C , the process of training the final classifier using the AdaBoost learning method is shown in formulas (4), (5), and (6): (4) (5) (6) in, For the m The weights of the decision tree weak classifiers, For the m A decision tree weak classifier, is the strong classifier obtained by ensemble learning, Function used to get m The weighted positive and negative of the weak classifier results, For the m The error rate of classification by the decision tree weak classifier; Step 5.2: Use the strong classifier trained in step 5.1 C ,The performance of the trained software defect classifier is tested on the test set using the precision, correctness, recall and F1-score performance indicators.
Citation Information
Patent Citations
VANETs vehicle accident risk prediction model based on AdaBoost-SO
CN109558969A