A method for predicting vehicle accident risks based on big data analysis

Abstract

The present invention discloses a method for predicting vehicle accident risk based on big data analysis, which relates to the field of vehicle risk prediction. The method first selects vehicle data that have had accidents in the past, and selects vehicle data that have not had accidents and corresponding accident information as feature labels in the same historical time interval to obtain the original data set of the model; then the original data set of the model is subjected to noise reduction, sampling and normalization processing; then the processed original data set is subjected to multicollinearity test and correlation analysis to determine the feature set; according to the feature set, the closed features are selected to construct a sample data set; a prediction model is constructed, and the model is trained using an integration method; according to the prediction accuracy and evaluation indicators, the prediction performance of the model is evaluated; and the prediction results are output. The prediction of the accident probability of the present invention no longer relies on subjective judgment or expert experience, but relies on the machine learning algorithm constructed by time series data to directly calculate the probability of accident, which is more accurate and objective, and has high credibility.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle risk prediction, and in particular to a method for predicting vehicle accident risks based on big data analysis. Background Art

[0002] Traditional vehicle insurance often adopts a unified pricing method. The personal information of the car owner (such as age, gender, marital status, driving experience, etc.) and the basic information of the vehicle (such as mileage, duration, etc.) are only collected once in the early stage. No secondary statistical analysis of the data is conducted, and no reasonable distinction is made between insured persons with different backgrounds and completely different driving styles to explore their deep value. In response to this situation, two new car insurance models have emerged in the industry. They respectively record the mileage and travel days of the car owner to verify the actual premium to be paid. Although this has improved the current situation to a certain extent, it is difficult to match the actual behavior of the car owner during the insurance period. There is a lack of understanding of the driver's driving habits and the risk of accidents cannot be controlled.

[0003] Therefore, effectively predicting the probability of vehicle accidents is crucial to the current field of auto insurance pricing. At present, most of the existing methods of this technology are based on the basic characteristics of the vehicle, and subjective judgments are made on the characteristic factors that affect the occurrence of vehicle accidents, and then data modeling is performed. The above auto insurance pricing model is single, and the actual driving behavior is not taken into account in the initial pricing. It is impossible to predict the probability of vehicle accidents in the future, which makes it difficult for insurance companies to distinguish between high-quality customers with low accident rates and identify risky customers with high accident rates when taking out insurance, resulting in pricing solidification and high auto insurance claims rates.

[0004] However, existing technologies do not include driving behavior-related factors in the feature range, the feature dimension is single, and there is no grasp of driving habits based on changes over time to measure the causes of accidents. The model results have a weak ability to portray risks. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for predicting vehicle accident risks based on big data analysis. By constructing a machine learning prediction model, driving behavior time series data, vehicle basic data and vehicle accident records are selected as data sets, useful features are analyzed and screened from massive data, and the model is trained to predict the probability of accidents without relying on expert experience and subjective judgment.

[0006] The objective of the present invention is achieved through the following technical solutions:

[0007] A method for predicting vehicle accident risk based on big data analysis, comprising:

[0008] S1: Obtain the original data, collect the basic vehicle data and driving behavior data in the continuous time windows t-1, t-2, and t-3, and predict the accident risk at time t; select the vehicle data that has had accidents in the past, and select the same number of vehicle data with good driving behavior and no accidents in the same historical time interval, and the corresponding accident information as feature labels to obtain the original data set of the model;

[0009] S2, data preprocessing, denoising, sampling and normalization of the sample data in the original data set of the model, and converting the characteristic values ​​of the samples to the same dimension;

[0010] S3, feature engineering, conduct multicollinearity test and correlation analysis based on driving behavior and the single attribute factors of the vehicle itself, and screen the analysis results to determine the feature set;

[0011] S4, model training, select key features according to the feature set to build a model sample data set, and divide the model sample data set into a training set, a test set, and a validation set; build a machine learning prediction model, and use the integration method to use the training set to learn and train the machine learning prediction model;

[0012] S5, model evaluation, based on the prediction accuracy and receiver operating characteristic curve (ROC) evaluation index, select the validation set data to evaluate the prediction performance of the machine learning prediction model;

[0013] S6, output the vehicle accident risk prediction result.

[0014] Specifically, the normalization process is performed by linear transformation according to the following formula:

[0015] Normalization is performed based on the following linear transformation:

[0016]

[0017] in, , Respectively represent t The time point i The company's j The standard value and original value of the characteristic index, j =1, 2, ..., J , i =1, 2, ..., I , J and I Represent the total number of features and the total number of vehicle data respectively. The linear transformation expands each variable to the interval [0,1], which usually produces better results for the prediction model.

[0018] Specifically, step S3 specifically includes the following sub-steps:

[0019] S31: Based on the multicollinearity test of driving behavior and the single attribute factors of the vehicle itself, the correlation degree between the single attribute factors is clarified to ensure the mutual independence of the factors;

[0020] S32: Based on the correlation analysis between vehicle accidents and each single attribute factor, the influence of each single attribute factor on the occurrence of the accident is clarified;

[0021] S33: Sort the correlation scores of the single attribute factors of each dimension from high to low;

[0022] S34: setting a correlation score threshold, screening all single attribute factors that are higher than the correlation score threshold, and then comprehensively considering the relationship between multiple key factors to form a key factor;

[0023] S35: Filter redundant features through the feature selection model and finally determine the feature set.

[0024] Specifically, step S4 includes the following sub-steps:

[0025] S41: using the feature set determined in step S35, for each key factor, m consecutive data points are selected in the time series and aggregated into one piece of data, and a model sample is constructed by selecting all key features;

[0026] S42: Divide the data set into training set, test set and validation set;

[0027] S43: Construct a machine learning prediction model suitable for predicting vehicle accident rates, bring the training set data into the model for learning and training, and use the integration method to integrate the feature selection process into the learning and training. Finally, use the test set to verify the prediction results.

[0028] Specifically, the machine learning prediction model is a logistic regression model composed of time series variables, which has the following general form:

[0029] ,

[0030] In the above formula, is a binary variable, if , it means that the vehicle i There was an accident in the current month, otherwise, the vehicle i There has never been an accident, ; represents the intercept, yes t - l The coefficient vector of the moment explanatory variables; It is a vehicle i In time t - l of p dimensional driving behavior feature vector, l =0, 1, 2, ..., L ; l and L is the number of time spans and the maximum span; At the beginning of the observation period, d is the length of the observation period.

[0031] Beneficial effects of the present invention:

[0032] 1. Aggregate and analyze the massive data collected by vehicle hardware equipment, and use feature selection algorithms to extract characteristic indicators that are helpful in evaluating driving behavior.

[0033] 2. The prediction of accident probability no longer relies on subjective judgment or expert experience. Instead, it directly calculates the probability of an accident using a machine learning algorithm built from time series data, which is more accurate, objective, and highly credible.

[0034] 3. Compared with complex black box models, the algorithm adopted in this invention has a sound theoretical basis and strong interpretability.

[0035] 4. Solve the pain points of the insurance industry and provide insurance companies with technical support for customer grading system and personalized auto insurance pricing, which helps reduce the compensation risk of insurance institutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of the method of the present invention;

[0037] Figure 2 It is the overall technical flow chart of the present invention;

[0038] Figure 3 It is a feature engineering process flow chart. DETAILED DESCRIPTION

[0039] In order to have a clearer understanding of the technical features, purposes and beneficial effects of the present invention, the technical solution of the present invention is now described in detail below. Obviously, the implementation cases described are part of the embodiments of the present invention, not all of the embodiments, and cannot be understood as limiting the scope of the implementation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the protection scope of the present invention. Example

[0040] In this embodiment, Figure 1As shown, S1: obtain the original data, collect the basic data and driving behavior data of the vehicle in the continuous time windows t-1, t-2, and t-3, and predict the accident risk at time t; select the vehicle data that has had accidents in the past, and select the same number of vehicle data with good driving behavior and no accidents in the same historical time interval, and the corresponding accident information as feature labels to obtain the original data set of the model;

[0041] S2, data preprocessing, denoising, sampling and normalization of the sample data in the original data set of the model, and converting the characteristic values ​​of the samples to the same dimension;

[0042] S3, feature engineering, conduct multicollinearity test and correlation analysis based on driving behavior and the single attribute factors of the vehicle itself, and screen the analysis results to determine the feature set;

[0043] S4, model training, select key features according to the feature set to build a model sample data set, and divide the model sample data set into a training set, a test set, and a validation set; build a machine learning prediction model, and use the integration method to use the training set to learn and train the machine learning prediction model;

[0044] S5, model evaluation, based on the prediction accuracy and receiver operating characteristic curve (ROC) evaluation index, select the validation set data to evaluate the prediction performance of the machine learning prediction model;

[0045] S6, output the vehicle accident risk prediction result.

[0046] The present invention comprehensively considers the basic characteristics of the vehicle, driving behavior characteristics, potential risk factors, etc., screens out the key factors that lead to accidents, and provides a method for predicting the occurrence of accidents for use in premium pricing. Figure 2 The present invention will be described in detail.

[0047] like Figure 2 The figure shows the overall steps of the technical solution, and the specific implementation steps of the method are as follows:

[0048] S1: Acquisition of raw data: The present invention collects continuous time windows t -1, t -2, t -3 based on vehicle basic data and driving behavior data to predict its time t The accident risk at the location. The data of vehicles that had accidents from 2020 to 2022 were selected, and the same number of vehicle data with good driving behavior and no accidents were selected in the same time period, as well as the corresponding accident information (for example: the cumulative driving behavior data of the previous three months corresponds to the current accident record) as feature labels to obtain the original data set of the model, as shown in Table 1 below;

[0049] Table 1 Composition of the vehicle accident rate prediction model dataset

[0050]

[0051] S2: Data preprocessing: The data in the production environment is complex and disordered, so some basic processing is required before model training, such as noise reduction, sampling, and normalization. Some dirty or unavailable data and outliers need to be removed. In addition, when the amount of data is extremely large, the massive data needs to be sampled, analyzed and modeled using the sampled data, and finally the characteristic values ​​of the samples need to be converted to the same dimension.

[0052] The present invention performs normalization based on the following linear transformation:

[0053]

[0054] in, , Respectively represent t The time point i The company's j The standard value and original value of the characteristic index, j =1, 2, ..., J , i =1, 2, ..., I , J and I Represent the total number of features and the total number of vehicle data respectively. The linear transformation expands each variable to the interval [0,1], which usually produces better results for the prediction model.

[0055] In the samples collected in reality, there is noise in the data, which will greatly affect the training effect of the model. The method of removing noise in the present invention is specifically: there are different processing methods according to different business scenarios, and only the 3σ principle of normal distribution is proposed here. Normal distribution is also called normal distribution. It is a kind of probability distribution of continuous random variables. A large number of phenomena in nature, human society, psychology, and education are normally distributed, such as the level of ability and the quality of students' grades. We can establish the quality distribution of the data set as a normal distribution. It will have different distribution forms depending on the mean, standard deviation and unit of the random variable. The normal distribution can be expressed as a probability density function.

[0056] In samples collected in reality, imbalance between positive and negative classes is a very common problem in real data. A classifier often has an accuracy of nearly 90%, but the recall for distinguishing minority samples is only about 10%. This is very unfavorable for us to correctly identify minority samples. Generally speaking, when the ratio of positive to negative samples exceeds 1:3, the classifier will tend to judge negative samples (manifested in that the recall of negative samples is too high, while the recall of positive samples is low, and the overall accuracy will still be very good). In this case, we can say that this classifier has failed because it cannot achieve our positioning of this group of people.

[0057] There are three ways to deal with sample imbalance:

[0058] (1) Undersampling. The undersampling method is to remove some majority samples in the training set so that the magnitude of the two types of data is close, and then the learning is carried out normally.

[0059] The disadvantage of this method is that it discards many counterexamples, which will cause the balanced training set to be smaller than the initial training set. Moreover, if the sampling randomly discards counterexamples, the information that has been collected will be lost, and often important information will be lost.

[0060] Undersampling improvement method 1: However, we can change the sampling method to improve the undersampling method. For example, we can divide the majority of samples into core samples and non-core samples. Non-core samples are samples with a lower probability of achieving the prediction target. We can consider deleting them from the non-core samples instead of randomly undersampling them. This ensures that the core sample data that requires machine learning judgment will not be lost.

[0061] For example, if we still aim to predict user registration, we can classify all sessions under a user name with a bounce rate of 100% as non-core samples, because the amount of information contained in the bounced users is very small (other behavioral features are empty). Excluding this part of the user samples can retain as much original data information as possible.

[0062] Undersampling improvement method 2: Another undersampling improvement method is the inheritance learning system proposed by EasyEnsemble, which divides the majority of samples into N sets, and then combines the divided sets with a minority of samples, thus forming N training sets, and each training set is undersampled, but from a global perspective, no information is lost.

[0063] (2) Oversampling. Oversampling is to expand the minority samples in the training set, that is, to increase the minority samples so that the number of two types of data is close, and then learn.

[0064] A simple and crude method is to copy the minority samples. The disadvantage is that although additional training data is introduced, no new information is added to the minority samples, which can easily cause overfitting.

[0065] Oversampling Improvement Method 1: Adding white noise (such as Gaussian noise) to minority class samples through sampling methods to generate new samples can alleviate this problem to a certain extent. For example, for age, original age = new age + random (0, 1).

[0066] The representative oversampling algorithm is the SMOTE algorithm. SMOTE obtains new samples by interpolating minority samples. For example, for each minority class sample a, sample b is selected from the samples closest to a, and then a point is randomly selected from ab as the new sample.

[0067] (3) Threshold movement. The central idea of ​​this type of method is not to rebalance the sample set, but to improve the decision-making process of the algorithm.

[0068] To give a simple example, when we usually classify the prediction results, when the predicted y (y represents the possibility of the positive class) value > 0.5, the prediction result is judged to be positive, otherwise it is negative.

[0069] Specify decision rules: If >1, the prediction is positive. It is not difficult to find that the threshold value is set to 0.5 only when the positive and negative ratio in the sample is 1:1. If the sample imbalance decision rule needs to be changed, let m + represents the number of positive examples, m - Represents the number of negative examples and improves the decision rule:

[0070] like , then the prediction is a positive example.

[0071] Because the training set is an unbiased sampling of the overall sample, the observed probability represents the true probability, and the decision rule Represents the observed probability of a positive example in the sample. As long as the predicted probability in the classifier is higher than the observed probability, the goal of improving the judgment result is achieved.

[0072] S3: Feature engineering, such as Figure 3 As shown in the figure, the feature engineering process is as follows:

[0073] S31: Based on the multicollinearity test of driving behavior and the single attribute factors of the vehicle itself, the correlation degree between the single attribute factors is clarified to ensure the mutual independence of the factors;

[0074] S32: Based on the correlation analysis between vehicle accidents and each single attribute factor, the influence of each single attribute factor on the occurrence of the accident is clarified;

[0075] S33: Sort the correlation scores of the single attribute factors of each dimension from high to low;

[0076] S34: setting a correlation score threshold, screening all single attribute factors that are higher than the correlation score threshold, and then comprehensively considering the relationship between multiple key factors to form a key factor;

[0077] S35: Filter redundant features through the feature selection model and finally determine the feature set.

[0078] The present invention proposes to set the penalty factor l The 1-norm is combined with the logistic regression model, and the integration method is used to integrate the feature selection process into the learning and training. That is, feature selection and classifier learning and training are carried out simultaneously, and the two processes are combined into one. The coefficients before some invalid features are compressed to zero, which reduces the complexity of the model and improves the learning efficiency of the classifier.

[0079] In this embodiment, the model training process in step S4 is specifically as follows:

[0080] S41: using the feature set determined in step S35, for each key factor, m consecutive data points are selected in the time series and aggregated into one piece of data, and a model sample is constructed by selecting all key features;

[0081] S42: Divide the data set into training set, test set and validation set;

[0082] S43: Build a machine learning prediction model suitable for predicting vehicle accident rates.

[0083] In the present invention, a logistic regression model is selected, the training set data is brought into the model for learning and training, and the prediction results are verified using the test set. Logistic regression, as a commonly used technical method in the field of accident prediction, has been widely used. The present invention proposes a logistic regression model composed of time series variables, which has the following general form:

[0084] ,

[0085] In the above formula, is a binary variable, if , it means that the vehicle i There was an accident in the current month, otherwise, the vehicle i There has never been an accident, ; represents the intercept, yes t - l The coefficient vector of the moment explanatory variables; It is a vehicle iIn time t - l of p dimensional driving behavior feature vector, l =0, 1, 2, ..., L ; l and L is the number of time spans and the maximum span; At the beginning of the observation period, d is the length of the observation period.

[0086] S5: Model evaluation: Based on the prediction accuracy and receiver operating characteristic (ROC) evaluation index, the validation set data was selected to evaluate the prediction performance of the model;

[0087] S6: Output prediction results.

[0088] The present invention constructs a machine learning prediction model, selects driving behavior time series data, vehicle basic data and vehicle accident records as data sets, analyzes and selects useful features from massive data, does not rely on expert experience and subjective judgment, and trains the model to predict the probability of accident occurrence. The present invention aggregates and analyzes the massive data collected by vehicle hardware equipment, and through the feature selection algorithm, it can extract characteristic indicators that are helpful in evaluating driving behavior. The prediction of accident probability no longer depends on subjective judgment or expert experience. It relies on the machine learning algorithm constructed by time series data to directly calculate the probability of accident, which is more accurate and objective, and has high credibility. Compared with complex black box models, the algorithm adopted by the present invention has a sound theoretical basis and strong interpretability. The present invention solves the pain points of the insurance industry, can provide insurance companies with technical support for customer grading system and personalized auto insurance pricing, and helps reduce the insurance company's payment risk.

[0089] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of the present invention to be protected is defined by the attached claims and their equivalents.

Claims

1. A method for predicting vehicle accident risk based on big data analysis, characterized in that: include: S1: Obtain the original data, collect the basic vehicle data and driving behavior data in the continuous time windows t-1, t-2, and t-3, and predict the accident risk at time t; select the vehicle data that has had accidents in the past, and select the same number of vehicle data with good driving behavior and no accidents in the same historical time interval, and the corresponding accident information as feature labels to obtain the original data set of the model; S2, data preprocessing, denoising, sampling and normalization of the sample data in the original data set of the model, and converting the characteristic values ​​of the samples to the same dimension; S3, feature engineering, performs multicollinearity test and correlation analysis based on driving behavior and the single attribute factors of the vehicle itself, and screens the analysis results to determine the feature set, which specifically includes the following sub-steps: S31: Based on the multicollinearity test of driving behavior and the single attribute factors of the vehicle itself, the correlation degree between the single attribute factors is clarified to ensure the mutual independence of the factors; S32: Based on the correlation analysis between vehicle accidents and each single attribute factor, the influence of each single attribute factor on the occurrence of the accident is clarified; S33: Sort the correlation scores of the single attribute factors of each dimension from high to low; S34: setting a correlation score threshold, screening all single attribute factors that are higher than the correlation score threshold, and then comprehensively considering the relationship between multiple key factors to form a key factor; S35: Filter redundant features through feature selection model and finally determine the feature set; S4, model training, selects key features according to the feature set to build a model sample data set, and divides the model sample data set into a training set, a test set, and a validation set; builds a machine learning prediction model, and uses the training set to learn and train the machine learning prediction model using an ensemble method, which specifically includes the following sub-steps: S41: using the feature set determined in step S35, for each key factor, m consecutive data points are selected in the time series and aggregated into one piece of data, and a model sample is constructed by selecting all key features; S42: Divide the data set into training set, test set and validation set; S43: Build a machine learning prediction model suitable for predicting vehicle accident rates, bring the training set data into the model for learning and training, and use the integration method to integrate the feature selection process into the learning and training, and finally use the test set to verify the prediction results; The machine learning prediction model is specifically a logistic regression model composed of time series variables, which has the following general form: , ； In the above formula, is a binary variable, if , it means that the vehicle i There was an accident in the current month, otherwise, the vehicle i There has never been an accident, ; represents the intercept, yes t - l The coefficient vector of the moment explanatory variables; It is a vehicle i In time t - l of p dimensional driving behavior feature vector, l =0, 1, 2, ..., L ; l and L is the number of time spans and the maximum span; At the beginning of the observation period, d is the length of the observation period; S5, model evaluation, based on the prediction accuracy and receiver operating characteristic curve (ROC) evaluation index, select the validation set data to evaluate the prediction performance of the machine learning prediction model; S6, output the vehicle accident risk prediction result.

2. The method for predicting vehicle accident risk based on big data analysis according to claim 1, characterized in that: The normalization process is specifically performed by the following linear transformation: Normalization is performed based on the following linear transformation: ； in, , Respectively represent t The time point i The company's j The standard value and original value of the characteristic index, j =1, 2, ..., J , i =1, 2, ..., I , J and I Represent the total number of features and the total number of vehicle data respectively. The linear transformation expands each variable to the interval [0,1], which usually produces better results for the prediction model.

Images