A dangerous driving behavior prediction method based on multi-source data

Abstract

The present invention discloses a method for predicting dangerous driving behavior based on multi-source data, including: collecting vehicle motion data and vehicle video data in a natural driving environment; selecting sudden acceleration, sudden deceleration and sharp turn as calibration objects, using longitudinal acceleration threshold to calibrate sudden acceleration and sudden deceleration, and distinguishing normal driving samples from abnormal driving samples; dividing abnormal driving samples into three categories, dividing abnormal driving samples into different feature sets by different combinations of the three categories of data, using different models to model each feature set, and selecting the model with the best comprehensive prediction ability as the best prediction model; by changing the input feature set and time window of the best prediction model, finding the best feature set and best time window for predicting dangerous driving behavior. The present invention uses vehicle video data for predicting driving behavior, and after fusing vehicle motion data and facial expression data, the accuracy of model prediction is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of driving risks, and in particular to a method for predicting dangerous driving behavior based on multi-source data. Background Art

[0002] With the rapid increase in population and the number of motor vehicles, road traffic safety issues have become increasingly prominent, and road traffic injuries have become an important cause of casualties. Abnormal driving behavior during driving is closely related to traffic accidents. Therefore, in order to improve road traffic safety and reduce the occurrence of road traffic accidents, research on driver driving behavior is very necessary. The identification and prediction of driver behavior is an urgently needed module in the driver safety system. Through appropriate warning feedback, potential unsafe driving behaviors of drivers can be prevented and traffic accidents can be reduced.

[0003] At present, the research on driving behavior has achieved rich results, but there are still the following shortcomings: most of the existing research focuses on the identification and classification of driving behavior, and there is less research on the prediction of driving behavior; in the selection of data sets, the data sets used are limited to objective vehicle motion data, and the data are relatively single, resulting in low prediction accuracy. Summary of the invention

[0004] Purpose of the invention: In view of the above problems, the purpose of the present invention is to provide a dangerous driving behavior prediction method based on multi-source data, comprehensively consider realistic factors, and provide a dangerous driving behavior prediction scheme for use in road environments.

[0005] Technical solution: A method for predicting dangerous driving behavior based on multi-source data of the present invention comprises the following steps:

[0006] S10, collecting vehicle motion data and vehicle video data in a natural driving environment, preprocessing the data, and retaining valid data;

[0007] S20, according to the obtained change relationship between the vehicle speed and the lateral acceleration, select sudden acceleration, sudden deceleration and sharp turn as calibration objects, combine the acceleration and speed data, use the lateral acceleration threshold to calibrate the sharp turn, use the longitudinal acceleration threshold to calibrate the sudden acceleration and sudden deceleration, and distinguish between normal driving samples and abnormal driving samples;

[0008] S30, classifying abnormal driving samples into three categories, namely, vehicle motion data, emotion data, and micro-expression data, and classifying abnormal driving samples into different feature sets by combining the three types of data in different ways, using different models to build models on each feature set, and selecting the model with the best comprehensive prediction ability as the best prediction model;

[0009] S40, by changing the input feature set and time window of the optimal prediction model, analyzing the change pattern of the prediction ability of each feature set with the time window, and finding the optimal feature set and optimal time window for predicting dangerous driving behavior.

[0010] Further, in step S10, preprocessing the data includes:

[0011] S101, splicing facial expression data and vehicle motion data according to timestamps, where the facial expression data is sourced from vehicle video data;

[0012] S102, clearing invalid data in the spliced ​​data set by adjacent value filling and direct deletion;

[0013] S103, using the SG filtering method to smooth the valid data, and using the Z-score method to standardize the smoothed data.

[0014] Further, in step S30, the feature set includes: emotion data set, micro-expression data set, vehicle motion data set, emotion data set + micro-expression data set, vehicle motion data set + emotion data set, vehicle motion data set + micro-expression data set, vehicle motion data set + emotion data set + micro-expression data set.

[0015] Further, in step S30, the model includes a decision tree model, a random forest model, a gradient boosting tree model, a support vector machine model and a naive Bayes model;

[0016] The evaluation criteria of the best prediction model is expressed by parameter F1, which is expressed as:

[0017] F1=(2*Precision*Recall) / (Precision+Recall)

[0018] Precision represents the accuracy, and Recall represents the recall rate.

[0019] Further, in step S30, using different models to perform modeling on each feature set includes:

[0020] S301, using the SMOTE algorithm to balance the number of positive and negative samples in the training set, generating new abnormal driving samples, so that the ratio of positive and negative samples in the training set is 1:1;

[0021] S302, before training each model, the model hyperparameters are adjusted using the Bayesian optimization algorithm, and during the search process, the performance of the model is evaluated using 5-fold cross validation;

[0022] S303, through the adjustment of hyperparameters and cross-validation, different feature sets are combined for modeling, and the prediction performance of different models is compared using the F1 value.

[0023] Further, in step S40, changing the time window includes:

[0024] The time window is divided into more detailed parts, and the starting point of the time window is slid forward from the moment when the driving behavior occurs, with a sliding step of Δt. Each starting point is modeled using a variety of time window lengths using the optimal prediction model. A grid search strategy is used to determine the time window and feature set combination with the best prediction effect based on the F1 value of the dangerous driving behavior model.

[0025] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0026] 1. This invention uses vehicle video data for the prediction of driving behavior for the first time. After integrating vehicle motion data and facial expression data, the accuracy of model prediction is greatly improved;

[0027] 2. The present invention fully considers the performance of different prediction models on different feature sets and screens the optimal prediction model for abnormal driving behavior;

[0028] 3. Existing driving behavior prediction research has not yet unified the observation time window. The present invention provides a method for comparing and optimizing the prediction feature set and the prediction time window. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flowchart of the present invention;

[0030] Figure 2 The schematic diagram of the vehicle force;

[0031] Figure 3 Schematic diagram of the SMOTE algorithm;

[0032] Figure 4 Schematic diagram of time window division. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0034] like Figure 1 , a dangerous driving behavior prediction method based on multi-source data described in this embodiment, the flow chart is as follows Figure 1 As shown, the prediction method includes the following steps:

[0035] S10, collecting vehicle motion data and vehicle video data in a natural driving environment, preprocessing the data, and retaining valid data.

[0036] The data of this embodiment comes from the natural driving experiment data. The motion data of the vehicle is collected by the high-precision GPS device VBOX. VBOX records GPS and other data information at a frequency of 10Hz, and the collected motion data is directly stored in the built-in memory card for export. The vehicle video data is shot by a dual-camera driving recorder with a picture resolution of 1080P. The driving recorder has a built-in 128G memory card to ensure one day of video storage. The driving recorder is installed above the front windshield, and the driver himself confirms that it does not interfere with the line of sight. The front and rear dual cameras are used to record the driver's facial expression video and the situation outside the car. The non-invasive collection method minimizes the interference of the equipment to the driver. In the process of adjusting the shooting angle of the driving recorder, it is ensured that the front camera can capture the driver's full face without capturing the passengers. The video screen will be subsequently input into professional software for processing to obtain numerical facial data. All data that need to be collected have been notified to the driver in advance, and the experiment is carried out with the driver's knowledge and consent.

[0037] Since the data is collected in a natural environment, the GPS receiver may be blocked by tall buildings or numbers, resulting in signal interruption, data noise and invalid values. Facial videos need to be extracted from expression data through algorithms and synchronized with VBOX data. Therefore, before modeling, data cleaning, missing value filling, data synchronization and other preprocessing work are required, including:

[0038] S101, splicing facial expression data and vehicle motion data according to timestamps, where the facial expression data is sourced from vehicle video data;

[0039] Vehicle video data cannot be used directly for modeling and needs to be converted into numerical data through processing. Since computer vision knowledge is too complicated, professional software FaceReader 8.0 is used for facial expression analysis. It can detect emotional expressions on the face, identify seven basic emotions as well as the valence and arousal of emotions, and measure the 20 most common facial micro-expression action units AU. The output data used in this embodiment contains 7 expressions, valence, arousal and the values ​​of 20 action units, with a frequency of 10Hz. The value of valence is between -1 and 1, and all other data are in the range of 0-1. The higher the value, the greater the degree.

[0040] During the working process, the online car-hailing driver will stop and turn off the engine due to waiting for passengers or taking a rest. VBOX will save a record file every time the vehicle is turned off, and the driving recorder will store a video file every minute. The test data of a driver is stored in multiple files. In order to obtain continuous data, it is necessary to splice the files of each driver vertically. The video data and vehicle motion data are collected by two devices. The VBOX device records the standard UTC time by default, while the driving recorder displays Beijing time. In this embodiment, the VBOX data is converted to Beijing time by adding 8 hours to achieve time zone unification. At the same time, the data time difference is determined by video, the timestamp is adjusted to synchronize the vehicle motion data and facial expression data, and horizontal splicing is performed. The total number of features of the spliced ​​data set is 34.

[0041] S102, clearing invalid data in the spliced ​​data set by adjacent value filling and direct deletion.

[0042] In this data set, invalid data includes erroneous data and missing values. The erroneous data mainly exists in the vehicle motion data. After the VBOX is started, it takes a while to lock at least 5 satellites to record accurate data. Therefore, the data with less than 5 satellites in the front part of each VBOX file are invalid data. During the vehicle movement process, due to the obstruction of trees or tall buildings, the GPS receiver cannot obtain satellite signals, resulting in missing data.

[0043] In vehicle video data, when the driver's head turns too much, such as looking back to talk to passengers, or when the eyes are blocked, missing values ​​will be generated because the face cannot be modeled. For missing values, it is necessary to adopt a suitable processing method according to the type, cause, quantity and distribution of the missing values ​​and the characteristics of the data. Since the invalid values ​​caused by the restart of VBOX exist at the beginning of each data file, the characteristics are obvious, and they exist in segments. The meaning of filling is not great, and this embodiment only requires motion data of a specific period of time, so it can be directly deleted. Because the invalid data generated by the GPS signal blocking, if the duration is short, there is a certain temporal sequence between vehicle data, and there will be no significant changes in a short time, so the method of using adjacent value filling is used for processing. If the satellite signal is not received for a long time, resulting in a long period of invalid data, the data will be discarded. Facial expression data will generate missing values ​​because the face is not captured, or the eyebrows are blocked. If a small amount of data is missing, adjacent values ​​are used to fill it, and if a large amount of data is missing, the segment is directly deleted.

[0044] S103, using the SG filtering method to smooth the valid data, and using the Z-score method to standardize the smoothed data.

[0045] In the process of data collection, due to equipment failure or data transmission problems, the original data is jittery and the track is jagged. It is necessary to smooth the data to improve the smoothness of the data and reduce the interference of noise on modeling. In this embodiment, the SG filtering method is used for data processing to make the data smoother and closer to the true value.

[0046] The dimension of the data will affect the speed of gradient descent. When the magnitude difference between features is too large, the speed of parameter descent is too slow, oscillating back and forth, and the descent path is complex, and even the global optimal solution cannot be found. Since the data used in this example has magnitude differences, such as emotion and micro-expression data between 0-1, and vehicle motion data between 0-200, before using the model that focuses on the magnitude of features (support vector machine), the Z-Score is used to scale the features. The expression is:

[0047]

[0048] Where X is the original eigenvalue, μ is the feature mean, δ is the feature standardization, X * is the transformed feature value.

[0049] S20, based on the obtained relationship between vehicle speed and lateral acceleration changes, select sudden acceleration, sudden deceleration and sharp turn as calibration objects, combine the acceleration and speed data, use the lateral acceleration threshold to calibrate the sharp turn, and use the longitudinal acceleration threshold to calibrate the sudden acceleration and sudden deceleration, so as to distinguish normal driving samples from abnormal driving samples.

[0050] (1) Lateral acceleration threshold

[0051] The radius of the curve has the most significant impact on the safety of the curve in road design. When a vehicle travels on a circular curve, it will be subjected to a radial force, usually called centrifugal force. Due to the centripetal acceleration, there is also an outward radial force acting on the center of curvature. The road is set with an inclination to balance the centripetal acceleration. This inclination of the road is called superelevation. The magnitude of the centripetal acceleration depends on the lateral friction between the tire and the road surface, as well as the weight of the vehicle on the inclined road surface. The centrifugal force F c The expression is:

[0052]

[0053] Where: m is the vehicle mass, a is c is the acceleration of the vehicle when it moves in a curve; v is the speed; R is the curve radius; W is the weight of the vehicle.

[0054] like Figure 2 As shown, when the vehicle is in equilibrium with respect to the slope (the vehicle is moving forward but neither sliding up nor down), F c Indicates the centrifugal force, FN Represents the supporting force, and the horizontal component force expression is:

[0055] F c cosα=Wsinα+Wf s cosα

[0056] where f s is the radial friction coefficient, then the curve radius R is expressed as:

[0057]

[0058] The acceleration of the vehicle when it moves in a curve is:

[0059]

[0060] If the road slope is set to 0, the radial friction coefficient is:

[0061]

[0062] Where G is gravity and the radial friction coefficient is:

[0063] f s =0.21-0.001V

[0064] Where V is the speed of the vehicle in km / h, then the lateral acceleration threshold is:

[0065] a c =(0.21-0.001V)g

[0066] (2) Longitudinal acceleration threshold

[0067] Most drivers decelerate at a rate higher than 4.5 m / s when they encounter unexpected objects on the road and need to stop. 2 , so in this embodiment, 4.5m / s is selected 2 (corresponding to 0.45g) as the longitudinal acceleration threshold.

[0068] In order to avoid the impact of sensor data fluctuations on the detection results and to obtain driving behaviors with a relatively high degree of danger, this embodiment sets the lateral acceleration exceeding the threshold continuously for more than 1 second as a sharp turn behavior, and the longitudinal acceleration exceeding 0.45g for more than 0.5 second as a sudden acceleration and deceleration behavior. The time point when the acceleration exceeds the threshold for the first time is set as the start time point of the dangerous driving behavior.

[0069] In this embodiment, the lateral acceleration threshold related to the road linear design and speed is used to calibrate the sharp turn, and the longitudinal acceleration threshold is used to determine the time point of the sudden acceleration and deceleration, and the data 15 seconds before the event is extracted. The pre-dangerous driving samples are selected according to the timestamp to ensure that the samples do not contain the data of the last dangerous driving, and finally 539 samples are obtained. At the same time, 3937 normal driving clips of the same time length are selected to form a complete data set.

[0070] S30, classifying abnormal driving samples into three categories, namely, vehicle motion data, emotion data, and micro-expression data, and classifying abnormal driving samples into different feature sets by combining the three types of data in different ways, using different models to build models on each feature set, and selecting the model with the best comprehensive prediction ability as the best prediction model;

[0071] The feature sets include: emotion dataset Emotion, micro-expression dataset AU, vehicle motion dataset Car_motion, emotion dataset + micro-expression dataset Emotion+AU, vehicle motion dataset + emotion dataset Car_motion+Emotion, vehicle motion dataset + micro-expression dataset Car_motion+AU, vehicle motion dataset + emotion dataset + micro-expression dataset All. The emotion dataset includes seven basic emotions, valence and arousal, the micro-expression dataset includes 20 action units, and the vehicle motion dataset includes speed, longitudinal speed, lateral acceleration, longitudinal acceleration, and heading angle.

[0072] In this embodiment, various representative algorithms are used for testing, including decision tree model DecisionTree, random forest model RandomForest, gradient boosting tree model GradientBoosting, support vector machine model SVM and naive Bayes model Naive Bayes model. The performance of different prediction models on each feature set is compared, and the evaluation criterion of the best prediction model is represented by parameter F1, which is expressed as:

[0073] F1=(2*Precision*Recall) / (Precision+Recall)

[0074] Precision represents the accuracy, and Recall represents the recall rate.

[0075] Using different models to model each feature set includes:

[0076] S301, to avoid inaccurate prediction caused by imbalanced data samples, the SMOTE algorithm is used to balance the number of positive and negative samples in the training set, and new abnormal driving samples are generated so that the ratio of positive and negative samples in the training set is 1:1;

[0077] S302, before training each model, the model hyperparameters are adjusted using the Bayesian optimization algorithm, and during the search process, the performance of the model is evaluated using 5-fold cross validation;

[0078] S303, through the adjustment of hyperparameters and cross-validation, different feature sets are combined for modeling, and the prediction performance of different models is compared using the F1 value.

[0079] The main concept of the SMOTE algorithm is to use existing abnormal driving samples to artificially generate new samples of minority groups through the nearest neighbor algorithm. Figure 3 As shown, first select a minority class sample, then select several samples from the k nearest neighbors, and finally generate new samples on the line.

[0080] By comparing the performance of different prediction models on various feature sets, the gradient boosting tree achieves the best prediction ability on all feature sets, as shown in Table 1. When using vehicle motion data + emotion data, it achieves the best overall prediction effect, with an F1 value of 92.66%. Therefore, the gradient boosting tree is selected as the best prediction model.

[0081] Table 1 Model prediction performance table

[0082]

[0083]

[0084] S40, by changing the input feature set and time window of the optimal prediction model, analyzing the change pattern of the prediction ability of each feature set with the time window, and finding the optimal feature set and optimal time window for predicting dangerous driving behavior.

[0085] Changing the time window includes:

[0086] The time window is divided into more detailed parts, and the starting point of the time window is slid forward from the moment when the driving behavior occurs, with a sliding step of Δt. Each starting point is modeled using a variety of time window lengths using the optimal prediction model. A grid search strategy is used to determine the time window and feature set combination with the best prediction effect based on the F1 value of the dangerous driving behavior model.

[0087] In this embodiment, the 15s data in advance is used to predict driving behavior, but it is not certain that this is the most suitable input data for prediction. In order to obtain better prediction results, the 15s data is divided more finely to determine the optimal input data. The time window slides forward from the moment the driving behavior occurs (planned to be time 0), and the sliding length is 1s. Each starting point attempts to use multiple time window lengths for modeling. As the starting point of the time window moves forward, the maximum time window length that can be taken continues to decrease. When the starting point moves to 14s before the behavior, the time window length is only 1s. A total of 120 prediction models were established, and a grid search strategy was used to determine the time window with the best prediction effect based on the performance of the dangerous driving behavior prediction model. The specific time window divisions are as follows: Figure 4 As shown in Table 2. The top 20 time windows of the prediction ability of each feature set are intercepted, as shown in Table 2. It can be seen that on the micro-expression feature set, the model uses 0-15s data for the best prediction effect, with an F1 of 60.45%. When using emotional data, the best results can be obtained by using the data 2-9s before the behavior, with an F1 of 71.82%. When using vehicle motion data, the first 10 time windows all start from 0s before the event, and the best result is using the data 0-3s before the event, with an F1 of 93.10%.

[0088] According to existing research, the average reaction time of drivers under 40 is less than 1.1s, and the average reaction time of drivers over 40 is less than 1.15s. The present invention fully considers the needs of actual driver reaction time, and the best prediction effect that can be obtained by using all features is 85.88% when 1s driver reaction time is reserved.

[0089] Table 2 Feature set F1 ranking

[0090]

[0091]

[0092]

Claims

1. A method for predicting dangerous driving behavior based on multi-source data, characterized in that: The prediction method includes the following steps: S10, collecting vehicle motion data and vehicle video data in a natural driving environment, preprocessing the data, and retaining valid data; S20, according to the obtained change relationship between the vehicle speed and the lateral acceleration, select sudden acceleration, sudden deceleration and sharp turn as calibration objects, combine the acceleration and speed data, use the lateral acceleration threshold to calibrate the sharp turn, use the longitudinal acceleration threshold to calibrate the sudden acceleration and sudden deceleration, and distinguish between normal driving samples and abnormal driving samples; S30, classifying abnormal driving samples into three categories, namely, vehicle motion data, emotion data, and micro-expression data, and classifying abnormal driving samples into different feature sets by combining the three types of data in different ways, using different models to build models on each feature set, and selecting the model with the best comprehensive prediction ability as the best prediction model; S40, by changing the input feature set and time window of the best prediction model, analyzing the change rule of the prediction ability of each feature set with the time window, and finding the best feature set and best time window for predicting dangerous driving behavior; In step S30, the model includes a decision tree model, a random forest model, a gradient boosting tree model, a support vector machine model and a naive Bayes model; The evaluation criteria of the best prediction model is expressed by parameter F1, which is expressed as: F1=(2*Precision*Recall) / (Precision+Recall) Precision represents the accuracy, and Recall represents the recall rate; In step S30, using different models to perform modeling on each feature set includes: S301, using the SMOTE algorithm to balance the number of positive and negative samples in the training set, generating new abnormal driving samples, so that the ratio of positive and negative samples in the training set is 1:1; S302, before training each model, the model hyperparameters are adjusted using the Bayesian optimization algorithm, and during the search process, the performance of the model is evaluated using 5-fold cross validation; S303, through the adjustment of hyperparameters and cross-validation, different feature sets are combined for modeling, and the prediction performance of different models is compared using the F1 value.

2. The method for predicting dangerous driving behavior according to claim 1, characterized in that: In step S10, preprocessing the data includes: S101, splicing facial expression data and vehicle motion data according to timestamps, where the facial expression data is sourced from vehicle video data; S102, clearing invalid data in the spliced ​​data set by adjacent value filling and direct deletion; S103, using the SG filtering method to smooth the valid data, and using the Z-score method to standardize the smoothed data.

3. The method for predicting dangerous driving behavior according to claim 2, characterized in that: In step S30, the feature set includes: emotion data set, micro-expression data set, vehicle motion data set, emotion data set + micro-expression data set, vehicle motion data set + emotion data set, vehicle motion data set + micro-expression data set, vehicle motion data set + emotion data set + micro-expression data set.

4. The method for predicting dangerous driving behavior according to claim 1, characterized in that: In step S40, changing the time window includes: The time window is divided into more detailed parts, and the starting point of the time window is slid forward from the moment when the driving behavior occurs, with a sliding step of Δt. Each starting point is modeled using a variety of time window lengths using the optimal prediction model. A grid search strategy is used to determine the time window and feature set combination with the best prediction effect based on the F1 value of the dangerous driving behavior model.

Images