Method and system for evaluating interactive distraction and driving risk of central control screen in natural driving state
By collecting and analyzing driver's behavioral data, a distraction and driving risk identification model is constructed, and using the InceptionTime network model and the double-headed attention mechanism, the problem of existing systems being unable to evaluate the distraction and driving risks during driver interactions is solved. Accurate risk assessment during driver interaction and improve driving safety.
Patent Information
- Application Number
- CN202411044129.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-03
AI Technical Summary
The existing driver monitoring system cannot effectively evaluate the driver's distraction and related driving risks when interacting with the on-board central control screen.
By collecting driver's visual, touch and manipulating behavior data, a distraction and driving risk identification model is constructed, and the improved InceptionTime network model and double-headed attention mechanism are used to extract feature indicators and perform factor analysis and clustering to identify drivers' Distraction level and driving risk level.
Accurate identification and evaluation of drivers' distractions and driving risks when interacting with the central control screen, improving driving safety.
Smart Images

Figure CN119058710A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of driving analysis, and in particular to a method and system for evaluating central control screen interactive distraction and driving risk in a natural driving state. Background Art
[0002] With the accelerated popularization of new energy vehicles, the degree of automobile intelligence and networking has gradually increased. Cars have transformed from simple means of transportation to multi-functional intelligent terminals. In this process, traditional cockpits have evolved into intelligent cockpits, and the functions of vehicle information systems (IVIS) have become increasingly powerful. The increase in the number and size of screens has become a new trend. More and more secondary tasks need to be completed through the full-touch human-computer interaction interface of the vehicle information system, which integrates a large amount of information and vehicle control parameters. This smart screen requires the driver to pay more attention to interact, which may cause driving distraction.
[0003] In order to reduce the safety risks caused by drivers, the driver monitoring system (DMS) plays a key auxiliary role as a core component of the intelligent connected car interaction system. Most mature DMS applications on the market mainly evaluate the driver's status by analyzing vehicle data, while the new generation of DMS based on vision technology has become an indispensable element of SAE Level 2 to 4 autonomous driving technology. These vision-based systems usually focus on monitoring the driver's fatigue level and behavior patterns, but cannot effectively evaluate the driver's distraction and related driving risks when interacting with the in-vehicle central control screen.
[0004] Therefore, in order to solve the above problems, a method and system for assessing distraction and driving risk of central control screen interaction under natural driving conditions is needed, which can accurately identify and assess the degree of distraction and driving risk of the driver when interacting with the central control screen. Summary of the invention
[0005] In view of this, the purpose of the present invention is to overcome the defects in the prior art and provide a method and system for evaluating distraction and driving risks caused by interaction with the central control screen under natural driving conditions, which can accurately identify and evaluate the degree of distraction and driving risks of the driver when interacting with the central control screen.
[0006] The method for evaluating central control screen interactive distraction and driving risk in a natural driving state of the present invention comprises the following steps:
[0007] Collecting behavioral data of the driver when performing interactive tasks in different environments; the behavioral data includes visual behavior data, touch behavior data, and manipulation behavior data;
[0008] Constructing a distraction and driving risk identification model; using the behavior data and the driving level level as input data to train the distraction and driving risk identification model to obtain a trained identification model; wherein the distraction level label and the driving risk level label are used as training labels;
[0009] The corresponding behavior data and driving level of the driver to be tested within the set time period are input into the trained recognition model, and the distraction level and driving risk level are output.
[0010] Further, the visual behavior data includes a gaze angle vector and a head rotation angle vector;
[0011] The touch behavior data includes the movement trajectory of the finger on the screen and the screen clicks of the finger;
[0012] The manipulation behavior data includes steering wheel angle, acceleration and brake pedal opening, longitudinal vehicle speed, lateral position and lateral speed.
[0013] Furthermore, the distraction level label and driving risk level label are constructed according to the following method:
[0014] Extracting features from the behavior data to obtain feature indicator data;
[0015] Performing difference analysis on the characteristic index data, and taking the characteristic index data with significant difference as the first index data;
[0016] Performing a correlation test on the first indicator data, and selecting second indicator data suitable for factor analysis from the first indicator data;
[0017] Performing factor analysis on the second indicator data to obtain a first factor related to the distraction level and a second factor related to the driving risk level; wherein the first factor includes a blinking factor, a sight factor, and a touch factor; and the second factor includes a braking fluctuation factor, a lateral fluctuation factor, and an acceleration fluctuation factor;
[0018] Clustering is performed on several factors in the first factor, and the distraction levels are divided into 1 to k levels from low to high; clustering is performed on several factors in the second factor, and the driving risk levels are divided into 1 to k levels from low to high.
[0019] Further, the characteristic index data includes eye movement index, finger trajectory index and driving control index;
[0020] The eye movement indicators include the road gaze duration, the central control screen gaze duration, and the blink duration;
[0021] The finger trajectory indicators include finger movement distance, finger touch times and finger movement duration ratio;
[0022] The driving control indexes include steering wheel turning rate, brake and accelerator pedal opening, lateral position standard deviation, lane deviation time distance, collision time distance, longitudinal speed and longitudinal acceleration.
[0023] Furthermore, the distraction and driving risk identification model includes an improved InceptionTime network model; the improved InceptionTime network model includes 5 parallel Inception networks and a dual-head attention mechanism;
[0024] The Inception network includes two residual blocks, each of which contains three Inception modules. The input of each residual block is added to the input of the next block through a residual connection, which alleviates the gradient vanishing by allowing the gradient to flow directly. After the residual block, the Inception network uses a global average pooling layer, which averages the output multivariate time series over the entire time dimension. Finally, the Inception network uses a fully connected Softmax layer as a classifier, and the number of neurons in this layer is equal to the number of categories in the data set for classification prediction.
[0025] The dual-headed attention mechanism is used to automatically identify and assign higher weights to features that are more critical to the prediction task during the feature extraction stage of the recognition model.
[0026] Further, the visual behavior data is obtained according to the following method:
[0027] The driver's sight line image data is collected in real time, and facial feature point detection is performed on the sight line image to obtain the feature point positions of the eyes, and then the closure ratio of both eyes is calculated; the facial image is then cropped and standardized, and then passed to the sight line trajectory recognition model, through which the gaze angle vector and the head rotation angle vector are calculated; wherein, the sight line trajectory recognition model adopts the ResNet-18 network model.
[0028] Further, the driving level level is determined according to the following method:
[0029] The average driving risk level of the subjects is obtained by clustering the driving risk levels of each subject, and then the average driving risk level of the subjects is clustered in one dimension to finally obtain the driving level level: the driving level level is divided into 1 to k levels from low to high.
[0030] A central control screen interactive distraction and driving risk assessment system under natural driving conditions, comprising a data acquisition unit, a model training unit and a risk assessment unit;
[0031] The data collection unit is used to collect the behavior data of the driver when performing interactive tasks in different environments; the behavior data includes visual behavior data, touch behavior data and manipulation behavior data;
[0032] The model training unit is used to construct a distraction and driving risk identification model; the behavior data and the driving level level are used as input data to train the distraction and driving risk identification model to obtain a trained identification model; wherein the distraction level label and the driving risk level label are used as training labels;
[0033] The risk assessment unit is used to input the behavior data and driving level level corresponding to the driver to be tested within a set time period into the trained recognition model, and output the distraction level and driving risk level.
[0034] Furthermore, the distraction and driving risk identification model includes an improved InceptionTime network model; the improved InceptionTime network model includes 5 parallel Inception networks and a dual-head attention mechanism;
[0035] The Inception network includes two residual blocks, each of which contains three Inception modules. The input of each residual block is added to the input of the next block through a residual connection, which alleviates the gradient vanishing by allowing the gradient to flow directly. After the residual block, the Inception network uses a global average pooling layer, which averages the output multivariate time series over the entire time dimension. Finally, the Inception network uses a fully connected Softmax layer as a classifier, and the number of neurons in this layer is equal to the number of categories in the data set for classification prediction.
[0036] The dual-headed attention mechanism is used to automatically identify and assign higher weights to features that are more critical to the prediction task during the feature extraction stage of the recognition model.
[0037] The beneficial effects of the present invention are as follows: a method and system for evaluating distraction and driving risk of interaction with a central control screen in a natural driving state disclosed in the present invention collects visual, touch and manipulation behavior data of the driver when performing interactive tasks in different environments, extracts multiple driving indicators, and determines the level of distraction and driving risk of the driver when performing interactive tasks through feature screening, factor analysis and clustering methods, and evaluates the driver's driving level according to the driving risk level; combining original data such as line of sight trajectory, finger trajectory, vehicle driving and steering wheel angle, and driving level information, the distraction level and driving risk level determined by clustering are used as labels for model training, and a distraction and driving risk identification model is constructed. The distraction and driving risk identification model can be used to accurately identify and evaluate the degree of distraction and driving risk of the driver when interacting with the central control screen. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:
[0039] Figure 1 It is a schematic diagram of the distraction and driving risk assessment principle of the present invention;
[0040] Figure 2 It is a driving scenario-interaction subtask-subjective scale relationship diagram of the present invention;
[0041] Figure 3 It is a schematic diagram of the processing flow of the driver's facial image of the present invention;
[0042] Figure 4 This is a schematic diagram of collecting finger touch behavior data of the present invention;
[0043] Figure 5 It is a schematic diagram of the manipulation behavior data collection method of the present invention;
[0044] Figure 6 This is an example diagram of the interactive distraction level clustering result of the present invention;
[0045] Figure 7 This is an example diagram of the clustering results of driving risk levels of the present invention;
[0046] Figure 8 (a) is a schematic diagram of the driving level clustering result of the present invention;
[0047] Figure 8 (b) is a schematic diagram of the probability distribution of driving level grades of the present invention;
[0048] Fig. 9 Schematic diagram of the Inception model for time series classification of the present invention;
[0049] Fig.10It is a schematic diagram of the structure of the distraction and driving risk identification model of the present invention;
[0050] Fig.11 (a) is a schematic diagram of the confusion matrix corresponding to the distraction level of the present invention;
[0051] Fig.11 (b) is a schematic diagram of the confusion matrix corresponding to the driving risk level of the present invention. DETAILED DESCRIPTION
[0052] The present invention is further described below in conjunction with the accompanying drawings, as shown in the drawings:
[0053] This embodiment discloses a method for assessing distraction and driving risk of central control screen interaction in a natural driving state, including the following steps:
[0054] Collecting behavioral data of the driver when performing interactive tasks in different environments; the behavioral data includes visual behavior data, touch behavior data, and manipulation behavior data;
[0055] Constructing a distraction and driving risk identification model; using the behavior data and the driving level level as input data to train the distraction and driving risk identification model to obtain a trained identification model; wherein the distraction level label and the driving risk level label are used as training labels;
[0056] The corresponding behavior data and driving level of the driver to be tested within the set time period are input into the trained recognition model, and the distraction level and driving risk level are output.
[0057] In this embodiment, in order to obtain data under various driving conditions, the present invention designs three typical road following conditions and four typical navigation interaction tasks, such as Figure 2 As shown, in each driving scenario and interactive task, the driver is required to make a comprehensive evaluation of six driving performance indicators.
[0058] Driving scenario: This driving scenario is set with 40km / h, 80km / h, and 120km / h as the base speeds, which correspond to the speed limits of urban roads (secondary highways to fourth-level highways), expressways (first-level highways), and highways respectively.
[0059] Interactive sub-tasks: In order to simulate the common central control screen interaction scenarios when the driver is driving naturally, the present invention designs four typical sub-tasks based on the AutoNavi Map App, covering multiple modalities such as vision, hearing and touch, and has the ability to distinguish complexity.
[0060] Subjective scale: In order to obtain the driver's subjective evaluation of the sub-task load of the in-vehicle information system interaction, this scale is based on the NASA-TLX scale. Combined with the actual situation of the driving sub-task, "physical demand" is changed to "visual occupation degree" and "task performance" is changed to "interaction performance". Each indicator is divided into 7 levels from low to high (as shown in Table 1). After each interactive sub-task is completed, the subjects are required to stop the car temporarily to complete the scale survey.
[0061] Table 1
[0062]
[0063] In this embodiment, during driving, various behaviors such as vision, touch, and manipulation reflect the possible distraction of the driver. Therefore, in order to effectively identify whether the driver is distracted, these objective behavior data must be collected to prepare data for subsequent indicator extraction.
[0064] In order to collect the driver's visual behavior data, a camera is first installed to capture the driver's line of sight in real time. The appropriate camera is selected according to the changes in ambient light: an RGB camera is used when there is sufficient light, and an infrared (IR) camera is used when there is insufficient light. The captured images are processed according to the following process (such as Figure 3 As shown in the figure): First, the input image is detected by facial feature points to obtain the eye feature point positions, and then the eye closure ratio (EAR) is calculated. Next, the facial image is cropped and normalized, and then passed to the gaze trajectory recognition model, which calculates the gaze angle vector and the head rotation angle vector.
[0065] Among them, the gaze trajectory recognition model adopts the ResNet-18 architecture trained based on the ETH-XGaze dataset (the gaze angle vector and head angle vector output are obtained from the facial image input). ResNet introduces residual connections, which allows the network to learn the residual by adding the input directly to the output, avoiding the gradient vanishing problem when training deeper networks. Compared with some deeper network structures, ResNet-18, which is composed of 18 main layers, has fewer parameters, but still maintains good performance and is suitable for deployment in real-time system scenarios. Among them, the ETH-XGaze dataset contains more than one million high-resolution images of different gaze positions under extreme head postures.
[0066] For the processing of touch behavior data, such as Figure 4As shown, it includes image-based finger motion trajectory recognition: first, the screen position needs to be calibrated, and then the hand feature points are detected through the image to obtain the position of the index finger tip. Next, the relative coordinates of the finger tip on the screen are calculated to obtain the finger motion trajectory on the screen. It also includes Android system event reading: by using Android Debug Bridge (ADB), connecting to the tablet computer in a wired or wireless manner, and capturing various system events and messages at a frequency of 60Hz, so as to calculate the finger screen click events in real time.
[0067] Regarding the processing of manipulation behavior data, in the case of distracted driving, the driver's manipulation of the car will be distorted or delayed. In order to accurately reflect the driver's manipulation behavior, the steering wheel angle and the opening change data of the acceleration / brake pedal can be collected. These data can provide important information about the driver's operation method and reaction time. In addition, the driving state of the vehicle itself can also reflect the driver's manipulation behavior. Therefore, by analyzing the car driving data, the driver's behavior pattern and driving state can be further understood.
[0068] according to Figure 5 As shown in the figure, there are many ways to collect manipulation behavior data. The on-board diagnostics system (OBD) can provide information such as steering wheel angle, acceleration / brake pedal opening, and longitudinal vehicle speed, while the gyroscope can provide data such as longitudinal and lateral speed. Since the OBD interface is usually difficult to open to individual users, this information can be obtained through a Bluetooth three-axis acceleration sensor. Similarly, high-precision gyroscopes are expensive, so GPS positioning can be used to obtain vehicle position information to calculate the longitudinal vehicle speed. At the same time, through lane line recognition technology, the position of the vehicle relative to the lane line can be determined, so that the lateral position and lateral speed can be calculated.
[0069] In this embodiment, the distraction level label and the driving risk level label are constructed according to the following method:
[0070] Feature extraction is performed on the behavior data to obtain feature index data; the feature index data includes eye movement index, finger trajectory index and driving control index; the eye movement index includes road gaze duration, central control screen gaze duration and blink duration; the finger trajectory index includes finger movement distance, finger touch times and finger movement duration ratio; the driving control index includes steering wheel turning rate, brake and accelerator pedal opening, lateral position standard deviation, lane deviation time, collision time, longitudinal speed and longitudinal acceleration;
[0071] Standard Deviation of Lateral Position (SDLP)
[0072]
[0073] Among them, SDLP can characterize the fluctuation of the lateral position of the vehicle on the lane. The larger the SDLP, the greater the fluctuation of the lateral position of the vehicle, the worse the driver's ability to maintain the vehicle laterally, and the higher the risk of lateral deviation. i represents the lateral position of the vehicle at time i, x represents the mean of the lateral position of the vehicle during this period, and n represents the number of vehicle position sampling points.
[0074] Time to Line-Crossing (TLC)
[0075]
[0076] TLC refers to the shortest time required for any part of the vehicle to reach the edge of the lane lines on both sides while maintaining the current state and continuing to travel on the current road. This method uses the lateral uniform acceleration method to calculate TLC data. l and r Indicates the distance between the vehicle and the left and right lane lines, and the lateral speed is L V , the lateral acceleration is L A , TLC can be solved according to the above equation.
[0077] Time to Collision (TTC)
[0078]
[0079] TTC refers to the shortest time required for a vehicle to collide with the vehicle in front at the current speed and direction. v1 represents the instantaneous longitudinal speed of the current vehicle, v2 represents the instantaneous longitudinal speed of the vehicle in front, and d represents the distance between the two vehicles after removing the length of the vehicle body.
[0080] Perform a difference analysis on the characteristic indicator data, and use the characteristic indicator data with significant differences as the first indicator data; specifically, the degree of driving distraction and risk level are affected by the complexity of the interactive sub-task and the changes in the driving environment. Therefore, it is necessary to test the differences of various indicators such as line of sight, finger trajectory, driving performance, etc. in different driving environments and different sub-task complexities to determine whether there are significant differences. This problem can be described as a two-way repeated measures analysis of variance (Two-Way Repeated Measures ANOVA) problem, in which the driving environment and sub-task difficulty are regarded as within-group variables, and the indicators are regarded as dependent variables. Assume that Yijk represents the indicator k observed under task difficulty level i and driving environment condition j, α i represents the main effect of task difficulty level i, β j represents the main effect of driving environment condition j, while αβ ij represents the interaction effect between task difficulty level and driving environment conditions. The null hypothesis tested is:
[0081]
[0082] The alternative hypothesis is:
[0083]
[0084] Two-way repeated measures ANOVA requires sample data to meet normality and homogeneity of variance. Due to the small number of experimental samples, the Shapiro-Wilk test was used to detect whether the data deviated from the normal distribution. For indicators that did not meet the conditions of the Shapiro-Wilk test, a graphical method was used to draw a QQ graph (Quantile-Quantile Plot) to visually check the distribution of data points.
[0085] A correlation test is performed on the first indicator data, and the second indicator data suitable for factor analysis is screened out from the first indicator data; specifically, factor analysis is a dimensionality reduction technique that can effectively reduce the number of variables in a data set while retaining most of the information in the original data set. In a multivariate data set, there is often a certain degree of correlation between variables. Highly correlated variables may cause multicollinearity problems, thereby affecting the stability and explanatory power of the model. Factor analysis can effectively reduce this multicollinearity by merging related variables into a few factors. First, the Kaiser-Mayer-Olkin (KMO) measure test and Bartlett's sphericity test are needed to determine whether there is a correlation between variables. The KMO measure test evaluates the correlation and commonality between variables, and its result should be greater than 0.5 to ensure that the data is suitable for factor analysis. The Bartlett's sphericity test tests whether there is a significant correlation between variables. If the p value is less than the significance level (usually set to 0.05), it means that the variables are correlated and suitable for factor analysis.
[0086] The second indicator data was subjected to factor analysis to obtain the first factor related to the distraction level and the second factor related to the driving risk level; specifically, for interactive distraction, it was mainly related to the eye movement index and the finger trajectory index. Three factors were extracted through factor analysis, and the maximum variance method was used to rotate the factors, hiding the coefficients less than 0.5. The rotation component matrix is shown in Table 2.
[0087] Table 2
[0088]
[0089] The indicators corresponding to factor 1 of interactive distraction are the number of blinks and the average blink duration, which is named the blink factor; the indicators corresponding to factor 2 are the screen gaze duration, the number of screen gazes, and the average duration of each screen gaze, which is named the sight factor; the indicators corresponding to factor 3 are the finger movement distance, the number of finger touches, and the finger movement time ratio, which is named the touch factor.
[0090] As for driving risk, it is mainly related to driving control indicators. Three factors were extracted through factor analysis. The maximum variance method was used to rotate the factors and the coefficients less than 0.5 were hidden. The rotation component matrix is shown in Table 3.
[0091] Table 3
[0092]
[0093] The indicators corresponding to factor 1 of driving risk are the standard deviation of lateral acceleration, the mean and standard deviation of brake pedal, and the standard deviation of longitudinal acceleration, which is named brake fluctuation factor; the indicators corresponding to factor 2 are the 2.5-degree steering wheel turning rate, the lateral position standard deviation, and TLC, which is named lateral fluctuation factor; the indicators corresponding to factor 3 are the 0.5-degree steering wheel turning rate, the accelerator pedal mean value and the accelerator pedal standard deviation, which is named acceleration fluctuation factor.
[0094] Clustering is performed on several factors in the first factor, and the distraction level is divided into 1 to k levels from low to high; clustering is performed on several factors in the second factor, and the driving risk level is divided into 1 to k levels from low to high. Specifically, Figure 6 , 7 As shown in the figure, the K-Means clustering method is used to cluster the three factors in the factor analysis of interactive distraction and driving risk. Due to the large number of indicator variables, in order to make the results more distinguishable and match the range of the task load scale, the distraction level is divided into 1 to 7 levels from low to high. Similarly, the distraction level is mapped similarly to divide the risk level from low to high into 1 to 7 levels.
[0095] Among them, the simulated driving experiment task data was divided based on a 3-second time window with a 50% overlap rate, and a total of 6048 valid data were extracted. Distraction and risk level mapping was performed on all time window data to obtain training labels.
[0096] In this embodiment, since the driver's personal driving skills have a significant impact on the change in risk level, the driving level is considered as a key characteristic indicator. In order to objectively evaluate the driving level, a one-dimensional K-Means cluster analysis was performed on the average driving risk level of different test subjects to obtain a purely objective driving level level. These levels are consistent with the 1-7 point range of subjective scores, distraction levels, and risk levels. That is, Figure 8 As shown in FIG. 1 , the average driving risk level of each subject is obtained by clustering the driving risk level of each subject, and then the average driving risk level of all subjects is clustered in one dimension to finally obtain the driving level clustering result.
[0097] In this embodiment, the improved InceptionTime network model includes 5 parallel Inception networks and a dual-head attention mechanism; wherein the prediction structure of each network in the 5 parallel Inception networks is assigned a uniform weight.
[0098] like Fig. 9 As shown in the figure, the Inception network includes two residual blocks, each of which contains three Inception modules; the input of each residual block is added to the input of the next block through a residual connection (shortcut linear connection), which alleviates the gradient disappearance by allowing the gradient to flow directly; after the residual block, the Inception network uses a global average pooling layer (Global Average Pooling, GAP), which averages the output multivariate time series over the entire time dimension to reduce the feature dimension; finally, the Inception network uses a fully connected Softmax layer as a classifier, the number of neurons in this layer is equal to the number of categories in the data set, for classification prediction;
[0099] In order to improve the accuracy of the InceptionTime model when dealing with multi-task output, this paper introduces a dual-headed attention mechanism in the basic model. The dual-headed attention mechanism can automatically identify and assign higher weights to features that are more critical to the prediction task during the feature extraction stage of the model, thereby achieving more accurate and robust predictions.
[0100] The distraction and driving risk identification model based on the improved InceptionTime architecture has better recognition effect than the traditional model structure, and the trained network model can identify the distraction and driving risk levels in real time.
[0101] like Fig.10As shown, by inputting the driver's sight data, touch data, objective manipulation data and driving level label within a certain period of time, the driver's distraction level and driving risk level (both ranging from 1 to 7) during this period can be evaluated simultaneously.
[0102] The present invention also relates to a central control screen interactive distraction and driving risk assessment system under natural driving conditions. The assessment system corresponds to the assessment method of the above embodiment and can be understood as a system for implementing the above assessment method. The system includes a data acquisition unit, a model training unit, and a risk assessment unit.
[0103] The data collection unit is used to collect the behavior data of the driver when performing interactive tasks in different environments; the behavior data includes visual behavior data, touch behavior data and manipulation behavior data;
[0104] The model training unit is used to construct a distraction and driving risk identification model; the behavior data and the driving level level are used as input data to train the distraction and driving risk identification model to obtain a trained identification model; wherein the distraction level label and the driving risk level label are used as training labels;
[0105] The risk assessment unit is used to input the behavior data and driving level level corresponding to the driver to be tested within a set time period into the trained recognition model, and output the distraction level and driving risk level.
[0106] In order to better evaluate the generalization ability of the recognition model of the present invention on a limited data set, the K-fold cross-validation method was used for model training. Before the model training began, the learning rate range test method proposed by Smith was used to find the optimal learning rate. In order to explore the impact of each modality on the prediction accuracy of distraction level and risk level, models with different modal time sequence data fusion input were trained separately, where F face Contains the gaze and head rotation vectors and the eye aspect ratio, F hand Contains the position coordinates of the finger on the central control screen, F input It includes driving control input (steering wheel angle, accelerator and brake pedal opening, speed, acceleration, lane deviation, distance to the vehicle ahead), F drive is the driver's driving level. The fusion data was normalized before training and scaled to between 0 and 1. The training time was 30 Epochs. The training results are shown in Table 4 below. The best indicator results of distraction level and risk level are bolded.
[0107] Table 4
[0108]
[0109] From Table 4, we can see that the prediction effect of distraction level is mainly affected by F face With F hand The prediction effect of risk level is mainly affected by F input With F drive When the model input is F face +F hand +F input When , the distraction level prediction performance is the best, while F face +F hand +F input +F drive When the model is input, the risk level prediction performance is the best. drive The addition of has caused some loss in the performance of distraction recognition, but the comprehensive prediction performance of distraction and risk is the best. At this time, the confusion matrix of the prediction results of the recognition model is as follows: Fig.11 shown.
[0110] The recognition model proposed in this paper not only has much higher performance than traditional time series classification models (GRU, LSTM and CNN), but also further improves on the benchmark of the InceptionTime model. This reflects the unique advantages of the model proposed in this paper in dealing with complex multi-task learning problems, highlights the effectiveness of the dual-headed attention mechanism in enhancing the model's learning ability and improving task-specific performance, and also provides a reference for the design and optimization of multivariate time series classification models.
[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.
Claims
1. A method for assessing distraction and driving risk of central control screen interaction in a natural driving state, characterized by: include: Collecting behavioral data of drivers when performing interactive tasks in different environments; The behavior data includes visual behavior data, touch behavior data and manipulation behavior data; Constructing a distraction and driving risk identification model; using the behavior data and the driving level level as input data to train the distraction and driving risk identification model to obtain a trained identification model; wherein the distraction level label and the driving risk level label are used as training labels; The corresponding behavior data and driving level of the driver to be tested within the set time period are input into the trained recognition model, and the distraction level and driving risk level are output.
2. The method for assessing central control screen interactive distraction and driving risk under natural driving conditions according to claim 1, characterized in that: The visual behavior data includes a gaze angle vector and a head rotation angle vector; The touch behavior data includes the movement trajectory of the finger on the screen and the screen clicks of the finger; The manipulation behavior data includes steering wheel angle, acceleration and brake pedal opening, longitudinal vehicle speed, lateral position and lateral speed.
3. The method for assessing central control screen interactive distraction and driving risk in a natural driving state according to claim 1, characterized in that: Construct distraction level labels and driving risk level labels according to the following method: Extracting features from the behavior data to obtain feature indicator data; Performing difference analysis on the characteristic index data, and taking the characteristic index data with significant difference as the first index data; Performing a correlation test on the first indicator data, and selecting second indicator data suitable for factor analysis from the first indicator data; Performing factor analysis on the second indicator data to obtain a first factor related to the distraction level and a second factor related to the driving risk level; wherein the first factor includes a blinking factor, a sight factor, and a touch factor; and the second factor includes a braking fluctuation factor, a lateral fluctuation factor, and an acceleration fluctuation factor; Clustering is performed on several factors in the first factor, and the distraction levels are divided into 1 to k levels from low to high; clustering is performed on several factors in the second factor, and the driving risk levels are divided into 1 to k levels from low to high.
4. The method for assessing central control screen interactive distraction and driving risk in a natural driving state according to claim 3, characterized in that: The characteristic index data include eye movement index, finger trajectory index and driving control index; The eye movement indicators include the road gaze duration, the central control screen gaze duration, and the blink duration; The finger trajectory indicators include finger movement distance, finger touch times and finger movement duration ratio; The driving control indexes include steering wheel turning rate, brake and accelerator pedal opening, lateral position standard deviation, lane deviation time distance, collision time distance, longitudinal speed and longitudinal acceleration.
5. The method for assessing central control screen interactive distraction and driving risk in a natural driving state according to claim 1, characterized in that: The distraction and driving risk identification model includes an improved InceptionTime network model; the improved InceptionTime network model includes 5 parallel Inception networks and a dual-head attention mechanism; The Inception network includes two residual blocks, each of which contains three Inception modules. The input of each residual block is added to the input of the next block through a residual connection, which alleviates the gradient vanishing by allowing the gradient to flow directly. After the residual block, the Inception network uses a global average pooling layer, which averages the output multivariate time series over the entire time dimension. Finally, the Inception network uses a fully connected Softmax layer as a classifier, and the number of neurons in this layer is equal to the number of categories in the data set for classification prediction. The dual-headed attention mechanism is used to automatically identify and assign higher weights to features that are more critical to the prediction task during the feature extraction stage of the recognition model.
6. The method for assessing central control screen interactive distraction and driving risk in a natural driving state according to claim 1, characterized in that: Obtain visual behavior data as follows: The driver's sight line image data is collected in real time, and facial feature point detection is performed on the sight line image to obtain the feature point positions of the eyes, and then the closure ratio of both eyes is calculated; the facial image is then cropped and standardized, and then passed to the sight line trajectory recognition model, through which the gaze angle vector and the head rotation angle vector are calculated; wherein, the sight line trajectory recognition model adopts the ResNet-18 network model.
7. The method for assessing central control screen interactive distraction and driving risk in a natural driving state according to claim 3, characterized in that: The driving proficiency level is determined as follows: The average driving risk level of the subjects is obtained by clustering the driving risk levels of each subject, and then the average driving risk level of the subjects is clustered in one dimension to finally obtain the driving level level: the driving level level is divided into 1 to k levels from low to high.
8. A central control screen interactive distraction and driving risk assessment system under natural driving conditions, characterized by: It includes data collection unit, model training unit and risk assessment unit; The data collection unit is used to collect the behavior data of the driver when performing interactive tasks in different environments; the behavior data includes visual behavior data, touch behavior data and manipulation behavior data; The model training unit is used to construct a distraction and driving risk identification model; the behavior data and the driving level level are used as input data to train the distraction and driving risk identification model to obtain a trained identification model; wherein the distraction level label and the driving risk level label are used as training labels; The risk assessment unit is used to input the behavior data and driving level level corresponding to the driver to be tested within a set time period into the trained recognition model, and output the distraction level and driving risk level.
9. The central control screen interactive distraction and driving risk assessment system under natural driving state according to claim 8, characterized in that: The distraction and driving risk identification model includes an improved InceptionTime network model; the improved InceptionTime network model includes 5 parallel Inception networks and a dual-head attention mechanism; The Inception network includes two residual blocks, each of which contains three Inception modules. The input of each residual block is added to the input of the next block through a residual connection, which alleviates the gradient vanishing by allowing the gradient to flow directly. After the residual block, the Inception network uses a global average pooling layer, which averages the output multivariate time series over the entire time dimension. Finally, the Inception network uses a fully connected Softmax layer as a classifier, and the number of neurons in this layer is equal to the number of categories in the data set for classification prediction. The dual-headed attention mechanism is used to automatically identify and assign higher weights to features that are more critical to the prediction task during the feature extraction stage of the recognition model.
