Driver takeover performance quantification method, system and device in auxiliary driving environment and medium

Abstract

The invention discloses a driver takeover performance quantification method, system and device in an auxiliary driving environment and a medium, and relates to the field of auxiliary driving data processing, and the method comprises the steps: obtaining the multi-modal physiological signal data of a driver in real time; constructing a feature vector based on the multi-modal physiological signal data; constructing an index prediction model; inputting the feature vector into an index prediction model to obtain a comprehensive index prediction value; determining a security level based on the comprehensive index predicted value; and generating driver takeover suggestions based on the safety level. According to the method, the state of the driver can be comprehensively monitored, evaluation and quantification of the state of the driver can be accurately realized in real time, a basis is provided for a personalized takeover strategy, and the driving safety is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of assisted driving data processing, and in particular to a method, system, device and medium for quantifying driver takeover performance in an assisted driving environment. Background Art

[0002] Traditional driver takeover methods typically rely on data from a single sensor. This data can be subject to errors or noise, and cannot fully and accurately reflect the driving environment and the driver's status. For example, cameras can be affected by external conditions such as lighting and weather, while radar can be interfered with by obstacles. Furthermore, relying on data from a single sensor prevents the acquisition of multi-angle and multi-dimensional information, and data processing capabilities are also limited.

[0003] To address the above issues, known technologies employ deep learning, big data analysis, and other techniques to process and analyze sensor data in real time to improve data accuracy and reliability. However, due to the complexity and variability of the driving environment, the data from a single sensor is difficult to fully reflect environmental changes, making it impossible to comprehensively monitor the driver's status, and thus unable to accurately and quantify the driver's status in real time. Summary of the Invention

[0004] The purpose of this application is to provide a method, system, equipment and medium for quantifying driver takeover performance in an assisted driving environment, which can comprehensively monitor the driver's status, and realize the evaluation and quantification of the driver's status in real time and accurately, thereby providing a basis for personalized takeover strategies to significantly improve driving safety.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for quantifying driver takeover performance in an assisted driving environment, comprising:

[0007] Acquire multimodal physiological signal data of the driver in real time;

[0008] constructing a feature vector based on the multimodal physiological signal data;

[0009] Construct indicator prediction models;

[0010] Inputting the characteristic vector into the indicator prediction model to obtain a comprehensive indicator prediction value;

[0011] Determining a safety level based on the predicted value of the comprehensive indicator;

[0012] A driver takeover recommendation is generated based on the safety level.

[0013] Optionally, the multimodal physiological signal data includes physiological data and eye movement data; and constructing a feature vector based on the multimodal physiological signal data includes:

[0014] performing filtering processing on the physiological data to obtain filtered physiological data;

[0015] Obtaining time domain features and frequency domain features based on filtered physiological data;

[0016] performing sliding window normalization processing on the eye movement data to obtain standard eye movement data;

[0017] The feature vector is constructed based on time domain features, frequency domain features and standard eye movement data.

[0018] Optionally, the time domain features include a MeanHR value of heart rate variability and a SDNN value of heart rate variability; the frequency domain features include an LF norm value of the frequency domain features of heart rate variability and a HF norm value of the frequency domain features of heart rate variability;

[0019] Standard eye movement data include: horizontal dispersion distance between fixations, average pupil diameter, blink frequency and average number of saccades.

[0020] Optionally, filtering the physiological data to obtain filtered physiological data includes:

[0021] The physiological data is filtered using a Butterworth filter to obtain filtered physiological data.

[0022] Optionally, construct an indicator prediction model, including:

[0023] Obtaining raw data; the raw data includes: multimodal historical physiological signal data of the driver, and driver reaction data and vehicle status historical data corresponding to the multimodal historical physiological signal data; the multimodal historical physiological signal data of the driver includes historical physiological data and historical eye movement data; the vehicle status historical data includes: maximum historical lateral offset, minimum historical collision time, and maximum historical longitudinal deceleration;

[0024] Preprocessing the raw data, and constructing a historical feature vector based on the preprocessed multimodal historical physiological signal data;

[0025] Using historical feature vectors as input samples and preprocessed driver response data and vehicle status history data corresponding to multimodal historical physiological signal data as output samples, a sample data set is obtained;

[0026] Dividing the sample data set into a plurality of mutually exclusive subsets;

[0027] The XGBoost algorithm is used to build a multi-objective regression prediction model;

[0028] One of the subsets was selected as the validation set, and the remaining subsets were used as the training set. The multi-objective regression prediction model was trained and verified using the ten-fold cross-validation method to obtain a multi-objective regression prediction model that met the prediction requirements.

[0029] A multi-objective regression prediction model that meets the prediction requirements is used as the indicator prediction model.

[0030] Optionally, in the process of acquiring the original data, the vehicle control signal in the vehicle status historical data is used as the reference time axis, and a dynamic time warping algorithm is used to align the historical physiological data and the historical eye movement data in time sequence.

[0031] Optionally, determining the safety level based on the predicted value of the comprehensive indicator includes:

[0032] Takeover time threshold, lateral offset threshold, collision time threshold and longitudinal deceleration threshold;

[0033] When the driver's reaction time, maximum lateral offset, minimum collision time and maximum longitudinal deceleration all meet the set conditions, the safety level is determined to be level 1;

[0034] When any one or two of the driver's takeover reaction time, maximum lateral offset, minimum collision time, and maximum longitudinal deceleration do not meet the set conditions, the safety level is determined to be the second level;

[0035] When any three or all of the driver's takeover reaction time, maximum lateral offset, minimum collision time and maximum longitudinal deceleration do not meet the set conditions, the safety level is determined to be the third level.

[0036] In a second aspect, the present application provides a system for quantifying driver takeover performance in an assisted driving environment, including:

[0037] Data acquisition module, used to obtain multimodal physiological signal data of the driver in real time;

[0038] A data processing module, configured to construct a feature vector based on the multimodal physiological signal data;

[0039] A model prediction module is used to construct an indicator prediction model and input the feature vector into the indicator prediction model to obtain a comprehensive indicator prediction value;

[0040] A feedback execution module is used to determine the safety level based on the predicted value of the comprehensive indicator, generate a driver takeover suggestion based on the safety level, and trigger a graded warning.

[0041] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for quantifying driver takeover performance in an assisted driving environment provided above.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for quantifying the driver takeover performance in the assisted driving environment provided above.

[0043] According to the specific embodiments provided in this application, this application has the following technical effects:

[0044] The present application provides a method, system, device and medium for quantifying the driver's takeover performance in an assisted driving environment. By acquiring the driver's multimodal physiological signal data in real time, the driver's status can be comprehensively monitored, which can solve the problem of insufficient accuracy of relying on a single sensor data. In addition, by constructing a feature vector based on the driver's multimodal physiological signal data acquired in real time, as the input of the indicator prediction model, a comprehensive indicator prediction value is obtained to determine the safety level, which can realize the real-time and accurate evaluation and quantification of the driver's status. Generating driver takeover recommendations based on the safety level can provide a basis for personalized takeover strategies, thereby significantly improving driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 A flowchart of a method for quantifying driver takeover performance in an assisted driving environment provided by one embodiment of the present application;

[0047] Figure 2 A schematic diagram of an implementation architecture of a method for quantifying driver takeover performance in an assisted driving environment provided by one embodiment of the present application;

[0048] Figure 3 A schematic diagram of the structure of a system for quantifying driver takeover performance in an assisted driving environment provided by another embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0052] In an exemplary embodiment, the present application provides a method for quantifying driver takeover performance in an assisted driving environment. The method is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, the method is applied to a server as an example for explanation. Figure 1 As shown, the method includes:

[0053] Step 100: Acquire multimodal physiological signal data of the driver in real time. The multimodal physiological signal data includes physiological data and eye movement data. Physiological data refers to physiological electrical signals, including heart rate.

[0054] In actual applications, photoplethysmography (PPG) and electrocardiogram (ECG) sensors can be used to wirelessly collect the driver's physiological electrical signals during the driver's takeover activities. Eye movement data can also be tracked and collected using an eye tracker.

[0055] Step 101: Construct a feature vector based on multimodal physiological signal data.

[0056] Step 102: Build an indicator prediction model.

[0057] Step 103: Input the feature vector into the indicator prediction model to obtain a comprehensive indicator prediction value. The comprehensive indicator prediction value is expressed as: Y = [y1, y2, y3, y4], where y1 is the takeover reaction time, y2 is the maximum lateral offset, y3 is TTCmin, or the minimum time to collision, and y4 is the maximum longitudinal deceleration.

[0058] Step 104: Determine the safety level based on the predicted value of the comprehensive indicator.

[0059] Step 105: Generate a driver takeover suggestion based on the safety level.

[0060] By implementing the above steps 100 to 105, the present application can comprehensively monitor the driver's status, solve the problem of insufficient accuracy of relying on a single sensor data, and can accurately realize the evaluation and quantification of the driver's status in real time, providing a basis for personalized takeover strategies, thereby significantly improving driving safety.

[0061] In another exemplary embodiment of the present application, the implementation process of the above step 101 may include:

[0062] Step 11: Filter the physiological data to obtain filtered physiological data. For example, a Butterworth filter is used to filter the physiological data to remove noise from the physiological signal.

[0063] Step 12: Obtain time domain features and frequency domain features based on the filtered physiological data. The time domain features include the MeanHR value and the SDNN value of heart rate variability (HRV). The frequency domain features include the LF norm value and the HF norm value of the heart rate variability frequency domain features.

[0064] Among them, (1) MeanHR value of heart rate variability (HRV) is the average heart rate value, which is used to measure the overall frequency of heart activity. Its formula is:

[0065]

[0066] Where, MeanHR refers to the MeanHR value of heart rate variability (HRV), HR k Refers to the number of heartbeats per minute (bpm), which is a basic indicator for measuring heart activity. N is the total number of RR intervals, that is, the total number of normal heartbeats in a specified time period. k is an integer from 1 to N, representing the sequence number of the RR interval. k is the kth RR interval, which refers to the distance between two R waves.

[0067] (2) The SDNN value of heart rate variability (HRV) is the standard deviation of the normal RR interval. It is used to evaluate the overall change and dispersion of HRV and is a sensitive indicator for evaluating the activity of the autonomic nervous system. The formula is:

[0068]

[0069] Where SDNN is the SDNN value of heart rate variability (HRV), It is the average of all RR intervals in a specified time period.

[0070] (3) The LF norm value of the heart rate variability frequency domain feature represents the normalized low-frequency power, and the formula is:

[0071]

[0072] Where LF norm is the LF norm value of the heart rate variability frequency domain feature, and TP is the total power in the range of 0-0.4 Hz, reflecting the overall variability of the signal: TP = ULF + VLF + LF + HF. ULF is the ultra-low frequency band (0-0.0033 Hz). VLF is the very low frequency band (0.0033-0.04 Hz). LF is the low frequency band (0.04-0.15 Hz). HF is the high frequency band (0.15-0.4 Hz).

[0073] (4) The HF norm value of the heart rate variability frequency domain feature represents the normalized high-frequency power, and the formula is:

[0074]

[0075] Where HF norm is the HF norm value of the heart rate variability frequency domain feature.

[0076] Step 13: Standardize the eye movement data using a sliding window (e.g., a window length of 3 seconds and a step size of 0.01 seconds) to obtain standardized eye movement data. Standardized eye movement data includes: horizontal dispersion distance between fixations, average pupil diameter, blink frequency, and average number of saccades.

[0077] Step 14: Construct a feature vector based on the time domain features, frequency domain features, and standard eye movement data. The feature vector is represented as F:

[0078] F=[f1, f2, f3, f4, f5, f6, f7, f8].

[0079] Where f1 is the MeanHR value of heart rate variability (HRV), f2 is the SDNN value of heart rate variability (HRV), f3 is the LF norm value of the frequency domain feature of heart rate variability, f4 is the HF norm value of the frequency domain feature of heart rate variability, f5 is the horizontal dispersion distance between gaze points, f6 is the average pupil diameter, f7 is the blink frequency, and f8 is the average number of saccades.

[0080] In another exemplary embodiment of the present application, in order to obtain the optimal prediction result, in this embodiment, the implementation process of the above step 102 may include:

[0081] Step 21. Continuously collect raw data within a preset time window before the assisted driving takeover request is triggered. The raw data includes: the driver's multimodal historical physiological signal data, and the driver's reaction data and vehicle status historical data corresponding to the multimodal historical physiological signal data. The driver's multimodal historical physiological signal data includes historical physiological data and historical eye movement data. The driver's reaction data includes: historical takeover reaction time, which represents the time interval from the moment the takeover request is triggered to the fastest operation (accelerator, brake, steering wheel) performed by the driver. The vehicle status historical data includes: the maximum historical lateral offset, the minimum historical collision time, and the maximum historical longitudinal deceleration.

[0082] The maximum historical lateral offset represents the maximum lateral position deviation of the vehicle from the moment of the takeover request after the driver issues the takeover request, indicating the stability of the lane change maneuver after the driver takes over control. The minimum historical collision time refers to the shortest time before the vehicle collides with the vehicle or obstacle ahead. It is a surrogate measure of safety and controllability. If a collision occurs, the minimum collision time is zero. The maximum historical longitudinal deceleration represents the maximum longitudinal deceleration of the vehicle after the driver takes over control and is often used to evaluate the longitudinal stability of the takeover.

[0083] In actual application, the driver's multimodal historical physiological signal data can be collected in the same manner as described above. The driver's reaction data and vehicle status historical data can be obtained through vehicle sensors.

[0084] Furthermore, in the process of obtaining the original data, the vehicle control signal in the vehicle status historical data can be used as the reference time axis, and the dynamic time warping (DTW) algorithm can be used to align the historical physiological data and historical eye movement data in time series.

[0085] Step 22: preprocess the original data and construct a historical feature vector based on the preprocessed multimodal historical physiological signal data.

[0086] Step 23: Using the historical feature vector as the input sample and the pre-processed driver reaction data and vehicle status history data corresponding to the multimodal historical physiological signal data as the output sample, a sample data set (corresponding to Figure 2 feature dataset in ).

[0087] Step 24: Divide the sample data set into multiple mutually exclusive subsets. For example, randomly divide the sample data set into 10 mutually exclusive subsets (D1 to D 10 ), maintaining the consistency of data distribution among each subset.

[0088] Step 25: Use the XGBoost algorithm to build a multi-objective regression prediction model.

[0089] Step 26: Select one of the subsets as the validation set (corresponding to Figure 2 The remaining subsets are used as training sets (corresponding to Figure 2 ), the multi-objective regression prediction model is trained and verified using the ten-fold cross-validation method to obtain a multi-objective regression prediction model that meets the prediction requirements. For example, the training and verification process is iterated 10 times, and each time one subset is selected as the validation set, and the remaining 9 subsets are combined into the training set. The multi-objective regression prediction model is fitted on each training set, and the mean of the target indicator is calculated on the validation set. The 10 validation results are aggregated to obtain a robust estimate of the performance of the multi-objective regression prediction model. The multi-objective regression prediction model is retrained using all the training data, and the parameters of the multi-objective regression prediction model are finally determined by retraining with all the training data.

[0090] Step 27: Use the multi-objective regression prediction model that meets the prediction requirements as the indicator prediction model.

[0091] Based on the above description, the loss function of the multi-objective regression prediction model provided in this application is expressed in the form of weighted mean square error, which is:

[0092]

[0093] Where, is the loss function, y i is the true value of the i-th target, is the predicted value of the i-th target, w i All are weights, w1:w2:w3:w4=0.4:0.2:0.2:0.2.

[0094] Furthermore, in order to explain the effect of the multi-objective regression prediction model, in this embodiment, the SHAP value can be used to calculate the contribution of each feature to the prediction result, which is:

[0095]

[0096] In the formula, F is the feature set, S is the subset of the feature set, f is the model prediction function, φ i is the SHAP value of feature i.

[0097] The SHAP value analysis results are used to quantify the contribution of each eigenvector to the prediction results and calculate its importance value to achieve the effect of explaining the multi-objective regression prediction model.

[0098] Furthermore, the training of the multi-objective regression prediction model can adopt a transfer learning strategy, and the pre-training data can be a historical driving dataset from multiple brands of vehicles, including the driver's takeover reaction time, maximum lateral offset, TTCmin, maximum longitudinal deceleration, etc.

[0099] In another exemplary embodiment of the present application, the implementation process of step 104 includes:

[0100] Step 41: When the driver's takeover reaction time, maximum lateral offset, minimum collision time, and maximum longitudinal deceleration all meet the set conditions, the safety level is determined to be Level 1. For example, if y1 ≤ t1, y2 ≤ t2, y3 ≥ t3, and y4 ≤ t4, the safety level is determined to be "Good." Here, y1 is the takeover reaction time, y2 is the maximum lateral offset, y3 is TTCmin, and y4 is the maximum longitudinal deceleration. t1 is the takeover reaction time threshold, t2 is the maximum lateral offset threshold, t3 is the TTCmin threshold, and t4 is the maximum longitudinal deceleration threshold.

[0101] Step 42: If any one or two of the driver's takeover reaction time, maximum lateral offset, minimum collision time, and maximum longitudinal deceleration do not meet the set conditions, the safety level is determined to be Level 2. For example, if any two of the conditions are not met, the safety level is determined to be "Fair."

[0102] Step 43: If any three or all of the driver's takeover reaction time, maximum lateral offset, minimum collision time, and maximum longitudinal deceleration do not meet the set conditions, the safety level is determined to be Level 3. For example, if three or more of the conditions are not met, the safety level is determined to be "inadequate."

[0103] In another exemplary embodiment of the present application, the safety level can be fed back in real time through the vehicle-mounted human-computer interaction interface, and the driver can be informed of the current safety level and the driver's takeover suggestions.

[0104] Based on the above description, the specific implementation process of the driver takeover performance quantification method provided in the assisted driving environment may include: real-time collection of the driver's physiological data (PPG, ECG), eye movement data (horizontal line of sight distance, average pupil diameter, blink frequency, average number of glances) and driving status data within 30 seconds before the automatic driving takeover request is issued, and constructing a time series feature set through preprocessing and feature extraction. The XGBoost algorithm is used to establish a multi-objective regression prediction model (corresponding to Figure 2 The XGBoost model in the above example is used to predict the comprehensive indicators after takeover (takeover reaction time, lateral offset, minimum collision time TTCmin, maximum deceleration), and the SHAP value method is used to perform interpretable analysis on the feature contribution. The prediction results are mapped into three levels: "good", "general", and "lack", and three new safety reminder methods are generated. Based on this, in actual application, the implementation architecture of the method provided by this application is as follows: Figure 2 shown.

[0105] In summary, this application solves the problem of insufficient accuracy of traditional reliance on single sensor data by quantifying the dynamic correlation between physiological and behavioral data. It can evaluate the driver's status in real time, provide a basis for personalized takeover strategies, and significantly improve driving safety.

[0106] Based on the same inventive concept, embodiments of the present application also provide a system for quantifying driver takeover performance in an assisted driving environment, which is used to implement the aforementioned method for quantifying driver takeover performance in an assisted driving environment. The solution provided by this system is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more embodiments of the system for quantifying driver takeover performance in an assisted driving environment provided below can be found in the aforementioned limitations on the method for quantifying driver takeover performance in an assisted driving environment, and will not be repeated here.

[0107] In an exemplary embodiment, Figure 3 As shown, a system for quantifying driver takeover performance in an assisted driving environment is provided, including: a data acquisition module 300, a data processing module 301, a model prediction module 302 and a feedback execution module 303.

[0108] The data acquisition module 300 is used to acquire the driver's multimodal physiological signal data in real time.

[0109] The data processing module 301 is used to construct a feature vector based on multimodal physiological signal data.

[0110] The model prediction module 302 is used to construct an indicator prediction model and to input the feature vector into the indicator prediction model to obtain a comprehensive indicator prediction value.

[0111] Feedback execution module 303 is used to determine the safety level based on the predicted values of the comprehensive indicators, generate driver takeover recommendations based on the safety level, and trigger graded warnings. These graded warnings can be triggered through visual, auditory, and tactile means, such as integrated HUD display, voice prompt unit, and seat vibration to transmit takeover signals.

[0112] As an optional implementation, the driver takeover performance quantification system in an assisted driving environment provided in the present application can support communication with a cloud server for online updating and data synchronization of the indicator prediction model.

[0113] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 4As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store quantitative data of the driver's takeover performance in an assisted driving environment. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for quantifying the driver's takeover performance in an assisted driving environment is implemented.

[0114] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0115] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0116] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0117] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0118] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0119] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (RRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0120] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0121] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for quantifying driver takeover performance in an assisted driving environment, characterized in that: include: Acquire multimodal physiological signal data of the driver in real time; constructing a feature vector based on the multimodal physiological signal data; Construct indicator prediction models; Inputting the characteristic vector into the indicator prediction model to obtain a comprehensive indicator prediction value; Determining a safety level based on the predicted value of the comprehensive indicator; A driver takeover recommendation is generated based on the safety level.

2. The method for quantifying driver takeover performance in an assisted driving environment according to claim 1, characterized in that: Multimodal physiological signal data includes physiological data and eye movement data; Constructing a feature vector based on the multimodal physiological signal data, comprising: performing filtering processing on the physiological data to obtain filtered physiological data; Obtaining time domain features and frequency domain features based on filtered physiological data; performing sliding window normalization processing on the eye movement data to obtain standard eye movement data; The feature vector is constructed based on time domain features, frequency domain features and standard eye movement data.

3. The method for quantifying driver takeover performance in an assisted driving environment according to claim 2, characterized in that: The time domain features include MeanHR value of heart rate variability and SDNN value of heart rate variability; The frequency domain features include: the LF norm value of the heart rate variability frequency domain feature and the HF norm value of the heart rate variability frequency domain feature; Standard eye movement data include: horizontal dispersion distance between fixations, average pupil diameter, blink frequency and average number of saccades.

4. The method for quantifying driver takeover performance in an assisted driving environment according to claim 2, characterized in that: Filtering the physiological data to obtain filtered physiological data includes: The physiological data is filtered using a Butterworth filter to obtain filtered physiological data.

5. The method for quantifying driver takeover performance in an assisted driving environment according to claim 1, characterized in that: Construct indicator prediction models, including: Obtaining raw data; the raw data includes: multimodal historical physiological signal data of the driver, and driver reaction data and vehicle status historical data corresponding to the multimodal historical physiological signal data; the multimodal historical physiological signal data of the driver includes historical physiological data and historical eye movement data; the vehicle status historical data includes: maximum historical lateral offset, minimum historical collision time, and maximum historical longitudinal deceleration; Preprocessing the raw data, and constructing a historical feature vector based on the preprocessed multimodal historical physiological signal data; The historical feature vector is used as an input sample, and the pre-processed driver reaction data and vehicle status history data corresponding to the multimodal historical physiological signal data are used as output samples to obtain a sample data set; Dividing the sample data set into a plurality of mutually exclusive subsets; The XGBoost algorithm is used to build a multi-objective regression prediction model; One of the subsets was selected as the validation set, and the remaining subsets were used as the training set. The multi-objective regression prediction model was trained and verified using the ten-fold cross-validation method to obtain a multi-objective regression prediction model that met the prediction requirements. A multi-objective regression prediction model that meets the prediction requirements is used as the indicator prediction model.

6. The method for quantifying driver takeover performance in an assisted driving environment according to claim 5, characterized in that: In the process of acquiring raw data, the vehicle control signal in the historical vehicle status data is used as the reference time axis, and the dynamic time warping algorithm is used to align the historical physiological data and historical eye movement data.

7. The method for quantifying driver takeover performance in an assisted driving environment according to claim 1, characterized in that: Determining the safety level based on the predicted value of the comprehensive indicator includes: Takeover time threshold, lateral offset threshold, collision time threshold and longitudinal deceleration threshold; When the driver's reaction time, maximum lateral offset, minimum collision time and maximum longitudinal deceleration all meet the set conditions, the safety level is determined to be level 1; When any one or two of the driver's takeover reaction time, maximum lateral offset, minimum collision time, and maximum longitudinal deceleration do not meet the set conditions, the safety level is determined to be the second level; When any three or all of the driver's takeover reaction time, maximum lateral offset, minimum collision time and maximum longitudinal deceleration do not meet the set conditions, the safety level is determined to be the third level.

8. A system for quantifying driver takeover performance in an assisted driving environment, characterized in that: include: Data acquisition module, used to obtain multimodal physiological signal data of the driver in real time; A data processing module, configured to construct a feature vector based on the multimodal physiological signal data; A model prediction module is used to construct an indicator prediction model and input the feature vector into the indicator prediction model to obtain a comprehensive indicator prediction value; A feedback execution module is used to determine the safety level based on the predicted value of the comprehensive indicator, generate a driver takeover suggestion based on the safety level, and trigger a graded warning.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for quantifying driver takeover performance in an assisted driving environment according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for quantifying driver takeover performance in an assisted driving environment according to any one of claims 1 to 7 is implemented.

Images