Driver supervision attention force quantification method fusing eye movement and physiological characteristics

Abstract

The present application relates to the technical field of automatic driving vehicle man-machine co-driving, and relates to a driving person supervision attention quantification method fusing eye movement and physiological characteristics. Index data of a driving person in an automatic driving scene is acquired, and the index data includes eye movement data, electrocardiogram signal data, electrodermal activity data, and subjective monitoring attention level questionnaire score. The index data is preprocessed to determine eye movement indexes and physiological indexes. Sensitivity analysis is performed on the eye movement indexes and the physiological indexes to screen out core sensitive indexes to construct a fusion index data set. Based on the fusion index data set, a driving person supervision attention quantification model is trained to quantize driving person supervision attention according to the trained driving person supervision attention quantification model. By screening sensitive features, the driving person supervision attention dynamic change characteristics in the man-machine co-driving scene are used to construct a driving person supervision attention quantification model using a LightGBM algorithm to accurately quantize the attention state in real time.

Description

Technical Field

[0001] This specification relates to the field of human-machine co-driving technology for autonomous vehicles, and in particular to a method for quantifying driver attention monitoring that integrates eye movement and physiological characteristics. Background Technology

[0002] With the development of autonomous driving technology, SAE Level 3 conditional automated driving systems can assume vehicle control tasks under specific conditions, but still require the driver to continuously monitor the driving environment and take over promptly when requested by the system. Driver Monitoring Attention (DMA) directly affects the safety of takeover and the overall reliability of the human-machine co-driving system. Insufficient DMA leads to a significant decrease in situational awareness and delayed takeover response, thereby increasing the risk of collisions. Therefore, real-time and accurate quantification of DMA is a key technology for ensuring the safety of human-machine co-driving.

[0003] Current methods for DMA quantification primarily rely on single modalities such as eye-tracking data or physiological signals. While eye-tracking data can reflect visual attention allocation, it is susceptible to ambient lighting interference and cannot distinguish situations where the gaze is on the road but cognitive resources are not allocated. Physiological indicators such as electrocardiograms (ECG) and conductance of skin can characterize emotional arousal and attention maintenance, but they suffer from limitations such as response delays and significant individual baseline variability, making it difficult to achieve accurate real-time quantification independently. Currently, DMA quantification methods can be broadly categorized into three types: First, attention assessment methods based on single eye-tracking features, which can capture visual attention dynamics but rely heavily on simple threshold judgments, lack physiological arousal verification, and are susceptible to environmental interference. Second, traditional monitoring methods based on physiological signals, which can reflect the state of autonomic nervous system activation, but lack systematic indicator selection, have a single analytical dimension, and cannot correlate with visual attention allocation. Third, machine learning models based on single modalities, which have insufficient adaptability and lack interpretability. Although eye-tracking and physiological signals are complementary in terms of attention representation dimensions, none of the above methods achieve deep integration of eye-tracking and physiological features, making it difficult to comprehensively characterize the multidimensional nature of DMA, resulting in limited quantification accuracy and affecting the safety of human-machine co-driving.

[0004] Therefore, this specification provides a method for quantifying driver monitoring attention by integrating eye movement and physiological characteristics. Summary of the Invention

[0005] This specification provides a method for quantifying driver monitoring attention by integrating eye movement and physiological characteristics, in order to partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification: This specification provides a method for quantifying driver monitoring attention by integrating eye movement and physiological characteristics, including: S1. Obtain driver indicator data in autonomous driving scenarios, including eye-tracking data, electrocardiogram signal data, skin conductance data, and subjective monitoring attention level questionnaire scores; S2. Preprocess the indicator data to determine eye movement indicators and physiological indicators; S3. Perform sensitivity analysis on the eye movement indicators and the physiological indicators, and select core sensitive indicators to construct a fusion indicator dataset; S4. Based on the fusion index dataset, train a driver supervision attention quantification model to quantify driver supervision attention according to the trained driver supervision attention quantification model.

[0007] Based on the aforementioned technical means, this solution provides a method for quantifying driver-monitored attention in autonomous vehicles by integrating eye-tracking and physiological characteristics. By selecting sensitive features and considering the dynamic changes in driver-monitored attention in human-machine co-driving scenarios, the LightGBM algorithm is used to construct a driver-monitored attention quantification model, accurately quantifying attention status in real time. This invention has small computational errors and strong interpretability, closely reflecting the dynamic distribution characteristics of driver attention in actual human-machine co-driving scenarios. It is beneficial for autonomous driving dynamic warning systems, optimizing takeover timing, and adjusting human-machine interaction strategies, providing technical support for preventing takeover failure accidents caused by insufficient attention.

[0008] Furthermore, S1 also includes step S11: Eye movement data, electrocardiogram (ECG) signal data, and electrodermal activity data of the driver were collected through a driving simulator experiment, and the time synchronization of the eye movement data, ECG signal data, and electrodermal activity data was achieved by using a pulse signal alignment mechanism.

[0009] Furthermore, S2 specifically includes: The eye movement data in the index data is subjected to median filtering to determine blinking events and saccade events; the ECG signal data in the index data is subjected to ECG R-wave peak detection to determine heart rate index and band index; and the skin conductance activity data in the index data is subjected to low-pass filtering and continuous decomposition analysis to determine skin conductance level and skin conductance response.

[0010] Furthermore, S3 specifically includes: For the blinking event, the saccade event, the heart rate index, the band index, the skin conductance level, and the skin conductance response, respectively, the standardized regression coefficients and significance levels are calculated; Indicators whose standardized regression coefficients and significance levels meet preset conditions are used as core sensitive indicators and stored in the fusion indicator dataset.

[0011] Furthermore, S3 also includes step S31: For each of the selected core sensitive indicators, calculate the Pearson correlation coefficient; Core sensitive indicators whose Pearson correlation coefficients reach a preset value are fused and stored in the fused indicator dataset.

[0012] Furthermore, in S4, based on the aforementioned fused index dataset, a driver monitoring attention quantification model is trained, specifically including: The metrics in the fusion metric dataset are used as training samples, and the subjective monitoring attention level questionnaire scores are used as labels. The training samples are input into the driver monitoring attention quantification model to be trained to obtain the driver's subjective monitoring attention output by the driver monitoring attention quantification model to be trained. Based on the difference between the driver's subjective monitoring attention and the label, the loss is determined, and the driver's monitoring attention quantification model to be trained is trained with the goal of minimizing the loss.

[0013] Furthermore, the driver monitoring attention quantification model to be trained is a model constructed using the LightGBM algorithm.

[0014] This specification provides a driver monitoring attention quantification device that integrates eye movement and physiological characteristics, including: The acquisition module is used to acquire the driver's indicator data in the autonomous driving scenario. The indicator data includes eye movement data, electrocardiogram signal data, skin conductance activity data, and subjective monitoring attention level questionnaire score. The preprocessing module is used to preprocess the indicator data to determine eye movement indicators and physiological indicators; The analysis module is used to perform sensitivity analysis on the eye movement indicators and the physiological indicators, and to screen out the core sensitive indicators to construct a fusion indicator dataset. The training module is used to train a driver supervision attention quantification model based on the fusion index dataset, so as to quantify driver supervision attention according to the trained driver supervision attention quantification model.

[0015] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for quantifying driver monitoring attention by integrating eye movements and physiological characteristics.

[0016] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements a method for quantifying driver monitoring attention by integrating eye movements and physiological characteristics.

[0017] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: This solution provides a method for quantifying driver-monitored attention in autonomous vehicles by integrating eye-tracking and physiological characteristics. By selecting sensitive features and considering the dynamic changes in driver-monitored attention in human-machine co-driving scenarios, a driver-monitored attention quantification model is constructed using the LightGBM algorithm to accurately quantify attention status in real time. This invention has small computational errors and strong interpretability, closely reflecting the dynamic distribution characteristics of driver attention in real-world human-machine co-driving scenarios. It is beneficial for autonomous driving dynamic warning systems, optimizing takeover timing, and adjusting human-machine interaction strategies, providing technical support for preventing takeover failure accidents caused by insufficient attention. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings: Figure 1 A flowchart illustrating a method for quantifying driver monitoring attention by integrating eye movements and physiological characteristics, provided in an embodiment of this specification. Figure 2a This is a schematic diagram of the electrocardiogram (ECG) data extraction process; Figure 2b This is a schematic diagram illustrating the process of extracting electrodermal activity data. Figure 3a The overall ROC curve for LightGBM; Figure 3b The ROC curves for LightGBM micro-average and macro-average are shown. Figure 3c ROC curves for each attention category in LightGBM; Figure 3d A summary chart of AUC for each category of LightGBM; Figure 4 A schematic diagram of a driver monitoring attention quantification device that integrates eye movement and physiological characteristics, as provided in this specification; Figure 5 This specification provides a corresponding Figure 1 A schematic diagram of the structure of an electronic device. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0020] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0021] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0022] Figure 1 A flowchart illustrating a method for quantifying driver monitoring attention by integrating eye movements and physiological characteristics, as provided in this specification, includes the following steps: S1: Obtain driver indicator data in autonomous driving scenarios, including eye-tracking data, electrocardiogram signal data, skin conductance data, and subjective monitoring attention level questionnaire scores.

[0023] This specification describes the process of quantifying driver supervision attention by fusing eye movements and physiological characteristics. In the embodiments described herein, this process can be executed by a server. However, this specification does not limit the type of device or platform used to perform this process; for example, a personal computer, mobile terminal, or other similar device or platform can also be used. For ease of description, the following explanation uses a server as the executing entity.

[0024] In one or more embodiments of this specification, the server can obtain driver indicator data in an autonomous driving scenario, including eye-tracking data, electrocardiogram signal data, skin conductance activity data, and subjective monitoring attention level questionnaire scores.

[0025] Typical failure scenarios for autonomous driving systems can be constructed using a driving simulator, including two types of emergencies: accidents involving vehicles ahead and road collapses. Non-driving-related tasks are designed to induce different levels of attentional shifts. These non-driving-related tasks include two levels of cognitive-auditory tasks and two levels of visual-manual tasks, using a differentiated cognitive resource competition mechanism to regulate the driver's supervisory attention. Simultaneously, multimodal objective indicators of the driver are collected, including eye-tracking data, electrocardiogram (ECG) data, and electrodermal activity (TEA) data. Subjective indicators of the driver are also collected, namely the Subjective Monitoring Attention (SMA) scale, using a 5-point scale (1 = very inattentive, 2 = somewhat inattentive, 3 = average, 4 = fairly focused, 5 = very focused), with higher scores indicating greater concentration.

[0026] Therefore, non-driving-related tasks can be designed using driving simulators to induce varying levels of attentional changes. Driver eye-tracking, electrocardiogram (ECG), and skin conductance data can be collected through simulator experiments, and a pulse signal alignment mechanism can be used to synchronize these data in time. The subjective monitoring attention level questionnaire scores the driver's level of supervisory attention within the analysis window 20 seconds prior to a takeover request.

[0027] Eye-tracking data is acquired using an eye tracker at a sampling frequency of 50 Hz. Three-point calibration is performed before acquisition to ensure the gaze point is aligned with the driving simulator position. An internal algorithm categorizes eye-tracking data into fixation events (relative eye stillness ≥120 ms) and saccade events (rapid movement between gazes). Eye-tracking data may include gaze point coordinates, pupil diameter, blink events, and saccade events. Electrocardiogram (ECG) signal data and electrodermal activity (EDA) data are acquired through a physiological feedback system.

[0028] Figure 2a This diagram illustrates the ECG data extraction process. ECG signal data was acquired at a sampling frequency of 100 Hz, with electrodes placed in the fifth left intercostal space and below the right clavicle. The horizontal axis represents the ECG signal acquisition time in seconds (s); the vertical axis represents the ECG signal amplitude in millivolts (mV). Figure 2a The 1-second ECG signal of a driver is selected as an example, and the black solid line represents the time sequence curve of the preprocessed ECG signal.

[0029] P, Q, R (R'), S, and T represent the positions of characteristic waves within a cardiac cycle. The P wave reflects atrial depolarization, the QRS complex reflects ventricular depolarization, and the T wave reflects ventricular repolarization. The time interval between two adjacent R waves (R and R') is the RR interval, used to calculate heart rate and heart rate variability. The PR interval and QT interval represent the duration of the corresponding electrocardiogram waveforms, and their measurement range is indicated by the horizontal line segment in the figure.

[0030] Figure 2b This diagram illustrates the process of extracting electrodermal activity (EDA) data. EDA data was collected at a sampling frequency of 100 Hz, with electrodes attached to the index and middle fingers of the left hand. The horizontal axis represents time relative to the time of the takeover request, in seconds (s), where 0 s represents the time the takeover request was issued, and negative values ​​represent the time 20 seconds prior to the request. The left vertical axis represents skin conductance, and the right vertical axis represents skin conductance response, both in microsiemens (μS).

[0031] Figure 2b The solid blue line representing skin conductance (SC) indicates the raw skin conductance signal after preprocessing; the dashed blue line representing skin conductance level (SCL) indicates the skin conductance level obtained through continuous decomposition, reflecting the slowly changing basic physiological arousal state; the dashed red line representing skin conductance response (SCR) indicates the skin conductance response, reflecting the short-term sympathetic response caused by stimulation or changes in attention. The raw skin conductance signal can be expressed as the sum of skin conductance level, skin conductance response, and measurement noise, i.e.:

[0032] In the formula, Represents the raw skin conductance signal at time t; Indicates the level of skin conductance under tension; This indicates phase-dependent skin conductance response; This represents measurement noise and unmodeled disturbances.

[0033] S2: Preprocess the indicator data to determine eye movement indicators and physiological indicators.

[0034] In one or more embodiments of this specification, the server may preprocess the indicator data to determine eye-tracking indicators and physiological indicators.

[0035] Specifically, median filtering can be applied to eye movement data in the indicator data to identify blinking and saccade events, as well as pupillary response characteristics and AttenD time series characteristics. ECG signal data in the indicator data can be processed to detect R-wave peak values, determining heart rate and waveband indicators, including the PR interval, QRS complex, and QT interval. Furthermore, skin conductance activity data in the indicator data can be subjected to 8th-order Butterworth low-pass filtering followed by continuous decomposition analysis (CDA) to determine skin conductance level (SCL) and skin conductance response (SCR). S3: Perform sensitivity analysis on the eye movement indicators and the physiological indicators, and select core sensitive indicators to construct a fusion indicator dataset.

[0036] In one or more embodiments of this specification, the server can perform sensitivity analysis on eye-tracking indicators and physiological indicators, and select core sensitive indicators to construct a fusion indicator dataset.

[0037] Specifically, standardized regression coefficients β and significance levels were calculated for blinking events, saccade events, heart rate indicators, waveband indicators, skin conductance levels, and skin conductance responses. p Then, indicators whose standardized regression coefficients and significance levels meet the preset conditions are used as core sensitive indicators and stored in the fusion indicator dataset.

[0038] Calculate the standardized regression coefficient β and the significance level. p Using eye-tracking metrics or physiological indicators as dependent variables, and driver subjective monitoring attention scores as fixed-effect independent variables, with driver identification numbers set as a random effect to control for repeated measurements and individual differences, the linear mixed-effects model is as follows:

[0039] In the formula For the first Driver No. Eye movement parameters or physiological parameters observed in each study The subjective monitoring attention level questionnaire was used for scoring. For a fixed intercept. These are the non-standardized regression coefficients. For driver random effects. This is random error.

[0040] To eliminate the influence of different indicator units, the independent and dependent variables are standardized:

[0041] In the formula, This indicates the subjective monitoring attention level questionnaire score. This indicates the eye movement or physiological indicators to be analyzed. and These represent the sample means of the two, respectively. and These represent the sample standard deviations of the two, respectively. and These represent the standardized variables. The standardized variables have a mean of 0 and a standard deviation of 1, which allows for comparison of the sensitivity of different indicators.

[0042] The standardized regression coefficients are:

[0043] In the formula, This represents the estimated value of the standardized regression coefficients; This represents the unstandardized regression coefficients obtained from the linear mixed-effects model; This indicates the conversion ratio between the standard deviations of the independent and dependent variables. When using the standardized values ​​directly... and When fitting the model, the regression coefficients output by the model are the standardized regression coefficients. Their positive or negative sign indicates the direction of change of the indicator and subjective monitoring attention; the larger the absolute value, the more sensitive the indicator is to changes in regulatory attention.

[0044] The standard errors of standardized regression coefficients are converted using the same proportional method.

[0045] in, The t-statistic is calculated from a linear mixed-effects model based on sample variation, residuals, and the structure of fixed and random effects. The t-statistic is calculated using the standardized regression coefficients and their standard errors.

[0046] In the formula, the numerator represents the estimated value of the standardized regression coefficient, and the denominator represents the degree of uncertainty of that estimate. The larger the absolute value of the t-value, the greater the regression coefficient relative to the random error. The two-sided significance level is calculated based on the t-value and the t-distribution of the corresponding degrees of freedom of the model:

[0047] In the formula, Describing the degrees of freedom as of The distribution and degrees of freedom are determined based on the sample size and model structure of the linear mixed-effects model. This can be achieved by satisfying... The indicator is defined as one that is significantly sensitive to the supervisory attention of drivers.

[0048] When the indicator data approximately follows a normal distribution and satisfies the sphericity assumption, repeated measures ANOVA is used; when the sphericity assumption is not met, the Greenhouse-Geisser method is used for correction; when the data does not meet the parametric test conditions, the Friedman test is used. Subjective monitoring attention level is used as a within-subjects factor, and eye-tracking or physiological indicators are used as the dependent variable. The test statistic is:

[0049] in:

[0050]

[0051] In the formula, This represents the sum of squares of the variation in the indicator caused by different levels of subjective monitoring attention. The sum of squares of the subjective monitoring attention level effect is used to represent the total variation of the indicator caused by different attention levels. This represents the sum of squared errors remaining after deducting the attention level effect and individual driver differences. Indicates the number of drivers. This represents the level of subjective monitoring attention, in this embodiment. , This represents the mean square of the attention rank effect.

[0052] Statistics are used to compare the relative magnitudes of the variation in indicators caused by attention levels with the variation in random error. The larger the value, the more significant the difference in indicators between different attention levels relative to random error. p When the value is less than 0.05, it is considered that there is a significant difference in the mean values ​​of the indicators under at least two subjective monitoring attention levels.

[0053] Each driver's indicator score at different attention levels is assigned a rank from lowest to highest. For the first The driver was in The rank under each level of attention, the first The rank sum of the attention levels is:

[0054] The Friedman test is used, and its statistic is:

[0055] In the formula, For the number of drivers, To determine the subjective attention level, this embodiment analyzes the data based on the original SMA score of 1-5. , For the first The rank sum of attention levels.

[0056] The statistic is used to test whether the rank distribution of the indicators is the same under different subjective monitoring attention levels. The higher the value, the more significant the difference in indicators between different attention levels. (Based on degrees of freedom) Calculate the significance level using the chi-square distribution p ,when p When <0.05, it is considered that there are significant differences in the overall distribution of indicators under different attention levels.

[0057] Repeated measures ANOVA Values ​​and Friedman test results The average value can only determine whether there is an overall difference between different attention levels, but cannot directly determine which specific levels show differences. Therefore, when the overall test results meet the requirements... p When the value is less than 0.05, further pairwise comparisons between attention levels are performed, and the Bonferroni method is used to correct for significance levels.

[0058] Furthermore, the Pearson correlation coefficient (r) can be calculated for each selected core sensitive indicator to quantify the correlation between indicators and eliminate redundant collinear indicators. For any two core sensitive indicators... and Based on its Calculate the Pearson correlation coefficient for each valid paired sample:

[0059] In the formula, and The first The index value of each sample and These are the sample means of the two indicators, respectively. This represents the number of valid samples.

[0060] The range of values ​​for is [ [1,1], where the positive or negative sign indicates the direction of correlation, and the larger the absolute value, the stronger the linear correlation. When two indicators satisfy |r|≥0.80, they are judged to have a strong correlation, and indicators with higher sensitivity, stronger significance, or better interpretability are retained to reduce feature redundancy.

[0061] S4: Based on the fusion index dataset, train a driver supervision attention quantification model to quantify driver supervision attention according to the trained driver supervision attention quantification model.

[0062] In one or more embodiments of this specification, the server can train a driver supervision attention quantification model based on a fusion index dataset. The driver supervision attention can be quantified according to the trained driver supervision attention quantification model. After that, the collected physiological indicators and eye movement indicators of the driver can be input into the driver supervision attention quantification model to quantify the driver's attention.

[0063] The method for training a driver's supervisory attention quantification model involves using the indicators from a fusion indicator dataset as training samples and the subjective monitoring attention level questionnaire scores as labels. The training samples are then input into the driver's supervisory attention quantification model to be trained, yielding the driver's subjective monitoring attention output by the model. Based on the difference between the driver's subjective monitoring attention and the labels, a loss is determined, and the model is trained with the goal of minimizing this loss.

[0064] The subjective monitoring attention level questionnaire score can be divided into three categories: inattentive, moderately attentive, and attentive. Specifically, the original 5-level SMA score is reconstructed into a three-category framework: inattentive state (original score 1-2, code 0), moderately attentive state (original score 3, code 1), and attentive state (original score 4-5, code 2). A three-category quantitative model of regulatory attention is trained to accurately quantify the driver's inattentive, moderately attentive, and attentive attention states in real time.

[0065] It is worth noting that the driver monitoring attention quantification model to be trained is a model built using the LightGBM (LightGradient Boosting Machine) algorithm.

[0066] Figure 3a This is the LightGBM composite ROC curve. Receiver operating characteristic (ROC) is commonly used in statistics and medical diagnosis (e.g., ROC curves). Figure 3a The horizontal axis represents the false positive rate, and the vertical axis represents the true positive rate. Different curves represent the model's recognition results for the three attention states. The closer the curve is to the top left corner, the larger the area under the ROC curve (AUC), and the better the model's classification performance.

[0067] Figure 3bThese are the ROC curves for the LightGBM micro-average and macro-average. The micro-average reflects the overall classification ability of all samples, while the macro-average reflects the model's balanced recognition ability across different classes.

[0068] Figure 3c These are the ROC curves for LightGBM at different attention levels. The three curves correspond to non-focused, moderately focused, and focused states, respectively, and are used to compare the model's recognition performance for different attention states.

[0069] Figure 3d This is a summary chart of AUC values ​​for each category in LightGBM. The horizontal axis represents different attention categories and the micro-average and macro-average results, while the vertical axis represents the AUC values, used to visually compare the model's ability to distinguish between categories.

[0070] The LightGBM model uses a histogram algorithm to discretize and bin continuous eye-tracking and physiological features, reducing the computational complexity of high-dimensional sparse data. It employs a leaf-wise growth strategy instead of level-wise growth, enhancing the model's ability to fit the transitional range of driver attention states while maintaining its lightweight nature, thus meeting the low-latency requirements of real-time monitoring systems. An interpretable quantitative model of driver supervisory attention was constructed, enabling global feature importance ranking, category-level contribution direction analysis, and individual prediction decision analysis, revealing the differentiated contribution mechanism of eye-tracking and physiological fusion features under different attention states. The model verified that the intermediate state is difficult to identify due to its transitional characteristics and high heterogeneity of physiological indicators, and that the three-classification framework better reflects the dynamic and gradual changes in attention states during actual driving compared to traditional binary classification. This invention effectively avoids problems such as the susceptibility of single-dimensional indicators to environmental interference, the oversimplification of traditional binary classification models, and the lack of mechanistic explanation in black-box models. It accurately identifies three types of attention states with high separation and their core physiological discrimination indicators, with good identification results, which is conducive to the formulation of early warning strategies and the improvement of takeover safety in autonomous driving systems.

[0071] Based on the driver monitoring attention quantification method integrating eye movement and physiological characteristics provided in one or more embodiments of this specification, following the same line of thought, this specification also provides a corresponding driver monitoring attention quantification device integrating eye movement and physiological characteristics, such as... Figure 4 As shown.

[0072] Figure 4 This specification provides a schematic diagram of a driver attention monitoring quantification device that integrates eye movement and physiological characteristics, specifically including: The acquisition module 400 is used to acquire the driver's indicator data in the autonomous driving scenario. The indicator data includes eye movement data, electrocardiogram signal data, skin conductance activity data, and subjective monitoring attention level questionnaire score. Preprocessing module 402 is used to preprocess the indicator data to determine eye movement indicators and physiological indicators; Analysis module 404 is used to perform sensitivity analysis on the eye movement indicators and the physiological indicators, and to screen out the core sensitive indicators to construct a fusion indicator dataset; Training module 406 is used to train a driver supervision attention quantification model based on the fusion index dataset, so as to quantify driver supervision attention according to the trained driver supervision attention quantification model.

[0073] Optionally, the acquisition module 400 is used to collect eye movement data, electrocardiogram signal data, and skin conductance activity data of the driver through a driving simulator experiment, and to use a pulse signal alignment mechanism to achieve time synchronization of the eye movement data, the electrocardiogram signal data, and the skin conductance activity data.

[0074] Optionally, the preprocessing module 402 is used to perform median filtering on the eye movement data in the indicator data to determine blinking events and saccade events; to perform ECG R-wave peak detection on the ECG signal data in the indicator data to determine heart rate indicators and band indicators; and to perform low-pass filtering and continuous decomposition analysis on the skin conductance activity data in the indicator data to determine skin conductance level and skin conductance response.

[0075] Optionally, the analysis module 404 is used to calculate the standardized regression coefficients and significance levels for the blinking event, the saccade event, the heart rate index, the band index, the skin conductance level, and the skin conductance response, respectively, and to store the indicators whose standardized regression coefficients and significance levels both reach the preset conditions as core sensitive indicators in the fusion index dataset.

[0076] Optionally, the analysis module 404 is further configured to calculate the Pearson correlation coefficient for each of the selected core sensitive indicators, and to fuse the core sensitive indicators whose Pearson correlation coefficient reaches a preset value and store them in the fused indicator dataset.

[0077] Optionally, the training module 406 is used to take the indicators in the fusion indicator dataset as training samples, the subjective monitoring attention level questionnaire score as labels, input the training samples into the driver monitoring attention quantification model to be trained, obtain the driver subjective monitoring attention output by the driver monitoring attention quantification model to be trained, determine the loss based on the difference between the driver subjective monitoring attention and the label, and train the driver monitoring attention quantification model to be trained with the minimum loss as the optimization objective.

[0078] Optionally, the driver monitoring attention quantification model to be trained in the training module 406 is a model constructed using the LightGBM algorithm.

[0079] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 This paper presents a method for quantifying driver attention by integrating eye movement and physiological characteristics.

[0080] This instruction manual also provides Figure 5 The diagram shows a schematic structural representation of the electronic device. Figure 5 As shown, at the hardware level, this electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above. Figure 1 This paper presents a method for quantifying driver attention by integrating eye movement and physiological characteristics.

[0081] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0082] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0083] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0084] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0085] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0086] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0091] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0092] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic or disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0093] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0094] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0096] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0097] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for quantifying driver attention by integrating eye movement and physiological characteristics, characterized in that, include: S1. Obtain driver indicator data in autonomous driving scenarios, including eye-tracking data, electrocardiogram signal data, skin conductance data, and subjective monitoring attention level questionnaire scores; S2. Preprocess the indicator data to determine eye movement indicators and physiological indicators; S3. Perform sensitivity analysis on the eye movement indicators and the physiological indicators, and select core sensitive indicators to construct a fusion indicator dataset; S4. Based on the fusion index dataset, train a driver supervision attention quantification model to quantify driver supervision attention according to the trained driver supervision attention quantification model.

2. The method for quantifying driver monitoring attention by integrating eye movement and physiological characteristics as described in claim 1, characterized in that, S1 also includes step S11: Eye movement data, electrocardiogram (ECG) signal data, and electrodermal activity data of the driver were collected through a driving simulator experiment, and the time synchronization of the eye movement data, ECG signal data, and electrodermal activity data was achieved by using a pulse signal alignment mechanism.

3. The method for quantifying driver monitoring attention by integrating eye movement and physiological characteristics as described in claim 1, characterized in that, S2 specifically includes: The eye movement data in the index data is subjected to median filtering to determine blinking events and saccade events; the ECG signal data in the index data is subjected to ECG R-wave peak detection to determine heart rate index and band index; and the skin conductance activity data in the index data is subjected to low-pass filtering and continuous decomposition analysis to determine skin conductance level and skin conductance response.

4. The method for quantifying driver monitoring attention by integrating eye movement and physiological characteristics as described in claim 3, characterized in that, S3 specifically includes: For the blinking event, the saccade event, the heart rate index, the band index, the skin conductance level, and the skin conductance response, respectively, the standardized regression coefficients and significance levels are calculated; Indicators whose standardized regression coefficients and significance levels meet preset conditions are used as core sensitive indicators and stored in the fusion indicator dataset.

5. The method for quantifying driver monitoring attention by integrating eye movement and physiological characteristics as described in claim 4, characterized in that, S3 also includes step S31: For each of the selected core sensitive indicators, calculate the Pearson correlation coefficient; Core sensitive indicators whose Pearson correlation coefficients reach a preset value are fused and stored in the fused indicator dataset.

6. The method for quantifying driver monitoring attention by integrating eye movement and physiological characteristics as described in claim 1, characterized in that, S4 trains a driver monitoring attention quantification model based on the aforementioned fused index dataset, specifically including: The metrics in the fusion metric dataset are used as training samples, and the subjective monitoring attention level questionnaire scores are used as labels. The training samples are input into the driver monitoring attention quantification model to be trained to obtain the driver's subjective monitoring attention output by the driver monitoring attention quantification model to be trained. Based on the difference between the driver's subjective monitoring attention and the label, the loss is determined, and the driver's monitoring attention quantification model to be trained is trained with the goal of minimizing the loss.

7. The method for quantifying driver monitoring attention by integrating eye movement and physiological characteristics as described in claim 6, characterized in that, The driver monitoring attention quantification model to be trained is a model constructed using the LightGBM algorithm.

8. A driver attention monitoring quantification device integrating eye movement and physiological characteristics, characterized in that, include: The acquisition module is used to acquire the driver's indicator data in the autonomous driving scenario. The indicator data includes eye movement data, electrocardiogram signal data, skin conductance activity data, and subjective monitoring attention level questionnaire score. The preprocessing module is used to preprocess the indicator data to determine eye movement indicators and physiological indicators; The analysis module is used to perform sensitivity analysis on the eye movement indicators and the physiological indicators, and to screen out the core sensitive indicators to construct a fusion indicator dataset. The training module is used to train a driver supervision attention quantification model based on the fusion index dataset, so as to quantify driver supervision attention according to the trained driver supervision attention quantification model.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 7.

Images