Driver state recognition method, device, equipment, medium and product in assisted driving environment

Abstract

The application discloses a driving person state recognition method, device, equipment, medium and product in an assisted driving environment, relates to the driving person state recognition field, and comprises the following steps: acquiring subjective data and physiological data of a driving person; preprocessing the physiological data; selecting time-frequency characteristic indexes of the preprocessed physiological data; determining an event type; dividing a driving process into three stages with a takeover warning time as a zero point; performing correlation analysis on the event type and the time-frequency characteristic indexes in the second stage and the third stage, and obtaining an analysis result; eliminating irrelevant time-frequency characteristic indexes based on the analysis result; performing importance sorting on the eliminated time-frequency characteristic indexes by using a random forest algorithm; selecting characteristic indexes based on the sorting result; and obtaining a driving person supervision state quantitative grade by using a K-means clustering algorithm based on the characteristic indexes and the subjective data. The application can more accurately capture the dynamic evolution of a driving state.

Description

Technical Field

[0001] This application relates to the field of driver state recognition, and in particular to a method, device, equipment, medium and product for driver state recognition in an assisted driving environment. Background Technology

[0002] With the rapid development of driver assistance systems, existing driver status recognition technologies mainly include three types: those based on physiological signals, behavioral characteristics, and vehicle status.

[0003] Monitoring based on physiological signals mainly collects EEG, ECG, skin conductance, and pulse signals, which can accurately determine the driver's fatigue and tension state. However, individual differences make model building difficult and data accuracy fluctuates greatly. In addition, it does not distinguish the importance of the indicators and is currently mainly used in experimental scenarios. Monitoring based on behavioral characteristics mainly uses cameras and sensors to collect the driver's facial expressions, eye or head state. However, in assisted driving, the driver's attention may be diverted due to reliance on assisted driving, which can easily lead to missed detections and increase the difficulty of recognition. Monitoring based on vehicle status infers the driver's state by analyzing the vehicle's driving trajectory, speed, acceleration, and lane departure. However, the automatic intervention of assisted driving systems can interfere with the judgment and affect the monitoring accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, equipment, medium and product for driver state recognition in an assisted driving environment, which can more accurately capture the dynamic evolution of driving state.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, this application provides a method for driver state recognition in an assisted driving environment, including:

[0007] Acquire driver's subjective and physiological data;

[0008] The physiological data are preprocessed;

[0009] Select time-frequency characteristic indicators of the preprocessed physiological data;

[0010] Determine the event type;

[0011] The driving process is divided into three stages with the takeover warning time as the zero point: the first stage is [-6s, -3s] before the takeover warning time, the second stage is [-3s, 0s] before the takeover warning time, and the third stage is the post-warning stage.

[0012] Correlation analysis was performed on the event types and the time-frequency characteristic indicators in the second and third stages to obtain the analysis results.

[0013] Irrelevant time-frequency feature indicators were eliminated based on the analysis results;

[0014] The importance of the removed time-frequency feature indicators was ranked using the random forest algorithm.

[0015] Feature indicators are selected based on the ranking results;

[0016] The driver's regulatory status level is obtained using the K-means clustering algorithm based on the aforementioned feature indicators and subjective data.

[0017] Optionally, the subjective data includes basic personal information, a driving status scale, and a status survey on scenario-related factors; the physiological data includes pulse wave signal (PPG), electrical skin signal (EDA), and electrical muscle signal (EMG).

[0018] Optionally, preprocessing the physiological data specifically includes the following steps:

[0019] A bandpass filter is used to filter the PPG and EMG signals;

[0020] A band-stop filter is used to perform secondary filtering on the bandpass filtered PPG and EMG signals;

[0021] A low-pass filter is used to filter the EDA signal;

[0022] The Gaussian smoothing noise reduction method is used to reduce noise in the filtered EDA signal;

[0023] Wavelet denoising was used to denoise the PPG and EMG signals after secondary filtering.

[0024] Optionally, the time-frequency characteristic index includes:

[0025] Heart rate (HR), percentage of intervals with a normal heart rate interval exceeding 20ms, PNN20, SDNN, RMSSD, SCL_mean, SCL_std, LF_ratio, MAV, RMS, MPF, MF, iEMG, and LF / HF.

[0026] Optionally, the event types include three categories: pedestrian crossing events at intersections, pedestrian crossing events in road sections, and pedestrian crossing events in blind spots of road sections.

[0027] Secondly, this application provides a driver state recognition device in an assisted driving environment, comprising:

[0028] The data acquisition module is used to acquire the driver's subjective and physiological data;

[0029] The preprocessing module is used to preprocess the physiological data;

[0030] The time-domain feature index selection module is used to select time-frequency feature indicators of the preprocessed physiological data;

[0031] The event type determination module is used to determine the event type;

[0032] The phase division module is used to divide the driving process into three phases with the takeover warning time as the zero point. The first phase is: before the takeover warning time [-6s, -3s], the second phase is: before the takeover warning time [-3s, 0s], and the third phase is: after the warning.

[0033] The correlation analysis module is used to perform correlation analysis on the event type and the time-frequency characteristic indicators in the second and third stages to obtain the analysis results.

[0034] The indicator elimination module is used to eliminate irrelevant time-frequency feature indicators based on the analysis results;

[0035] The importance ranking module is used to rank the importance of the removed time-frequency feature indicators using the random forest algorithm;

[0036] The feature selection module is used to select feature indicators based on the ranking results;

[0037] The regulatory status level determination module is used to obtain the driver's regulatory status level based on the aforementioned feature indicators and subjective data using the K-means clustering algorithm.

[0038] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the driver state recognition method in the assisted driving environment described in any one of the above.

[0039] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the driver state recognition method in the assisted driving environment described above.

[0040] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the driver state recognition method in the assisted driving environment described above.

[0041] According to the specific embodiments provided in this application, this application has the following technical effects:

[0042] This application provides a method, device, equipment, medium, and product for driver state recognition in assisted driving environments. It integrates three physiological signal indicators (PPG, EDA, and EMG) and applies different filtering and noise reduction methods to different physiological signals to prevent feature loss. A standardized questionnaire design and analysis method is used to process the comprehensive driving state score. Multi-dimensional data fusion reduces individual differences and external interference, resulting in greater accuracy compared to single-data monitoring. After dimensionality reduction of the physiological indicators, key physiological indicators are selected within a 3-second time window using a random forest algorithm, effectively reducing state assessment delay. Multi-dimensional physiological signal monitoring is less affected by the environment and can adapt to different assisted driving scenarios. Simultaneously, the data is divided into three stages (baseline stage, pre-warning stage, and post-warning stage) to more accurately capture the dynamic evolution of the driving state and improve algorithm adaptability. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart illustrating a driver state recognition method in an assisted driving environment, provided as an embodiment of this application;

[0045] Figure 2 This is a schematic diagram showing the order of importance of physiological indicators provided in an embodiment of this application;

[0046] Figure 3 A schematic diagram of the functional modules of a driver state recognition device in an assisted driving environment is provided for another embodiment of this application;

[0047] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] In one exemplary embodiment, such as Figure 1 As shown, a method for driver state recognition in an assisted driving environment is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. The method includes the following steps:

[0051] Step 101: Obtain the driver's subjective and physiological data.

[0052] The subjective data includes basic personal information, a driving status scale, and a status survey on scenario-related factors; the physiological data includes pulse wave signals (PPG), electrical skin signals (EDA), and electrical muscle signals (EMG) collected by physiological facilities.

[0053] The comprehensive driving status scale includes a five-point rating system for aggression, driving aversion, fatigue tendency, hazard monitoring, and stimulation seeking.

[0054] Step 102: Preprocess the physiological data.

[0055] The specific processing procedure is as follows:

[0056] Filtering: The effective components of PPG and EMG are located in the mid-frequency band. To simultaneously remove baseline drift of low-frequency signals and high-frequency noise, bandpass filters are used to filter them. Pulse wave signals in the 1-20Hz range are filtered out, and EMG electromyography signals in the 5-150Hz range are filtered out. Band-stop filters are added to both to remove 50Hz power frequency noise. The effective components of EDA are in the low-frequency band, so only high-frequency noise needs to be removed. A low-pass filter is used with a cutoff frequency set to 2Hz.

[0057] Noise Reduction: Since the effective frequency components of PPG and EMG are concentrated in different frequency bands compared to EDA, different noise reduction methods are chosen. EDA signal noise typically manifests as high-frequency random fluctuation interference signals, separated from the main signal band of EDA. However, considering the sparse and large amplitude of the phase component in EDA, it remains identifiable after smoothing and noise reduction. Gaussian smoothing noise reduction can effectively filter out high-frequency noise while preserving low-frequency trends; therefore, it is chosen for noise reduction of the EDA signal. Both PPG and EMG contain steeply rising high-frequency signals: the main peak reflecting the rising phase of the heartbeat in PPG, and the sudden high-frequency pulses generated by high-intensity muscle activity in EMG. Since Gaussian smoothing noise reduction blurs high-frequency signals, leading to feature loss, wavelet noise reduction is combined with its ability to separate high-frequency transient features and preserve local abrupt changes in the signal. Matching wavelet noise reduction achieves noise separation and suppression.

[0058] Step 103: Select time-frequency characteristic indicators of the preprocessed physiological data.

[0059] Specifically, the steps include the following:

[0060] First, a 3-second time window is uniformly determined, and then the time-frequency characteristic indicators of the three types of signals within this time window are extracted.

[0061] (1) PPG signal

[0062] Heart rate (HR, bpm): This is the most common indicator of cardiac activity. The calculation first identifies the peak pulse value (i.e., the main peak, with a peak threshold set at 70%), then calculates the pulse wave peak interval (i.e., pulse interval IBI, similar to the RR interval on an electrocardiogram). Abnormal main peak interval IBI values ​​(such as intervals too short, less than 0.3s) are filtered out to obtain the NN interval.

[0063]

[0064] PNN20(ms): The percentage of intervals with an NN interval exceeding 20ms within a specified time period. A high PNN20 value indicates a large variation in heart rate intervals and high heart rate variability, which also suggests that the driver has strong physical adaptability and can flexibly respond to emergencies.

[0065] SDNN (ms): The standard deviation of all normal NN intervals within a specified time period, suitable for reflecting the overall range of change. A higher SDNN value indicates greater heart rate variation, better overall condition, and a stronger ability for the driver to adapt to environmental changes, demonstrating the body's flexibility in responding to environmental changes.

[0066]

[0067] RMSSD (ms): The root mean square of the difference between adjacent IBI intervals within a specified time period, reflecting sudden changes in normal heartbeat intervals. A high RMSSD indicates that the body can quickly adjust its heartbeat and has strong recovery capabilities.

[0068]

[0069] Where N is the number of sample points within the time period.

[0070] (2)EDA signal

[0071] EDA signals mainly consist of tonic data and pulsed data, which are located in different frequency bands. The tonic data ranges from 0 to 0.05 Hz, while the pulsed data ranges from 0.05 to 2 Hz. High-pass filtering can roughly separate them. The data from the EDA acquisition device after filtering and denoising preprocessing is called skin conductance (SC) data. The separated tonic data yields skin conductance level (SCL) data, and the pulsed data yields skin conductance response (SCR) data. The EDA signal (SC) is decomposed into SCL and SCR through signal processing, but this is not a simple arithmetic sum. SCL is the baseline value of skin conductance, representing the continuous conductance level in a resting state without external stimuli, suitable for monitoring environmental adaptability. SCR is the transient change in skin conductance triggered by external stimuli, suitable for special stimulus and event-related testing experiments. In this application, due to the short study period, there may be cases where no SCR events occur within the analysis timeframe; therefore, only SCL is analyzed.

[0072] SCL_mean(μS): The mean of skin conductance level (SCL), which is the average value of SCL data over a period of time.

[0073] SCL_std(μS): The standard deviation of SCL, used to measure the dispersion of SCL data.

[0074] LF_ratio(ratio): The low-frequency power ratio of SCL in the frequency domain, used to reflect sympathetic nerve fluctuations.

[0075] (3) EMG signal

[0076] Temporal characteristics of electromyographic signals:

[0077] MAV (μV): The average absolute value of electromyographic activity intensity over a specified time period.

[0078] RMS (μV): The root mean square of the intensity of electromyographic activity over a specified time period. Both MAV and RMS reflect the intensity of muscle activity. As MAV and RMS increase, the degree of muscle fatigue also increases accordingly, and the muscle is in a state of continuous tension. MAV directly reflects the average intensity of the signal, while RMS reflects the energy intensity. The two are complementary.

[0079] iEMG (μV): Integrated electromyography value, used to calculate the cumulative intensity of muscle activity and to assess the overall load.

[0080]

[0081] Where N is the number of sample points within the time period, x i Let be the muscle activity intensity value of the i-th sample.

[0082] Frequency domain characteristics of muscle signals:

[0083] MPF(Hz): The average power frequency of the EMG in the frequency domain, obtained by power spectrum estimation based on Fourier transform.

[0084] MF(Hz): The median power frequency of EMG in the frequency domain, obtained by power spectrum estimation based on Fourier transform. The decrease in both MPF and MF corresponds to muscle fatigue.

[0085] LF / HF (ratio): The low-frequency / high-frequency power ratio of EMG in the frequency domain, obtained by power spectrum estimation based on Fourier transform. An increase in the proportion of low-frequency power in LF / HF may indicate that the muscle is too stiff.

[0086] Step 104: Determine the event type.

[0087] In the experimental design, events of varying urgency were designed as indicators of the regulatory state level, and feature indicators were selected according to different types of emergency events. The experimental design considered three emergency events: pedestrian crossing at an intersection (Event 1), pedestrian crossing on a road segment (Event 2), and pedestrian crossing in a blind spot on a road segment (Event 3). Under normal circumstances, for Event 1, vehicles need to slow down when approaching an intersection, and it is judged that the impact of a pedestrian crossing at a zebra crossing on the driver's state is relatively small. For Event 2, vehicles are driving normally on the road when a pedestrian appears, and the driver may experience emotional fluctuations due to delayed perception and reaction, thus placing this event at a higher level of regulatory state. For Event 3, drivers encounter blind spots created by parked vehicles while driving normally, losing the ability to perceive the blind spot. Therefore, the driver is already at a higher baseline level than the previous two events until the pedestrian appears, causing more intense emotional fluctuations, thus placing this event at the highest level of regulatory state.

[0088] Step 105: Divide the driving process into three stages with the takeover warning time as the zero point. The first stage is: before the takeover warning time [-6s, -3s], the second stage is: before the takeover warning time [-3s, 0s], and the third stage is: after the warning.

[0089] When selecting input data, the takeover warning moment was used as a key time point to divide the driver's driving process into three physiological data extraction phases. Physiological indicators were extracted within three ranges: the first phase was before the takeover warning moment [-6s, -3s]; the second phase was before the takeover warning moment [-3s, 0s]; and the third phase was from the issuance of the takeover warning until the driver finished decelerating. The first phase served as a baseline segment representing normal driving, the second phase was the pre-takeover warning phase caused by the event, and the third phase was the post-warning phase.

[0090] Step 106: Perform correlation analysis on the event type and the time-frequency characteristic indicators in the second and third stages to obtain the analysis results.

[0091] Step 107: Eliminate irrelevant time-frequency feature indicators based on the analysis results.

[0092] Since the guiding test of emergency event types for the regulatory status should correspond to the second and third stages after the scope is divided, and considering that the stimulation brought to the driver before and after autonomous driving takeover will weaken the state difference brought about by event types, physiological indicators in the second and third stages of effective events in the manual driving scenario are extracted for dimensionality reduction to obtain the basic indicators for quantifying the driving regulatory status. Before dimensionality reduction of the above indicators, a correlation analysis was performed on event types and physiological indicators. The results are shown in Table 1. It shows that as the event type changes, the root mean square (rms), integral electromyography (iemg), and absolute mean (mav) of electromyography signals, the mean heart rate (AVHR) and mean NN interval (AVNN) of pulse wave signals, and the low-frequency power ratio (SCL_std) of skin conductance level also change accordingly.

[0093] Table 1. Correlation analysis between event type and physiological indicators

[0094]

[0095] **: Significant correlation at the 0.01 level; *: Significant correlation at the 0.05 level

[0096] Step 108: Use the random forest algorithm to rank the importance of the removed time-frequency feature indicators.

[0097] Step 109: Select feature indicators based on the ranking results.

[0098] Random forest algorithms perform prediction and classification by constructing multiple decision trees. They are not only suitable for data prediction and classification but also effectively assess the importance of input features. Using random forests to rank feature importance is a common method for feature dimensionality reduction. Through the correspondence between (x, y), random forests train multiple decision trees to fit the model. In dimensionality reduction applications, it is also necessary to calculate the contribution of each feature across all decision trees, and the final summation is the importance score. A higher score indicates a greater impact of the feature on the model's prediction results.

[0099] The indicators selected in this application are based on the random forest algorithm. Significantly relevant physiological indicators related to event type within the extracted range are denoted as x, and event type as y. The importance of indicators x is then ranked. The results are as follows: Figure 2As shown in the figure, a comprehensive comparison reveals that the mean skin conductance level (SCL_std) and the mean absolute value of electromyography (mav) within the time-domain characteristic phase of electromyography intensity are more important. Based on the random forest importance ranking results, SCL_std and mav, two objective indicators, are selected and combined with the comprehensive driving status score as input data to quantify the regulatory status.

[0100] Step 110: Based on the aforementioned feature indicators and subjective data, the K-means clustering algorithm is used to obtain the driver's regulatory status level.

[0101] Cluster analysis was used to classify driving supervision status levels based on a comprehensive score of physiological indicators and driving status:

[0102] Clustering method selection: First, observe the data characteristics and plot a three-dimensional scatter plot of the comprehensive driving status score and two physiological indicators. Select a clustering method based on the approximate cluster shape of the scatter plot. It was found that the data points were distributed in at least two clusters, with relatively regular cluster shapes but blurred boundaries. Commonly used clustering methods include K-means clustering, hierarchical clustering, and DBSCAN clustering.

[0103] Clustering method selection: Hierarchical clustering is suitable for handling spherical or highly regular clusters. However, for loosely distributed or irregularly shaped clusters, it may lead to erroneous merging near noise or outliers. In practical applications, DBSCAN clustering results after parameter adjustment exhibit problems such as excessive cluster numbers and unbalanced sample distribution, with different sample ratios reaching 100:1, leading to low model accuracy and other issues in subsequent analysis and modeling. Therefore, considering both subjective and objective factors, K-means clustering is chosen for classifying regulatory status levels.

[0104] Clustering results and state level matching: K-means clustering was used to obtain sample sizes for different state levels, namely category 0 (1050), category 1 (871), and category 2 (107). The changes in physiological indicators correspond to the following actual physiological information: the larger the SCL_std, the greater the fluctuation in skin conductance, indicating increased sympathetic nerve activity and individual tension. MAV is used to assess changes in muscle tension. An increase in MAV indicates increased muscle tension, which is usually associated with active sympathetic nerve activity. At the same time, based on the mean of the three indicators, category 0 is at the lowest level in both SCL_std and the overall state score. For the MAV indicator, the difference between category 0 and 1 is not significant compared to category 2. Therefore, category 0 is associated with a low level of supervision. Based on the large gap between the highest values ​​of SCL_std and MAV in categories 1 and 2, and the inaccuracy of driving status due to individual differences, category 2 is considered as a high level of supervision. Category 1 corresponds to a medium level of supervision.

[0105] Based on the same inventive concept, this application also provides a driver state recognition device for implementing the driver state recognition method in the assisted driving environment described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the driver state recognition device in the assisted driving environment provided below can be found in the limitations of the driver state recognition device in the assisted driving environment described above, and will not be repeated here.

[0106] In one exemplary embodiment, such as Figure 3 As shown, a driver state recognition device in an assisted driving environment is provided, comprising:

[0107] The data acquisition module is used to acquire the driver's subjective and physiological data;

[0108] The preprocessing module is used to preprocess the physiological data;

[0109] The time-domain feature index selection module is used to select time-frequency feature indicators of the preprocessed physiological data;

[0110] The event type determination module is used to determine the event type;

[0111] The phase division module is used to divide the driving process into three phases with the takeover warning time as the zero point. The first phase is: before the takeover warning time [-6s, -3s], the second phase is: before the takeover warning time [-3s, 0s], and the third phase is: after the warning.

[0112] The correlation analysis module is used to perform correlation analysis on the event type and the time-frequency characteristic indicators in the second and third stages to obtain the analysis results.

[0113] The indicator elimination module is used to eliminate irrelevant time-frequency feature indicators based on the analysis results;

[0114] The importance ranking module is used to rank the importance of the removed time-frequency feature indicators using the random forest algorithm;

[0115] The feature selection module is used to select feature indicators based on the ranking results;

[0116] The regulatory status level determination module is used to obtain the driver's regulatory status level based on the aforementioned feature indicators and subjective data using the K-means clustering algorithm.

[0117] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 4As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores driver status recognition data in an assisted driving environment. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a driver status recognition method in an assisted driving environment.

[0118] Those skilled in the art will understand that Figure 4 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0119] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0120] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0122] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0123] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0124] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for driver state recognition in an assisted driving environment, characterized in that, The driver state recognition method in the assisted driving environment includes: Acquire driver's subjective and physiological data; The physiological data are preprocessed; Select time-frequency characteristic indicators of the preprocessed physiological data; Determine the event type; The driving process is divided into three stages with the takeover warning time as the zero point: the first stage is [-6s, -3s] before the takeover warning time, the second stage is [-3s, 0s] before the takeover warning time, and the third stage is the post-warning stage. Correlation analysis was performed on the event types and the time-frequency characteristic indicators in the second and third stages to obtain the analysis results. Irrelevant time-frequency feature indicators were eliminated based on the analysis results; The importance of the removed time-frequency feature indicators was ranked using the random forest algorithm. Feature indicators are selected based on the ranking results; Based on the aforementioned feature indicators and subjective data, the K-means clustering algorithm is used to obtain the quantitative level of the driver's regulatory status; The subjective data includes basic personal information, a driving status scale, and a status survey on scenario-related factors; the physiological data includes pulse wave signal (PPG), skin conductance signal (EDA), and muscle conductance signal (EMG); the driving status scale includes a comprehensive driving status score obtained by a five-point scale of aggression, driving aversion, fatigue tendency, hazard monitoring, and stimulation seeking. The time-frequency characteristic indicators include: Heart rate (HR), percentage of intervals with a normal heart rate interval exceeding 20ms (PNN20), standard deviation of all normal heart rate intervals within a specified time period (SDNN), root mean square (RMS) of the difference between adjacent IBI intervals within a specified time period (SD), mean (SCL_mean) of skin conductance level (SCL), standard deviation (SCL_std) of SCL, low-frequency power ratio (LF_ratio) of SCL in the frequency domain, mean absolute value (MAV) of electromyographic activity intensity within a specified time period, root mean square (RMS) of electromyographic activity intensity within a specified time period, average power frequency (MPF) of EMG in the frequency domain, median power frequency (MF) of EMG in the frequency domain, integrated electromyographic value (iEMG), and low-frequency / high-frequency power ratio (LF / HF) of EMG in the frequency domain. The event types include three categories: pedestrian crossing events at intersections, pedestrian crossing events on road sections, and pedestrian crossing events in blind spots of road sections. The regulatory status quantification level includes three levels: low, medium, and high, specifically: The low regulatory level corresponds to 2.19μV≤MAV≤111.45μV, 8.22E-04μS≤SCL_std≤0.20μS, and 56≤Comprehensive Driving Status Score≤87; The medium-level regulatory status corresponds to 2.18μV≤MAV≤79.09μV, 5.74E-04μS≤SCL_std≤0.22μS, and 78≤Comprehensive Driving Status Score≤101; The high level of supervision corresponds to 2.53μV≤MAV≤269.10μV, 4.63E-03μS≤SCL_std≤0.74μS, and 56≤Comprehensive Driving Status Score≤98.

2. The driver state recognition method in an assisted driving environment according to claim 1, characterized in that, The preprocessing of the physiological data specifically includes the following steps: A bandpass filter is used to filter the PPG and EMG signals; A band-stop filter is used to perform secondary filtering on the bandpass filtered PPG and EMG signals; A low-pass filter is used to filter the EDA signal; The Gaussian smoothing noise reduction method is used to reduce noise in the filtered EDA signal; Wavelet denoising was used to denoise the PPG and EMG signals after secondary filtering.

3. A driver status recognition device in an assisted driving environment, characterized in that, The driver status recognition device in the assisted driving environment includes: The data acquisition module is used to acquire the driver's subjective and physiological data; The preprocessing module is used to preprocess the physiological data; The time-domain feature index selection module is used to select time-frequency feature indicators of the preprocessed physiological data; The event type determination module is used to determine the event type; The phase division module is used to divide the driving process into three phases with the takeover warning time as the zero point. The first phase is: before the takeover warning time [-6s, -3s], the second phase is: before the takeover warning time [-3s, 0s], and the third phase is: after the warning. The correlation analysis module is used to perform correlation analysis on the event type and the time-frequency characteristic indicators in the second and third stages to obtain the analysis results. The indicator elimination module is used to eliminate irrelevant time-frequency feature indicators based on the analysis results; The importance ranking module is used to rank the importance of the removed time-frequency feature indicators using the random forest algorithm; The feature selection module is used to select feature indicators based on the ranking results; The regulatory status level determination module is used to obtain the quantitative level of the driver's regulatory status using the K-means clustering algorithm based on the aforementioned feature indicators and subjective data. The subjective data includes basic personal information, a driving status scale, and a status survey on scenario-related factors; the physiological data includes pulse wave signal (PPG), skin conductance signal (EDA), and muscle conductance signal (EMG); the driving status scale includes a comprehensive driving status score obtained by a five-point scale of aggression, driving aversion, fatigue tendency, hazard monitoring, and stimulation seeking. The time-frequency characteristic indicators include: Heart rate (HR), percentage of intervals with a normal heart rate interval exceeding 20ms (PNN20), standard deviation of all normal heart rate intervals within a specified time period (SDNN), root mean square (RMS) of the difference between adjacent IBI intervals within a specified time period (SD), mean (SCL_mean) of skin conductance level (SCL), standard deviation (SCL_std) of SCL, low-frequency power ratio (LF_ratio) of SCL in the frequency domain, mean absolute value (MAV) of electromyographic activity intensity within a specified time period, root mean square (RMS) of electromyographic activity intensity within a specified time period, average power frequency (MPF) of EMG in the frequency domain, median power frequency (MF) of EMG in the frequency domain, integrated electromyographic value (iEMG), and low-frequency / high-frequency power ratio (LF / HF) of EMG in the frequency domain. The event types include three categories: pedestrian crossing events at intersections, pedestrian crossing events on road sections, and pedestrian crossing events in blind spots of road sections. The regulatory status quantification level includes three levels: low, medium, and high, specifically: The low regulatory level corresponds to 2.19μV≤MAV≤111.45μV, 8.22E-04μS≤SCL_std≤0.20μS, and 56≤Comprehensive Driving Status Score≤87; The medium-level regulatory status corresponds to 2.18μV≤MAV≤79.09μV, 5.74E-04μS≤SCL_std≤0.22μS, and 78≤Comprehensive Driving Status Score≤101; The high level of supervision corresponds to 2.53μV≤MAV≤269.10μV, 4.63E-03μS≤SCL_std≤0.74μS, and 56≤Comprehensive Driving Status Score≤98.

4. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the driver state recognition method in an assisted driving environment as described in any one of claims 1-2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the driver state recognition method in the assisted driving environment as described in any one of claims 1-2.

6. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the driver state recognition method in the assisted driving environment as described in any one of claims 1-2.

Images