Driver state sensing system based on physiological index and external behavior analysis

Abstract

The invention discloses a driver state sensing system based on physiological indexes and external behavior analysis, which relates to the technical field of vehicle active safety control, and comprises a sensor group integrated in a driver contact or close area and a vehicle control part and used for collecting physiological signals and behavior characteristic signals of a driver in real time, the physiological signals at least comprise heart rate and skin electric signals, and the behavior characteristic signals at least comprise eyelid movement, head posture and steering wheel operation signals. According to the driver state sensing system, the comprehensiveness and accuracy of driver state sensing are remarkably improved by integrating physiological indexes and external behavior analysis, the limitation of single signal judgment is effectively overcome through a multi-source signal fusion mechanism, the reliability of state evaluation is ensured, and the advanced signal preprocessing technology adopted by the system is high in reliability. The data anti-interference capability is enhanced, the feature extraction precision is improved, and the robustness of the model is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle active safety control, and in particular to a driver state perception system based on physiological indicators and external behavior analysis. Background Art

[0002] Amid the rapid development of intelligent vehicles and active safety technologies, driver state awareness, as a core component of ensuring driving safety, has garnered widespread attention. Driver fatigue or distraction are significant factors in traffic accidents. Real-time, accurate perception of these states and timely intervention are crucial for reducing accident risks. Currently, driver state awareness technologies primarily focus on physiological indicator monitoring or external behavior analysis. Physiological indicators such as heart rate and electrodermal signals can reflect the driver's internal physiological changes, while external behaviors such as eyelid movement, head posture, and steering wheel operation can reflect their external driving state. The combination of these two approaches enables more comprehensive state awareness.

[0003] However, existing technologies still have limitations: some systems rely on a single type of signal for judgment, making it difficult to achieve both comprehensive and accurate state perception. In some fusion perception solutions, data preprocessing lacks robustness against interference, limiting feature extraction accuracy. Furthermore, the robustness of these models needs to be improved in distinguishing between fatigue and distraction, adapting to individual driver differences, and addressing complex driving environments. Furthermore, the hierarchical level of response mechanisms is insufficiently refined, potentially impacting the timeliness and rationality of interventions. To address this, we propose a driver state perception system based on physiological indicators and external behavioral analysis. Summary of the Invention

[0004] To solve the above technical problems, a driver status perception system based on physiological indicators and external behavior analysis is provided. This technical solution solves the above-mentioned problems of partial reliance on a single signal, lack of comprehensiveness and accuracy; weak anti-interference ability of data preprocessing and limited feature extraction accuracy; insufficient robustness of the model in distinguishing fatigue from distraction, adapting to individual differences and complex environments, and insufficient fineness of response grading, which affects the intervention effect.

[0005] In order to achieve the above objects, the technical solution adopted by the present invention is:

[0006] A driver status perception system based on physiological indicators and external behavior analysis, including:

[0007] A sensor group, integrated in areas of contact or proximity with the driver and vehicle control components, for collecting in real time physiological signals and behavioral characteristic signals of the driver, wherein the physiological signals include at least heart rate and electrodermal signals, and the behavioral characteristic signals include at least eyelid movement, head posture, and steering wheel operation signals;

[0008] The data processing module is in communication connection with the sensor group, and contains a feature extraction unit and a fusion decision unit. The feature extraction unit is used to extract time sequence features reflecting the state of the driver from the physiological signals and behavior characteristic signals. The fusion decision unit performs fusion analysis on the extracted physiological features and behavior features based on a machine learning or deep learning model, and outputs a judgment result and a level of the fatigue or distraction state of the driver.

[0009] The response execution module is in communication connection with the data processing module, and contains a hierarchical triggering unit and a closed-loop execution unit. The hierarchical triggering unit generates corresponding response instructions according to the state judgment level output by the fusion decision unit. The closed-loop execution unit executes the response instructions, so as to realize hierarchical closed-loop response from early warning prompt to active intervention of the vehicle.

[0010] The central processing module controls the data acquisition time of the sensor group, the feature extraction and fusion decision process of the data processing module, and the instruction transmission of the response execution module.

[0011] Preferably, the sensor group specifically includes:

[0012] The physiological sensor contains a heart rate monitoring unit arranged in the steering wheel holding area or the seat contact area, and is used to non-contact or contact acquisition of the heart rate and heart rate variability signals of the driver.

[0013] The skin conductance sensor arranged in the steering wheel holding area is used to detect the skin conductance activity change of the palm of the driver.

[0014] The behavior sensor contains an in-vehicle camera facing the face of the driver, which is configured as an eye movement tracking unit and is used to continuously capture the eyelid opening and closing state, blink frequency and line of sight direction of the driver.

[0015] The inertial measurement unit or depth camera arranged above the head of the driver or in the headrest area of the seat is configured as a head posture analysis unit and is used to detect the pitch, yaw angle and nodding frequency of the head of the driver.

[0016] The torque / angle sensor and grip sensor integrated in the steering column or the steering wheel body are configured as a steering wheel operation perception unit, and are used to monitor the steering angle change rate, steering angle retention stability, operation force and hand disengagement state.

[0017] The physiological sensor and the behavior sensor transmit the original data to the data processing module through the vehicle-mounted sensor network.

[0018] Preferably, the feature extraction unit of the data processing module specifically performs the following operations:

[0019] For physiological signals, the time domain and frequency domain features of heart rate variability in the heart rate signal are extracted as physiological features, and the skin conductance level change amplitude, rising rate and event-related skin electrical response features are extracted from the skin electrode signal;

[0020] For behavioral characteristic signals, based on the eyelid activity image sequence, the proportion of eyelid closure time per unit time, blink duration, blink interval, and microsleep event characteristics in which eyelid closure lasts longer than a threshold are extracted as eye movement features. Based on the head posture data, the duration and frequency of the head pitch angle exceeding the preset threshold, the duration and frequency of the head yaw angle exceeding the preset threshold, and the amplitude and frequency of the nodding movement are extracted as head posture features. Based on the steering wheel operation data, the standard deviation of the steering wheel angle, the frequency and amplitude of the steering wheel correction movement, the steering wheel stationary holding time, the duration of the hand leaving the steering wheel, and the steering wheel grip change pattern are extracted as operation features.

[0021] The extracted physiological features and behavioral features are all time series feature vectors.

[0022] Preferably, the fusion decision unit is specifically implemented as follows:

[0023] Receive the multi-source time series feature vectors output by the feature extraction unit, synchronize the physiological feature vectors with the behavioral feature vectors, and fuse them to form a fused feature vector;

[0024] The machine learning or deep learning model adopts a hybrid model including long short-term memory network and convolutional neural network components, the model takes the fused feature vector as input and outputs a classification probability or regression value representing the driver state;

[0025] The fusion decision unit applies a preset judgment threshold based on the model output value to classify the driver's status into multiple levels: normal, mild fatigue / distraction, moderate fatigue / distraction, and severe fatigue / distraction, and outputs a corresponding status judgment result and level signal;

[0026] The determination threshold is fine-tuned through model adaptive adjustment according to individual differences of drivers or driving environment.

[0027] Preferably, the hierarchical triggering unit of the response execution module generates a response instruction according to a preset strategy based on the state determination level output by the fusion decision unit:

[0028] When it is determined to be a mild state, the first-level response instruction is triggered, and a control instruction is generated to make the closed-loop execution unit issue a mild warning prompt;

[0029] When it is determined to be in a moderate state, the secondary response instruction is triggered, generating a control instruction to cause the closed-loop execution unit to issue an enhanced warning prompt accompanied by a mild vehicle intervention;

[0030] When determining as severe state, triggering three-level response instruction, generating control instruction to make closed-loop execution unit execute strong intervention measures;

[0031] The closed-loop execution unit comprises a pre-warning prompt unit for executing first-level and second-level response instructions, outputting visual, audible or somatosensory pre-warning signals through a vehicle-mounted display, a loudspeaker or a haptic device;

[0032] A vehicle active intervention unit for executing second-level and third-level response instructions, communicating with a vehicle body control system through a vehicle bus to control an actuator to realize lane-keeping auxiliary enhancement intervention, adaptive cruise speed limiting regulation, emergency brake pre-filling or triggering automatic deceleration or parking operation under safe conditions.

[0033] Preferably, the central processing module specifically comprises:

[0034] An embedded hardware platform integrating a multi-core processor, a real-time operating system, a memory and a storage unit;

[0035] A system software framework deploying and running algorithm models and response execution logic of the data processing module;

[0036] A communication interface including a dedicated interface or a vehicle-mounted bus interface connecting a sensor group, a high-speed internal bus for inter-module communication and a vehicle bus interface connecting a vehicle control network;

[0037] The central processing module schedules sensor data acquisition cycles, controls real-time execution of feature extraction and fusion decision algorithms, manages determination and response instruction generation of the hierarchical triggering logic, coordinates actions of the closed-loop execution unit and monitors running states of system components.

[0038] Preferably, the data processing module further comprises a signal pre-processing unit located before the feature extraction unit, for processing original signals collected by the sensor group:

[0039] For physiological signals, digital filtering is applied to remove power frequency interference and high-frequency noise, wavelet transform or adaptive filtering is used to suppress motion artifacts, and baseline correction is performed;

[0040] For behavioral signals, face detection and key point tracking are performed on image data to extract eyelid and head features, and de-bouncing and outlier filtering are performed on steering wheel sensor data;

[0041] The signal pre-processing unit realizes time synchronization of sensor data with different sampling rates through interpolation or resampling and marks time stamps.

[0042] Preferably, the fusion decision unit analyzes the temporal correlation and fusion feature pattern of the physiological characteristics and behavioral characteristics through its deep learning model, distinguishes the driver's fatigue state and distraction state, and determines their severity levels.

[0043] Preferably, the fusion decision unit establishes independent determination sub-models for fatigue state and distraction state respectively, and each sub-model uses the coordinated temporal variation pattern of physiological characteristics and behavioral characteristics to perform state classification.

[0044] Preferably, it also includes a self-test and calibration module for:

[0045] When the system starts, it performs a self-test process to verify the sensor function and communication link;

[0046] During the initial driving phase after the driver first uses the system or starts the vehicle, baseline physiological signals and behavioral characteristic data under normal driving conditions are collected to establish an individualized characteristic baseline;

[0047] Dynamically adjust the image sensor sensitivity or feature extraction parameters when a sudden change in ambient light intensity or a change in road type is detected;

[0048] Continuously monitor sensor data quality, diagnose signal loss or distortion, and trigger compensation or fault notifications.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The driver state perception system proposed in the present invention significantly improves the comprehensiveness and accuracy of driver state perception by integrating physiological indicators and external behavior analysis. Its multi-source signal fusion mechanism effectively overcomes the limitations of single signal judgment and ensures the reliability of state assessment. The advanced signal preprocessing technology adopted by the system enhances the data's anti-interference ability and improves the accuracy of feature extraction. Through the deep learning model, the system can accurately distinguish between the driver's fatigue and distraction states, and adaptively adjust the judgment threshold to adapt to individual differences and complex driving environments, thereby enhancing the robustness of the model. The hierarchical closed-loop response mechanism implemented by the system provides refined measures from early warning to active intervention according to the severity of the state, ensuring the timeliness and rationality of the intervention, and providing strong support for improving driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a system module diagram of the present invention;

[0052] Figure 2 It is a system workflow diagram of the present invention. DETAILED DESCRIPTION

[0053] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0054] Reference Figure 1 and Figure 2 As shown, a driver status perception system based on physiological indicators and external behavior analysis builds a closed-loop safety assurance system by deeply integrating multi-dimensional real-time monitoring and intelligent decision-making technologies. The sensor array serves as the system's sensory nerve endings, and its deployment strategy is carefully designed based on ergonomic principles and driving interaction characteristics. Regarding physiological sensors, the heart rate monitoring unit integrated into the steering wheel grip area uses reflective photoplethysmography (PPG), utilizing infrared LEDs and photodetectors beneath the steering wheel's surface to capture pulsation signals from capillaries in the fingertips or palms. This placement avoids the discomfort of the wearable device and ensures stable contact between the driver's palm and the steering wheel during normal driving, effectively reducing motion artifacts. The heart rate monitoring unit built into the seat back utilizes a piezoelectric film sensor array, capturing heartbeat signals by detecting micro-vibrations on the back, providing a redundant backup for the steering wheel solution. The galvanic skin sensor, embedded in the steering wheel rim in the form of interdigitated electrodes, operates based on the physiological mechanism of galvanic skin response (GSR): sympathetic nerve activation leads to increased sweat gland secretion, thereby increasing skin conductivity. The palm area can provide more stable and high signal-to-noise ratio skin conduction signals compared to fingers or wrists due to its rich distribution of sweat glands and continuous contact characteristics.

[0055] The behavior sensor network builds multi-dimensional driving behavior capture capability. The in-vehicle camera selects a global shutter CMOS sensor with an 850 nm near-infrared light source to ensure stable operation in scenarios with dramatic changes in light, such as day and night and tunnel entry and exit. Its built-in computer vision algorithm realizes millisecond-level face detection based on the improved Viola-Jones framework, and combines shape regression-based face key point tracking (such as the 68-point model) to accurately segment the eye region of interest (ROI). The eye movement tracking unit uses the gray value vertical projection method to calculate the eye aspect ratio (EAR), and dynamically determines the closed-eye state through the adaptive threshold. In view of the special physiological characteristics of Asians, such as single eyelid, the algorithm integrates the iris visible area ratio as an auxiliary criterion. The head posture analysis adopts a multi-sensor fusion strategy: the ceiling millimeter wave radar captures head micro-movements through the Doppler effect, and its penetration can eliminate the influence of hair obstruction; the nine-axis inertial measurement unit (IMU) built into the headrest includes a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, which converts the raw data into pitch, yaw, and roll angles through a quaternion solution algorithm. The angle resolution of the two sensors after Kalman filter fusion can reach 0.5 degrees, effectively overcoming the failure risk of pure visual solutions in strong backlight or obstruction. The steering wheel operation perception unit is composed of multiple physical quantity sensors: the non-contact reluctance torque sensor integrated into the steering column measures the steering torque applied by the driver, and its full-bridge circuit design eliminates temperature drift; the piezoresistive thin film sensor array (usually 8-16 sensing units) embedded in the steering wheel spokes generates a real-time grip distribution heat map, which distinguishes between active hand release (such as gear shifting) and disabled hand release through pattern recognition algorithms.

[0056] The signal preprocessing unit of the data processing module adopts a hierarchical processing architecture to cope with complex working conditions. The physiological signal processing chain first eliminates power interference through a 50Hz / 60Hz adaptive notch filter, and then suppresses electromyographic noise through a fourth-order Butterworth low-pass filter (cutoff frequency 35Hz). In view of the motion artifacts caused by driving bumps, the system combines the vibration frequency spectrum data of the steering wheel torque sensor and uses a normalized least mean squares (NLMS) adaptive filter for dynamic compensation. In behavior signal preprocessing, the video stream is enhanced through a contrast enhancement algorithm based on histogram equalization to improve the eyelid recognition rate in low light environments; the steering wheel sensor data uses a sliding window outlier detection method (based on median absolute deviation) to eliminate abnormal peaks caused by road shocks. The time synchronization of multiple source signals is realized through hardware timestamps combined with software interpolation: each sensor data packet carries a synchronization marker distributed by the central processing module, which is accurate to milliseconds, and the feature extraction unit uses a cubic spline interpolation method to unify data with different sampling rates (such as heart rate 1Hz, eye movement 30Hz, and steering wheel torque 100Hz) to a 100Hz time reference.

[0057] The core value of the feature extraction unit is to transform the original signal into a state indicator with physiological significance. Physiological feature extraction focuses on the analysis of autonomic nervous system activity: calculating the standard deviation of adjacent R-R intervals (SDNN) from heart rate signals reflects overall autonomic nervous tension, and the ratio of low-frequency power (0.04-0.15 Hz) to high-frequency power (0.15-0.4 Hz) (LF / HF) represents the balance between sympathetic and parasympathetic nerves; the continuous decomposition method (CDA) is used for skin conductance signal analysis to extract features such as latency, rise time, and amplitude of event-related skin conductance response (ER-SCR), with special attention to the density distribution of microseismic oscillations in the 0.2-1.0 μS range. Behavioral feature extraction is related to specific operation modes: eye movement analysis calculates the percentage of time that the eyelid is closed more than 80% per minute (PERCLOS), the average duration of eye blinks (with 100 ms as the fatigue threshold), and microsleep events (defined as a closed-eye state lasting more than 1 second); head posture features are quantified as the percentage of cumulative time with a pitch angle exceeding 20 degrees, the standard deviation of yaw angle, and the frequency of nodding (local extreme value detection based on angular velocity integration); steering wheel operation features include steering angle entropy within a 10-second window (reflecting the degree of operation disorder), coefficient of variation of grip strength, zero torque holding time (indicating operation interruption), and torque mutation rate (detecting sudden operation errors).

[0058] The fusion decision unit constructs an intelligent analysis engine for spatio-temporal feature fusion. The multi-source time series feature vector is first aligned in time by the dynamic time warping (DTW) algorithm, forming a unified dimensional spatio-temporal feature matrix. The mixed deep learning model uses a dual-flow network architecture: the physiological feature flow input contains a three-layer one-dimensional convolutional neural network (1D-CNN) with convolution kernel sizes of 5 / 3 / 3, which is used to extract multi-scale time-frequency features of heart rate variability; the behavioral feature flow uses a bidirectional long short-term memory network (Bi-LSTM) with 128 hidden units to model the long-term dependence between eye movement and steering wheel operation. The dual-flow features are weighted and concatenated in the fusion layer through an attention mechanism, and a fully connected layer outputs three parallel branches: the fatigue state classifier focuses on capturing the positive correlation between PERCLOS and head nodding frequency; the distraction state detector emphasizes the joint analysis of gaze deviation angle and steering wheel turning angle entropy; and the state severity regressor implements more detailed grade division based on continuous value prediction. The adaptive mechanism of the model includes two dimensions: the individual calibration module collects 15 minutes of baseline driving data at the beginning of vehicle startup to establish an individual parameter profile including resting heart rate, baseline skin conductance level, and typical gaze patterns; the environmental adaptation module dynamically adjusts feature weights based on light sensors (>10000 lux triggers strong light mode) and GPS road types (highway mode), such as reducing the judgment weight of gaze deviation features in strong glare environments.

[0059] The tiered strategy for the response execution module strictly adheres to functional safety standards. Level 1 (mild) activates the primary warning layer of the human-machine interface (HMI): the steering wheel grip sensor generates 3Hz pulsed haptic feedback (amplitude 0.5N), and the instrument cluster displays an amber coffee cup icon. Level 2 (medium) initiates a combined warning and mild vehicle intervention: a flashing red warning symbol is projected on the head-up display (HUD), and an 85dB intermittent chime is emitted from the car's audio system. The vehicle dynamics control system increases the Lane Keeping Assist (LKA) steering torque to 140% of the base value, and the Adaptive Cruise Control (ACC) speed limit is reduced by 15%. Level 3 (severe) triggers the active safety chain: the Electronic Stability Program (ESP) pre-pressurizes the brake lines to 5 bar, and the brake-by-wire system applies progressive braking at a deceleration of 0.3g. If the vehicle speed drops below 30 km / h and the driver does not respond, the hazard lights are automatically activated and the vehicle is pulled into the emergency lane. All response actions interact with the chassis control system in real time through the vehicle bus (CAN / FlexRay), ensuring that the end-to-end delay from status determination to response execution is strictly controlled within 250ms.

[0060] The central processing module serves as the nerve center of the system, and its embedded platform adopts a heterogeneous computing architecture: a dual-core Cortex-A72 processor runs the Linux system and deep learning inference engine, while real-time control tasks are handled by the Cortex-R5 core in lockstep mode, ensuring that key functions meet ASIL-B requirements. The system software framework is developed based on the AUTOSAR architecture, and the application layer implements modular algorithm containers. The feature extraction container executes at a frequency of 100Hz, the fusion decision container runs in a cycle of 50ms, and the response control container is refreshed at a cycle of 10ms. The communication interface adopts a layered design: sensor data is transmitted with low latency via a dedicated SPI / I2C interface, inter-module communication uses time-triggered Ethernet (TTEthernet), and vehicle control network interaction is achieved through a CAN FD gateway with a hardware security module (HSM).

[0061] The self-test and calibration module builds a full life cycle protection system. The power-on self-test (POST) process includes sensor loop impedance testing, camera focus checking, and model loading verification. The dynamic calibration mechanism performs baseline data collection after each ignition: within 10 minutes after the vehicle starts (vehicle speed > 40km / h and no sudden acceleration or deceleration), the driver's normal state parameters are recorded to establish a personalized feature baseline library. The runtime diagnostic system continuously monitors signal quality indicators (such as heart rate signal-to-noise ratio, eye tracking confidence), and automatically switches to a degraded mode when it detects continuous camera occlusion or electrode contact failure - for example, relying only on a combined analysis of steering wheel torque and heart rate variability. All abnormal events generate diagnostic trouble codes (DTCs) that comply with the ISO 14229 standard, and continuous optimization of model parameters is achieved through the OTA update mechanism.

[0062] The system significantly improves the robustness of state recognition through cross-validation of multimodal sensing. A typical case is when the system detects a sudden drop in steering wheel grip accompanied by a sudden increase in the amplitude of the skin electrode response. Combined with the fact that the torque sensor has no abnormal fluctuation characteristics, it can accurately distinguish between sudden muscle weakness (requiring emergency intervention) and the driver actively raising his hand to adjust the air conditioner (ignorable). During the engineering verification phase, its fatigue detection accuracy reached 92.3% in highway scenarios (based on the EEG gold standard), the F1 value of distraction state recognition reached 87.6%, and the false alarm rate was controlled below 1.2 times / 1,000 hours, meeting the reliability requirements of pre-installed mass production systems.

[0063] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A driver status perception system based on physiological indicators and external behavior analysis, characterized in that: include: A sensor group, integrated in areas of contact or proximity with the driver and vehicle control components, for collecting in real time physiological signals and behavioral characteristic signals of the driver, wherein the physiological signals include at least heart rate and electrodermal signals, and the behavioral characteristic signals include at least eyelid movement, head posture, and steering wheel operation signals; a data processing module, communicatively connected to the sensor group, comprising a feature extraction unit and a fusion decision unit. The feature extraction unit is configured to extract time series features reflecting the driver's status from the physiological signals and behavioral feature signals. The fusion decision unit, based on a machine learning or deep learning model, fuses and analyzes the extracted physiological and behavioral features to output a determination result and level of the driver's fatigue or distraction status; A response execution module, in communication with the data processing module, comprises a hierarchical triggering unit and a closed-loop execution unit. The hierarchical triggering unit generates corresponding response instructions based on the status determination level output by the fusion decision unit, and the closed-loop execution unit executes the response instructions to achieve a hierarchical closed-loop response from early warning prompts to active vehicle intervention; The central processing module controls the data collection timing of the sensor group, the feature extraction and fusion decision process of the data processing module, and the instruction transmission of the response execution module.

2. The driver status perception system based on physiological indicators and external behavior analysis according to claim 1 is characterized in that: The sensor group specifically includes: Physiological sensors, including a heart rate monitoring unit located in the steering wheel grip area or the seat contact area, for collecting the driver's heart rate and heart rate variability signals in a non-contact or contact manner; and a galvanic skin sensor located in the steering wheel grip area to detect changes in skin conductance activity on the driver's palm; Behavioral sensors, including an in-car camera facing the driver's face, configured as an eye tracking unit to continuously capture the driver's eyelid opening and closing status, blink frequency, and gaze direction; An inertial measurement unit or depth camera located above the driver's head or in the headrest area of ​​the seat, configured as a head posture analysis unit, is used to detect the driver's head pitch, yaw angle, and nodding frequency; A torque / angle sensor and grip force sensor integrated into the steering column or steering wheel body, configured as a steering wheel operation sensing unit, is used to monitor the steering wheel angle change rate, angle stability, operating force, and hand release status; The physiological sensor and the behavioral sensor transmit raw data to the data processing module through the vehicle-mounted sensor network.

3. The driver status perception system based on physiological indicators and external behavior analysis according to claim 1 is characterized in that: The feature extraction unit of the data processing module specifically performs the following operations: For physiological signals, the time domain and frequency domain features of heart rate variability in the heart rate signal are extracted as physiological features, and the skin conductance level change amplitude, rising rate and event-related skin electrical response features are extracted from the skin electrode signal; For behavioral characteristic signals, based on the eyelid activity image sequence, the proportion of eyelid closure time per unit time, blink duration, blink interval, and microsleep event characteristics in which eyelid closure lasts longer than a threshold are extracted as eye movement features. Based on the head posture data, the duration and frequency of the head pitch angle exceeding the preset threshold, the duration and frequency of the head yaw angle exceeding the preset threshold, and the amplitude and frequency of the nodding movement are extracted as head posture features. Based on the steering wheel operation data, the standard deviation of the steering wheel angle, the frequency and amplitude of the steering wheel correction movement, the steering wheel stationary holding time, the duration of the hand leaving the steering wheel, and the steering wheel grip change pattern are extracted as operation features. The extracted physiological features and behavioral features are all time series feature vectors.

4. The driver status perception system based on physiological indicators and external behavior analysis according to claim 1 is characterized in that: The fusion decision unit is specifically implemented as follows: Receive the multi-source time series feature vectors output by the feature extraction unit, synchronize the physiological feature vectors with the behavioral feature vectors, and fuse them to form a fused feature vector; The machine learning or deep learning model adopts a hybrid model including long short-term memory network and convolutional neural network components, the model takes the fused feature vector as input and outputs a classification probability or regression value representing the driver state; The fusion decision unit applies a preset judgment threshold based on the model output value to classify the driver's status into multiple levels: normal, mild fatigue / distraction, moderate fatigue / distraction, and severe fatigue / distraction, and outputs a corresponding status judgment result and level signal; The determination threshold is fine-tuned through model adaptive adjustment according to individual differences of drivers or driving environment.

5. The driver status perception system based on physiological indicators and external behavior analysis according to claim 1 is characterized in that: The hierarchical triggering unit of the response execution module generates a response instruction according to the preset strategy based on the status determination level output by the fusion decision unit: When it is determined to be a mild state, the first-level response instruction is triggered, and a control instruction is generated to make the closed-loop execution unit issue a mild warning prompt; When it is determined to be in a moderate state, the secondary response instruction is triggered, generating a control instruction to cause the closed-loop execution unit to issue an enhanced warning prompt accompanied by a mild vehicle intervention; When it is determined to be a severe state, the third-level response instruction is triggered, and a control instruction is generated to make the closed-loop execution unit perform strong intervention measures; The closed-loop execution unit includes: a warning prompt unit for executing the first and second level response instructions and outputting visual, auditory or somatosensory warning signals through the vehicle display, speaker or tactile device; The vehicle active intervention unit is used to execute secondary and tertiary response instructions, communicate with the body control system through the vehicle bus, and control the actuator to achieve lane keeping assist enhanced intervention, adaptive cruise speed limit adjustment, emergency brake pre-filling, or trigger automatic deceleration or parking operations under safe conditions.

6. The driver status perception system based on physiological indicators and external behavior analysis according to claim 1 is characterized in that: The central processing module specifically includes: Embedded hardware platform, integrating multi-core processor, real-time operating system, memory and storage unit; System software framework, deploying and running the algorithm model and response execution logic of the data processing module; Communication interfaces, including dedicated interfaces or vehicle bus interfaces for connecting to sensor groups, high-speed internal buses for inter-module communication, and vehicle bus interfaces for connecting to the vehicle control network; The central processing module schedules the sensor data acquisition cycle, controls the real-time execution of feature extraction and fusion decision algorithms, manages the determination and response instruction generation of hierarchical trigger logic, coordinates the actions of closed-loop execution units, and monitors the operating status of various system components.

7. The driver status perception system based on physiological indicators and external behavior analysis according to claim 1 is characterized in that: The data processing module also includes a signal preprocessing unit, which is located before the feature extraction unit and is used to process the original signal collected by the sensor group: For physiological signals, digital filtering is applied to remove power frequency interference and high-frequency noise, wavelet transform or adaptive filtering is used to suppress motion artifacts, and baseline correction is performed; For behavioral signals, face detection and key point tracking are performed on image data to extract eyelid and head features, and steering wheel sensor data is de-jittered and outlier filtered; The signal preprocessing unit synchronizes the sensor data with different sampling rates through interpolation or resampling, and marks the timestamps.

8. The driver status perception system based on physiological indicators and external behavior analysis according to claim 1 is characterized in that: The fusion decision unit analyzes the temporal correlation and fusion feature pattern of the physiological characteristics and behavioral characteristics through its deep learning model, distinguishes the driver's fatigue state and distraction state, and determines their severity levels.

9. The driver status perception system based on physiological indicators and external behavior analysis according to claim 1 is characterized in that: The fusion decision unit establishes independent judgment sub-models for fatigue state and distraction state respectively, and each sub-model uses the coordinated temporal change pattern of physiological characteristics and behavioral characteristics to classify the state.

10. The driver status perception system based on physiological indicators and external behavior analysis according to claim 1, characterized in that: Also includes self-test and calibration modules for: When the system starts, it performs a self-test process to verify the sensor function and communication link; During the initial driving phase after the driver first uses the system or starts the vehicle, baseline physiological signals and behavioral characteristic data under normal driving conditions are collected to establish an individualized characteristic baseline; Dynamically adjust the image sensor sensitivity or feature extraction parameters when a sudden change in ambient light intensity or a change in road type is detected; Continuously monitor sensor data quality, diagnose signal loss or distortion, and trigger compensation or fault notifications.

Images