Driver state sensing method based on image recognition

Abstract

The invention relates to the technical field of computer vision application, and discloses a driver state sensing method based on image recognition. According to the method, starting from acquisition of a driver face image sequence, environmental illumination interference is eliminated through multistage filtering processing, standardized image data is generated, and a stable basis is provided for subsequent analysis. Then, performing region segmentation on the standardized image data, accurately extracting key feature points of eye, mouth and head postures, constructing a dynamic feature vector set, and comprehensively capturing behavior features of the driver; and establishing a state transition probability model according to the time continuity of the dynamic feature vector set, and calculating the real-time change rate of the blink frequency, the yawn frequency and the head deflection angle so as to reflect the state change of the driver. And inputting the real-time change rate into a hierarchical decision engine, and judging that the driver is in a fatigue state, a distraction state or a normal state at present through primary threshold comparison and secondary time-space correlation analysis.

Description

Technical Field

[0001] This invention relates to the field of computer vision application technology, specifically to a driver state perception method based on image recognition. Background Technology

[0002] In the transportation system, the driver, as the core element, has a psychological and physiological state that is closely related to driving safety. A driver's emotional fluctuations, fatigue level, and level of concentration are all reflected in their behavior and facial expressions, thus affecting driving operations. For example, when a driver is fatigued, their reaction speed decreases significantly, and their ability to judge and respond to emergencies weakens, greatly increasing the risk of accidents such as lane departure and rear-end collisions. Distracted driving, such as using a mobile phone or talking to passengers, diverts the driver's attention from the road, making it impossible to detect potential dangers in time, also increasing the risk of accidents. Statistics show that a significant proportion of traffic accidents are caused by driver fatigue, distraction, and other adverse conditions. Therefore, accurately perceiving the driver's condition, promptly detecting abnormalities, and taking appropriate measures are of great significance in reducing the risk of traffic accidents and ensuring road safety.

[0003] Comparison of Contact and Non-Contact Devices: Currently, devices used for driver state perception are mainly divided into contact and non-contact types. Contact devices, such as wearable monitoring devices, can accurately acquire the driver's physiological data, such as heart rate and skin conductance, to infer the driver's fatigue and stress levels. However, these devices require the driver to wear them, making them inconvenient to use and potentially interfering with normal driving operations, causing discomfort and limiting their practical application. Non-contact devices, especially cameras, show a broader application prospect in the field of driver state perception due to their ability to directly acquire facial images and behavioral information without contact with the driver. They can capture the driver's facial expressions and head posture in real time, providing a rich data source for state analysis.

[0004] Current Status and Challenges of Image Recognition-Based Driver State Perception Methods: Most research in image recognition-based driver state perception methods focuses on a single direction: emotion recognition or behavior estimation. For example, some studies analyze facial expressions to determine a driver's emotional state, identifying emotions such as anger or anxiety that may affect driving safety; others emphasize estimating driver fatigue or attention levels by detecting behavioral features such as head movements and eye closure. However, these methods often consider only a single factor and struggle to comprehensively and accurately reflect the driver's true state.

[0005] Current recognition methods based on 2D RGB images are susceptible to interference from environmental factors such as lighting and occlusion. Under direct sunlight or backlighting, images captured by the camera may be overexposed or underexposed, making facial feature extraction difficult and affecting recognition accuracy. Furthermore, when the driver's face is partially obscured, such as by wearing sunglasses or a scarf, the performance of the recognition algorithm will significantly decrease. In addition, while methods based on convolutional neural networks or transformers perform well in image feature extraction and classification, their high computational complexity and time cost make them difficult to meet the stringent real-time requirements of actual driving scenarios. Summary of the Invention

[0006] The purpose of this invention is to provide a driver state perception method based on image recognition to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides a driver state perception method based on image recognition, the method comprising: A sequence of driver facial images is acquired, and ambient light interference is eliminated through multi-level filtering to generate standardized image data. Standardized image data is segmented into regions to extract key feature points of the eyes, mouth, and head posture, and a dynamic feature vector set is constructed. Based on the temporal continuity of the dynamic feature vector set, a state transition probability model is established to calculate the real-time rate of change of blink frequency, number of yawns, and head deflection angle. The real-time rate of change is input into the hierarchical decision engine, which then determines whether the current state is fatigued, distracted, or normal through primary threshold comparison and secondary spatiotemporal correlation analysis. When the state is determined to be fatigued or distracted, the multimodal feedback channel is activated to simultaneously adjust the sampling frequency of feature extraction and the sensitivity parameters of the decision engine. A driver behavior pattern library is built based on historical state judgment results, and the threshold comparison rules and spatiotemporal correlation weights in the hierarchical decision engine are dynamically updated. Track the displacement trajectory of feature points in consecutive image frames and correct misjudged state results through motion consistency detection; By integrating vehicle driving parameters and status determination results, a driving behavior risk assessment matrix is ​​established. Adjust the focal length and field of view of the image acquisition device based on the output of the risk assessment matrix; The corrected state determination results are iteratively optimized with the risk assessment matrix to form a closed-loop control system.

[0008] Preferably, the multi-stage filtering process includes: An adaptive median filter is used to eliminate impulse noise in the image; Brightness gradient features of facial contours are extracted using the difference of Gaussian operator; The nonlocal means algorithm is used to suppress noise interference in repetitive texture regions; By combining the confidence weights of the output results of each filter, the final standardized image data is generated.

[0009] Preferably, the key feature point extraction includes: Locate the center coordinates of the pupil and the outlines of the upper and lower eyelids in the eye area; Calculate the angle of inclination between the line connecting the corners of the mouth and the horizontal plane in the mouth area; The pitch and yaw angles of the head relative to the driver's seat are estimated using a 3D reconstruction algorithm. The above spatial coordinate parameters are converted into a dynamic feature vector set in time series form.

[0010] Preferably, the process of establishing the state transition probability model includes: The distribution of blink interval duration in ten consecutive frames of images was statistically analyzed. The threshold for measuring the duration of mouth opening during a yawn; Record the duration of the head tilt angle exceeding a critical value; The transition probability matrix of each feature parameter is constructed based on the Markov chain principle.

[0011] Preferably, the workflow of the hierarchical decision engine includes: Primary threshold comparison stage: The real-time rate of change is compared with the preset physiological characteristic thresholds item by item; Secondary spatiotemporal correlation analysis stage: Detect concurrent anomalies in eye and mouth features; The status determination result is triggered when the same anomaly occurs for three consecutive analysis cycles.

[0012] Preferably, the activation conditions for the multimodal feedback channel include: The fatigue status assessment result persists for more than five seconds without being eliminated; The result of the distracted state assessment is accompanied by an acceleration of the vehicle's lateral displacement; The switching frequency between normal and abnormal states exceeds three times per minute.

[0013] Preferably, the process of constructing the driver behavior pattern library includes: Store the average closure duration of eye features categorized by driving time period; Record typical variation curves of head deflection angle based on road type; Establish a benchmark parameter table for mouth feature detection under different lighting conditions.

[0014] Preferably, the implementation steps of the motion consistency detection include: Calculate the change in Euclidean distance between feature points in adjacent frames; Eliminate overall image displacement interference caused by vehicle bumps; Verify the biological rationale for the movement of feature points during abnormal states.

[0015] Preferably, the method for generating the driving behavior risk assessment matrix includes: Correlation analysis was performed between steering wheel angle data and head deflection angle; Compare the ratio of the actual vehicle speed to the current road speed limit; The coupling coefficient between lane keeping status and blink frequency.

[0016] Preferably, the iterative optimization process of the closed-loop control system includes: The sensitive regions of feature extraction are corrected based on the latest ten state determination results; Verify the characteristic patterns of historical misjudgment cases by combining vehicle control signals; Dynamically balance the allocation of computing resources between image processing accuracy and real-time requirements.

[0017] Compared with the prior art, the beneficial effects of the present invention are: Traditional driver state perception technologies face numerous limitations when dealing with complex driving environments. In actual driving, lighting conditions are highly variable, ranging from direct sunlight during the day to dim lighting at night, and sudden changes in light levels inside and outside tunnels. These lighting interferences can severely degrade the image quality captured by cameras, leading to overexposure, underexposure, or shadows, significantly reducing the accuracy of image-based judgments. For example, in bright sunlight during the day, highlights may appear on the driver's face, obscuring facial features and making accurate extraction difficult; at night, images may become blurry due to insufficient light, further complicating feature recognition.

[0018] Traditional methods often focus on a single behavioral characteristic of the driver, such as judging fatigue solely by detecting the degree of eye closure or judging distraction solely by head movements. These methods struggle to comprehensively and accurately assess the driver's true condition. In actual driving, drivers may simultaneously exhibit multiple abnormal states; for example, fatigue may be accompanied by distraction. Relying on a single characteristic for judgment can easily lead to overlooking crucial information, resulting in misjudgments or missed diagnoses. Furthermore, traditional systems typically use fixed parameter settings, failing to adaptively adjust parameters based on different driver habits, changes in the driving environment, and feedback information during system operation, thus failing to meet complex and ever-changing practical needs.

[0019] The driver state perception method based on image recognition proposed in this patent has significant advantages in solving the aforementioned problems. Through multi-level filtering, it can effectively eliminate ambient light interference and generate stable and reliable standardized image data. This process is like finely "polishing" the image, removing imperfections caused by lighting issues, so that subsequent feature extraction and analysis can be carried out on a high-quality data basis, providing a solid foundation for accurately judging the driver's state.

[0020] The dynamic feature vector set, combined with a state transition probability model, comprehensively captures changes in driver behavior. Like a keen observer, it not only focuses on key feature points of the driver's eyes, mouth, and head posture, but also accurately calculates the real-time rate of change in blink frequency, yawn count, and head tilt angle based on the temporal continuity of these feature points. This comprehensive and dynamic monitoring method can more fully and accurately reflect the driver's true state, greatly improving the accuracy of state judgment.

[0021] The hierarchical decision engine, through primary threshold comparison and secondary spatiotemporal correlation analysis, acts like a sophisticated decision-making brain, comprehensively considering multiple factors to screen and judge the driver's state layer by layer. This dual analysis mechanism avoids the one-sidedness of a single judgment method, further improving the accuracy of judgment and effectively reducing misjudgments and omissions.

[0022] When the system determines that the driver is fatigued or distracted, the multimodal feedback channel is activated promptly, providing clear and explicit alerts to the driver through various means such as sound, vibration, and lights, thereby attracting the driver's attention. Simultaneously, the system adjusts the sampling frequency of feature extraction and the sensitivity parameters of the decision engine, enabling the system to adapt more flexibly and accurately to different driving scenarios and changes in driver state, achieving dynamic and adaptive monitoring of the driver's condition.

[0023] A driver behavior pattern library is built based on historical state judgment results, and the threshold comparison rules and spatiotemporal correlation weights in the hierarchical decision engine are dynamically updated, giving the system the ability to "learn". It can continuously accumulate experience and automatically optimize judgment strategies according to the characteristics of different drivers and changes in the driving environment, improving the system's performance and adaptability, making the system increasingly "intelligent".

[0024] By tracking the displacement trajectories of feature points in consecutive image frames and correcting erroneous state results through motion consistency detection, this process acts like adding an "insurance" layer to the judgment results. It can promptly correct potential misjudgments, ensuring the reliability of the judgment results and providing a more reliable guarantee for driving safety. Attached Figure Description

[0025] Figure 1This is a schematic diagram illustrating the working principle of the driver state perception method based on image recognition described in this invention. Figure 2 This is a flowchart of a multi-stage filtering process; Figure 3 The flowchart for extracting key feature points. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Please see Figure 1 This invention provides a driver state perception method based on image recognition. The method includes initiating a process by acquiring a sequence of driver facial images, applying multi-level filtering to the original image data to eliminate ambient light interference, and generating standardized image data. The standardized image data undergoes region segmentation to extract key feature points of the eyes, mouth, and head posture, constructing a dynamic feature vector set. The temporal continuity of the dynamic feature vector set is used to establish a state transition probability model, calculating the real-time change rate of blink frequency, yawn count, and head deflection angle. The real-time change rate is input into a hierarchical decision engine, which sequentially determines the current state as fatigued, distracted, or normal through primary threshold comparison and secondary spatiotemporal correlation analysis. When a fatigued or distracted state is determined, a multimodal feedback channel is activated, synchronously adjusting the sampling frequency of feature extraction and the sensitivity parameters of the decision engine. Historical state determination results are used to construct a driver behavior pattern library, dynamically updating the threshold comparison rules and spatiotemporal correlation weights in the hierarchical decision engine. Feature point displacement trajectories are tracked in consecutive image frames, and motion consistency detection is used to correct misjudged state results. Vehicle driving parameters are fused with the state determination results to establish a driving behavior risk assessment matrix. The output of the risk assessment matrix adjusts the focal length and viewing angle of the image acquisition device. The corrected state determination result is then iteratively optimized with the risk assessment matrix to form a closed-loop control system.

[0028] Example 1: See Figure 2Multi-level filtering includes an adaptive median filter to eliminate impulse noise in the image. The adaptive median filter operates based on sliding window analysis, with the window size dynamically adjusted according to the local pixel intensity variance. Impulse noise is identified by comparing statistical outliers in neighboring pixel values, and the filtering operation replaces noisy pixels with the median pixel value in the neighborhood. A Gaussian difference operator extracts the brightness gradient features of the facial contour. This operator uses two Gaussian kernels with different standard deviations for image convolution. The standard deviation parameter is selected based on the image resolution and the expected contour thickness. The brightness gradient features are obtained by calculating the pixel-level difference between the Gaussian blur results, and the gradient magnitude is used to highlight the edge structure of the eyebrow, eye, and lip regions. A nonlocal mean algorithm suppresses noise interference in repetitive texture regions. This algorithm searches for image blocks with similar gray-level distributions, and similarity is calculated based on the Euclidean distance between blocks. The weighted averaging process assigns greater weight to high-similarity blocks, and repetitive texture regions, such as facial pores or beard areas, are smoothed. The confidence weights of each filter output are assigned based on historical filtering performance metrics. The confidence weight of the adaptive median filter depends on the noise density estimate. The confidence weight of the Gaussian difference operator is related to the signal-to-noise ratio of the gradient magnitude. The confidence weight of the nonlocal mean algorithm is determined by the block matching error. The weighted fusion process uses a linear combination method to generate the final standardized image data.

[0029] The standardized image data input module extracts key feature points. For the eye region, it locates the pupil center coordinates and the upper and lower eyelid contours. The pupil center coordinates are calculated from the binarized pupil region using an ellipse fitting method. The binarization process utilizes the high contrast between the pupil and iris. The upper and lower eyelid contours are fitted using an active appearance model, with training data including eyelid morphology from different ethnicities and lighting conditions. For the mouth region, it calculates the tilt angle between the line connecting the corners of the mouth and the horizontal plane. Mouth corner point detection is based on a Haar feature classifier, trained using a labeled mouth corner image dataset. The tilt angle is obtained by connecting the left and right mouth corner points to form a vector and calculating its angle with the horizontal axis. The 3D reconstruction algorithm estimates the pitch and yaw angles of the head relative to the driver's seat. This algorithm employs a stereo vision-based triangulation principle, using binocular camera calibration parameters to map 2D feature points to 3D space. The pitch and yaw angles are calculated from the rotation matrix, with the driver's seat coordinate system serving as the world coordinate system reference. Spatial coordinate parameters are converted into a dynamic feature vector set in time series form. The conversion process involves coordinate normalization and time alignment. The dynamic feature vector set contains the three-dimensional coordinates, motion velocity, and acceleration components of the feature points. The vector dimension is fixed at three times the number of feature points.

[0030] The adaptive median filter in the multi-stage filtering process has an adjustable maximum window size, which limits the computational complexity of the filter. The noise detection threshold is adaptively set based on the local statistical characteristics of the image. The standard deviation of the Gaussian difference operator follows scale space theory; a large standard deviation Gaussian kernel captures coarse contour features, while a small standard deviation Gaussian kernel preserves fine edge details. The difference results are normalized to eliminate differences in illumination intensity. The search window size and similarity block size parameters of the nonlocal mean algorithm are set according to the image resolution. High-resolution images allow for a larger search range, and similarity calculation is accelerated using Fast Fourier Transform. The confidence weight calculation uses a sliding window to evaluate the stability of the filter output, and the weight values ​​are normalized to ensure a consistent range of image data after weighted fusion. The calculation of the pupil center coordinates extracted from key feature points combines the dark pupil effect and circularity constraints. The dark pupil effect refers to the pixel value in the pupil area being significantly lower than that in the surrounding area, and the circularity constraint eliminates non-circular noise interference. The upper and lower eyelid contour lines are fitted using a point distribution model. The model parameters are reduced in dimensionality through principal component analysis, and the fitting process minimizes the error between the model points and the image edges. The calculation of the tilt angle of the corner of the mouth incorporates a robustness check to exclude abnormal angle values ​​caused by facial expression distortion. The baseline distance of the binocular camera in the 3D reconstruction algorithm is optimized to the typical distance of the driver's face. Feature point matching uses a scale-invariant feature transformation descriptor, and mismatch exclusion is based on polar geometry constraints. The time-series construction of the dynamic feature vector set employs a fixed-length sliding window, with the window length covering typical physiological action cycles. Vector elements are standardized to eliminate individual size differences. Multi-level filtering and key feature point extraction are pipelined for hardware acceleration. Image data is managed in memory using a circular buffer, and computational tasks are parallelized across multiple computing units of the image processor. The noise model update of the adaptive median filter is based on continuous inter-frame difference analysis. Gradient features of the Gaussian difference operator are used for non-maximum suppression to refine edges. The computational complexity of the non-local mean algorithm is optimized using integral images. Real-time tracking of the pupil center coordinates uses Kalman filtering to predict the motion trajectory. The upper and lower eyelid contour update mechanism avoids inter-frame jitter. Corner of the mouth point detection combined with color space information improves robustness. The 3D reconstruction algorithm incorporates inertial measurement unit data to assist in attitude estimation.

[0031] The accuracy of feature point extraction is evaluated using a reprojection error metric, which measures the deviation of 3D points projected back to the image plane. An error threshold triggers feature point reinitialization. The temporal continuity of the dynamic feature vector set is checked based on cosine similarity between vectors; outlier vectors are interpolated or removed to maintain data quality. The adaptive parameter adjustment mechanism for multi-level filtering relies on an image quality assessment module, which calculates image sharpness and contrast metrics, iteratively optimizing filtering parameters to maximize these metrics. Failure handling mechanisms for key feature point extraction include backup feature detection algorithms, based on template matching or deep learning models. The primary / backup switching logic is based on feature point confidence scores. The entire implementation is deployed on an embedded vision processor, with statically allocated memory reserved for intermediate filtering results, and the dynamic feature vector set stored in a circular buffer. Computational load balancing considers the difference in computational intensity between the filtering and feature extraction stages, with the task scheduler prioritizing the real-time requirements of feature extraction. The image acquisition driver is configured in trigger mode, with exposure parameters adjusted in conjunction with filtering. Standardized image data is uniformly formatted as YUV420 to reduce transmission bandwidth. The pupil center coordinate calculation module outputs additional pupil diameter information, which is used to assist in fatigue assessment. The upper and lower eyelid contour lines output the percentage of eyelid closure, and the corner of the mouth tilt angle outputs a confidence score. The 3D reconstruction algorithm outputs a quaternion representation of the head posture. The quaternion representation avoids the gimbal lock problem, and the timestamp of the dynamic feature vector set is strictly synchronized with the vehicle CAN bus data.

[0032] The parallel implementation of the multi-stage filtering process adopts a multi-threaded architecture, with each filter running in an independent thread and thread synchronization achieved through a barrier mechanism. Data flow between modules for key feature point extraction employs double buffering to avoid processing conflicts, and feature point coordinate transfer uses a shared memory region. Hardware acceleration of the adaptive median filter utilizes dedicated median filtering instructions from the image processor; the convolution operation of the Gaussian difference operator is optimized through separable convolution; and the similarity calculation of the non-local means algorithm uses approximate nearest neighbor search to improve speed. Ellipse fitting for pupil detection uses a random sampling consensus algorithm to exclude outliers; the active appearance model for eyelid contour fitting updates shape parameters online; and the classifier for mouth corner detection uses a cascaded structure to accelerate computation. During system operation, computational resource utilization is continuously monitored; when resource utilization exceeds a threshold, image resolution or the number of feature points is dynamically reduced. The parameter table for multi-stage filtering is dynamically queried based on ambient light sensor data; the ambient light sensor data is fused with camera exposure parameters; and the model parameters for key feature point extraction are personalized based on the driver ID. The storage format of the dynamic feature vector set is optimized to a byte-aligned layout, which improves memory access efficiency. The feature vector is transferred to the state determination module using DMA to reduce CPU usage.

[0033] Example 2: See Figure 3 The process of establishing the state transition probability model includes statistically analyzing the interval duration distribution of blinking actions in ten consecutive frames of images. Blinking action recognition is based on the time-series change in the percentage of eyelid closure, calculated from the vertical distance between the upper and lower eyelid contours. The interval duration distribution uses a histogram statistical method to record the time difference between consecutive blinking events. The distribution model fitting uses kernel density estimation to characterize blinking pattern features under different driving scenarios. The duration threshold of mouth opening during yawning is measured. Mouth opening is quantified by the ratio of the distance between the corners of the mouth to the baseline state. The baseline state is taken from the average distance between the corners of the mouth under a calm driver expression. The duration threshold is set according to the duration range of typical yawning actions in physiological studies, and the threshold dynamic adjustment mechanism is calibrated in conjunction with individual differences and environmental factors. The duration segment when the head yaw angle exceeds a critical value is recorded. The head yaw angle is extracted from the yaw angle data stream output by the 3D reconstruction algorithm. The critical value is set to a 15-degree angular displacement according to safe driving regulations. The duration segment uses a sliding window counting method to accumulate the number of consecutive frames of abnormal postures. The transition probability matrix of each feature parameter is constructed based on the Markov chain principle. The state space of the Markov chain is defined as three discrete states: normal state, fatigue state, and distraction state. The initialization of the transition probability matrix is ​​learned from the labeled dataset using the maximum likelihood estimation method. The matrix update mechanism absorbs newly generated state transition data in real time through an online learning algorithm.

[0034] The hierarchical decision engine's workflow includes a primary threshold comparison stage, which compares the real-time rate of change with preset physiological characteristic thresholds. These thresholds include blink frequency (15 times per minute), yawn frequency (5 times per hour), and head deflection angle change rate (10 degrees per second). The comparison operation employs a parallel processing architecture to simultaneously process data streams from multiple feature channels. The secondary spatiotemporal correlation analysis stage detects concurrent anomalies in eye and mouth features. The determination of concurrent anomalies is based on the overlap of the feature anomaly time windows. Eye feature anomalies refer to microsleep phenomena where eyelid closure lasts longer than 200 milliseconds, and mouth feature anomalies refer to a mouth corner distance expansion exceeding 150% of the baseline value and remaining there for more than 300 milliseconds. Spatiotemporal correlation analysis introduces a graph model to characterize the causal relationships between features. When the same type of anomaly occurs for three consecutive analysis cycles, a state determination result is triggered. The analysis cycle is set to a fixed time window of 2 seconds. The determination of the same type of anomaly uses a pattern matching algorithm to identify recurring combinations of feature anomalies. The output format of the state determination result includes information in two dimensions: state type and confidence score.

[0035] The training data for the state transition probability model comes from labeled datasets collected from multi-scenario driving experiments. These datasets include facial feature data under different driving conditions, such as urban roads, highways, and nighttime driving. Data preprocessing includes time alignment and feature normalization. A third-order Markov chain is chosen to balance model complexity and prediction accuracy. Laplace smoothing is used to smooth the state transition probability matrix, avoiding zero-probability issues. An outlier removal mechanism is introduced to statistically analyze the blink interval distribution, based on box plot rules to exclude physiologically unreasonable extreme values. The calibration process for the yawn duration threshold considers individual differences in oral cavity structure and is achieved during system initialization by guiding the driver to complete standard facial expression capture. The hierarchical decision engine's initial threshold comparison stage employs a multi-level threshold design, including warning and danger levels. The threshold comparison results generate a preliminary state probability distribution. The secondary spatiotemporal correlation analysis stage constructs a dynamic Bayesian network model. The nodes of this model correspond to different facial feature dimensions, and the network parameters are trained from historical data using the expectation-maximization algorithm. The decision-making logic for continuous analysis cycles introduces state persistence constraints, requiring that abnormal states must persist for a minimum duration before being confirmed. The output interface of the decision engine directly interfaces with the vehicle warning system, and the output data format conforms to automotive electronic system communication protocol standards.

[0036] Data exchange between the state transition probability model and the hierarchical decision engine is achieved through a shared memory region, which uses a circular buffer structure to store feature data from the latest 50 analysis cycles. The synchronization mechanism between the model update thread and the decision thread uses read-write locks to protect data consistency, and a real-time performance monitoring module tracks model inference time and decision latency metrics. The blink interval duration distribution statistics module integrates an anomaly detection algorithm to identify outlier data points caused by sensor malfunctions. The yawn duration threshold management module implements temperature compensation, adjusting the sensitivity parameters of mouth feature detection based on the in-vehicle ambient temperature. The head deflection angle recording module combines inertial measurement unit (IMU) data to improve angle measurement accuracy; IMU data and visual data are fused using Kalman filtering. The online learning of the Markov chain model employs an incremental update strategy, processing only newly generated state transition samples each time without retraining the entire model. The parallel processing architecture of the primary threshold comparison stage is deployed on a digital signal processor (DSP), whose parallel computing units simultaneously process data from multiple feature streams. The dynamic Bayesian network inference for secondary spatiotemporal correlation analysis uses an approximate inference algorithm, which reduces computational complexity while ensuring inference accuracy.

[0037] The triggering mechanism for state determination results includes anti-jitter processing logic, which eliminates transient abnormal interference to ensure the stability of state determination. The sensitivity parameters of the decision engine are dynamically adjusted according to the driving scenario; a higher sensitivity setting is used for urban road scenarios, and a lower sensitivity setting is used for highway scenarios to reduce the false alarm rate. The state definition of the state transition probability model is extended to include transition states, which characterize the intermediate process from a normal state to an abnormal state, improving model granularity. The decision rules of the hierarchical decision engine use configurable rule templates, which support modifying threshold parameters through configuration files without recompiling the code. Model data is persistently stored in flash memory to prevent data loss in the event of power failure, and the latest trained model parameters are automatically loaded when the system starts. Real-time performance monitoring data is output through the on-board diagnostic interface for analysis by a remote monitoring platform, and the system self-check module automatically verifies the working status of each component at startup. The historical record of state determination results is saved to a circular buffer, designed to store the driving data of the most recent 8 hours for post-event analysis. The model update mechanism supports over-the-air (OTA) upgrades, allowing remote updates of model parameters without recalling the vehicle.

[0038] Example 3: The activation condition of the multimodal feedback channel includes the logic for determining if the fatigue state judgment result persists for more than five seconds without being eliminated. The fatigue state judgment result originates from the output data stream of the hierarchical decision engine. The five-second duration threshold is implemented through a sliding time window counter. The continuous existence check of the fatigue state marker within the window uses a bitmap data structure to record the state sequence. The determination of the non-elimination condition is based on Boolean operations on whether the state marker has never jumped back to the normal state within the time window. The method for detecting the lateral displacement acceleration of the vehicle accompanied by the distraction state judgment result is as follows: the distraction state judgment result is timestamped with the lateral acceleration data transmitted by the vehicle's CAN bus. Lateral displacement acceleration is obtained by calculating the first derivative of the lateral acceleration. The acceleration change rate threshold is set to 0.5 meters per second squared seconds. The confirmation of the accompanying relationship uses a correlation analysis algorithm based on time overlap windows. The statistical method for the switching frequency between normal and abnormal states exceeds three times per minute is to use a circular queue to record state transition events in the most recent minute. A state transition event is defined as a jump from normal to abnormal or from abnormal to normal. The frequency is calculated by counting the number of jump events in the queue. The threshold of three times per minute corresponds to an average state switching frequency of once every twenty seconds.

[0039] The response actions after the multimodal feedback channel is activated include synchronously adjusting the sampling frequency of feature extraction and the sensitivity parameters of the decision engine. The feature extraction sampling frequency adjustment is achieved by reconfiguring the frame rate parameters of the image acquisition device, which can be dynamically adjusted between 15 and 30 frames per second. The sampling frequency increase rule follows a positive correlation adjustment strategy based on the degree of state anomaly. The modification of the decision engine's sensitivity parameters involves the threshold scaling factor in the primary threshold comparison stage and the confidence threshold in the secondary spatiotemporal correlation analysis stage. The threshold scaling factor is adjusted in the range of 0.8 to 1.2 using linear interpolation, and the confidence threshold is dynamically adjusted to a floating-point value between 0.1 and 0.9 based on the persistence of the state. The warning output module of the multimodal feedback channel integrates three modes: visual warning, auditory warning, and tactile warning. Visual warnings are implemented through changes in dashboard icon colors and text prompts; auditory warnings use multi-band sound signals to distinguish warning levels; and tactile warnings are generated by the vibration motor of the driver's seat, producing different modes of tactile stimulation.

[0040] The activation determination of the feedback channel adopts a multi-condition fusion algorithm based on weighted integrals, and the algorithm formula is expressed as follows:

[0041] Where: characters Represents the overall activation level of the feedback channel, character Indicates the condition index number (values ​​1, 2, 3), character This represents the weight coefficient of the i-th condition, character Represents the standardized measurement value of the i-th condition, character This represents the deviation compensation term for the i-th condition. Weighting coefficient. The standardized measurement value was obtained through reverse optimization using historical false alarm rate data. Normalize the original data for each condition to the 0-1 interval, and then apply the deviation compensation term. Used to eliminate dimensional differences between different conditions. When the overall activation level A exceeds the preset threshold of 0.75, the multimodal feedback channel is fully activated.

[0042] The time window for fatigue state duration determination uses a configurable length parameter, which adaptively adjusts based on vehicle speed. A longer window is used in highway driving scenarios to reduce false alarms, while a shorter window is used in urban road scenarios to improve sensitivity. The bitmap data structure update mechanism is synchronized with the image acquisition frame rate. Each video frame corresponds to a state flag bit, and bitmap shifting operations are implemented using hardware-accelerated bit manipulation instructions. Detection of vehicle lateral displacement acceleration incorporates filtering, using a first-order low-pass filter to eliminate high-frequency noise from the sensor. Acceleration differential calculation employs the central difference method to improve accuracy. The circular queue for state transition frequency statistics is implemented as a fixed-length circular buffer, with the buffer size precisely corresponding to the data storage requirements of a one-minute time span. State transition detection uses an edge-triggered mechanism to record the transition time stamp.

[0043] The feature extraction sampling frequency adjustment strategy considers system power consumption constraints. When increasing the sampling frequency, it is gradually increased in stages to avoid sudden resource fluctuations. After the normal state is restored, the sampling frequency is gradually reduced using an exponential decay method. The adjustment of the decision engine sensitivity parameters introduces a hysteresis control mechanism to prevent frequent oscillations of parameters near critical values. The parameter change rate is limited to a gradual step size of no more than 0.1 units per second. The warning output of the multimodal feedback channel implements an intensity grading strategy, with warning intensity divided into three levels: prompt, warning, and danger. Each level corresponds to a different combination of icon color, sound frequency, and vibration mode. The icon design for visual warnings follows automotive human-machine interface design specifications. The icon color transitions from green to yellow to red to indicate escalating risk, and the text prompts are dynamically generated based on the specific state type. The sound signal design for auditory warnings avoids sharp frequency stimulation, with the basic frequency range controlled between 500-2000 Hz. The sound pulse mode uses intermittent sound emission to reduce driver irritation. The vibration pattern encoding of tactile warnings adopts a combination of time and space. The time pattern includes continuous vibration, pulse vibration and rhythmic vibration, while the spatial pattern generates directional tactile cues by controlling the start and stop sequence of multiple vibration motors.

[0044] The monitoring of feedback channel activation conditions runs in a timed task of the real-time operating system. The task execution cycle is strictly aligned with the output rhythm of the image processing pipeline, and the time consistency check of the condition judgment results prevents false activation caused by momentary interference. The adjustment command of the feature extraction sampling frequency is sent to the image acquisition module through an inter-process communication mechanism. The dynamic reconfiguration process of the acquisition module ensures uninterrupted video stream continuity. The modification of the decision engine sensitivity parameters adopts atomic operations to ensure data integrity, and the parameter update timing is selected during the blank period between frames processed by the decision engine. The coordinated control of multimodal warning output is achieved through a central warning manager. The warning manager handles the priority and mutual exclusion relationship of different modal warnings to avoid information overload caused by multiple warnings being output simultaneously. During system operation, the effectiveness of the feedback channel is continuously monitored. After a warning is output, the improvement of the driver's state is tracked. If the state does not improve, the warning intensity level is upgraded. The deactivation condition setting of the feedback channel is based on the continuous stabilization time after the state returns to normal. The stabilization time threshold adopts a progressively extended strategy to prevent premature exit from the warning state. All activation events and warning operations are recorded in non-volatile memory. The recorded data includes timestamps, detailed activation condition parameters, and system response actions, supporting post-event analysis and system optimization. The overall response latency of the multimodal feedback channel is measured using a timestamp chain. The end-to-end latency from state determination to warning output is controlled within 100 milliseconds to meet real-time requirements. Testing and verification of the feedback channel utilizes a hardware-in-the-loop simulation platform. The platform simulates various driving scenarios and driver state combinations, with test cases covering boundary conditions and abnormal situations. Weighting coefficients are also included. The optimization is trained on a historical dataset using the gradient descent algorithm, with the training objective being to minimize the weighted sum of the false positive and false negative rates. Bias compensation term. The calibration is completed during the system deployment phase through on-site data acquisition. The calibration process requires the driver to perform standardized state transition actions. The resource management module of the multimodal feedback channel monitors the system's computational load. When the load is too high, it automatically reduces the computational priority of the warning function to ensure the normal operation of the core recognition function.

[0045] Example 4: The construction process of the driver behavior pattern library includes storing the average closure duration of eye features categorized by driving time period. The driving time period classification adopts a joint judgment logic based on GPS geographic location and real-time clock. The average closure duration of eye features is statistically calculated from continuously monitored eyelid movement data. The storage structure adopts a hierarchical database design, indexing data by date and time period. Typical change curves of head deflection angle are recorded according to road type. Road type identification is determined by fusing road level information provided by the vehicle navigation system with visual lane feature recognition. The typical change curves of head deflection angle are smoothly fitted to the original angle data using cubic spline interpolation. The characteristic parameters of the change curves include maximum value, minimum value, and average rate of change. A benchmark parameter table for mouth feature detection under different lighting conditions is established. Lighting condition classification is determined comprehensively using camera exposure parameters and ambient light sensor data. The benchmark parameter table for mouth feature detection includes the benchmark value of mouth corner distance, lip contour clarity threshold, and color space distribution range. The implementation steps of motion consistency detection include calculating the change in Euclidean distance between feature points in adjacent frames. This change is obtained through differential operations on feature point coordinates, which are derived from the spatial coordinate sequence output by the 3D reconstruction algorithm. The Euclidean distance calculation is accelerated using a floating-point unit to ensure real-time performance. The system also eliminates overall image displacement interference caused by vehicle bumps. Vehicle bump interference is identified based on a rigid body transformation model of feature point motion. Overall image displacement is detected through motion consistency analysis of background feature points. The interference elimination algorithm uses a random sampling consensus algorithm to separate the real facial motion from the vehicle bump components. Finally, the biological rationality of feature point motion during abnormal states is verified. This verification is based on kinematic boundaries set according to human anatomical constraints. The verification of the biological rationality of feature point motion includes maximum motion velocity checks and motion acceleration continuity checks.

[0046] The driver behavior pattern database adopts a relational database structure, containing three main tables: driving time, road type, and lighting conditions. These tables are linked via foreign keys. The average eye closure duration is statistically analyzed using a sliding window mechanism with a five-minute interval, and the results are updated and stored in minutes. Typical head deflection angle curves are represented parametrically, storing curve features as Fourier descriptor coefficients to reduce storage space. An adaptive update mechanism is established for the benchmark parameter table for mouth feature detection, automatically adjusting parameters by comparing the current detection results with historical benchmarks. Outlier filtering is introduced for calculating Euclidean distance changes in motion consistency detection, with dynamic thresholds set based on the statistical distribution of historical distance changes. Compensation for vehicle bump interference is aided by inertial measurement unit (IMU) data, which is fused with visual feature point motion data using a Kalman filter. The kinematic model for biological validity verification includes motion constraints for 27 facial keypoints, predefined based on muscle connectivity and skeletal movement limitations.

[0047] Referring to Table 1, the driver behavior pattern database update strategy adopts an incremental learning approach. This approach processes only newly generated valid data samples each time, avoiding the computational overhead of retraining with the entire dataset. The motion consistency detection results are fed back to the state determination module, and these results are used to correct the output confidence of the hierarchical decision engine. The database query interface is optimized to support real-time concurrent access, and a read-write lock mechanism is used to ensure data consistency. The smoothing of feature point motion trajectories uses a bidirectional filtering algorithm, which considers both temporal and spatial continuity constraints.

[0048] Table 1: Driver Behavior Pattern Database Parameter Table

[0049] The data acquisition for the driver behavior pattern database includes a quality control phase, which checks the completeness and rationality of the data, marking invalid data and excluding it from training samples. The motion consistency detection algorithm employs a multi-scale analysis strategy, validating motion consistency at different levels of the image pyramid. Differential coding technology is used for database compression storage, reducing storage redundancy across consecutive time periods. Biological constraints on feature point motion are based on internationally standardized anthropometric data; violations of these constraints trigger data quality warnings. The construction process of the driver behavior pattern database integrates data anonymization, removing personally identifiable information while retaining behavioral feature patterns. The motion consistency detection calculation outputs a confidence score, which is used for weighted fusion of multi-source decision information. The database backup mechanism employs an incremental backup strategy, with daily scheduled backups ensuring data security. The sliding window size for feature point trajectory analysis is dynamically adjusted based on vehicle speed, using a larger window at high speeds to increase stability.

[0050] Access control for the driver behavior pattern library is divided into three levels: general read access, data update access, and system management access. Real-time motion consistency detection is guaranteed through a dedicated hardware accelerator, which handles matrix operations on feature point coordinates. The database index structure uses a B+ tree to optimize query performance, supporting range queries and fast location. The feature point motion analysis caching mechanism stores the calculation results of the most recent 100 frames, avoiding redundant calculations and improving efficiency. The driver behavior pattern library's data export function supports standard formats, including CSV and JSON. The motion consistency detection debugging interface outputs intermediate calculation results and is accessed through a dedicated debugging port. Database storage encryption uses the AES-256 algorithm, with the encryption key managed by a hardware security module. A visualization tool for feature point motion trajectories provides an intuitive analysis interface and runs on the accompanying host computer software.

[0051] The construction process of the driver behavior pattern library and the implementation steps of motion consistency detection exchange data through shared memory, with a double-buffering mechanism in the shared memory area to avoid access conflicts. A data synchronization mechanism ensures timestamp alignment between the two modules, achieving millisecond-level accuracy. An error handling mechanism detects anomalies in data transmission, triggering an automatic recovery process. System resource monitoring tracks memory usage and computational load in real time, automatically triggering data cleanup and optimization operations when resources are insufficient. The long-term learning capability of the driver behavior pattern library is achieved through a neural network model, which automatically extracts the temporal evolution patterns of behavioral features. Adaptive improvements to motion consistency detection are based on feedback learning from historical detection results, adjusting detection parameters and threshold settings. The collaborative work of the two modules forms a closed-loop optimization system, continuously improving the accuracy and reliability of state determination. Comprehensive monitoring of the system's operational status ensures the stable operation of each component, with monitoring data uploaded to a cloud analysis platform via the vehicle network.

[0052] Example 5: The method for generating the driving behavior risk assessment matrix includes specific operations for correlation analysis between steering wheel angle data and head yaw angle. Steering wheel angle data is collected from the control unit of the electric power steering system at a frequency of 50 sampling points per second. The head yaw angle is calculated and output in real time using the three-dimensional reconstruction algorithm described in Example 1. The correlation analysis uses a sliding window Pearson correlation coefficient, with a window length set to 200 sampling points corresponding to a 4-second time span. The correlation coefficient threshold is set to 0.6 to determine whether a significant correlation exists. The calculation process compares the ratio of the actual vehicle speed to the current road speed limit. The actual vehicle speed is measured by pulse counting from wheel speed sensors, and the current road speed limit information is obtained from a high-precision map database combined with GPS positioning data. The formula for calculating the ratio of the difference is... The proportion is set in three levels: below 10% is the green safety zone, 10%-20% is the yellow warning zone, and above 20% is the red danger zone. The method for determining the coupling coefficient between lane keeping status and blink frequency is as follows: lane keeping status is quantified by identifying lane line deviation distance using a forward-looking camera; blink frequency is calculated by counting the reciprocal of the interval duration in the eyelid movement sequence; and the coupling coefficient is calculated using grey relational analysis to determine the correlation between the two time-series data, with the correlation value fluctuating between 0 and 1.

[0053] The iterative optimization process of the closed-loop control system includes the specific implementation of correcting the sensitive regions extracted from feature extraction based on the latest ten state judgment results. These results are stored in a circular queue data structure, with the oldest result being pushed out each time a new result enters. Sensitive region correction is based on the correlation analysis between the state judgment results and the success rate of feature point detection. The correction operation is achieved by adjusting the coordinates of the region of interest in image processing. The analysis process combines vehicle control signals to verify the characteristic patterns of historical misjudgment cases. Vehicle control signals include brake pedal travel, accelerator opening, and turn signal status. Historical misjudgment cases are extracted from system logs, including false alarms and missed alarms. Feature pattern analysis uses a decision tree algorithm to mine common patterns of facial features in misjudgment cases. A scheduling strategy for dynamically balancing image processing accuracy and real-time requirements is implemented. Image processing accuracy is controlled by adjusting image resolution and the number of feature points, while the real-time requirement is set to a maximum processing latency of 100 milliseconds. Computational resource allocation uses a priority-based dynamic scheduling algorithm with a scheduling cycle of 1 second. Taking a specific example of highway driving, when the vehicle is traveling at 100 km / h, the correlation analysis between the steering wheel angle and head yaw angle in the driving behavior risk assessment matrix detected a high correlation coefficient of 0.75, indicating that the driver's head yaw and steering wheel operation are highly synchronized, which is within the normal driving mode. The difference between the actual vehicle speed and the road speed limit is calculated to be +5%, which is in the green safety zone. The coupling coefficient between lane keeping status and blink frequency is 0.3, indicating a low correlation. At this time, the iterative optimization process of the closed-loop control system gradually narrows the sensitive area of ​​feature extraction to the core area of ​​the eye based on the results of the most recent ten "normal state" judgments. Vehicle control signal verification shows that all state judgments match the driving operation without any misjudgments. Computational resource allocation reduces the image processing precision to 720p resolution to ensure the real-time performance of the system.

[0054] In another example of a frequently stop-and-go urban road scenario, the driving behavior risk assessment matrix detected that the correlation coefficient between steering wheel angle and head yaw angle dropped to 0.2, while the ratio of the actual vehicle speed to the speed limit fluctuated between -15% and +20%, and the coupling coefficient between lane keeping status and blink frequency increased to 0.7. The closed-loop control system dynamically adjusts the sensitive area coverage for the eyes and mouth based on the mixed state judgment results. The temporal correlation analysis between the vehicle's emergency braking signal and the "distraction state" judgment results reveals specific characteristic patterns. Computational resource allocation is increased to 1080p resolution while simultaneously increasing processor frequency. The generation of the driving behavior risk assessment matrix employs a multi-threaded parallel computing architecture. Steering wheel angle data preprocessing includes signal filtering and outlier removal steps, head yaw angle data undergoes Kalman filtering for smoothing, and correlation analysis calculations use a fast Fourier transform acceleration algorithm. A caching mechanism is established for acquiring road speed limit data to reduce database query latency, and an inertial weighting factor is introduced in the difference ratio calculation to avoid drastic numerical fluctuations. The calculation of lane keeping status includes lane line recognition confidence weighting, and the blink frequency statistics adopt an adaptive threshold to eliminate the influence of individual differences. The coupling coefficient calculation is implemented in a sliding window real-time update mode.

[0055] The iterative optimization process of the closed-loop control system is implemented using a negative feedback regulation mechanism. The correction amount for sensitive areas extracted by feature extraction is calculated by a PID controller, and the adjustment step size of the sensitive area coordinates is limited to ±10 pixels. The feature pattern analysis module for historical misjudgment cases includes a case clustering function. Case clustering is based on the feature vector distance using the K-means algorithm, and pattern extraction is achieved through association rule mining. Dynamic balancing of computing resource allocation uses a load monitoring algorithm, which tracks CPU utilization and memory bandwidth in real time. The resource allocation strategy queries a pre-set allocation strategy table based on the system load status. The output of the driving behavior risk assessment matrix is ​​formatted as a standard JSON data packet. The JSON data packet contains three fields: risk level, risk score, and risk dimension details. The data packet is broadcast to each subsystem via the vehicle Ethernet. The control commands of the closed-loop control system are distributed using a publish-subscribe model. The feature extraction module subscribes to sensitive area update commands, the image acquisition module subscribes to resolution adjustment commands, and the decision engine subscribes to parameter update commands. During system operation, the accuracy index of the risk assessment matrix is ​​continuously recorded. The accuracy index is calculated by comparing with actual driving events, and the assessment results are fed back to the adaptive adjustment loop of the matrix parameters.

[0056] In a specific example of a tunnel passage scenario, the driving behavior risk assessment matrix handles sudden changes in lighting conditions. The matrix input data includes a sudden drop in ambient light intensity and automatic headlight activation signals. The risk assessment results temporarily switch to a conservative mode to improve sensitivity. The closed-loop control system simultaneously activates a lighting compensation scheme, expanding the sensitive area of ​​feature extraction to include the entire facial region. Image processing accuracy is maintained at a minimum to ensure processing speed, and historical misjudgment case feature pattern analysis ignores short-term abnormal lighting data. A nighttime highway driving example demonstrates the system's workflow in fatigue driving recognition. The driving behavior risk assessment matrix detects a decrease in steering wheel micro-correction movements and a loss of correlation with head posture. The actual vehicle speed fluctuates around the set cruise speed, and lane keeping status shows the vehicle slowly drifting within the lane. Based on the "fatigue state" judgment result, the closed-loop control system records and adjusts the sensitive area to focus on monitoring eye features. Vehicle control signal analysis shows that prolonged non-braking operation verifies the state judgment, and computing resources are reallocated to prioritize ensuring the accuracy of blink frequency calculation.

[0057] The generation process of the driving behavior risk assessment matrix includes a data quality inspection step. Steering angle signal quality is tested through signal variance detection, head deflection angle data is filtered using confidence scores, and road speed limit data is verified through multi-source validation to ensure accuracy. The iterative optimization process of the closed-loop control system establishes a version management mechanism, recording the version number and timestamp for each parameter modification, and the system supports rapid rollback to a previous stable version. The data flow between matrix generation and control system optimization adopts an asynchronous communication mode, avoiding blocking issues caused by processing speed mismatches. Data integrity is guaranteed through a checksum mechanism. The system's performance under complex weather conditions demonstrates robustness. In heavy rain, the driving behavior risk assessment matrix is ​​adjusted in conjunction with the wiper operating frequency to change the risk assessment model. The closed-loop control system activates an anti-interference mode to reduce the accuracy requirements of sensitive areas in feature extraction, and historical misjudgment case feature pattern analysis filters weather-related abnormal patterns. In winter icy road scenarios, the matrix input includes ESP system intervention frequency data, and control system optimization focuses on the collaborative analysis of vehicle dynamics and driver status. The update cycle of the driving behavior risk assessment matrix is ​​synchronized with the vehicle's CAN bus signal, and matrix parameters dynamically switch according to the driving mode (Economy / Sport / Comfort). The iteration optimization speed of the closed-loop control system is positively correlated with the vehicle speed. When driving at high speed, the optimization cycle is shortened, which improves the system responsiveness.

[0058] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0059] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A driver state perception method based on image recognition, characterized in that, Includes the following steps: A sequence of driver facial images is acquired, and ambient light interference is eliminated through multi-level filtering to generate standardized image data. Standardized image data is segmented into regions to extract key feature points of the eyes, mouth, and head posture, and a dynamic feature vector set is constructed. Based on the temporal continuity of the dynamic feature vector set, a state transition probability model is established to calculate the real-time rate of change of blink frequency, number of yawns, and head deflection angle. The real-time rate of change is input into the hierarchical decision engine, which then determines whether the current state is fatigued, distracted, or normal through primary threshold comparison and secondary spatiotemporal correlation analysis. When the state is determined to be fatigued or distracted, the multimodal feedback channel is activated to simultaneously adjust the sampling frequency of feature extraction and the sensitivity parameters of the decision engine. A driver behavior pattern library is built based on historical state judgment results, and the threshold comparison rules and spatiotemporal correlation weights in the hierarchical decision engine are dynamically updated. Track the displacement trajectory of feature points in consecutive image frames and correct misjudged state results through motion consistency detection; By integrating vehicle driving parameters and status determination results, a driving behavior risk assessment matrix is ​​established. Adjust the focal length and field of view of the image acquisition device based on the output of the risk assessment matrix; The corrected state determination results are iteratively optimized with the risk assessment matrix to form a closed-loop control system.

2. The driver state perception method based on image recognition according to claim 1, characterized in that, The multi-stage filtering process includes: An adaptive median filter is used to eliminate impulse noise in the image; Brightness gradient features of facial contours are extracted using the difference of Gaussian operator; The nonlocal means algorithm is used to suppress noise interference in repetitive texture regions; By combining the confidence weights of the output results of each filter, the final standardized image data is generated.

3. The driver state perception method based on image recognition according to claim 2, characterized in that, The key feature point extraction includes: Locate the center coordinates of the pupil and the outlines of the upper and lower eyelids in the eye area; Calculate the angle of inclination between the line connecting the corners of the mouth and the horizontal plane in the mouth area; The pitch and yaw angles of the head relative to the driver's seat are estimated using a 3D reconstruction algorithm. The above spatial coordinate parameters are converted into a dynamic feature vector set in time series form.

4. The driver state perception method based on image recognition according to claim 3, characterized in that, The process of establishing the state transition probability model includes: The distribution of blink interval duration in ten consecutive frames of images was statistically analyzed. The threshold for measuring the duration of mouth opening during a yawn; Record the duration of the head tilt angle exceeding a critical value; The transition probability matrix of each feature parameter is constructed based on the Markov chain principle.

5. The driver state perception method based on image recognition according to claim 4, characterized in that, The workflow of the hierarchical decision engine includes: Primary threshold comparison stage: The real-time rate of change is compared with the preset physiological characteristic thresholds item by item; Secondary spatiotemporal correlation analysis stage: Detect concurrent anomalies in eye and mouth features; The status determination result is triggered when the same type of anomaly occurs for three consecutive analysis cycles.

6. The driver state perception method based on image recognition according to claim 5, characterized in that, The activation conditions for the multimodal feedback channel include: The fatigue status assessment result persists for more than five seconds without being eliminated; The result of the distracted state assessment is accompanied by an acceleration of the vehicle's lateral displacement; The switching frequency between normal and abnormal states exceeds three times per minute.

7. The driver state perception method based on image recognition according to claim 6, characterized in that, The process of constructing the driver behavior pattern library includes: Store the average closure duration of eye features categorized by driving time period; Record typical variation curves of head deflection angle based on road type; Establish a benchmark parameter table for mouth feature detection under different lighting conditions.

8. The driver state perception method based on image recognition according to claim 7, characterized in that, The implementation steps of the motion consistency detection include: Calculate the change in Euclidean distance between feature points in adjacent frames; Eliminate overall image displacement interference caused by vehicle bumps; Verify the biological rationale for the movement of feature points during abnormal states.

9. The driver state perception method based on image recognition according to claim 8, characterized in that, The method for generating the driving behavior risk assessment matrix includes: Correlation analysis was performed between steering wheel angle data and head deflection angle; Compare the ratio of the actual vehicle speed to the current road speed limit; The coupling coefficient between lane keeping status and blink frequency.

10. The driver state perception method based on image recognition according to claim 9, characterized in that, The iterative optimization process of the closed-loop control system includes: The sensitive regions of feature extraction are corrected based on the latest ten state determination results; Verify the characteristic patterns of historical misjudgment cases by combining vehicle control signals; Dynamically balance the allocation of computing resources between image processing accuracy and real-time requirements.

Images