A non-contact driver state recognition method, system and medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TIANJIN HAISHI INTELLIGENT TECHNOLOGY CO LTD
- Filing Date
- 2026-07-07
- Publication Date
- 2026-08-07
AI Technical Summary
[0003]目前，现有的检测方式多为接触式的，通过佩戴手环等仪器来监测，操作繁琐且易干扰驾驶动作，不适用于实时驾驶场景
本发明的方法采用图像采集的方式，无需驾驶员接触设备，适配各类驾驶场景，不干扰驾驶动作；基于通过人脸特征提取生成rPPG信号，基于rPPG信号生成生理特征参数，通过对生理特征参数进行情绪识别，在确保生理参数提取准确率的同时，能够通过综合参数有效识别驾驶员的情绪，解决了现有识别维度少的问题；有利于后续的预警提示以及驾驶员的安全保障。
Smart Images
Figure CN122530993A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology, specifically to a non-contact driver status recognition method, system, and medium. Background Technology
[0002] A driver's physical health and emotional state directly affect driving safety; fatigue and emotional instability are major causes of traffic accidents. Drivers also frequently experience abnormal physical symptoms and emotional fluctuations due to prolonged driving and complex road conditions.
[0003] Currently, most existing detection methods are contact-based, relying on devices such as wristbands for monitoring. These methods are cumbersome to operate and can easily interfere with driving actions, making them unsuitable for real-time driving scenarios. Furthermore, existing non-contact methods mostly focus on single-parameter early warning analysis, resulting in low parameter extraction accuracy and failing to meet practical application needs. Summary of the Invention
[0004] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a non-contact driver status recognition method.
[0005] To achieve the above objectives, the present invention provides the following technical solution: A contactless driver status recognition method, the method comprising: Acquire facial images; The face image is preprocessed; Feature extraction is performed on the preprocessed face image to generate physiological feature parameters, which include one or more of heart rate, blood oxygen saturation, respiratory rate, and HRV features; The physiological feature parameters are input into the state recognition model to generate state reference data, which includes emotion category and level. The feature extraction of the preprocessed face image includes: Channel-wise spatial averaging is performed on the skin pixels within the target region of the preprocessed face image to generate the original time series R(t), G(t), and B(t); Modeling RGB signals based on a dichroic reflection model: And generate rPPG physiological signals; Where I(t) is the RGB channel light intensity signal acquired at time t; DC(t) is the DC slow converter component; AC(t) is the AC light intensity amplitude that fluctuates with the heartbeat and is the effective component of the rPPG signal; ω is the angular frequency corresponding to the heartbeat. This refers to the signal phase shift.
[0006] In this invention, preferably, the method further includes: The status reference data is classified into levels, and graded early warning signals are generated based on the levels.
[0007] In this invention, preferably, acquiring the face image includes: A facial image is generated by acquiring no fewer than 200 consecutive images of the driver's face and a fixed area above his head using an image acquisition device.
[0008] In this invention, preferably, the preprocessing of the face image includes: The face image is then filtered and color corrected. The processed face image is used to locate features, determine the target region, and optimize the target region. Calculate the variance of facial feature point coordinates within the target area, and select more than 200 consecutive valid images with no significant motion noise.
[0009] In this invention, preferably, the step of calculating the variance of facial feature point coordinates within the target area and filtering out more than 200 consecutive valid images with no significant motion noise includes: Coordinate detection is performed on facial feature points in each frame of the acquired continuous images, and 5 facial feature points are selected as the basis for motion detection. Select the first window with a time length of tu in the continuous image, which contains continuous Q-frame images; The first window is divided into tw sub-windows at preset intervals; Calculate the x-variance and y-variance of the facial feature point coordinates within each sub-window to obtain the variance value for each group; If the number of sub-window groups with variance values less than a preset threshold is greater than or equal to a preset number, the Q-frame images of the first window are determined as valid images with more than 200 consecutive frames; otherwise, the window is slid along the time sequence with a step size step, and the above steps of selecting a first window with a time length of tu in the continuous images, including continuous Q-frame images, are repeated to calculate the x-variance and y-variance of the coordinates of the facial feature points in each sub-window to obtain the variance value of each group.
[0010] In this invention, preferably, the physiological characteristic parameters include one or more of heart rate, blood oxygen saturation, respiratory rate, and HRV characteristics. Based on the rPPG physiological signal, the physiological characteristic parameters are generated, including: The AC dynamic component signal of the rPPG physiological signal is acquired, and an observation model of a single-frequency real sinusoidal signal is established based on the AC dynamic component signal. The heart rate is estimated and generated using an improved covariance algorithm; and / or The amplitude ratio (R / G, B / G) and phase difference of the rPPG physiological signal are obtained, and the blood oxygen saturation is calculated using a model of hemoglobin's absorption characteristics for different wavelengths of light; and / or Acquire the low-frequency fluctuation signal of the rPPG physiological signal and analyze the low-frequency fluctuation signal to generate the respiratory rate; and / or The time-domain features, frequency-domain features, and nonlinear features of the rPPG physiological signal are extracted to generate the HRV features.
[0011] In this invention, preferably, the state recognition model adopts a trained SVM model, and the training process of the SVM model includes: rPPG physiological signals of drivers under different emotional states were collected, and the rPPG physiological signals were labeled with emotion type and level to construct a training set; The SVM model is trained using 10-fold cross-validation based on the training set. The SVM model is then optimized using grid search, resulting in the trained SVM model.
[0012] The present invention also provides a contactless driver status recognition system, the system comprising: The acquisition module is used to acquire facial images; An image preprocessing module is used to preprocess the face image; The parameter extraction module is used to extract features from the preprocessed face image and generate physiological feature parameters. The state recognition module is used to input the physiological feature parameters into the state recognition model to generate state reference data; The parameter extraction module is also used for: Channel-wise spatial averaging is performed on the skin pixels within the target region of the preprocessed face image to generate the original time series R(t), G(t), and B(t); Modeling RGB signals based on a dichroic reflection model: And generate rPPG physiological signals; Where I(t) is the RGB channel light intensity signal acquired at time t; DC(t) is the DC slow converter component; AC(t) is the AC light intensity amplitude that fluctuates with the heartbeat and is the effective component of the rPPG signal; ω is the angular frequency corresponding to the heartbeat. This refers to the signal phase shift.
[0013] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the non-contact driver state recognition method described above.
[0014] Compared with the prior art, the beneficial effects of the present invention are: The method of this invention uses image acquisition, which eliminates the need for the driver to contact the device, adapts to various driving scenarios, and does not interfere with driving actions. Based on the generation of rPPG signals through facial feature extraction, physiological feature parameters are generated from the rPPG signals. By performing emotion recognition on the physiological feature parameters, the method can effectively identify the driver's emotions through comprehensive parameters while ensuring the accuracy of physiological parameter extraction, thus solving the problem of limited recognition dimensions in existing methods. This is beneficial for subsequent early warning prompts and driver safety. Attached Figure Description
[0015] Figure 1 This is a flowchart illustrating the non-contact driver status recognition method described in this invention.
[0016] Figure 2 This is a schematic diagram of the installation of the image acquisition device described in this invention.
[0017] Figure 3 This is a schematic diagram of the location of facial feature points as described in this invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0020] Please see Figure 1 A preferred embodiment of the present invention provides a non-contact driver status recognition method, the method comprising: S1. Obtain the face image; S2. Preprocess the face image; S3. Extract features from the preprocessed face image to generate physiological feature parameters, including one or more of the following: heart rate, blood oxygen saturation, respiratory rate, and HRV. S4. Input the physiological feature parameters into the state recognition model to generate state reference data, which includes emotion category and level.
[0021] In this embodiment, the method further includes: S5. Divide the status reference data into levels and generate graded early warning signals based on the levels.
[0022] In this embodiment, acquiring the face image includes: The image acquisition device captures at least 200 consecutive images of the driver's face and a fixed area above his head.
[0023] Please see Figure 2 Specifically, the image acquisition device can be a black light camera, fixed approximately 10cm above the vehicle's dashboard, with a distance of 50cm-100cm from the driver's face. It can also be installed on the A-pillar or below the rearview mirror, using the same installation method as the vehicle's DMS or OMS. The shooting angle is 15°-30° upward, with the camera's deviation angle from the face not exceeding ±20°. The frame rate is 30-60fps, ensuring clear acquisition of continuous full-color images of the driver's face and the fixed background area above their head. Each acquisition should contain at least 200 valid full-color face and background images. The black light camera can output high-quality full-color images in low-light or dark environments, adapting to different driving scenarios and requiring no additional light source.
[0024] In this embodiment, step S2, the preprocessing of the face image, includes: S21. Perform filtering and color correction processing on the face image; S22. Perform feature localization on the processed face image, determine the target region, and optimize the target region; S23. Calculate the variance of facial feature point coordinates within the target area and filter out more than 200 consecutive valid images without significant motion noise.
[0025] Specifically, continuously acquired full-color face images are subjected to a 5×5 kernel Gaussian filter and a 3×3 kernel median filter to suppress noise from the image acquisition device. Then, white balance and gamma correction are used to eliminate the effects of illumination nonlinearity in the face images, completing color correction and ensuring the stability of the RGB signals in the face images. Please refer to [link to relevant documentation]. Figure 3Next, the face in the face image is detected using the Dlib algorithm, and 34 feature points are located. The envelope regions formed by connecting feature points 14, 0, 4, and 13, and the envelope regions formed by connecting feature points 17, 21, 31, and 35 are selected as target regions. Skin segmentation is performed using a YCbCr skin color threshold, where Cb∈[77,127] and Cr∈[133,173], to exclude non-skin regions such as hair, eyes, and lips. A U-Net network is used for pixel-level skin masking to further exclude highlight and shadow regions, achieving accurate skin region selection. Finally, the variance of the facial feature point coordinates in each frame of the face image is calculated. Based on this variance, it is compared with a set threshold, and the face images in frames with coordinate variances within the threshold range are marked as valid images. All frames are traversed, and more than 200 consecutive valid images are selected as face images for subsequent processing.
[0026] In this embodiment, step S23, calculating the variance of facial feature point coordinates within the target area and filtering out more than 200 consecutive valid images with no significant motion noise, includes: S231. Perform coordinate detection on the facial feature points in each frame of the acquired continuous images, and select 5 facial feature points as the basis for motion detection, including 2 on the left, 1 in the middle, and 2 on the right. S232. Select a first window with a time length of tu in the continuous image, which contains Q consecutive frames of images, where Q≥200; S233. Divide the first window into tw sub-windows at a preset interval, with each sub-window having a duration of 1 second; S234. Calculate the x-variance and y-variance of the coordinates of the facial feature points within each sub-window to obtain the variance values for each group. S235. If the number of sub-window groups with variance values less than a preset threshold is greater than or equal to a preset number, the Q-frame images of the first window are determined as valid images with more than 200 consecutive frames; otherwise, the window is slid along the time sequence with a step size step, and the above steps of selecting a first window with a time length of tu in the continuous images, including continuous Q-frame images, to calculating the x-variance and y-variance of the coordinates of the facial feature points in each sub-window, and obtaining the variance value of each group are repeated.
[0027] In this embodiment, the feature extraction of the preprocessed face image in step S3 includes: S31. Perform channel-wise spatial averaging on the skin pixels within the target region of the preprocessed face image to generate three original time series: R(t), G(t), and B(t). ; Where G(t) is the green channel brightness value, N is the effective skin pixel value in the target area, and i is the traversal index of the effective skin pixel. i (t) represents the green channel brightness value of the i-th effective skin pixel at time t. Similarly, the red channel signal R(t) and the blue channel signal B(t) are obtained. By spatial averaging, single-point pixel noise is suppressed, thereby improving the signal-to-noise ratio of the rPPG signal.
[0028] S32. Modeling RGB signals based on a dichroic reflection model: ; Where I(t) is the RGB channel light intensity signal acquired at time t; DC(t) is the DC slow-varying component, which contains static / slow-varying information such as skin tissue, average blood volume, ambient light, and skin color, and needs to be removed; AC(t) is the AC light intensity amplitude that fluctuates with the heartbeat, which is the effective component of the rPPG signal; ω is the angular frequency corresponding to the heartbeat, ω=2πf, where f is the fundamental frequency of the heart rate; This is for signal phase shift; the green channel G is most sensitive to hemoglobin absorption and has the highest signal intensity. This invention uses the G channel as the core feature extraction channel.
[0029] The DC static component is removed by a 5-frame moving average and a 0.5-0.8Hz high-pass filter, while the AC dynamic component is retained, to generate the rPPG physiological signal.
[0030] In this embodiment, in step S3, physiological characteristic parameters are generated based on the rPPG physiological signal, including: In step S33, the Lucas-Kanade optical flow method is first used to complete frame alignment and motion compensation to eliminate signal distortion caused by driver head movement. The feature signal is then normalized using X′(t) = X(t) / mean(X(t)), where X′(t) is the normalized signal, X(t) is the original rPPG time series signal (which can be any R, G, or B channel), t is the time frame number, and mean(X(t)) is the average of the original signal X(t) over a time window to eliminate signal interference caused by illumination fluctuations.
[0031] S34. Obtain the AC dynamic component signal of the rPPG physiological signal, and establish an observation model of a single-frequency real sinusoidal signal based on the AC dynamic component signal. Combine this with an improved covariance algorithm to estimate and generate the heart rate. Specifically, the following formula can be used: ; Where P(n) is the AC dynamic component observation signal at the nth sampling point, A The signal amplitude; f 0 represents the fundamental frequency of the heartbeat; fS The image sampling frame rate; n The discrete sampling point number; Φ 0 represents the initial phase of the signal; g ( n () is zero-mean Gaussian white noise. and / or The amplitude ratios (R / G, B / G) and phase differences of the rPPG physiological signals in the RGB three channels are obtained. The blood oxygen saturation is then calculated based on the absorption characteristics of hemoglobin for different wavelengths of light. Specifically, an RGB three-channel rPPG blood oxygen calculation model can be established based on the Lambert-Beer law and the optical absorption characteristics of hemoglobin. First, the AC pulsation component is separated from the three-channel rPPG physiological signals. AC R 、AC G 、AC B Calculate the red / green light amplitude ratio Ratio R / G Blue light / green light amplitude ratio Ratio B / G The phase difference Δ between channels is obtained by Hilbert transform. φ R / G Δ φ B / G By combining the absorption differences between oxyhemoglobin and deoxyhemoglobin in different wavelength bands, a blood oxygen saturation estimation model is constructed: ; ; Δ φ R / G= φ R- φ G ; Δ φ B / G= φ B- φ G ; ; Among them: | AC R |、| AC G |、| AC B These represent the amplitudes of the red, green, and blue AC components, respectively. φ R 、φ G 、φ B These represent the signal phase shifts for the red, green, and blue channels, respectively.Ratio R / G , Ratio B / G The amplitude ratio; Δ φ R / G Δ φ B / G The phase difference between channels; a , b , c , d , e These are model coefficients calibrated using clinical data; SpO 2 represents the final output blood oxygen saturation. And / or Acquiring the 0.1-0.3Hz low-frequency fluctuation signal of the rPPG physiological signal and analyzing the low-frequency fluctuation signal to generate the respiratory rate, specifically includes: The AC dynamic component signal p(n) is input into the bandpass filter to obtain the breathing frequency band filtered signal b(n), i.e., b(n) = BandpassFilter(p(n), 0.1Hz, 0.3Hz); Next, a Hilbert transform is performed on the respiratory envelope signal b(n) to obtain the respiratory envelope signal Env(n), i.e. ; Finally, the number N of peak values of the respiratory envelope signal Env(n) within the time window was determined. peak The respiratory rate was calculated. RR ,Right now RR =(N peak / T)×60, where T is the signal duration. and / or The temporal, frequency, and nonlinear features of the rPPG physiological signal are extracted to obtain the intercardiac interval sequence. RR i , RR 1、 RR 2 、...、RR N This generates the HRV features. The HRV features include... SD 1. SD 2. SD1 / SD2 , SDSD (Standard deviation of adjacent differences) pNN20 (Percentage of differences > 20ms) pNN 50 (percentage of differences > 50ms) MeanRR (mean RR interval) SDNN (Standard deviation of all heartbeat intervals) RMSSD (Root mean square of adjacent RR differences) LF (Low-frequency power), HF (High-frequency power) LF / HF , nLF ,nHF , TP .in, Mean RR interval MeanRR : ; Standard deviation of all heartbeat intervals SDNN : ; Root mean square of adjacent RR differences RMSSD for: ; In the formula, [[ID=5 i+1 For the first i +1 heartbeat interval; adjacent difference standard deviation for: ; In the formula, Δ mean Adjacent i The average of the difference sequence, ; Percentage of differences > 50ms 50 is: ; Percentage of differences > 20ms 20 is: ; right i Power spectral density (PSD) analysis was performed during the interval, and the results were as follows: , , ,in This is extremely low frequency power, specifically 0.003–0.04 Hz. This refers to low-frequency power, specifically 0.04–0.15Hz. This refers to high-frequency power, specifically 0.15–0.4 Hz.
[0032] The sympathetic / parasympathetic balance ratio LF / HF is: ; (Standardized LF) is: ; (Standardized HF) is: ; (Total power) is: ; Nonlinear index 1. 2. This is obtained from the following formula: ; .
[0033] Specifically, in step S34, the G channel rPPG signal is selected first, and physiological characteristic parameters are calculated in conjunction with the R channel and B channel signals.
[0034] In this embodiment, the state recognition model uses a trained SVM model, and the training process of the SVM model includes: rPPG physiological signals of drivers under different emotional states were collected, and the rPPG physiological signals were labeled with emotion type and level to construct a training set; The SVM model is trained using 10-fold cross-validation based on the training set. The SVM model was then optimized using grid search, and the kernel function was determined to be a polynomial kernel with a penalty coefficient of C=11 and gamma=auto, thus completing the SVM model training.
[0035] In this embodiment, in step S4, the rPPG physiological signal is input into the state recognition model. A One-vs-Rest strategy is used to generate state reference data, which includes emotion categories and levels. The emotion categories include four types: fatigue, depression, irritability, and anxiety. Each emotion category is further divided into four levels: none, mild, moderate, and severe. The level is determined by the probability value corresponding to the emotion output by the state recognition model: a probability ≥80% is considered severe, a probability between 60% and 79% is considered moderate, a probability between 40% and 59% is considered mild, and a probability <40% is considered no corresponding emotion.
[0036] In this embodiment, in step S5, when the corresponding emotion level is severe, an 800Hz high-frequency sound alert and a rapidly flashing red indicator light are triggered for 10 seconds to forcibly attract the driver's attention. When the corresponding emotion level is moderate, a 600Hz mid-frequency sound alert and a flashing yellow indicator light are triggered for 5 seconds. When the corresponding emotion level is mild, a 400Hz low-frequency sound alert and a slowly flashing blue indicator light are triggered for 3 seconds. When the driver is in a normal state, the green indicator light remains constantly on, and images are continuously collected for real-time monitoring without sound alerts to avoid interfering with driving.
[0037] Another preferred real-time method of the present invention provides a contactless driver status recognition system, the system comprising: The acquisition module is used to acquire facial images; An image preprocessing module is used to preprocess the face image; The parameter extraction module is used to extract features from the preprocessed face image and generate physiological feature parameters. The state recognition module is used to input the physiological characteristic parameters into the state recognition model to generate state reference data.
[0038] In this embodiment, the system also includes an early warning module, which is used to classify the status reference data into levels and generate graded early warning signals based on the levels.
[0039] In this embodiment, the acquisition module is also used to acquire at least 200 consecutive images of the driver's face area and a fixed area above his head using an image acquisition device.
[0040] In this embodiment, the image preprocessing module includes: The preprocessing submodule is used to perform filtering and color correction on the face image; The target region determination module is used to perform feature localization on the processed face image and determine the target region. The filtering module is used to calculate the variance of the facial feature point coordinates within the target area and filter out more than 200 consecutive valid images without significant motion noise.
[0041] In this embodiment, the filtering module further includes: The first filtering submodule is used to detect the coordinates of facial feature points in each frame of the acquired continuous images, and select 5 facial feature points as the basis for motion detection. The second filtering submodule is used to select a first window with a time length of tu from the continuous images, which contains continuous Q-frame images; The third filtering submodule is used to divide the first window into tw sub-windows according to a preset interval; The fourth filtering submodule is used to calculate the x-variance and y-variance of the coordinates of facial feature points within each sub-window, and obtain the variance value for each group. The fifth filtering submodule determines the Q-frame images of the first window as valid images with more than 200 consecutive frames if the number of sub-window groups with variance values less than a preset threshold is greater than or equal to a preset number; otherwise, it slides the window along the time sequence with a step size step and repeats the processing steps of the second to fourth filtering submodules.
[0042] In this embodiment, the parameter extraction module is specifically used for: Channel-wise spatial averaging is performed on the skin pixels within the target region of the preprocessed face image to generate the original time series R(t), G(t), and B(t); Modeling RGB signals based on a dichroic reflection model: ; And generate rPPG physiological signals.
[0043] In this embodiment, the physiological characteristic parameters include one or more of heart rate, blood oxygen saturation, respiratory rate, and HRV characteristics. Based on the rPPG physiological signal, the physiological characteristic parameters are generated, including: The AC dynamic component signal of the rPPG physiological signal is acquired, and an observation model of a single-frequency real sinusoidal signal is established based on the AC dynamic component signal. The heart rate is estimated and generated using an improved covariance algorithm; and / or The amplitude ratio (R / G, B / G) and phase difference of the rPPG physiological signal are obtained, and the blood oxygen saturation is calculated using a model of hemoglobin's absorption characteristics for different wavelengths of light; and / or Acquire the low-frequency fluctuation signal of the rPPG physiological signal and analyze the low-frequency fluctuation signal to generate the respiratory rate; and / or The time-domain features, frequency-domain features, and nonlinear features of the rPPG physiological signal are extracted to generate the HRV features.
[0044] In this embodiment, the state recognition model uses a trained SVM model, and the training process of the SVM model includes: rPPG physiological signals of drivers under different emotional states were collected, and the rPPG physiological signals were labeled with emotion type and level to construct a training set; The SVM model is trained using 10-fold cross-validation based on the training set. The SVM model is then optimized using grid search, resulting in the trained SVM model.
[0045] In some other preferred embodiments of the present invention, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as described in the above embodiments.
[0046] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0047] The above description is a detailed description of the preferred embodiments of the present invention. However, the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modifications made under the technical spirit of the present invention should fall within the patent scope covered by the present invention.
Claims
1. A non-contact driver status recognition method, characterized in that, The method includes: Acquire facial images; The face image is preprocessed; Feature extraction is performed on the preprocessed face image to generate physiological feature parameters, which include one or more of heart rate, blood oxygen saturation, respiratory rate, and HRV features; The physiological feature parameters are input into the state recognition model to generate state reference data, which includes emotion category and level. The feature extraction of the preprocessed face image includes: Channel-wise spatial averaging is performed on the skin pixels within the target region of the preprocessed face image to generate the original time series R(t), G(t), and B(t); Modeling RGB signals based on a dichroic reflection model: And generate rPPG physiological signals; Where I(t) is the RGB channel light intensity signal acquired at time t; DC(t) is the DC slow converter component; AC(t) is the AC light intensity amplitude that fluctuates with the heartbeat and is the effective component of the rPPG signal; ω is the angular frequency corresponding to the heartbeat. This refers to the signal phase shift.
2. The non-contact driver status recognition method according to claim 1, characterized in that, The method further includes: The status reference data is classified into levels, and graded early warning signals are generated based on the levels.
3. The non-contact driver status recognition method according to claim 1, characterized in that, The acquisition of the face image includes: The image acquisition device captures at least 200 consecutive images of the driver's face and a fixed area above his head.
4. The non-contact driver status recognition method according to claim 1, characterized in that, The preprocessing of the facial image includes: The face image is then filtered and color corrected. The processed face image is used to locate features and determine the target region; Calculate the variance of facial feature point coordinates within the target area, and select at least 200 consecutive valid images with no significant motion noise.
5. The non-contact driver status recognition method according to claim 4, characterized in that, The calculation of the variance of facial feature point coordinates within the target area, and the selection of more than 200 consecutive valid images with no significant motion noise, includes: Coordinate detection is performed on facial feature points in each frame of the acquired continuous images, and 5 facial feature points are selected as the basis for motion detection. Select the first window with a time length of tu in the continuous image, which contains continuous Q-frame images; The first window is divided into tw sub-windows at preset intervals; Calculate the x-variance and y-variance of the facial feature point coordinates within each sub-window to obtain the variance value for each group; If the number of sub-window groups with variance values less than a preset threshold is greater than or equal to a preset number, the Q-frame images of the first window are determined as valid images with more than 200 consecutive frames; otherwise, the window is slid along the time sequence with a step size step, and the above steps of selecting a first window with a time length of tu in the continuous images, including continuous Q-frame images, are repeated to calculate the x-variance and y-variance of the coordinates of the facial feature points in each sub-window to obtain the variance value of each group.
6. The non-contact driver status recognition method according to claim 1, characterized in that, Based on the rPPG physiological signal, physiological characteristic parameters are generated, including: The AC dynamic component signal of the rPPG physiological signal is acquired, and an observation model of a single-frequency real sinusoidal signal is established based on the AC dynamic component signal. The heart rate is estimated and generated using an improved covariance algorithm; and / or The amplitude ratio (R / G, B / G) and phase difference of the rPPG physiological signal are obtained, and the blood oxygen saturation is calculated using a model of hemoglobin's absorption characteristics for different wavelengths of light; and / or Acquire the low-frequency fluctuation signal of the rPPG physiological signal and analyze the low-frequency fluctuation signal to generate the respiratory rate; and / or The time-domain features, frequency-domain features, and nonlinear features of the rPPG physiological signal are extracted to generate the HRV features.
7. The non-contact driver status recognition method according to claim 1, characterized in that, The state recognition model uses a trained SVM model, and the training process of the SVM model includes: rPPG physiological signals of drivers under different emotional states were collected, and the rPPG physiological signals were labeled with emotion type and level to construct a training set; The SVM model is trained using 10-fold cross-validation based on the training set. The SVM model is then optimized using grid search, resulting in the trained SVM model.
8. A non-contact driver status recognition system, characterized in that, The system includes: The acquisition module is used to acquire facial images; An image preprocessing module is used to preprocess the face image; The parameter extraction module is used to extract features from the preprocessed face image and generate physiological feature parameters. The state recognition module is used to input the physiological feature parameters into the state recognition model to generate state reference data; The parameter extraction module is also used for: Channel-wise spatial averaging is performed on the skin pixels within the target region of the preprocessed face image to generate the original time series R(t), G(t), and B(t); Modeling RGB signals based on a dichroic reflection model: And generate rPPG physiological signals; Where I(t) is the RGB channel light intensity signal acquired at time t; DC(t) is the DC slow converter component; AC(t) is the AC light intensity amplitude that fluctuates with the heartbeat and is the effective component of the rPPG signal; ω is the angular frequency corresponding to the heartbeat. This refers to the signal phase shift.
9. A storage medium, characterized in that, The system contains a computer program that, when executed by a processor, causes the processor to perform the steps of a non-contact driver state recognition method as described in any one of claims 1-7.