Fatigue driving monitoring method and system and storage medium

Abstract

The invention provides a fatigue driving monitoring method and system and a storage medium, and the method comprises the steps: obtaining a DMS video stream, carrying out the feature extraction of the DMS video stream, and obtaining a face ROI region; inputting the face ROI region into a regression model to extract skin brightness change characteristics of the driver, and obtaining multiple vital signs of the driver based on the skin brightness change characteristics; the vital signs at least comprise heart rate, respiratory rate, oxyhemoglobin saturation, blood pressure and heart rate variability; the vital signs are judged according to a preset time period and multiple vital sign threshold ranges, and when at least two of the multiple vital signs exceed the corresponding vital sign threshold ranges, it is judged that the driver is in the fatigue driving state. According to the method provided by the invention, multiple vital signs are detected by depending on one independent regression model, and the accuracy and timeliness of fatigue driving detection are improved.

Description

Technical Field

[0001] The present application belongs to the field of automobile safety technology, and specifically relates to a fatigue driving monitoring method, system and storage medium. Background Art

[0002] At present, common fatigue driving monitoring algorithms usually judge fatigue driving based on the driver's facial, eye, body and other features. Often, the driver is already in a fatigued state before the physiological symptoms of fatigue appear, and there is a disadvantage of untimely detection. For example, when the driver is driving, the physiological manifestations of fatigue are generally manifested by closing the eyes, lowering the head, and yawning, but when the driver shows these physiological characteristics of fatigue, the driver is already in a fatigued state. Therefore, by identifying the physiological characteristic behaviors of closing eyes or yawning, the fatigue monitoring algorithm has the problem of untimely monitoring; during the fatigue monitoring process, due to factors such as changes in the driver's expression, the fatigue monitoring algorithm will have the problem of misidentification; when the driver wears sunglasses while driving, in scenes with low visibility of the glasses, the fatigue judgment function by closing eyes will fail.

[0003] The existing technology for monitoring vital signs only detects respiratory rate and heart rate, which has great limitations. In addition, most of the current visual vital sign solutions can only realize the recognition of respiratory rate and heart rate. The technical field of vital sign monitoring has not yet made a breakthrough in the detection of all vital sign indicators by relying on a single model. Vital sign monitoring still needs to rely on multiple networks to achieve, making the monitoring of all vital signs highly complex and inefficient. Summary of the invention

[0004] In order to solve the above technical problems, the present application proposes a high-efficiency and high-accuracy fatigue driving monitoring method, system and storage medium relying on a single model.

[0005] Specifically, the present application proposes a fatigue driving monitoring method, comprising: S100: Obtain a DMS video stream, perform feature extraction on the DMS video stream, and obtain a face ROI area.

[0006] S200: Inputting the face ROI area into a regression model to extract the driver's skin brightness change characteristics, and obtaining multiple vital signs of the driver based on the skin brightness change characteristics; the vital signs include at least heart rate, respiratory rate, blood oxygen saturation, blood pressure and heart rate variability.

[0007] And, S300: judging the vital signs according to a preset time period and multiple vital sign threshold ranges, and determining that the driver is in a fatigue driving state when at least two of the multiple vital signs exceed the corresponding vital sign threshold ranges.

[0008] In the above technical solution, a single regression model is used to detect multiple vital signs of the driver. The efficiency of fatigue driving is improved by extracting the face ROI area. By detecting multiple vital signs, fatigue driving detection is made more comprehensive, multi-dimensional and multi-faceted, which improves the accuracy of fatigue driving and can monitor the driver's fatigue driving status more timely.

[0009] As an implementation manner, the step S100 may include: S110: Perform full-image detection on the DMS video stream to obtain a valid face image.

[0010] S120: Perform key point detection on the valid face image to obtain facial key feature points.

[0011] S130: performing standardization processing on the facial key feature points; and S140: cropping and splicing the face according to the standardized facial key feature points to obtain the face ROI area.

[0012] Among them, by detecting key points of the acquired valid face image and standardizing the key facial feature points, face detection can be made more accurate, and the standardization can reduce image changes caused by factors such as posture and expression. By cropping and splicing the face ROI area, the efficiency of vital sign detection will be higher than directly using the entire face, filtering out the interference of redundant feature information, and the recognition performance will also be better.

[0013] Furthermore, the step S110 includes: The effective face image is tracked in real time to store the same face parameters extracted from each frame into the face information list. By tracking the effective face image obtained by detection in real time, the detection process of vital signs can be monitored in real time and dynamically, and the detection data can be updated in time, so that the detection of fatigue driving is more timely.

[0014] Furthermore, the step S120 includes: Presetting key detection parts, and performing preprocessing based on the face information list and the valid face image to obtain a face detection image; The facial detection image is input into a key point detection model to obtain facial key feature points based on the detection key parts.

[0015] By performing key point detection on the facial detection image through the key point detection model, the key feature points of the face can be more accurately identified, thereby improving the accuracy of subsequent analysis. Effective key point detection can reduce the problem of misidentification caused by facial posture changes or facial occlusion. It can better track facial features at different angles, postures or expressions.

[0016] Furthermore, the step S140 includes: According to the facial key feature points after the standardization process, a face cropping area is obtained.

[0017] The valid face images are cropped and spliced ​​according to the face cropping area to obtain the face ROI area.

[0018] The face cropping area at least includes the forehead and cheek areas.

[0019] The areas of the key facial feature points where blood flow caused by heartbeats and subtle brightness changes of the skin are obviously cropped as ROI areas, and the cropped facial areas are spliced. This can make the detection efficiency of fatigue driving higher and filter out redundant feature information, thereby improving the recognition performance.

[0020] Furthermore, in step S200, multiple vital signs of the driver are obtained based on the skin brightness change characteristics, including: Extracting a plurality of life change features based on the skin brightness change feature; wherein the life change features at least include blood flow change features, pulse change features and light reflectivity features; Perform a vital sign analysis according to the multiple life change characteristics, and obtain multiple vital signs based on the analysis results of the multiple vital signs.

[0021] The skin brightness change feature can be extracted at least using rPPG technology. By analyzing the blood flow change feature, pulse change feature and light reflectivity feature to obtain the heart rate, respiratory rate, blood oxygen saturation, blood pressure and heart rate variability vital signs, it is possible to avoid the problem of misidentification of the fatigue monitoring algorithm due to factors such as changes in the driver's expression, and to avoid the failure of the fatigue identification function by closing eyes when the driver wears sunglasses and the visibility of the glasses is low during driving, making fatigue driving more accurate.

[0022] Furthermore, the step S200 further includes: The detection of the heart rate variability specifically includes: optimizing the acquired pulse signal to obtain the time domain signal and the frequency domain signal of the heart rate variability.

[0023] The optimization process at least includes detrending, filtering and peak detection processing.

[0024] By detrending the acquired pulse signal, the long-term trend or systematic change in the signal can be removed, making the signal more stable and the heart rate detection more accurate. Filtering can remove noise and interference in the signal, extract the useful signal part, and reduce the artifacts caused by movement, environmental noise or equipment noise, so that the subsequent heart rate variability analysis is more accurate.

[0025] Based on the same inventive concept, the present application also proposes a system for fatigue driving monitoring method, the system comprising: Region acquisition module: used to acquire DMS video stream and perform feature extraction on the DMS video stream to obtain the face ROI region.

[0026] Detection module: used to input the face ROI area into the regression model to extract the driver's skin brightness change characteristics, and obtain multiple vital signs of the driver based on the skin brightness change characteristics; the vital signs include at least heart rate, respiratory rate, blood oxygen saturation, blood pressure and heart rate variability.

[0027] And, a fatigue judgment module is used to judge the vital signs according to a preset time period and multiple vital sign threshold ranges, and when at least two of the multiple vital signs exceed the corresponding vital sign threshold ranges, it is determined that the driver is in a fatigue driving state.

[0028] Furthermore, the region acquisition module includes: The face detection module is used to perform full-image detection on the DMS video stream to obtain a valid face image.

[0029] The face tracking module is used to track the effective face image in real time.

[0030] The key point detection module is used to perform key point detection on the effective face image, obtain facial key feature points and perform standardization on the key points.

[0031] And, a cropping and splicing module is used to crop and splice the face according to the facial key feature points after the standardization process to obtain the face ROI area.

[0032] Based on the same inventive concept, the present application also proposes a domain controller, including a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions can be read by the domain controller to execute the fatigue driving monitoring method.

[0033] Compared with the prior art, this application has at least the following beneficial effects: This application detects a valid face image in the DMS video stream, extracts the ROI area of ​​the valid face image, and inputs the face ROI area into the regression model to extract the driver's skin brightness change characteristics. This avoids the problem of misidentification due to changes in the driver's expression during fatigue monitoring, and avoids the problem of fatigue monitoring failure in scenarios where the driver wears sunglasses and the visibility of the glasses is low. It achieves the detection of multiple vital signs of the driver by relying on a single regression model, making fatigue driving more efficient, and improving the accuracy and timeliness of fatigue driving detection through the judgment of multiple vital signs. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of a fatigue driving monitoring method shown in an embodiment of the present application.

[0035] Figure 2 It is a schematic diagram of a fatigue driving monitoring system shown in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] Embodiment 1: Please refer to Figure 1 The fatigue driving monitoring method mainly includes steps S100 to S300.

[0038] Wherein, the step S100 mainly includes: obtaining a DMS video stream, performing feature extraction on the DMS video stream, and obtaining a face ROI area. Wherein, the DMS video stream is a driver video captured by a camera in a smart cockpit DMS (Driver Monitor System). The smart cockpit DMS is usually used in automobiles to monitor and analyze the driver's status to improve driving safety. For example, the driver's attention level, fatigue state and other factors that may affect driving safety can be detected. The feature extraction is preferably a real-time detection of the driver's face. The face ROI area (Region of Interest) is preferably the cheek and forehead area of ​​the face.

[0039] The step S200 mainly includes: inputting the face ROI area into the regression model to extract the skin brightness change characteristics of the driver, and obtaining multiple vital signs of the driver based on the skin brightness change characteristics; the vital signs at least include heart rate, respiratory rate, blood oxygen saturation, blood pressure and heart rate variability. The regression model is preferably rPPG technology (remote photoplethysmography). The principle of the rPPG technology is to use reflected ambient light to measure the subtle brightness changes of the skin. The subtle brightness changes of the skin are caused by the blood flow caused by the heartbeat. The areas with relatively obvious changes include the forehead and the two cheeks. Therefore, based on the key points of the eyebrows, the key points of the corners of the eyes, and the key points of the corners of the mouth, the forehead and the two cheeks are cut, and the ROI area is horizontally or vertically spliced ​​to obtain the face ROI area as the feature input of the regression model. Compared with directly using the entire face, the efficiency of extracting effective features will be higher, the interference of redundant feature information will be filtered, and the recognition performance will be better.

[0040] The step S300 mainly includes: judging the vital signs according to a preset time period and multiple vital sign threshold ranges, and judging that the driver is in a fatigue driving state when at least two of the multiple vital signs exceed the corresponding vital sign threshold ranges. The time period is preferably 30 seconds, and the multiple vital sign threshold ranges preferably include a heart rate threshold range, a respiratory rate threshold range, a blood oxygen saturation threshold range, a diastolic pressure threshold range, a systolic pressure threshold range, a time domain signal threshold range, and a frequency domain signal threshold range. Among them, the heart rate threshold range is preferably 60-100bpm, the respiratory rate threshold range is preferably 12-30rpm, the blood oxygen saturation threshold range is preferably 95%-100%, the diastolic pressure threshold range is preferably 60-90mmHg, the systolic pressure threshold range is preferably 90-140mmHg, the time domain signal threshold range and the frequency domain signal threshold range are preferably a time domain signal threshold range of 141±39ms, and the frequency domain signal threshold range is preferably 1.5-2.0.

[0041] For example, in the vital sign detection, the driver's vital signs are heart rate 82bpm, breathing rate 20rpm, blood oxygen saturation 96%, diastolic pressure 100mmHg, systolic pressure 118mmHg, time domain signal 134ms, frequency domain signal 1.6. According to the driver's vital signs and the vital sign thresholds, only the diastolic pressure is no longer within the diastolic pressure threshold range, so it is determined that the driver is not currently driving fatigued.

[0042] In this embodiment, step S100 may include steps S110 to S140: The step S110 may include: performing full-image detection on the DMS video stream to obtain a valid face image. The step also includes real-time tracking of the valid face image to store the same face parameters extracted from each frame into a face information list. The face parameters may include face contour, eye features, eyebrow features, nose features, mouth features, face proportions, and face feature symmetry.

[0043] The step S120 may include: performing key point detection on the valid face image to obtain key facial feature points. The key point detection may include pre-setting key detection parts, and performing pre-processing based on the face information list and the valid face image to obtain a face detection image; inputting the face detection image into a key point detection model to obtain key facial feature points based on the key detection parts. In the key point detection module, multiple key points may be used to cover various parts of the face, and the key facial feature points are identified based on the key points. The key points are preferably 68 key points, and the 68 key points will cover the eyebrows, glasses, nose, mouth, and cheek contours of the face.

[0044] The step S130 may include: performing standardization processing on the key facial feature points, wherein the key facial feature points, such as eyes, nose tip, mouth corners, etc., are located and standardized to reduce image changes caused by factors such as posture and expression.

[0045] The step S140 may include: cropping and splicing the face according to the standardized facial key feature points to obtain the face ROI area. The cropping and splicing of the face is preferably to obtain the face cropping area according to the standardized facial key feature points; and cropping and splicing the valid face image according to the face cropping area to obtain the face ROI area. The face cropping area at least includes the forehead and cheek area.

[0046] In this embodiment, the step S200 acquires multiple vital signs of the driver based on the skin brightness change characteristics, including: Based on the skin brightness change characteristics, multiple life change characteristics are extracted; wherein the life change characteristics at least include blood flow change characteristics, pulse change characteristics and light reflectivity characteristics; vital signs analysis is performed based on the multiple life change characteristics, and multiple vital signs are obtained based on the analysis results of the multiple vital signs.

[0047] Among them, at least the skin brightness change characteristics are extracted by using rPPG technology, and the blood flow change characteristics mainly include detecting the periodic fluctuations of blood flow through tiny brightness changes on the skin surface, reflecting the changes in heart rate and blood oxygen level, and the heart rate and blood oxygen saturation can be obtained according to the blood flow change characteristics. The light reflectivity characteristics mainly include monitoring the reflectivity changes of the skin under different wavelengths of light to extract signals related to blood flow. The pulse change characteristics mainly include various changes in the pulse waveform over a period of time, through which important information about cardiovascular health and physical condition can be obtained. For example, pulse frequency, pulse rhythm, pulse amplitude, pulse waveform, pulse variability, pulse wave conduction time, and ups and downs of the pulse.

[0048] In this embodiment, the step S200 further includes: The detection of the heart rate variability specifically includes: optimizing the acquired pulse signal to obtain the time domain signal and frequency domain signal of the heart rate variability. The optimization process at least includes detrending, filtering and peak detection. Among them, the time domain signal detection in the heart rate variability detection is mainly to measure the statistical indicators of the time interval between heartbeats, such as standard deviation (SDNN) and root mean square deviation (RMSSD). The frequency domain signal detection is mainly to analyze the spectrum of the heart rate signal through Fourier transform, including high frequency (HF) and low frequency (LF) components.

[0049] The detrending process mainly removes long-term trends and deviations in the pulse signal to improve the stability of the data. The filtering process mainly reduces the noise and interference in the signal to improve the accuracy of the analysis. For example, high-frequency noise is removed and the main signal components are retained. Or low-frequency interference, especially trends or slow-changing signals, is removed. The peak detection process mainly includes identifying and extracting significant peaks in the pulse signal.

[0050] In the specific implementation process, the vital signs are judged according to the preset time period and multiple vital sign threshold ranges. When the driver is driving fatigued, the driver's heart rate will speed up or slow down, the breathing rate will become rapid, and the heartbeat variability will be abnormal. Fatigue will cause abnormal regulation of the parasympathetic and sympathetic nerves in the body, increase the tension of the sympathetic nerves, cause excessive excitement of the body, and activate the renin, angiotensin and aldosterone systems, causing abnormal contraction of blood vessels, and significantly increase the resistance in the blood vessels, so that it will cause an increase in blood pressure. Fatigue will reduce blood oxygen saturation. When at least two of the driver's heart rate, respiratory rate, blood oxygen saturation, diastolic pressure, systolic pressure, time domain signal and frequency domain signal exceed the preset heart rate threshold range, respiratory rate threshold range, blood oxygen saturation threshold range, diastolic pressure threshold range, systolic pressure threshold range, time domain signal threshold range and frequency domain signal threshold range, it is judged that the driver is in a fatigued driving state.

[0051] For example, the driver's fatigue state is detected with a time period of 30 seconds. When the vital signs obtained through the detection are: heart rate 110bpm, respiratory rate 35rpm, blood oxygen saturation 93%, diastolic pressure 100mmHg, systolic pressure 110mmHg, time domain signal 185ms and frequency domain signal 2.1, the preset vital sign threshold ranges are heart rate threshold range 60-100bpm, respiratory rate threshold range 12-30rpm, blood oxygen saturation threshold range 95%-100%, diastolic pressure threshold range 60-90mmHg, systolic pressure threshold range 90-140mmHg, time domain signal threshold range 141±39ms, and frequency domain signal threshold range preferably 1.5-2.0. According to the judgment that 6 of the vital signs are not within the preset range, the current driver is in a fatigue driving state.

[0052] In summary, when a driver is in a state of fatigue, various indicators of vital signs will show abnormal changes. DMS visual vital signs monitoring can identify the driver's fatigue state more timely and accurately, and can provide safe driving warning prompts in advance before the driver shows signs of fatigue.

[0053] Embodiment 2: Please refer to Figure 2 The present application also proposes a system for fatigue driving monitoring method, which mainly includes: a region acquisition module, a detection module and a fatigue judgment module.

[0054] Region acquisition module: used to acquire DMS video stream, perform feature extraction on the DMS video stream, and obtain face ROI region. The feature extraction on the DMS video stream preferably includes identifying the driver's face and performing feature extraction, comparing the obtained driver's face with the driver's face in the historical data, and cropping the driver's face to obtain the ROI region. After feature extraction, it can also include real-time face tracking. By continuously tracking the face in real time, the data of fatigue driving detection can be updated in a timely manner, thereby improving the timeliness of fatigue driving detection.

[0055] Detection module: used to input the face ROI area into the regression model to extract the driver's skin brightness change characteristics, and obtain multiple vital signs of the driver based on the skin brightness change characteristics; the vital signs at least include heart rate, respiratory rate, blood oxygen saturation, blood pressure and heart rate variability. The regression model at least uses rPPG technology to extract the skin brightness change characteristics. The detection module realizes the detection of multiple vital signs of the driver by relying on a single regression model.

[0056] The fatigue judgment module is used to judge the vital signs according to a preset time period and multiple vital sign threshold ranges, and judge the driver to be in a fatigue driving state when at least two of the multiple vital signs exceed the corresponding vital sign threshold ranges. The preset time period is preferably 30 seconds.

[0057] During the specific implementation process, the region acquisition module includes: The face detection module is used to perform full-image detection on the DMS video stream to obtain a valid face image.

[0058] The face tracking module is used to track the effective face image in real time.

[0059] The key point detection module is used to perform key point detection on the effective face image, obtain facial key feature points and perform standardization on the key points.

[0060] And, a cropping and splicing module is used to crop and splice the face according to the facial key feature points after the standardization process to obtain the face ROI area.

[0061] Among them, the effective face image is tracked in real time in the face tracking module, and then the same face parameters extracted from each frame are stored in the face information list. In the key point detection module, the face parameters in the face information list and the effective face image are detected by key points, and finally all faces are cropped and spliced ​​by the cropping and splicing module to obtain the face ROI area in each frame of the DMS video stream in a timely manner. In the feature extraction of the driver's skin brightness change in the detection module, since the skin brightness of the cheek and forehead areas of the face changes significantly due to the blood flow caused by the heartbeat, the cropping of the face ROI area can make the fatigue driving detection system more efficient and the detection result more accurate.

[0062] Embodiment three: The present application also proposes a computer-readable storage medium, the computer-readable storage medium comprising: The computer-readable storage medium stores computer-executable instructions.

[0063] The computer executable instructions implement the fatigue driving monitoring method when executed by the control processor.

[0064] In the computer-readable storage medium, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.

[0065] In summary, this application extracts the ROI area of ​​the effective face image detected in the DMS video stream, relies on the regression model to extract the driver's skin brightness change characteristics, and obtains multiple vital signs of the driver based on the skin brightness change characteristics, and judges whether the driver is in a fatigue driving state based on the multiple vital signs. The detection of multiple vital signs of the driver by a single regression model is realized, making the detection of vital signs more efficient. The judgment of fatigue driving through multiple vital signs can identify the driver's fatigue state more timely and accurately, and can provide safe driving warning prompts in advance before the driver shows signs of fatigue.

[0066] In several embodiments provided in the present application, it is understood that each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.

[0067] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for an electronic device to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0068] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A method for monitoring fatigue driving, characterized in that: include: S100: Acquire a DMS video stream, perform feature extraction on the DMS video stream, and acquire a face ROI area; S200: inputting the face ROI region into a regression model to extract the driver's skin brightness change characteristics, and obtaining multiple vital signs of the driver based on the skin brightness change characteristics; the vital signs at least include heart rate, respiratory rate, blood oxygen saturation, blood pressure and heart rate variability; And, S300: judging the vital signs according to a preset time period and multiple vital sign threshold ranges, and determining that the driver is in a fatigue driving state when at least two of the multiple vital signs exceed the corresponding vital sign threshold ranges.

2. The fatigue driving monitoring method according to claim 1, characterized in that: The feature extraction in step S100 includes: S110: Perform full-image detection on the DMS video stream to obtain a valid face image; S120: Perform key point detection on the valid face image to obtain key facial feature points; S130: performing standardization processing on the facial key feature points; and S140: cropping and splicing the face according to the standardized facial key feature points to obtain the face ROI area.

3. The fatigue driving monitoring method according to claim 2, characterized in that: The step S110 further includes: The effective face image is tracked in real time, and the same face parameters extracted from each frame are stored in a face information list.

4. The fatigue driving monitoring method according to claim 3 is characterized in that: The step S120 includes: Presetting key detection parts, and performing preprocessing based on the face information list and the valid face image to obtain a face detection image; The facial detection image is input into a key point detection model to obtain facial key feature points based on the detection key parts.

5. The fatigue driving monitoring method according to claim 2, characterized in that: The step S140 includes: Obtaining a face cropping area according to the facial key feature points after the standardization process; The effective face images are cropped and spliced ​​according to the face cropping area to obtain the face ROI area; The face cropping area at least includes the forehead and cheek areas.

6. The fatigue driving monitoring method according to claim 1, characterized in that: The step S200 acquires multiple vital signs of the driver based on the skin brightness change characteristics, including: Extracting a plurality of life change features based on the skin brightness change feature; wherein the life change features at least include blood flow change features, pulse change features and light reflectivity features; Perform a vital sign analysis according to the multiple life change characteristics, and obtain multiple vital signs based on the analysis results of the multiple vital signs.

7. The fatigue driving monitoring method according to claim 1, characterized in that: The step S200 further includes: The detection of the heart rate variability specifically includes: optimizing the acquired pulse signal to obtain a time domain signal and a frequency domain signal of the heart rate variability; The optimization process at least includes detrending, filtering and peak detection processing.

8. A system based on the fatigue driving monitoring method according to any one of claims 1 to 7, characterized in that: The system comprises: Region acquisition module: used to acquire DMS video stream, perform feature extraction on the DMS video stream to obtain the face ROI region; Detection module: used for inputting the face ROI area into the regression model to extract the skin brightness change characteristics of the driver, and obtaining multiple vital signs of the driver based on the skin brightness change characteristics; the vital signs at least include heart rate, respiratory rate, blood oxygen saturation, blood pressure and heart rate variability; And, a fatigue judgment module is used to judge the vital signs according to a preset time period and multiple vital sign threshold ranges, and when at least two of the multiple vital signs exceed the corresponding vital sign threshold ranges, it is determined that the driver is in a fatigue driving state.

9. The fatigue driving monitoring system according to claim 8, characterized in that: The region acquisition module comprises: A face detection module is used to perform full-image detection on the DMS video stream to obtain a valid face image; A face tracking module, used for tracking the valid face image in real time; A key point detection module is used to perform key point detection on the valid face image, obtain key facial feature points and perform standardization on the key points; And, a cropping and splicing module is used to crop and splice the face according to the facial key feature points after the standardization process to obtain the face ROI area.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by the control processor, the fatigue driving monitoring method as described in any one of claims 1-7 is executed.

Images