Fatigue detection method based on visual sensor
The driver's head, eyes and face dynamic images are captured through multispectral vision sensors, and the fatigue prediction timing model is used to analyze the fatigue degree and alarm, which solves the problem of driver fatigue detection and improves driving safety.
Patent Information
- Application Number
- CN202510886367.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-30
- Publication Date
- 2025-08-12
AI Technical Summary
Drivers are tired and unconscious during driving, which increases driving risk, and it is difficult for the prior art to effectively detect and early warning.
The driver's head, eye and face dynamic images are captured through multispectral vision sensors, and dynamic features are analyzed using the trained fatigue prediction timing model, the fatigue degree is judged and an alarm is issued when the predicted value exceeds the threshold.
The driving safety is improved, and through the combination of multi-spectral vision sensors and fatigue prediction timing model, effective detection and early warning of driver fatigue status is achieved, reducing the risk of traffic accidents caused by fatigue.
Smart Images
Figure CN120458584A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of fatigue detection, and in particular to a fatigue detection method based on a visual sensor. Background Art
[0002] The driver's mental state is a major factor influencing driving safety. If a driver becomes fatigued while driving, accidents are highly likely to occur. However, sometimes, fatigue occurs unnoticed by the driver, and if fellow passengers or the driver alone are unaware, this can significantly increase the risk of driving. Summary of the Invention
[0003] Based on this, the purpose of this application is to provide a fatigue detection method based on a visual sensor, which can overcome the shortcomings of the existing technology.
[0004] In order to achieve the above objectives, the technical solutions adopted in this application are:
[0005] A fatigue detection method based on a visual sensor, comprising:
[0006] Capture the driver's head dynamic image through multispectral vision sensors;
[0007] Acquiring a head dynamic detection result, an eye dynamic detection result, and a facial dynamic detection result of the driver based on the head dynamic image;
[0008] Inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain a predicted fatigue value of the driver;
[0009] If the predicted fatigue value is greater than the preset fatigue threshold, a fatigue driving alarm is issued.
[0010] As an embodiment, the step of obtaining the driver's head dynamic detection result, eye dynamic detection result, and facial dynamic detection result based on the head dynamic image includes:
[0011] Inputting the head dynamic image into a trained head posture estimation model to extract head dynamic features;
[0012] If the extracted head dynamic features indicate that the driver's head nods and / or deflects, the head dynamic detection result is determined to be an abnormal result.
[0013] As an embodiment, the step of obtaining the driver's head dynamic detection result, eye dynamic detection result, and facial dynamic detection result based on the head dynamic image includes:
[0014] Inputting the head dynamic image into a trained eye feature extraction network to extract eye dynamic features;
[0015] If the extracted eye dynamic features indicate that the driver's continuous eye closure time is greater than a preset time threshold, and / or the blinking frequency is greater than a preset frequency threshold, the eye dynamic detection result is determined to be an abnormal result.
[0016] As an embodiment, the step of obtaining the driver's head dynamic detection result, eye dynamic detection result, and facial dynamic detection result based on the head dynamic image includes:
[0017] Inputting the head dynamic image into a trained facial micro-expression recognition module to extract facial dynamic features;
[0018] If the extracted facial dynamic features indicate that the driver has a facial expression of yawning and / or drooping eyebrows, the facial dynamic detection result is determined to be an abnormal result.
[0019] In one embodiment, the step of inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver includes:
[0020] If the number of abnormal results in the head dynamic detection results, the eye dynamic detection results, and the facial dynamic detection results is greater than a preset number threshold, obtaining the driving time of the driver;
[0021] The driving duration, the head dynamic detection result, the eye dynamic detection result and the facial dynamic detection result are input into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver.
[0022] In one embodiment, the step of inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver includes:
[0023] If the number of abnormal results in the head dynamic detection results, the eye dynamic detection results, and the facial dynamic detection results is greater than a preset number threshold, obtaining the real-time time of the driver's location;
[0024] The real-time time, the head dynamic detection result, the eye dynamic detection result and the facial dynamic detection result are input into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver.
[0025] In one embodiment, the step of inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver includes:
[0026] If the number of abnormal results in the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result is greater than a preset number threshold, obtaining the complexity of the road on which the driver is traveling;
[0027] The complexity, the head dynamic detection result, the eye dynamic detection result and the facial dynamic detection result are input into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver.
[0028] As an embodiment, the step of issuing a fatigue driving alarm if the predicted fatigue value is greater than a preset fatigue threshold includes:
[0029] If the predicted fatigue value is greater than a preset fatigue threshold, the main driver's seat is driven to vibrate to alert and wake up the driver.
[0030] As an embodiment, the step of issuing a fatigue driving alarm if the predicted fatigue value is greater than a preset fatigue threshold includes:
[0031] If the predicted fatigue value is greater than a preset fatigue threshold, the sound playing module in the driving vehicle emits a prompt sound to alert and wake up the driver.
[0032] As an embodiment, the step of issuing a fatigue driving alarm if the predicted fatigue value is greater than a preset fatigue threshold includes:
[0033] If the predicted fatigue value is greater than a preset fatigue threshold, the cooling temperature of the air-conditioning module in the vehicle is lowered and the air output of the air-conditioning module is increased to alert and wake up the driver.
[0034] Compared with traditional technologies, the beneficial effects of this application are:
[0035] The present application can capture the driver's head dynamic image through a multi-spectral visual sensor to obtain the driver's head dynamic detection results, eye dynamic detection results and facial dynamic detection results, and then input the head dynamic detection results, the eye dynamic detection results and the facial dynamic detection results into a trained fatigue prediction time series model to obtain the driver's predicted fatigue value, so as to issue a fatigue driving alarm when the predicted fatigue value is greater than a preset fatigue threshold, thereby improving driving safety.
[0036] For better understanding and implementation, the present application is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 This is a flowchart of a fatigue detection method based on a visual sensor according to an embodiment of the present application. DETAILED DESCRIPTION
[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
[0039] It should be clear that the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the embodiments of the present application.
[0040] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. In the description of this application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to the specific circumstances. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates other meanings. The words "if" / "if" used herein can be interpreted as "at the time of" or "when" or "in response to determination".
[0041] In addition, in this application, unless otherwise specified, "plurality" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.
[0042] See also Figure 1, which is a flow chart of the fatigue detection method based on a visual sensor of the present application, including:
[0043] S1: Capture the driver's head dynamic image through a multispectral vision sensor.
[0044] Among them, the multispectral vision sensor is set in the car to capture the dynamic image of the driver's head. Specifically, the multispectral vision sensor can capture the dynamic image of the driver's head through multiple bands, such as visible light, near-infrared and other bands.
[0045] Optionally, after obtaining the dynamic image of the driver's head, the ambient light interference in the dynamic image of the head can be eliminated through an adaptive lighting compensation algorithm, such as adaptive threshold processing, histogram equalization, gamma correction, ambient light compensation and backlight adjustment, high-pass filter technology, chromaticity and brightness interference compensation, etc.
[0046] Among them, the technical principle of adaptive threshold processing is: dynamically calculate the threshold according to the brightness of the local area of the image to overcome the shortcomings of the global threshold when the lighting is uneven.
[0047] The technical principle of histogram equalization is to enhance contrast by redistributing pixel intensity, which is suitable for image enhancement under different lighting conditions.
[0048] The technical principle of gamma correction is to adjust the gamma value of the image to change the brightness distribution and optimize the details of overly bright or dark areas.
[0049] The technical principle of ambient light compensation and backlight adjustment is to automatically adjust camera parameters to balance brightness, such as the ambient light compensation function of OpenMV.
[0050] The technical principle of high-pass filter technology is to extract high-frequency details of the image and reduce low-frequency light interference.
[0051] The technical principle of chromatic aberration compensation is to compensate for the optical interference between backlight segments, thereby improving dimming depth and image quality.
[0052] S2: Acquire the driver's head dynamic detection result, eye dynamic detection result, and facial dynamic detection result based on the head dynamic image.
[0053] Among them, since the head dynamic image includes multiple image frames, it is necessary to obtain the driver's head dynamic detection results, eye dynamic detection results and facial dynamic detection results based on the feature changes in the multiple image frames of the head dynamic image or the target area changes obtained according to feature recognition.
[0054] S3: Inputting the head dynamic detection result, the eye dynamic detection result and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver.
[0055] The fatigue prediction time series model is an LSTM time series network, which can be used to predict the driver's fatigue level as a predicted fatigue value. The LSTM time series network, also known as the long short-term memory network, is an improved recurrent neural network (RNN).
[0056] The fatigue prediction time series model can be trained by multiple head dynamic detection result samples, multiple eye dynamic detection result samples, multiple facial dynamic detection result samples and corresponding multiple fatigue level samples.
[0057] S4: If the predicted fatigue value is greater than a preset fatigue threshold, a fatigue driving alarm is issued.
[0058] Among them, the fatigue driving alarm can be to remind the driver by activating the function of the in-vehicle equipment, or it can be to issue a fatigue driving alarm notification to the traffic management department or road bureau that has jurisdiction over the current road. Among them, when issuing a fatigue driving alarm notification to the traffic management department or road bureau that has jurisdiction over the current road, the fatigue driving alarm notification is generated based on the pre-recorded license plate number, vehicle model, vehicle color, real-time vehicle positioning and the predicted fatigue value of the current driving vehicle, and then sent to the traffic management department or road bureau.
[0059] Compared with the existing technology, the present application can capture the driver's head dynamic image through a multispectral visual sensor to obtain the driver's head dynamic detection results, eye dynamic detection results and facial dynamic detection results, and then input the head dynamic detection results, the eye dynamic detection results and the facial dynamic detection results into a trained fatigue prediction time series model to obtain the driver's predicted fatigue value, so as to issue a fatigue driving alarm when the predicted fatigue value is greater than a preset fatigue threshold, thereby improving driving safety.
[0060] In a feasible embodiment, the step S2: obtaining the driver's head dynamic detection result, eye dynamic detection result, and facial dynamic detection result based on the head dynamic image, includes:
[0061] S211: Inputting the head dynamic image into a trained head posture estimation model to extract head dynamic features.
[0062] The head pose estimation model includes a feature extraction layer, a single-dimensional pose layer, and a comprehensive-dimensional pose layer. During training, the total loss function of the head pose estimation model is determined based on the single-dimensional pose loss function of the single-dimensional pose layer and the comprehensive-dimensional pose loss function of the comprehensive-dimensional pose layer. For example, the total loss function of the head pose estimation model is shown in the following formula:
[0063] L=α(L yaw +L pitch +L roll )+βL cls
[0064] Among them, L is the output of the total loss function, L yaw is the single-dimensional posture loss function of the Yaw angle, L pitch is the single-dimensional posture loss function of the Pitch angle, L roll is the single-dimensional posture loss function of the Roll angle, L cls is the comprehensive dimensional posture loss function, α is the weight coefficient of the single dimension posture loss function, and β is the weight coefficient of the comprehensive dimensional posture loss function.
[0065] S212: If the extracted head dynamic features indicate that the driver's head nods and / or turns, determine that the head dynamic detection result is an abnormal result.
[0066] The head dynamic features can be input into a trained first classifier to obtain the driver's head region in each image frame of the head dynamic image. Then, based on the changes in the head region in each image frame, it is determined whether the driver has nodded and / or tilted their head. The head pose estimation model and the first classifier can be trained using samples of head dynamic images with the head region labeled.
[0067] In a feasible embodiment, the step S2: obtaining the driver's head dynamic detection result, eye dynamic detection result, and facial dynamic detection result based on the head dynamic image, includes:
[0068] S221: Inputting the head dynamic image into a trained eye feature extraction network to extract eye dynamic features.
[0069] Among them, the eye feature extraction network can also adopt a lightweight spatiotemporal convolutional network.
[0070] S222: If the extracted eye dynamic features indicate that the driver's continuous eye closure time is greater than a preset time threshold, and / or the blinking frequency is greater than a preset frequency threshold, the eye dynamic detection result is determined to be an abnormal result.
[0071] The eye dynamic features can be input into a trained second classifier to obtain the driver's eye regions in each frame of the head dynamic image. Based on the changes in the eye regions across the frames, it is then determined whether the driver has closed their eyes or blinked, thereby obtaining the continuous eye closure time and blink frequency. The eye feature extraction network and the second classifier can be trained using samples of head dynamic images with the eye regions labeled.
[0072] In a feasible embodiment, the step S2: obtaining the driver's head dynamic detection result, eye dynamic detection result, and facial dynamic detection result based on the head dynamic image, includes:
[0073] S231: Inputting the head dynamic image into a trained facial micro-expression recognition module to extract facial dynamic features.
[0074] Among them, the facial micro-expression recognition module can also adopt a lightweight spatiotemporal convolutional network.
[0075] Before inputting the head dynamic image into the trained facial micro-expression recognition module, the grayscale image of the head dynamic image is linearly transformed. The linear transformation formula is as follows:
[0076]
[0077] Among them, G(x,y) is the output of the linear transformation formula, a1, a2 and a3 are grayscale value weights, f(x,y) is the grayscale value at the point (x,y), b1, b2 and b3 are different preset constant values, and f1, f2 and f3 are preset grayscale thresholds.
[0078] S232: If the extracted facial dynamic features indicate that the driver has a facial expression of yawning and / or drooping eyebrows, determine that the facial dynamic detection result is an abnormal result.
[0079] The facial dynamic features can be input into a trained third classifier to obtain the driver's facial region and / or eyebrow region in each image frame of the facial dynamic image. Based on the changes in the facial region and / or eyebrow region in each image frame, it is then determined whether the driver has a yawning and / or drooping eyebrow facial expression. The facial pose estimation model and the third classifier can be trained using facial dynamic image samples labeled with facial regions. The facial dynamic image samples can also be labeled with eyebrow regions.
[0080] Among them, the lightweight spatiotemporal convolutional network is a convolutional network with feature extraction capabilities. Since the above-mentioned head posture estimation model, eye feature extraction network, and facial micro-expression recognition module all use lightweight spatiotemporal convolutional networks, they can be combined with the fatigue prediction timing model using the LSTM timing network to obtain a timing model with a CNN-LSTM structure. This combines the local feature extraction capability of CNN (convolutional network) with the timing modeling capability of LSTM, and is suitable for complex space-time patterns.
[0081] In a feasible embodiment, the step of S3: inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the driver's predicted fatigue value includes:
[0082] S311: If the number of abnormal results in the head dynamic detection results, the eye dynamic detection results, and the facial dynamic detection results is greater than a preset number threshold, the driving time of the driver is obtained.
[0083] S312: Input the driving duration, the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain a predicted fatigue value of the driver.
[0084] The longer the driving time, the greater the predicted fatigue value. In this embodiment, the fatigue prediction time series model can be trained using multiple driving time samples, multiple head dynamic detection result samples, multiple eye dynamic detection result samples, multiple facial dynamic detection result samples, and corresponding multiple fatigue level samples.
[0085] In a feasible embodiment, the step of S3: inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the driver's predicted fatigue value includes:
[0086] S321: If the number of abnormal results in the head dynamic detection results, the eye dynamic detection results, and the facial dynamic detection results is greater than a preset number threshold, obtain the real-time time of the area where the driver is located.
[0087] S322: Input the real-time time, the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver.
[0088] The closer the real-time time is to the time interval between 11 pm and 6 am, the greater the predicted fatigue value. In this embodiment, the fatigue prediction time series model can be trained using multiple real-time time samples, multiple head dynamic detection result samples, multiple eye dynamic detection result samples, multiple facial dynamic detection result samples, and corresponding multiple fatigue level samples.
[0089] In a feasible embodiment, the step of S3: inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the driver's predicted fatigue value includes:
[0090] S331: If the number of abnormal results in the head dynamic detection results, the eye dynamic detection results, and the facial dynamic detection results is greater than a preset number threshold, the complexity of the road on which the driver is traveling is obtained.
[0091] S332: Input the complexity, the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain a predicted fatigue value of the driver.
[0092] The higher the road complexity, the greater the predicted fatigue value. In this embodiment, the fatigue prediction time series model can be trained using multiple road complexity samples, multiple head dynamic detection result samples, multiple eye dynamic detection result samples, multiple facial dynamic detection result samples, and corresponding multiple fatigue level samples.
[0093] In a feasible embodiment, the step of issuing a fatigue driving alarm if the predicted fatigue value is greater than a preset fatigue threshold comprises:
[0094] If the predicted fatigue value is greater than a preset fatigue threshold, the main driver's seat is driven to vibrate to alert and wake up the driver.
[0095] In a feasible embodiment, the step of issuing a fatigue driving alarm if the predicted fatigue value is greater than a preset fatigue threshold comprises:
[0096] If the predicted fatigue value is greater than a preset fatigue threshold, the sound playing module in the driving vehicle emits a prompt sound to alert and wake up the driver.
[0097] In a feasible embodiment, the step of issuing a fatigue driving alarm if the predicted fatigue value is greater than a preset fatigue threshold comprises:
[0098] If the predicted fatigue value is greater than a preset fatigue threshold, the cooling temperature of the air-conditioning module in the vehicle is lowered and the air output of the air-conditioning module is increased to alert and wake up the driver.
[0099] The present application is based on a fatigue detection method of a visual sensor. The method obtains the driver's predicted fatigue value based on the head dynamic detection results, eye dynamic detection results, facial dynamic detection results and a trained fatigue prediction time series model obtained according to the driver's head dynamic image extraction. When the predicted fatigue value is greater than a preset fatigue threshold and a fatigue driving alarm is issued, the method can drive the main driver's seat to vibrate, and / or drive the sound playback module in the car to emit a prompt sound, and / or reduce the cooling temperature of the air-conditioning module in the car, increase the air output of the air-conditioning module, etc., to warn and wake up the driver, so as to reduce the probability of the driver falling asleep due to too high fatigue value, thereby improving the driver's driving safety and reducing the probability of traffic accidents due to too high driver fatigue value.
[0100] The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application. Those of ordinary skill in the art can understand and implement the present invention without inventive effort.
[0101] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0102] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the function selected in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 function selected in a box or multiple boxes.
[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 steps for the function selected in a box or multiple boxes.
[0104] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.
[0105] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0106] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.
[0107] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.
[0108] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.
Claims
1. A fatigue detection method based on a visual sensor, characterized in that: include: Capture the driver's head dynamic image through multispectral vision sensors; Acquiring a head dynamic detection result, an eye dynamic detection result, and a facial dynamic detection result of the driver based on the head dynamic image; Inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain a predicted fatigue value of the driver; If the predicted fatigue value is greater than the preset fatigue threshold, a fatigue driving alarm is issued.
2. The fatigue detection method based on a visual sensor according to claim 1, characterized in that: The step of obtaining the driver's head dynamic detection result, eye dynamic detection result, and facial dynamic detection result based on the head dynamic image includes: Inputting the head dynamic image into a trained head posture estimation model to extract head dynamic features; If the extracted head dynamic features indicate that the driver's head nods and / or deflects, the head dynamic detection result is determined to be an abnormal result.
3. The fatigue detection method based on a visual sensor according to claim 1, characterized in that: The step of obtaining the driver's head dynamic detection result, eye dynamic detection result, and facial dynamic detection result based on the head dynamic image includes: Inputting the head dynamic image into a trained eye feature extraction network to extract eye dynamic features; If the extracted eye dynamic features indicate that the driver's continuous eye closure time is greater than a preset time threshold, and / or the blinking frequency is greater than a preset frequency threshold, the eye dynamic detection result is determined to be an abnormal result.
4. The fatigue detection method based on a visual sensor according to claim 1, characterized in that: The step of obtaining the driver's head dynamic detection result, eye dynamic detection result, and facial dynamic detection result based on the head dynamic image includes: Inputting the head dynamic image into a trained facial micro-expression recognition module to extract facial dynamic features; If the extracted facial dynamic features indicate that the driver has a facial expression of yawning and / or drooping eyebrows, the facial dynamic detection result is determined to be an abnormal result.
5. The fatigue detection method based on a visual sensor according to claim 1, characterized in that: The step of inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the driver's predicted fatigue value includes: If the number of abnormal results in the head dynamic detection results, the eye dynamic detection results, and the facial dynamic detection results is greater than a preset number threshold, obtaining the driving time of the driver; The driving duration, the head dynamic detection result, the eye dynamic detection result and the facial dynamic detection result are input into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver.
6. The fatigue detection method based on a visual sensor according to claim 1, characterized in that: The step of inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the driver's predicted fatigue value includes: If the number of abnormal results in the head dynamic detection results, the eye dynamic detection results, and the facial dynamic detection results is greater than a preset number threshold, obtaining the real-time time of the driver's location; The real-time time, the head dynamic detection result, the eye dynamic detection result and the facial dynamic detection result are input into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver.
7. The fatigue detection method based on a visual sensor according to claim 1, characterized in that: The step of inputting the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result into a trained fatigue prediction time series model to obtain the driver's predicted fatigue value includes: If the number of abnormal results in the head dynamic detection result, the eye dynamic detection result, and the facial dynamic detection result is greater than a preset number threshold, obtaining the complexity of the road on which the driver is traveling; The complexity, the head dynamic detection result, the eye dynamic detection result and the facial dynamic detection result are input into a trained fatigue prediction time series model to obtain the predicted fatigue value of the driver.
8. The fatigue detection method based on a visual sensor according to claim 1, characterized in that: The step of issuing a fatigue driving alarm if the predicted fatigue value is greater than a preset fatigue threshold comprises: If the predicted fatigue value is greater than a preset fatigue threshold, the main driver's seat is driven to vibrate to alert and wake up the driver.
9. The fatigue detection method based on a visual sensor according to claim 1, characterized in that: The step of issuing a fatigue driving alarm if the predicted fatigue value is greater than a preset fatigue threshold comprises: If the predicted fatigue value is greater than a preset fatigue threshold, the sound playing module in the driving vehicle emits a prompt sound to alert and wake up the driver.
10. The fatigue detection method based on a visual sensor according to claim 1, characterized in that: The step of issuing a fatigue driving alarm if the predicted fatigue value is greater than a preset fatigue threshold comprises: If the predicted fatigue value is greater than a preset fatigue threshold, the cooling temperature of the air-conditioning module in the vehicle is lowered and the air output of the air-conditioning module is increased to alert and wake up the driver.
Citation Information
Cited By
Fatigue driving real-time detection method and system based on monitoring camera
CN121196554A