Fatigue driving detection method and related equipment

Abstract

The invention provides a fatigue driving detection method and related equipment, and relates to the field of vehicle safe driving, and the method comprises the steps: obtaining a first face image of a driver in response to a preset control event for a vehicle; based on the first face image, determining time sequence data of the eyelid opening degree of the driver; performing eye movement event segmentation on the time sequence data of the eyelid opening degree, and determining an eye closing detection threshold value of the driver; on the basis of the eye closing detection threshold value, the time sequence data of the eyelid opening degree and a pre-trained time sequence network, determining eyelid movement time sequence characteristics of the driver, and on the basis of the eyelid movement time sequence characteristics, determining a fatigue alarm threshold value of the driver; acquiring a second face image of the driver; and triggering fatigue driving alarm and / or fatigue driving intervention under the condition of determining that the driver is in fatigue driving based on the second face image, the eye closing detection threshold and the fatigue alarm threshold. The method can adapt to eye feature differences of different drivers, and can accurately carry out fatigue driving detection and early warning.

Description

Technical Field

[0001] This application relates to the field of vehicle safe driving technology, and in particular to a fatigue driving detection method and related equipment. Background Technology

[0002] With the widespread use of modern transportation and the increase in driving frequency, fatigued driving has become one of the major causes of traffic accidents. Drivers are prone to fatigue after prolonged driving, leading to slowed reaction times, impaired judgment, and lack of concentration, seriously threatening road traffic safety.

[0003] Currently, driver fatigue detection technologies typically use standardized fatigue detection thresholds (such as blinking frequency and eye closure time). This approach ignores the individual differences between drivers and cannot effectively adapt to the differences in eye characteristics among different drivers, which can easily lead to false alarms or missed alarms. Summary of the Invention

[0004] In view of this, this application provides a fatigue driving detection method and related equipment, which can adapt to the differences in eye characteristics of drivers of different vehicles and accurately detect and warn of fatigue driving.

[0005] In a first aspect, embodiments of this application provide a fatigue driving detection method applied to a vehicle. The fatigue driving detection method includes: in response to a preset control event for the vehicle, acquiring a first facial image of the vehicle driver; determining temporal data of the eyelid opening and closing degree of the vehicle driver based on the first facial image; performing eye movement event segmentation on the temporal data of the eyelid opening and closing degree to determine a closed-eye detection threshold for the vehicle driver; determining temporal features of eyelid movement of the vehicle driver based on the closed-eye detection threshold, the temporal data of the eyelid opening and closing degree, and a pre-trained temporal network; and determining a fatigue alarm threshold for the vehicle driver based on the temporal features of eyelid movement, wherein the temporal features of eyelid movement include eyelid opening and closing degree fluctuation information, closed-eye information, and / or blinking information; acquiring a second facial image of the vehicle driver, wherein the second facial image is an image acquired at a time after the first facial image; and triggering a fatigue driving alarm and / or fatigue driving intervention when it is determined that the vehicle driver is in a fatigue driving state based on the second facial image, the closed-eye detection threshold, and the fatigue alarm threshold.

[0006] Compared with related technologies, the embodiments of this application have at least the following advantages: by calculating the eyelid opening and closing degree of the vehicle driver and performing eye movement event segmentation, an individualized eye closure detection threshold can be dynamically set based on the eye physiological characteristics of the vehicle driver, adapting to the eye differences between individual vehicle drivers, significantly improving the accuracy of eye closure detection for different vehicle drivers. Furthermore, by introducing a temporal network to learn the temporal characteristics of the vehicle driver's eyelid movement end-to-end, the personalized fatigue alarm threshold matched with the vehicle driver can be accurately predicted based on the learned eyelid movement temporal characteristics, effectively overcoming the inherent defects of the existing "one-size-fits-all" alarm threshold, significantly improving the accuracy of fatigue monitoring for different vehicle drivers, and having a highly robust vehicle driver alarm threshold setting.

[0007] In some possible embodiments, the first facial image includes multiple frames of facial images arranged in chronological order of image acquisition. The step of determining the temporal data of the eyelid opening degree of the vehicle driver based on the first facial image includes: determining the eyelid opening degree of each frame of the facial image based on the eye key points of each frame of the facial image; determining the temporal data of the eyelid opening degree based on the eyelid opening degree of each frame of the facial image and preset camera calibration parameters, wherein the preset camera calibration parameters are the calibration parameters of the camera module used to acquire the first facial image, and the preset camera calibration parameters are used to convert the pixel distance of the eyelid opening degree of each frame of the facial image into physical distance.

[0008] Compared with related technologies, the embodiments of this application have at least the following advantages: by calculating the pixel distance of the eyelid opening and closing degree of each frame of facial image and converting the pixel distance of the eyelid opening and closing degree of each frame of facial image into physical distance, it is possible to obtain the real physical eyelid opening and closing degree of the vehicle driver, which is convenient for subsequent accurate eye movement event segmentation.

[0009] In some possible embodiments, the step of performing eye-tracking event segmentation on the temporal data of eyelid opening and closing to determine the eye-closing detection threshold for the vehicle driver includes: processing the temporal data of eyelid opening and closing based on a sliding window to determine the rate of change of the standard deviation of opening and closing between adjacent windows; performing eye-tracking event segmentation on the temporal data of eyelid opening and closing based on the rate of change to determine a stable open-eye segment; and determining the eye-closing detection threshold for the vehicle driver based on the temporal data of the stable open-eye segment.

[0010] Compared with related technologies, the embodiments of this application have at least the following advantages: by accurately segmenting the stable open eye segment through the window mutation detection algorithm, it is possible to dynamically set an individualized closed eye detection threshold based on the eye physiological characteristics of the vehicle driver, and adapt to the eye differences between individual vehicle drivers.

[0011] In some possible embodiments, determining the eye-closed detection threshold of the vehicle driver based on the time-series data of the stable eye-opening segment includes: determining the average eyelid opening and closing angle and the standard deviation of the eyelid opening and closing angle of the vehicle driver based on the time-series data of the stable eye-opening segment; and determining the eye-closed detection threshold of the vehicle driver based on the average eyelid opening and closing angle and the standard deviation of the eyelid opening and closing angle.

[0012] Compared with related technologies, the embodiments of this application have at least the following advantages: a Gaussian distribution model is established based on time-series data of stable eye-opening segments to determine the average value and standard deviation of eyelid opening and closing. The calculated eye-closing detection threshold can cover the normal eye-opening state, so that the determined eye-closing detection threshold is an eye-closing detection threshold that is suitable for vehicle drivers, avoiding misjudgment of vehicle drivers with small eyes as having closed eyes due to an excessively high eye-closing detection threshold, and avoiding missed detection of vehicle drivers with large eyes due to an excessively high eye-closing detection threshold.

[0013] In some possible embodiments, determining the fatigue alarm threshold of the vehicle driver based on the eyelid movement temporal features includes: determining a first fatigue alarm threshold and a second fatigue alarm threshold of the vehicle driver based on the pre-trained temporal network and the eyelid movement temporal features. The first fatigue alarm threshold includes a threshold for the number of frames with eyes closed within a preset time period, and the second fatigue alarm threshold includes a threshold for the number of frames with eyes continuously closed within the preset time period. The fatigue level corresponding to the first fatigue alarm threshold is lower than the fatigue level corresponding to the second fatigue alarm threshold. The training samples of the pre-trained temporal network include multiple facial images of vehicle drivers in historical driving environments.

[0014] Compared with related technologies, the embodiments of this application have at least the following advantages: by introducing a temporal network to learn the temporal characteristics of the driver's eyelid movements end-to-end, personalized first fatigue alarm threshold and second fatigue alarm threshold are automatically generated to meet the needs of alarming or intervening for different degrees of driving fatigue.

[0015] In some possible embodiments, the first facial image is a depth image, and the step of determining the temporal features of the vehicle driver's eyelid movements based on the closed-eye detection threshold, the temporal data of the eyelid opening and closing degree, and a pre-trained temporal network includes: acquiring depth information of the eye region in the first facial image; and determining the temporal features of the vehicle driver's eyelid movements based on the closed-eye detection threshold, the temporal data of the eyelid opening and closing degree, the depth information of the eye region, and a pre-trained temporal network.

[0016] Compared with related technologies, the embodiments of this application have at least the following advantages: by introducing depth information of the eye region, the temporal data of eyelid opening and closing can be calibrated, so that the temporal network can learn the real temporal characteristics of the driver's eyelid movement more accurately, thereby improving the accuracy of the network in predicting the fatigue alarm threshold.

[0017] In some possible embodiments, the vehicle includes a camera module for performing facial image acquisition and a supplementary lighting module for providing supplementary lighting for image acquisition of the camera module. The method further includes: acquiring environmental data of the vehicle's location; setting the supplementary lighting band and intensity of the supplementary lighting module based on the environmental data, and processing parameters for image enhancement processing of the first facial image and the second facial image acquired by the camera module.

[0018] Compared with related technologies, the embodiments of this application have at least the following advantages: clear facial images can be captured in complex and ever-changing driving environments, avoiding the decrease in eye feature recognition accuracy due to overexposure or blurring of the captured facial images, which in turn leads to false fatigue alarms. At the same time, by performing adaptive image enhancement processing on the captured facial images, the image contrast is enhanced to better match the actual lighting conditions, which can further improve the accuracy of subsequent eye feature recognition and reduce the occurrence of false fatigue alarms.

[0019] Secondly, embodiments of this application also provide a fatigue driving detection device, applied to a vehicle, the device comprising: A first acquisition module is used to acquire a first facial image of the vehicle driver in response to a preset control event for the vehicle; a first determination module is used to determine the temporal data of the eyelid opening and closing degree of the vehicle driver based on the first facial image; a second determination module is used to perform eye movement event segmentation on the temporal data of the eyelid opening and closing degree to determine the eye closure detection threshold of the vehicle driver; a third determination module is used to determine the eyelid movement temporal features of the vehicle driver based on the eye closure detection threshold, the temporal data of the eyelid opening and closing degree, and a pre-trained temporal network, and to determine the fatigue alarm threshold of the vehicle driver based on the eyelid movement temporal features, wherein the eyelid movement temporal features include eyelid opening and closing degree fluctuation information, eye closure information, and / or blinking information; a second acquisition module is used to acquire a second facial image of the vehicle driver, wherein the second facial image is an image acquired at a time after the first facial image; a control module is used to trigger a fatigue driving alarm and / or fatigue driving intervention when it is determined that the vehicle driver is in a fatigue driving state based on the second facial image, the eye closure detection threshold, and the fatigue alarm threshold.

[0020] Thirdly, embodiments of this application also provide an electronic device, which includes a processor and a memory. The memory is used to store instructions, and the processor is used to call the instructions in the memory, causing the electronic device to execute the fatigue driving detection method as described in the first aspect.

[0021] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the fatigue driving detection method as described in the first aspect.

[0022] The technical effects achieved by the second, third, and fourth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect, and will not be repeated here. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the structure of a vehicle provided in one embodiment of this application.

[0024] Figure 2 A flowchart illustrating the steps of a fatigue driving detection method provided in an embodiment of this application.

[0025] Figure 3 A flowchart illustrating the steps of a fatigue driving detection method provided in another embodiment of this application.

[0026] Figure 4 This is a functional block diagram of a fatigue driving detection device provided in an embodiment of this application.

[0027] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0029] The following description sets forth many specific details to provide a full understanding of this application. The described embodiments are only some, not all, of the embodiments of this application.

[0030] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this application's specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0031] It should be further noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0032] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence.

[0033] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0034] Figure 1 This is a functional block diagram of the vehicle 100 provided in the embodiments of this application.

[0035] The vehicle 100 may include an environmental perception module 101, a supplementary lighting module 102, a camera module 103, and a data processing module 104.

[0036] The environmental perception module 101 can be used to collect environmental data about the location of the vehicle 100. For example, the environmental perception module 101 may include a light sensor and a rain sensor, wherein the light sensor is used to detect the ambient light intensity and the rain sensor is used to detect the rainfall intensity.

[0037] In other embodiments, the environmental perception module 101 may also include a multispectral sensor and an image rain detector, using a neural network algorithm to analyze rainfall intensity in real time based on rain images.

[0038] The supplementary lighting module 102 may include various infrared light sources and pulse width modulation (PWM) driving circuits for driving the infrared light sources. Different infrared light sources can be used for supplementary lighting in different shooting environments. For example, the supplementary lighting module 102 may include an 850nm infrared LED array and a 940nm infrared LED array. The 850nm infrared LED array can be used for supplementary lighting in low-light environments, while the 940nm infrared LED array can be used for supplementary lighting in strong light / rain and fog environments.

[0039] In some embodiments, the data processing module 104 can select to enable an 850nm infrared LED array or a 940nm infrared LED array based on environmental data collected by the environmental sensing module 101, and can control the supplementary lighting module 102 to dynamically adjust the supplementary lighting intensity. The supplementary lighting intensity can be adjusted based on the current illumination intensity collected by the light sensor, and can be adjusted every time t, which can be set according to actual needs. For example, supplementary lighting intensity = reference value × (1 + current illumination intensity / preset threshold), where the reference value and the preset threshold can be preset according to actual supplementary lighting needs. This embodiment does not limit this, and the current illumination intensity is collected by the light sensor.

[0040] For example, when the ambient light intensity detected by the light sensor exceeds 10,000 lux, or when the rainfall intensity detected by the rain sensor exceeds 5 millimeters per hour, the 940nm infrared LED array is activated for supplemental lighting. Or, for example, when the ambient light intensity detected by the light sensor is below 100 lux, the 850nm infrared LED array is activated for supplemental lighting.

[0041] The camera module 103, which can be pointed at the vehicle driver, includes an image acquisition sensor, a polarizing filter, and a wide-angle lens. The image acquisition sensor is used to acquire images; for example, the image acquisition sensor has a frame rate of 30 fps and a resolution of 1280×720. The polarizing filter is used to suppress reflections caused by vehicle windows, objects inside the cabin, and the driver's glasses, etc., and the wide-angle lens is used to ensure that the image acquisition sensor covers the range of motion of the driver's head.

[0042] In some embodiments, the data processing module 104 can also adjust the processing parameters for image enhancement processing of the image acquired by the camera module 103 based on the environmental data collected by the environmental perception module 101. For example, based on the illumination intensity at the time of image acquisition, the image contrast can be enhanced by combining the adaptive histogram equalization (CLAHE) algorithm. This avoids the problem of "over-enhancement" or "under-enhancement" of the traditional CLAHE algorithm under different lighting scenarios, making the image contrast enhancement more in line with the actual lighting conditions. This can improve the accuracy of subsequent eye feature recognition and reduce the occurrence of false alarms in fatigue warnings.

[0043] The data processing module 104 may include a processor and a memory. The memory can be used to store images acquired by the camera module 103. The processor can be used for image processing and control of other modules. The processor can also be used to execute the fatigue driving detection method provided in the embodiments of this application.

[0044] In some embodiments, the processor may reuse a control processor, an intelligent sensing / computing processor, or a body / cockpit interaction processor deployed in vehicle 100. The processor may also be a dedicated processor in a driver status monitoring system (DMS) configured in vehicle 100. Specifically, the processor may be a microcontroller unit (MCU), an advanced reduced instruction set machine (ARM) processor, an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), etc.

[0045] The hardware architecture described above for fatigue driving detection can solve the problems of poor environmental adaptability, high hardware cost, and low reliability of existing vehicle driver status monitoring systems.

[0046] The fatigue driving detection method provided in this application is described in detail below.

[0047] The fatigue driving detection method provided in this application can be applied to vehicles, including gasoline-powered vehicles, pure electric vehicles, hybrid vehicles, range-extended vehicles, and so on.

[0048] Figure 2 This is a flowchart illustrating a fatigue driving detection method provided in an embodiment of this application. Figure 2 As shown, this fatigue driving detection method can be applied to the aforementioned data processing module, and the fatigue driving detection method may specifically include: Step S210: In response to a preset control event for the vehicle, a first facial image of the vehicle driver is acquired.

[0049] In some embodiments, the preset control event may be an event that triggers the acquisition of a first facial image of the vehicle driver, which can be set according to the actual needs of vehicle fatigue monitoring, and this application embodiment does not limit this. For example, the preset control event is a vehicle power start event (engine start event or drive motor start event). Another example is an event where the vehicle gear shifts to a forward gear.

[0050] In some embodiments, a camera module in the vehicle can capture real-time facial images of the driver after the vehicle is started. A data processing module can communicate with the camera module to acquire a first facial image of the driver.

[0051] For vehicle drivers, they are generally in a relatively excited state (low fatigue level) shortly after getting into the vehicle. The first facial image can be a facial image of the driver shortly after getting into the vehicle, which facilitates the accurate determination of the driver's closed-eye detection threshold based on the first facial image. For example, taking the receipt of a preset control event as the starting time, the first facial image is a facial image collected for a preset time period starting from the starting time. The preset time period can be set according to the actual fatigue detection needs, and this application embodiment does not limit it. For example, if the preset time period is 30 seconds, for a camera module with a frame rate of 30fps, 900 frames of facial images can be collected, that is, the first facial image includes 900 frames of facial images.

[0052] Step S220: Based on the first facial image, determine the temporal data of the driver's eyelid opening and closing.

[0053] In some embodiments, the first facial image may include multiple frames of facial images of the vehicle driver captured by the camera module. For example, if 900 frames of facial images are included, the data processing module may use a preset facial key point detection algorithm (e.g., the MediaPipe Face Mesh algorithm) to locate the key points of the eyes in each frame of the facial image and calculate the eyelid opening degree h of each frame of the facial image. eye Eyelid opening degree h eye It can be defined as the midpoint P of the upper eyelid upper P, the midpoint of the lower eyelid lower The vertical distance.

[0054] In some embodiments, the eyelid opening h of each frame of facial image eye It can be the average of the eyelid opening and closing angles of the left and right eyes, or the eyelid opening and closing angles of the left or right eye.

[0055] Eyelid opening h in each frame of facial image eye The value represents the pixel distance, and is used to obtain the eyelid opening h of each frame of the facial image. eye Subsequently, based on the eyelid opening / closing degree of each frame of facial images and preset camera calibration parameters, the pixel distance can be converted into physical distance. Then, based on the physical distance of the eyelid opening / closing degree of each frame of facial images, the temporal data of eyelid opening / closing degree can be obtained. The unit of physical distance can be millimeters. The preset camera calibration parameters can refer to the calibration parameters of the camera module. These calibration parameters can include the mapping relationship between pixel distance and physical distance, thereby enabling the conversion of the pixel distance of eyelid opening / closing degree into the physical distance of eyelid opening / closing degree. For 900 frames of facial images arranged in the acquisition sequence, the temporal data H of eyelid opening / closing degree can be expressed as: H = h1, h2, h3, …, h 900Where h1 is the physical distance of eyelid opening in the first frame of the 900 facial images, h2 is the physical distance of eyelid opening in the second frame of the 900 facial images, and h3 is the physical distance of eyelid opening in the third frame of the 900 facial images. 900 The physical distance for eyelid opening in the 900th frame of a face image out of 900 frames.

[0056] Step S230: Perform eye movement event segmentation on the temporal data of eyelid opening and closing to determine the detection threshold for eye closure of the vehicle driver.

[0057] In some embodiments, the data processing module can perform eye movement event segmentation on the temporal data of eyelid opening and closing using a sliding window mutation detection method. When the fluctuation rate of eyelid activity in adjacent windows exceeds a set threshold, it is marked as a blink boundary. Then, based on the blink boundary, the temporal data H of eyelid opening and closing is divided into temporal data of stable eye-opening segment and temporal data of blink event segment. Subsequently, a personalized eye opening and closing model is established based on the temporal data of stable eye-opening segment, and then the eye-closing detection threshold that conforms to the physiological characteristics of vehicle drivers can be calculated based on the personalized eye opening and closing model.

[0058] Specifically, segmenting eye movement events from temporal data on eyelid opening and closing to determine the eye closure detection threshold for a vehicle driver may include the following steps: (a) Based on the time-series data of the eyelid opening and closing degree, process the sliding window and determine the rate of change of the standard deviation of the opening and closing degree between adjacent windows; In some embodiments, the size of the sliding window can be set according to actual needs, and this application embodiment does not limit this. For example, the size of the sliding window is set to 15 frames. The rate of change Δ of the standard deviation of the opening degree between adjacent windows. σ i It can be represented as: , , in, h k Indicates the first k The degree of eyelid opening in a frame of facial images. w Indicates the size of the sliding window. i This indicates the location index of the facial image currently being analyzed. i This represents the average eyelid opening / closing degree within the current window. σ i This represents the standard deviation of the current window (used to reflect the degree of fluctuation in eyelid opening and closing within the current window). σ i+wThis represents the standard deviation of windows adjacent to the current window (used to reflect the degree of fluctuation in eyelid opening within windows adjacent to the current window), for example, when... i When =100, σ i+w Δ represents the standard deviation of eyelid opening / closing in frames 115-129. σ i This represents the rate of change of the standard deviation of the eyelid opening angle between the current window and its adjacent windows, Δ. σ i It can be used to quantify the proportion of changes in the degree of eyelid movement fluctuations between adjacent time periods.

[0059] In some embodiments, alternative sliding window mutation detection, such as wavelet transform modulus maxima detection or hidden Markov models, can also be used for eye-tracking event segmentation.

[0060] (b) The rate of change of the standard deviation of eyelid opening between adjacent windows is used to segment the temporal data of eyelid opening and closing into eye movement events to determine the stable eye-opening segment; In some embodiments, the fluctuation of eyelid opening and closing is gradual when the eyes are normally open, but during blinking, the eyelids rapidly close and open, causing drastic changes in eyelid opening and closing. The rate of change of the standard deviation of eyelid opening and closing between adjacent windows can be compared with a set threshold. If the rate of change of the standard deviation of eyelid opening and closing between adjacent windows Δ σ i The set threshold is marked as the blink boundary. After marking all blink boundaries, a stable eye-opening segment (i.e., a continuous eye-opening interval without blinking) is segmented based on the marked blink boundaries. The set threshold can be set according to actual detection needs, and this application embodiment does not limit it. For example, the set threshold is 200%, that is, if Δ σ i >200%, marked as blink boundary.

[0061] After segmentation, the resulting segmentation can include stable eye-opening segments. S open And the blinking event segment S blink : in, t start The moment when the blinking action begins. t end The moment the blinking action ends. h j For the first j Eyelid opening / closing in a frame of facial image. Stable eye-opening segment. S openIt can be assembled from multiple segments that are open to the naked eye.

[0062] (c) Determine the eye-closed detection threshold for vehicle drivers based on time-series data of stable open-eye segments.

[0063] After determining the stable eye-opening segment, a personalized eye-opening and closing degree model can be established based on the time-series data of the stable eye-opening segment to determine the eye-closed detection threshold for vehicle drivers. For example, Gaussian distribution modeling can be performed based on the time-series data of the stable eye-opening segment, and the eye-closed detection threshold for vehicle drivers can be determined based on the Gaussian distribution modeling results. Alternatively, a non-parametric kernel density estimation algorithm can be used to determine the eye-closed detection threshold for vehicle drivers based on the time-series data of the stable eye-opening segment.

[0064] Taking Gaussian distribution modeling based on time-series data of stable eye-opening segments as an example, determining the eye-closed detection threshold for vehicle drivers based on the time-series data of stable eye-opening segments can include: determining the average and standard deviation of eyelid opening and closing angles of vehicle drivers based on the time-series data of stable eye-opening segments; determining the eye-closed detection threshold for vehicle drivers based on the average and standard deviation of eyelid opening and closing angles, so that the determined eye-closed detection threshold is an eye-closed detection threshold adapted to vehicle drivers, avoiding misjudgment of vehicle drivers with small eyes as having closed eyes due to excessively high eye-closed detection thresholds, and avoiding underreporting of vehicle drivers with large eyes due to excessively high eye-closed detection thresholds.

[0065] For example, in determining the average eyelid opening and closing angle. μ eye Standard deviation of eyelid opening σ eye Set the eye-closed detection threshold a= μ eye -2.5 σ eye The average eyelid opening and closing degree μ eye =4mm, standard deviation of eyelid opening / closing σ eye Taking a value of 0.6mm as an example, the closed-eye detection threshold a = 4 - 2.5 × 0.6 = 2.5mm. It is understood that in other embodiments, a formula for calculating the closed-eye detection threshold based on the average eyelid opening and closing angle and the standard deviation of eyelid opening and closing angle can be set according to the actual detection results, and a value is not necessarily a given. μ eye -2.5 σ eye For restrictions.

[0066] In some embodiments, after performing Gaussian distribution modeling based on time-series data of stable eye-opening segments, the average eyelid opening degree is... μ eye Standard deviation of eyelid opening σeye It can be expressed by the following formula: , in, S open To stabilize the time series data during the eye-opening phase, | S open |The amount of time-series data for the stable eye-opening phase (which can be equated to the number of time-series data). h To stabilize the opening and closing of individual eyelids during the open eye segment.

[0067] Step S240: Based on the eye-closed detection threshold, the temporal data of eyelid opening and closing degree, and the pre-trained temporal network, determine the temporal characteristics of the driver's eyelid movement, and determine the driver's fatigue alarm threshold based on the temporal characteristics of eyelid movement.

[0068] In some embodiments, after obtaining the eye-closing detection threshold for the vehicle driver, the data processing module can input the eye-closing detection threshold and the temporal data of eyelid opening and closing into a pre-trained temporal network. The temporal network learns the temporal characteristics of the vehicle driver's eyelid movements, and then determines and outputs the vehicle driver's fatigue alarm threshold based on the eyelid movement temporal characteristics. In other embodiments, only the temporal data of eyelid opening and closing can be input into the pre-trained temporal network, which can then learn the temporal characteristics of the vehicle driver's eyelid movements to determine and output the vehicle driver's fatigue alarm threshold.

[0069] In some embodiments, the eyelid movement temporal features may include eyelid opening and closing fluctuation information, eye closure information, and / or blinking information. That is, in this embodiment, the eyelid movement temporal features include, in addition to eyelid opening and closing fluctuation information, at least one of eye closure information and blinking information. Specifically, the eyelid opening and closing fluctuation information can be used to reflect the fluctuation characteristics of eyelid opening and closing acquired in chronological order, including but not limited to fluctuation amplitude, fluctuation frequency, fluctuation trend, and abnormal fluctuations. The eye closure information can be used to reflect the duration and amplitude of eye closure for the vehicle driver. The blinking information can be used to reflect the blinking frequency, blinking duration, and blinking amplitude for the vehicle driver. For the pre-trained temporal network, by learning the eyelid movement temporal features representing the physiological eyelid movements of the vehicle driver end-to-end, and then predicting the fatigue alarm threshold of the vehicle driver based on the eyelid movement temporal features, the computational complexity is significantly reduced while ensuring accuracy. This achieves low-complexity and highly robust determination of the fatigue alarm threshold using only a short duration of first image data.

[0070] In some embodiments, a pre-trained temporal network can share temporal data on eyelid opening and closing based on an eye-closing detection threshold, thereby enabling the extraction of temporal features of eye-closing information and / or blinking information.

[0071] Specifically, the pre-trained temporal network can use a single-layer temporal convolution (e.g., a kernel width of 5 and a channel count of 4) to extract temporal features such as eyelid opening and closing, eye closure information, and blinking information. The predicted fatigue alarm threshold is then output by a fully connected layer. For example, the fatigue alarm threshold may include a first fatigue alarm threshold and a second fatigue alarm threshold. The first fatigue alarm threshold and the second fatigue alarm threshold correspond to different fatigue levels, and different fatigue levels can correspond to different fatigue driving alarms or fatigue driving interventions.

[0072] For example, the first fatigue alarm threshold includes a threshold for the number of frames with eyes closed within a preset time, and the second fatigue alarm threshold includes a threshold for the number of frames with eyes continuously closed within a preset time. The fatigue level corresponding to the first fatigue alarm threshold is lower than the fatigue level corresponding to the second fatigue alarm threshold. The preset time can be set according to actual needs, and this application embodiment does not limit it. For example, the preset time is 10s or 20s.

[0073] In some embodiments, the temporal network can be a miniature temporal convolutional network (TCN network), a lightweight temporal transformer, etc. This application does not limit this, as it is convenient to deploy in vehicles with limited hardware computing power. Furthermore, before deploying the pre-trained temporal network to the vehicle, data volume can be compressed through network quantization (e.g., INT8 quantization), and convolution operations can be optimized based on the processor's instruction set, enabling the vehicle to complete online learning and updating of fatigue alarm thresholds in a short time. Compared to the traditional fixed setting scheme for fatigue alarm thresholds, this approach, driven by the driver's eye data, adapts to different physiological differences among drivers (such as congenital ptosis, postoperative recovery period, etc.) and supports incremental network learning. For example, when a user reports a false fatigue alarm, the weights of the last layer of the temporal network can be locally fine-tuned, and online learning and fatigue alarm threshold determination can be performed again based on the driver's temporal data.

[0074] In some embodiments, the training samples for the pre-trained temporal network may include facial images of multiple vehicle drivers in historical driving environments. The temporal network can be trained in a supervised manner by adding a label (fatigue warning threshold) to each vehicle driver. For example, the training samples are based on a large-scale set of facial images of multiple vehicle drivers. This dataset may be collected in controlled driving environments and / or natural driving environments and is an anonymized dataset. The trained temporal network can be deployed to vehicles. During the network application phase, the trained temporal network can perform transfer learning (fine-tuning) based on the temporal features of the current vehicle driver (e.g., temporal data of eyelid opening and closing, and eye closure detection threshold) to predict the fatigue warning threshold of the current vehicle driver.

[0075] Taking a temporal network (TCN) as an example, a TCN network can include an input layer, convolutional layers, a feature fusion layer, and an output layer. The input layer receives input features, the convolutional layers extract temporal features and model time dependencies, the feature fusion layer fuses features, and the output layer can be a fully connected layer, outputting a vector composed of a first fatigue warning threshold x and a second fatigue warning threshold y. During the training phase of the TCN network, the weights of the fully connected layers can be optimized through weight update rules, and the network parameters can be iteratively updated using gradient descent to gradually minimize the loss value.

[0076] Step S250: Obtain a second facial image of the vehicle driver, which is an image captured at a time after the first facial image.

[0077] In some embodiments, after determining the fatigue alarm threshold for the vehicle driver, the data processing module can perform real-time fatigue driving monitoring of the vehicle driver based on the fatigue alarm threshold and the eye-closing detection threshold. The data processing module can acquire a second facial image of the vehicle driver captured by the camera module in real time, analyze the second facial image to determine whether the vehicle driver is in a state of fatigue driving, and then determine whether to trigger a fatigue driving alarm or fatigue driving intervention based on the determination result.

[0078] Step S260: If it is determined that the driver of the vehicle is in a state of fatigued driving based on the second facial image, the closed eye detection threshold and the fatigue alarm threshold, trigger a fatigue driving alarm and / or fatigue driving intervention.

[0079] In some embodiments, when it is determined that the vehicle driver is in a state of fatigued driving based on the second facial image, the closed-eye detection threshold, and the fatigue alarm threshold, the data processing module may trigger other functional modules in the vehicle to issue fatigue driving alarms and / or intervene in fatigue driving. When it is determined that the vehicle driver is not in a state of fatigued driving based on the second facial image, the closed-eye detection threshold, and the fatigue alarm threshold, the data processing module does not trigger other functional modules in the vehicle to issue fatigue driving alarms or intervene in fatigue driving.

[0080] In some embodiments, the fatigue alarm threshold includes a first fatigue alarm threshold x and a second fatigue alarm threshold y. It can be determined by analyzing the key points of the eyes in the second facial image to determine the eyelid opening and closing degree of each frame of the facial image, and comparing it with the closed eye detection threshold to determine whether there is a facial image with closed eyes. Then, based on the number of frames of the closed eye facial image, the number of frames with continuously closed eyes, and the fatigue alarm threshold, it can be determined whether the vehicle driver is in a state of fatigued driving and the corresponding fatigue level.

[0081] For example, the first fatigue alarm threshold x is equivalent to the threshold of a level 1 fatigue alarm, and the second fatigue alarm threshold y is equivalent to the threshold of a level 2 fatigue alarm. A level 1 fatigue alarm can correspond to one or more of the following: dashboard text alerts, sound alerts, and steering wheel vibration alerts. A level 2 fatigue alarm, in addition to including one or more of the following: text alerts, sound alerts, and steering wheel vibration alerts, also includes fatigue driving intervention linked to other vehicle functions, such as controlling vehicle deceleration. For example, when a level 2 fatigue alarm is triggered and the advanced driver assistance system (ADAS) is activated, it is necessary to reconfirm the driver's status (e.g., the driver needs to slowly turn the steering wheel). If it is confirmed that the driver does not have the ability to drive safely, the vehicle's safe stopping function is triggered.

[0082] The aforementioned fatigue driving detection method calculates the driver's eyelid opening and closing degree and uses a sliding window mutation detection algorithm to accurately segment stable open eye segments. This enables the dynamic setting of individualized closed eye detection thresholds based on the driver's eye physiological characteristics, adapting to the differences in eye characteristics among individual drivers and significantly improving the accuracy of closed eye detection for different drivers. Furthermore, by introducing a temporal network to learn the temporal characteristics of the driver's eyelid movements end-to-end, the method accurately predicts personalized fatigue alarm thresholds that match the driver based on the learned eyelid movement temporal characteristics. This effectively overcomes the inherent defects of existing "one-size-fits-all" alarm thresholds, significantly improving the accuracy of fatigue monitoring for different drivers and demonstrating highly robust driver alarm threshold setting.

[0083] Please see Figure 3 This is a flowchart illustrating a fatigue driving detection method provided in an embodiment of this application. Figure 3 As shown, this fatigue driving detection method can be applied to vehicles, and... Figure 2 In comparison, this fatigue driving detection method, by incorporating depth information from the eye region, can more accurately generate personalized fatigue warning thresholds. Specifically, this fatigue driving detection method may include: Step S310: In response to a preset control event for the vehicle, a first facial image of the vehicle driver is acquired.

[0084] Step S310 of this embodiment is similar to step S210 of the previous embodiment, and will not be described again here.

[0085] Step S320: Based on the first facial image, determine the temporal data of the driver's eyelid opening and closing.

[0086] Step S320 in this embodiment is similar to step S220 in the previous embodiment, and will not be described again here.

[0087] Step S330: Perform eye movement event segmentation on the temporal data of eyelid opening and closing to determine the detection threshold for eye closure of the vehicle driver.

[0088] Step S330 in this embodiment is similar to step S230 in the previous embodiment, and will not be described again here.

[0089] Step S340: Obtain depth information of the eye region in the first facial image; determine the eyelid movement temporal features of the vehicle driver based on the closed eye detection threshold, temporal data of eyelid opening and closing, depth information of the eye region, and a pre-trained temporal network; and determine the fatigue alarm threshold of the vehicle driver based on the eyelid movement temporal features.

[0090] In some embodiments, the first facial image is a depth image, and the depth information of the eye region in the first facial image can be the depth information of the eye region of each frame of the facial image in the first facial image. By introducing the depth information of the eye region, the temporal data of eyelid opening and closing can be calibrated, enabling the temporal network to learn the real temporal characteristics of the driver's eyelid movements more accurately, thereby improving the prediction accuracy of the temporal network for fatigue alarm thresholds.

[0091] In some embodiments, when determining the eyelid opening and closing degree, the depth information of the eye region can also be combined with the midpoint P of the upper eyelid. upper P, the midpoint of the lower eyelid lower The vertical distance is corrected to accurately determine the eyelid opening degree of each frame of facial image.

[0092] Step S350: Acquire a second facial image of the vehicle driver, the second facial image being an image captured at a time after the first facial image.

[0093] Step S350 in this embodiment is similar to step S250 in the previous embodiment, and will not be described again here.

[0094] Step S360: If it is determined that the driver of the vehicle is in a state of fatigued driving based on the second facial image, the closed eye detection threshold and the fatigue alarm threshold, trigger a fatigue driving alarm and / or fatigue driving intervention.

[0095] Step S360 of this embodiment is similar to step S260 of the previous embodiment, and will not be described again here.

[0096] The aforementioned fatigue driving detection method calculates the driver's eyelid opening and closing degree and uses a sliding window mutation detection algorithm to accurately segment stable open eye segments. This enables the dynamic setting of individualized closed eye detection thresholds based on the driver's eye physiological characteristics, adapting to the differences in eye characteristics among individual drivers and significantly improving the accuracy of closed eye detection for different drivers. Furthermore, by introducing depth information of the eye region, the calculated eyelid movement temporal features of the driver can be calibrated. By introducing a temporal network to learn the driver's eyelid movement temporal features end-to-end, the learned eyelid movement temporal features accurately predict personalized fatigue alarm thresholds matching the driver, effectively overcoming the inherent defects of existing "one-size-fits-all" alarm thresholds. This significantly improves the accuracy of fatigue monitoring for different drivers and demonstrates highly robust driver alarm threshold setting.

[0097] like Figure 4 As shown, this application embodiment also provides a fatigue driving detection device 40, which can be integrated into a vehicle. The fatigue driving detection device 40 may include: The first acquisition module 401 is used to acquire a first facial image of the vehicle driver in response to a preset control event for the vehicle.

[0098] The first determining module 402 is used to determine the temporal data of the eyelid opening and closing degree of the vehicle driver based on the first facial image.

[0099] The second determining module 403 is used to perform eye movement event segmentation on the temporal data of eyelid opening and closing to determine the eye closure detection threshold of the vehicle driver.

[0100] The third determining module 404 is used to determine the temporal characteristics of the driver's eyelid movements based on the eye-closing detection threshold, temporal data of eyelid opening and closing, and a pre-trained temporal network, and to determine the driver's fatigue alarm threshold based on the eyelid movement temporal characteristics. The eyelid movement temporal characteristics may include eyelid opening and closing fluctuation information, eye-closing information, and / or blinking information.

[0101] The second acquisition module 405 is used to acquire a second facial image of the vehicle driver, wherein the second facial image is an image acquired at a time after the first facial image.

[0102] The control module 406 is used to trigger a fatigue driving alarm and / or fatigue driving intervention when it is determined that the vehicle driver is in a fatigue driving state based on the second facial image, the closed eye detection threshold and the fatigue alarm threshold.

[0103] The aforementioned modules can be programmable software instructions stored in memory and executable by the processor. It is understood that in other embodiments, the aforementioned modules can also be program instructions or firmware embedded in the processor.

[0104] Please refer to Figure 5 , Figure 5 This is a schematic diagram of an embodiment of the electronic device of this application. The electronic device 1000 can be integrated into a vehicle. For example, the electronic device 1000 can be... Figure 1 The data processing module 104 shown.

[0105] The electronic device 1000 includes a memory 1010, a processor 1020, and a computer program 1030 stored in the memory 1010 and executable on the processor 1020. When the processor 1020 executes the computer program 1030, it implements the steps described in the fatigue driving detection method embodiment above, for example... Figure 2 , Figure 3 The steps are shown.

[0106] For example, computer program 1030 can also be divided into one or more modules / units, one or more of which are stored in memory 1010 and executed by processor 1020. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, and the instruction segments are used to describe the execution process of computer program 1030 in electronic device 1000.

[0107] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 1000 and does not constitute a limitation on the electronic device 1000. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device 1000 may also include input / output devices, network access devices, buses, etc.

[0108] Processor 1020 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors, single-chip microcomputers, or processor 1020 can be any conventional processor.

[0109] The memory 1010 can be used to store computer programs 40 and / or modules / units. The processor 1020 implements various functions of the electronic device 1000 by running or executing the computer programs and / or modules / units stored in the memory 1010 and by calling data stored in the memory 1010. The memory 1010 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 1000 (such as audio data), etc. In addition, the memory 1010 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0110] If the modules / units integrated in the electronic device 1000 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0111] This embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the fatigue driving detection method in the above embodiment.

[0112] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method of detecting fatigue driving, characterized by, The method is applied to a vehicle and comprises: in response to a preset control event for the vehicle, acquiring a first facial image of a driver of the vehicle; based on the first facial image, determining time sequence data of eyelid opening and closing of the driver of the vehicle; performing eye movement event segmentation on the time sequence data of the eyelid opening and closing to determine a closed-eye detection threshold of the driver of the vehicle; based on the closed-eye detection threshold, the time sequence data of the eyelid opening and closing and a pre-trained time sequence network, determining eyelid movement time sequence features of the driver of the vehicle, and based on the eyelid movement time sequence features, determining a fatigue warning threshold of the driver of the vehicle, the eyelid movement time sequence features including eyelid opening and closing fluctuation information, closed-eye information and / or blink information; acquiring a second facial image of the driver of the vehicle, the second facial image being an image captured at a time point after the first facial image; in a case where it is determined based on the second facial image, the closed-eye detection threshold and the fatigue warning threshold that the driver of the vehicle is in a fatigue driving state, triggering fatigue driving warning and / or fatigue driving intervention.

2. The fatigue driving detection method according to claim 1, characterized by, The first facial image comprises a plurality of facial images arranged in time sequence according to image capture time, and the determination of the time sequence data of the eyelid opening and closing of the driver of the vehicle based on the first facial image comprises: based on eye key points of each facial image in the plurality of facial images, determining eyelid opening and closing of the each facial image; based on the eyelid opening and closing of the each facial image and preset camera calibration parameters, determining the time sequence data of the eyelid opening and closing, the preset camera calibration parameters being calibration parameters of a camera module used for capturing the first facial image, the preset camera calibration parameters being used for converting pixel distance of the eyelid opening and closing of the each facial image into physical distance.

3. The method of detecting driver fatigue according to claim 1, wherein The eye movement event segmentation on the time sequence data of the eyelid opening and closing to determine the closed-eye detection threshold of the driver of the vehicle comprises: based on sliding window processing of the time sequence data of the eyelid opening and closing, determining a change rate of standard deviation of opening and closing between adjacent windows; based on the change rate, performing eye movement event segmentation on the time sequence data of the eyelid opening and closing to determine a stable open-eye section; based on time sequence data of the stable open-eye section, determining the closed-eye detection threshold of the driver of the vehicle.

4. The drowsy driving detection method according to claim 3, characterized by, The determination of the closed-eye detection threshold of the driver of the vehicle based on the time sequence data of the stable open-eye section comprises: based on the time sequence data of the stable open-eye section, determining an eyelid opening and closing average value and an eyelid opening and closing standard deviation of the driver of the vehicle; based on the eyelid opening and closing average value and the eyelid opening and closing standard deviation, determining the closed-eye detection threshold of the driver of the vehicle.

5. The method of detecting driver fatigue according to claim 1, wherein The determination of the fatigue warning threshold of the driver of the vehicle based on the eyelid movement time sequence features comprises: determine a first fatigue warning threshold and a second fatigue warning threshold of the vehicle driver based on the pre-trained time sequence network and the eyelid movement time sequence feature, the first fatigue warning threshold comprising a closed-eye frame number threshold within a preset time, the second fatigue warning threshold comprising a continuous closed-eye frame number threshold within the preset time, the fatigue degree corresponding to the first fatigue warning threshold being lower than the fatigue degree corresponding to the second fatigue warning threshold, and the training sample of the pre-trained time sequence network comprising face images of a plurality of vehicle drivers in historical driving environments.

6. The method of detecting driver fatigue according to claim 1, wherein The first face image is a depth image, and the determination of the eyelid movement time sequence feature of the vehicle driver based on the closed-eye detection threshold, the time sequence data of the eyelid opening degree, and the pre-trained time sequence network comprises: obtaining depth information of an eye region in the first face image; determining the eyelid movement time sequence feature of the vehicle driver based on the closed-eye detection threshold, the time sequence data of the eyelid opening degree, the depth information of the eye region, and the pre-trained time sequence network.

7. The method of detecting driver fatigue according to claim 1, wherein The vehicle comprises a camera module for performing face image acquisition and a light supplement module for supplementing light for image acquisition of the camera module, and the method further comprises: obtaining environmental data of the vehicle; setting a light supplement waveband and a light supplement intensity of the light supplement module based on the environmental data, and setting a processing parameter for image enhancement processing of the first face image and the second face image acquired by the camera module.

8. A drowsy driving detection device characterized by comprising: The device is applied to a vehicle and comprises: a first obtaining module configured to obtain a first face image of a vehicle driver in response to a preset control event of the vehicle; a first determining module configured to determine time sequence data of eyelid opening degree of the vehicle driver based on the first face image; a second determining module configured to perform eye movement event segmentation on the time sequence data of the eyelid opening degree to determine a closed-eye detection threshold of the vehicle driver; a third determining module configured to determine an eyelid movement time sequence feature of the vehicle driver based on the closed-eye detection threshold, the time sequence data of the eyelid opening degree, and a pre-trained time sequence network, and determine a fatigue warning threshold of the vehicle driver based on the eyelid movement time sequence feature, the eyelid movement time sequence feature comprising eyelid opening degree fluctuation information, closed-eye information, and / or blink information; a second obtaining module configured to obtain a second face image of the vehicle driver, the second face image being an image acquired at a time point after the first face image; a control module configured to trigger fatigue driving warning and / or fatigue driving intervention in a case where it is determined based on the second face image, the closed-eye detection threshold, and the fatigue warning threshold that the vehicle driver is in a fatigue driving state. 9.An electronic device comprising a processor and a memory, wherein, The memory is configured to store instructions, and the processor is configured to invoke the instructions in the memory to enable the electronic device to perform the fatigue driving detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions which, when executed on the electronic device, cause the electronic device to perform the fatigue driving detection method of any one of claims 1-7.

Images