Fatigue driving detection method based on fusion of multiple detection methods

Abstract

The invention provides a fatigue driving detection method integrating multiple detection methods, which comprises the following steps of: acquiring a skin electric signal of a driver based on a physiological signal detection module, and detecting an eye state of the driver in real time based on a facial image recognition module; the driving behavior analysis module is used for collecting vehicle driving parameters and continuous driving time of a driver, the data fusion processing module is used for receiving and comprehensively analyzing data of the modules, the fatigue state of the driver is evaluated, and the fatigue early warning module is used for sending out an early warning signal when detecting the fatigue state of the driver. The fatigue driving detection method and the fatigue driving early warning module are combined, so that when it is detected that the driver is in the fatigue driving state, the method can give an alarm in time through the buzzer to enable the driver to be sober for a short time and remind the driver to have a rest in time, the fatigue driving phenomenon is effectively reduced, and the driving safety is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fatigue detection, and in particular to a fatigue driving detection method that integrates multiple detection methods. Background Art

[0002] With the development of the automobile industry and the increase in the number of cars, traffic safety issues have become increasingly prominent, and traffic accidents caused by fatigue driving are increasing. When a driver is in a fatigued driving state, his attention and reaction speed will drop sharply, and his ability to control the car will drop sharply, which can easily lead to traffic accidents and pose a serious threat to people's lives and property safety. Fatigue driving detection methods can greatly reduce traffic accidents caused by driver fatigue driving. Existing fatigue driving detection technologies usually rely on a single fatigue detection method, such as detection based on physiological signals, detection based on facial image recognition, heart rate detection or eye tracking. These methods lack comprehensiveness and have certain limitations.

[0003] Therefore, it is necessary to design a fatigue driving detection method that integrates multiple detection methods. Summary of the Invention

[0004] In order to overcome the deficiencies of the prior art, the present invention aims to provide a fatigue driving detection method that integrates multiple detection methods.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] The present invention provides a fatigue driving detection method based on the fusion of multiple detection methods, including:

[0007] Step 1: Collect the driver's skin electrical signals based on the physiological signal detection module;

[0008] Step 2: Detect the driver's eye status in real time based on the facial image recognition module;

[0009] Step 3: Collect vehicle driving parameters and driver continuous driving time based on the driving behavior analysis module;

[0010] Step 4: The data fusion processing module receives and comprehensively analyzes the data from the above modules to assess the driver's fatigue status;

[0011] Step 5: The fatigue warning module issues a warning signal when the driver's fatigue state is detected.

[0012] Preferably, in step 1, collecting the driver's skin electrical signal based on the physiological signal detection module is specifically as follows:

[0013] The physiological signal detection module is a wristband that can detect skin electrical signals with a sampling frequency of 50 Hz. After collecting the skin electrical signals, the wristband transmits them to the data fusion processing module via Bluetooth.

[0014] Preferably, in step 2, the driver's eye state is detected in real time based on the facial image recognition module, specifically:

[0015] Based on the camera, a set of facial frame videos of the driver is collected. After obtaining the driver's facial video, each frame of the collected video is first pre-processed by grayscale using the average method, that is, the average of the three components R, B, and G is taken. Then, the grayscale image is de-noised using the morphological opening method. After that, the Canny edge detection is used to extract the edge of the eyeball area. The Canny edge detection formula is:

[0016]

[0017] Where G x and G y are the gradients of the image in the x and y directions respectively;

[0018] Since the human eye is approximately spherical in shape, the minimum enclosing circle of the eyeball contour is calculated using the minEnclosingcircle algorithm. The center and radius of the circle are (x c ,x y ) and r, so the area of the eyeball is S t =πr 2 The calculated eyeball area data is transmitted to the data fusion processing module in real time. In order to accurately determine the eye data of each driver, when the car is driving, the facial image recognition module will use the frame video of the previous 5 minutes as the initialization frame set, and use the efficient detector Dlib to detect the maximum value of the driver's eye closure duration and the average value of the eyeball area S0 in the previous 5 minutes as the judgment criteria and transmit them to the data fusion processing module.

[0019] Preferably, in step 3, the vehicle driving parameters and the driver's continuous driving time are collected based on the driving behavior analysis module, specifically:

[0020] The driving behavior analysis module includes a vehicle speed sensor and an acceleration sensor arranged in the middle of the vehicle chassis, and a pressure sensor arranged on the driver's seat, which are used to obtain the vehicle's speed, acceleration and the driver's continuous driving time. The driving behavior analysis module also obtains the throttle position through the Can bus to obtain the throttle change rate.

[0021] Preferably, in step 4, the data fusion processing module receives and comprehensively analyzes the data of the above modules to evaluate the driver's fatigue state, specifically:

[0022] The data fusion processing module receives the electrodermal signal from the physiological signal detection module and calculates the current electrodermal signal intensity, which is:

[0023]

[0024] In the formula, R is the current electrodermal signal, EDR min and EDR max are the minimum and maximum values of the electrodermal signal under normal conditions; since the fatigue characteristic of the electrodermal signal is manifested as a lower EDR value, the smaller S is, the higher the fatigue degree is. The lower limit S0 of S is set to 0.2. That is, when S ≤ 0.2, it is considered that the driver is in a fatigue driving state;

[0025] The data fusion processing module receives the maximum value of the driver's continuous eye - closing duration within the first 5 minutes and the continuous eye - closing duration during driving from the facial image recognition module. The eye - closing duration refers to the total time of the driver's continuous eye - closing. Assuming the frame rate is f, the eye - closing duration is:

[0026]

[0027] In the formula, I(·) is the indicator function, which takes 1 when S t < S0 and 0 otherwise. N is the total number of frames, S t is the eye area at time t, S threshold is the eye area limit, and S threshold is defined as 0.2S0; in the fatigue state, the reaction speed of the nervous system slows down, resulting in an increase in the eyelid closing time and difficulty in keeping the eyes open for a long time when fatigued. As fatigue deepens, the continuous eye - closing duration increases, and the driver's attention decreases or approaches the sleep state. In this invention, when the continuous eye - closing duration reaches 5 seconds or more within 5 minutes, it is considered that the driver is in fatigue driving;

[0028] The data fusion processing module receives the vehicle speed, acceleration, throttle change rate, and continuous driving time from the driving behavior analysis module. When the driver is in fatigue driving, sudden braking, unstable vehicle speed, and significantly reduced reaction rate often occur. Therefore, a comprehensive evaluation method can be used for driving behavior. The specific calculation method is as follows:

[0029]

[0030] In the formula, A is the sudden braking frequency (times / minute). Since the braking deceleration of ordinary vehicles under normal conditions is about 3 - 5m / s 2 [[ID=3,8]]when the detected braking deceleration is greater than 5.5m / s 2 it is considered that the driver makes an emergency braking operation; N brakes is the number of sudden braking times within time T; T is the observation time (minutes);

[0031] Definition of vehicle speed instability parameter: refers to the inconsistency of vehicle speed control, measured by the throttle change rate, which is usually calculated by the following formula:

[0032]

[0033] Where Δt is the time interval, T(t) is the throttle opening at the current moment, and T (t-Δt) is the throttle opening at the previous moment, the throttle change rate Indicates the rate of change of the throttle opening per unit time;

[0034] Definition of driver reaction rate: measured by reaction time. The longer the reaction time, the higher the fatigue level.

[0035]

[0036] Where, T reaction is the average reaction time; N is the number of measured reactions; T i is the reaction time of the i-th measurement;

[0037] The indicators of the comprehensive driving behavior analysis module define the comprehensive evaluation indicators as follows:

[0038]

[0039] Where F is the fatigue driving evaluation index, w1, w2, and w3 are the weights of each parameter, and w1 = 0.2, w2 = 0.3, and w3 = 0.5. When F ≥ 0.8, the driver is considered to be in a fatigue driving state.

[0040] In addition, as long-term driving can lead to decreased attention and slower reaction speed, accumulated fatigue can cause inattention, continuous driving time exceeding 4 hours is considered fatigue driving;

[0041] If the data from any module in the data receiving and computing part of the data fusion processing module reaches the preset value, it is determined that the driver is in a fatigue driving state. At this time, the data fusion processing module ends the detection and sends a signal to the fatigue driving warning module;

[0042] If all data do not reach the preset data, but at least one of the data reaches 90% of the preset data, the driver is not in a fatigued driving state, but the various indicators are close to the preset values. At this time, further judgment is required. The data receiving and calculation parts corresponding to each module send data to the logic judgment part; the logic judgment part judges the driver's status based on this data and the weights of each module. At this time, the weights of the data from the physiological signal detection module, the facial image recognition module, and the driving behavior analysis module are 0.5, 0.3, and 0.2 respectively. Therefore, the driver's status score is:

[0043]

[0044] When the driver status score Score ≥ 0.8, the driver is considered to be in a fatigue driving state, and the data fusion processing module sends a signal to the fatigue driving warning module.

[0045] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0046] The present invention provides a fatigue driving detection method that integrates multiple detection methods. The method includes collecting the driver's skin electrical signals based on a physiological signal detection module, detecting the driver's eye status in real time based on a facial image recognition module, collecting vehicle driving parameters and the driver's continuous driving time based on a driving behavior analysis module, receiving and comprehensively analyzing the data of the above modules based on a data fusion processing module, evaluating the driver's fatigue state, and issuing a warning signal when the driver's fatigue state is detected based on a fatigue warning module. The present invention combines the fatigue driving detection method with the fatigue driving warning module. Therefore, when it is detected that the driver is in a fatigue driving state, the method can also timely alarm the driver through a buzzer to make the driver temporarily awake and remind the driver to rest in time, which can effectively reduce the occurrence of fatigue driving and improve driving safety. The system can not only be used for driver fatigue detection, but can also be widely used in various work scenarios that require long-term concentration. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 Schematic diagram of the process of the present invention;

[0049] Figure 2 This is a schematic diagram of the flow chart of the data fusion processing module of the present invention;

[0050] Figure 3 Schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] The purpose of the present invention is to provide a fatigue driving detection method that integrates multiple detection methods, combining the fatigue driving detection method with a fatigue driving warning module. Therefore, when it is detected that the driver is in a fatigue driving state, the method can promptly alarm through a buzzer to make the driver temporarily awake and remind the driver to take a rest in time, effectively reducing the occurrence of fatigue driving and improving driving safety.

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] like Figure 3 As shown, the present invention provides a fatigue driving detection system based on the fusion of multiple detection methods, including:

[0055] Physiological signal detection module, the skin electrical signal of the physiological signal detection module is collected by a wristband that can detect skin electrical signals. In order to capture subtle changes in the skin electrical signal response, the required sampling frequency is 50Hz. The skin electrical signal collected by the wristband is transmitted to the data fusion processing module via Bluetooth for further processing;

[0056] A facial image recognition module, which collects a set of facial frame videos of the driver through a camera installed in the vehicle with a frame frequency of 60 Hz, and monitors the driver's eye status in real time;

[0057] The driving behavior analysis module collects the vehicle's speed and acceleration through a speed sensor and an acceleration sensor installed in the middle of the vehicle chassis, detects the driver's continuous driving time through a pressure sensor installed on the driver's seat, obtains the throttle position through the CAN bus, and thus obtains the throttle change rate. The module then transmits the key data collected by the sensors in real time to the data fusion processing module;

[0058] Data fusion processing module, which includes two parts: data reception and operation, and logic judgment. Each of the above modules corresponds to an independent data reception and operation unit, which is used to receive and comprehensively analyze the data from the above modules to assess the driver's fatigue status;

[0059] The fatigue warning module receives signals from the data fusion processing module to perform corresponding operations. When the data fusion processing module determines that the driver is in a fatigue driving state, it sends a signal to the fatigue driving warning module. After receiving the signal, the warning module immediately transmits a trigger signal to the connected buzzer, which starts the buzzer to sound an alarm to provide a stimulus signal to remind the driver to wake up. At the same time, the buzzer alarm also reminds the driver to immediately pull over and take a proper rest to ensure driving safety.

[0060] On the contrary, when the data fusion processing module determines that the driver is in a non-fatigue driving state, it will not send any signal to the fatigue driving warning module. Therefore, the warning module remains in standby mode and does not activate the buzzer. In this way, the fatigue driving warning module is only activated when necessary, avoiding unnecessary interference and improving the intelligence of the system and user experience.

[0061] like Figure 1 As shown, a fatigue driving detection method based on the fusion of multiple detection methods is characterized by including:

[0062] Step 1: Collect the driver's skin electrical signals based on the physiological signal detection module;

[0063] Step 2: Detect the driver's eye status in real time based on the facial image recognition module;

[0064] Step 3: Collect vehicle driving parameters and driver continuous driving time based on the driving behavior analysis module;

[0065] Step 4: The data fusion processing module receives and comprehensively analyzes the data from the above modules to assess the driver's fatigue status;

[0066] Step 5: The fatigue warning module issues a warning signal when the driver's fatigue state is detected.

[0067] Regarding step 1, since the system has been introduced above, it will not be described in detail here;

[0068] In step 2, the driver's eye status is detected in real time based on the facial image recognition module, specifically:

[0069] Based on the camera, a set of facial frame videos of the driver is collected. After obtaining the driver's facial video, each frame of the collected video is first pre-processed by grayscale using the average method, that is, the average of the three components R, B, and G is taken. Then, the grayscale image is de-noised by morphological opening operation (erosion followed by dilation). After that, the Canny edge detection is used to extract the edge of the eyeball area. The Canny edge detection formula is:

[0070]

[0071] Where G x and G y are the gradients of the image in the x and y directions respectively;

[0072] Since the human eye is approximately spherical in shape, the minimum enclosing circle of the eyeball contour is calculated using the minEnclosingcircle algorithm. The center and radius of the circle are (x c ,x y ) and r, so the area of the eyeball is S t =πr 2 The calculated eyeball area data is transmitted to the data fusion processing module in real time. In order to accurately determine the eye data of each driver, when the car is driving, the facial image recognition module will use the frame video of the previous 5 minutes as the initialization frame set, and use the efficient detector Dlib to detect the maximum value of the driver's eye closure duration and the average value of the eyeball area S0 in the previous 5 minutes as the judgment criteria and transmit them to the data fusion processing module.

[0073] The driving behavior analysis module in step 3 is also introduced in the system as a whole and will not be described in detail here;

[0074] In step 4, the data fusion processing module receives and comprehensively analyzes the data from the above modules to assess the driver's fatigue status, specifically:

[0075] like Figure 2 As shown, the data fusion processing module of the present invention includes two parts: data receiving operation and logic judgment. The above modules correspond to an independent data receiving operator. The processing of each module is introduced as follows:

[0076] 1. The data fusion processing module receives the skin electrical signal from the physiological signal detection module and calculates the current skin electrical signal strength as:

[0077]

[0078] Where R is the current skin electrical signal, EDR min and EDR maxis the minimum and maximum values of the skin electrical signal under normal conditions; since the fatigue characteristic of the skin electrical signal is manifested as a lower EDR value, the smaller S is, the higher the fatigue degree is. The lower limit S0 of S is set to 0.2, that is, when S ≤ 0.2, it is considered that the driver is in a fatigue driving state;

[0079] 2. The data fusion processing module receives the maximum value of the driver's continuous eye-closure duration within the first 5 minutes from the facial image recognition module and the continuous eye-closure duration during driving. The continuous eye-closure duration refers to the total time of the driver's continuous eye-closure. Let the frame rate (i.e., the number of frames per second) be f, then the continuous eye-closure duration is:

[0080]

[0081] In the formula, I(·) is an indicator function, which takes 1 when S t < S0 and 0 otherwise. N is the total number of frames, S t is the eye area at time t, S threshold is the eye area limit, and S threshold is defined as 0.2S0; in the fatigue state, the reaction speed of the nervous system slows down, resulting in an increase in the eyelid closure time and difficulty in keeping the eyes open for a long time when fatigued. As the fatigue deepens, the continuous eye-closure duration increases, and the driver's attention decreases or approaches the sleep state. In this invention, when the continuous eye-closure duration reaches 5 seconds or more within 5 minutes, it is considered that the driver is in a fatigue driving state;

[0082] 3. The data fusion processing module receives the vehicle speed, acceleration, throttle change rate, and continuous driving time from the driving behavior analysis module. When the driver is in a fatigue driving state, sudden braking, unstable vehicle speed, and significantly reduced reaction rate often occur. Therefore, a comprehensive evaluation method can be used for driving behavior. The specific calculation method is as follows:

[0083]

[0084] In the formula, A is the sudden braking frequency (times / minute). Since the braking deceleration of ordinary vehicles under normal conditions is about 3 - 5 m / s 2 , when the detected braking deceleration is greater than 5.5 m / s 2 , it is considered that the driver makes an emergency braking operation; N brakes is the number of sudden braking times within time T; T is the observation time (minutes);

[0085] Define the vehicle speed instability parameter: which refers to the incoherence of vehicle speed control and is measured by the throttle change rate. The throttle change rate is usually calculated by the following formula:

[0086]

[0087] Where Δt is the time interval, T(t) is the throttle opening at the current moment, and T (t-Δt) is the throttle opening at the previous moment, the throttle change rate Indicates the rate of change of the throttle opening per unit time;

[0088] Definition of driver reaction rate: measured by reaction time. The longer the reaction time, the higher the fatigue level.

[0089]

[0090] Where, T rection is the average reaction time; N is the number of measured reactions; T i is the reaction time of the i-th measurement;

[0091] The indicators of the comprehensive driving behavior analysis module define the comprehensive evaluation indicators as follows:

[0092]

[0093] Where F is the fatigue driving evaluation index, w1, w2, and w3 are the weights of each parameter, and w1 = 0.2, w2 = 0.3, and w3 = 0.5. When F ≥ 0.8, the driver is considered to be in a fatigue driving state.

[0094] In addition, as long-term driving can lead to decreased attention and slower reaction speed, accumulated fatigue can cause inattention, continuous driving time exceeding 4 hours is considered fatigue driving;

[0095] If the data from any module in the data receiving and computing part of the data fusion processing module reaches the preset value, it is determined that the driver is in a fatigue driving state. At this time, the data fusion processing module ends the detection and sends a signal to the fatigue driving warning module;

[0096] If all data do not reach the preset data, but at least one of the data reaches 90% of the preset data, the driver is not in a fatigued driving state, but the various indicators are close to the preset values. At this time, further judgment is required. The data receiving and calculation parts corresponding to each module send data to the logic judgment part; the logic judgment part judges the driver's status based on this data and the weights of each module. At this time, the weights of the data from the physiological signal detection module, the facial image recognition module, and the driving behavior analysis module are 0.5, 0.3, and 0.2 respectively. Therefore, the driver's status score is:

[0097]

[0098] When the driver status score Score ≥ 0.8, the driver is considered to be in a fatigue driving state, and the data fusion processing module sends a signal to the fatigue driving warning module.

[0099] In step 5, the fatigue warning module issues a warning signal when the driver's fatigue state is detected, specifically:

[0100] The fatigue driving warning module in the present invention performs corresponding operations by receiving signals from the data fusion processing module. When the data fusion processing module determines that the driver is in a fatigue driving state, it sends a signal to the fatigue driving warning module. After receiving the signal, the warning module immediately transmits a trigger signal to the buzzer connected thereto, starting the buzzer to sound an alarm, providing a stimulus signal to remind the driver to wake up. At the same time, the buzzer alarm also prompts the driver to immediately pull over and take a proper rest to ensure driving safety.

[0101] Conversely, if the data fusion processing module determines that the driver is not fatigued, it does not send any signals to the fatigue warning module. As a result, the warning module remains in standby mode and the buzzer is not activated. This way, the fatigue warning module activates only when necessary, avoiding unnecessary interference and improving system intelligence and user experience.

[0102] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0103] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A fatigue driving detection method based on the fusion of multiple detection methods, characterized in that: include: Step 1: Collect the driver's skin electrical signals based on the physiological signal detection module; Step 2: Detect the driver's eye status in real time based on the facial image recognition module; Step 3: Collect vehicle driving parameters and driver continuous driving time based on the driving behavior analysis module; Step 4: The data fusion processing module receives and comprehensively analyzes the data from the above modules to assess the driver's fatigue status; Step 5: The fatigue warning module issues a warning signal when the driver's fatigue state is detected.

2. The method according to claim 1, characterized in that In step 1, the driver's skin electrical signals are collected based on the physiological signal detection module, specifically: The physiological signal detection module is a wristband that can detect skin electrical signals with a sampling frequency of 50 Hz. After collecting the skin electrical signals, the wristband transmits them to the data fusion processing module via Bluetooth.

3. The method according to claim 2, characterized in that In step 2, the driver's eye status is detected in real time based on the facial image recognition module, specifically: Based on the camera, a set of facial frame videos of the driver is collected. After obtaining the driver's facial video, each frame of the collected video is first pre-processed by grayscale using the average method, that is, the average of the three components R, B, and G is taken. Then, the grayscale image is de-noised using the morphological opening method. After that, the Canny edge detection is used to extract the edge of the eyeball area. The Canny edge detection formula is: Where G x and G y are the gradients of the image in the x and y directions respectively; Since the human eye is approximately spherical in shape, the minimum enclosing circle of the eyeball contour is calculated using the minEnclosingcircle algorithm. The center and radius of the circle are (x c ,x y ) and r, so the area of the eyeball is S t =πr 2 The calculated eyeball area data is transmitted to the data fusion processing module in real time. In order to accurately determine the eye data of each driver, when the car is driving, the facial image recognition module will use the frame video of the first 5 minutes as the initialization frame set, and use the efficient detector Dlib to detect the maximum value of the driver's eye closure duration and the average value of the eyeball area S0 in the first 5 minutes as the judgment criteria and transmit them to the data fusion processing module.

4. The method according to claim 3, characterized in that In step 3, the vehicle driving parameters and the driver's continuous driving time are collected based on the driving behavior analysis module, specifically: The driving behavior analysis module includes a vehicle speed sensor and an acceleration sensor arranged in the middle of the vehicle chassis, and a pressure sensor arranged on the driver's seat, which are used to obtain the vehicle's speed, acceleration and the driver's continuous driving time. The driving behavior analysis module also obtains the throttle position through the Can bus to obtain the throttle change rate.

5. The method according to claim 4, characterized in that In step 4, the data fusion processing module receives and comprehensively analyzes the data from the above modules to assess the driver's fatigue status, specifically: The data fusion processing module receives the skin electrical signal from the physiological signal detection module and calculates the current skin electrical signal strength as: Where R is the current skin electrical signal, EDR min and EDR max are the minimum and maximum values of the skin electrode signal under normal conditions; since the fatigue characteristic of the skin electrode signal is manifested as a lower EDR value, the smaller S is, the higher the degree of fatigue. The lower limit S0 is set to 0.2, that is, when S≤0.2, the driver is considered to be in a fatigued driving state; The data fusion processing module receives the maximum eye-closing duration of the driver in the first 5 minutes and the eye-closing duration during driving from the facial image recognition module. The eye-closing duration refers to the total time the driver closes his eyes continuously. If the frame rate is f, the eye-closing duration is: Where, I(·) is an indicator function, which takes 1 when S t < is less than S0 and 0 otherwise. N is the total number of frames, and S t is the eye area at time t, and S threshold is the eye area limit, and it is defined that S threshold = 0.2S0; In the fatigue state, the reaction speed of the nervous system slows down, resulting in an increase in the eyelid closing time. When fatigued, it is difficult for the eyes to stay open for a long time. As the fatigue deepens, the duration of eye closure increases, and the driver's attention decreases or approaches the sleep state. In the present invention, when the duration of eye closure reaches 5 seconds or more within 5 minutes, it is considered that the driver is fatigued while driving; The data fusion processing module receives the vehicle speed, acceleration, throttle change rate and continuous driving time from the driving behavior analysis module. When the driver is in a state of fatigue driving, sudden braking, unstable vehicle speed and significantly reduced reaction rate often occur. Therefore, a comprehensive evaluation method can be used for driving behavior. The specific calculation method is as follows: Emergency braking frequency: Where A is the frequency of emergency braking (times / minute). Since the braking deceleration of ordinary vehicles under normal circumstances is about 3-5m / s 2 Therefore, when the braking deceleration is detected to be greater than 5.5m / s 2 When the driver makes an emergency braking operation; N brakes is the number of emergency brakes within time T; T is the observation time (minutes); Definition of vehicle speed instability parameter: refers to the inconsistency of vehicle speed control, measured by the throttle change rate, which is usually calculated by the following formula: Where Δt is the time interval, T(t) is the throttle opening at the current moment, and T (t-Δt) is the throttle opening at the previous moment, the throttle change rate Indicates the rate of change of the throttle opening per unit time; Definition of driver reaction rate: measured by reaction time. The longer the reaction time, the higher the fatigue level. Where, T reaction is the average reaction time; N is the number of measured reactions; T i is the reaction time of the i-th measurement; The indicators of the comprehensive driving behavior analysis module define the comprehensive evaluation indicators as follows: Where F is the fatigue driving evaluation index, w1, w2, and w3 are the weights of each parameter, and w1 = 0.2, w2 = 0.3, and w3 = 0.

5. When F ≥ 0.8, the driver is considered to be in a fatigue driving state. In addition, as long-term driving can lead to decreased attention and slower reaction speed, accumulated fatigue can cause inattention, continuous driving time exceeding 4 hours is considered fatigue driving; If the data from any module in the data receiving and computing part of the data fusion processing module reaches the preset value, it is determined that the driver is in a fatigue driving state. At this time, the data fusion processing module ends the detection and sends a signal to the fatigue driving warning module; If all the data do not reach the preset data, but at least one of the data reaches 90% of the preset data, then the driver is not in a fatigue driving state, but the indicators are close to the preset values. At this time, further judgment is required, and the data receiving and calculation parts corresponding to each module send data to the logic judgment part; The logic judgment part judges the driver's status based on this data and the weight of each module. At this time, the weights of the data from the physiological signal detection module, the facial image recognition module, and the driving behavior analysis module are 0.5, 0.3, and 0.2 respectively. Therefore, the driver's status score is: When the driver status score Score ≥ 0.8, the driver is considered to be in a fatigue driving state, and the data fusion processing module sends a signal to the fatigue driving warning module.

Images