Driver state monitoring method and system based on multi-sensor fusion

Abstract

The invention discloses a driver state monitoring method based on multi-sensor fusion. The method comprises the steps: carrying out zooming of an obtained image at different scales, generating an image pyramid, and carrying out the positioning of a human face and five sense organs; as for the human face and the five sense organs obtained through positioning, extracting eye and mouth area featuresthrough a convolutional neural network, performing statistics on the eye closing duration within a certain period of time, if the eye closing duration exceeds a threshold value, judging that the driver is in a first fatigue state, performing statistics on the mouth opening duration within a certain period of time, if the mouth opening duration exceeds the threshold value, judging that the driver is in a second fatigue state; collecting heart rate information of the driver, calculating a heart rate variance D through a continuous time queue, and when D is less than a threshold value T, judgingjudged that the driver is in a third fatigue state; and setting the weights of the three fatigue states, superposing the three fatigue states, judging the fatigue level, and reminding in a voice broadcast and / or seat vibration mode. The face recognition result and the heart rate monitoring result are dynamically fused, the fatigue driving state can be accurately judged and the precision is higher.

Description

technical field [0001] The invention relates to the technical field of fatigue driving detection, in particular to a driver state monitoring method and system based on multi-sensor fusion. Background technique [0002] At present, there are many research methods for the detection of driver fatigue state, which can be roughly divided into detection based on driver’s physiological signal, detection based on driver’s operation behavior, detection based on vehicle state information and detection based on driver’s physiological response. feature detection methods. [0003] The accuracy of judging fatigue driving based on physiological signals (EEG signals, ECG signals, etc.) is high, and for all healthy drivers, the physiological signals have little difference and have commonalities, but the traditional physiological signal acquisition method requires The use of contact measurement brings a lot of inconvenience and limitations to the practical application of driver fatigue detec...

AI Technical Summary

Problems solved by technology

[0005] The driver's fatigue state can also be estimated by using vehicle driving state information such as vehicle trajectory changes and lane line deviations, but the driving state of the vehicle is also related to many environmental factors such as vehicle characteristics and roads, and is related to the driver's driving experience and driving habits. Therefore, there are many interference factors that need to be considered in judging fatigue driving based on vehicle state information

[0006] The fatigue driving discrimination method based on the driver's physiological response characteristics refers to the use of the driver's eye characteristics, mouth movement characteristics, etc. to infer the driver's fatigue state. Closing time and yawning actions can be directly used to detect fatigue, but the main shortcomings are as follows: (1) complex changes in illumination (2) variable head postures of drivers (3) individual differences among different drivers

Affects the accuracy of driver's face detection and facial features positioning, thereby reducing the robustness of the driver's driving behavior model space

[0007] However, due to the individual differences of drivers, the detection method of a single detection index has limitations, mainly manifested in low accuracy, prone to deviation, etc.

Images