Fatigue driving detection method based on multi-modal information fusion

A fatigue driving and detection method technology, applied in the detection field, can solve the problems of incomplete driver fatigue state, poor robustness and stability, complicated preprocessing operation, etc., achieve good robustness and stability, improve accuracy, Characterize the full effect

CN112528815APending Publication Date: 2021-03-19XIDIAN UNIV
8 Cites 12 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Publication Date
2021-03-19

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a fatigue driving detection method based on multi-modal information fusion. The method comprises the steps: collecting electroencephalogram signals, facial images and vehicle driving lane images at the same time, constructing and training convolutional neural networks respectively, collecting and inputting to-be-detected data into the trained convolutional neural networks for preliminary prediction, and judging whether a driver is in a fatigue state or not by integrating preliminary prediction results. The method comprises the following specific steps: generating a training set, respectively constructing and training electroencephalogram signal classification, positioning eye and mouth areas, judging opening and closing states of eyes and mouth, positioning a convolutional neural network of lane line positions, collecting to-be-detected data, preliminarily predicting a fatigue driving state, and fusing and judging a fatigue state of a driver. According to the method, the problems of single driver fatigue state judgment basis mode and incomplete representation of the driver fatigue state are solved, the method has good robustness and stability, and the fatigue driving detection precision is improved.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention belongs to the technical field of detection, and further relates to a fatigue driving detection method based on multimodal information fusion in the technical field of information detection. The present invention can collect the driver's EEG signal, facial image and vehicle lane image to carry out multi-modal information fusion analysis to judge whether the driver is in a fatigue driving state. Background technique

[0002] Fatigue driving refers to the phenomenon that the driver's physiological reaction is obviously slowed down in the fatigue state, which leads to the phenomenon that the driving skills are significantly reduced. Accurate and real-time detection of the driver's fatigue state is of great significance to driving safety. At present, research on fatigue driving detection mainly includes fatigue detection based on driver's physiological indicators, fatigue driving detection based on driver's behavior characteristics, and fatig...

Examples

Embodiment Construction

[0056] Attached below figure 1 The specific steps for realizing the present invention are further described.

[0057] Step 1, generate a training set.

[0058] A set of data is composed of the EEG signals, facial images, and images of the motor vehicle track where the driver is in a fatigue state or non-fatigue state collected synchronously each time, and at least 200 groups are collected; the EEG signal sampling frequency is 256Hz, and each The data collection time of the group is 2 minutes, and 10 images of the driver's face and 20 images of the motor vehicle track where the vehicle is located are collected per second.

[0059] Extract the EEG signals, facial images, and vehicle track images from each set of data to form the EEG signal training set, facial image training set, and vehicle track image training set. For each face image training set, The locations of the eyes and mouth regions in the image are annotated.

[0060] Step 2, construct and train a convolutional ne...