The invention discloses a
pilot dangerous driving
behavior monitoring method based on multi-
modal data fusion, and belongs to the field of multi-
modal data fusion monitoring, and the method comprises the steps: obtaining the physiological data of a
pilot in the take-off and landing states through a multi-element sensor, judging the fatigue state of the
pilot, and obtaining the dangerous driving behavior of the pilot. An eye tracker and a camera are used for conducting data collection on a pilot in the take-off and landing states, behavior data of the blinking frequency, the
eyelid closure degree, the
pupil change, the yawning frequency, the nodding frequency, the average
moving speed of an operating lever and the moving direction of the pilot are obtained, and the fatigue state of the pilot is judged; the method comprises the following steps: constructing a multi-
modal data fusion network based on Transform, and training the multi-
modal data fusion network; the driving behavior of a pilot is monitored by using the trained network based on the Transform multi-
modal data fusion; through a multi-channel
interaction technology, the comprehensiveness and accuracy of information are improved, the influence of environment and individual differences is reduced, and the stability and robustness are enhanced.