A non-contact in-vehicle driver fatigue detection method, system and electronic device

Abstract

The present invention discloses a non-contact in-transit fatigue detection method, system and electronic device for drivers, which relates to the field of fatigue state detection. The method includes: obtaining the heartbeat signal of a target driver based on an FMCW millimeter-wave radar; obtaining physiological characteristics by applying heart rate variability analysis according to the heartbeat signal; obtaining the pressing frequency of the target driver pressing the vigilance button; respectively normalizing the time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN and the pressing frequency; applying a preset weight to calculate the comprehensive characteristics of the normalized time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN and the pressing frequency; and obtaining the fatigue state prediction result of the target driver by applying a logistic regression method according to the comprehensive characteristics. The present invention can effectively detect the fatigue state of train drivers in a non-contact manner.

Description

Technical Field

[0001] The present invention relates to the field of fatigue state detection, and in particular to a non-contact driver on-the-road fatigue detection method, system and electronic equipment. Background Art

[0002] Train drivers play one of the most important safety roles among railway transport personnel. They often work in harsh environments, with long tasks, heavy loads, and irregular work and rest schedules. As the level of automation in the railway industry continues to improve, the drivers' on-the-road driving work has shifted from more physical requirements to cognitive requirements. This constant vigilance may cause drivers to suffer more severe mental fatigue, threatening train driving safety. How to realize driving fatigue detection in the existing train driver driving environment has become an urgent need.

[0003] Existing technology solution 1, "A Method for Detecting Driver Fatigue on the Road and a Remote Monitoring System for Train Operation Status," identifies driver fatigue by identifying factors such as the upward curve of the mouth corners and the degree of eyelid closure. This technology is not only susceptible to individual driving attributes (such as eye size and glasses) and external environmental factors (such as frequent changes in tunnel lighting), but also suffers from poor generalization of the overall fatigue model based on a single driver behavior dimension.

[0004] The second existing technology solution, "An EEG-Based High-Speed Train Driver Fatigue Detection Method and Device," uses an EEG sampling module to obtain EEG data, extracts features from the EEG data to obtain EEG feature quantities, and constructs an EEG prediction model based on an intelligent algorithm based on the EEG feature quantities and the high-speed train driver's pedaling frequency. This technology uses EEG electroencephalogram (EEG) to identify train driver fatigue. Although EEG fatigue identification has a high accuracy rate, EEG is a highly contact-based device that requires prolonged wear, which can place a severe physiological and psychological burden on the driver and create resistance. Furthermore, train driving time is long, making it impossible to wear EEG devices for long periods of time. Summary of the Invention

[0005] The purpose of the present invention is to provide a non-contact driver fatigue detection method, system and electronic equipment, which can effectively detect the fatigue state of a train driver in a non-contact manner.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A non-contact method for detecting driver fatigue on the road, the method comprising:

[0008] Based on FMCW millimeter wave radar, the heartbeat signal of the target driver is obtained;

[0009] Based on the heartbeat signal, apply heart rate variability analysis to obtain physiological characteristics; the physiological characteristics include the time-domain index IBI interval, the time-domain index RMSSD, and the time-domain index SDNN;

[0010] Obtain the pressing frequency of the target driver pressing the vigilance button;

[0011] Normalize the time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency respectively;

[0012] Apply preset weights to calculate the comprehensive characteristics of the normalized time-domain index IBI interval, time-domain index RMSSD, time-domain index SDNN, and pressing frequency;

[0013] Based on the comprehensive characteristics, apply the logistic regression method to obtain the fatigue state prediction result of the target driver; the prediction result is a fatigue state or a non-fatigue state.

[0014] Optionally, the obtaining of the heartbeat signal of the target driver based on the FMCW millimeter-wave radar specifically includes:

[0015] Based on the FMCW millimeter-wave radar, obtain the distance of the chest cavity of the target driver fluctuating over time;

[0016] According to the distance of the chest cavity fluctuating over time, determine the phase signal of the FMCW millimeter-wave radar;

[0017] Filter the phase signal to obtain an initial heartbeat signal;

[0018] Apply the VMD modal decomposition algorithm to optimize the initial heartbeat signal to obtain a heartbeat signal.

[0019] Optionally, obtain the pressing frequency of the target driver pressing the vigilance button through an adjustment module; wherein, the adjustment module includes a single-channel resistive pressure sensor, a data conversion module, a communication module, and a host computer connected in sequence;

[0020] The single-channel resistive pressure sensor is used to convert the pressure of the target driver pressing the vigilance button into a resistance value;

[0021] The data conversion module is used to convert the resistance value from an analog quantity to a digital quantity to obtain a digital resistance value;

[0022] The communication module is used to transmit the digital resistance value to the host computer;

[0023] The host computer determines the pressing frequency according to the digital resistance value.

[0024] Optionally, the calculation formula of the comprehensive feature is:

[0025] x = 0.15 × IBI interval + 2.23 × RMSSD + 0.1 × SDNN + 0.5 × vigilant button frequency.

[0026] Optionally, according to the comprehensive feature, applying the logistic regression method to obtain the fatigue state prediction result of the target driver, specifically including:

[0027] According to the comprehensive feature, apply Calculate the fatigue state prediction value of the target driver; where x is the comprehensive feature; y is the fatigue state prediction value;

[0028] When the fatigue state prediction value of the target driver is greater than the set threshold, the target driver is in a fatigue state; otherwise, the target driver is in a non-fatigue state.

[0029] A non-contact in-transit driver fatigue detection system applies the above non-contact in-transit driver fatigue detection method, and the detection system includes:

[0030] A heartbeat signal acquisition module, configured to acquire the heartbeat signal of the target driver based on an FMCW millimeter-wave radar;

[0031] A physiological feature determination module, configured to obtain physiological features according to the heartbeat signal by applying heart rate variability analysis; the physiological features include the time-domain index IBI interval, the time-domain index RMSSD, and the time-domain index SDNN;

[0032] A pressing frequency acquisition module, configured to acquire the pressing frequency of the target driver pressing the vigilant button;

[0033] A normalization module, configured to perform normalization processing on the time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency respectively;

[0034] A calculation module, configured to calculate the comprehensive feature of the normalized time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency by applying a preset weight;

[0035] A prediction module, configured to obtain the fatigue state prediction result of the target driver according to the comprehensive feature by applying the logistic regression method; the prediction result is a fatigue state or a non-fatigue state.

[0036] An electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above non-contact in-transit driver fatigue detection method.

[0037] Optionally, the memory is a readable storage medium.

[0038] According to the specific embodiments provided by the present invention, the following technical effects are disclosed by the present invention:

[0039] The present invention provides a non-contact in-transit fatigue detection method, system and electronic device for drivers. By acquiring the heartbeat signal of a target driver based on an FMCW millimeter-wave radar, the pressing frequency of the target driver pressing a vigilance button is also acquired; physiological characteristics are obtained by applying heart rate variability analysis to the heartbeat signal; the physiological characteristics include the time-domain index IBI interval, the time-domain index RMSSD, and the time-domain index SDNN; the time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency are respectively normalized; a comprehensive feature of the normalized time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency is calculated by applying a preset weight; according to the comprehensive feature, a fatigue state prediction result of the target driver is obtained by applying a logistic regression method; the prediction result is a fatigue state or a non-fatigue state. Therefore, the present invention effectively detects the fatigue state of train drivers in a non-contact manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 It is a flowchart of the non-contact in-transit fatigue detection method for drivers of the present invention;

[0042] Figure 2 It is a flowchart of the VMD modal decomposition algorithm of the present invention;

[0043] Figure 3 It is a flowchart of the variable mode decomposition based on the genetic algorithm and signal permutation entropy of the present invention;

[0044] Figure 4 It is a schematic diagram of the behavior data acquisition of the train driver vigilance button of the present invention;

[0045] Figure 5 It is a flowchart of the actual application of the non-contact in-transit fatigue detection method for drivers of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] The purpose of the present invention is to provide a non-contact driver fatigue detection method, system and electronic equipment, which can effectively detect the fatigue state of the train driver in a non-contact manner.

[0048] The main steps of this patent are divided into three steps. The first step is to use FMCW millimeter wave radar equipment to collect the driver's physiological signals; the second step is to use the alert button device to collect the driver's behavioral signals; and the third step is to combine the driver's physiological signals and the alert button signal to determine the driver's fatigue state.

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] Example 1

[0051] like Figure 1 As shown, the present invention provides a non-contact method for detecting driver fatigue on the road, the detection method comprising:

[0052] Step S1: Acquire the target driver's heartbeat signal based on the FMCW millimeter wave radar.

[0053] In practical applications, FMCW millimeter-wave radar is used to collect physiological data from train drivers, aiming to measure the driver's physiological characteristics using non-contact equipment. FMCW millimeter-wave radar can detect changes in the target's distance dimension by transmitting and receiving reflected waveforms in real time, and is commonly used for target position detection. Physiological sign detection involves the weak vibrations caused by human breathing and heartbeat. Generally, a heartbeat causes chest wall displacement of 1mm to 2mm. The heartbeat waveform itself exhibits a certain periodicity, and the heartbeat signal frequency primarily ranges from 0.8Hz to 2.0Hz. The radar's phase sensitivity to subtle motion displacements can be exploited to analyze and derive physiological parameter information.

[0054] S1 specifically includes:

[0055] Step S11: Based on the FMCW millimeter wave radar, the fluctuation distance of the target driver's chest over time is obtained.

[0056] Step S12: Determine the phase signal of the FMCW millimeter-wave radar according to the undulating distance of the chest over time.

[0057] In practical applications, the basic formula for chest vibration obtained from the millimeter-wave FMCW radar signal is:

[0058]

[0059] Where, is the phase signal of the radar; d(t) is the distance change of the chest over time; is the wavelength of the radar signal.

[0060] Step S13: Filter the phase signal to obtain an initial heartbeat signal.

[0061] In practical applications, after obtaining the phase waveform as described above, band-pass filtering is performed to obtain the heartbeat signal. Specifically, the IIR filter parameters of the heartbeat signal are shown in Table 1.

[0062] Table 1 IIR filter parameter settings for the heartbeat signal

[0063]

[0064] Step S14: Apply the VMD modal decomposition algorithm to optimize the initial heartbeat signal to obtain a heartbeat signal.

[0065] In practical applications, based on the obtained initial heartbeat signal, the VMD modal decomposition algorithm is used for further noise reduction and optimization.

[0066] Although the peak frequency of the heart rate can be clearly found in the initial heartbeat signal, there are always harmonic and noise frequencies in the separated heartbeat frequency band spectrum, which affects the extraction of the heart rate peak and the subsequent extraction of heart rate variability characteristics, and further noise reduction and optimization are required. The VMD modal decomposition algorithm is as Figure 2 shown. After determining two hyperparameters, the decomposition layer number k and the penalty factor α, signal modal decomposition can be performed. The parameter settings of k and α will both affect the signal decomposition quality. If the hyperparameter k is small, modal aliasing will occur, causing some noise to be included in the useful signal. When the hyperparameter k is large, important components in the signal will be decomposed into multiple modes; the hyperparameter α is used to balance the bandwidth constraint. When it is too large or too small, the bandwidth constraint will be too tight or too loose, affecting the signal decomposition quality. Therefore, how to set the two hyperparameters k and α in different application scenarios is the difficulty of signal variable modal decomposition.

[0067] As Figure 3 shown, use the genetic algorithm and signal permutation entropy to optimize the K and α parameters in the VMD modal decomposition algorithm. Thus, a relatively pure heartbeat signal can be obtained according to the phase signal of the radar.

[0068] Step S2: Based on the heartbeat signal, apply heart rate variability analysis to obtain physiological characteristics; the physiological characteristics include the time-domain index IBI interval, the time-domain index RMSSD, and the time-domain index SDNN.

[0069] In practical applications, the physiological characteristics of the driver are extracted according to the heartbeat signal. Heart rate variability analysis is one of the non-invasive techniques used to evaluate the state of the autonomic nervous system, and it is an important indicator of an individual's ability to effectively respond to life stress and daily stress. The indexes extracted in the present invention are:

[0070] (1) The time-domain index IBI (Inter-Beat Interval, heartbeat interval) interval represents the distance value between adjacent two consecutive heartbeat peak values.

[0071] (2) The time-domain index RMSSD is used to measure the change of consecutive IBI. RMSSD is the root mean square value of the difference between adjacent heartbeat intervals throughout the process, and the calculation method is as follows:

[0072]

[0073] where N IBI is the total number of measured IBI; IBI(i) - IBI(i - 1) is the adjacent IBI interval, IBI(i) is the heartbeat interval at the i-th moment; IBI(i - 1) is the heartbeat interval at the (i - 1)-th moment.

[0074] (3) The time-domain index SDNN is used to measure the standard deviation of all IBI, and to measure the difference of IBI. SDNN refers to the standard deviation of all heartbeat intervals during the detection time, and the calculation method is as follows:

[0075]

[0076] In the formula: N IBI is the total number of measured IBI; is the average value of each measured IBI.

[0077] Step S3: Obtain the pressing frequency of the target driver pressing the vigilance button.

[0078] Specifically, the pressing frequency of the target driver pressing the vigilance button is obtained through the conditioning module; wherein, the conditioning module includes a single-channel resistive pressure sensor, a data conversion module, a communication module, and a host computer connected in sequence.

[0079] The single-channel resistive pressure sensor is used to convert the pressure of the target driver pressing the vigilance button into a resistance value. The data conversion module is used to convert the resistance value from an analog quantity to a digital quantity, obtaining a digital resistance value. The communication module is used to transmit the digital resistance value to the host computer. The host computer determines the pressing frequency according to the digital resistance value.

[0080] In practical applications, obtaining the driver's action data (vigilance button behavior data) aims to obtain the change characteristics of the vigilance button frequency during the process of driver fatigue generation.

[0081] The hardware of the train driver vigilance button behavior data acquisition and analysis platform mainly consists of a flexible thin-film pressure sensor, a single-channel resistive pressure sensor conditioning module, a voltage analog input to RS485 module, an RS485 to USB module, and a mushroom head button box.

[0082] The flexible thin-film pressure sensor is a resistive sensor, and its output resistance changes with the pressure magnitude attached to the surface. It is placed above the mushroom head of the vigilance button box, as Figure 4 shown. After the driver presses the vigilance button, the single-channel resistive pressure sensor conditioning module converts the corresponding resistance value change into the pressure magnitude according to a specific pressure-resistance curve, converts the voltage signal into the RS485 interface standard ModBus-RTU communication protocol through the analog signal to RS485 module, and finally completes the data acquisition and real-time display on the PC side through the RS485 to USB module.

[0083] Step S4: Normalize the time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency respectively.

[0084] In practical applications, as Figure 5 shown, due to the different data dimensions, such as the heart rate frequency is 60Bpm - 100Bpm, the breathing frequency is 8Bpm - 20Bpm, and the vigilance button tapping frequency is 0 - 1, the data with larger dimensions will dominate in the objective function, weakening the contribution of the data with smaller dimensions.

[0085] The present invention performs a normalization operation on physiological and behavioral data, and the normalization formula is:

[0086]

[0087] Step S5: Apply the preset weights to calculate the comprehensive characteristics of the normalized time-domain index IBI interval, time-domain index RMSSD, time-domain index SDNN, and pressing frequency.

[0088] The calculation formula for the comprehensive characteristics is:

[0089] x = 0.15 × IBI interval + 2.23 × RNSSD + 0.1 × SDNN + 0.5 × Vigilance button frequency.

[0090] Step S6: According to the comprehensive feature, apply the logistic regression method to obtain the fatigue state prediction result of the target driver; the prediction result is a fatigue state or a non-fatigue state.

[0091] S6 specifically includes:

[0092] Step S61: According to the comprehensive feature, apply Calculate the fatigue state prediction value of the target driver; where x is the comprehensive feature; y is the fatigue state prediction value.

[0093] Step S62: When the fatigue state prediction value of the target driver is greater than the set threshold, the target driver is in a fatigue state; otherwise, the target driver is in a non-fatigue state.

[0094] Specifically, when y < 0.5, the state is considered a non-fatigue state, and when y > 0.5, the state is considered a fatigue state.

[0095] Embodiment 2

[0096] In order to execute the method corresponding to the above Embodiment 1 to achieve the corresponding functions and technical effects, the following provides a non-contact in-transit fatigue detection system for drivers, and the detection system includes:

[0097] A heartbeat signal acquisition module, configured to acquire the heartbeat signal of the target driver based on an FMCW millimeter-wave radar.

[0098] A physiological feature determination module, configured to obtain physiological features according to the heartbeat signal by applying heart rate variability analysis; the physiological features include the time-domain index IBI interval, the time-domain index RMSSD, and the time-domain index SDNN.

[0099] A pressing frequency acquisition module, configured to acquire the pressing frequency of the target driver pressing the vigilance button.

[0100] A normalization module, configured to perform normalization processing on the time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency respectively.

[0101] A calculation module, configured to calculate the comprehensive features of the normalized time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency by applying preset weights.

[0102] A prediction module, configured to apply a logistic regression method based on the comprehensive features to obtain a prediction result of the fatigue state of the target driver; the prediction result is a fatigue state or a non-fatigue state.

[0103] Embodiment III

[0104] An embodiment of the present invention provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the non-contact in-route fatigue detection method for drivers in Embodiment I.

[0105] Optionally, the above electronic device may be a server.

[0106] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the non-contact in-route fatigue detection method for drivers in Embodiment I.

[0107] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0108] In this article, specific examples are used to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A non-contact method for detecting driver fatigue on the road, characterized in that, The detection method includes: Based on the FMCW millimeter-wave radar, obtaining the heartbeat signal of the target driver; According to the heartbeat signal, applying heart rate variability analysis to obtain physiological characteristics; the physiological characteristics include the time-domain index IBI interval, the time-domain index RMSSD, and the time-domain index SDNN; Obtaining the pressing frequency of the target driver pressing the vigilance button; Normalizing the time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency respectively; Applying a preset weight to calculate the comprehensive characteristics of the normalized time-domain index IBI interval, time-domain index RMSSD, time-domain index SDNN, and pressing frequency; According to the comprehensive characteristics, applying the logistic regression method to obtain the fatigue state prediction result of the target driver; the prediction result is the fatigue state or the non-fatigue state.

2. The non-contact in-vehicle driver fatigue detection method according to claim 1, characterized in that The obtaining of the heartbeat signal of the target driver based on the FMCW millimeter-wave radar specifically includes: Based on the FMCW millimeter-wave radar, obtaining the distance of the chest cavity of the target driver fluctuating over time; According to the distance of the chest cavity fluctuating over time, determining the phase signal of the FMCW millimeter-wave radar; Filtering the phase signal to obtain the initial heartbeat signal; Applying the VMD modal decomposition algorithm to optimize the initial heartbeat signal to obtain the heartbeat signal.

3. The non-contact in-transit driver fatigue detection method according to claim 1, wherein Obtaining the pressing frequency of the target driver pressing the vigilance button through the conditioning module; wherein, the conditioning module includes a single-channel resistive pressure sensor, a data conversion module, a communication module, and a host computer connected in sequence; The single-channel resistive pressure sensor is used to convert the pressure of the target driver pressing the vigilance button into a resistance value; The data conversion module is used to convert the resistance value from analog quantity to digital quantity to obtain the digital resistance value; The communication module is used to transmit the digital resistance value to the host computer; The host computer determines the pressing frequency according to the digital resistance value.

4. The non-contact in-transit driver fatigue detection method according to claim 1, characterized in that, The calculation formula of the comprehensive characteristics is: x0.15×IBI interval + 2.23×RMSSD + 0.1×SDNN + 0.5×vigilance button frequency.

5. The non-contact in-vehicle driver fatigue detection method according to claim 1, wherein According to the comprehensive characteristics, applying the logistic regression method to obtain the fatigue state prediction result of the target driver specifically includes: According to the comprehensive feature, apply to calculate the fatigue state prediction value of the target driver; where x is the comprehensive feature and y is the fatigue state prediction value; When the fatigue state prediction value of the target driver is greater than the set threshold, the target driver is in the fatigue state; otherwise, the target driver is in the non-fatigue state.

6. A non-contact in-vehicle driver fatigue detection system, characterized in that, The detection system includes: A heartbeat signal acquisition module, which is used to obtain the heartbeat signal of the target driver based on the FMCW millimeter-wave radar; A physiological characteristic determination module, which is used to obtain physiological characteristics according to the heartbeat signal by applying heart rate variability analysis; the physiological characteristics include the time-domain index IBI interval, the time-domain index RMSSD, and the time-domain index SDNN; A pressing frequency acquisition module, which is used to obtain the pressing frequency of the target driver pressing the vigilance button; A normalization module, which is used to normalize the time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency respectively; A calculation module, configured to apply a preset weight to calculate a comprehensive feature of the normalized time-domain index IBI interval, the time-domain index RMSSD, the time-domain index SDNN, and the pressing frequency; A prediction module, configured to apply a logistic regression method based on the comprehensive feature to obtain a prediction result of the fatigue state of the target driver; the prediction result is a fatigue state or a non-fatigue state.

7. An electronic device, characterized in that, It includes a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the non-contact in-transit fatigue detection method for drivers according to any one of claims 1 to 5.

8. An electronic device according to claim 7, wherein The memory is a readable storage medium.

Images