Driving monitoring system and driving monitoring method

Abstract

The invention provides a driving monitoring system and a driving monitoring method. The driving monitoring method comprises the following steps: detecting a first heart beat signal of driving through a radar; extracting a heartbeat interval signal from the first heart beat signal; generating heart rate variability data according to the heartbeat interval signal, wherein the heart rate variability data comprises low-frequency power and high-frequency power; judging whether driving is fatigue or not according to the heart rate variation data; and outputting an alarm message in response to the judgment of the driving fatigue.

Description

Technical Field

[0001] This invention relates to a monitoring technology, and more particularly to a driving monitoring system and a driving monitoring method. Background Technology

[0002] Traditional driver monitoring systems (DMS) use cameras to monitor the interior of the vehicle and determine the driver's physiological state or fatigue level based on the images captured by the cameras. However, image recognition-based DMS systems are prone to misjudging the driver's state. For example, a DMS might mistakenly determine that the driver is asleep because their eyes appear too small. Furthermore, using images to monitor driving may infringe on the privacy of the driver or passengers. Therefore, providing a driver monitoring system that overcomes these shortcomings is one of the important issues in this field. Summary of the Invention

[0003] This invention provides a driving monitoring system and a driving monitoring method, which can monitor the driver's physiological state to determine whether the driver is fatigued.

[0004] The present invention discloses a driving monitoring system comprising a radar and a processor. The processor is coupled to the radar and is configured to perform: detecting a first heartbeat signal of the driver via the radar; extracting a heartbeat interval signal from the first heartbeat signal; generating heart rate variability data based on the heartbeat interval signal, wherein the heart rate variability data includes low-frequency power and high-frequency power; determining whether the driver is fatigued based on the heart rate variability data; and outputting an alarm message in response to the determination of driver fatigue.

[0005] In one embodiment of the present invention, the processor described above is configured to further perform: obtaining a historical data set, wherein the historical data set includes historical heart rate variability data, wherein the historical heart rate variability data includes historical low-frequency power and historical high-frequency power; calculating the P-value of the heart rate variability data based on the historical data set and the heart rate variability data; and determining whether the driver is fatigued based on the P-value.

[0006] In one embodiment of the present invention, the processor described above is configured to further perform: in response to a P value being less than a threshold, determine driver fatigue.

[0007] In one embodiment of the present invention, the processor calculates the P-value based on one of the following: variance analysis, T test, or F test.

[0008] In one embodiment of the invention, the processor described above is configured to further perform: adding heart rate variability data to a historical data set to update the historical data set.

[0009] In one embodiment of the present invention, the processor is configured to further perform: detecting a first heartbeat signal and a second heartbeat signal of the driver based on a preset period, wherein the second heartbeat signal is later than the first heartbeat signal; and determining whether the driver is fatigued based on an updated set of historical data and the second heartbeat signal.

[0010] In one embodiment of the present invention, the aforementioned heart rate variability data further includes the standard deviation of the heartbeat interval signal and the ratio of low-frequency power to high-frequency power.

[0011] In one embodiment of the present invention, the aforementioned heartbeat interval signal includes a first heartbeat interval, wherein the processor is configured to further perform: detecting a first peak and a second peak of the first heartbeat signal, wherein the first peak and the second peak are adjacent peaks; and calculating the time interval between the first peak and the second peak to obtain the first heartbeat interval.

[0012] In one embodiment of the invention, the processor described above is configured to further perform: performing a discrete Fourier transform on the heartbeat interval signal to obtain a frequency response; and obtaining low-frequency power and high-frequency power from the frequency response.

[0013] In one embodiment of the invention, the processor described above is configured to further perform: detecting the field in the vehicle by radar to determine whether driving is present; and in response to determining that driving is present, detecting a first heartbeat signal of the driver by radar.

[0014] The present invention provides a driving monitoring method, comprising: detecting a driver's first heartbeat signal via radar; extracting a heartbeat interval signal from the first heartbeat signal; generating heart rate variability data based on the heartbeat interval signal, wherein the heart rate variability data includes low-frequency power and high-frequency power; determining whether the driver is fatigued based on the heart rate variability data; and outputting an alarm message in response to the determination of driver fatigue.

[0015] In one embodiment of the present invention, determining whether a driver is fatigued based on heart rate variability data includes: obtaining a historical data set, wherein the historical data set includes historical heart rate variability data, wherein the historical heart rate variability data includes historical low-frequency power and historical high-frequency power; calculating the P-value of the heart rate variability data based on the historical data set and the heart rate variability data; and determining whether a driver is fatigued based on the P-value.

[0016] In one embodiment of the present invention, determining whether a driver is fatigued based on a P value includes: determining driver fatigue in response to a P value being less than a threshold.

[0017] In one embodiment of the present invention, the above-described driving monitoring method further includes calculating a P-value according to one of the following: variance analysis, T-test, or F-test.

[0018] In one embodiment of the present invention, the above-described driving monitoring method further includes: adding heart rate variability data to a historical data set to update the historical data set.

[0019] In one embodiment of the present invention, the above-described driving monitoring method further includes: detecting a first heartbeat signal and a second heartbeat signal of the driver based on a preset period, wherein the second heartbeat signal is later than the first heartbeat signal; and determining whether the driver is fatigued based on an updated historical data set and the second heartbeat signal.

[0020] In one embodiment of the present invention, the aforementioned heart rate variability data further includes the standard deviation of the heartbeat interval signal and the ratio of low-frequency power to high-frequency power.

[0021] In one embodiment of the present invention, the heartbeat interval signal includes a first heartbeat interval, wherein extracting the heartbeat interval signal from the first heartbeat signal includes: detecting a first peak and a second peak of the first heartbeat signal, wherein the first peak and the second peak are adjacent peaks; and calculating the time interval between the first peak and the second peak to obtain the first heartbeat interval.

[0022] In one embodiment of the present invention, generating heart rate variability data based on a heartbeat interval signal includes: performing a discrete Fourier transform on the heartbeat interval signal to obtain a frequency response; and obtaining low-frequency power and high-frequency power from the frequency response.

[0023] In one embodiment of the present invention, the above-described driving monitoring method further includes: detecting the field in the vehicle by radar to determine whether driving exists; and in response to determining that driving exists, detecting a first heartbeat signal of the driver by radar.

[0024] Based on the above, the driving monitoring system of the present invention can detect the driver's heartbeat signal through radar, and determine the driver's sympathetic or parasympathetic nerve activity based on the heartbeat signal, thereby determining whether the driver is fatigued. Attached Figure Description

[0025] Figure 1 A schematic diagram of a driving monitoring system is shown according to an embodiment of the present invention;

[0026] Figure 2 A flowchart of a driving monitoring method is shown according to an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of a heartbeat signal is shown according to an embodiment of the present invention;

[0028] Figure 4 A schematic diagram of the frequency response is shown according to an embodiment of the present invention;

[0029] Figure 5A flowchart of a driving monitoring method is shown according to an embodiment of the present invention. Detailed Implementation

[0030] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same element references are used in the drawings and description to denote the same or similar parts.

[0031] Figure 1 A schematic diagram of a driving monitoring system 100 is shown according to an embodiment of the present invention. The driving monitoring system 100 may include a processor 110, a storage medium 120, a transceiver 130, and a radar 140.

[0032] Processor 110 may be, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field-programmable gate array (FPGA), or other similar elements or combinations thereof. Processor 110 may be coupled to storage medium 120, transceiver 130, and radar 140, and access and execute multiple modules and various applications stored in storage medium 120.

[0033] Storage medium 120 may be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar components or combinations thereof, for storing multiple modules or various applications that can be executed by processor 110.

[0034] Transceiver 130 transmits or receives signals wirelessly or via a wired connection. Transceiver 130 may also perform operations such as low-noise amplification, impedance matching, mixing, up or down frequency conversion, filtering, amplification, and similar functions.

[0035] Radar 140 can transmit radio frequency signals and receive reflected signals of radio frequency signals. Processor 110 can use radar 140 to detect objects. Radar 140 may include, but is not limited to, millimeter-wave (mmWave) radar or frequency-modulated continuous wave (FMCW) radar.

[0036] Figure 2 A flowchart of a driving monitoring method is shown according to an embodiment of the present invention, wherein the driving monitoring method may be performed by, for example... Figure 1 The driving monitoring system 100 shown is implemented. In step S201, the processor 110 can detect the presence of a driver via radar 140. For example, the processor 110 can transmit a radio frequency signal to a field in the vehicle (e.g., the driver's seat) via radar 140 and receive the reflected signal of the radio frequency signal. The processor 110 can analyze the reflected signal to determine whether a driver is present in the field. If the processor 110 detects the presence of a driver, it proceeds to step S202. If the processor 110 does not detect the presence of a driver, it can re-execute step S201 after waiting for a period of time. In one embodiment, the processor 110 can re-execute step S201 based on a preset period to detect the presence of a driver.

[0037] In step S202, the processor 110 can detect the driver's heartbeat signal via the radar 140. For example, the processor 110 can detect the driver's chest displacement via the radar 140 to obtain the driver's heartbeat signal.

[0038] In step S203, the processor 110 may extract a beat-to-beat interval signal from the heartbeat signal and generate heart rate variability (HRV) data based on the beat interval. In one embodiment, the beat interval signal is, for example, an RR interval signal.

[0039] The heartbeat interval signal may contain one or more heartbeat intervals. Specifically, processor 110 can detect two adjacent peaks in the heartbeat signal. Then, processor 110 can calculate the time interval between the two adjacent peaks to obtain the heartbeat interval. Figure 3 For example, Figure 3 A schematic diagram of a heartbeat signal 30 is shown according to an embodiment of the present invention. The processor 110 can detect peaks 31 and 32 of the heartbeat signal 30, wherein peaks 31 and 32 are adjacent peaks. The processor 110 then calculates the time interval 300 between peaks 31 and 32 to obtain a heartbeat interval. The processor 110 can obtain one or more heartbeat intervals from the heartbeat signal 30, thereby generating a heartbeat interval signal.

[0040] In one embodiment, the processor 110 may perform time-domain analysis on the heartbeat interval signal to obtain one or more time-domain HRV data. The time-domain HRV data may include data related to the standard deviation of the heartbeat interval signal, such as the standard deviation of NN intervals (SDNN) or the standard deviation of average NN intervals (SDANN).

[0041] In one embodiment, the processor 110 may perform a discrete Fourier transform (DFT) on the heartbeat interval signal to obtain a frequency response, and obtain one or more frequency domain HRV data based on the frequency response. The frequency domain HRV data may include total power (TP), very low frequency power (VLFP), low frequency power (LFP) representing sympathetic and parasympathetic activity, high frequency power (HFP) representing parasympathetic activity, normalized low frequency power (nLFP) representing a quantitative indicator of sympathetic activity, normalized high frequency power (nHFP) representing a quantitative indicator of parasympathetic activity, or the LF / HF ratio representing autonomic balance. LFP may include power in the frequency band from 0.04 Hz to 0.15 Hz. HFP can contain power in the frequency band from 0.15Hz to 0.4Hz.

[0042] by Figure 4 For example, Figure 4 A schematic diagram of a frequency response 40 is shown according to an embodiment of the present invention. Processor 110 may perform a DFT on a heartbeat interval signal to obtain the frequency response 40, wherein the frequency response 40 may include a low-frequency power 41 and a high-frequency power 42. Processor 110 may divide the low-frequency power 41 by the high-frequency power 42 to obtain an LF / HF ratio.

[0043] In one embodiment, the processor 110 may perform interpolation or resampling on the heartbeat interval signal before performing a DFT on the heartbeat interval signal.

[0044] Back Figure 2 In step S204, processor 110 may calculate the p-value of the HRV data. In one embodiment, processor 110 may calculate the p-value of the HRV data based on analysis of variance (ANOVA), T-test, or F-test.

[0045] Specifically, before acquiring HRV data and calculating its P-value, processor 110 can acquire historical data sets by receiving them via transceiver 130 or by detecting driving via radar 140. These historical data sets may contain multiple historical HRV data points, and each historical HRV data point may contain data such as historical LFP or historical HFP. Processor 110 can determine if the number of historical HRV data points in the historical data set is sufficient. If the number of historical HRV data points is less than or equal to a threshold, processor 110 can continuously acquire new historical HRV data points and add them to the historical data set. If the number of historical HRV data points exceeds the threshold, processor 110 can calculate the mean and standard deviation of the historical data set to obtain its distribution.

[0046] After obtaining the distribution of the historical dataset, processor 110 can calculate the T-value or F-value of the HRV data based on the mean and standard deviation of the historical dataset and the HRV data. Next, processor 110 can calculate the cumulative distribution function (CDF) value based on the distribution of the historical dataset and the values ​​of the HRV data (i.e., the T-value or F-value) to obtain the P-value of the HRV data.

[0047] In one embodiment, after acquiring HRV data, the processor 110 can add the HRV data to a historical data set to update the historical data set. The updated historical data set can be used to calculate the P-value of new HRV data acquired by the processor 110 in the future.

[0048] In step S205, processor 110 determines whether the P value of the HRV data is less than a first threshold, where the first threshold is, for example, equal to 0.01. If the P value is less than the first threshold, the process proceeds to step S206. In step S206, processor 110 determines that the driver is in a state of fatigue, and that the degree of driver fatigue is very severe. Processor 110 can output an alarm message to indicate that the driver's fatigue level is very severe. For example, processor 110 can communicate with an output device such as a display device or a speaker via transceiver 130, and output the alarm message through the output device. On the other hand, if the P value is greater than or equal to the first threshold, the process proceeds to step S207.

[0049] In step S207, processor 110 determines whether the P value of the HRV data is less than a second threshold, where the second threshold is greater than a first threshold. The second threshold is, for example, equal to 0.05. If the P value is less than the second threshold, the process proceeds to step S208. In step S208, processor 110 determines that the driver is fatigued. Processor 110 can output an alarm message to alert the driver that they are fatigued. For example, processor 110 can communicate with an output device such as a display device or speaker via transceiver 130 and output the alarm message through the output device.

[0050] On the other hand, if the P value is greater than or equal to the second threshold, then proceed to step S209. In step S209, the processor 110 may wait for a period of time based on a preset cycle. Afterward, the processor 110 may re-execute step S201 or step S202.

[0051] Figure 5 A flowchart of a driving monitoring method is shown according to an embodiment of the present invention, wherein the driving monitoring method may be composed of, for example, Figure 1 The driving monitoring system 100 shown is implemented. In step S501, the driver's first heartbeat signal is detected by radar. In step S502, the heartbeat interval signal is extracted from the first heartbeat signal. In step S503, heart rate variability data is generated based on the heartbeat interval signal, wherein the heart rate variability data includes low-frequency power and high-frequency power. In step S504, the driver's fatigue is determined based on the heart rate variability data. In step S505, in response to the determination of driver fatigue, an alarm message is output.

[0052] In summary, the driving monitoring system of this invention can detect the driver's heartbeat signal using radar, and determine the driver's sympathetic or parasympathetic nerve activity based on the heartbeat signal, thereby determining whether the driver is fatigued. Compared with traditional driving monitoring systems that use image recognition technology, the driving monitoring system of this invention has higher accuracy and can protect the privacy of the driver and passengers.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A driving monitoring system characterized by comprising: include: radar; as well as A processor, coupled to the radar, wherein the processor is configured to perform: The radar detects the driver's first heartbeat signal; Extract the heartbeat interval signal from the first heartbeat signal; Heart rate variability data is generated based on the heartbeat interval signal, wherein the heart rate variability data includes low-frequency power and high-frequency power; Determine whether the driver is fatigued based on the heart rate variability data; as well as In response to the assessment of driver fatigue, an alarm message is output.

2. The driving monitoring system of claim 1, wherein the processor is configured to further perform: Obtain a historical data set, wherein the historical data set includes historical heart rate variability data, wherein the historical heart rate variability data includes historical low-frequency power and historical high-frequency power; Calculate the P-value of the heart rate variability data based on the historical data set and the heart rate variability data; and The driver's fatigue level is determined based on the P-value.

3. The driving monitoring system according to claim 2, wherein the processor is configured to further perform: The driver is determined to be fatigued if the P value is less than the threshold.

4. The driving monitoring system according to claim 2, wherein the processor calculates the P value according to one of the following: variance analysis, T test, or F test.

5. The driving monitoring system of claim 2, wherein the processor is configured to further perform: The heart rate variability data is added to the historical data set to update the historical data set.

6. The driving monitoring system of claim 5, wherein the processor is configured to further perform: The driver's first and second heartbeat signals are detected based on a preset period, wherein the second heartbeat signal is later than the first heartbeat signal; and The driver is determined to be fatigued based on the updated historical data set and the second heartbeat signal.

7. The driving monitoring system according to claim 1, wherein the heart rate variability data further includes the standard deviation of the heartbeat interval signal and the ratio of the low-frequency power to the high-frequency power.

8. The driving monitoring system of claim 1, wherein the heartbeat interval signal includes a first heartbeat interval, and wherein the processor is configured to further perform: Detecting a first peak and a second peak of the first heartbeat signal, wherein the first peak and the second peak are adjacent peaks; and The time interval between the first peak and the second peak is calculated to obtain the first heartbeat interval.

9. The driving monitoring system of claim 1, wherein the processor is configured to further perform: Perform a Discrete Fourier Transform on the heartbeat interval signal to obtain the frequency response; and The low-frequency power and the high-frequency power are obtained from the frequency response.

10. The driving monitoring system of claim 1, wherein the processor is configured to further perform: The radar detects the area within the vehicle to determine if the driver is present; and In response to the determination of the presence of the driver, the first heartbeat signal of the driver is detected by the radar.

11. A driving monitoring method characterized by, include: Detect the driver's first heartbeat signal using radar; Extract the heartbeat interval signal from the first heartbeat signal; Heart rate variability data is generated based on the heartbeat interval signal, wherein the heart rate variability data includes low-frequency power and high-frequency power; Determine whether the driver is fatigued based on the heart rate variability data; as well as In response to the assessment of driver fatigue, an alarm message is output.

12. The driving monitoring method according to claim 11, wherein determining whether the driver is fatigued based on the heart rate variability data includes: Obtain a historical data set, wherein the historical data set includes historical heart rate variability data, wherein the historical heart rate variability data includes historical low-frequency power and historical high-frequency power; Calculate the P-value of the heart rate variability data based on the historical data set and the heart rate variability data; as well as The driver's fatigue level is determined based on the P-value.

13. The driving monitoring method according to claim 12, wherein determining whether the driver is fatigued based on the P-value includes: The driver is determined to be fatigued if the P value is less than the threshold.

14. The driving monitoring method according to claim 12 further includes calculating the P value according to one of the following: variance analysis, T test, or F test.

15. The driving monitoring method according to claim 12, further comprising: The heart rate variability data is added to the historical data set to update the historical data set.

16. The driving monitoring method according to claim 15, further comprising: The driver's first heartbeat signal and second heartbeat signal are detected based on a preset period, wherein the second heartbeat signal is later than the first heartbeat signal; as well as The driver is determined to be fatigued based on the updated historical data set and the second heartbeat signal.

17. The driving monitoring method according to claim 11, wherein the heart rate variability data further includes the standard deviation of the heartbeat interval signal and the ratio of the low-frequency power to the high-frequency power.

18. The driving monitoring method according to claim 11, wherein the heartbeat interval signal includes a first heartbeat interval, wherein extracting the heartbeat interval signal from the first heartbeat signal includes: The first peak and the second peak of the first heartbeat signal are detected, wherein the first peak and the second peak are adjacent peaks; as well as The time interval between the first peak and the second peak is calculated to obtain the first heartbeat interval.

19. The driving monitoring method according to claim 11, wherein generating the heart rate variability data based on the heartbeat interval signal comprises: Perform a discrete Fourier transform on the heartbeat interval signal to obtain the frequency response; as well as The low-frequency power and the high-frequency power are obtained from the frequency response.

20. The driving monitoring method according to claim 11, further comprising: The radar detects the area within the vehicle to determine whether the driver is present. as well as In response to the determination of the presence of the driver, the first heartbeat signal of the driver is detected by the radar.

Images