A millimeter wave radar-based driver fatigue monitoring method, system, and device

Abstract

The application discloses a kind of driver fatigue monitoring method, system and equipment based on millimeter wave radar, the method is according to the characteristics that breathing will be shallow when tired, the following steps are used: step 1, extract breathing signal from millimeter wave radar interference phase signal, set initial window size win, initial flag variable flag, window change step step and breathing amplitude threshold;Step 2, scan the window, find the window peak and carry out window size updating operation;Step 3, label operation is carried out to the scanned window, and the driver fatigue time period is judged out.The breathing signal collected by millimeter wave radar is used for monitoring in the application, direct contact with the human body is not needed, meanwhile, the millimeter wave radar is not affected by weather and sunshine, the comfort level is higher and the reliability is higher, the window size can be adjusted when scanning the window, and the detection accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of human anti-fatigue detection, and particularly relates to a driver fatigue monitoring method, system and device based on a millimeter wave radar. BACKGROUND

[0002] With the improvement of people's living standards, private cars have become the preferred mode of transportation for most people, and the incidence of traffic accidents is increasing. Fatigue driving is an important factor leading to traffic accidents. Fatigue phenomena during driving cannot be completely avoided, but using effective methods to detect abnormal physiological signals emitted by the driver's body in real time, making correct judgments and issuing fatigue warning commands to the driver when the driver is just tired, can greatly reduce the incidence of traffic accidents.

[0003] In recent years, research on fatigue driving monitoring methods has increased, including using electroencephalogram, electrocardiogram, electromyogram to monitor the physiological state of the driver, although the measurement accuracy is high, but it needs to be pasted on the human body electrode, which will bring discomfort to the driver, and it is not convenient to carry. Using a camera to capture the driver's eye movement, but this is easily affected by the surrounding environment, light and temperature, and cannot protect the driver's personal privacy. Using a pressure sensor to monitor the grip of the steering wheel, this contact-based measurement will make the driver very sensitive and will interfere with normal driving. SUMMARY

[0004] The present application aims to overcome the shortcomings of the prior art and proposes a driver fatigue monitoring method, system and device based on a millimeter wave radar, a fatigue detection method without contacting the human body, which can accurately monitor the fatigue of the driver according to the physiological state of the driver. The technical solution adopted by the present application is as follows:

[0005] A driver fatigue monitoring method based on a millimeter wave radar, comprising the following steps:

[0006] Step 1, extract the respiratory signal from the millimeter wave radar interference phase signal, set the initial window size win, the initial flag variable flag, the window change step step and the respiratory amplitude threshold, wherein the initial value of flag is 0;

[0007] Step 2, scan the window, find the window peak and perform window size update operation;

[0008] Step 3, label the scanned window, and determine the driver fatigue time period.

[0009] Further, the setting of the breath amplitude threshold in step 1 comprises the following: analyzing the amplitude range of the breath signal, determining a threshold coefficient, then multiplying the mean value representing the overall trend of the signal peak point by the threshold coefficient to obtain the threshold; applying the threshold to the entire signal, and the breath state of the breath signal greater than the threshold is normal breathing, and the breath state of the breath signal less than the threshold is shallow breathing.

[0010] Further, step 2 comprises the following steps:

[0011] Step 2-1, analyze the waveform characteristics of the breath signal, find the positions of the wave peaks and troughs, and obtain the peak value of the breath signal;

[0012] Step 2-2, find the peak value of the breath segment in the window;

[0013] If there is no peak value in the window, it is judged whether the flag is 0: if the flag is 0, the current window size is enlarged to win=win+step, and step 2-4 is jumped to; if the flag is not 0, the breath state of the current window is equal to the breath state of the last window, and the scanning is ended;

[0014] If there is a peak value in the window, step 2-3 is performed;

[0015] Step 2-3, if all the peak values in the window are greater than or less than the threshold, that is, all the breath segments in the window are normal or shallow, it is judged whether the flag is 0: if the flag is 0, frequency domain analysis is performed by using Fourier transform, the period cycle is calculated according to the frequency f, cycle=1 / f, the window size for the next scanning is set as win=cycle / (1 / Fs), where Fs is the sampling frequency of the signal, and step 2-4 is performed; if the flag is 1, the scanning is ended;

[0016] If the peak values in the window are not all greater than or less than the threshold, that is, there are normal breath segments and shallow breath segments in the window, the size of the current window is reduced to win=win-step, the flag is set to 0, and step 2-1 is jumped to;

[0017] Step 2-4, it is judged whether the size of the current scanning window exceeds the breath end time: if it exceeds the breath end time, the size of the window is adjusted, the window size is from the starting position of the window to the breath end time, the flag is set to 1, and step 2-1 is jumped to; if it does not exceed the breath end time, step 2-1 is directly jumped to.

[0018] Further, step 3 comprises:

[0019] The windows are labeled, the window with shallow breathing is marked as 1, and the window with normal breathing is marked as 0; according to the label from front to back, adjacent windows of the same type are merged to obtain the normal breathing section and the shallow breathing section of the whole breathing signal.

[0020] Further, the shallow breathing section is the time period when the driver's body appears fatigue.

[0021] Further, the present application also provides a driver fatigue monitoring system based on a millimeter wave radar, comprising:

[0022] The millimeter wave radar module collects driver physiological activity information through a millimeter wave radar to obtain a breathing signal.

[0023] The peak detection module analyzes the breathing signal obtained in the millimeter wave radar module to extract the peak value of the breathing signal in the window.

[0024] The window size updating module updates the size of the window according to the peak value information extracted in the peak detection module.

[0025] The state recognition module recognizes the state of the window to distinguish the driver fatigue time period.

[0026] The present application also provides an electronic device comprising a memory, a processor and program instructions stored in the memory and executable by the processor to implement the steps of the method of the present application.

[0027] Compared with the prior art, the present application has the following advantages:

[0028] 1. The present application monitors the fatigue state of the driver using the breathing signal collected by the millimeter wave radar, without direct contact with the human body, and the millimeter wave radar is not affected by weather and sunlight, so it is more comfortable and reliable.

[0029] 2. The present application can automatically adjust the size of the breathing signal window when identifying the state of the driver, better adapt to the changes of the breathing signal, and improve the accuracy of state recognition. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 It is a working flowchart of the driver fatigue monitoring method based on a millimeter wave radar in the embodiments of the present application.

[0031] Figure 2 It is a physiological signal detection principle diagram of the driver fatigue monitoring method based on a millimeter wave radar in the embodiments of the present application.

[0032] Figure 3 It is a shallow breathing recognition result image in the embodiments of the present application. DETAILED DESCRIPTION

[0033] The present application is described in further detail below with reference to the accompanying drawings, so that those skilled in the art can have a better understanding of the present application and can implement it, but the following examples are only used to explain the present application, not as a limitation of the present application.

[0034] The millimeter wave radar does not need to be in direct contact with the human body, is not easily affected by light and weather, can monitor the physiological state of the driver all day long, and can protect the personal privacy of the driver. As shown in Figure 2 The transmitting antenna of the millimeter wave radar emits electromagnetic waves, which are returned after encountering the human body, and the receiving antenna receives the signals reflected by the human body. The receiving signals contain the weak changes in the chest cavity caused by the physiological activities of the human body, and the physiological information of the human body is obtained through the weak changes.

[0035] As shown in Figure 1 A driver fatigue monitoring method based on a millimeter wave radar includes the following steps:

[0036] Step 1: Extract the respiratory signal from the millimeter wave radar interference phase signal, set the initial window size win, the initial flag variable flag, the window change step step, and the respiratory amplitude threshold, wherein the initial value of flag is 0;

[0037] Step 2: Scan the window, find the window peak value, and perform window size updating operation;

[0038] Step 3: Perform label operation on the scanned window, and determine the driver fatigue time period.

[0039] In step 1, setting the respiratory amplitude threshold includes the following contents: analyzing the amplitude range of the respiratory signal, determining the threshold coefficient, then multiplying the threshold coefficient by the mean value representing the overall trend of the signal peak point to obtain the threshold; apply the threshold to the entire signal, the respiratory state of the respiratory signal greater than the threshold is normal breathing, and the respiratory state of the respiratory signal less than the threshold is shallow breathing.

[0040] In step 2, the following steps are included:

[0041] Step 2-1: Analyze the waveform characteristics of the respiratory signal, find the positions of the wave peaks and troughs, and obtain the peak value of the respiratory signal;

[0042] Step 2-2: Find the peak value of the respiratory segment in the window;

[0043] If there is no peak in the window, it is determined whether the flag is 0: if the flag = 0, the current window size is enlarged to win = win + step, and step 2-4 is jumped to; if the flag is not 0, the current window breathing state is equal to the last window breathing state, and the scanning is ended;

[0044] If there is a peak in the window, step 2-3 is performed.

[0045] In step 2-3, if all the window peaks are greater than or less than the threshold, that is, all the window is normal breathing section or all is shallow breathing section, it is determined whether the flag is 0: if the flag = 0, frequency domain analysis is performed by using Fourier transform, the period cycle is calculated according to the frequency f, cycle = 1 / f, the window size of the next scanning is set to win = cycle / (1 / Fs), where Fs is the sampling frequency of the signal, and step 2-4 is performed; if the flag = 1, the scanning is ended.

[0046] If the window peak is not all greater than or less than the threshold, that is, there are normal breathing and shallow breathing sections in the window, the current window size is reduced to win = win-step, the flag is set to 0, and step 2-1 is jumped to.

[0047] In step 2-4, it is determined whether the size of the current scanning window exceeds the end time of breathing: if it exceeds the end time of breathing, the size of the window is adjusted, the window size is from the start position of the window to the end time of breathing, the flag is set to 1, and step 2-1 is jumped to; if it does not exceed the end time of breathing, step 2-1 is directly jumped to.

[0048] Step 3 includes:

[0049] The window is labeled, the shallow breathing window is marked as 1, the normal breathing window is marked as 0, the normal breathing section and the shallow breathing section of the whole breathing signal are obtained by merging the windows of the same type between the adjacent windows from front to back according to the labels.

[0050] Figure 3 It is a shallow breathing recognition result image in the embodiment of the application, in the fatigue simulation experiment, the millimeter wave radar directly faces the chest cavity part of the test personnel, the detected person stably sits on the chair, and the relative distance between the radar and the chair is about 0.5 m, and the test personnel are required to hold their breath for a period of time during data acquisition. It can be seen from the figure that at the 9th second, the breathing begins to be shallow until the 19th second, at the 28th second, the breathing begins to be shallow again, and the duration is about 5 seconds.

[0051] The application further provides a driver fatigue monitoring system based on a millimeter wave radar, which comprises:

[0052] The millimeter wave radar module collects driver physiological activity information by millimeter wave radar, so as to obtain a breathing signal;

[0053] The peak detection module analyzes the breathing signal obtained in the millimeter wave radar module, and extracts a peak value of the breathing signal in a window;

[0054] The window size updating module updates the size of the window according to the peak value information extracted in the peak detection module;

[0055] The state recognition module recognizes the state of the window, and distinguishes a driver fatigue time period.

[0056] Finally, the application provides an electronic device, which includes a memory, a processor and program instructions stored in the memory and executable by the processor, and the processor executes the program instructions to implement each step of the video anomaly detection method based on scene classification.

[0057] It should be noted that the description of the system and device of the embodiments of the application is similar to the description of the above method embodiments, and has similar beneficial effects to the method embodiments, and therefore will not be described again.

[0058] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program codes can be executed entirely on a machine, partially on a machine, partially on a machine as a separate software package, and partially on a remote machine or server.

[0059] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the application. It should be understood that the above description is only for specific embodiments of the application, and is not intended to limit the scope of the application. Any equivalent changes and modifications made by those skilled in the art without departing from the concept and principles of the application shall fall within the scope of the application.

Claims

1. A method for monitoring driver fatigue based on millimeter-wave radar, characterized in that, Includes the following steps: Step 1: Extract the breathing signal from the millimeter-wave radar interferometric phase signal, and set the initial window size win, initial flag variable flag, window change step size step, and breathing amplitude threshold, where the initial value of flag is 0; Step 1, setting the breathing amplitude threshold includes: analyzing the amplitude range of the breathing signal, determining the threshold coefficient, and then multiplying the threshold coefficient by the mean value representing the overall trend of the peak points of the breathing signal to obtain the breathing amplitude threshold; applying the breathing amplitude threshold to the breathing signal, the breathing state of the breathing signal greater than the breathing amplitude threshold is normal breathing, and the breathing state of the breathing signal less than the breathing amplitude threshold is shallow breathing; Step 2: Scan the window to find the peak value of the respiratory signal within the window and update the window size. Step 3: Label the scanned window to determine the time period during which the driver experienced fatigue; Step 2 includes the following steps: Step 2-1: Analyze the waveform characteristics of the respiratory signal, find the positions of the peaks and troughs, and obtain the peak value of the respiratory signal; Step 2-2: Locate the peak value of the respiratory signal within the window; If there is no peak value within the window, then check if the flag is 0: if Then expand the current window size to Proceed to steps 2-4; if flag is not 0, use the breathing state of the previous window as the breathing state of the current window and end the scan. If there is a peak value within the window, proceed to steps 2-3; Steps 2-3: If the peak values ​​of the respiratory signals within the window are all greater than the respiratory amplitude threshold, meaning the window represents the normal breathing segment; if the peak values ​​of the respiratory signals within the window are all less than the respiratory amplitude threshold, meaning the window represents the shallow breathing segment; then determine if flag is 0: If Frequency domain analysis is performed using Fourier transform, and the period *cycle* is calculated based on the frequency *f*. Set the window size for the next scan to [size missing]. Where Fs is the sampling frequency of the respiratory signal, proceed to steps 2-4; if End scan; If the peak values ​​of the respiratory signals within the window are not all greater than or all less than the respiratory amplitude threshold (i.e., the window contains both normal breathing segments and shallow breathing segments), reduce the size of the current window. ,make Proceed to step 2-1; Steps 2-4: Determine if the current window size exceeds the end of the breathing time: If it does, adjust the window size to match the time from the window's initial position to the end of the breathing time, and set... If the breathing has not ended, proceed directly to step 2-1; Step 3 includes: The windows are labeled, with shallow breathing marked as 1 and normal breathing marked as 0. Based on the labels, adjacent windows of the same type are merged from front to back to obtain the normal breathing segment and the shallow breathing segment of the breathing signal. The period when breathing becomes shallow is the time when the driver's body becomes fatigued.

2. A millimeter-wave radar-based driver fatigue monitoring system for implementing the driver fatigue monitoring method based on millimeter-wave radar according to claim 1, characterized in that, include: The millimeter-wave radar module collects information about the driver's physiological activities using millimeter-wave radar, thereby obtaining respiratory signals; The peak detection module analyzes the respiratory signal obtained by the millimeter-wave radar module and extracts the peak value of the respiratory signal within the window; The window size update module updates the window size based on the peak value extracted by the peak detection module. The status recognition module identifies the status of the window and determines the time period during which the driver is fatigued.

3. An electronic device, comprising a memory and a processor, wherein the memory stores program instructions for execution by the processor, characterized in that, The processor executes the program instructions to implement the steps of the driver fatigue monitoring method based on millimeter-wave radar according to claim 1.

Images