Driver respiration monitoring method and system based on directional loudspeaker

Abstract

The invention relates to a driver respiration monitoring method and system based on a directional loudspeaker. The method comprises the following steps that a transmitting signal sent by the directional loudspeaker and a reflected signal received by a microphone after the transmitting signal is reflected by the thorax of a driver are obtained; performing signal frequency mixing on the transmitted signal and the reflected signal, and performing signal processing on the mixed intermediate frequency signal to obtain a phase-time diagram of the intermediate frequency signal in a fixed interval as a coarse-grained breathing waveform; and inputting the coarse-grained respiration waveform into a Bi-LSTM respiration monitoring network, outputting a fine-grained respiration waveform, and carrying out respiration monitoring on the driver. Compared with the prior art, the method has the advantages that accurate breathing monitoring of the driver can be achieved in a complex scene, and the like.

Description

Technical Field

[0001] The present invention relates to the field of automobile control technology, and in particular to a driver breathing monitoring method and system based on a directional loudspeaker. Background Art

[0002] Driving a car is the most common way of traveling. With the vigorous development of highways, the driving distance and time are gradually increasing. The physical and mental health and driving experience of drivers need to be paid more attention to, which shows that a safe and intelligent driving environment is becoming more and more important. Similarly, the application of acoustic sensors in driving environments is becoming more and more common, and directional sound fields (such as prototypes such as directional speakers) have been widely studied in different scenarios. Consider a scenario where drivers can monitor their breathing while listening to music and navigating, thereby ensuring their driving experience and health during driving.

[0003] Unlike directional speakers, conventional acoustic sensing uses ordinary speakers, which makes the current acoustic breathing monitoring system based on conventional speakers face several common challenges related to collecting breathing signals, such as requiring a quiet environment and the driver to remain still. In the cockpit environment, it is necessary to dynamically measure the driver's fine-grained breathing waveform in a noisy and vibrating vehicle environment with passengers to accurately monitor the driver's physical condition, which makes the conventional method have great limitations.

[0004] Although current directional speakers can provide directional sound fields, which is useful in driving environments where different people have different listening needs, such as drivers listening to navigation and passengers listening to music, and directional sound waves can be customized according to the needs of users, these studies did not, however, utilize directional speakers to explore their potential capabilities for respiratory monitoring.

[0005] Utilizing the high directivity of directional speakers, high-precision directional user breathing detection can be achieved in a complex cockpit environment, but there are some challenges in this process: 1) There are multipath reflections from various objects and passenger interference in the driving environment, making it difficult to accurately extract which object or person the signal represented by the breathing comes from. 2) Synchronization of breathing monitoring and communication: The frequency band used for breathing monitoring must be inaudible to the user and will not interfere with the original directional sound playback function of the directional speaker. There is currently no relevant research on how to design the breathing monitoring signal to achieve this synchronous operation. 3) The vibration of a moving vehicle will significantly affect the accuracy of breathing monitoring. Summary of the invention

[0006] The purpose of the present invention is to provide a driver breathing monitoring method and system based on a directional speaker, which can transmit a high-frequency FMCW (Frequency-Modulated Continuous Wave) signal, and represent the displacement change of the driver's chest through the phase change of the intermediate frequency signal generated by the FMCW transmission signal and the reflection signal, and realize the fitting of the breathing waveform through a bidirectional Long Short-Term Memory (Bi-LSTM) network, generate a fine-grained breathing waveform, and realize high-precision breathing monitoring.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A driver breathing monitoring method based on a directional speaker comprises the following steps:

[0009] Acquire a transmission signal sent by a directional loudspeaker and a reflection signal received by a microphone after the transmission signal is reflected by the driver's chest, wherein the transmission signal is a frequency modulated continuous wave signal;

[0010] Perform signal mixing on the transmitted signal and the reflected signal, perform signal processing on the mixed intermediate frequency signal, and obtain a phase-time diagram of the intermediate frequency signal in a fixed interval as a coarse-grained breathing waveform;

[0011] The coarse-grained breathing waveform is input into the Bi-LSTM breathing monitoring network, and a fine-grained breathing waveform is output to monitor the driver's breathing.

[0012] The signal processing includes low-pass filtering, fast Fourier transform, peak selection, band-pass filtering and Gaussian filtering. After the intermediate frequency signal after mixing is subjected to low-pass filtering to remove the noise of the intermediate frequency signal, the signal is converted into the frequency domain using the fast Fourier transform. The peak value caused by breathing is found from the frequency domain signal through the peak selection algorithm. The frequency interval where the peak value caused by breathing is located is filtered out through band-pass filtering, and the frequency offset of the filtered frequency interval is removed by Gaussian filtering to obtain the phase-time diagram of the intermediate frequency signal in a fixed interval.

[0013] The frequency domain signal of the intermediate frequency signal after fast Fourier transformation includes only two peaks, one is the peak generated by the driver's breathing, and the other is the peak generated by the direct path of the speaker and the microphone. The peak selection algorithm searches for the signal peak generated by the driver's breathing by setting a threshold.

[0014] The Bi-LSTM respiratory monitoring network converts the input pulse sequence of length Lin into a corresponding standard respiratory sequence of length Lout using a standard encoder-decoder structure. For the input sequence, the Bi-LSTM respiratory monitoring network uses a bidirectional LSTM as an encoder to discover and utilize long-range dependencies in the data and extract features; then, after the Batch ReLU layer, another bidirectional LSTM network with the same structure is used as a decoder to decode the features, and CNN is used to transform its length to the target output length Lout.

[0015] The training process of the Bi-LSTM breathing monitoring network is as follows: during the data collection process, the driver's reference breathing waveform measured under the standard breathing belt and the coarse-grained breathing waveform after signal processing are simultaneously obtained, the two waveforms are input into the Bi-LSTM breathing monitoring network in a one-to-one correspondence, the coarse-grained breathing waveform is used as the network input, and the reference breathing waveform is used as the network output, and the network is trained to realize the fitting process from the coarse-grained waveform to the fine-grained waveform.

[0016] A driver breathing monitoring system based on a directional speaker, comprising:

[0017] A directional loudspeaker, used to send a transmission signal to the chest of a human body, wherein the transmission signal is a frequency modulated continuous wave signal with a frequency range of 70KHz-80KHz and a chirp of 0.1s;

[0018] A microphone, used for receiving a reflected signal of the transmitted signal after being reflected by the driver's chest;

[0019] A signal processing module is used to perform signal mixing on the transmission signal and the reflection signal, and perform signal processing on the mixed intermediate frequency signal to obtain a phase-time diagram of the intermediate frequency signal in a fixed interval as a coarse-grained breathing waveform;

[0020] The breathing monitoring module is used to input the coarse-grained breathing waveform into the Bi-LSTM breathing monitoring network, output the fine-grained breathing waveform, and monitor the driver's breathing.

[0021] The signal processing module performs low-pass filtering, fast Fourier transform, peak selection, band-pass filtering and Gaussian filtering operations. After the mixed intermediate frequency signal is subjected to low-pass filtering to remove the noise of the intermediate frequency signal, the signal is converted into the frequency domain using the fast Fourier transform. The peak value caused by breathing is found from the frequency domain signal through the peak selection algorithm. The frequency interval where the peak value caused by breathing is located is filtered out through band-pass filtering, and the frequency offset of the filtered frequency interval is removed by Gaussian filtering to obtain a phase-time diagram of the intermediate frequency signal in a fixed interval.

[0022] The frequency domain signal of the intermediate frequency signal after fast Fourier transformation includes only two peaks, one is the peak generated by the driver's breathing, and the other is the peak generated by the direct path of the speaker and the microphone. The peak selection algorithm searches for the signal peak generated by the driver's breathing by setting a threshold.

[0023] The Bi-LSTM respiratory monitoring network converts the input pulse sequence of length Lin into a corresponding standard respiratory sequence of length Lout using a standard encoder-decoder structure. For the input sequence, the Bi-LSTM respiratory monitoring network uses a bidirectional LSTM as an encoder to discover and utilize long-range dependencies in the data and extract features; then, after the Batch ReLU layer, another bidirectional LSTM network with the same structure is used as a decoder to decode the features, and CNN is used to transform its length to the target output length Lout.

[0024] The training process of the Bi-LSTM breathing monitoring network is as follows: during the data collection process, the driver's reference breathing waveform measured under the standard breathing belt and the coarse-grained breathing waveform after signal processing are simultaneously obtained, the two waveforms are input into the Bi-LSTM breathing monitoring network in a one-to-one correspondence, the coarse-grained breathing waveform is used as the network input, and the reference breathing waveform is used as the network output, and the network is trained to realize the fitting process from the coarse-grained waveform to the fine-grained waveform.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] (1) The present invention realizes directional breathing monitoring and communication of the driver by using a directional speaker, which can effectively improve the driver's driving experience and driving safety.

[0027] (2) The present invention builds a bidirectional Long Short-Term Memory (Bi-LSTM) respiratory monitoring network to fit the measured coarse-grained phase waveform image end-to-end into a standard, fine-grained respiratory waveform curve, so that the obtained waveform curve has higher accuracy.

[0028] (3) The present invention verifies that the task accuracy can still be maintained in the case of down jackets and sweaters under different clothing conditions. It also provides performance verification in actual application scenarios, proving that the present invention can still maintain good task accuracy in complex traffic environments and rugged mountain roads, solving the impact of driver activities and vehicle shaking on respiratory monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of the method of the present invention;

[0030] Figure 2 Schematic diagram of the Bi-LSTM respiratory monitoring network structure in one embodiment;

[0031] Figure 3 is a schematic diagram of a respiratory waveform monitoring result in an embodiment;

[0032] Figure 4 A schematic diagram of PCC results of respiratory monitoring using the method of the present invention in an embodiment;

[0033] Figure 5 A schematic diagram of REE results of respiratory monitoring using the method of the present invention in an embodiment;

[0034] Figure 6 It is a schematic diagram of WEE results of respiratory monitoring using the method of the present invention in an embodiment. DETAILED DESCRIPTION

[0035] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0036] This embodiment provides a driver breathing monitoring method based on a directional speaker. First, a FMCW signal is emitted by a directional speaker. After being reflected by the human chest, a microphone is used to collect breathing data and extract breathing waveforms. The time difference between the transmission signal and the reflected signal of the FMCW signal can detect the distance. The intermediate frequency signal obtained after mixing the two signals x is passed through fast Fourier transform, bandpass filtering and Gaussian filtering to obtain a phase-time diagram of the intermediate frequency signal, that is, a coarse-grained breathing waveform. Secondly, the coarse-grained breathing waveform corresponding to each other when collecting data and the fine-grained breathing waveform measured with a standard breathing belt are input into the Bi-LSTM neural network to train the network parameters. When used, the fine-grained breathing waveform can be directly obtained through the coarse-grained breathing waveform.

[0037] like Figure 1 As shown, the method specifically comprises the following steps:

[0038] S1, obtaining a transmission signal sent by a directional loudspeaker and a reflection signal received by a microphone after the transmission signal is reflected by the driver's chest.

[0039] In this embodiment, a directional loudspeaker is used to transmit a 70KHz-80KHz FMCW signal with a chirp of 0.1s to the chest of the human body, and a microphone is used to receive the reflected signal.

[0040] S2, performing signal mixing on the transmission signal and the reflection signal, performing signal processing on the mixed intermediate frequency signal, and obtaining a phase-time diagram of the intermediate frequency signal in a fixed interval as a coarse-grained breathing waveform.

[0041] Signal processing includes low-pass filtering, fast Fourier transform, peak selection, band-pass filtering and Gaussian filtering. After the mixed intermediate frequency signal is low-pass filtered to remove the noise of the intermediate frequency signal, the signal is converted into the frequency domain using the fast Fourier transform (FFT), and the peak value caused by breathing is found from the frequency domain signal using the peak selection algorithm. The frequency interval where the peak value caused by breathing is filtered out by band-pass filtering, and the frequency offset (phase drift) of the filtered frequency interval is removed by Gaussian filtering, and the phase-time diagram of the intermediate frequency signal in a fixed interval is obtained as a coarse-grained breathing waveform.

[0042] Since the present invention adopts a directional loudspeaker, the frequency domain signal of the intermediate frequency signal after fast Fourier transform only includes two peaks, one is the peak generated by the driver's breathing, and the other is the peak generated by the direct path between the loudspeaker and the microphone. Therefore, the peak selection algorithm in this embodiment sets a threshold to find the signal peak generated by the driver's breathing.

[0043] S3, inputs the coarse-grained breathing waveform into the Bi-LSTM breathing monitoring network, outputs the fine-grained breathing waveform, and performs breathing monitoring on the driver.

[0044] like Figure 2 As shown in the figure, the Bi-LSTM respiratory monitoring network converts the input pulse sequence of length Lin into the corresponding standard respiratory sequence of length Lout with a standard encoder-decoder structure. For the input sequence, the Bi-LSTM respiratory monitoring network uses a bidirectional LSTM as an encoder to discover and utilize the long-range dependencies in the data and extract features; then, after the Batch ReLU layer, another bidirectional LSTM network with the same structure is used as a decoder to decode the features, and CNN is used to transform its length to the target output length Lout.

[0045] The training process of the Bi-LSTM respiratory monitoring network is as follows: During the data collection process, the driver's reference respiratory waveform measured under the standard respiratory belt and the coarse-grained respiratory waveform after signal processing are obtained at the same time, and the two waveforms are input into the Bi-LSTM respiratory monitoring network one by one. The coarse-grained respiratory waveform is used as the network input, and the reference respiratory waveform is used as the network output. The network is trained to achieve the fitting process from the coarse-grained waveform to the fine-grained waveform. This process can solve the impact of vehicle shaking on respiratory monitoring during driving. The specific example can be seen in Figure 3, where the blue line is the coarse-grained breathing waveform measured when the car is shaking, the green line is the breathing waveform measured under the standard breathing belt, and the orange part is the fine-grained breathing waveform generated by Bi-LSTM fitting. Through the training of network model parameters with a large amount of data in the early stage (compared with the standard breathing waveform, Pearson Correlation Coefficient (PCC) is greater than 0.85, Rate Estimation Error (REE) is less than 0.4, and Waveform Estimation Error (WEE) is less than 0.15), fine-grained breathing waveforms can be directly output. Figure 4 , 5 As shown in Figure 6, we show the PCC, REE and WEE of the 10 drivers tested in the dataset, where Figure 5 , 6 It is presented using a box plot.

[0046] Because the present invention measures the breathing waveform by the rise and fall of the chest, the thickness of the clothes will be an important influencing factor. This embodiment tests the different clothes of the driver (shirt, sweater, jacket, down jacket), and the present invention obtains good results, and the PCC is greater than 0.85.

[0047] Because human activities and car shaking can have a great impact on respiratory monitoring, this embodiment is also tested in different road environments and traffic environments. The present invention can achieve good results, and the PCC is greater than 0.82.

[0048] This embodiment also provides a driver breathing monitoring system based on a directional speaker, comprising:

[0049] A directional loudspeaker, used to send a transmission signal to the chest of a human body, wherein the transmission signal is a frequency modulated continuous wave signal with a frequency range of 70KHz-80KHz and a chirp of 0.1s;

[0050] A microphone, used for receiving a reflected signal of the transmitted signal after being reflected by the driver's chest;

[0051] A signal processing module is used to perform signal mixing on the transmission signal and the reflection signal, and perform signal processing on the mixed intermediate frequency signal to obtain a phase-time diagram of the intermediate frequency signal in a fixed interval as a coarse-grained breathing waveform;

[0052] The breathing monitoring module is used to input the coarse-grained breathing waveform into the Bi-LSTM breathing monitoring network, output the fine-grained breathing waveform, and monitor the driver's breathing.

[0053] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0054] The present invention proposes a driver breathing monitoring system using a directional speaker. Compared with a traditional speaker, the present invention can achieve directional communication and breathing monitoring. In addition, the present invention also proposes a set of data preprocessing methods, which can use FMCW signals, solve noise and slow phase drift through bandpass filtering and Gaussian filtering, and finally use the phase of the intermediate frequency signal to roughly represent the breathing waveform. At the same time, a LSTM-based neural network Bi-LSTM is constructed to achieve end-to-end breathing waveform fitting and high-precision breathing waveform extraction.

[0055] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A driver breathing monitoring method based on a directional speaker, characterized in that: The following steps are involved: Acquire a transmission signal sent by a directional loudspeaker and a reflection signal received by a microphone after the transmission signal is reflected by the driver's chest, wherein the transmission signal is a frequency modulated continuous wave signal; Perform signal mixing on the transmitted signal and the reflected signal, perform signal processing on the mixed intermediate frequency signal, and obtain a phase-time diagram of the intermediate frequency signal in a fixed interval as a coarse-grained breathing waveform; The coarse-grained breathing waveform is input into the Bi-LSTM breathing monitoring network, and a fine-grained breathing waveform is output to monitor the driver's breathing.

2. A driver breathing monitoring method based on a directional speaker according to claim 1, characterized in that: The signal processing includes low-pass filtering, fast Fourier transform, peak selection, band-pass filtering and Gaussian filtering. After the intermediate frequency signal after mixing is subjected to low-pass filtering to remove the noise of the intermediate frequency signal, the signal is converted into the frequency domain using the fast Fourier transform. The peak value caused by breathing is found from the frequency domain signal through the peak selection algorithm. The frequency interval where the peak value caused by breathing is located is filtered out through band-pass filtering, and the frequency offset of the filtered frequency interval is removed by Gaussian filtering to obtain the phase-time diagram of the intermediate frequency signal in a fixed interval.

3. A driver breathing monitoring method based on a directional speaker according to claim 2, characterized in that: The frequency domain signal of the intermediate frequency signal after fast Fourier transformation includes only two peaks, one is the peak generated by the driver's breathing, and the other is the peak generated by the direct path of the speaker and the microphone. The peak selection algorithm searches for the signal peak generated by the driver's breathing by setting a threshold.

4. The driver breathing monitoring method based on a directional speaker according to claim 1, characterized in that: The Bi-LSTM respiratory monitoring network converts the input pulse sequence of length Lin into the corresponding standard respiratory sequence of length Lout using a standard encoder-decoder structure. For the input sequence, the Bi-LSTM respiratory monitoring network uses a bidirectional LSTM as an encoder to discover and utilize long-range dependencies in the data and extract features. Then, after the Batch ReLU layer, another bidirectional LSTM network with the same structure is used as a decoder to decode the features, and CNN is used to transform its length to the target output length Lout.

5. The driver breathing monitoring method based on a directional speaker according to claim 1, characterized in that: The training process of the Bi-LSTM breathing monitoring network is as follows: during the data collection process, the driver's reference breathing waveform measured under the standard breathing belt and the coarse-grained breathing waveform after signal processing are simultaneously obtained, the two waveforms are input into the Bi-LSTM breathing monitoring network in a one-to-one correspondence, the coarse-grained breathing waveform is used as the network input, and the reference breathing waveform is used as the network output, and the network is trained to realize the fitting process from the coarse-grained waveform to the fine-grained waveform.

6. A driver breathing monitoring system based on a directional speaker, characterized in that: include: A directional loudspeaker, used to send a transmission signal to the chest of a human body, wherein the transmission signal is a frequency modulated continuous wave signal with a frequency range of 70KHz-80KHz and a chirp of 0.1s; A microphone, used for receiving a reflected signal of the transmitted signal after being reflected by the driver's chest; A signal processing module is used to perform signal mixing on the transmission signal and the reflection signal, and perform signal processing on the mixed intermediate frequency signal to obtain a phase-time diagram of the intermediate frequency signal in a fixed interval as a coarse-grained breathing waveform; The breathing monitoring module is used to input the coarse-grained breathing waveform into the Bi-LSTM breathing monitoring network, output the fine-grained breathing waveform, and monitor the driver's breathing.

7. A driver breathing monitoring system based on a directional speaker according to claim 6, characterized in that: The signal processing module performs low-pass filtering, fast Fourier transform, peak selection, band-pass filtering and Gaussian filtering operations. After the mixed intermediate frequency signal is subjected to low-pass filtering to remove the noise of the intermediate frequency signal, the signal is converted into the frequency domain using the fast Fourier transform. The peak value caused by breathing is found from the frequency domain signal through the peak selection algorithm. The frequency interval where the peak value caused by breathing is located is filtered out through band-pass filtering, and the frequency offset of the filtered frequency interval is removed by Gaussian filtering to obtain a phase-time diagram of the intermediate frequency signal in a fixed interval.

8. A driver breathing monitoring system based on a directional speaker according to claim 7, characterized in that: The frequency domain signal of the intermediate frequency signal after fast Fourier transformation includes only two peaks, one is the peak generated by the driver's breathing, and the other is the peak generated by the direct path of the speaker and the microphone. The peak selection algorithm searches for the signal peak generated by the driver's breathing by setting a threshold.

9. A driver breathing monitoring system based on a directional speaker according to claim 6, characterized in that: The Bi-LSTM respiratory monitoring network converts the input pulse sequence of length Lin into the corresponding standard respiratory sequence of length Lout using a standard encoder-decoder structure. For the input sequence, the Bi-LSTM respiratory monitoring network uses a bidirectional LSTM as an encoder to discover and utilize long-range dependencies in the data and extract features. Then, after the Batch ReLU layer, another bidirectional LSTM network with the same structure is used as a decoder to decode the features, and CNN is used to transform its length to the target output length Lout.

10. The driver breathing monitoring system based on a directional speaker according to claim 6, characterized in that: The training process of the Bi-LSTM breathing monitoring network is as follows: during the data collection process, the driver's reference breathing waveform measured under the standard breathing belt and the coarse-grained breathing waveform after signal processing are simultaneously obtained, the two waveforms are input into the Bi-LSTM breathing monitoring network in a one-to-one correspondence, the coarse-grained breathing waveform is used as the network input, and the reference breathing waveform is used as the network output, and the network is trained to realize the fitting process from the coarse-grained waveform to the fine-grained waveform.

Images