A radar-based in-vehicle occupant heart rate monitoring method and system

Abstract

The application discloses a radar-based in-vehicle personnel heart rate monitoring method and system, and belongs to the technical field of vital sign detection.The method comprises the following steps: acquiring radar echo signals for monitoring vital signs of in-vehicle personnel; determining phase signals of a distance gate where a human target is located and distance gates in the vicinity of the distance gate according to the acquired radar echo signals; extracting spatial features of the phase signals of the distance gate where the human target is located and the distance gates in the vicinity of the distance gate; extracting time sequence features from the extracted spatial features; obtaining a heartbeat signal of the in-vehicle personnel according to the time sequence features; and acquiring a heart rate of the in-vehicle personnel according to the heartbeat signal of the in-vehicle personnel. The accuracy of in-vehicle personnel heart rate monitoring is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vital sign detection, in particular to a radar-based in-vehicle personnel heart rate monitoring method and system. BACKGROUND

[0002] When the vital sign signal of the driver is abnormal, it will affect the safety of vehicle driving, and the heart rate as an important indicator of vital signs, accurate monitoring of the heart rate of the driver is an important measure to ensure safe driving of the vehicle.

[0003] In related technologies, there is a method of detecting the vital signs of the driver by using radar; the method obtains the radar echo signal, then extracts the heartbeat signal from the radar echo signal, and finally calculates the frequency of the breathing and heartbeat signals in the frequency domain by using discrete Fourier transform to realize the detection of the vital signs of the driver.

[0004] However, in the heart rate detection based on the frequency-modulated continuous wave radar, the vital sign-related heartbeat signal in the radar echo is very weak and is often submerged in strong clutter, and the harmonics of the breathing signal have great interference on the heartbeat signal, so the method of detecting the vital signs of the driver by using radar in related technologies cannot handle the interference of the breathing harmonics on the heartbeat signal, resulting in limited detection accuracy of the heart rate of the driver. SUMMARY

[0005] The present application provides a radar-based in-vehicle personnel heart rate monitoring method and system, which can solve the problem that the heartbeat signal detection is inaccurate due to the interference of the breathing signal on the heartbeat signal, and finally leads to inaccurate heart rate detection.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] On the one hand, a radar-based in-vehicle personnel heart rate monitoring method is provided, comprising:

[0008] obtaining a radar echo signal for monitoring the vital signs of the in-vehicle personnel;

[0009] determining the phase signal of the distance gate where the human target is located and the distance gate near the distance gate according to the obtained radar echo signal;

[0010] extracting the spatial features of the phase signal of the distance gate where the human target is located and the distance gate near the distance gate;

[0011] extracting the time sequence features from the extracted spatial features;

[0012] obtaining the heartbeat signal of the in-vehicle personnel according to the time sequence features;

[0013] obtaining the heart rate of the in-vehicle personnel according to the heartbeat signal of the in-vehicle personnel.

[0014] Optionally, a distance dimension fast Fourier transform is performed on the radar echo signal to obtain a frequency domain signal;

[0015] The frequency domain signal is stored in a two-dimensional matrix R, which includes two dimensions of a slow time dimension and a fast time dimension;

[0016] A variance is calculated for the matrix R along the slow time dimension;

[0017] A maximum value of the variance is determined along the fast time dimension, and the maximum value is the distance gate where the person is located, and the distance gates on the left and right sides of the distance gate where the person is located are distance gates near the distance gate where the person is located;

[0018] Phase extraction and unwrapping are performed on the frequency domain signal at the distance gate where the human target is located and the distance gates near the distance gate to obtain a phase signal of the distance gate where the human target is located and the distance gates near the distance gate.

[0019] Optionally, the I / Q signal of the radar echo signal is compensated and filtered to obtain a compensated and filtered signal, and a distance dimension fast Fourier transform is performed on the compensated and filtered signal to obtain a frequency domain signal.

[0020] Optionally, a convolutional neural network is used to extract spatial features of the phase signal of the distance gate where the human target is located and the distance gates near the distance gate, and a bidirectional long short-term memory neural network is used to extract time sequence features from the extracted spatial features.

[0021] Optionally, peaks with amplitudes greater than a set heartbeat signal threshold are searched for in the time domain of the heartbeat signal, and an average time interval of the peaks is calculated to obtain a heartbeat period and a heart rate.

[0022] Optionally, it is determined whether the heart rate of the person in the vehicle is abnormal, and when the heart rate of the person in the vehicle is abnormal, an alarm information is sent.

[0023] On the other hand, a radar-based heart rate monitoring system for a person in a vehicle is provided, comprising:

[0024] A signal acquisition module for acquiring a radar echo signal for monitoring vital signs of a person in a vehicle;

[0025] A phase signal extraction module for determining a phase signal of a distance gate where a human target is located and distance gates near the distance gate according to the acquired radar echo signal;

[0026] A heartbeat signal extraction module for extracting spatial features of the phase signal of the distance gate where the human target is located and the distance gates near the distance gate, extracting time sequence features from the extracted spatial features, and obtaining a heartbeat signal of the person in the vehicle according to the time sequence features;

[0027] A heart rate determination module for obtaining a heart rate of the person in the vehicle according to the heartbeat signal of the person in the vehicle.

[0028] In another aspect, a computer device is provided, the device comprising:

[0029] a processor adapted to execute a computer program;

[0030] a computer readable storage medium having stored therein a computer program which, when executed by the processor, implements the radar-based in-vehicle personnel heart rate monitoring method of the first aspect.

[0031] In another aspect, a computer readable storage medium is provided, the computer readable storage medium storing a computer program which is adapted to be loaded and executed by a processor to implement the radar-based in-vehicle personnel heart rate monitoring method of the first aspect.

[0032] In another aspect, a computer program product is provided, the computer program product comprising a computer program which, when executed by a processor, implements the radar-based in-vehicle personnel heart rate monitoring method of the first aspect.

[0033] The technical solutions provided in the present application can bring at least the following beneficial effects:

[0034] The radar-based in-vehicle personnel heart rate monitoring method and system provided in the present application determine the phase signals of the distance gate where the human target is located and the distance gates near the distance gate according to the radar echo signals, and use the phase signals of multiple distance gates to extract the heartbeat signals, thereby improving the accuracy of heartbeat signal extraction. On the basis of accurately obtaining the heartbeat signals, accurate monitoring of the heart rate of the in-vehicle personnel is realized. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0036] Figure 1 is a flowchart of the radar-based in-vehicle personnel heart rate monitoring method provided in the embodiments of the present application;

[0037] Figure 2 is a principle diagram of phase signal extraction provided in the embodiments of the present application;

[0038] Figure 3 is a flowchart of phase signal extraction provided in the embodiments of the present application;

[0039] Figure 4A structural diagram of a vital sign information extraction model provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0040] To make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0041] Before the vital sign signal monitoring method provided by the embodiments of the present application is explained in detail, the application scenarios provided by the embodiments of the present application are introduced.

[0042] The radar-based in-vehicle personnel heart rate monitoring method provided by the embodiments of the present application can be applied to various scenarios. For example, in a vehicle driving scenario, if the heart rate of the driver is abnormal, the safety of vehicle driving will be affected. At this time, the heart rate of the driver can be monitored by the radar installed in the vehicle according to the radar-based in-vehicle personnel heart rate monitoring method provided by the embodiments of the present application, and alarm information is generated in the case that the heart rate of the driver is abnormal, so as to remind the driver to avoid possible risks in advance.

[0043] The radar-based in-vehicle personnel heart rate monitoring method provided by the embodiments of the present application can be executed by a vehicle. Further, the radar-based in-vehicle personnel heart rate monitoring method provided by the embodiments of the present application can also be implemented by a vital sign signal monitoring device, and the vehicle is taken as an example for subsequent description.

[0044] The vital sign signal monitoring device can be any electronic product that can interact with a user through one or more of a keyboard, a touchpad, a touch screen, a remote controller, voice interaction, or a handwriting device, such as a PC (Personal Computer), a mobile phone, a smart phone, a PDA (Personal Digital Assistant), a PPC (Pocket PC), a tablet computer, a smart car machine, a smart television, a smart speaker, and the like.

[0045] Those skilled in the art should understand that the above-mentioned application scenarios and vital sign signal monitoring devices are only examples, and other existing or future application scenarios and vital sign signal monitoring devices, such as those applicable to the embodiments of the present application, should also be included in the protection scope of the embodiments of the present application, and are hereby included by reference.

[0046] It should be noted that the business scenarios described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0047] Next, a radar-based in-vehicle personnel heart rate monitoring method provided by the embodiments of the present application will be explained in detail. Figure 1 is a flowchart of a radar-based in-vehicle personnel heart rate monitoring method provided by the embodiments of the present application, please refer to Figure 1 The method comprises the following steps.

[0048] Step 101: Obtain radar echo signals for monitoring vital signs of in-vehicle personnel.

[0049] The embodiments of the present application use the radar in the vehicle to emit frequency-modulated continuous wave (FMCW) radar signals to the in-vehicle personnel to monitor the vital signs of the in-vehicle personnel. The emitted radar signals produce backscattering after encountering the human body, and then the radar echo signals are formed by the antenna receiving.

[0050] The frequency-modulated continuous wave radar signal is the most common linear frequency-modulated continuous wave signal, that is, the emitted radar signal x T (t) changes linearly with time, which can be expressed as:

[0051]

[0052] Where A T is the amplitude of the transmitted signal, f c is the center frequency, B is the radar signal bandwidth, Tc is the radar signal duration, is the phase noise of the transmitter. The radar echo signal x R (t) is the time-delayed signal after the transmitted signal is reflected by the human body, which can be expressed as:

[0053]

[0054] Where AR is the amplitude of the received radar echo signal, t d is the time delay of the radar echo related to the detection target, which can be expressed as:

[0055]

[0056] Where R0 is the distance from the radar to the center of the chest cavity of the in-vehicle personnel, R(t) is the distance change due to the movement of the human chest cavity, and c is the speed of light. The intermediate frequency signal phase is the difference between the transmitted signal and the received signal phase, which can be expressed as:

[0057]

[0058] In reality, td is very small, and π, B, T c The values ​​of t and 2d can be ignored. Furthermore, when using frequency-modulated continuous wave radar to measure objects at short ranges, the residual phase noise... This can also be ignored, thus the simplified expression for the intermediate frequency signal phase is:

[0059]

[0060] Where, f IF =2B(R0+R(t)) / cT c , f IF and Both include information related to thoracic cavity displacement R(t), but because the thoracic cavity displacement caused by respiration and heartbeat is very small, it is difficult to detect in practice. IF To obtain the displacement of the thoracic cavity, and It is sensitive to changes in thoracic cavity displacement, therefore it is necessary to detect intermediate frequency signals. The changes in chest cavity displacement are obtained by observing the changes in chest cavity displacement, thereby obtaining the characteristics of human respiration and heartbeat.

[0061] Step 102: Based on the acquired radar echo signal, determine the phase signal of the range gate where the human target is located and the range gates near it.

[0062] In this context, the human target refers to the chest cavity reflex point. Analysis of the frequency-modulated continuous wave radar detection principle reveals that breathing and heartbeat cause chest cavity displacement. By detecting this displacement, information related to breathing and heartbeat can be obtained. Since the phase of the intermediate frequency signal is linearly related to the chest cavity displacement, detecting vital signs requires extracting the phase signal from the chest cavity reflex point. The principle diagram for phase signal extraction is shown below. Figure 2 As shown,

[0063] Perform a range-dimensional Fast Fourier Transform (FFT) on the radar echo signal to obtain the frequency domain signal;

[0064] The frequency domain signal is stored in a two-dimensional matrix R, which includes two dimensions: a slow time dimension and a fast time dimension.

[0065] Calculate the variance of matrix R along the slow time dimension;

[0066] The maximum value of the variance is determined along the fast time dimension. The maximum value is the distance gate corresponding to the human target. The distance gates to the left and right of the distance gate where the human target is located are the distance gates near the distance gate where the human target is located.

[0067] Phase extraction and unwrapping are performed on the frequency domain signals at the distance gate where the human target is located and the distance gates near the distance gate, to obtain phase signals at the distance gate where the human target is located and the distance gates near the distance gate.

[0068] Preferably, the I / Q signals of the radar echo signals are compensated and filtered to obtain compensated and filtered signals; and distance dimension fast Fourier transform is performed on the compensated and filtered signals to obtain the frequency domain signals.

[0069] In the embodiment of the application, the frequency domain signals are composed into a two-dimensional matrix R along the slow time dimension, and the frequency domain signals are processed in an observation window with a size of N to determine the position of the human target, and finally the phase signals at the target position are extracted.

[0070] A flowchart of the phase signal extraction is shown in Figure 3 Due to the characteristics of the devices in the I / Q two channels not being completely matched, phase offsets and amplitude errors are caused in the I / Q two channels, so that the amplitudes of the intermediate frequency signals in the I / Q two channels are not equal and the phases are not completely orthogonal. Therefore, compensation needs to be performed on the two signals, and the compensation of the I / Q two signals is performed using the algorithm in the literature. In addition, in actual radar data, not only the chest movement related signals of the target, but also the static clutter related to the environment around the target are included, and these clutters not only affect the positioning of the target, but also introduce phase noise. Therefore, the clutters need to be removed.

[0071] Therefore, the I / Q signals of the radar echo signals are compensated and filtered to obtain compensated and filtered signals; and distance dimension fast Fourier transform is performed on the compensated and filtered signals to obtain the frequency domain signals.

[0072] Preferably, the phasor mean cancellation method is used to eliminate the static clutter, and the implementation formula is:

[0073]

[0074] R[m, n] = R[m, n] - C[m]

[0075] Wherein, R is a matrix composed of N distance dimension frequency domain data (FFT data), m is a fast time dimension sampling point, and n is a slow time dimension sampling point.

[0076] Since the heartbeat and respiration can cause slight fluctuations in the phase intermediate frequency signals, the variance of the matrix R can be calculated along the slow time dimension, and then the maximum value of the variance in the fast time dimension is calculated, and the maximum value is the distance gate corresponding to the human target.

[0077] Compared with the AT demodulation algorithm of directly calculating the inverse tangent, the DACM demodulation algorithm can avoid the occurrence of direct current bias and quadrature imbalance to a certain extent. Therefore, the distance gate where the target is located is used to perform phase extraction and unwrapping on the FFT data by using the DACM demodulation algorithm.

[0078] Assuming that the signal of the distance gate where the target is located is R(t) = I(t) + jQ(t), the derivative of the phase signal is:

[0079]

[0080] where I(t)' and Q(t)' are the derivatives of I(t) and Q(t) respectively, and I(t) and Q(t) are echo data I / Q signals when the sampling point is t.

[0081] Further discretization and summation of the above formula can obtain the discrete form of the phase signal expression as:

[0082]

[0083] Step 103: Extract the spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate; extract the time sequence features from the extracted spatial features; and obtain the heartbeat signal of the person in the vehicle according to the time sequence features.

[0084] The embodiment of the application extracts the spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate by using a convolutional neural network; and extracts the time sequence features from the extracted spatial features by using a bidirectional long short-term memory neural network.

[0085] The phase signal contains the breathing and heartbeat related information of the target human body. After obtaining the phase signal, the breathing signal and the heartbeat signal can be extracted from the phase signal. The embodiment of the application uses a trained life feature extraction model to extract the breathing signal and the heartbeat signal of the person in the vehicle; and the life feature extraction model is obtained by using a convolutional bidirectional long short-term memory neural network (CNN-BiLSTM).

[0086] In order to make full use of the information in the radar echo and improve the detection accuracy, unlike the past only using the data of one range gate for vital sign detection, the distance FFT data of the range gate where the target is located and the range gates near the target are used for extraction of the respiration signal and the heartbeat signal in the embodiments of the present application, that is, the phase signals from the range gate where the target is located and the range gates near the target are extracted as the input of the neural network. Since in the distance dimension FFT data matrix, the human target occupies multiple adjacent range gates, and these range gates all contain vital sign related information. Therefore, using the phase signals of multiple range gates as the input of the neural network can improve the extraction accuracy of the respiration signal and the heartbeat signal. Moreover, due to the random movement of the human body and clutter interference, the range gate of the actual target will fluctuate near the detected range gate. The traditional algorithm only uses a single range gate and is difficult to solve this problem. Using multiple range gates can reduce the influence of the random movement of the human body and the clutter interference and increase the stability of the algorithm. The convolution bidirectional long short-term memory neural network structure is as shown in Figure 4 The network is composed of a one-dimensional convolutional neural network (CNN), a bidirectional long short-term memory neural network (BiLSTM) and two multi-layer fully connected networks. The one-dimensional convolutional neural network includes three convolutional layers and three transposed convolutional layers, which normalize the phase signals of the range gate where the human target is located and the range gates near the target, input the normalized phase signals into the one-dimensional convolutional neural network, and obtain the spatial features of the phase signals of the range gate where the human target is located and the range gates near the target after sequentially passing through the three convolutional layers and the three transposed convolutional layers; the spatial features output by the one-dimensional convolutional neural network are input into the bidirectional long short-term memory network, and the time sequence features of the spatial features are extracted through the bidirectional long short-term memory network; then, the time sequence features generate the respiration signal and the heartbeat signal through the two multi-layer fully connected networks respectively.

[0087] The bidirectional long short-term memory network uses bidirectional overlapping calculation to make up for the disadvantage that the long short-term memory network cannot extract information from back to front at the same time, and can better capture bidirectional timing characteristics. However, the bidirectional long short-term memory neural network has poor applicability in expressing spatial dimension characteristics of data, and it is difficult to fully extract spatial position characteristics of data. Therefore, a hybrid structure of a one-dimensional convolutional neural network and a bidirectional long short-term memory neural network is used to extract breathing and heartbeat related characteristics. The advantages of the convolutional neural network in spatial feature extraction are combined with the advantages of the bidirectional long short-term memory unit in timing feature extraction, the feature information in the space and time aspects is fully utilized, the data mining capability of the model is enhanced, and the accurate extraction of the breathing signal and the heartbeat signal is realized. In addition, the one-dimensional convolutional neural network part adopts a serial structure of convolution plus transposed convolution. In the convolution part, a large step is used to reduce the size of the output feature map and increase the receptive field, and then a plurality of layers of transposed convolution are used to complete the up-sampling. Through this structure, the spatial feature extraction process can be optimized, and the performance of the model can be improved.

[0088] The phase signal of the heartbeat signal and the breathing signal is labeled, and the constructed vital sign extraction model is trained. After training, a trained vital sign extraction model is obtained.

[0089] Step 104: According to the heartbeat signal of the person in the vehicle, the heart rate of the person in the vehicle is obtained.

[0090] After obtaining the heartbeat signal of the person in the vehicle, the peak value with an amplitude greater than a set heartbeat signal threshold value is searched in the time domain of the heartbeat signal, and the average time interval of the peak values is calculated to obtain the heartbeat period and the heart rate.

[0091] Since the heartbeat signal and the breathing signal are obtained at the same time in the embodiment of the application, when the heartbeat period and the heart rate are determined, the peak value with an amplitude greater than a set breathing signal threshold value is searched in the time domain of the breathing signal, and the average time interval of the peak values is calculated to obtain the breathing period and the breathing frequency.

[0092] In actual detection, there may be serious interference in the radar echo signal, which causes the neural network to be unable to effectively extract the breathing and heartbeat signals from the phase signal, and at this time the amplitude of the neural network output signal is usually small. In order to reduce the influence of interference on the estimation of vital sign parameters, a threshold is used to filter the peak value. This processing can improve the accuracy of vital sign parameter estimation.

[0093] After obtaining the heart rate of the person in the vehicle, the embodiment of the application further judges whether the heart rate of the person in the vehicle is abnormal; when the heart rate of the person in the vehicle is abnormal, an alarm information is sent.

[0094] Preferably, in combination with the host voice assistant and the IHU, the heart rate condition of the in-vehicle person is continuously monitored, when the heart rate of the in-vehicle person exceeds a set heart rate threshold, it is determined that the heart rate of the in-vehicle person is abnormal, and an alarm information is issued through the voice and the IHU to prompt the heart rate health condition.

[0095] wherein the heartbeat signal threshold, the respiration signal threshold and the heart rate threshold are all set in advance and can be adjusted according to different needs.

[0096] In the vehicle driving scenario, when the heart rate of the in-vehicle person is abnormal, an alarm information is generated to prompt the in-vehicle person. In this way, the in-vehicle person can take relevant measures in time to maintain personal safety. Moreover, after the alarm information lasts for a preset time length, the central gateway can also control the vehicle braking based on the alarm information to further maintain the personal safety of the in-vehicle person.

[0097] For example, in the case that the vehicle is not in the driving state and the heartbeat signal or the respiration signal is not zero, the in-vehicle temperature and / or the vehicle unlocking information are acquired, and in the case that the in-vehicle temperature is not in a target temperature range or the vehicle unlocking information indicates illegal unlocking, an alarm information is generated. That is, if the vehicle is not in the driving state and the heartbeat signal or the respiration signal is not zero, it indicates that there is a left person in the vehicle or that a foreign person has broken into the vehicle. At this time, in the case that the in-vehicle temperature is not in the target temperature range, an alarm information is generated to prompt the driver that the current in-vehicle temperature is high and there is a left person in the vehicle. In the case that the vehicle unlocking information indicates illegal unlocking, an alarm information is generated to prompt the driver that there is a foreign person who has illegally unlocked and broken into the vehicle in the vehicle.

[0098] wherein the target temperature range is set in advance. Moreover, the target temperature range can also be adjusted according to different needs.

[0099] In the vehicle non-driving scenario, when the heartbeat signal or the respiration signal of the in-vehicle person is not zero, an alarm information is generated to prompt the driver. In this way, the driver can avoid leaving children or the elderly in the vehicle with high temperature, and can also avoid the problem of property loss caused by the breaking into of foreign persons.

[0100] Next, a radar-based in-vehicle personnel heart rate monitoring process provided by the embodiment of the application is completely described. The radar echo signal received by the radar installed in the vehicle is subjected to fast Fourier transform, and the phase signal of the distance gate where the human target is located and the distance gate near the distance gate are determined. Then, the spatial features of the phase signal of the distance gate where the human target is located and the distance gate near the distance gate are extracted. The timing features are extracted from the extracted spatial features. The heartbeat signal of the in-vehicle personnel is obtained according to the timing features, and then the heart rate of the in-vehicle personnel is obtained. Since the phase signal of the distance gate where the human target is located and the distance gate near the distance gate is comprehensively considered when the heartbeat signal is extracted, and the spatial features and timing features of the phase signal are fully utilized, the accuracy of the heartbeat signal extraction is improved, and then the accuracy of the heart rate monitoring is ensured.

[0101] The radar-based in-vehicle personnel heart rate monitoring system provided by the embodiment of the application comprises:

[0102] The signal acquisition module is configured to acquire a radar echo signal for monitoring the vital signs of the in-vehicle personnel.

[0103] The phase signal extraction module is configured to determine the phase signal of the distance gate where the human target is located and the distance gate near the distance gate according to the acquired radar echo signal.

[0104] The heartbeat signal extraction module is configured to extract the spatial features of the phase signal of the distance gate where the human target is located and the distance gate near the distance gate, extract the timing features from the extracted spatial features, and obtain the heartbeat signal of the in-vehicle personnel according to the timing features.

[0105] The heart rate determination module is configured to obtain the heart rate of the in-vehicle personnel according to the heartbeat signal of the in-vehicle personnel.

[0106] It should be noted that the radar-based in-vehicle personnel heart rate monitoring system provided by the above embodiment only uses the division of the above functional modules as an example to illustrate the in-vehicle personnel heart rate monitoring, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the system is divided into different functional modules to complete all or part of the functions described above. In addition, the radar-based in-vehicle personnel heart rate monitoring system provided by the above embodiment and the radar-based in-vehicle personnel heart rate monitoring method embodiment belong to the same concept, and the specific implementation process is described in the method embodiment, which will not be described here.

[0107] The computer device provided by the embodiment of the application comprises:

[0108] The processor is adapted to execute the computer program.

[0109] A computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program, when executed by the processor, implements the radar-based in-vehicle personnel heart rate monitoring method disclosed in Embodiment 1.

[0110] The computer readable storage medium provided in the embodiments of the application stores a computer program, and the computer program is adapted to be loaded and executed by the processor to implement the radar-based in-vehicle personnel heart rate monitoring method disclosed in Embodiment 1.

[0111] The computer program product provided in the embodiments of the application comprises a computer program, and the computer program, when executed by the processor, implements the radar-based in-vehicle personnel heart rate monitoring method disclosed in Embodiment 1.

[0112] The method provided in the embodiments of the application can be directly embodied as hardware processor execution, or be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically programmable read-only memory, a register, or the like. The storage medium is located in the storage, and the processor reads information in the storage, and combines hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0113] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software mode depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0114] Although the specific embodiments of the application are described above in combination with the drawings, it is not a limitation on the protection scope of the application. Those skilled in the art should understand that various modifications or changes made by those skilled in the art on the basis of the technical solutions of the application without creative labor are still within the protection scope of the application.

Claims

1. A radar-based in-vehicle occupant heart rate monitoring method, characterized by, The method comprises the following steps: acquiring radar echo signals for monitoring vital signs of people in a vehicle; determining a distance gate where a human target is located and phase signals of distance gates near the distance gate according to the acquired radar echo signals; performing distance dimension fast Fourier transform on the radar echo signals to obtain frequency domain signals; storing the frequency domain signals in a two-dimensional matrix R, which comprises two dimensions of a slow time dimension and a fast time dimension; calculating variance of the matrix R along the slow time dimension; determining a maximum value of the variance along the fast time dimension, and the maximum value is the distance gate where the person is located, and distance gates on both sides of the distance gate where the person is located are distance gates near the distance gate where the person is located; performing phase extraction and unwrapping on the frequency domain signals at the distance gate where the human target is located and distance gates near the distance gate to obtain phase signals of the distance gate where the human target is located and the distance gates near the distance gate; extracting spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate; extracting spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate by using a convolutional neural network; and extracting time sequence features from the extracted spatial features by using a bidirectional long short-term memory neural network; the convolutional bidirectional long short-term memory neural network structure is composed of a one-dimensional convolutional neural network, a bidirectional long short-term memory neural network and two multilayer fully connected networks; the one-dimensional convolutional neural network comprises three layers of convolution and three layers of transposed convolution, and the phase signals of the distance gate where the human target is located and the distance gates near the distance gate are normalized, and the normalized phase signals are input into the one-dimensional convolutional neural network, and after sequentially passing through the three layers of convolution and the three layers of transposed convolution, the spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate are obtained; the spatial features output by the one-dimensional convolutional neural network are input into the bidirectional long short-term memory network, and the time sequence features of the spatial features are extracted by the bidirectional long short-term memory network; then, the time sequence features pass through the two multilayer fully connected networks to generate a breathing signal and a heartbeat signal respectively; a mixed structure of the one-dimensional convolutional neural network and the bidirectional long short-term memory neural network is used to extract features related to breathing and heartbeat; the advantages of the convolutional neural network in spatial feature extraction and the advantages of the bidirectional long short-term memory unit in time sequence feature extraction are combined to fully utilize feature information in space and time, enhance the data mining capability of the model, and realize accurate extraction of the breathing signal and the heartbeat signal; in addition, the one-dimensional convolutional neural network part adopts a serial structure of convolution and transposed convolution; in the convolution part, a large step is used to reduce the size of the output feature map and increase the receptive field, and then a plurality of layers of transposed convolution are used to complete up sampling; time sequence features are extracted from the extracted spatial features; a heartbeat signal of the person in the vehicle is obtained according to the time sequence features; a heart rate of the person in the vehicle is acquired according to the heartbeat signal of the person in the vehicle.

2. The radar-based in-vehicle occupant heart rate monitoring method of claim 1, wherein, The I / Q signals of the radar echo signals are compensated and filtered to obtain compensated and filtered signals; and distance dimension fast Fourier transform is performed on the compensated and filtered signals to obtain frequency domain signals.

3. The radar-based in-vehicle occupant heart rate monitoring method of claim 1, wherein, In the time domain of the heartbeat signal, peaks with amplitudes greater than a set heartbeat signal threshold value are searched, and the average time interval of the peaks is calculated to obtain a heartbeat period and a heart rate.

4. The radar-based in-vehicle occupant heart rate monitoring method of claim 1, wherein, Determine whether the heart rate of the person in the vehicle is abnormal; when the heart rate of the person in the vehicle is abnormal, an alarm information is sent out.

5. A radar-based in-vehicle occupant heart rate monitoring system, characterized by, The method comprises the steps of: The signal acquisition module is configured to acquire radar echo signals for monitoring vital signs of a person in a vehicle. The phase signal extraction module is configured to determine phase signals of a distance gate where the human target is located and distance gates near the distance gate according to the acquired radar echo signals. The radar echo signals are subjected to distance dimension fast Fourier transform to obtain frequency domain signals. The frequency domain signals are stored in a two-dimensional matrix R, which comprises a slow time dimension and a fast time dimension. The variance of the matrix R is calculated along the slow time dimension. The maximum value of the variance along the fast time dimension is determined, and the maximum value is the distance gate where the person is located. The phase signals of the distance gate where the human target is located and the distance gates near the distance gate are obtained by phase extraction and unwrapping. The heartbeat signal extraction module is configured to extract spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate; and extract time sequence features from the extracted spatial features. The heartbeat signal of the person in the vehicle is obtained according to the time sequence features. The spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate are extracted by using a convolutional neural network. The time sequence features are extracted from the extracted spatial features by using a bidirectional long short-term memory neural network. The convolutional bidirectional long short-term memory neural network structure is composed of a one-dimensional convolutional neural network, a bidirectional long short-term memory neural network and two multi-layer fully connected networks. The one-dimensional convolutional neural network comprises three convolutional layers and three transposed convolutional layers. The phase signals of the distance gate where the human target is located and the distance gates near the distance gate are normalized. The normalized phase signals are input into the one-dimensional convolutional neural network. The spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate are obtained after the normalized phase signals pass through the three convolutional layers and the three transposed convolutional layers in sequence.

6. An electronic device, comprising: The spatial features output by the one-dimensional convolutional neural network are input into the bidirectional long short-term memory network. The time sequence features of the spatial features are extracted by the bidirectional long short-term memory network. Then, the time sequence features pass through the two multi-layer fully connected networks to generate a breathing signal and a heartbeat signal respectively. The mixed structure of the one-dimensional convolutional neural network and the bidirectional long short-term memory neural network is used to extract features related to breathing and heartbeat. The advantages of the convolutional neural network in spatial feature extraction and the advantages of the bidirectional long short-term memory unit in time sequence feature extraction are combined. The feature information in space and time is fully utilized to enhance the data mining capability of the model and realize accurate extraction of the breathing signal and the heartbeat signal. In addition, the one-dimensional convolutional neural network part adopts a serial structure of convolution and transposed convolution. In the convolution part, a large step is used to reduce the size of the output feature map and increase the receptive field. The maximum value of the variance along the fast time dimension is determined, and the maximum value is the distance gate where the person is located. The phase signals of the distance gate where the human target is located and the distance gates near the distance gate are obtained by phase extraction and unwrapping. The heartbeat signal extraction module is configured to extract spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate; and extract time sequence features from the extracted spatial features. The heartbeat signal of the person in the vehicle is obtained according to the time sequence features. The spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate are extracted by using a convolutional neural network. The time sequence features are extracted from the extracted spatial features by using a bidirectional long short-term memory neural network. The convolutional bidirectional long short-term memory neural network structure is composed of a one-dimensional convolutional neural network, a bidirectional long short-term memory neural network and two multi-layer fully connected networks. The one-dimensional convolutional neural network comprises three convolutional layers and three transposed convolutional layers. The phase signals of the distance gate where the human target is located and the distance gates near the distance gate are normalized. The normalized phase signals are input into the one-dimensional convolutional neural network. The spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate are obtained after the normalized phase signals pass through the three convolutional layers and the three transposed convolutional layers in sequence. The spatial features output by the one-dimensional convolutional neural network are input into the bidirectional long short-term memory network. The time sequence features of the spatial features are extracted by the bidirectional long short-term memory network. Then, the time sequence features pass through the two multi-layer fully connected networks to generate a breathing signal and a heartbeat signal respectively. The mixed structure of the one-dimensional convolutional neural network and the bidirectional long short-term memory neural network is used to extract features related to breathing and heartbeat. The advantages of the convolutional neural network in spatial feature extraction and the advantages of the bidirectional long short-term memory unit in time sequence feature extraction are combined. The feature information in space and time is fully utilized to enhance the data mining capability of the model and realize accurate extraction of the breathing signal and the heartbeat signal. In addition, the one-dimensional convolutional neural network part adopts a serial structure of convolution and transposed convolution. In the convolution part, a large step is used to reduce the size of the output feature map and increase the receptive field. The maximum value of the variance along the fast time dimension is determined, and the maximum value is the distance gate where the person is located. The phase signals of the distance gate where the human target is located and the distance gates near the distance gate are obtained by phase extraction and unwrapping. The heartbeat signal extraction module is configured to extract spatial features of the phase signals of the distance gate where the human target is located and the distance gates near the distance gate; and extract time sequence features from the extracted spatial features. The heartbeat signal of the person in the vehicle is obtained according to the time sequence features. The device comprises: A processor adapted to execute a computer program; A computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program, when executed by a processor, implements the radar-based heart rate monitoring method for a person in a vehicle according to any one of claims 1-4.

7. A computer readable storage medium characterized by The computer readable storage medium stores a computer program, and the computer program is adapted to be loaded and executed by the processor to implement the radar-based heart rate monitoring method for a person in a vehicle according to any one of claims 1-4.

8. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program, when executed by a processor, implements the radar-based heart rate monitoring method for a person in a vehicle according to any one of claims 1-4.

Images