Device and method for extracting heartbeat data based on wireless radar signals

Abstract

An embodiment of the present application provides a device and method for extracting heartbeat data based on wireless radar signals. The method includes: collecting wireless radar signals obtained by sensing a detection object through a radar; performing band-pass filtering on the obtained wireless radar data in a first frequency band; calculating the spectral energy at each moment accumulated through a time window; performing feature extraction on the data after band-pass filtering to obtain band-pass filtering feature data, and / or performing feature extraction on the spectral energy at each moment within a second frequency band to obtain spectral energy feature data; inputting the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain into a detection model based on a neural network; and obtaining the heartbeat interval and heart rate of the detection object based on the waveform data. Thus, it is possible to accurately obtain heartbeat data in a non-contact manner, thereby realizing the monitoring of resting heart rate and heart rate variability; not only can heartbeat monitoring be conveniently performed at any time, but also the user experience is comfortable and the accuracy is high.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of radar detection. Background Art

[0002] With the improvement of living standards, people's health awareness has been gradually enhanced, and it is necessary to regularly evaluate the heart rate to monitor the health status. Generally speaking, there are two indicators for evaluating the heart rate: resting heart rate and heart rate variability (HRV). The resting heart rate refers to the number of heartbeats per minute of a normal person in a quiet state, and the heart rate variability refers to the change in the difference between successive heartbeats. The indicators for evaluating the heart rate have very important guiding significance for disease diagnosis and health management; the heart rate variability can also be used to evaluate the stress index and identify emotions.

[0003] Medically, electrocardiogram (ECG) is usually used to monitor the resting heart rate and heart rate variability of patients. The accuracy of ECG monitoring is very high, but it is necessary to paste patches at designated positions on the skin during the test, which is inconvenient to use and prone to discomfort, and it is also not convenient to use ECG for monitoring at any time.

[0004] With the popularization of smart wearable devices, heart rate monitoring has been integrated into most smart bracelets / watches. However, most of these devices can only provide heart rate indicators and are difficult to accurately evaluate the heart rate. Moreover, due to the need for long-term wearing, the elderly are prone to forget and it is troublesome to wear. On the other hand, non-contact heart rate monitoring solutions have emerged, such as analyzing the signals reflected by the radar through the human body and extracting signals related to the heart rate.

[0005] It should be noted that the above introduction to the technical background is only for the convenience of clearly and completely explaining the technical solutions of the present application and facilitating the understanding of those skilled in the art. It cannot be considered that the above technical solutions are well-known to those skilled in the art just because these solutions are described in the background art part of the present application. Summary of the Invention

[0006] However, the inventors found that: since the physiological signals include not only the heartbeat signals but also the respiration signals, the extracted heartbeat signals are easily affected by the respiration signals. In the current non-contact heart rate monitoring solutions, the presence or absence of the heartbeat signals can generally be detected, but the detection accuracy is low, and it is difficult to extract accurate heartbeat data.

[0007] To address at least one of the above technical problems, the embodiments of the present application provide a device and method for extracting heartbeat data based on wireless radar signals. It is expected to accurately obtain the heartbeat data non-contactly, so as to realize the monitoring of the resting heart rate and heart rate variability.

[0008] According to one aspect of the embodiments of the present application, there is provided a device for extracting heartbeat data based on wireless radar signals, including:

[0009] A collection unit that collects wireless radar signals obtained by a radar sensing a detection object;

[0010] A filtering unit that performs band-pass filtering on the obtained wireless radar signals in a first frequency band, where the first frequency band is a predetermined frequency band related to the heartbeat of the detection object;

[0011] A calculation unit that calculates the spectral energy at each moment accumulated by the data after band-pass filtering through a time window;

[0012] A feature extraction unit that extracts features from the data after band-pass filtering to obtain band-pass filtering feature data, and / or extracts features from the spectral energy at each moment within a second frequency band to obtain spectral energy feature data;

[0013] A detection unit that inputs the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain into a neural network-based detection model, and uses the detection model to obtain waveform data after detection; and

[0014] An acquisition unit that obtains the heartbeat interval and heart rate of the detection object based on the waveform data.

[0015] According to another aspect of the embodiments of the present application, there is provided a method for extracting heartbeat data based on wireless radar signals, including:

[0016] Collecting wireless radar signals obtained by a radar sensing a detection object;

[0017] Performing band-pass filtering on the obtained wireless radar signals in a first frequency band, where the first frequency band is a predetermined frequency band related to the heartbeat of the detection object;

[0018] Calculating the spectral energy at each moment accumulated by the data after band-pass filtering through a time window;

[0019] Extracting features from the data after band-pass filtering to obtain band-pass filtering feature data, and / or extracting features from the spectral energy at each moment within a second frequency band to obtain spectral energy feature data;

[0020] Inputting the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain into a neural network-based detection model, and using the detection model to obtain waveform data after detection; and

[0021] Obtaining the heartbeat interval and heart rate of the detection object based on the waveform data.

[0022] One of the beneficial effects of the embodiments of the present application lies in: obtaining band-pass filtering feature data and / or spectral energy feature data based on wireless radar signals, inputting the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain into a detection model based on a neural network, and obtaining waveform data after detection by using the detection model; and obtaining the heartbeat interval and heart rate of the detection object based on the waveform data. Thus, heartbeat data can be accurately obtained in a non-contact manner, so as to realize the monitoring of resting heart rate and heart rate variability; not only can heartbeat monitoring be conveniently carried out at any time, but also the user experience is comfortable and the accuracy is high.

[0023] Referring to the following description and drawings, specific embodiments of the embodiments of the present application are disclosed in detail, indicating the ways in which the principles of the embodiments of the present application can be adopted. It should be understood that the embodiments of the present application are not limited in scope thereby. Within the spirit and terms of the appended claims, the embodiments of the present application include many changes, modifications and equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings included are used to provide a further understanding of the embodiments of the present application, which form a part of the specification, are used to illustrate the embodiments of the present application, and are used to explain the principles of the present application together with the written description. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other embodiments according to these drawings without creative efforts. In the drawings:

[0025] Figure 1 is a schematic diagram of an electrocardiogram (ECG) signal obtained through an electrocardiogram;

[0026] Figure 2 is a schematic diagram of a method for extracting heartbeat data based on wireless radar signals according to an embodiment of the present application;

[0027] Figure 3 is an example diagram of wireless radar data according to an embodiment of the present application;

[0028] Figure 4 is an example diagram of data after band-pass filtering according to an embodiment of the present application;

[0029] Figure 5 is an example diagram of spectral energy at multiple moments according to an embodiment of the present application;

[0030] Figure 6 is an example diagram of an electrocardiogram signal obtained through an ECG according to an embodiment of the present application;

[0031] Figure 7 is an example diagram of band-pass filtering feature data according to an embodiment of the present application;

[0032] Figure 8 It is another schematic diagram of the method for extracting heartbeat data based on wireless radar signals according to an embodiment of the present application;

[0033] Figure 9 It is an example diagram of comparing the electrocardiogram signal obtained through ECG with triangular wave data according to an embodiment of the present application;

[0034] Figure 10 It is an example diagram of the loss and histogram of heart rate and heartbeat interval according to an embodiment of the present application;

[0035] Figure 11 It is a schematic diagram of the device for extracting heartbeat data based on wireless radar signals according to an embodiment of the present application;

[0036] Figure 12 It is a schematic diagram of the electronic device according to an embodiment of the present application. Detailed implementation manners

[0037] Referring to the accompanying drawings and through the following description, the foregoing and other features of the embodiments of the present application will become apparent. In the description and drawings, specific embodiments of the present application are disclosed, which show some embodiments in which the principles of the embodiments of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the embodiments of the present application include all modifications, variations, and equivalents falling within the scope of the appended claims.

[0038] In the embodiments of the present application, terms such as "first" and "second" are used to distinguish different elements in terms of name, but do not indicate the spatial arrangement or time sequence of these elements, and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. Terms such as "comprising", "including", and "having" mean the presence of the stated features, elements, components, or assemblies, but do not exclude the presence or addition of one or more other features, elements, components, or assemblies.

[0039] In the embodiments of the present application, the singular forms "a", "the", etc. include the plural forms and should be broadly understood as "a kind of" or "a class of" rather than being limited to the meaning of "one"; in addition, the term "the" should be understood to include both the singular form and the plural form unless the context clearly indicates otherwise. In addition, the term "according to" should be understood as "at least partially according to...", and the term "based on" should be understood as "at least partially based on...", unless the context clearly indicates otherwise.

[0040] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or instead of features in other embodiments. The term "comprising" as used herein refers to the presence of features, whole things, steps, or components, but does not exclude the presence or addition of one or more other features, whole things, steps, or components.

[0041] In an embodiment of the present application, the radar may be a millimeter-wave (mmWave) radar, but is not limited thereto. The radar emits electromagnetic waves through a transmitting antenna, and after being reflected by different objects, receives corresponding reflected waves (which can be called radar echo information). By analyzing the radar echo information, information such as the position of the object relative to the radar and the radial movement speed can be effectively extracted, and this information can meet the requirements of many application scenarios.

[0042] In an embodiment of the present application, the object to be detected may be people of various age groups. For example, it may be the elderly, or it may be children, or it may be the elderly and / or caregivers, children and / or guardians. The present application is not limited thereto, and the object to be detected may also be an animal with vital signs, etc. The following will take the human body as an example for illustration.

[0043] Figure 1 is a schematic diagram of an electrocardiogram (ECG) signal, as Figure 1 shown, the electrocardiogram signal has at least two R-wave information (peaks) 101. The beat interval (IBI, Inter-Beat Interval) and heart rate can be calculated through this electrocardiogram signal.

[0044] In an embodiment of the present application, the heartbeat data includes the heartbeat interval of the detection object (which can be characterized by IBI) and the heart rate; the heart rate can characterize the index of resting heart rate, and the heartbeat interval can be used to analyze heart rate variability. For the specific content of these concepts, reference can be made to related technologies.

[0045] Embodiments of the first aspect

[0046] An embodiment of the present application provides a method for extracting heartbeat data based on wireless radar signals.

[0047] Figure 2 is a schematic diagram of the method for extracting heartbeat data based on wireless radar signals in an embodiment of the present application, as Figure 2 shown, the method includes:

[0048] 201, collect wireless radar signals obtained by the radar sensing the detection object;

[0049] 202. Perform band-pass filtering on the obtained wireless radar data in a first frequency band, where the first frequency band is a predetermined frequency band related to the heartbeat of the detection object.

[0050] 203. Calculate the spectral energy at each moment accumulated by the data after band-pass filtering through a time window.

[0051] 204. Extract features from the data after band-pass filtering to obtain band-pass filtering feature data, and / or extract features from the spectral energy at each moment within a second frequency band to obtain spectral energy feature data.

[0052] 205. Input the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain into a detection model based on a neural network, and use the detection model to obtain the detected waveform data; and

[0053] 206. Obtain the heartbeat interval and heart rate of the detection object based on the waveform data.

[0054] It should be noted that the above appendix Figure 2 only schematically illustrates the embodiments of the present application, but the present application is not limited thereto. For example, the execution order between various operations can be appropriately adjusted. In addition, some other operations can be added or some of the operations can be reduced. Those skilled in the art can make appropriate modifications according to the above content, not limited to the records in the above appendix Figure 2 .

[0055] In some embodiments, a continuous wave radar device with single-frequency modulation can be used to sense the external space through the wireless signal transmitted by the radar transmitter. The detection object is within the radiation range of the radar device. For example, it can sit or lie still at a certain distance from the radar device, such as less than 60 cm. One radar device can be used to monitor a certain detection object, but the present application is not limited thereto. For example, for a radar device with enhanced capabilities, multiple detection objects can be monitored simultaneously.

[0056] In some embodiments, the radar receiver can receive the radar echo (wireless radar signal), and use a data acquisition card to acquire the wireless radar signal at a first sampling rate S1 to obtain wireless radar data. The first sampling rate S1 can be relatively high, for example, 2500 Hz.

[0057] Figure 3 is an example diagram of the wireless radar data of the embodiments of the present application. As Figure 3 shown, the wireless radar data includes, for example, an in-phase signal radar_I(n) and a quadrature signal radar_Q(n).

[0058] In some embodiments, the first frequency band is a predetermined frequency band related to the heartbeat of the detection object; for example, the first frequency band is [8, 100] Hz, or [-100, -8][8, 100] Hz. Through this band-pass filter, the influence of low-frequency respiration signals and high-frequency noise signals can be eliminated to a certain extent, thus providing a basis for accurately extracting heartbeat data.

[0059] For example, perform band-pass filtering on radar_I(n) and radar_Q(n) to obtain the data mainly modulated by heartbeat fluctuations after band-pass filtering: radar_I_bp(n) and radar_Q_bp(n).

[0060] In some embodiments, the DC frequency offset can also be eliminated from the data after band-pass filtering.

[0061] For example, the following operations can be performed to eliminate the DC offset:

[0062] radar_I_bp(n) = radar_I_bp(n) - mean(radar_I_bp(n));

[0063] radar_Q_bp(n) = radar_Q_bp(n) - mean(radar_Q_bp(n)).

[0064] Figure 4 is an example diagram of the data after band-pass filtering in the embodiments of the present application. As Figure 4 shown, the data after band-pass filtering includes, for example, the in-phase signal radar_I_bp(n) and the quadrature signal radar_Q_bp(n) after eliminating the DC offset.

[0065] In some embodiments, the data after band-pass filtering can be converted from a time-domain signal to a frequency-domain signal by sliding a time window to obtain the spectral energy at each moment. For example, the short-time Fourier transform (STFT) or wavelet transform, etc., can be used to convert the data after band-pass filtering from a time-domain signal to a frequency-domain signal.

[0066] For example, the processed IQ complex signal is transformed from the time domain to the frequency domain, and the complex signal is

[0067] S(n) = radar_I_bp(n) + j * radar_Q_bp(n);

[0068] The time window can be represented by a sequence, and n can represent each moment.

[0069] For example, a complex signal S(n) can be segmented into n sequences S(k) of k points, and a frequency-domain transformation (such as FFT, etc.) is performed on each sequence S(k, i) to obtain the spectrum z(t(i), f(m), i) of the sequence. After merging the spectra of each sequence, the spectrum z(t(n), f(m)) of the complex signal S(n) is obtained.

[0070] Among them, S(k, i) represents the i-th sequence with a sequence length of k, i = 0, 1, 2,..., n, and k represents the number of points of the sequence for frequency-domain transformation. This number of points can be a fixed value, such as 512, etc. z(t(i), f(m), i) represents the spectral energy of the i-th sequence (i.e., at time t(i), t(i) = i * f s , f s is the sampling frequency) at each frequency point f(m), m = 0, 1, 2,..., k - 1.

[0071] Figure 5 is an example diagram of the spectral energy at multiple moments in an embodiment of the present application. Among them, the segmented time window is, for example, 204.8 ms (512-point FFT), and the sliding step is 0.4 ms; for example, it can slide point by point to ensure that the sampling rate is consistent with the sampling rate of the IQ signal. For example, within the range of [-30, -8][8, 30] Hz, the spectral energy at each moment can be obtained (as shown by the gray and white parts in Figure 5 ).

[0072] Figure 6 is an example diagram of the electrocardiogram signal obtained through ECG in an embodiment of the present application. As shown in Figure 5 and 6 , Figure 5 the spectral energy distribution is basically consistent with the electrocardiogram signal distribution of Figure 6 . Thus, it can also be preliminarily verified that the embodiments of the present application can improve the accuracy of heartbeat data.

[0073] The following first describes how to obtain spectral energy feature data.

[0074] In some embodiments, the second frequency band can be a partial frequency band within the first frequency band. For example, the second frequency band is [8, 30] Hz, or [-30, -8][8, 30] Hz. By performing spectral aggregation analysis within the second frequency band, the influence of noise signals can be further eliminated, and the accuracy of heartbeat data can be improved.

[0075] In some embodiments, feature extraction is performed on the spectral energy at each moment within the second frequency band to obtain spectral energy feature data, and the spectral energy feature data includes one or more of the following:

[0076] - Data obtained by summing the spectral energy at each moment of a partial frequency band within the second frequency band;

[0077] For example, data obtained by summing the spectral energies in the frequency band [8, 30] Hz:

[0078]

[0079] For another example, data obtained by summing the spectral energies in the frequency band [-30, -8] Hz:

[0080]

[0081] - Data obtained by finding the maximum value of the spectral energies at each moment in some frequency bands within the second frequency band;

[0082] For example, data obtained by finding the maximum value of the spectral energies in the frequency band [8, 30] Hz:

[0083]

[0084] For another example, finding the maximum value of the spectral energies in the frequency band [-30, -8] Hz

[0085]

[0086] - Data obtained by summing the spectral energies at each moment in all frequency bands within the second frequency band;

[0087] For example, data obtained by summing the spectral energies in the frequency bands [8, 30] Hz and [-30, -8] Hz:

[0088]

[0089] - Data obtained by finding the maximum value of the spectral energies at each moment in all frequency bands within the second frequency band;

[0090] For example, data obtained by finding the maximum value of the spectral energies in the frequency bands [8, 30] Hz and [-30, -8] Hz:

[0091]

[0092] The above gives a schematic description of the spectral energy characteristic data, but the present application is not limited thereto, and other characteristic data obtained by feature extraction based on spectral energy can also be used. Some or all of these spectral energy characteristic data can be input into a detection model based on a neural network, and the detected waveform data can be obtained using the detection model.

[0093] The following further describes how to obtain the band-pass filtering characteristic data.

[0094] In some embodiments, feature extraction may also be performed on the band-pass filtered data to obtain band-pass filtered feature data. The band-pass filtered feature data includes one or more of the following:

[0095] - Data obtained by band-pass filtering the quadrature signal of the radar data;

[0096] For example, radar_Q(n);

[0097] - Data obtained by band-pass filtering the in-phase signal of the radar data;

[0098] For example, radar_I_bp(n);

[0099] - Data obtained by band-pass filtering the radar data and then calculating the amplitude;

[0100] For example,

[0101] where S(n) = radar_I_bp(n) + j * radar_Q_bp(n).

[0102] Figure 7 is an example diagram of the band-pass filtered feature data of the embodiments of the present application, showing the data radar_I_bp after band-pass filtering and removing the DC bias, the data radar_Q_bp after band-pass filtering and removing the DC bias, and the data iq_am after calculating the amplitude.

[0103] The above is a schematic description of the band-pass filtered feature data, but the present application is not limited thereto, and other feature data obtained by performing feature extraction on the band-pass filtered data may also be used. Some or all of these band-pass filtered feature data may be input into a neural network-based detection model, and the detection model is used to detect the spectral energy feature data and the band-pass filtered feature data to obtain the detected waveform data.

[0104] Figure 8 is another schematic diagram of the method for extracting heartbeat data based on wireless radar signals in the embodiments of the present application, schematically showing the case of using both the band-pass filtered feature data and the spectral energy feature data as the feature data input to the detection model. As Figure 8 shown, the method includes:

[0105] 801, collecting wireless radar signals obtained by sensing a detection object through a radar;

[0106] 802, performing band-pass filtering on the obtained wireless radar data in a first frequency band, where the first frequency band is a predetermined frequency band related to the heartbeat of the detection object;

[0107] 803. Calculate the spectral energy at each moment accumulated by the data after band-pass filtering through a time window;

[0108] 804. Extract features from the spectral energy at each moment within the second frequency band to obtain spectral energy feature data.

[0109] As Figure 8 shown, the method further includes:

[0110] 805. Extract features from the data after band-pass filtering to obtain band-pass filtering feature data;

[0111] 806. Input the spectral energy feature data and the band-pass filtering feature data into a neural network-based detection model, and use the detection model to obtain the waveform data after detection; and

[0112] 807. Obtain the heart rate interval and heart rate of the detection object based on the waveform data.

[0113] It should be noted that the above appendix Figure 8 only schematically illustrates the embodiments of the present application, but the present application is not limited thereto. For example, the execution order between various operations can be appropriately adjusted. In addition, some other operations can be added or some of the operations can be reduced. Those skilled in the art can make appropriate modifications according to the above content, not limited to the records in the above appendix Figure 8 .

[0114] In some embodiments, before inputting the feature data (spectral energy feature data and / or band-pass filtering feature data) into a neural network-based detection model, the feature data can also be resampled at a second sampling rate S2; where the second sampling rate S2 is lower than the first sampling rate S1.

[0115] For example, the first sampling rate S1 is 2500 Hz and the second sampling rate S2 is 200 Hz. Through the higher first sampling rate, a wireless radar signal that accurately reflects the heartbeat characteristics can be obtained, so that accurate feature data can be obtained; through the lower second sampling rate, data distortion can be ensured and the complexity of the deep learning model can be reduced.

[0116] In some embodiments, the resampled feature data can also be normalized.

[0117] For example, the min-max normalization method is used to normalize the feature information of the radar.

[0118] Taking radar_I_bp as an example,

[0119] In some embodiments, the feature data within a period of time can be input into the detection model; where the period of time is a predetermined time (such as 2 s) that can ensure including at least two R-wave information. Thus, the accuracy of the detection data can be further guaranteed, and the response time is relatively fast.

[0120] In some embodiments, the waveform data output by the detection model is triangular wave data, whereby the waveform data can be simply and conveniently processed.

[0121] Figure 9 It is an example diagram of comparing the electrocardiogram signal obtained by ECG with the triangular wave data in an embodiment of the present application. As Figure 9 shown, the loss (as shown by the broken line in Figure 9 ) between the triangular wave data in the embodiment of the present application and the electrocardiogram signal obtained by ECG is controlled within a certain range. Therefore, the accuracy of the heartbeat data in the embodiment of the present application is relatively high.

[0122] In some embodiments, noise can be added to the triangular wave data output from the detection model; and the maximum value of the triangular wave data after adding noise is detected to obtain the heartbeat interval and heart rate of the detection object.

[0123] For example, to avoid detecting the same maximum value point, a small amount of noise can be added to the triangular wave data:

[0124] ypred = ypred + rand(length(ypred)) * 0.0001.

[0125] In some embodiments, the heartbeat interval and heart rate of the detection object can be obtained based on the triangular wave data after adding noise. Specifically, the maximum value of the triangular wave data can be corresponded to the R-wave information of the electrocardiogram signal obtained by electrocardiogram. The heartbeat interval of the detection object is obtained based on the time interval between adjacent maximum values, and the heart rate of the detection object is obtained based on the obtained heartbeat interval of the detection object. Thus, detecting the maximum value of the triangular wave after adding noise and corresponding the position of the maximum value point to the position of the R wave in the electrocardiogram can further improve the detection accuracy.

[0126] For example, the current IBI is obtained by averaging the values of the adjacent R-wave intervals (R-R interval) within a 5 s window:

[0127] Assume that the data sampling rate is s0, the window duration is 5 s, there are m R waves, and the corresponding positions of the R waves are {l0, l1,... l m-1}}, then,

[0128]

[0129] The heart rate at the current moment is obtained from the IBI at the current moment, that is,

[0130]

[0131] Figure 10 is an example diagram of the loss and histogram of the heart rate and heartbeat interval in the embodiments of the present application. As Figure 10 shown, through the solution of the embodiments of the present application, the heartbeat data can be accurately extracted from the wireless radar signal.

[0132] The above has schematically described how to extract the heartbeat data. The following briefly describes the detection model and training.

[0133] In some embodiments, the detection model based on a neural network can be a deep learning neural network model. For example, Resnet or VGG can be used, or it can also be a 1D fully convolutional network. The loss function can be KLDivLoss, etc. During the training process, the electrocardiogram signal detected by the ECG can be used as the ground truth for training.

[0134] The embodiments of the present application can obtain a set / multiple sets of optimal parameters through a supervised training method; then apply the parameters to the detection model. The embodiments of the present application do not limit the structure of the detection model and can refer to related technologies. In addition, the specific training method, etc. is not limited either. For example, SGD (Stochastic Gradient Descent) optimization, Adam (Adaptive Moment Estimation) optimization, etc. can be used.

[0135] The above only describes the steps or processes related to the present application, but the present application is not limited thereto. The method for extracting the heartbeat data may further include other steps or processes. For the specific content of these steps or processes, reference can be made to the prior art.

[0136] The above embodiments only exemplarily illustrate the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications can also be made on the basis of the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0137] As can be seen from the above embodiments, band-pass filtering feature data and / or spectral energy feature data are obtained based on wireless radar signals, the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain are input into a detection model based on a neural network, and the waveform data after detection is obtained by using the detection model; and the heart rate interval and heart rate of the detection object are obtained based on the waveform data. Thus, the heart rate data can be accurately obtained in a non-contact manner, so as to realize the monitoring of resting heart rate and heart rate variability; not only can the heart rate be conveniently monitored at any time, but also the user experience is comfortable and the accuracy is high.

[0138] Embodiments of the second aspect

[0139] The embodiments of the present application provide a device for extracting heart rate data based on wireless radar signals. The same content as that in the embodiments of the first aspect will not be repeated.

[0140] Figure 11 is a schematic diagram of the device for extracting heart rate data based on wireless radar signals in the embodiments of the present application. As Figure 11 shown, the device 1100 for extracting heart rate data based on wireless radar signals includes:

[0141] An acquisition unit 1101 that acquires wireless radar signals obtained by sensing a detection object through a radar;

[0142] A filtering unit 1102 that performs band-pass filtering on the obtained wireless radar signals in a first frequency band, and the first frequency band is a predetermined frequency band related to the heartbeat of the detection object;

[0143] A calculation unit 1103 that calculates the spectral energy at each moment accumulated by the data after band-pass filtering through a time window;

[0144] A feature extraction unit 1104 that extracts features from the data after band-pass filtering to obtain band-pass filtering feature data, and / or extracts features from the spectral energy at each moment within a second frequency band to obtain spectral energy feature data;

[0145] A detection unit 1105 that inputs the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain into a detection model based on a neural network, and uses the detection model to obtain the waveform data after detection; and

[0146] An acquisition unit 1106 that obtains the heart rate interval and heart rate of the detection object based on the waveform data.

[0147] In some embodiments, the calculation unit 1103 converts the data after band-pass filtering from a time domain signal to a frequency domain signal through a sliding time window to obtain the spectral energy at each moment.

[0148] In some embodiments, the computing unit 1103 uses the short-time Fourier transform or the wavelet transform to convert the data after band-pass filtering from a time-domain signal to a frequency-domain signal.

[0149] In some embodiments, the spectral energy feature data includes one or more of the following:

[0150] Data obtained by summing the spectral energies at each moment of partial frequency bands within the second frequency band;

[0151] Data obtained by taking the maximum value of the spectral energies at each moment of partial frequency bands within the second frequency band;

[0152] Data obtained by summing the spectral energies at each moment of all frequency bands within the second frequency band;

[0153] Data obtained by taking the maximum value of the spectral energies at each moment of all frequency bands within the second frequency band.

[0154] In some embodiments, the band-pass filtering feature data includes one or more of the following:

[0155] Data obtained by performing band-pass filtering on the quadrature signal of the radar data;

[0156] Data obtained by performing band-pass filtering on the in-phase signal of the radar data;

[0157] Data obtained by performing band-pass filtering on the radar data and then calculating the amplitude.

[0158] In some embodiments, the acquisition unit 1101 samples at a first sampling rate; the device further includes:

[0159] A resampling unit 1107 that resamples the feature data at a second sampling rate before inputting the feature data into the neural network-based detection model; where the second sampling rate is lower than the first sampling rate; and

[0160] A normalization unit 1108 that normalizes the resampled feature data.

[0161] In some embodiments, the detection unit 1105 inputs the feature data within a period of time into the detection model; where the period of time is a predetermined time that can ensure at least two R-wave information including the heartbeat.

[0162] In some embodiments, the waveform data output by the detection model is triangular wave data.

[0163] In some embodiments, the acquisition unit 1106 is further configured to: add noise to the triangular wave data output from the detection model; and detect the maximum value of the triangular wave data after adding noise to obtain the heartbeat interval and heart rate of the detection object.

[0164] In some embodiments, the acquisition unit 1106 is specifically configured to: correspond the maximum value of the triangular wave data to the R-wave information of the electrocardiogram signal obtained through electrocardiogram, obtain the heartbeat interval of the detection object based on the time interval between adjacent maximum values, and obtain the heart rate of the detection object based on the obtained heartbeat interval of the detection object.

[0165] It should be noted that only the components or modules related to the present application are described above, but the present application is not limited thereto. The device 1100 for extracting heartbeat data based on wireless radar signals may further include other components or modules. For the specific content of these components or modules, reference may be made to the related art.

[0166] For simplicity, Figure 11 only the connection relationships or signal directions between the various components or modules are exemplarily shown in the figure, but those skilled in the art should be clear that various related technologies such as bus connection can be adopted. The above-mentioned various components or modules can be implemented by means of hardware facilities such as a processor and a memory; the embodiments of the present application do not limit this.

[0167] The above embodiments only exemplarily illustrate the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications can also be made on the basis of the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0168] As can be seen from the above embodiments, band-pass filtering feature data and / or spectral energy feature data are obtained based on wireless radar signals, the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain are input into a detection model based on a neural network, and the detected waveform data is obtained by using the detection model; and the heartbeat interval and heart rate of the detection object are obtained based on the waveform data. Thereby, heartbeat data can be accurately obtained in a non-contact manner, so as to realize the monitoring of resting heart rate and heart rate variability; not only can heartbeat monitoring be conveniently carried out at any time, but also the user experience is comfortable and the accuracy is high.

[0169] Embodiments of the third aspect

[0170] The embodiments of the present application provide an electronic device, including the device 1100 for extracting heartbeat data based on wireless radar signals as described in the embodiments of the second aspect, the content of which is incorporated herein. The electronic device may be, for example, a computer, a server, a workstation, a laptop computer, a smart phone, etc.; but the embodiments of the present application are not limited thereto.

[0171] Figure 12 is a schematic diagram of the electronic device according to an embodiment of the present application. As Figure 12 shown, the electronic device 1200 may include: a processor (such as a central processing unit CPU) 1210 and a memory 1220; the memory 1220 is coupled to the central processor 1210. The memory 1220 can store various data; in addition, a program 1221 for information processing is also stored, and the program 1221 is executed under the control of the processor 1210.

[0172] In some embodiments, the function of the device 1100 for extracting heartbeat data based on wireless radar signals is integrated into the processor 1210. Among them, the processor 1210 is configured to implement the method for extracting heartbeat data based on wireless radar signals as described in the embodiments of the first aspect.

[0173] In some embodiments, the device 1100 for extracting heartbeat data based on wireless radar signals is separately configured from the processor 1210. For example, the device 1100 for extracting heartbeat data based on wireless radar signals can be configured as a chip connected to the processor 1210, and the function of the device 1100 for extracting heartbeat data based on wireless radar signals is realized through the control of the processor 1210.

[0174] For example, the processor 1210 is configured to perform the following controls: collect wireless radar signals for sensing a detection object through a radar; perform band-pass filtering on the obtained wireless radar data in a first frequency band, where the first frequency band is a predetermined frequency band related to the heartbeat of the detection object; calculate the spectral energy at each moment accumulated through a time window for the data after band-pass filtering; perform feature extraction on the data after band-pass filtering to obtain band-pass filtering feature data, and / or perform feature extraction on the spectral energy at each moment in a second frequency band to obtain spectral energy feature data; input the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain into a detection model based on a neural network, and use the detection model to obtain the detected waveform data; and obtain the heartbeat interval and heart rate of the detection object based on the waveform data.

[0175] In addition, as Figure 12 shown, the electronic device 1200 may further include: an input / output (I / O) device 1230, a display 1240, etc.; among them, the functions of the above components are similar to those in the prior art and will not be described in detail here. It should be noted that the electronic device 1200 does not necessarily have to include Figure 12 all the components shown in; in addition, the electronic device 1200 may further include Figure 12 components not shown in, and reference can be made to the related art.

[0176] An embodiment of the present application further provides a computer-readable program, which, when executed on an electronic device, causes the computer to execute the method for extracting heartbeat data based on wireless radar signals as described in the embodiments of the first aspect in the electronic device.

[0177] An embodiment of the present application further provides a storage medium storing a computer-readable program, wherein the computer-readable program causes the computer to execute the method for extracting heartbeat data based on wireless radar signals as described in the embodiments of the first aspect in the electronic device.

[0178] The above devices and methods of the present application can be implemented by hardware or by a combination of hardware and software. The present application relates to such a computer-readable program that, when executed by a logic component, can cause the logic component to implement the above-mentioned device or component, or cause the logic component to implement the above-mentioned various methods or steps. The present application also relates to a storage medium for storing the above program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0179] The method / device described in combination with the embodiments of the present application can be directly embodied as hardware, a software module executed by a processor, or a combination of the two. For example, one or more of the functional block diagrams shown in the figure and / or a combination of one or more of the functional block diagrams can correspond to each software module in the computer program flow, and can also correspond to each hardware module. These software modules can respectively correspond to the respective steps shown in the figure. These hardware modules can be implemented by, for example, using a field programmable gate array (FPGA) to solidify these software modules.

[0180] The software module can be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium can be coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium can be a component of the processor. The processor and the storage medium can be located in an ASIC. The software module can be stored in the memory of the mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) uses a larger-capacity MEGA-SIM card or a large-capacity flash device, the software module can be stored in the MEGA-SIM card or the large-capacity flash device.

[0181] One or more of the functional blocks described in the accompanying drawings and / or one or more combinations of functional blocks can be implemented as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any suitable combination thereof for performing the functions described in this application. One or more of the functional blocks described in the accompanying drawings and / or one or more combinations of functional blocks can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication combination with a DSP, or any other such configuration.

[0182] The present application has been described in conjunction with specific embodiments, but those skilled in the art should understand that these descriptions are exemplary and not a limitation on the protection scope of the present application. Those skilled in the art can make various variations and modifications to the present application based on the principles of the present application, and these variations and modifications are also within the scope of the present application.

[0183] Regarding the embodiments including the above embodiments, the following supplementary notes are also disclosed:

[0184] Supplementary Note 1. A method for extracting heartbeat data based on wireless radar signals, comprising:

[0185] Collecting wireless radar signals obtained by sensing a detection object through a radar;

[0186] Performing band-pass filtering on the obtained wireless radar data in a first frequency band, where the first frequency band is a predetermined frequency band related to the heartbeat of the detection object;

[0187] Calculating the spectral energy at each moment accumulated by the band-pass filtered data through a time window;

[0188] Performing feature extraction on the band-pass filtered data to obtain band-pass filtered feature data, and / or performing feature extraction on the spectral energy at each moment within a second frequency band to obtain spectral energy feature data;

[0189] Inputting the band-pass filtered feature data in the time domain and / or the spectral energy feature data in the frequency domain into a detection model based on a neural network, and using the detection model to obtain detected waveform data; and

[0190] Obtaining the heartbeat interval and heart rate of the detection object based on the waveform data.

[0191] Supplementary Note 2. The method according to Supplementary Note 1, wherein calculating the spectral energy at each moment accumulated by the band-pass filtered data through a time window comprises:

[0192] Convert the data after band-pass filtering from the time-domain signal to the frequency-domain signal by sliding a time window to obtain the spectral energy at each moment.

[0193] Remark 3. According to the method described in Remark 2, wherein the data after band-pass filtering is converted from the time-domain signal to the frequency-domain signal using the short-time Fourier transform (STFT) or wavelet transform.

[0194] Remark 4. According to the method described in any one of Remarks 1 to 3, wherein the spectral energy characteristic data includes one or more of the following:

[0195] Data obtained by summing the spectral energy at each moment of partial frequency bands within the second frequency band;

[0196] Data obtained by taking the maximum value of the spectral energy at each moment of partial frequency bands within the second frequency band;

[0197] Data obtained by summing the spectral energy at each moment of all frequency bands within the second frequency band;

[0198] Data obtained by taking the maximum value of the spectral energy at each moment of all frequency bands within the second frequency band.

[0199] Remark 5. According to the method described in any one of Remarks 1 to 4, wherein the method further includes:

[0200] Eliminating the DC frequency offset from the data after band-pass filtering.

[0201] Remark 6. According to the method described in any one of Remarks 1 to 5, wherein the band-pass filtering characteristic data includes one or more of the following:

[0202] Data obtained by performing band-pass filtering on the quadrature signal of the radar data;

[0203] Data obtained by performing band-pass filtering on the in-phase signal of the radar data;

[0204] Data obtained by performing band-pass filtering on the radar data and then calculating the amplitude.

[0205] Remark 7. According to the method described in any one of Remarks 1 to 6, wherein the wireless radar signal is sampled at a first sampling rate.

[0206] Remark 8. According to the method described in Remark 7, wherein before inputting the characteristic data into the neural network-based detection model, the method further includes:

[0207] Resampling the characteristic data at a second sampling rate; wherein the second sampling rate is lower than the first sampling rate; and

[0208] Normalizing the resampled characteristic data.

[0209] Supplementary Note 9. The method according to any one of Supplementary Notes 1 to 8, wherein characteristic data within a period of time is input into the detection model; wherein the period of time is a predetermined time (such as 2 s) capable of ensuring at least two R-wave information including heartbeats.

[0210] Supplementary Note 10. The method according to any one of Supplementary Notes 1 to 9, wherein the waveform data output by the detection model is triangular wave data.

[0211] Supplementary Note 11. The method according to Supplementary Note 10, wherein the method further comprises:

[0212] adding noise to the triangular wave data output from the detection model; and

[0213] detecting the maximum value of the triangular wave data after adding noise to obtain the heart rate interval and heart rate of the detection object.

[0214] Supplementary Note 12. The method according to Supplementary Note 11, wherein detecting the maximum value of the triangular wave data after adding noise to obtain the heart rate interval and heart rate of the detection object comprises:

[0215] corresponding the maximum value to the R-wave information of the electrocardiogram signal obtained by electrocardiogram;

[0216] obtaining the heart rate interval of the detection object based on the time interval between adjacent maximum values; and

[0217] obtaining the heart rate of the detection object based on the obtained heart rate interval of the detection object.

[0218] Supplementary Note 13. An electronic device, comprising a memory and a processor, the memory storing a computer program, the processor being configured to execute the computer program to implement the method for extracting heartbeat data based on wireless radar signals according to any one of Supplementary Notes 1 to 12.

[0219] Supplementary Note 14. A storage medium storing a computer-readable program, wherein the computer-readable program causes a computer to execute the method for extracting heartbeat data based on wireless radar signals according to any one of Supplementary Notes 1 to 12 in an electronic device.

Claims

1. A device for extracting heartbeat data based on wireless radar signals, characterized in that, The device includes: a collection unit that collects wireless radar signals obtained by sensing a detection object through a radar; a filtering unit that performs band-pass filtering on the obtained wireless radar signals in a first frequency band, where the first frequency band is a predetermined frequency band related to the heartbeat of the detection object; a calculation unit that calculates the spectral energy at each moment accumulated through a time window for the data after band-pass filtering; a feature extraction unit that extracts features from the data after band-pass filtering to obtain band-pass filtering feature data, and / or extracts features from the spectral energy at each moment within a second frequency band to obtain spectral energy feature data; wherein the second frequency band is a partial frequency band within the first frequency band; a detection unit that inputs the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain into a detection model based on a neural network, and uses the detection model to obtain detected triangular wave data; and an acquisition unit that, based on the triangular wave data, obtains the heartbeat interval and heart rate of the detection object by detecting the maximum value of the triangular wave data.

2. The device according to claim 1, wherein The calculation unit converts the data after band-pass filtering from a time-domain signal to a frequency-domain signal through a sliding time window to obtain the spectral energy at each moment.

3. The apparatus according to claim 2, wherein, The calculation unit uses short-time Fourier transform or wavelet transform to convert the data after band-pass filtering from a time-domain signal to a frequency-domain signal.

4. The device according to claim 1, wherein, The spectral energy feature data includes one or more of the following: data obtained by summing the spectral energy at each moment of partial frequency bands within the second frequency band; data obtained by finding the maximum value of the spectral energy at each moment of partial frequency bands within the second frequency band; data obtained by summing the spectral energy at each moment of all frequency bands within the second frequency band; data obtained by finding the maximum value of the spectral energy at each moment of all frequency bands within the second frequency band.

5. The apparatus according to claim 1, wherein The band-pass filtering feature data includes one or more of the following: data obtained by performing band-pass filtering on the quadrature signal of the radar data obtained by sampling the wireless radar signal; data obtained by performing band-pass filtering on the in-phase signal of the radar data; data obtained by performing band-pass filtering on the radar data and then finding the amplitude.

6. The device according to claim 1, wherein, The collection unit samples at a first sampling rate; the device further includes: a resampling unit that resamples the feature data at a second sampling rate before inputting the feature data into the detection model based on a neural network; where the second sampling rate is lower than the first sampling rate; and a normalization unit that normalizes the resampled feature data.

7. The device according to claim 1, wherein The detection unit inputs the feature data within a period of time into the detection model; where the period of time is a predetermined time that can ensure at least two R-wave information including heartbeats.

8. The device according to claim 1, wherein, the acquisition unit is further configured to: add noise to the triangular wave data output from the detection model; and detect the maximum value of the triangular wave data after adding noise to obtain the heartbeat interval and heart rate of the detection object.

9. The device according to claim 8, wherein, The obtaining unit corresponds the maximum value to the R-wave information of the electrocardiogram signal obtained by electrocardiogram, obtains the heartbeat interval of the detection object based on the time interval between adjacent maximum values, and obtains the heart rate of the detection object based on the obtained heartbeat interval of the detection object.

10. A method for extracting heartbeat data based on wireless radar signals, comprising: Collecting wireless radar signals obtained by sensing a detection object through a radar; Performing band-pass filtering on the obtained wireless radar data in a first frequency band, where the first frequency band is a predetermined frequency band related to the heartbeat of the detection object; Calculating the spectral energy at each moment accumulated by the data after band-pass filtering through a time window; Performing feature extraction on the data after band-pass filtering to obtain band-pass filtering feature data, and / or performing feature extraction on the spectral energy at each moment in a second frequency band to obtain spectral energy feature data; where the second frequency band is a partial frequency band within the first frequency band; Inputting the band-pass filtering feature data in the time domain and / or the spectral energy feature data in the frequency domain into a detection model based on a neural network, and using the detection model to obtain detected triangular wave data; And Based on the triangular wave data, obtaining the heartbeat interval and heart rate of the detection object by detecting the maximum value of the triangular wave data.

Images