Non-contact electrocardiosignal measuring device and measuring method thereof

Abstract

The invention discloses a non-contact electrocardiosignal measuring device and a measuring method thereof, and relates to the technical field of radar signal detection, the non-contact electrocardiosignal measuring device comprises a multi-domain joint modulation micro-motion signal deentanglement module and a signal reconstruction network based on time-frequency domain transformation; the multi-domain joint modulation micro-motion signal de-entanglement module is configured to receive a radar waveform obtained by electrocardiosignal acquisition by a millimeter wave frequency modulation continuous wave radar and extract a heart mechanical motion signal from the radar waveform; the signal reconstruction network based on time-frequency domain transformation is configured to receive the extracted heart mechanical motion signal, learn and construct a nonlinear relation between the heart mechanical motion signal and the electrocardiosignal, and output a predicted electrocardiosignal according to the heart mechanical motion signal. The problem of how to extract the heart mechanical motion signal by using a radar technology and avoid a noise signal generated by interference factors is solved, and the predicted electrocardiosignal is obtained through the heart mechanical motion signal.

Description

Technical Field

[0001] The present application relates to the field of radar signal detection technology, and in particular to a non-contact electrocardiogram signal measurement device and a measurement method thereof. Background Art

[0002] As a key tool for assessing heart health, the electrocardiogram (ECG) has traditionally relied on contact electrodes to record the heart's electrical activity. This method requires the electrode patch to fit tightly to the skin and be connected to the ECG recording device via wires. These cumbersome steps not only increase the complexity of the measurement, but may also cause inconvenience to the subject's daily activities. With the advancement of technology, wearable biosensors have provided new possibilities for ECG measurements, but these devices still need to be in direct contact with the skin and are easily affected by a variety of factors, such as perspiration, sensor fit, and ambient temperature, which limits the reliability, adaptability, and continuity of continuous monitoring.

[0003] In recent years, remote photoplethysmography (rPPG) based on face video has become a research hotspot. Non-contact rPPG technology indirectly reflects heartbeat information by analyzing changes in facial skin color, but it is easily affected by skin type, facial expression and lighting changes, which limits its measurement accuracy.

[0004] Therefore, how to avoid the limited reliability, adaptability and continuity of continuous monitoring caused by direct contact of wearable biosensors with the skin, and at the same time avoid the insufficient measurement accuracy of non-contact rPPG technology, has become a problem that needs to be solved urgently. Summary of the invention

[0005] In an embodiment of the present application, a non-contact ECG signal measurement device is provided to solve the problem of how to avoid the limitations on the reliability, adaptability and continuity of continuous monitoring caused by direct contact of wearable biosensors with the skin, while avoiding the problem of insufficient measurement accuracy of non-contact rPPG technology.

[0006] In a first aspect, an embodiment of the present application provides a contactless ECG signal measurement device, comprising a multi-domain joint modulation micro-motion signal de-entanglement module and a signal reconstruction network based on time-frequency domain transformation; the multi-domain joint modulation micro-motion signal de-entanglement module is configured to receive a radar waveform obtained from a millimeter-wave frequency-modulated continuous wave radar for ECG signal acquisition, and extract a cardiac mechanical motion signal from the radar waveform; the signal reconstruction network based on time-frequency domain transformation is configured to receive the extracted cardiac mechanical motion signal, learn and construct a nonlinear relationship between the cardiac mechanical motion signal and the ECG signal, so as to output a predicted ECG signal based on the cardiac mechanical motion signal.

[0007] In a possible implementation, a radar control module is also included; the radar control module is configured to design and simulate the millimeter-wave frequency modulated continuous wave radar using millimeter-wave design software, and configure parameters of the millimeter-wave frequency modulated continuous wave radar.

[0008] In the second aspect, an embodiment of the present application provides a measurement method for a contactless ECG signal measurement device, including: receiving a radar waveform obtained from a millimeter-wave frequency-modulated continuous wave radar for ECG signal acquisition in a multi-domain joint modulation micro-motion signal de-entanglement module, and extracting a cardiac mechanical motion signal from the radar waveform; receiving the extracted cardiac mechanical motion signal in a signal reconstruction network based on time-frequency domain transformation, learning and constructing a nonlinear relationship between the cardiac mechanical motion signal and the ECG signal, so as to output a predicted ECG signal based on the cardiac mechanical motion signal; using millimeter-wave design software in a radar control module to design and simulate a millimeter-wave frequency-modulated continuous wave radar, and configuring the millimeter-wave parameters of the millimeter-wave frequency-modulated continuous wave radar.

[0009] In a possible implementation, the method of extracting the cardiac mechanical motion signal from the radar waveform includes: performing time-frequency domain data smoothing processing on each channel data in the radar waveform to obtain a zero-frequency component signal; performing spatial channel superposition on the zero-frequency component signal to obtain a signal matrix after spatial channel superposition; performing target selection on the signal matrix after spatial channel superposition to obtain a screened target position set; and performing time-frequency filtering on the screened target position set to extract the cardiac mechanical motion signal.

[0010] In a possible implementation, the time-frequency domain data smoothing processing is performed on each channel data in the radar waveform to obtain a zero-frequency component signal, including: the time-frequency domain data smoothing processing time-domain smoothing processing and frequency-domain processing; each channel data in the radar waveform includes a cardiac mechanical motion signal, a chest motion signal and a noise signal; the time-domain smoothing processing includes: multiplying the channel data with a Hanning window to generate a first smoothed signal; the frequency-domain processing includes: performing Fourier transform on the first smoothed signal along the time dimension to convert it into the frequency domain to obtain a frequency-domain signal; multiplying the frequency-domain signal with another Hanning window to generate a second smoothed signal; performing Fourier transform on the second smoothed signal according to a characteristic dimension to generate a characteristic frequency-domain signal; using a two-dimensional second-order Fourier transform to process each signal channel in the characteristic frequency-domain signal to obtain a processed signal, and using a fast Fourier transform shift function to move the zero-frequency component of the processed signal to the center of the frequency spectrum to obtain a zero-frequency component signal.

[0011] In a possible implementation, performing spatial channel superposition on the zero-frequency component signal to obtain a signal matrix after spatial channel superposition includes: an expression of spatial channel superposition is: ;in, is the signal matrix after the spatial channels are superimposed, is the channel index, is the total channel data, is a zero frequency component signal, Indicates the absolute value operation.

[0012] In a possible implementation, the target selection is performed on the signal matrix after the spatial channels are superimposed to obtain a screened target position set, including: intercepting two signal clips of the same length from the head and tail of the signal matrix after the spatial channels are superimposed; using a connection operation to connect the two signal clips of the same length with the signal matrix after the spatial channels are superimposed to generate an extended signal matrix; traversing each sampling point in the extended signal matrix to obtain the background clutter power of each sampling point; averaging the background clutter powers of the sampling points within a preset range as the average background clutter power; calculating the difference between the signal value of the sampling points within the preset range and the average background clutter power; and judging whether the difference is greater than A preset threshold is set; if the difference is greater than the preset threshold, the judgment result is that there is a target at the location of the sampling point; if the difference is less than or equal to the preset threshold, the judgment result is that there is no target at the location of the sampling point; a target detection result matrix is ​​constructed based on the judgment result, and a peak search algorithm is performed on the target detection result matrix to identify potential local maximum points, and the potential local maximum points are used as potential target positions; wherein each element in the target detection result matrix marks whether there is a target at the corresponding sampling point; according to the preset maximum and minimum distance ranges from the millimeter wave frequency modulated continuous wave radar to the human body, the potential target positions are screened, and the target positions within the distance range are retained; and the screened target position set is output.

[0013] In a possible implementation, the performing of a time-frequency filtering operation on the filtered target position set to extract the cardiac mechanical motion signal includes: performing a second-order difference processing on the signal in the filtered target position set to reduce low-frequency interference; the expression of the second-order difference processing is: ;in, is the result signal obtained after performing second-order difference processing on the signals in the filtered target position set. is the second signal in the filtered target position set, is the first signal in the filtered target position set, is the last signal in the filtered target position set, The signal before the last signal in the filtered set of target positions, for Function; Create FIR filter; The expression of FIR filter is: ;in, To create a good FIR filter, is the length of the filter, is the passband boundary frequency range, , , is the sampling frequency, is the upper limit of the heartbeat amplitude, is the lower limit of the heartbeat amplitude, is the Kaiser window function, is the parameter of Kaiser window, A filtering operation is performed for an FIR filter; and the result signal is filtered using the FIR filter to extract the cardiac mechanical motion signal.

[0014] In a possible implementation, the method of receiving the extracted cardiac mechanical motion signal in a signal reconstruction network based on time-frequency domain transformation, learning and constructing the nonlinear relationship between the cardiac mechanical motion signal and the electrocardiogram signal, and outputting a predicted electrocardiogram signal according to the cardiac mechanical motion signal comprises: the signal reconstruction network based on time-frequency domain transformation receives the cardiac mechanical motion signal as input; the signal reconstruction network based on time-frequency domain transformation comprises a plurality of multi-channel time-frequency fusion extraction modules; a convolutional neural network is used to perform preliminary feature extraction on the cardiac mechanical motion signal; the extracted features are normalized and processed with an activation function, and then input into a plurality of multi-channel time-frequency fusion extraction modules; a convolutional neural network is used to perform preliminary feature extraction on the cardiac mechanical motion signal; and the extracted features are normalized and processed with an activation function, and then input into a plurality of multi-channel time-frequency fusion extraction modules. In the multi-channel time-frequency fusion extraction module, nonlinear feature mapping is performed, and the ECG signal is used as the target signal to learn and construct the nonlinear relationship between the cardiac mechanical motion signal and the ECG signal; wherein, the multi-channel time-frequency fusion extraction module includes a residual structure, which includes a convolutional neural network, batch normalization and an activation function; the signal reconstruction network based on the time-frequency domain transformation outputs the predicted ECG signal; the signal reconstruction network based on the time-frequency domain transformation is trained using a supervised learning method, and its parameters are optimized by minimizing the loss function; the evaluation coefficient is used to evaluate the similarity between the predicted ECG signal and the ground truth ECG signal in the time domain; the expression of the evaluation coefficient is: ;in, is the evaluation coefficient, To predict the ECG signal, is the ground truth ECG signal, is the average value of the ground truth ECG signal, is the predicted average value of the ECG signal, is the standard deviation of the ground truth ECG signal, is the standard deviation of the predicted ECG signal; the mean absolute error is used to calculate the distance between the predicted ECG signal and the ground truth ECG signal in the frequency domain; the expression of the mean absolute error is: ;in, is the mean absolute error, To predict the ECG signal, is the ground truth ECG signal, is the fast Fourier transform, is the total number of frequency domain data points, is the frequency domain data point index, h is the function used to calculate the mean absolute error, Indicates the absolute value operation; The loss function is obtained based on the evaluation coefficient and the mean absolute error; the loss function is: ;in, is the total loss, is the evaluation coefficient, is the mean absolute error.

[0015] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects: The embodiment of the present application provides a non-contact ECG signal measurement device, which uses a millimeter-wave frequency-modulated continuous wave radar to collect ECG signals. It does not need to be in direct contact with the subject's skin, avoids the cumbersome steps of electrode patches and wire connections in traditional ECG measurements, and greatly improves the convenience of measurement and user experience. The multi-domain joint modulation micro-motion signal de-entanglement module can accurately separate the cardiac mechanical motion signal from the complex radar waveform, effectively eliminate the influence of interference factors, and ensure the purity and accuracy of the extracted signal. The signal reconstruction network based on time-frequency domain transformation can accurately output the predicted ECG signal by constructing a nonlinear relationship between the cardiac mechanical motion signal and the ECG signal, thereby realizing real-time monitoring and dynamic analysis of the ECG signal. It solves the problem of how to avoid the reliability, adaptability and continuity of continuous monitoring caused by direct contact of wearable biosensors with the skin, and at the same time avoids the problem of insufficient measurement accuracy of non-contact rPPG technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A schematic diagram of a non-contact electrocardiogram signal measuring device provided in an embodiment of the present application; Figure 2 A flow chart of a measurement method of a non-contact electrocardiogram signal measurement device provided in an embodiment of the present application; Figure 3A waveform diagram of performing time domain smoothing and frequency domain processing on a single channel data in a radar waveform provided in an embodiment of the present application; Figure 4 A waveform diagram of two-dimensional Fourier transform and spatial channel superposition provided in an embodiment of the present application; Figure 5 A schematic diagram of using potential local maximum points as potential target positions in a target detection process provided by an embodiment of the present application; Figure 6 A schematic diagram of performing second-order difference processing on signals in a filtered target position set provided in an embodiment of the present application; Figure 7 A schematic diagram of a signal reconstruction network based on time-frequency domain transformation provided in an embodiment of the present application being trained under the supervision of a ground truth electrocardiogram signal; Figure 8 A schematic diagram of a non-contact ECG signal measurement server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The following describes some of the techniques involved in the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, some descriptions of well-known functions and structures are omitted in the following description.

[0020] The present application provides a non-contact electrocardiogram signal measuring device. Figure 1As shown, it includes a multi-domain joint modulation micro-motion signal de-entanglement module and a signal reconstruction network based on time-frequency domain transformation. The multi-domain joint modulation micro-motion signal de-entanglement module is configured to receive the radar waveform obtained from the millimeter-wave frequency modulation continuous wave radar for ECG signal acquisition, and extract the cardiac mechanical motion signal from the radar waveform. The signal reconstruction network based on time-frequency domain transformation is configured to receive the extracted cardiac mechanical motion signal, learn and construct the nonlinear relationship between the cardiac mechanical motion signal and the ECG signal, so as to output the predicted ECG signal according to the cardiac mechanical motion signal. It also includes a radar control module. The radar control module is configured to design and simulate the millimeter-wave frequency modulation continuous wave radar using millimeter-wave design software, and configure the parameters of the millimeter-wave frequency modulation continuous wave radar.

[0021] The present application embodiment provides a measurement method of a non-contact electrocardiogram signal measurement device, such as Figure 2 As shown, the method includes steps S101 to S103. Among them, Figure 2 This is only an execution sequence shown in the embodiment of the present application, and does not represent the only execution sequence of a measurement method of a non-contact electrocardiogram signal measurement device. In the case that the final result can be achieved, Figure 2 The steps shown may be performed in parallel or reversed.

[0022] S101: receiving a radar waveform obtained from a millimeter-wave frequency-modulated continuous-wave radar for collecting electrocardiographic signals in a multi-domain joint-modulation micro-motion signal de-entanglement module, and extracting a cardiac mechanical motion signal from the radar waveform.

[0023] Furthermore, extracting the cardiac mechanical motion signal from the radar waveform includes: performing time-frequency domain data smoothing processing on each channel data in the radar waveform to obtain a zero-frequency component signal. Performing spatial channel superposition on the zero-frequency component signal to obtain a signal matrix after spatial channel superposition. Performing target selection on the signal matrix after spatial channel superposition to obtain a filtered target position set. Performing time-frequency filtering operation on the filtered target position set to extract the cardiac mechanical motion signal.

[0024] Specifically, each channel data in the radar waveform is smoothed in the time and frequency domain to obtain a zero-frequency component signal, including: time and frequency domain data smoothing, time domain smoothing, and frequency domain processing. Each channel data in the radar waveform includes a cardiac mechanical motion signal, a chest motion signal, and a noise signal. The time and frequency domain data smoothing process can remove noise other than the human body motion noise generated by the human heart mechanical motion and breathing.

[0025] The expression of each channel data in the radar waveform is: .in, For the Channel data, is the mechanical motion signal of the heart, It is a chest movement signal. is the noise signal. Among them, Sources include ground echoes, intermodulation products, discrete reflectors, vibrations of components inside the receiving equipment, and human motion.

[0026] The time domain smoothing process includes: performing a multiplication operation on the channel data and the Hanning window to generate a first smoothed signal. The purpose of this step is to smooth the channel data in the time domain to reduce the influence of noise.

[0027] The expression of the first smoothed signal is: .in, is the first smoothed signal generated, For the Channel data, , For the signal channel, is the feature dimension, is the time dimension, is the set of real numbers, For the Hanning window, is a hyperparameter, is the first weight parameter of the Hanning window. Specifically, when the channel data is one-dimensional, The value can be 1, and the hyperparameter can be 10.

[0028] The frequency domain processing includes: performing Fourier transform on the first smoothed signal along the time dimension, converting it into the frequency domain, and obtaining a frequency domain signal.

[0029] The frequency domain signal is multiplied by another Hanning window to generate a second smoothed signal. The purpose of this step is to further smooth the channel data in the frequency domain to further reduce the impact of noise.

[0030] The expression of the second smoothed signal is: .in, is the second smoothed signal, is the frequency domain signal, For another Hanning window, is a hyperparameter, is the second weight parameter of the Hanning window. Specifically, when the channel data is one-dimensional, The value can be 1, and the hyperparameter can be 10.

[0031] Perform Fourier transform on the second smoothed signal according to the characteristic dimension to generate a characteristic frequency domain signal.

[0032] A two-dimensional second-order Fourier transform is used to process each signal channel in the characteristic frequency domain signal to obtain a processed signal, and a fast Fourier transform shift function is used to move the zero-frequency component of the processed signal to the center of the spectrum to obtain a zero-frequency component signal.

[0033] The expression of the zero frequency component signal is: .in, is a zero frequency component signal, is the Fourier transform dimension, is the characteristic frequency domain signal, is the fast Fourier transform shift function.

[0034] Specifically, for the remaining channel data, the above-mentioned time-frequency domain data smoothing process is repeated. Since the noise signal mainly comes from the human body motion in the channel data and the interference of discrete reflectors, it is not efficient to process these noise signals one by one. Therefore, it is possible to directly extract the cardiac mechanical motion signal. Chest movement signal caused by breathing The combination of can effectively remove noise signals from channel data .

[0035] The zero-frequency component signal is subjected to spatial channel superposition to obtain a signal matrix after spatial channel superposition, including: The expression of spatial channel superposition is: .in, is the signal matrix after the spatial channels are superimposed, is the channel index, is the total channel data, is a zero frequency component signal, Indicates the absolute value operation.

[0036] Specifically, through spatial channel superposition, multiple channel data are combined into a signal matrix after spatial channel superposition. This process reduces the size of the data while amplifying the useful signals that exist in all channels. Through this process, the mechanical motion signal of the heart and the chest motion signal generated by breathing are successfully extracted.

[0037] Specifically, target selection aims to identify targets contained in the signal matrix after the spatial channels are superimposed.

[0038] This process mainly relies on calculating the difference between the sampling point and the average background clutter power to determine whether there is a target at the sampling point. However, this method will encounter errors when processing the first and last positions of the signal matrix after the spatial channel superposition, because these are single-sided signal data.

[0039] Therefore, target selection is performed on the signal matrix after the spatial channels are superimposed to obtain a screened target position set, including the following steps.

[0040] Two signal clips of the same length are respectively cut from the head and tail of the signal matrix after the spatial channels are superimposed.

[0041] Using the concatenation operation, two signal clips of the same length are concatenated with the signal matrix after the spatial channels are superimposed to generate an extended signal matrix.

[0042] The expression of the extended signal matrix is: .in, is the expansion matrix, For the connection operation, This is the signal clip captured from the head of the signal matrix after the spatial channels are superimposed. It is the signal clipping taken from the tail of the signal matrix after the spatial channels are superimposed. It is the signal matrix after the spatial channels are superimposed.

[0043] Each sampling point in the extended signal matrix is ​​traversed to obtain the background clutter power of each sampling point.

[0044] The background clutter powers of the sampling points within the preset range are averaged to be the average background clutter power.

[0045] It should be noted that the preset range may be a range of three consecutive sampling points centered at the current sampling point and including the previous sampling point and the next sampling point. The average background clutter power is the average value of all sampling points in the extended signal matrix.

[0046] Calculate the difference between the signal value of the sampling point within the preset range and the average background clutter power.

[0047] Determine whether the difference is greater than a preset threshold.

[0048] If the difference is greater than a preset threshold, the result is that there is a target at the location of the sampling point.

[0049] If the difference is less than or equal to the preset threshold, the judgment result is that there is no target at the location of the sampling point.

[0050] Specifically, the expression for determining whether there is a target at the sampling point is: .in, For sampling point The target detection result, that is, whether there is a target at the location of the sampling point, its value is True (target exists) or False (target does not exist). is the preset threshold, is the signal value of the sampling point within the preset range, is the average background clutter power.

[0051] The expression for calculating the difference between the signal value of the sampling point within the preset range and the average background clutter power is: .in, is the difference, is the signal value of the sampling point within the preset range, is the average background clutter power.

[0052] Based on the judgment results, a target detection result matrix is ​​constructed, and a peak search algorithm is performed on the target detection result matrix to identify potential local maximum points, and the potential local maximum points are used as potential target positions. Among them, each element in the target detection result matrix marks whether there is a target at the corresponding sampling point.

[0053] According to the preset maximum and minimum distance ranges from the millimeter wave frequency modulated continuous wave radar to the human body, potential target positions are screened and target positions within the distance range are retained. The preset maximum and minimum distance ranges can be 0.15 meters to 0.45 meters.

[0054] Output the filtered set of target locations.

[0055] The filtered target position set is subjected to a time-frequency filtering operation to extract the cardiac mechanical motion signal, including the following steps.

[0056] The signals in the filtered target position set are processed by second-order difference to reduce low-frequency interference.

[0057] Specifically, the low-frequency interference may be chest movement interference caused by breathing. The second-order difference processing can quickly capture the rapid changes in the signal, and then extract the slope or curvature information of the signal, making the subsequent calculation process more concise and efficient.

[0058] The expression of second-order difference processing is: .in, is the result signal obtained after performing second-order difference processing on the signals in the filtered target position set. is the second signal in the filtered target position set, is the first signal in the filtered target position set, is the last signal in the filtered target position set, The signal before the last signal in the filtered set of target positions, for function.

[0059] Create a FIR filter.

[0060] The expression of FIR filter is: .in, To create a good FIR filter, is the length of the filter, is the passband boundary frequency range, , , is the sampling frequency, is the upper limit of the heartbeat amplitude, is the lower limit of the heartbeat amplitude, is the Kaiser window function, is the parameter of Kaiser window, Performs filtering operations for FIR filters.

[0061] The resulting signal is filtered using a FIR filter to extract the cardiac mechanical motion signal.

[0062] The expression of the filtering process is: .in, is the mechanical motion signal of the heart, Perform filtering operation for FIR filter, The result signal is obtained after performing second-order difference processing on the signals in the filtered target position set.

[0063] S102: receiving the extracted cardiac mechanical motion signal in a signal reconstruction network based on time-frequency domain transformation, learning and constructing a nonlinear relationship between the cardiac mechanical motion signal and the electrocardiogram signal, so as to output a predicted electrocardiogram signal according to the cardiac mechanical motion signal.

[0064] The extracted cardiac mechanical motion signal is received in a signal reconstruction network based on time-frequency domain transformation, and the nonlinear relationship between the cardiac mechanical motion signal and the electrocardiogram signal is learned and constructed to output a predicted electrocardiogram signal according to the cardiac mechanical motion signal, including the following steps.

[0065] The signal reconstruction network based on time-frequency domain transformation receives the cardiac mechanical motion signal as input. The signal reconstruction network based on time-frequency domain transformation includes multiple multi-channel time-frequency fusion extraction modules. Specifically, the English name of the signal reconstruction network based on time-frequency domain transformation is RadarNet. The English name of the multi-channel time-frequency fusion extraction module is TFEM (time-frequency fusion extraction modules).

[0066] Convolutional neural network is used to perform preliminary feature extraction of cardiac mechanical motion signals.

[0067] Specifically, the convolutional neural network used in the present application may be a 1DCNN (one-dimensional convolutional neural network) with a kernel size of 3.

[0068] The following data sets correspond to the training sets of machine learning models in signal reconstruction networks based on time-frequency domain transformation.

[0069] Specifically, the radar physiological signal dataset (RadarPhys-30) of this application contains radar physiological data collected from 30 volunteers, which are obtained by collecting ECG signals through millimeter-wave frequency modulated continuous wave radar. The data collection device is TI IWR1843BOOST FMCW (a model of millimeter-wave frequency modulated continuous wave radar), which is facilitated by TIADC1000 (a model of analog-to-digital converter used with radar) to facilitate real-time data transmission. The radar parameter settings are listed in Table 1, including start frequency, frequency slope, chirp idle time, slope end time, sampling point, sampling rate, periodicity, number of chirp signal cycles, and frame. Synchronous ECG signal and HR (heart rate) acquisition uses BIOPAC MP160 (physiological data acquisition system) and AcqKarawlk5.0 software. The radar physiological signal dataset contains radar physiological data of 30 volunteers, with a male-female ratio of 6:4 (i.e. 60% male and 40% female). The age range of the participants was between 18-25 years old, the weight ranged from 50 kg to 80 kg, and all were from Asia. Two conditions were considered in the data collection process: the subjects were in a lying state and a sitting state. In the lying state, the subjects were completely relaxed and 50 seconds of signals were collected, which were then divided into five separate segments. In the sitting state, 30 seconds of signals were collected and then divided into three segments. The antenna of the millimeter-wave frequency-modulated continuous wave radar was pointed directly at the subject's body, maintaining a distance between 0.15 meters and 0.45 meters. Electrodes on the subject's right ankle and right wrist were used to facilitate the acquisition of ECG signals. After completing the data collection of one state, the subject was allowed to rest for 1 minute before starting the next signal collection state. In this way, a radar physiological signal dataset was obtained.

[0070] It should be noted that the radar physiological signal data set mentioned in this application is only an exemplary data set for illustrating the principle and specific implementation of this application. The core content and protection scope of this application are not limited to this data set or the specific data contained therein.

[0071] Table 1

[0072] After being normalized and processed by activation functions, the extracted features are input into multiple multi-channel time-frequency fusion extraction modules for nonlinear feature mapping, and the ECG signal is used as the target signal to learn and construct the nonlinear relationship between the cardiac mechanical motion signal and the ECG signal. Among them, the multi-channel time-frequency fusion extraction module includes a residual structure, which includes a convolutional neural network, batch normalization and an activation function.

[0073] Specifically, the activation function in the present application may be a ReLU activation function.

[0074] The ECG signal predicted by the signal reconstruction network output based on time-frequency domain transformation.

[0075] A supervised learning method is used to train a signal reconstruction network based on time-frequency domain transformation, and its parameters are optimized by minimizing the loss function. Specifically, the errors in the time domain and the errors in the frequency domain can be filtered out to minimize the loss function and optimize its parameters.

[0076] The similarity between the predicted ECG signal and the ground truth ECG signal in the time domain is evaluated using the evaluation coefficient.

[0077] The expression of the evaluation coefficient is: .in, is the evaluation coefficient, To predict the ECG signal, is the ground truth ECG signal, is the average value of the ground truth ECG signal, is the predicted average value of the ECG signal, is the standard deviation of the ground truth ECG signal, is the standard deviation of the predicted ECG signal.

[0078] The mean absolute error is used to calculate the distance between the predicted ECG signal and the ground truth ECG signal in the frequency domain.

[0079] The expression of mean absolute error is: .in, is the mean absolute error, To predict the ECG signal, is the ground truth ECG signal, is the fast Fourier transform, is the total number of frequency domain data points, is the frequency domain data point index, h is the function used to calculate the mean absolute error, Indicates the absolute value operation.

[0080] The loss function is obtained based on the evaluation coefficient and the mean absolute error.

[0081] The loss function is: .in, is the total loss, is the evaluation coefficient, is the mean absolute error.

[0082] S103: Designing and simulating a millimeter-wave frequency-modulated continuous-wave radar by using millimeter-wave design software in a radar control module, and configuring millimeter-wave parameters of the millimeter-wave frequency-modulated continuous-wave radar.

[0083] Specifically, the millimeter wave design software used in this application is mmWave Studio.

[0084] Furthermore, the present application also provides a function for measuring heart rate (HR) and intercardiac beat interval (IBI). The following is a detailed description of this function.

[0085] Predicted ECG signal Perform frequency domain transformation to analyze its frequency components. Calculate the data after frequency domain transformation The power spectral density (PSD) of The noise that may exist in the signal will produce high-frequency components in the power spectrum density. In order to reduce the impact of noise on heart rate calculation, a low-pass Butterworth filter is used to filter the signal. The design parameters of the low-pass Butterworth filter include the order of the filter and the normalized cutoff frequency , these parameters are set according to the characteristics of PSD and the required filtering effect. The implementation formula of the low-pass Butterworth filter is: .in, is the order of the filter, is the normalized cutoff frequency, is the power spectral density, is the signal after filtering.

[0086] According to the filtered signal , find the maximum peak position in its spectrum. This peak position corresponds to The main frequency component of the signal, that is, the frequency corresponding to the heart rate. The calculation formula for heart rate is: .in, is the maximum value in the spectrum of the filtered signal, is the time constant, For heart rate.

[0087] exist In the signal, detect the R peak (i.e., the peak value of the QRS complex in the electrocardiogram). The R peak is the key point for calculating the IBI. According to the detected R peak position, calculate the time interval between adjacent R peaks, i.e., the cardiac interval (IBI). The formula for calculating the cardiac interval is: .in, is the difference between adjacent R peak positions, is the sampling frequency, The cardiac interval.

[0088] Figure 3 A waveform diagram of performing time domain smoothing and frequency domain processing on a single channel data in a radar waveform provided in an embodiment of the present application. Figure 3 (a) in the figure is the thunder waveform before time domain smoothing and frequency domain processing. Figure 3 (b) is the radar waveform after time domain smoothing and frequency domain processing. By comparison, it can be clearly seen that Figure 3 The main component peak in (b) is clearer and the noise is effectively filtered out. Figure 3 Before Smooth means before time domain smoothing and frequency domain processing are performed, Smooth means after time domain smoothing and frequency domain processing are performed, Range means that the x-axis represents the data range, Speed ​​means that the z-axis represents the data speed, and Amplitude means that the y-axis represents the data amplitude.

[0089] Figure 4 A waveform diagram of two-dimensional Fourier transform and spatial channel superposition provided in an embodiment of the present application. Figure 4 (a) in the figure is the result after two-dimensional Fourier transform. Figure 4 (b) in the figure is the result after the spatial channels are superimposed. The improved signal is first subjected to a two-dimensional Fourier transform, and then the signal is amplified in the frequency domain. The visualization effect produced by this processing flow is shown in the following figure on a single channel. Figure 4 As shown. During the data acquisition process, since the human body is in a quasi-static state (whether sitting or lying), most of the peaks are concentrated on the zero-velocity plane. The height of these peaks is directly related to the energy intensity of the radar waveform, indicating that there is a high probability that there are objects at these locations. Among them, the improved signal is the zero-frequency component signal mentioned above. Figure 4 The FFT in the figure represents two-dimensional Fourier transform, and the Channel Stack represents spatial channel superposition.

[0090] Figure 5 A schematic diagram of using potential local maximum points as potential target positions during target detection provided by an embodiment of the present application. Figure 5 Specifically, a target detection result matrix is ​​constructed based on the judgment results, a peak search algorithm is executed on the target detection result matrix to identify potential local maximum points, and the potential local maximum points are used as potential target positions. This is a schematic diagram of the process. Figure 5(a) The possible target position after target detection in the figure contains a significant main peak and multiple sub-peaks surrounding it. Although these sub-peaks have lower energy, they may still be identified as potential targets during the target detection process, thereby introducing unnecessary noise or misjudgment. Figure 5 (b) in the figure is the local maximum point determined by the peak search algorithm. The peak search algorithm is an effective signal processing technology that can accurately identify the peak point with the highest energy, that is, the main peak, in a complex signal environment. In this way, sub-peaks with lower energy and less impact on the experimental results can be effectively ignored, thereby improving the accuracy and reliability of target detection. Figure 5 TargetDetection is target detection, and Peak Search is peak search.

[0091] Figure 6 A schematic diagram of performing second-order difference processing on signals in a filtered target position set provided in an embodiment of the present application. Figure 6 (a) in FIG. 1 is the result signal obtained after the second-order difference processing of the signal in the filtered target position set. The result signal is the cardiac mechanical motion signal and the chest motion signal separated from the channel data. Figure 6 (b) in the figure shows the process of filtering the resulting signal using an FIR filter. Figure 6 (c) in the figure is the result of filtering the result signal using the FIR filter. After being processed by the FIR filter, the pure cardiac mechanical motion signal is successfully separated from the cardiac mechanical motion signal and the chest motion signal. Figure 6 Target Data is the result signal, Diference is the difference, CMM is the cardiac mechanical motion signal, and Signal Index is the signal index.

[0092] Figure 7 A schematic diagram of a signal reconstruction network based on time-frequency domain transformation provided in an embodiment of the present application being trained under the supervision of a ground truth electrocardiogram signal. Figure 7 (a) shows the cardiac mechanical motion signal CMM (indicated by the green line) and the predicted ECG signal ( , represented by the blue line) and the ground truth ECG signal ( , represented by the red line) in the time domain. By observing, we can find The signal is not only at key peaks such as PQRST The signals are highly similar and show amazing consistency in the overall waveform trend. It is particularly worth mentioning that at the R peak position, The signal showed significant synchronization with the cardiac mechanical motion signal CMM, which further verified the accuracy and reliability of the signal reconstruction network based on time-frequency domain transformation in ECG reconstruction. Figure 7 (b) in the figure is the power spectrum density diagram related to the above signal. Through this diagram, important information such as heart rate (HR) and inter-cardiac interval (IBI) can be easily obtained from the RR interval. In this example, the heart rate of the sample is determined to be 75 beats per minute (bpm), and the inter-cardiac interval is 0.8ms. The RR interval is the difference between the horizontal coordinates of each peak of the chain. Figure 7 The horizontal axis is frequency and the vertical axis is power spectrum density.

[0093] Some modules in the apparatus described in the present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0094] The devices or modules described in the above application embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described in various modules according to their functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, the module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0095] The methods, devices or modules described in this application can be implemented in the form of computer-readable program codes. The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program codes (such as software or firmware) that can be executed by the (micro)processor, logic gates, switches, application-specific integrated circuits (English: Application Specific Integrated Circuit, referred to as: ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program codes, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included in it for implementing various functions can also be regarded as structures within the hardware component. Or even, the means for realizing various functions may be regarded as both a software module for realizing the method and a structure within a hardware component.

[0096] like Figure 8 As shown, an embodiment of the present application also provides a non-contact ECG signal measurement server, including a memory 801 and a processor 802; the memory 801 is used to store computer executable instructions; the processor 802 is used to execute computer executable instructions to implement a measurement method of a non-contact ECG signal measurement device described above in an embodiment of the present application.

[0097] The embodiment of the present application also provides a computer-readable storage medium, which stores executable instructions. When a computer executes the executable instructions, it can implement the measurement method of the non-contact electrocardiogram signal measurement device described above in the embodiment of the present application.

[0098] It can be seen from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, or it can be reflected in the implementation process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the method described in the embodiment of the present application.

[0099] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of this application can be used in many general or special computer system environments or configurations.

[0100] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.

Claims

1. A non-contact electrocardiogram signal measuring device, characterized in that: It includes a multi-domain joint modulation micro-motion signal disentanglement module and a signal reconstruction network based on time-frequency domain transformation; The multi-domain joint modulation micro-motion signal de-entanglement module is configured to receive a radar waveform obtained from the millimeter-wave frequency-modulated continuous wave radar for electrocardiographic signal acquisition, and extract the cardiac mechanical motion signal from the radar waveform; The signal reconstruction network based on time-frequency domain transformation is configured to receive the extracted cardiac mechanical motion signal, learn and construct the nonlinear relationship between the cardiac mechanical motion signal and the electrocardiogram signal, so as to output a predicted electrocardiogram signal according to the cardiac mechanical motion signal.

2. The non-contact ECG signal measuring device according to claim 1, characterized in that: Also included is the radar control module; The radar control module is configured to design and simulate the millimeter-wave frequency modulation continuous wave radar using millimeter-wave design software, and configure parameters of the millimeter-wave frequency modulation continuous wave radar.

3. The measurement method of the non-contact ECG signal measurement device according to any one of claims 1 to 2, characterized in that: include: The multi-domain joint modulation micro-motion signal de-entanglement module receives the radar waveform obtained from the millimeter-wave frequency-modulated continuous wave radar for electrocardiographic signal collection, and extracts the cardiac mechanical motion signal from the radar waveform; Receiving the extracted cardiac mechanical motion signal in a signal reconstruction network based on time-frequency domain transformation, learning and constructing a nonlinear relationship between the cardiac mechanical motion signal and the electrocardiogram signal, so as to output a predicted electrocardiogram signal according to the cardiac mechanical motion signal; In the radar control module, the millimeter wave design software is used to design and simulate the millimeter wave frequency modulated continuous wave radar, and the millimeter wave parameters of the millimeter wave frequency modulated continuous wave radar are configured.

4. The measurement method of the non-contact ECG signal measurement device according to claim 3, characterized in that: The step of extracting the cardiac mechanical motion signal from the radar waveform comprises: Perform time-frequency domain data smoothing on each channel data in the radar waveform to obtain a zero-frequency component signal; Perform spatial channel superposition on the zero-frequency component signal to obtain a signal matrix after spatial channel superposition; Performing target selection on the signal matrix after spatial channel superposition to obtain a screened target position set; The filtered target position set is subjected to time-frequency filtering to extract the cardiac mechanical motion signal.

5. The measurement method of the non-contact ECG signal measurement device according to claim 4, characterized in that: The step of performing time-frequency domain data smoothing processing on each channel data in the radar waveform to obtain a zero-frequency component signal includes: The time-frequency domain data smoothing process is time-domain smoothing processing and frequency-domain processing; Each channel data in the radar waveform includes cardiac mechanical motion signal, chest motion signal and noise signal; The time domain smoothing process includes: performing a multiplication operation on the channel data and the Hanning window to generate a first smoothed signal; The frequency domain processing includes: performing Fourier transform on the first smoothed signal along the time dimension to convert it into the frequency domain to obtain a frequency domain signal; Multiplying the frequency domain signal with another Hanning window to generate a second smoothed signal; Performing Fourier transform on the second smoothed signal according to the characteristic dimension to generate a characteristic frequency domain signal; A two-dimensional second-order Fourier transform is used to process each signal channel in the characteristic frequency domain signal to obtain a processed signal, and a fast Fourier transform shift function is used to move the zero-frequency component of the processed signal to the center of the spectrum to obtain a zero-frequency component signal.

6. The measurement method of the non-contact ECG signal measurement device according to claim 5, characterized in that: The performing spatial channel superposition on the zero-frequency component signal to obtain a signal matrix after spatial channel superposition includes: The expression of spatial channel superposition is: ;in, is the signal matrix after the spatial channels are superimposed, is the channel index, is the total channel data, is a zero frequency component signal, Indicates the absolute value operation.

7. The measurement method of the non-contact ECG signal measurement device according to claim 6, characterized in that: The target selection is performed on the signal matrix after the spatial channels are superimposed to obtain a screened target position set, including: For the head and tail of the signal matrix after the spatial channels are superimposed, two signal clips of the same length are respectively cut; Using a concatenation operation, two signal clips of the same length are concatenated with the signal matrix after the spatial channels are superimposed to generate an extended signal matrix; Traverse each sampling point in the extended signal matrix to obtain the background clutter power of each sampling point; Taking the average value of the background clutter power of the sampling points within the preset range as the average background clutter power; Calculate the difference between the signal value of the sampling point within the preset range and the average background clutter power; Determine whether the difference is greater than a preset threshold; If the difference is greater than the preset threshold, the result is that there is a target at the location of the sampling point; If the difference is less than or equal to the preset threshold, the judgment result is that there is no target at the location of the sampling point; A target detection result matrix is ​​constructed based on the judgment results, and a peak search algorithm is performed on the target detection result matrix to identify potential local maximum points, and the potential local maximum points are used as potential target positions; wherein each element in the target detection result matrix marks whether there is a target at the corresponding sampling point; According to the preset maximum and minimum distance ranges of the millimeter wave frequency modulated continuous wave radar to the human body, potential target positions are screened and target positions within the distance range are retained; Output the filtered set of target locations.

8. The measurement method of the non-contact ECG signal measurement device according to claim 7, characterized in that: The step of performing a time-frequency filtering operation on the filtered target position set to extract the cardiac mechanical motion signal comprises: Performing second-order difference processing on the signals in the filtered target position set to reduce low-frequency interference; The expression of second-order difference processing is: ;in, is the result signal obtained after performing second-order difference processing on the signals in the filtered target position set. is the second signal in the filtered target position set, is the first signal in the filtered target position set, is the last signal in the filtered target position set, The signal before the last signal in the filtered set of target positions, for function; Create FIR filter; The expression of FIR filter is: ;in, To create a good FIR filter, is the length of the filter, is the passband boundary frequency range, , , is the sampling frequency, is the upper limit of the heartbeat amplitude, is the lower limit of the heartbeat amplitude, is the Kaiser window function, is the parameter of Kaiser window, Perform filtering operation for FIR filter; The resulting signal is filtered using a FIR filter to extract the cardiac mechanical motion signal.

9. The measurement method of the non-contact ECG signal measurement device according to claim 3, characterized in that: The method of receiving the extracted cardiac mechanical motion signal in a signal reconstruction network based on time-frequency domain transformation, learning and constructing a nonlinear relationship between the cardiac mechanical motion signal and the electrocardiogram signal, and outputting a predicted electrocardiogram signal according to the cardiac mechanical motion signal, comprises: The signal reconstruction network based on time-frequency domain transformation receives the cardiac mechanical motion signal as input; the signal reconstruction network based on time-frequency domain transformation includes a plurality of multi-channel time-frequency fusion extraction modules; Use convolutional neural network to perform preliminary feature extraction of cardiac mechanical motion signals; After being normalized and processed by activation functions, the extracted features are input into multiple multi-channel time-frequency fusion extraction modules for nonlinear feature mapping, and the ECG signal is used as the target signal to learn and construct the nonlinear relationship between the cardiac mechanical motion signal and the ECG signal; wherein the multi-channel time-frequency fusion extraction module includes a residual structure, which includes a convolutional neural network, batch normalization and an activation function; The ECG signal predicted by the signal reconstruction network output based on time-frequency domain transformation; Use supervised learning method to train the signal reconstruction network based on time-frequency domain transformation, and optimize its parameters by minimizing the loss function; The similarity between the predicted ECG signal and the ground truth ECG signal in the time domain is evaluated using the evaluation coefficient; The expression of the evaluation coefficient is: ;in, is the evaluation coefficient, To predict the ECG signal, is the ground truth ECG signal, is the average value of the ground truth ECG signal, is the predicted average value of the ECG signal, is the standard deviation of the ground truth ECG signal, is the standard deviation of the predicted ECG signal; The distance between the predicted ECG signal and the ground truth ECG signal in the frequency domain is calculated using the mean absolute error; The expression of mean absolute error is: ;in, is the mean absolute error, To predict the ECG signal, is the ground truth ECG signal, is the fast Fourier transform, is the total number of frequency domain data points, is the frequency domain data point index, h is the function used to calculate the mean absolute error, Indicates the absolute value operation; Obtain the loss function based on the evaluation coefficient and the mean absolute error; The loss function is: ;in, is the total loss, is the evaluation coefficient, is the mean absolute error.

Images