A heart rate variability feature analysis method based on deep learning

Abstract

A deep learning-based heart rate variability feature analysis method relates to the computer field and can effectively extract information contained in heart rate variability. The method includes the following steps: acquiring electrocardiogram (ECG) data via a sensor and transmitting it to a host computer; preprocessing the ECG signal using interpolation and filtering; intercepting the received ECG data using a sliding window, performing real-time updates and Fourier transforms to obtain a spectral feature graph; and parsing the spectral graph and extracting useful information using a deep convolutional neural network. This includes frequency domain feature extraction using a residual learning method based on ResNet18, time-frequency domain feature fusion based on an attention mechanism, and classification and output of feature information using a multilayer perceptron.

Description

Technical Field

[0001] The present invention belongs to the field of electronic information and is a heart rate variability feature analysis method based on deep learning and neural networks. Background Art

[0002] Heart rate variability (HRV) is an indicator of heart rhythm. By analyzing HRV, the activity of the nervous system can be evaluated.

[0003] The existing mainstream HRV analysis methods are divided into visual analysis and statistical analysis methods. Currently, HRV analysis technology and applications based on deep learning and neural networks are still in their infancy.

[0004] In recent years, deep learning algorithms have achieved remarkable results in fields such as image recognition and natural language processing. In image recognition, deep learning algorithms, using techniques such as convolutional neural networks, can more accurately extract and analyze features in images. Therefore, deep learning algorithms offer new insights into HRV analysis, potentially improving its accuracy.

[0005] The present invention combines ResNet, attention mechanism and multi-layer perceptron technologies to construct a neural network, and uses the electrocardiogram (ECG) signal obtained by the sensor as input to perform real-time heart rate variability feature analysis. Summary of the Invention

[0006] In order to solve the problems of HRV analysis methods based on traditional statistical means being susceptible to external interference and having poor flexibility, the present invention combines ResNet, attention mechanism and multi-layer perceptron technology to construct an HRV analysis neural network to realize heart rate variability feature monitoring.

[0007] The present invention relates to the following four aspects:

[0008] (1) The sensor obtains the subject's ECG data and transmits it to the host computer via low-power Bluetooth;

[0009] (2) Preprocess the obtained ECG signal using interpolation, filtering and other means to improve signal accuracy and eliminate interference spikes;

[0010] (3) A sliding window is used to cache the received ECG signal, and the ECG data in the sliding window is updated in real time according to the first-in-first-out principle; at the same time, a Fourier transform is performed on the ECG data in the sliding window to obtain a spectrum feature map;

[0011] (4) Apply Resnet, attention mechanism, and multi-layer perceptron technologies to construct a deep convolutional neural network to analyze ECG data in real time and try to extract useful information from it.

[0012] The core algorithm of the present invention:

[0013] (1) ECG signal preprocessing based on cubic spline interpolation and filter

[0014] Typically, ECG data collected from sensors are easily affected by factors such as breathing and movement, resulting in missing data points or noise, which affects subsequent ECG analysis.

[0015] In order to accurately extract the characteristic vector of heart rate variability, the present invention first uses cubic spline interpolation to upsample the original discrete ECG data to improve data accuracy.

[0016] Then, a high-pass Butterworth filter is applied to the sample to reduce the potential DC component offset and baseline drift of 0.15-0.3Hz in the signal. The Butterworth filter, also known as the maximally flat filter, was first proposed by British engineer and physicist Stephen Butterworth in 1930. It has the characteristics of a flat and ripple-free frequency response curve in the passband and gradually decreases to zero in the stopband. It is very suitable for filtering baseline drift in ECG signals. The transfer function of the high-pass Butterworth filter is shown in Equation (1):

[0017]

[0018] Where n is the filter order, D(u,v) represents the distance from the frequency midpoint to the frequency plane, and D0 is the cutoff frequency. When D(u,v) is greater than D0, the corresponding H(u,v) gradually approaches 1, allowing high frequencies to pass through. When D(u,v) is less than D0, H(u,v) gradually approaches 0, filtering out low frequencies.

[0019] Finally, the data is smoothed using a Savitzky-Golay filter, a filtering method based on local polynomial least squares fitting in the time domain. The key advantage of this filter is that it removes noise while preserving the shape and width of the signal, thus maintaining temporal accuracy.

[0020] (2) Dynamic analysis of ECG signal characteristics in time and frequency domains based on sliding windows

[0021] Since ECG data is continuous streaming data input continuously by sensors, it has the characteristics of continuity and long-term. Traditional static data analysis methods cannot be directly applied to this scenario. Therefore, the present invention uses sliding window technology to preprocess the raw ECG data.

[0022] Specifically, the data is divided into windows of adjustable lengths of 5, 10, or 20 seconds. The so-called "window" is a segment of the original signal from a certain time point to a certain time point, such as Figure 2As shown in the rectangular box in the figure, peak detection and feature analysis are then performed independently on the data within each window. A timestamp is set for each window, and the time and frequency domain feature analysis results are saved. As new data is continuously input, the sliding window is advanced, and the ECG data and its time and frequency domain features within the sliding window are updated in real time according to the first-in, first-out principle.

[0023] (3) Signal power spectrum feature extraction based on FFT and PSD

[0024] The frequency characteristics of the signal are calculated using the Fast Fourier Transform FFT algorithm. For a discrete data point sequence u with a length of N, N (n), the formula of FFT algorithm is shown in formula (2):

[0025]

[0026] Where j is the imaginary unit, ω is the frequency, n is the number of frequency points, U n (ω) is the result of Fourier transform of ECG signal, which is a complex sequence with the same length as the input sequence, including the real part, i.e. the amplitude of each frequency component, and the imaginary part, i.e. the phase of each frequency component. Then, the modulus of (2) is squared to obtain the estimated power spectral density (PSD) function. Among them U n (ω) is the Fourier transform result sequence of the ECG signal obtained in (2), N is the sequence length, ω is the frequency, and n is the number of frequency points:

[0027]

[0028] By plotting the power spectrum density curve, we can obtain the three-channel RGB image of the frequency domain characteristics of the heartbeat interval in the sliding window; Figure 3 As shown, it is convenient for subsequent neural network processing.

[0029] (4) Analysis of Heart Rate Variability Characteristics Based on Neural Network

[0030] The neural network architecture used in the present invention is as follows Figure 4 shown.

[0031] First, the RGB image generated in step (3) is input into the Resnet layer. The Resnet layer consists of multiple convolutional blocks and residual blocks, which are used to extract the frequency domain features of multiple images. Then, the fusion layer uses the attention mechanism to concatenate and fuse the frequency domain feature vector with the time domain feature vector. Finally, a fully connected layer (also known as a multi-layer perceptron) and a softmax layer are used to classify the feature vector and select the category with the highest probability as the inference output.

[0032] (5) Frequency domain feature extraction based on ResNet18 residual learning method

[0033] As the network depth increases, the neural network will have the problem of gradient disappearance, resulting in a decrease in the accuracy of network detection. ResNet introduces the residual learning method to eliminate the gradient disappearance problem caused by the increase in the depth of the neural network. Each residual module contains two paths, such as Figure 5 As shown in Figure 1, one path directly passes the input feature; the other path performs several convolution operations on the feature to obtain the residual feature F(x). Weights are assigned to the two paths. Ultimately, the sum of the results of the two paths serves as the output of the current network layer and the input of the next network layer. The addition of the residual module can improve the model's fitting ability and mitigate the negative impact of increasing the number of network layers.

[0034] The structure of Resnet18 is as follows Figure 6 As shown in Figure 2, the input layer consists of a convolutional layer and a maximum pooling layer. The middle layer contains four convolutional blocks containing the aforementioned residual modules, represented by dotted boxes. Finally, the feature values ​​are output through an average pooling layer.

[0035] The image generated after PSD transformation (RGB three-channel image with a resolution of 224*224) is used as the input of the above network. It is first convolved with a convolution kernel of size 7*7 and stride 2, and then max pooled with a kernel of size 3*3 and stride 2. After the above processing, the image will become a 56*56 feature map.

[0036] The middle convolutional layer, consisting of four convolutional blocks, is the core component of ResNet18. It extracts features from the original data while maintaining the spatial continuity of the image. Furthermore, these convolution kernels can share parameters, effectively reducing the number of parameters in the convolutional layer.

[0037] The first convolutional layer has a kernel size of 3*3, a stride of 2, a padding of 1, and 64 output channels, resulting in an output of 64*112*112. The second convolutional layer has a kernel size of 3*3, a stride of 1, and a padding of 1, resulting in an output of 128*56*56. The final two convolutional layers have a kernel size of 1*1 and undergo downsampling. Each layer has a stride of 2 and a padding of 1, doubling the output channels and halving the output vector size. The output vector is 512*7*7. Finally, this vector passes through an average pooling layer, outputting a 512*1*1 frequency-domain feature vector, which serves as the input for subsequent time-frequency feature fusion.

[0038] (6) Time-frequency domain feature fusion based on attention mechanism

[0039] The frequency domain feature vector generated by Resnet18 and the calculated time domain feature vector are used as the input of the attention mechanism. Then the correlation between the vector q and each input vector is calculated by the scoring function f(x):

[0040]

[0041] Where x is the query, x i is the key, y i For the value corresponding to the key, "query x and x i The attention weights between ∝(x,x i ) means that if a key x i The closer to a given query x, the more likely the value y is assigned to this key. i The function will produce a score, the higher the score, the higher the weight.

[0042] Then, the concat function is used to perform weighted concatenation of the feature vectors. Finally, the classification results are output through the fully connected layer and the softmax layer.

[0043] Effects of the Invention

[0044] Experiments have shown that the above signal preprocessing process and the model that integrates residual network, attention mechanism and multi-layer perceptron based on HRV time-frequency domain features can achieve an accuracy rate of 92% in identifying the effective information contained therein. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Cubic spline interpolation example

[0046] Figure 2 Sliding Window

[0047] Figure 3 Power spectral density curve of three-channel RGB image

[0048] Figure 4 Neural Network Architecture

[0049] Figure 5 Residual Network

[0050] Figure 6 Resnet18 architecture DETAILED DESCRIPTION

[0051] The specific implementation of the present invention is divided into 3 parts:

[0052] (1) Training of neural network models

[0053] This paper utilizes a variety of open-source ECG signal datasets, inputting continuous ECG waveforms and outputting custom classification labels. All training was performed on a graphics workstation with an E5-2620 CPU, 128GB of memory, and a Tesla M40 graphics accelerator.

[0054] (2) Development of host computer software

[0055] The host computer software is written in Python 3.9. Python supports the development of graphical user interface window programs through the built-in Tkinter library and waveform display through the Matplotlib library. It also provides extensive library support for serial communication, Bluetooth communication, and deep learning. Python is highly object-oriented, flexible, and has concise syntax, making it easy to deploy.

[0056] (3) Bluetooth communication between the sensor and the host PC

[0057] The sensor uses the Nuanxinjia EPCM-001 module, which integrates a RISC-V microcontroller, BLE Bluetooth, ECG patch electrode interface, and finger photoplethysmography sensor, and can be easily connected to a computer in a wired or wireless manner. In order to facilitate the subjects to wear the device and move around, the present invention uses a wireless communication method based on BLE Bluetooth to interact with the host computer program. Usually, laptop computers have built-in Bluetooth receivers. If it is a desktop computer, you may need to use a receiver that can receive Bluetooth signals and install the corresponding drivers and software to communicate with the sensor.

Claims

1. A heart rate variability feature analysis method based on deep learning, characterized by: (1) The sensor obtains the subject's ECG data and transmits it to the host computer via low-power Bluetooth; (2) Apply interpolation and filtering to preprocess the obtained ECG signal to improve the signal accuracy and eliminate interference spikes; (3) A sliding window is used to cache the received ECG signal, and the ECG data in the sliding window is updated in real time according to the first-in-first-out principle; at the same time, a Fourier transform is performed on the ECG data in the sliding window to obtain a spectrum feature map; (4) Apply Resnet, attention mechanism and multi-layer perceptron to construct a deep convolutional neural network to analyze ECG data in real time and extract useful information from it; (1) ECG signal preprocessing based on cubic spline interpolation and filter Firstly, cubic spline interpolation is used to upsample the original discrete ECG data to improve data accuracy; Then, a high-pass Butterworth filter is applied to the sample to reduce the potential DC component offset and 0.15-0.3 Hz baseline drift in the signal; the transfer function of the high-pass Butterworth filter is shown in formula (1): Where n is the order of the filter, D(u,v) represents the distance from the frequency midpoint to the frequency plane, and D0 is the cutoff frequency. When D(u,v) is greater than D0, the corresponding H(u,v) gradually approaches 1, allowing the high-frequency part to pass through. When D(u,v) is less than D0, H(u,v) gradually approaches 0, filtering the low-frequency part and smoothing the data. (2) Dynamic analysis of ECG signal characteristics in time and frequency domains based on sliding windows The data is divided into windows of adjustable lengths of 5, 10, or 20 seconds. A "window" is a segment of the original signal from a certain time point to a certain time point. Peak detection and feature analysis are then performed independently on the data within each window. A timestamp is set for each window, and the time and frequency domain feature analysis results are saved. As new data is continuously input, the sliding window is advanced, and the ECG data and its time and frequency domain features within the sliding window are updated in real time according to the first-in-first-out principle; (3) Signal power spectrum feature extraction based on FFT and PSD The frequency characteristics of the signal are calculated using the Fast Fourier Transform FFT algorithm. For a discrete data point sequence u with a length of N, N (n), the formula of FFT algorithm is shown in formula (2): Where j is the imaginary unit, ω is the frequency, n is the number of frequency points, U n (ω) is the result of Fourier transform of ECG signal, which is a complex sequence with the same length as the input sequence, including the real part, i.e. the amplitude of each frequency component, and the imaginary part, i.e. the phase of each frequency component. Then, the modulus of (2) is squared to obtain the estimated power spectral density (PSD) function. Among them U n (ω) is the Fourier transform result sequence of the ECG signal obtained in (2), N is the sequence length, ω is the frequency, and n is the number of frequency points: The power spectrum density curve is plotted to obtain a three-channel RGB image of the frequency domain characteristics of the heartbeat interval in the sliding window; (4) Analysis of Heart Rate Variability Characteristics Based on Neural Network The RGB image generated in step (3) is input to the Resnet layer; the Resnet layer consists of multiple convolutional blocks and residual blocks, which are used to extract the frequency domain features of multiple images; then, the fusion layer uses the attention mechanism to splice and fuse the frequency domain feature vector with the time domain feature vector; finally, the fully connected layer, also known as the multi-layer perceptron and the softmax layer, is used to classify the feature vector and select the category with the highest probability as the inference result output; (5) Frequency domain feature extraction based on ResNet18 residual learning method As the network depth increases, the neural network will experience the problem of vanishing gradients, resulting in a decrease in the accuracy of network detection. ResNet introduces residual learning methods to eliminate the gradient vanishing problem caused by the increase in neural network depth. Each residual module contains two paths, one of which is a direct path for input features; the other path performs several convolution operations on the feature to obtain the residual feature F(x), and weights are assigned to the two paths. Finally, the sum of the results of the two paths is used as the output of the current layer network and the input of the next layer network; Resnet18; the input layer consists of a convolutional layer and a maximum pooling layer; the middle layer contains 4 convolutional blocks containing the above residual modules, and the feature values ​​are output through an average pooling layer; The image generated after PSD transformation, that is, the RGB three-channel image with a resolution of 224*224, is used as the input of the above network. It is first convolved with a convolution kernel of size 7*7 and stride 2, and then max pooled with a kernel of size 3*3 and stride 2. After the above processing, the image will become a 56*56 feature map. The middle convolution layer has four convolution blocks. The convolution kernel size of the first convolution layer is 3*3, the stride is 2, the padding is 1, the output channel is 64, and the output of this layer is 64*112*112; the convolution kernel size of the second convolution layer is 3*3, the stride is 1, the padding is 1, and the output of this layer is 128*56*56; the kernel size of the last two convolution layers is 1*1, and they undergo a downsampling; the stride of each layer is 2, the padding is 1, the output channel of each layer is doubled, and the output vector size is halved; the size of the output vector is 512*7*7; finally, the vector passes through an average pooling layer to output a 512*1*1 frequency domain feature vector, which serves as the input for subsequent time-frequency domain feature fusion; (6) Time-frequency domain feature fusion based on attention mechanism The frequency domain feature vector generated by Resnet18 and the calculated time domain feature vector are used as the input of the attention mechanism. Then the correlation between the vector q and each input vector is calculated by the scoring function f(x): Where x is the query, x i is the key, y i For the value corresponding to the key, "query x and x i The attention weights between ∝(x,x i ) represents; then the concat function is used to perform weighted concatenation of the feature vectors; finally, the classification results are output through the fully connected layer and the softmax layer.

Images