A deep learning method for arrhythmia classification based on two-dimensional images

Abstract

The present invention relates to a deep learning arrhythmia classification method based on two-dimensional images, and belongs to the technical field of medical signal processing. The method comprises: classifying one-dimensional original electrocardiogram signals into abnormal intervals and abnormal amplitudes; performing two-dimensional imaging processing on one-dimensional electrocardiogram signals with abnormal intervals and abnormal amplitudes; dividing the obtained two-dimensional image set into a training set and a test set in a ratio of 7:3, with no crossover data between them; using transfer learning to transfer the relevant parameters of the trained and adjusted VGG16 network model to a new data set for training; and outputting the arrhythmia classification result corresponding to the current target electrocardiogram signal. The present invention can improve the classification accuracy of arrhythmia.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical signal processing and relates to a deep learning arrhythmia classification method based on two-dimensional images. Background Art

[0002] Patients with heart disease will have abnormal heartbeats. Changes in the rhythm rate of the heartbeats will cause abnormal ECG fluctuations, which is called dysrhythmia. Timely detection and diagnosis of arrhythmias can effectively prevent the occurrence of malignant heart diseases. Analysis of ECG signals is of great significance in the diagnosis and prevention of heart problems. There are many types of abnormal heart rhythms, and the collection of ECG signals is easily affected by multiple factors, resulting in large differences in ECGs. At the same time, the differences are subtle and difficult to observe. Therefore, a convenient, accurate, efficient and real-time arrhythmia classification and detection method is extremely important.

[0003] The feature extraction method of ECG signal can be realized based on time domain and frequency domain. The time domain method uses the amplitude, time interval, width and other time domain features of ECG signal to classify and identify ECG signal. Rahhal et al. proposed a classification method of ECG signal based on deep learning, using autoencoder to learn appropriate feature representation in an unsupervised manner, and then adding softmax regression layer, thus combining the method of deep neural network. Experiments show that a higher accuracy rate can be obtained. Some scholars proposed a new deep learning method for 17 types of arrhythmia detection based on long-term ECG signal analysis, and designed a new 1D-CNN using 10s ECG segments to realize the recognition of 17 types of arrhythmia diseases, with an accuracy rate of 91.33%. P.Bhardwaj et al. used support vector machine technology to classify five types of arrhythmias of ECG signals, combined with the time domain features of ECG signals for classification, and the recognition rate can reach 95.21%. Ganeshkumar R et al. proposed a frequency domain feature extraction method based on discrete cosine transform (DCT), two different arrhythmia conditions and normal ECG Figure 1 The average classification accuracy of this method reached 90%. Salem M et al. used two-dimensional feature extraction and SVM classifier to classify four types of arrhythmias, and the classification accuracy of this method reached 89.04%.

[0004] Although many scholars have used different methods to classify ECG arrhythmias, they all have one or more of the following limitations: 1) the beat frequency of ECG is lost in the noise filtering and feature extraction schemes; 2) the number of ECG arrhythmia types used for classification is limited; 3) the classification performance adopted in practical applications is relatively low.

[0005] The traditional method is to extract the feature information of time domain and frequency domain separately, and to analyze the frequency domain information with obvious features in combination with the time domain to obtain different abnormal types of ECG signals. ECG signals are a kind of time domain signals, but the research on ECG signals does not only stop at time domain analysis, but also needs to be analyzed and processed in the frequency domain. Combining time domain features and frequency domain features is of great significance for the classification and recognition of ECG signals. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a deep learning arrhythmia classification method based on two-dimensional images, which converts one-dimensional electrocardiogram signals into two-dimensional images and then combines them with deep learning convolutional neural networks for feature extraction, thereby improving the classification accuracy of arrhythmias.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A deep learning arrhythmia classification method based on two-dimensional images comprises the following steps:

[0009] S1: Classify the abnormal interval and amplitude of the one-dimensional original ECG signal;

[0010] S2: Perform two-dimensional imaging processing on the one-dimensional ECG signals with abnormal intervals and amplitudes to obtain a two-dimensional image set;

[0011] S3: Divide the obtained two-dimensional image set into a training set and a test set in a ratio of 7:3;

[0012] S4: Use transfer learning to transfer the relevant parameters of the trained and adjusted VGG16 network model to the new dataset for training;

[0013] S5: Output the arrhythmia classification result corresponding to the current target ECG signal.

[0014] Further, in step S2, obtaining a two-dimensional image set specifically includes:

[0015] (1) For abnormal intervals: extract 3000 points of ECG signals of about 10 seconds and divide them into 5 parts, which are arranged from top to bottom. The horizontal axis is the 600 points of each signal segment, which is also the corresponding time change. The color change is used to represent the change of amplitude to obtain a color two-dimensional image.

[0016] (2) For abnormal amplitude: obtain the ECG waveform of each cycle based on the position of the R wave of the one-dimensional signal, perform Fourier transform on it, combine the transformed signal with time, and finally plot the waveform of 10 consecutive cycles into a two-dimensional time-frequency image, with the horizontal axis representing the change of time and the vertical axis representing the time-frequency of frequency.

[0017] Further, in step S2, obtaining a two-dimensional image set specifically includes:

[0018] (1) For abnormal intervals: the entire set of data is divided into two categories, one is data that is marked as normal for all 10 seconds, and the other is data with abnormal intervals within 10 seconds;

[0019] (2) For abnormal amplitude: resize all images to 224*224, call the split() function to split the original RGBA image format into four channels, and then call image.merge() under PIL to convert it into an appropriate RGB image and save it.

[0020] Further, in step S4, the adjusted VGG16 network model includes: an input layer, a convolutional layer, a maximum pooling layer, a fully connected layer and a Softmax classifier connected in sequence;

[0021] The convolution layer is used to extract the representative features of the two-dimensional image set and complete the two-dimensional convolution feature extraction of the sample. The convolution kernel size is 3*3;

[0022] The maximum pooling layer is used to reduce the number of features of the convolutional layer and thus reduce the operation parameters, reduce the sensitivity of the convolutional layer to position, and control overfitting;

[0023] The fully connected layer integrates multi-channel data to reduce the impact of feature position on classification;

[0024] Softmax classifier, classification based on labels.

[0025] Furthermore, in step S4, the calculation formula of the convolutional layer is:

[0026]

[0027] Among them, (i, j) represents the coordinates of the center pixel, m represents the width, n represents the height, g represents the output image of the convolution layer, f represents the original image, h represents the convolution kernel, and * is the convolution operator.

[0028] Furthermore, in step S4, the fully connected layer is specifically as follows: the fully connected second layer in the original VGG16 model is extracted, that is, one fully connected layer is reduced, and the feature dimension is 4096 at this time; the activation function of the fully connected layer selects the Relu function, and then uses the principal component analysis algorithm to reduce the feature dimension to 5.

[0029] Further, in step S4, the Softmax classifier includes: using the Adam optimizer when training the Softmax classifier, selecting the cross entropy loss function as the loss function, and the number of cycles is 30 times.

[0030] The beneficial effects of the present invention are:

[0031] 1) The present invention directly converts one-dimensional ECG signals into two-dimensional images, which no longer requires noise filtering and feature extraction, reduces the complexity of the operation, and can reduce complex mathematical calculations to extract feature information of ECG signals. However, traditional separation methods will ignore some ECG beats during noise filtering and feature extraction, resulting in reduced authenticity of classification; therefore, the classification results of the present invention are more authentic and reliable.

[0032] 2) The traditional arrhythmia recognition method is to extract the feature information of the time domain and frequency domain separately, while the present invention combines the time domain and frequency domain information on a color image, and then migrates it to the VGG16 model for feature extraction and classification and recognition with normal signals, so that it can be observed more intuitively, explore deeper information, and the correlation between various different information is also better.

[0033] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0035] Figure 1 It is a flow chart of the deep learning arrhythmia classification method based on two-dimensional images of the present invention;

[0036] Figure 2 Image processing for interval abnormalities;

[0037] Figure 3 Image processing for abnormal amplitude;

[0038] Figure 4 It is a two-dimensional image dataset under normal circumstances;

[0039] Figure 5 It is a two-dimensional image dataset with interval abnormalities;

[0040] Figure 6 is the time-frequency image set of normal situation;

[0041] Figure 7 is a set of time-frequency images with amplitude anomalies;

[0042] Figure 8 The algorithm network structure of the present invention is based on the VGG16 model;

[0043] Fig. 9 This is the specific structure diagram of the softmax classifier;

[0044] Fig.10 Visualization of the results for interval abnormalities;

[0045] Fig.11 This is a graph showing the accuracy and loss value changes of the interval abnormality classification;

[0046] Fig.12 This is a visualization of the results of the amplitude anomaly;

[0047] Fig.13 This is a graph showing the accuracy and loss value changes for amplitude anomaly classification;

[0048] Fig.14 A flow chart comparing the present invention and the traditional classification method. DETAILED DESCRIPTION

[0049] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0050] See also Figure 1 to Figure 14 In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following is a further description of the data set, model, framework and experimental results used in the experiment. In this experiment, the MIT-BIH arrhythmia data set is used as the data input of the model, the WINDOWS operating system is used, the memory is 8G, the processor is i5-7400, the programming environment is Python3.7, and the framework structure is based on the TensorFlow backend.

[0051] This experiment uses the arrhythmia dataset from the public database MIT-BIH, which contains half an hour of monitoring data from 48 subjects, with a sampling frequency of 360Hz, and a total of more than 100,000 single heartbeats. This application divides abnormal heart rhythm into two situations for classification and identification experiments.

[0052] This embodiment converts the one-dimensional ECG signal into a two-dimensional image and then combines it with the VGG16 convolutional neural network to extract features. Then, PCA dimensionality reduction is performed based on the extracted multiple features to obtain obvious classification results, thereby improving the accuracy of ECG signal classification. In order to achieve the above purpose, the classification method of the present invention is as follows Figure 1 As shown, the specific steps include:

[0053] 1) Convert one-dimensional data into two-dimensional image stage:

[0054] For abnormal time intervals, the ECG signal is directly processed into images, such as Figure 2 As shown, 3000 points of ECG signals of about 10 seconds are extracted and divided into 5 parts from top to bottom. The 600 points on the horizontal axis represent the change of time. The color represents the change of amplitude to get a colorful two-dimensional image. The image size is adjusted to 224*224, and the split() function is called to split the original RGBA image format into four channels. Then call image.merge() under PIL to change it to a suitable RGB image and save it. The whole set of data is divided into two categories, one is the data marked as normal for all 10 seconds, and the other is the data with abnormal intervals within 10 seconds. Among them, Figure 3 is a two-dimensional image dataset under normal circumstances, Figure 4 It is a two-dimensional image dataset with interval abnormalities.

[0055] The second type of abnormal heart rhythm is the abnormal amplitude of the ECG signal, so the ECG signal is converted into a time-frequency image in combination with the frequency domain. The waveform of each P, Q, R, S, and T wave is drawn through the position of the R wave marker, and then the ECG signal of each cycle is Fourier transformed:

[0056] STFT(τ,f)=∫x(t)g(t-τ)e -2jπft dt

[0057] Among them, x(t) is the original signal and g is the window function.

[0058] Combine the Fourier transformed signal with time, and finally draw the waveform of 10 consecutive cycles into a two-dimensional image as shown in Figure 5 As shown, the horizontal axis is 10 consecutive cycles, which is also the change of time, and the vertical axis is the frequency obtained after Fourier transform. The color represents the amplitude change of the ECG signal. Figure 6 is the time-frequency image set under normal circumstances, Figure 7 is a set of time-frequency images with amplitude anomalies.

[0059] The data set of abnormal intervals was divided into training set and test set in a ratio of 7:3. There were 47 data in the normal case, 33 images were randomly selected as the training set, and the remaining 14 were used as the test set; there were 41 image data in the abnormal case, 29 were randomly selected as the training set, and the remaining 12 were used as the test set.

[0060] The data with abnormal amplitude are divided into two categories: one is that all the ECG signals of 10 consecutive cycles are marked as normal, and the other is that there are problems with abnormal amplitude in these 10 ECG signals. The ECGs of these two categories are classified and identified. There are 136 normal data sets, which are randomly divided into training sets and test sets at a ratio of 7:3. The training set has 98 data and the test set has 38 data; there are 71 image data in the abnormal situation, which are also randomly divided at a ratio of 7:3. There are 52 training sets and 19 test sets.

[0061] 2) Network training phase:

[0062] The two-dimensional convolutional neural network model is: modified on the original VGG16 model as a transfer learning and training model, specifically:

[0063] First, build the original VGG16 network and connect the input layer, convolution layer, maximum pooling layer, fully connected layer and softmax classifier in sequence.

[0064] Then, based on the VGG16 network model, the model input dimension is 224*224*3; the original structure of the VGG16 model for image feature extraction is maintained, and the fully connected second layer in the VGG16 model is extracted. At this time, the feature dimension is 4096, and then PCA is used to reduce the feature dimension to 5. Finally, the results are visualized based on the first two principal components. And based on the 4096-dimensional features proposed by the VGG16 model, the adjusted classifier is added to classify according to the label. The network structure is as follows Figure 8 As shown in the figure, the VGG16 model consists of 5 convolutional layers with a convolution kernel size of 3*3 and a pooling layer with a maximum pooling size of 2*2. Block One and Block Two contain two pooling layers and one convolutional layer, and Block Three, Block Four, and Block Five all contain three convolutional layers and one pooling layer. The calculation formula of the convolutional layer is:

[0065]

[0066] Among them, f is the original image, g is the "new image", h is the convolution kernel, * is the convolution operator; (i, j) represents the coordinates of the center pixel, i = 1, 2, ..., h, j = 1, 2, ..., w, where h is the image height and w is the image width. The full set needs to traverse the entire image.

[0067] The adjusted classifier is as follows: the softmax classifier is used for classification recognition, the feature data extracted after transfer learning is added to the fully connected layers of 256 dimensions, 128 dimensions, and 64 dimensions, and finally a 3-classification softmax classifier is added, such as Fig. 9As shown, the activation function of the fully connected layer selects the Relu function, and the formula is as follows: Relu(x) = max(0,x); the Adam optimizer is used when training the classifier, and the parameters are set as follows: α = 10 -5 , β1=0.9, β2=0.999, ε=10 -8 , the loss function uses the cross entropy loss function (categorical crossentropy), the number of cycles is 30, and finally the accuracy and loss obtained by training are visualized.

[0068]

[0069]

[0070] Among them, the PCA algorithm is specifically:

[0071] Suppose there are m data in the data set X, X=(x1,x2,x3,....x m ), each data has n feature attributes X i =(x i1 ,x i2 ,x i3 ,...,x in )(i=1,2,...,m), standardize X to get

[0072]

[0073] Find the covariance matrix U of X after standardization:

[0074]

[0075] Find the eigenvalue λ of the matrix U i and the eigenvector α i .

[0076] According to the eigenvalue λ i The values ​​of are arranged from large to small to select k, and the corresponding k λ i The feature vector α i Form the matrix A.

[0077] Construct k data vectors into a new space, namely Y=AX.

[0078] Will Figure 5 and Figure 7 The two types of data sets are migrated to the VGG16 model for training and learning, and the high-dimensional feature information is reduced in dimension using the PCA algorithm. The resulting visualization image after dimensionality reduction is as follows Fig.10As shown, the red dots are all normal signals concentrated in one area, and the blue dots are images with abnormal heart rhythm.

[0079] Based on the above, the present invention further improves the fully connected layer part of the VGG16 model to obtain specific accuracy and loss values.

[0080] 3) The classification results are as follows: the above-mentioned dataset of interval abnormalities after image processing is transferred to the VGG16 model for training and learning, and the high-dimensional feature information obtained is reduced in dimension using the PCA algorithm. The visualization image of the result after dimensionality reduction is as follows: Fig.10 As shown, the red dots are all normal signals concentrated in one area, and the blue dots are images with abnormal heart rhythm.

[0081] Verification experiment: K-fold cross validation was added, and the specific formula and settings are as follows:

[0082]

[0083] Wherein, MSE is mean square error; K=4.

[0084] Fig.11 The specific accuracy and loss values ​​obtained after adding the fully connected layer are as follows: for the ECG signal classification with abnormal intervals, the accuracy is 96.15%, the precision is 96.45%, the recall is 96.15%, and the F1 value is 96.16%.

[0085] A four-fold cross validation was performed on the data, and the mean square error (MSE) was [0.020, 0.008, 0.012, 0.046] and the mean absolute error was [0.107, 0.050, 0.078, 0.069], respectively. Both reflect the error between the true value and the predicted value, and the results are close to 0.

[0086] The above-mentioned dataset of amplitude anomalies after image processing is transferred to the VGG16 model for training and learning, and the high-dimensional feature information obtained is reduced in dimension using the PCA algorithm. The resulting visualization image after dimensionality reduction is as follows: Fig.12 As shown, the red dots are all normal signals concentrated in one area, and the blue dots are images with abnormal heart rhythm.

[0087] Fig.13 The specific accuracy and loss values ​​obtained after adding the fully connected layer are as follows: for ECG signal classification with abnormal amplitude, the accuracy is 94.67%, the precision is 93.04%, the recall is 92.98%, and the F1 value is 92.88%.

[0088] A four-fold cross validation was performed on the data, and the mean square error (MSE) was [0.022, 0.070, 0.185, 0.048] and the mean absolute error was [0.066, 0.144, 0.293, 0.119], respectively. Both reflect the error between the true value and the predicted value, and the results are close to 0.

[0089] The accuracy of classifying ECG signals with abnormal intervals can reach 96.60%, the precision is 96.45%, the recall is 96.25%, and the F1 value is 96.16%; at the same time, the accuracy of classifying ECG signals with abnormal amplitudes can reach 93.62%, the precision is 93.04%, the recall is 92.98%, and the F1 value is 92.88%. According to the data ratio of abnormal intervals and abnormal amplitudes, the average accuracy of the classification method in this paper is 95.84%, the precision is 94.50%, the recall is 94.39%, and the F1 value is 94.29%. Table 1 compares the present invention with the traditional ECG classification algorithm.

[0090] Table 1 is a comparison between the classification model of the present invention and other classification models:

[0091]

[0092] As can be seen from Table 1, although the accuracy of the method of the present invention is not the highest among the above methods, the methods with higher accuracy, such as one-dimensional CNN and FFNN, have one or more of the following limitations: the beat frequency of ECG is lost in the noise filtering and feature extraction scheme; the number of ECG arrhythmia types used for classification is limited; the classification performance used in practical applications is relatively low. The method of this paper converts the one-dimensional ECG signal into a two-dimensional image, and no longer requires noise filtering and feature extraction. This is very important because some ECG beats will be ignored in noise filtering and feature extraction, which is more reliable and authentic. In addition, the precision, recall and F1 value of this paper are all at a high level, and there is no phenomenon that only the accuracy is high and other values ​​are low in the one-dimensional CNN method.

[0093] Fig.14 The flowchart of the present invention is compared with the traditional two-dimensional image ECG signal classification method. Fig.14 It can be seen that the present invention does not perform pre-processing such as filtering on the ECG signal, because in the noise filtering and feature extraction scheme, it will cause some beat frequency loss of the ECG signal, reducing the recognition rate; and the traditional method is to perform deep learning and classification on the two-dimensional heartbeat image of the ECG signal, while the present invention analyzes the two-dimensional image by combining the time domain and frequency domain information of the one-dimensional signal.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A deep learning arrhythmia classification method based on two-dimensional images, characterized in that: The method comprises the following steps: S1: Classify the abnormal interval and amplitude of the one-dimensional original ECG signal; S2: Perform two-dimensional imaging processing on the one-dimensional ECG signals with abnormal intervals and amplitudes to obtain a two-dimensional image set, which specifically includes: (1) For abnormal intervals: extract 3000 points of 10s ECG signal and divide it into 5 parts from top to bottom. The horizontal axis is the 600 points of each signal segment, which is also the corresponding time change. The color change is used to represent the change of amplitude to obtain a colorful two-dimensional image. The whole set of data is divided into two categories, one is the data marked as normal for all 10s, and the other is the data with abnormal intervals within 10s. (2) For abnormal amplitude: obtain the ECG waveform of each cycle based on the position of the R wave of the one-dimensional signal, perform Fourier transform on it, combine the transformed signal with time, and finally draw the waveform of 10 consecutive cycles into a two-dimensional time-frequency image, with the horizontal axis representing the change of time and the vertical axis representing the time-frequency of frequency; Resize all images to 224*224, call the split() function to split the original RGBA image format into four channels, then call image.merge() under PIL to convert it to a suitable RGB image and save it; S3: Divide the obtained two-dimensional image set into a training set and a test set; S4: Use transfer learning to transfer the relevant parameters of the trained and adjusted VGG16 network model to the new dataset for training; The adjusted VGG16 network model includes: sequentially connected input layer, convolution layer, maximum pooling layer, fully connected layer and Softmax classifier; the convolution layer is used to extract the representative features of the two-dimensional image set and complete the two-dimensional convolution feature extraction of the sample, and the convolution kernel size is 3*3; the maximum pooling layer is used to reduce the number of features of the convolution layer; the fully connected layer integrates multi-channel data; the Softmax classifier is classified according to the label; The operation formula of the convolutional layer is: Among them, (i, j) represents the coordinates of the center pixel, m represents the width, n represents the height, g represents the output image of the convolution layer, f represents the original image, h represents the convolution kernel, and * is the convolution operator; S5: Output the arrhythmia classification result corresponding to the current target ECG signal.

2. The deep learning arrhythmia classification method according to claim 1, characterized in that: Step S3 specifically includes: dividing the obtained two-dimensional image set into a training set and a test set in a ratio of 7:

3.

3. The deep learning arrhythmia classification method according to claim 1, characterized in that: In step S4, the fully connected layer is specifically as follows: the fully connected second layer in the original VGG16 model is extracted, that is, one fully connected layer is reduced; the activation function of the fully connected layer selects the Relu function, and then the principal component analysis algorithm is used to reduce the feature dimension.

4. The deep learning arrhythmia classification method according to claim 1, characterized in that: In step S4, the Softmax classifier includes: using an Adam optimizer when training the Softmax classifier, and selecting a cross entropy loss function as a loss function.

Images