Multi-label multi-lead arrhythmia classification method based on fusion neural network

Abstract

The application discloses a multi-label multi-lead arrhythmia classification method based on a fusion neural network, and specifically comprises the following steps: inputting original electrocardio signals after preprocessing into an electrocardio local feature extraction module composed of a convolutional neural network to perform local saliency feature extraction of the multi-lead electrocardio signals; inputting shallow electrocardio signal features extracted by the convolutional neural network into an electrocardio time sequence feature extraction module composed of a recurrent neural network to perform time sequence feature extraction of the multi-lead electrocardio signals; fusing the electrocardio local saliency features and the electrocardio time sequence features to form electrocardio features used for classification finally; inputting the fused electrocardio features into a linear classification layer to output multi-label categories; and using a Softmax extended to a multi-label scene in combination with a cross-entropy loss function to perform network training. The application can extract local saliency features and time sequence features of electrocardio signals simultaneously, can relieve the data imbalance problem of a multi-label data set, and has good performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electrocardiosignal classification, and particularly relates to a multi-label multi-lead arrhythmia classification method based on a fusion neural network. BACKGROUND

[0002] Cardiovascular diseases have a great threat to human life, health and safety. According to the World Health Organization's 2018 report, "2018 World Health Statistics Report", about 17.9 million people died of cardiovascular diseases worldwide in 2016, accounting for about 31.4% of the global death toll that year and 44% of the death toll of chronic non-communicable diseases that year. How to give full play to the advantages of information science, especially the rapid development of artificial intelligence technology in recent years, through medical and engineering cross-innovation, to achieve early prevention, early detection and early treatment of cardiovascular diseases, effectively reduce the burden of doctors, improve disease prevention and treatment efficiency, and reduce the waste of medical resources in the process of diagnosis and treatment, has become the key and focus of cardiovascular disease prevention and treatment.

[0003] Electrocardiogram detection plays an irreplaceable role in the early warning of arrhythmia, myocardial infarction and other cardiovascular diseases, and is one of the most commonly used diagnostic techniques in clinical practice. With the development of machine learning technology and deep learning technology, the increase of medical field data, the research of computer-aided electrocardiogram automatic diagnosis algorithm has also been developed, and its application is becoming more and more widely. In the field of computer-aided electrocardiogram automatic diagnosis research, certain achievements have been made in the diagnosis of electrocardiogram for binary classification and multi-classification. In actual clinical practice, various types of arrhythmia exist in the form of cross-over and mutual transformation, and a cardiovascular disease patient may correspond to multiple arrhythmia types at the same time, which is called electrocardio multi-label phenomenon. Electrocardio multi-label phenomenon is common in clinical electrocardiogram databases.

[0004] It is one of the problems to be solved in the multi-label multi-lead arrhythmia classification task to design a reasonable network structure to extract the local saliency features and the time sequence features of the ECG signal to improve the performance of the ECG multi-label classification model. The research on the application of deep learning technology in the multi-label ECG classification task usually refers to the network structure in the research on the ECG multi-classification task. According to the network structure, it can be divided into three categories: deep convolutional neural network CNN, recurrent neural network RNN, and the network structure combining deep convolutional neural network CNN and recurrent neural network RNN. The first category of deep convolutional neural network CNN can extract more abundant shallow features and deep semantic features of the signal. However, the ECG signal is a time sequence signal with rich time sequence features, and the deep convolutional neural network can only capture the local features of the ECG signal waveform, but cannot capture the time sequence features of the ECG signal. The second category of recurrent neural network RNN depends on the information of the current sequence to learn the current information, and has good performance in extracting time sequence features, and is widely used in ECG multi-classification tasks, and can capture the time sequence features of the ECG signal. The disadvantage of the recurrent neural network is that the local features of the ECG signal are discarded. The third category of network combining deep convolutional neural network CNN and recurrent neural network RNN can capture the local features and time sequence features of the ECG signal. This kind of network structure usually combines LSTM and CNN, but the number of layers of CNN in the existing research is too small, which makes the generalization performance of the model poor. At the same time, due to the complexity of the LSTM network structure, a large amount of time and resources are consumed.

[0005] In addition, how to design a reasonable objective function to alleviate the imbalance problem of the data set is one of the problems to be solved in the multi-label multi-lead arrhythmia classification task. The multi-label ECG data set generally has the problem of data imbalance, mainly in two aspects: one is the imbalance between positive and negative samples, and the number of negative samples is much larger than the number of positive samples; the other is the imbalance between different abnormal types, for example, the number of samples with P-wave label is much smaller than the number of samples with sinus rhythm label. In the multi-label ECG classification task, the task of the model is to judge whether a sample simultaneously exists in all ECG types in the label space.

[0006] The solution to the multi-label classification problem is usually to convert it into multiple binary classification problems, and the objective function is a combination of sigmoid function and cross-entropy loss function. This scheme pays equal attention to positive and negative samples. In the case of data distribution imbalance, the class with less sample quantity plays a smaller role in the network training process than the class with more sample quantity, which makes it difficult for the model to correctly learn the information contained in the class with less sample quantity, and finally the prediction result of the model tends to the class with more sample quantity. SUMMARY

[0007] The application aims to provide a multi-label multi-lead arrhythmia classification method based on a fusion neural network, solve the problem that the existing neural network model cannot simultaneously extract and utilize the local saliency features and the time sequence features of an ECG signal, and simultaneously propose a loss function design scheme combining a softmax activation function and a cross-entropy loss, thereby relieving the data imbalance problem of a multi-label electrocardio data set.

[0008] The technical scheme adopted by the application is a multi-label multi-lead arrhythmia classification method based on a fusion neural network, comprising the following steps:

[0009] a) inputting the original electrocardio signal after preprocessing to an electrocardio local feature extraction module composed of a convolutional neural network to perform local saliency feature extraction of the multi-lead electrocardio signal.

[0010] b) inputting the shallow electrocardio signal features extracted by the convolutional neural network to an electrocardio time sequence feature extraction module composed of a recurrent neural network to perform time sequence feature extraction of the multi-lead electrocardio signal.

[0011] c) fusing the electrocardio local saliency features and the electrocardio time sequence features to form electrocardio features finally used for classification.

[0012] d) inputting the fused electrocardio features to a linear classification layer to output multi-label categories.

[0013] e) using a Softmax combining a cross-entropy loss function extended to a multi-label scenario to perform network training.

[0014] As a further improvement of the application, the specific process of preprocessing the original electrocardio signal is as follows: filtering the original electrocardio signal to filter out baseline drift interference and power frequency interference, performing data enhancement processing to increase the samples for model training, and finally performing resampling processing.

[0015] As a further improvement of the application, the filtering processing in step a) is to use an IIR high-pass filter to process the ECG signal to filter out the baseline drift interference of the ECG signal, the cutoff frequency of the IIR high-pass filter being 0.1 Hz, use an IIR notch filter to process the ECG signal after filtering out the baseline drift interference to filter out the power frequency interference, the filtering frequency of the IIR notch filter being 50 Hz, and finally obtain the filtered ECG signal. The data enhancement processing is to randomly perform vertical flip up and down and random horizontal translation of 10 or fewer sampling points on the filtered ECG signal to increase the samples of the data. The resampling processing is to resample all the samples of the data set after data enhancement into C*2048 signals, C being the number of leads of the ECG signal.

[0016] As a further improvement of the application, the ECG local feature extraction module of step a) is a one-dimensional ResNet18 model structure, and the module input is the preprocessed ECG signal. The module structure is divided into a first convolutional layer, a maximum pooling layer, a residual convolutional structure layer, and an average pooling layer. The first convolutional layer uses 64 convolutional kernels with a scale of 15, a sliding step of 2, and an edge padding value of 7. The maximum pooling layer has a sliding step of 2. The number of convolutional kernels in the residual convolutional layer is [64 128 256 512], the convolutional kernel scale is 7, and the edge padding value is 3. The structure of the residual convolutional layer is [2 2 22], each residual layer contains two identical basic residual structures, and each residual basic structure contains two convolutional layers with the same number of convolutional kernels. The residual basic structure maps the input of a certain layer to any layer through a jump connection, solving the problems of gradient disappearance and gradient dispersion caused by network degradation.

[0017] As a further improvement of the application, the ECG local feature extraction module of step a) is a one-dimensional ResNet18 model structure, and the module input is the preprocessed ECG signal. The module structure is divided into a first convolutional layer, a maximum pooling layer, a residual convolutional structure layer, and an average pooling layer. The first convolutional layer uses 64 convolutional kernels with a scale of 15, a sliding step of 2, and an edge padding value of 7. The maximum pooling layer has a sliding step of 2. The number of convolutional kernels in the residual convolutional layer is [64 128 256 512], the convolutional kernel scale is 7, and the edge padding value is 3. The structure of the residual convolutional layer is [2 2 22], each residual layer contains two identical basic residual structures, and each residual basic structure contains two convolutional layers with the same number of convolutional kernels. The residual basic structure maps the input of a certain layer to any layer through a jump connection, solving the problems of gradient disappearance and gradient dispersion caused by network degradation.

[0018] As a further improvement of the application, the ECG local feature extraction module of step a) is a one-dimensional ResNet18 model structure, and the module input is the preprocessed ECG signal. The module structure is divided into a first convolutional layer, a maximum pooling layer, a residual convolutional structure layer, and an average pooling layer. The first convolutional layer uses 64 convolutional kernels with a scale of 15, a sliding step of 2, and an edge padding value of 7. The maximum pooling layer has a sliding step of 2. The number of convolutional kernels in the residual convolutional layer is [64 128 256 512], the convolutional kernel scale is 7, and the edge padding value is 3. The structure of the residual convolutional layer is [2 2 22], each residual layer contains two identical basic residual structures, and each residual basic structure contains two convolutional layers with the same number of convolutional kernels. The residual basic structure maps the input of a certain layer to any layer through a jump connection, solving the problems of gradient disappearance and gradient dispersion caused by network degradation.

[0019] As a further improvement of the application, the linear classification layer of step d) is a Linear layer, and the input of the Linear layer is the fused ECG feature, and the output of the Linear layer is the label category of the multi-label ECG data set.

[0020] As a further improvement of the application, the multi-label loss function of step e) is an extension of the combination of the softmax activation function and the cross-entropy loss function in the single-label multi-classification scenario. Assuming that the scores of each class output by the network output layer are s1, s2,..., sn , the cross-entropy loss function is equivalent to making all non-target class scores s1, s2,...s t-1 t+1 ,...,s n s t s0 as small as possible. For multi-label classification, since the value of class k output by the model is uncertain, it is necessary to determine which classes the model outputs according to the threshold value, and for this purpose, an additional 0 class is introduced, and we hope that the scores of the target classes are all greater than the score s0 of the 0 class, while the scores of the non-target classes are all less than s0, at this time the loss function can be written as:

[0021]

[0022] Where neg, pos represent the positive and negative sample sets respectively, and C represents the number of label classes. Assuming that s0 = 0, the formula of the above loss function is simplified as:

[0023]

[0024] Compared with the prior art, the beneficial effects of the present application are as follows:

[0025] 1. The present application combines convolutional neural network and recurrent neural network, the convolutional neural network adopts ResNet structure, which can effectively extract local saliency features of electrocardio signals, and its residual structure can effectively avoid network degradation problem caused by gradient disappearance through jump connection. The recurrent neural network adopts GRU unit, which can effectively extract time sequence features of time sequence signals, and has fast training speed and good performance. The fusion of ResNet and GRU enables the model to have the ability to extract local saliency features and time sequence features of electrocardio signals at the same time, which can effectively extract features of electrocardio signals and improve the performance of the model on multi-label electrocardio data set.

[0026] 2. The present application no longer converts the multi-label classification problem into multiple binary classification problems, but regards it as a problem of comparing the target class score with the non-target class score, which can automatically balance the weight between classes and alleviate the imbalance problem of multi-label electrocardio data set, and at the same time, without the need for additional weight matrix to alleviate the imbalance, the complexity of the algorithm is reduced, and the performance of the model on multi-label electrocardio data set is improved. DETAILED DESCRIPTION

[0027] Figure 1 It is a flow structure diagram for multi-label multi-lead arrhythmia classification based on fusion neural network.

[0028] Figure 2 It is a residual base structure diagram.

[0029] ​Figure 3 is a structure diagram of a GRU unit. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying Figure 1 , the accompanying Figure 2 , the accompanying Figure 3 The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying

[0031] Referring to Figure 1 , the multi-label multi-lead arrhythmia classification method based on the fusion neural network comprises the following steps:

[0032] a) inputting the original electrocardio signal after preprocessing to an electrocardio local feature extraction module composed of a convolutional neural network to perform local saliency feature extraction of the multi-lead electrocardio signal.

[0033] b) inputting the shallow electrocardio signal features extracted by the convolutional neural network to an electrocardio time sequence feature extraction module composed of a recurrent neural network to perform time sequence feature extraction of the multi-lead electrocardio signal.

[0034] c) fusing the electrocardio local saliency features and the electrocardio time sequence features to form electrocardio features finally used for classification.

[0035] d) inputting the fused electrocardio features to a linear classification layer to output multi-label categories.

[0036] e) using a Softmax extended to a multi-label scenario in combination with a cross-entropy loss function to perform network training.

[0037] The specific process of preprocessing of the original electrocardio signal in step a) is as follows: performing filtering processing on the original electrocardio signal to filter out baseline drift interference and power frequency interference, performing data enhancement processing to increase samples for model training, and finally performing resampling processing. The filtering processing is to use an IIR high-pass filter with a cutoff frequency of 0.1 Hz to process the ECG signal to filter out the baseline drift interference of the ECG signal, and use an IIR notch filter to process the ECG signal after filtering out the baseline drift interference to filter out the power frequency interference of 50 Hz, to obtain the filtered ECG signal. The data enhancement processing is to randomly perform vertical flip up and down and random horizontal translation of 10 or fewer sampling points on the filtered ECG signal to increase the samples of the data. The resampling processing is to resample all the data set samples after data enhancement to C*2048 signals, where C is the number of leads of the ECG signal.

[0038] The electrocardio local feature extraction module in step a) is a one-dimensional ResNet18 model, referring to Figure 2, which uses the basic residual structure to map the output of any layer to another layer, avoiding the network degradation problem caused by gradient disappearance or gradient explosion, making it possible to deepen the network and better extract the local saliency features of the ECG signal.

[0039] The one-dimensional ResNet18 network model structure is shown in the table, including a first convolutional layer, a maximum pooling layer, a residual convolutional structure layer, and an average pooling layer. The first convolutional layer uses 64 convolutional kernels with a scale of 15, a sliding step of 2, and an edge padding value of 7. The maximum pooling layer has a sliding step of 2. The number of convolutional kernels in the residual convolutional layer is [64 128 256 512], the convolutional kernel scale is 7, and the edge padding value is 3. The structure of the residual convolutional layer is [2 2 2 2], each residual layer contains two identical basic residual structures, and each residual basic structure contains two convolutional layers with the same number of convolutional kernels. The input of the ResNet18 network is C*2048 multi-lead multi-label ECG data, and the output is 512*1 ECG local features.

[0040]

[0041]

[0042] The ECG time sequence feature extraction module in step b) is a 3-layer GRU network, and the input is the shallow ECG feature output by the first convolutional layer of the ECG local feature extraction module in step a). See Figure 3 The GRU unit is composed of an update gate and a reset gate, where the update gate determines how much information from the current time state and the previous time state needs to be continued. The input is the concatenation of the current input and the previous input after linear transformation, and the probability value of retaining the current information and the previous information is obtained through the sigmoid activation function. The reset gate determines how much past information to forget. The input is the same as the update gate, and the difference is only in the different parameters and uses of linear transformation. First, the gating signal is reset through the reset gate, and then concatenated with the input. The concatenated data is scaled to the range of -1~1 by the tanh activation function, obtaining data containing the current input. In the update memory stage, forgetting and memory operations are performed simultaneously.

[0043] The 3-layer GRU network structure is shown in the table, and the hidden layer parameters of the GRU unit are [64 64 1]. The input of the GRU network is 64*1024 shallow ECG features, and the output is 64*1 ECG time sequence features.

[0044]

[0045] The ECG local feature and the ECG time sequence feature in step c) are fused by feature splicing in the first dimension, and the 512*1 ECG local feature and the 64*1 ECG time sequence feature are spliced into 576*1 ECG features for network training.

[0046] The linear classification layer in step d) is a Linear layer, the input of which is the fused 576*1 ECG features, and the output is the label category of the multi-label ECG data set, and N is the number of label categories in the data set.

[0047] The loss function for network training in step e) is a Softmax combined cross-entropy loss function extended to a multi-label scenario. The scores of each category output by the network output layer are defined as s1, s2,...,s n The scores of all non-target categories s1, s2,...s t-1 , t+1 ,...,s n are compared with the score s t of the target category, and the maximum difference between the scores of the non-target categories and the score of the target category is made as small as possible.

[0048]

[0049] Wherein, neg and pos represent the positive and negative sample sets respectively, and C represents the number of label categories. Assuming that s0=0, the formula of the above loss function is simplified as:

[0050]

[0051] Finally, it should be noted that: the above only describes a more reasonable embodiment of the present application, and is not used to limit the present application, although the present application has been described in detail with reference to the foregoing examples, for those skilled in the art, the technical solutions recorded in the foregoing examples can still be modified, or some technical features can be replaced. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-label, multi-lead arrhythmia classification method based on fused neural networks, characterized in that, Includes the following steps: a) After preprocessing, the raw ECG signal is input into the ECG local feature extraction module composed of a convolutional neural network to extract the local saliency features of the multi-lead ECG signal; b) Input the shallow ECG signal features extracted by the convolutional neural network into the ECG temporal feature extraction module composed of a recurrent neural network to extract the temporal features of multi-lead ECG signals; c) Integrate local saliency features with temporal features to form the final electrocardiogram features used for classification; d) Input the fused ECG features into a linear classification layer and output multi-label categories; e) Use Softmax, which is extended to multi-label scenarios, combined with the cross-entropy loss function for network training; The ECG local feature extraction module described in step a) uses a one-dimensional ResNet18 model as the network structure. The network input is C*2048 ECG data, where C is the number of leads in the ECG signal. The convolutional layer uses 64 convolutional kernels with a scale of 15, a stride of 2, and an edge padding value of 7. This convolutional layer outputs 64*1024 shallow ECG signal features. This convolutional layer is followed by a max pooling layer with a stride of 2. After the max pooling layer, a residual convolutional structure is connected. All convolutional kernels in the residual convolutional structure have a scale of 7 and an edge padding value of 3. The residual convolutional structure has four layers, stacked in a [2 2 2 2] configuration. Each residual convolutional structure contains two convolutional layers, repeated twice, for a total of 16 convolutional layers. The number of convolutional kernels in each residual convolutional structure is [64 128 256]. [512] The last convolutional layer outputs 512*64 features; after the convolutional layer, an average pooling layer is connected, which outputs local saliency features with a dimension of 512*1; The ECG time series feature extraction module described in step b) uses a 3-layer GRU as the network structure; its input is the extracted shallow ECG signal features with an input dimension of 64*1024, the hidden layer dimension of the GRU is [64 64 1], and the output is a 64*1 time series feature. In step e), the scheme of combining the softmax activation function with the cross-entropy loss function is extended to multi-label datasets, transforming the multi-label classification problem into a problem of comparing the scores of the target class and the scores of the non-target class pairwise, and automatically balancing the weights between the classes. Its objective function is ; in , These represent the sets of positive and negative samples, respectively.

2. The multi-label, multi-lead arrhythmia classification method based on a fusion neural network according to claim 1, characterized in that, In step c), local saliency features and temporal features are fused. The fusion operation used is a splicing operation. After fusion, the final ECG feature dimension used for classification is 576*1.

3. The multi-label, multi-lead arrhythmia classification method based on a fusion neural network according to claim 1, characterized in that, In step d), the fused ECG features are input into the linear classification layer, and the output is a multi-label category; the linear layer input is 576*1, and the output is N*1, where N is the number of categories in the ECG dataset.

4. The multi-label, multi-lead arrhythmia classification method based on a fusion neural network according to claim 1, characterized in that, The specific process of preprocessing the original ECG signal in step a) is as follows: the original ECG signal is filtered to remove baseline drift interference and power frequency interference; data augmentation is performed to increase the number of samples for model training; and finally, resampling is performed.

5. The multi-label, multi-lead arrhythmia classification method based on a fusion neural network according to claim 4, characterized in that, The filtering process involves using an IIR high-pass filter to process the ECG signal, removing baseline drift interference. The cutoff frequency of the IIR high-pass filter is 0.1Hz. Then, an IIR notch filter is used to process the baseline drift interference-free ECG signal, removing power frequency interference. The filtering frequency of the IIR notch filter is 50Hz. Finally, the filtered ECG signal is obtained. The data augmentation process involves randomly flipping the filtered ECG signal vertically and horizontally, and randomly shifting it vertically by no more than 10 sampling points to increase the number of data samples. The resampling process involves resampling all samples of the augmented dataset into a C*2048 signal, where C is the number of leads in the ECG signal.

Images