Multi-lead electrocardiogram classification and identification method based on convolution and self-attention mechanism

Abstract

The application discloses a multi-lead electrocardiogram classification and recognition method based on convolution and self-attention mechanism, which mainly comprises a hierarchical self-attention network and a convolution attention network. The hierarchical self-attention network can capture long-range dependence features across multiple heartbeats by using the self-attention mechanism. The lead-sensitive mechanism uses a window-based self-attention mechanism to calculate self-attention scores from two views of the time dimension and the lead dimension, thereby focusing on the specificity differences of different lead electrocardiogram signals. The convolution attention network uses the inductive bias that the local electrocardiogram signal waveform is invariant to time translation, uses convolution to extract local waveform features, and focuses on local key waveform information. The multi-lead electrocardiogram classification and recognition method provided by the application can be used in an electrocardiogram multi-label classification task and has a high classification accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer science, and particularly relates to a method for solving a multi-label classification and identification task of multi-lead electrocardiogram data based on deep learning technology. BACKGROUND

[0002] Heart disease is an important factor threatening human health and has been widely concerned all over the world. Electrocardiogram (ECG) is a technology for recording the electrical activity changes of the heart during each cardiac cycle from the body surface using an electrocardiograph, and is one of the most commonly used non-invasive, simple and economical methods for screening and diagnosing arrhythmia and cardiovascular diseases. Due to the complexity and diversity of electrocardiogram, the same type of arrhythmia may vary from person to person, which makes even experienced doctors need time to accurately diagnose electrocardiogram. Using computer technology to automatically diagnose electrocardiogram can assist cardiologists in clinical practice and help patients in areas with insufficient medical resources. As mentioned above, it is a challenge to automatically diagnose arrhythmia through electrocardiogram, so it is crucial to improve the accuracy of electrocardiogram automatic classification.

[0003] Conventionally, electrocardiogram classification algorithms first segment the electrocardiogram signal into several heartbeats, and use expert systems or statistical-based machine learning methods (such as support vector machines) to extract manually designed medical features for heartbeat-level classification. However, since expert feature selection involves a lot of medical field knowledge, and the complex preprocessing process required for feature extraction, its effectiveness is limited by the effectiveness of the extracted expert features. Therefore, its accuracy is significantly lower than that of a cardiologist.

[0004] In recent years, deep learning technology has made remarkable achievements in computer vision, natural language processing, speech recognition and other fields. It has been widely applied in the medical field, such as medical image segmentation, drug development, genetic analysis, etc. Deep neural networks can automatically extract features from large-scale data, and use raw data for end-to-end training and reasoning, eliminating the tedious feature extraction work performed by humans. In the electrocardiogram classification task, various types of convolutional neural network and recurrent neural network models and their hybrid models significantly improve the accuracy of classification and diagnosis.

[0005] The most commonly used standard static electrocardiogram in clinic is an electrocardiogram with a length of 10 seconds and containing 12 lead electrocardiogram signals. The multi-label classification task of the multi-lead static electrocardiogram with a length of 10 seconds is a common and application-valued task, that is, the input is a 10-second multi-lead electrocardiogram signal sequence, and the output is one or more corresponding heart rhythm types. The main challenges of this task are as follows: on the one hand, the electrocardiogram with a length of ten seconds is a long time sequence containing multiple heartbeats, which makes it difficult for methods commonly used for time series such as long short-term memory network to capture the long-range dependence relationship of multiple heartbeats. Therefore, many existing works divide the entire electrocardiogram signal into heartbeats or short segments of fixed length. Recently, some works use residual convolutional networks or convolutional neural networks that fuse long short-term memory networks or convolutional neural networks that fuse Transformers to directly classify the entire electrocardiogram signal. However, they mostly focus on multi-classification tasks rather than multi-label classification tasks, and the latter is more suitable for clinical needs because there may be multiple heart rhythm types on the same electrocardiogram; on the other hand, different leads reflect the potential information of the heart activity from different spatial angles, so the multi-lead electrocardiogram can comprehensively reflect the condition of the heart. Therefore, the standard 12-lead electrocardiogram can observe more abnormal conditions than the single-lead electrocardiogram. For example, diagnosing right ventricular hypertrophy requires observing the abnormal waveforms of V1, V2 and V6 leads, and a single-lead electrocardiogram containing only I lead or II lead cannot diagnose this abnormal heart rhythm. Existing electrocardiogram classification methods usually concatenate the multi-lead electrocardiogram signal into a matrix and send it into a neural network to extract overall features, without extracting features specific to the lead to improve the classification result. SUMMARY

[0006] The present application aims to overcome the shortcomings of the prior art and proposes an end-to-end multi-lead electrocardiogram multi-label classification recognition method based on convolution and self-attention mechanism. The convolution attention network can focus on the local waveform features of the electrocardiogram signal, and the hierarchical self-attention network can extract the dependence relationship across multiple heartbeats. The present application also proposes a window-based lead-aware mechanism that can fully utilize the physical meaning of each lead in the multi-lead electrocardiogram to extract lead-specific information and improve the classification accuracy of the electrocardiogram model.

[0007] The specific technical solutions to achieve the objectives of the present application are as follows:

[0008] A multi-lead electrocardiogram classification recognition method based on convolution and self-attention mechanism, the electrocardiogram classification recognition method comprising:

[0009] Step 1: Preprocess the multi-lead electrocardiogram data; set X represents multi-lead ECG data with a sequence length of N and a number of leads of m, i.e., X is a multivariate time series, represented as a two-dimensional matrix as shown in formula (1); data preprocessing specifically includes the following steps:

[0010]

[0011] Step 1.1: Use a high-pass filter to filter out signals with frequencies in the range of 0Hz to 0.5Hz;

[0012] Step 1.2: Then, Fourier transform is used to convert the multi-lead ECG data. The resampling is a time series of length n, and the output after resampling is: Where n is any positive integer that is a multiple of 32;

[0013] Step 2: Employing a hierarchical self-attention network consisting of multiple stages, feature vectors containing long-range cross-beat dependence and specific lead characteristics are extracted from the ECG signal data output after Step 1 processing; specifically including the following steps:

[0014] Step 2.1: The ECG signal output after processing in Step 1 First, the ECG segment embedding layer is used to divide the ECG signal into non-overlapping segments. Each segment contains 1×4 sampling points; then, a linear layer embeds each segment into a feature vector; the length of the embedded feature vector is the hyperparameter C, which can be set to any positive integer; the output of this step is...

[0015] Step 2.2: Add absolute position encoding in the lead dimension; adjust the output of Step 2.1. Add a learnable parameter Right now in and This represents the input and output of the j-th time segment of the i-th lead during the absolute position encoding process. It is the absolute position encoding parameter of the i-th lead;

[0016] Step 2.3: Construct two window-based Transformer modules; replace the multi-head self-attention mechanism in the standard Transformer module with a window-based multi-head attention mechanism, which only computes within the window; the specific construction steps are as follows:

[0017] Suppose a window contains l segments. The computation process within each window using the window-based multi-head self-attention mechanism is as follows:

[0018]

[0019]

[0020]

[0021] wherein is the input of a window, the number of segments is l and the feature dimension of each segment is d, represents the output of a window after calculation, Q i , K i and represent query, key and value, h represents the number of self-attention heads, H i represents the i-th attention head, and are learnable weight parameters, Concat represents vector concatenation operation, and Attention represents self-attention operation; three window-based multi-head self-attention mechanisms are constructed based on the mechanism according to different window division manners, which are as follows:

[0022] Firstly, the mechanism input is Lxf segments, L and T represent the number of segments in the lead dimension and the time dimension respectively, the window is divided in the time dimension, and the size of each window is 1xT', T' is a hyperparameter satisfying T'≤T, representing the length of the window in the time dimension, the input segments are divided into non-overlapping one-dimensional windows, and then the multi-head self-attention mechanism is calculated in the window; it is called time-sensitive window-based multi-head self-attention mechanism; secondly, the mechanism input is LxT segments, the window is divided in the lead dimension, and the size of the window is Lx1, the input segments are divided into 1xT non-overlapping one-dimensional windows, and then the multi-head self-attention mechanism is calculated in the window; it is called lead-sensitive window-based multi-head self-attention mechanism; thirdly, after the window is divided by the time-sensitive window-based multi-head self-attention mechanism described in the first aspect, the window is moved leftward along the time dimension by l segments, and then the multi-head self-attention mechanism is calculated in the moved window; it is called time-sensitive multi-head self-attention mechanism based on moving window.

[0023] For the time-sensitive window-based multi-head self-attention mechanism and the time-sensitive multi-head self-attention mechanism based on moving window, relative position encoding in the time dimension is added, that is, the calculation process of the multi-head self-attention is replaced from formula (3) to formula (5):

[0024]

[0025] wherein the value in B i is selected from ​​a learnable relative position encoding parameter representing the relative position range of two segments in the time dimension within [-l+1, l-1];

[0026] According to the three window-based multi-head self-attention mechanisms, two window-based Transformer modules are constructed, specifically:

[0027] One of them is divided into 3 sub-modules, and the calculation process of the continuous 3 sub-modules is shown in formulas (6)-(11), wherein and respectively represent the output of the window-based attention layer and the multi-layer perceptron MLP based on the sub-module b, LN represents batch normalization, TW-MSA represents time-sensitive window-based multi-head self-attention mechanism, LW-MSA represents lead-sensitive window-based multi-head self-attention mechanism, TSW-MSA represents time-sensitive moving window-based multi-head self-attention mechanism, and is called time-lead Transformer module;

[0028]

[0029]

[0030]

[0031]

[0032]

[0033]

[0034] The second one is divided into 2 sub-modules, and the calculation process of the continuous 2 sub-modules is shown in formulas (12)-(15), wherein the symbol meanings are the same as those in the aforementioned time-lead Transformer module; it is called time Transformer module;

[0035]

[0036]

[0037]

[0038]

[0039] Step 2.4: Constructing two segment merging layers;

[0040] One, for an input containing LxT segments, let the feature dimension of each segment be C', concatenate the features of each two adjacent segments on the same lead, and then perform layer normalization on the new features generated after concatenation; after this layer operation, the number of segments in the time dimension changes from T to The feature dimension of each segment changes from C' to 2C'; referred to as the time segment merging layer;

[0041] Two, for an input containing LxT segments, let the feature dimension of each segment be C', concatenate all the segments of each lead at each time point, and after concatenation, the number of segments in the lead dimension decreases from L to 1, and the feature dimension of each segment changes from C' to LxC'; then use a fully connected layer to reduce the feature dimension from LxC' to C', and finally perform layer normalization operation on the new features; referred to as the lead segment merging layer;

[0042] Step 2.5: using the structure constructed in steps 2.4 and 2.5, construct a 5-stage hierarchical self-attention network, and use the network to extract features from the output of step 2.1 The feature of each stage is:

[0043] Stage 1, the structure of this stage is composed of 1 time-lead Transformer module described in step 2.3 and 1 time segment merging layer described in step 2.4; the output of step 2.1 After the operation of the structure in turn, the output is

[0044] Stage 2, the structure of this stage is composed of 2 time-lead Transformer modules described in step 2.3 and 1 lead segment merging layer described in step 2.4; the output of stage 1 After the operation of the structure in turn, the output is

[0045] Stage 3, the structure of this stage is composed of 3 time Transformer modules described in step 2.3 and 1 time segment merging layer described in step 2.4; the output of stage 2 After the operation of the structure in turn, the output is

[0046] Stage 4, the structure of this stage is composed of 1 time Transformer module described in step 2.3 and 1 time segment merging layer described in step 2.4; the output of stage 3 After the operation of the structure in turn, the output is

[0047] Stage 5, the structure of this stage consists of 1 time Transformer module described in step 2.3; the output of stage 4 After sequentially operating the structure, the output is

[0048] Step 3: A multi-stage convolutional attention network is used to extract local waveform features from the output of step 1, and a 256-channel feature map is output Specifically, the following steps are included:

[0049] Step 3.1: For the input of the electrocardiogram signal output in step 1 First, use a convolutional layer with a kernel length of 15 and a step of 2 to downsample the input scale to Then use batch normalization and ReLU activation function on it; then use a maximum pooling kernel with a length of 3 and a step of 2 to further reduce the dimension of the feature to and output

[0050] Step 3.2: Construct a convolutional attention module, which consists of 1 one-dimensional convolutional layer with a kernel length of 7, 1 batch normalization layer, 1 ReLU activation function, 1 one-dimensional convolutional layer with a kernel length of 7, 1 batch normalization layer, and a one-dimensional spatial and channel attention module in sequence; The module applies the calculation process of the structure in sequence, and applies residual connection;

[0051] Step 3.3: Use the structure constructed in step 3.2 to construct a 3-stage hierarchical self-attention network, and use the network to extract features from the output of step 3.1 The features of each stage are:

[0052] Stage 1: The structure of this stage consists of 3 convolutional attention modules described in step 3.2; the output of step 2.1 After sequentially operating the structure, the output is

[0053] Stage 2: The structure of this stage consists of 6 convolutional attention modules described in step 3.2; the output of stage 1 After sequentially operating the structure, the output is

[0054] Stage 3: The structure of this stage consists of 6 convolutional attention modules described in step 3.2. The output of stage 2 After sequentially operating the structure, the output is

[0055] Step 4: Fuse the features extracted in steps 2 and 3, and output the model classification prediction result, which includes the following steps:

[0056] Step 4.1: Firstly, layer normalization is performed on the outputs of step 2 and step 3 respectively, and then dimension reduction is performed by generalized average pooling to obtain and respectively. and Then, Y p and Z p are spliced to obtain the fusion feature

[0057] Step 4.2: The output of step 4.1 is finally outputted by a fully connected layer and a sigmoid function to obtain a classification result.

[0058] The application proposes a multi-lead electrocardiogram classification and recognition method based on convolution and self-attention mechanism, which mainly includes a hierarchical self-attention network and a convolution attention network. The hierarchical self-attention network can capture long-range dependence features across multiple heartbeats by using the self-attention mechanism. The lead sensitive mechanism uses a window-based self-attention mechanism to calculate self-attention scores from two views of time dimension and lead dimension, so as to focus on the specificity differences of different lead electrocardiogram signals. The convolution attention network uses the inductive bias that the local electrocardiogram waveform is invariant to time translation, uses convolution to extract local waveform features, and focuses on local key waveform information. The application can achieve the highest accuracy in the multi-lead electrocardiogram multi-label classification task. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is an example diagram of the model structure in the method proposed by the application, which mainly includes a hierarchical self-attention network and a convolution attention network.

[0060] Figure 2 is an example diagram of a window-based Transformer module structure in the model constructed by the application.

[0061] Figure 3 is an example diagram of the window-based multi-head self-attention mechanism proposed by the application.

[0062] Figure 4 is an example diagram of the convolution attention module constructed by the application.

[0063] Figure 5 is an example diagram of an embodiment. DETAILED DESCRIPTION

[0064] The technical solutions of the application will be further described below with reference to the drawings.

[0065] Figure 1 is an example diagram of the model structure in the method proposed by the application, which mainly includes a hierarchical self-attention network and a convolution attention network.

[0066] Traditional convolutional network-based methods are difficult to capture the global view, so the present application is based on the Transformer model, and a hierarchical self-attention network is designed. Transformer is a model based on self-attention mechanism, which guides the model to pay more attention to the features that are more critical and relevant to improving the prediction effect. Transformer calculates the correlation weight between all feature vectors, so it can extract long-range dependencies farther than convolutional neural network (CNN) and long short-term memory network (LSTM).

[0067] The hierarchical self-attention network proposed in the present application not only introduces a multi-level structure design, but also designs a lead sensitive mechanism for multi-lead electrocardiogram signals, so it can not only capture multi-scale features in the time dimension in the global view, but also pay attention to key information on each lead, fully mining the characteristics of multi-lead electrocardiogram signals from different dimensions of time and lead.

[0068] Because the electrocardiogram signal is a long time sequence, directly feeding the original electrocardiogram signal sequence into the Transformer will be infeasible due to the excessively long sequence. Therefore, the present application designs an electrocardiogram embedding layer to divide the electrocardiogram signal into several non-overlapping segments. However, unlike the method of concatenating the values of the RGB channels in the image segment and then dividing it, which is used in the field of image processing and its variants, the method proposed in the present application divides the different lead channels into different electrocardiogram segments. This segmentation method preserves the independence of each lead electrocardiogram signal, and creates conditions for the lead-sensitive self-attention mechanism based on the window to pay attention to the key information of a specific lead. Specifically, for a given input The signal of each lead is divided into non-overlapping segments, and each segment contains 4 sampling points, so after segmentation segments will be generated. Figure 3 (a) shows an example of electrocardiogram segment division when the number of leads m = 4 and inch, an example of electrocardiogram segment division. Then a linear layer is used to embed each segment into a feature vector with a length of C The above segmentation and embedding operations can be replaced by a two-dimensional convolution layer with a convolution kernel size and a step of 1 x C. Finally, layer normalization is used to standardize the embedded features.

[0069] The window-based Transformer module proposed in the application replaces the continuous standard Transformer multi-head attention mechanism with a time-sensitive window-based multi-head self-attention mechanism, a lead-sensitive window-based multi-head self-attention mechanism and a time-sensitive moving window-based multi-head self-attention mechanism. Thus, two window-based Transformer modules are constructed: a time-lead Transformer module and a time Transformer module, which respectively contain 2 and 3 continuous sub-modules. As shown in Figure 2 Fig. (a) is a time-lead Transformer module, and Fig. (b) is a time Transformer module; each sub-module contains a window-based multi-head self-attention mechanism, followed by a 2-layer multi-layer perception with GELU as the activation function. In each module, layer normalization is used before the multi-head self-attention mechanism and the multi-layer perception, and residual connection is used in each layer.

[0070] Due to the large amount of calculation and memory consumption, the standard global multi-head self-attention mechanism is not suitable for electrocardiogram segments. Therefore, the multi-head self-attention mechanism is calculated in the non-overlapping window in the application. In addition, an induction bias specific to the electrocardiogram field is introduced, that is, a cardiologist usually pays attention to the changes of the waveform on the same lead over time or the relationship between electrocardiogram waveforms on different leads at the same time. For example, a doctor will observe the interval between the peaks of two adjacent R waves on the same lead to determine whether tachycardia or bradycardia occurs, and will also pay attention to whether the P wave is upright on lead II and inverted on lead AVR at the same time to determine whether atrioventricular block occurs. In order to introduce the above induction bias, a one-dimensional window is used in the time and lead dimensions respectively, and the multi-head self-attention mechanism is calculated in the window. Assuming that the input of the current module is LxT segments, the size of the window is 1xT' in the time-sensitive window-based multi-head self-attention mechanism, and the size of the window is Lx1 in the lead-sensitive multi-head self-attention mechanism. Then in the time-sensitive window-based multi-head self-attention mechanism, the input segments are divided into non-overlapping one-dimensional windows; in the lead-sensitive multi-head self-attention mechanism, the input segments are divided into 1xT non-overlapping one-dimensional windows. As shown in Figure 3 (a) (b) (c) shows (a) an electrocardiogram segment example, (b) a time-sensitive window-based multi-head self-attention mechanism, and (c) a lead-sensitive window-based multi-head self-attention mechanism; for an input of 4x16 segments, in the time-sensitive window-based multi-head self-attention mechanism, it is divided into 4x4=16 windows with a size of 1x4, and in the lead-sensitive multi-head self-attention mechanism, it is divided into 1x16 windows with a size of 4x1.

[0071] Because window-based self-attention is computed only within each non-overlapping window, time-sensitive window-based multi-head self-attention mechanisms lack information exchange between different time windows, limiting the model's ability to model long-range dependencies. This invention designs a one-dimensional time-shifting window mechanism to achieve feature extraction across time windows. This mechanism is called a time-sensitive moving-window-based multi-head self-attention mechanism. Given an input of L×T segments and a window of size 1×T′, the window partitioning strategy mentioned in the time-sensitive window-based multi-head self-attention mechanism will partition the input into... A non-overlapping one-dimensional window. Next, the partitioning window is translated along the time dimension. A fragment. Figure 3 (d) illustrates an example of a time-sensitive, moving-window-based multi-head self-attention mechanism; for an input of size 4×16 segments, the time-sensitive, window-based multi-head self-attention mechanism divides it into 4×4 windows of size 1×4 segments. This is achieved by translating along the time dimension. After a few segments, the number of windows becomes 4 × 5 = 20, and the window size also changes to 1 × 4 or 1 × 5. However, because the window sizes are not consistent, calculating the multi-head self-attention mechanism in the new windows cannot be accelerated by batch computation, and loop computation is inefficient. Therefore, a left-leaning loop-shifting approach is adopted to achieve batch computation, such as... Figure 3 As shown in (d), after cyclic movement along the time dimension, the rightmost window contains two previously non-adjacent sub-windows. A masking mechanism is employed to avoid calculating self-attention between different sub-windows. After cyclic movement, the number of windows is the same as in time-sensitive window-based multi-head self-attention mechanisms, and the window sizes are consistent, thus enabling efficient batch computation. After self-attention computation, the cyclically moved segment is moved back to its original position.

[0072] In order to construct hierarchical representations, such as Figure 1 As shown, this invention uses a time-segment merging layer to merge adjacent segments in the time dimension at the end of stages 1, 3, and 4, and a lead-segment merging layer to merge information extracted from all leads at the end of stage 2. In the time-segment merging layer, features of time-adjacent segments on the same lead are concatenated, and then normalized by a layer. Therefore, the number of segments in the time dimension is halved, while the feature dimension of each segment is doubled. In the lead-segment merging layer, the segment features from all leads at the same time point are concatenated, then passed through a fully connected layer to reduce the feature dimension back to the level before concatenation, and finally passed through a normalization layer. Therefore, the number of segments in the lead dimension is reduced to 1, while the number of segments in the time dimension remains unchanged.

[0073] Because the self-attention mechanism itself does not consider position information, adding hybrid position encoding when selecting the self-attention mechanism solves the problem of lacking position information. In the lead dimension, use the learnable absolute position encoding adopted in ViT, BERT, and in the time dimension, use the relative position encoding adopted in SwinTransformer.

[0074] Convolution-based models are widely used in ECG classification tasks, and many state-of-the-art works have emerged and shown good performance. Convolution operation has good locality features and can extract local waveform patterns. Similarly, images have translational invariance in space, and waveforms also have translational invariance in time, so convolution can capture the special waveform pattern of ECG signal sequences. The present application uses a convolutional attention network to extract local waveform features, which follows a similar structure to the convolutional residual network and uses spatial and channel attention modules. The convolutional residual network model introduces residual connections, making it easier to train deeper neural networks; spatial and channel attention modules can make the model pay more attention to important features in the spatial and channel dimensions. Since ECG signals are a kind of time series data, all two-dimensional convolution layers are replaced with one-dimensional convolution layers, and the two-dimensional spatial attention feature map in the spatial and channel attention module is replaced with a one-dimensional time attention feature map.

[0075] The convolutional attention network structure is shown in Figure 1 For a given ECG signal input First, use a convolutional layer with a kernel length of 15 and a step of 2 to downsample the input to Then use a batch normalization layer and a ReLU activation function. Subsequently, a max pooling layer with a kernel length of 3 and a step of 2 further reduces the dimension of the feature to a dimension of Next, it is sent to 3 one-dimensional convolutional residual networks and spatial and channel attention modules containing 3, 4, 6, and 3 groups, respectively. As shown in Figure 4 Each one-dimensional convolutional residual network and spatial and channel attention module consists of two one-dimensional convolutional layers with a kernel length of 7, a ReLU activation function, two batch normalization layers, a spatial and channel attention module, and a residual connection. Finally, the output feature vector is subjected to L2 regularization. The output of this module is a 256-channel feature map.

[0076] After feature extraction by the hierarchical self-attention network and the convolutional attention module, the features containing global features and lead information and the local waveform features are obtained. In order to fuse the two types of features, the features are reduced to Y' and Z' are then concatenated to obtain the fused feature

[0077] Generalized mean pooling can adaptively choose the appropriate pooling method between max pooling and average pooling. Assuming that the feature is to be pooled in the last dimension, the generalized mean pooling is calculated as follows:

[0078]

[0079] where g i represents the i-th value of the output after pooling, and p is a learnable pooling exponent parameter. When p = 1, the generalized mean pooling is equivalent to the average pooling; when p approaches 0 infinitely, its behavior is approximately the same as the maximum pooling.

[0080] Embodiments

[0081] Figure 5 (a) and (b) are an electrocardiogram input when the method of the present application is applied. The electrocardiogram is an eight-lead electrocardiogram, and the eight leads are I, II, V1, V2, V3, V4, V5, and V6 leads, respectively. Figure 5 (c) is Figure 5 (a) is an enlarged view of the V3 lead. Its sampling rate is 500 Hz, the time length is 10 seconds, and there are 5000 sampling points in total. Its heart rhythm type label is "sinus rhythm, T wave change". After resampling it to 2048 sampling points, it is input into the model constructed by the present application, and the output prediction result is "sinus rhythm, T wave change", which is correct. By observing Figure 5 , it is found that "T wave change" only occurs in the V3 lead. In the region marked by the circle, the normal T wave changes to "camel T wave". However, the T wave in other leads is normal. That is, the model needs to accurately capture the change information of the electrocardiogram signal on different leads. The lead-sensitive mechanism proposed by the present application is able to focus on important information on different leads through the lead-sensitive window-based multi-head self-attention mechanism. Therefore, the model proposed by the present application can acutely capture the change of the T wave in the V3 lead in the example.

[0082] Comparative Example

[0083] The multi-lead electrocardiogram classification and recognition method proposed by the present application is studied using a public electrocardiogram dataset, and compared with existing literature and work to demonstrate the effect of the present application.

[0084] Experimental Dataset

[0085] The dataset used for the electrocardiogram multi-label classification task has two, both of which are publicly available multi-lead multi-label electrocardiogram datasets on the Internet. The number of label categories, data volume, data division, length, lead number, and sampling rate of the effective data are shown in Table 1.

[0086] Table 1

[0087]

[0088] Where dataset A comes from a certain competition dataset, and dataset B comes from a certain university collected dataset. They are divided into training set, validation set and test set by stratified sampling according to categories, and the number is shown in Table 1.

[0089] In the implementation of the present comparative example, each piece of data is resampled to 2048 sampling points. For dataset A, the number of leads m is 8; for dataset B, the number of leads m is 12. The segment embedding feature dimension in the electrocardio segment embedding layer is set to C = 96, and the feature dimension in each attention head is taken as a fixed value of 8, so the number of attention heads in the 5 stages is [12, 24, 24, 48, 96] respectively. The window size in the time-sensitive window-based multi-head attention mechanism is set to 1 x T = 1 x 32. In order to alleviate the problem of overfitting, Dropout is used after the ReLU activation function of the one-dimensional convolution residual network and the spatial and channel attention module, and random depth is applied to each block in the Transformer and convolution module. The training loss function adopts a weighted binary cross-entropy function, which can to some extent alleviate the problem of unbalanced training sample data by increasing the loss weight for the category with less sample data. Specifically, given the predicted output and label The weighted binary cross-entropy loss function is calculated as follows:

[0090]

[0091]

[0092] Where w i represents the weight of the i-th heart rhythm type, Loss WBCE represents the loss function value, u represents the number of label categories, and count(i) represents the number of samples belonging to the i-th label category in the training set. The model training optimizer selects Adam, performs learning rate decay in a fixed number of training rounds, and loads the model with the highest micro-average F1 value on the validation set so far to continue training. The model with the highest micro-average F1 value on the validation set is selected as the final model, and the training is ended after 128 training rounds. Finally, the prediction classification result threshold is 0.5.

[0093] The classification and recognition method proposed in the present application is compared with the following five end-to-end electrocardio classification and recognition methods to verify the effectiveness of the method.

[0094] (1) 16-layer CNN: The classification method for 10-second ECG proposed in the paper "Arrythmia detection using deep convolutional neural network with long duration ECG signals" published in 2018, a 16-layer convolutional neural network based on one-dimensional convolution;

[0095] (2) ECGNet: A multi-scale one-dimensional convolution and long short-term memory network (LSTM) fusion network for arrhythmia classification proposed in the paper "Deep Network for Arrhythmia Classification" published in 2018;

[0096] (3) DDNN: A twelve-lead atrial fibrillation classification model based on DenseNet and channel attention module proposed in the paper "Accurate detection of atrial fibrillation from 12-lead ECG using deep neural network" published in 2020

[0097] (4) ResNet-1D: A one-dimensional convolutional network based on ResNet for automatic diagnosis of twelve-lead ECG proposed in the paper "Automatic diagnosis of the 12-lead ECG using a deep neural network" published in 2020;

[0098] (5) STCT: A convolution and Transformer combined model proposed in the paper "STCT: Spatial-Temporal Conv-Transformer Network for Cardiac Arrhythmias Recognition" published in 2022, which can use one-dimensional convolutional network to extract spatial information, use CvT to extract time information, and thus extract spatiotemporal features in ECG signals for cardiac arrhythmia detection task.

[0099] In this comparative example, the micro-average F1 value is used to evaluate the model effect, which is defined as formulas (19)-(21). Where TP i , FP i , FN i represent the number of true positives, false positives, and false negatives for the i-th class, respectively, and u represents the number of label types.

[0100]

[0101]

[0102]

[0103] Table 2

[0104]

[0105] The experimental results of the method proposed in the present application and five baseline methods on two public data sets are shown in Table 2, LHTC-Net represents the method proposed in the present application, and the indicators shown in the table are micro-average F1 values. It can be observed that the performance of the method proposed in the present application on the two data sets exceeds all five baseline methods, verifying the effectiveness thereof.

Claims

1. A multi-lead electrocardiogram classification and recognition method based on convolution and self-attention mechanisms, characterized in that, The electrocardiogram classification and recognition method includes: Step 1: Preprocess the multi-lead ECG data; assuming... X represents multi-lead ECG data with a sequence length of N and a number of leads of m, i.e., X is a multivariate time series, represented as a two-dimensional matrix as shown in formula (1); data preprocessing specifically includes the following steps: Step 1.1: Use a high-pass filter to filter out signals with frequencies in the range of 0Hz to 0.5Hz; Step 1.2: Then, Fourier transform is used to convert the multi-lead ECG data. The resampling is a time series of length n, and the output after resampling is: Where n is any positive integer that is a multiple of 32; Step 2: Employing a hierarchical self-attention network consisting of multiple stages, feature vectors containing long-range cross-beat dependence and specific lead characteristics are extracted from the ECG signal data output after Step 1 processing; specifically including the following steps: Step 2.1: The ECG signal output after processing in Step 1 First, the ECG segment embedding layer is used to divide the ECG signal into non-overlapping segments. Each segment contains 1×4 sampling points; then, a linear layer embeds each segment into a feature vector; the length of the embedded feature vector is the hyperparameter C, which can be set to any positive integer; the output of this step is... Step 2.2: Add absolute position encoding in the lead dimension; adjust the output of Step 2.

1. Add a learnable parameter Right now in and This represents the input and output of the j-th time segment of the i-th lead during the absolute position encoding process. It is the absolute position encoding parameter of the i-th lead; Step 2.3: Construct two window-based Transformer modules; replace the multi-head self-attention mechanism in the standard Transformer module with a window-based multi-head attention mechanism, which only computes within the window; the specific construction steps are as follows: Suppose a window contains l segments. The computation process within each window using the window-based multi-head self-attention mechanism is as follows: Q i = IS i Q ,K i = IS i K ,V i = IS i V ,1≤i≤h (2) in The input is a window containing l segments, each segment having a feature dimension of d. Q represents the output of a window after calculation. i K i and This represents the query, key, and value; h represents the number of self-attention heads. i W represents the i-th attention head. i Q W i K , and These are learnable weight parameters. Concat represents vector concatenation, and Attention represents self-attention. Based on the aforementioned mechanism, three window-based multi-head self-attention mechanisms are constructed according to different window partitioning methods: First, let the input of the mechanism consist of L×T segments, where L and T represent the number of segments in the lead dimension and time dimension, respectively. Windows are divided in the time dimension, with each window having a size of 1×T′, where T′ is a hyperparameter satisfying T′≤T, representing the length of the window in the time dimension. The input segments will be divided into... First, a non-overlapping one-dimensional window is used, and then a multi-head self-attention mechanism is computed within the window; this is called a time-sensitive window-based multi-head self-attention mechanism. Second, assuming the mechanism input is L×T segments, the window is divided along the lead dimension, with a window size of L×1. The input segments are divided into 1×T non-overlapping one-dimensional windows, and then a multi-head self-attention mechanism is computed within the window; this is called a lead-sensitive window-based multi-head self-attention mechanism. Third, after dividing the window using the time-sensitive window-based multi-head self-attention mechanism described in the first step, the window is cyclically moved to the left along the time dimension. The multi-head self-attention mechanism is then computed within the moved window; this is called a time-sensitive moving window-based multi-head self-attention mechanism. For the time-sensitive window-based multi-head self-attention mechanism and the time-sensitive moving window-based multi-head self-attention mechanism, a relative position encoding in the time dimension is added, that is, the calculation process of multi-head self-attention is replaced by formula (5) instead of formula (3): Among them B i The value in is from Selected from It represents a learnable relative position encoding parameter in the time dimension, where the relative position of two segments is within the range of [-l+1, l-1]. Based on the three window-based multi-head self-attention mechanisms mentioned above, the following two window-based Transformer modules are constructed respectively: Firstly, it is divided into 3 sub-modules. The calculation process of the 3 consecutive sub-modules is shown in formulas (6)-(11), where and Y b These represent the outputs of the window-based attention layer and the multilayer perceptron (MLP) for submodule b, respectively. LN represents batch normalization, TW-MSA represents the time-sensitive window-based multi-head self-attention mechanism, LW-MSA represents the lead-sensitive window-based multi-head self-attention mechanism, and TSW-MSA represents the time-sensitive moving window-based multi-head self-attention mechanism, collectively referred to as the time-lead Transformer module. Secondly, it is divided into two sub-modules. The calculation process of the two consecutive sub-modules is shown in formulas (12)-(15). The meaning of the symbols in the formulas is the same as that in the aforementioned time-lead Transformer module; it is called the time Transformer module. Step 2.4: Construct a layer that merges the two types of fragments; Firstly, for an input containing L×T segments, let the feature dimension of each segment be C′. The features of every two adjacent segments on the same lead are concatenated, and then the resulting new features are subjected to layer standardization. After this layer operation, the number of segments in the time dimension changes from T to... The feature dimension of each segment changes from C′ to 2C′; this is called the time segment merging layer. Secondly, for an input containing L×T segments, let the feature dimension of each segment be C′. The segments of all leads at each time point are concatenated. After concatenation, the number of segments in the lead dimension is reduced from L to 1, and the feature dimension of each segment is changed from C′ to L×C′. Then, a fully connected layer is used to reduce the feature dimension from L×C′ back to C′, and finally, layer standardization is performed on the new features; this is called the lead segment merging layer. Step 2.5: Using the structures built in Steps 2.4 and 2.5, construct a 5-stage hierarchical self-attention network, and use this network to extract the output from Step 2.

1. Features are extracted from it, and the features at each stage are as follows: Phase 1, the structure of which consists of one time-lead Transformer module as described in step 2.3 and one time-segment merging layer as described in step 2.4; the output of step 2.1 After undergoing the structural operations in sequence, the output is: Phase 2, the structure of which consists of two time-lead Transformer modules as described in step 2.3 and one lead segment merging layer as described in step 2.4; the output of Phase 1 After undergoing the structural operations in sequence, the output is: Phase 3, the structure of which consists of three time Transformer modules as described in step 2.3 and one time segment merging layer as described in step 2.4; the output of Phase 2 After undergoing the structural operations in sequence, the output is: Phase 4, this phase consists of one time Transformer module as described in step 2.3 and one time segment merging layer as described in step 2.4; the output of phase 3. After undergoing the structural operations in sequence, the output is: Stage 5, the structure of which consists of one Time Transformer module as described in step 2.3; the output of Stage 4 After undergoing the structural operations in sequence, the output is: Step 3: Using a multi-stage convolutional attention network, local waveform features are extracted from the ECG signal data output after Step 1, and a 256-channel feature map is output. Specifically, the following steps are included: Step 3.1: Input the ECG signal output from Step 1. First, the input size is downsampled using a convolutional layer with a kernel length of 15 and a stride of 2. Then, batch normalization and ReLU activation are applied; subsequently, max pooling with a kernel length of 3 and a stride of 2 is used to further reduce the dimensionality of the features and output them. Step 3.2: Construct a convolutional attention module, whose structure consists of a one-dimensional convolutional layer with a kernel length of 7, a batch normalization layer, a ReLU activation function, a one-dimensional convolutional layer with a kernel length of 7, a batch normalization layer, and a one-dimensional spatial and channel attention module in sequence; when this module is applied, the calculation process of the above structure is executed sequentially, and residual connections are applied. Step 3.3: Using the structure constructed in Step 3.2, build a 3-stage hierarchical self-attention network, and use this network to extract the output from Step 3.

1. Features are extracted from it, and the features at each stage are as follows: Phase 1: This phase consists of three convolutional attention modules described in step 3.2; the output of step 2.1 After undergoing the structural operations in sequence, the output is: Stage 2: This stage consists of six convolutional attention modules as described in step 3.2; the output of Stage 1. After undergoing the structural operations in sequence, the output is: Stage 3: This stage consists of six convolutional attention modules as described in step 3.2; the output of Stage 2. After undergoing the structural operations in sequence, the output is: Step 4: Combine the features extracted in Step 2 and Step 3, and output the model classification prediction results. This includes the following steps: Step 4.1: First, regarding the output of Step 2... and the output of step 3 After performing layer standardization, the dimensionality is reduced to generalized average pooling. and Then Y p and Z p The splicing results in fusion features Step 4.2: The output of Step 4.1 is finally passed through a fully connected layer and a Sigmoid function to output the classification result.

Images