An electrocardio multi-modal contrast learning method for single-lead arrhythmia diagnosis

Abstract

The application discloses a kind of electrocardiogram multi-modal contrast learning methods for single-lead arrhythmia diagnosis, it is related to arrhythmia diagnosis technical field.The present application at least includes the following steps: S1: build pre-training framework, pre-training framework at least includes contrast time series learning module and contrast image learning module;S2: data preprocessing is carried out, signal slice is first recorded and is cut into consistent length segment, to ensure the network is simplified and local feature is strengthened;S3: contrast time series learning module is defined and applied;S4: contrast image learning module is defined and applied;S5: design multi-modal consistency loss.The present application realizes unlabelled data on unsupervised pre-training, from which the information effective to target task is mined, and further this utilizes gram angle field and converts electrocardiogram sample into image data, to utilize the advantage of CNN in image feature extraction, so as to more comprehensively mine the precious information contained in electrocardiogram signal.

Description

Technical Field

[0001] This invention relates to the field of cardiac arrhythmia diagnosis technology, specifically to an electrocardiogram multimodal contrastive learning method for diagnosing single-lead cardiac arrhythmias. Background Technology

[0002] Cardiac arrhythmia is a common cardiovascular syndrome. Abnormalities in the origin, conduction, or other parts of cardiac electrical stimulation can cause arrhythmia, which is a major challenge in the clinical treatment of cardiovascular diseases.

[0003] An electrocardiogram (ECG) is a record of the electrical activity of countless myocardial cells, reflecting the electrophysiological processes of the heart. ECG is a primary diagnostic tool for arrhythmias; different types of arrhythmias exhibit different waveforms and frequencies on the ECG. Timely detection of arrhythmias through effective measures plays a significant role in the prevention and diagnosis of cardiovascular diseases, and has crucial clinical significance. Self-supervised learning provides a pre-training strategy that can extract valuable information from unlabeled data.

[0004] Sarkar et al. performed six different signal transformations on electrocardiogram (ECG) signals and used transformation recognition as the given task to develop a self-supervised learning framework for emotion recognition based on ECG. Contrastive learning is a special type of self-supervised learning method that aims to learn an encoder that maps the input to a feature space, thereby bringing positive samples closer together while increasing the distance between negative sample pairs through contrastive loss.

[0005] Oord et al. proposed Contrastive Predictive Coding (CPC), which predicts the representation of future segments by extracting latent features from the current segment.

[0006] Tian et al. proposed a contrastive multi-view learning framework that can learn unsupervised representations from multiple views of a dataset. Based on the principle of maximizing mutual information, it treats multiple views of the same sample as a “shared context”.

[0007] MoCo, SimCLR, and BYOL used instance discrimination as a prerequisite task, employing different random augmentations from the same instance as query and key images. They explored the advantages of data augmentation and showed that random cropping and color perturbation were the most effective.

[0008] However, these data augmentation methods cannot be easily extended to the time series domain. With recent advances in self-supervised learning in fields such as computer vision and natural language processing, some studies have also explored the application of contrastive learning in the field of electrocardiography.

[0009] Cheng et al. introduced subject-specific learning through contrastive loss and adversarial training for specific topics to promote subject invariance in the self-supervised learning process, but did not study its application on external unlabeled datasets.

[0010] Kiyasseh et al. utilized the time and space invariance of electrocardiograms (ECGs) to redefine positive samples as different ECGs of the same patient (e.g., ECGs in different leads and ECGs at different time intervals), and proposed a patient-level contrastive learning method. However, this method requires labeled patient IDs in the pre-trained dataset, limiting its applicability.

[0011] Soltanieh et al. explored the effectiveness of various enhancement methods in contrastive self-supervised learning of electrocardiogram signals.

[0012] Chen et al. proposed an instance-level contrastive learning scheme by using random pruning and wavelet transform for contrastive learning enhancement. Zhang et al. proposed a self-supervised contrastive pre-training framework for time series by utilizing the time-frequency consistency of time series signals.

[0013] In recent years, deep learning methods have been applied in many fields, achieving significant success in the diagnosis of electrocardiogram (ECG) arrhythmias. Currently, most deep learning-based ECG classification methods focus on designing network architectures for ECG signals to extract more expressive representations. However, these methods are limited by the size of the labeled dataset.

[0014] Although the development of wearable ECG devices generates a large amount of unlabeled ECG data daily, the high cost of labeling data (requiring professional medical personnel) makes building large, high-quality datasets extremely difficult. This results in the underutilization of this data. Supervised deep learning ECG classification models also suffer from severe overfitting due to insufficient training data. While some methods have achieved some success in unsupervised pre-training of ECG data, their accuracy in diagnosing arrhythmias using ECG signals still needs improvement. Summary of the Invention

[0015] The purpose of this invention is to provide a multimodal contrastive learning method for ECG for the diagnosis of single-lead arrhythmias, in order to solve the existing problem: although some methods have achieved some success in unsupervised pre-training of ECG, their accuracy in the diagnosis of arrhythmias using ECG signals still needs to be improved.

[0016] To achieve the above objectives, the present invention provides the following technical solution: a multimodal contrastive learning method for diagnosing single-lead arrhythmias, comprising at least the following steps:

[0017] S1: Build a pre-training framework, which should include at least a contrastive temporal learning module and a contrastive image learning module;

[0018] S2: Perform data preprocessing. For signal slicing, first, randomly cut the records into segments of the same length to ensure network simplification and enhance local features;

[0019] S3: Define and apply the comparative time series learning module;

[0020] S4: Define and apply the contrastive image learning module;

[0021] S5: Design multimodal consistency loss.

[0022] Preferably, the pre-trained framework uses the BYOL architecture, and the pre-trained framework includes weights... Defined online networks and weights Defined target network;

[0023] The weight The defined online network includes encoders. Projector head and predictor ;

[0024] The weight The defined target network includes the encoder. and projector head ;

[0025] Apart from the predictor, the online network and the target network have the same architecture, but use different sets of weights;

[0026] The target network provides an online network for training regression targets, and the parameters of the target network are... The exponential moving average formula is used for updating, given the target decay rate. , The updated formula is:

[0027] .

[0028] Preferably, the random pruning in S2 is a random data transformation, and the random data transformation is the construction of an electrocardiogram signal enhancement transformation library. The random data transformation includes four different data transformation methods, which are as follows:

[0029] Signal scaling The electrocardiogram (ECG) signal is multiplied by a random scalar in a Gaussian distribution, thereby changing the magnitude of the signal value. Let represent the mean and variance of the Gaussian distribution, respectively.

[0030] Segmented random cropping First, the signal is divided into k equal segments, each containing n signal points. Then, within each segment, consecutive signals are randomly selected... Set each signal point to zero, among which The proportion of random cropping;

[0031] Random permutation To shuffle the position of a signal, the signal is first sliced ​​into n equal segments, and then these segments are rearranged to create a new signal. Random permutation is a simple way to randomly shuffle the temporal position of events within a window.

[0032] Add Gaussian noise Standard deviation of noise It is a hyperparameter that needs to be determined in advance.

[0033] Preferably, step S3 includes at least the following steps:

[0034] For a given electrocardiogram signal sample Through ECG signal enhancement library B: Generate an augmented dataset ;

[0035] Enhanced View It is an electrocardiogram signal. It is derived through enhancement transformation;

[0036] Here, an enhancement library was developed to generate multiple enhancements instead of a single type, exposing the model to complex temporal dynamics, resulting in more robust embeddings;

[0037] For ECG signal samples Randomly select an enhanced view Then enhance the view Input target network Received and mapping ;

[0038] Will Enter online network Received Mapping output and predicted output = , respectively and L2 regularization yields: , ;

[0039] Compared with temporal loss, the regularized prediction output is... Mapping output to the target network The mean square error between them is used as the loss. The specific calculations are as follows:

[0040]

[0041] Similarly, by Enter the online network and The loss obtained from the input target network ;

[0042] The comparison time-series loss function is used with samples The unit is defined as:

[0043] .

[0044] Preferably, step S4 includes at least the following steps:

[0045] After converting ECG signal data into image data, we fully utilize the advantages of CNN in image feature extraction to perform modeling;

[0046] For input ECG signal samples Using the Gram angle field to transmit electrocardiogram signals Convert to image ;

[0047] For images It randomly selects an enhancement method from the image enhancement library to generate its enhanced view. The image enhancement library includes at least random cropping, color jittering, and rotation;

[0048] Contrast image loss;

[0049] By Enter online network Obtain its representation , mapping and predicted output = ;

[0050] At the same time, Input target network Obtain its representation and mapping ;

[0051] With samples The mean squared error loss is obtained by setting the unit as follows:

[0052]

[0053] By Input target network ,Will Enter online network Loss Therefore, the contrast image loss is defined as follows:

[0054] .

[0055] Preferably, in step S5, the electrocardiogram sample is... The multimodal consistency loss calculation formula is defined as:

[0056] .

[0057] Compared with the prior art, the beneficial effects of the present invention are:

[0058] 1. This invention proposes multimodal contrastive learning of electrocardiograms, which enables unsupervised pre-training on unlabeled data to extract information effective for the target task, thereby effectively improving the detection performance of the single-lead arrhythmia diagnosis model.

[0059] 2. This invention utilizes Gram angle field to convert ECG samples into image data, thereby leveraging the advantages of CNN in image feature extraction to more comprehensively uncover the valuable information contained in ECG signals.

[0060] 3. This invention utilizes the property that the features of different modalities of the same electrocardiogram signal should be closer to those of the enhanced sample, and designs a multimodal consistency loss to measure the distance between embeddings extracted based on different modalities. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a schematic diagram of the ECG multimodal contrast pre-training framework of the present invention;

[0063] Figure 2 This is a schematic diagram illustrating a data transformation example of the present invention;

[0064] Figure 3 This is a schematic diagram comparing the transfer performance of the present invention with other pre-training methods under different training scales.

[0065] Figure 4 This is a schematic diagram of the model fine-tuned on the target dataset according to the present invention. Detailed Implementation

[0066] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0067] This invention proposes a novel ECG multimodal contrastive learning method for the diagnosis of single-lead arrhythmias, achieving unsupervised pre-training on unlabeled data to extract information relevant to the target task. The invention utilizes Gram angle fields to convert ECG samples into image data, leveraging the advantages of CNNs in image feature extraction to more comprehensively uncover the valuable information contained in the ECG signal. Furthermore, by utilizing the property that features from different modalities of the same ECG signal should be closer to those from augmented samples, a multimodal consistency loss is designed to measure the distance between embeddings extracted based on different modalities.

[0068] See Figures 1-2 A multimodal contrastive learning method for diagnosing single-lead arrhythmias includes at least the following steps:

[0069] S1: Build a pre-training framework, which should include at least a contrastive temporal learning module and a contrastive image learning module;

[0070] S2: Data preprocessing is performed, including signal slicing. Since ECG recordings are generally quite long and contain a large amount of information and various features, using the entire recording as network input would require configuring more convolutional kernels to extract this information, resulting in a very bloated network that is not conducive to focusing on local features. However, ECG signal abnormalities sometimes precisely manifest in some local changes. Therefore, for ECG signals, the recording is first randomly cropped into segments of uniform length to ensure network conciseness and enhance local features.

[0071] S3: Define and apply the comparative time series learning module;

[0072] S4: Define and apply the contrastive image learning module;

[0073] S5: Design multimodal consistency loss.

[0074] The pre-trained framework uses the BYOL architecture, which consists of weights. Defined online networks and weights Defined target network;

[0075] By weight The defined online network includes encoders. Projector head and predictor ;

[0076] By weight The defined target network includes the encoder. and projector head ;

[0077] Apart from the predictor, the online network and the target network have the same architecture, but use different sets of weights;

[0078] Target Network provides an online network for training regression targets. The parameters of the target network... Updates are performed using the exponential moving average (EMA) formula, given a target decay rate. , The updated formula is:

[0079] .

[0080] The random cropping in S2 is a random data transformation, which in turn involves building an ECG signal enhancement transformation library. Random data transformation includes four different data transformation methods, which are as follows:

[0081] Signal scaling ,like Figure 2 As shown in (b), the electrocardiogram signal is multiplied by a random scalar in a Gaussian distribution, thereby changing the magnitude of the signal value, where Let represent the mean and variance of the Gaussian distribution, respectively.

[0082] Segmented random cropping ,like Figure 2 As shown in (c), the signal is first divided into k equal segments, each containing n signal points. Then, within each segment, consecutive signals are randomly selected... Set each signal point to zero, among which The proportion of random cropping;

[0083] Random permutation ,like Figure 2 As shown in (d), in order to shuffle the position of the signal, the signal is first sliced ​​into n equal-length segments, and then these segments are rearranged to create a new signal. Random permutation is a simple way to randomly shuffle the temporal position of events within a window.

[0084] Add Gaussian noise ,like Figure 2 As shown in (e), the standard deviation of the noise It is a hyperparameter that needs to be determined in advance.

[0085] S3 includes at least the following steps:

[0086] For a given electrocardiogram signal sample Through ECG signal enhancement library B: Generate an augmented dataset ;

[0087] Enhanced View It is an electrocardiogram signal. It is derived through enhancement transformation;

[0088] Here, an enhancement library was developed to generate multiple enhancements instead of a single type, exposing the model to complex temporal dynamics, resulting in more robust embeddings;

[0089] For ECG signal samples Randomly select an enhanced view Then enhance the view Input target network Received and mapping ;

[0090] Will Enter online network Received Mapping output and predicted output = , respectively and L2 regularization yields: , ;

[0091] Compared with temporal loss, the regularized prediction output is... Mapping output to the target network The mean square error between them is used as the loss. The specific calculations are as follows:

[0092]

[0093] Similarly, by Enter the online network and The loss obtained from the input target network ;

[0094] The comparison time-series loss function is used with samples The unit is defined as:

[0095] .

[0096] S4 includes at least the following steps:

[0097] Gram angle fields can convert time-series data into image data, preserving both the complete information of the signal and its time dependence. After converting ECG signal data into image data, the advantages of CNNs in image feature extraction are fully utilized for modeling.

[0098] For input ECG signal samples Using the Gram angle field to transmit electrocardiogram signals Convert to image ;

[0099] For images It randomly selects an enhancement method from the image enhancement library to generate its enhanced view. The image enhancement library includes at least random cropping, color jittering, and rotation;

[0100] Contrast image loss;

[0101] By Enter online network Obtain its representation , mapping and predicted output = ;

[0102] At the same time, Input target network Obtain its representation and mapping ;

[0103] With samples The mean squared error loss is obtained by setting the unit as follows:

[0104]

[0105] By Input target network ,Will Enter online network Loss Therefore, the contrast image loss is defined as follows:

[0106] .

[0107] S5 will include ECG samples The multimodal consistency loss calculation formula is defined as:

[0108] .

[0109] The pushing process is as follows:

[0110] For the same ECG signal sample, the features extracted from its different modalities should be close to each other, that is, the features extracted by the timing encoder, the features extracted by the image encoder, and the features of their augmented samples should be close to each other.

[0111] To ensure that the distance between features is measurable, it is done separately through the projection head. , Representation from one-dimensional temporal space and representations from two-dimensional image space Mapped to a joint temporal-image space;

[0112] For each input ECG signal There are four embeddings, which are: , , and ;

[0113] The first two embeddings are representations extracted by a timing encoder. , The latter two embeddings are generated from representations extracted by the image encoder. , Generated;

[0114] Design a multimodal consistency loss term to measure the distance between time-based embeddings and image-based embeddings;

[0115] use express and The distance between them, and similar ones , , ;

[0116] The normalized temperature-scale cross-entropy loss (NT-Xent), widely used in contrastive learning, is adopted as the distance function. For example, the distance calculation formula is:

[0117]

[0118] in, Represents cosine similarity. Indicates temperature parameter, It is an index function, when When equal to 0, Time equals 1;

[0119] Embeddings representing different ECG signal samples or their enhanced samples;

[0120] Here, we will not consider and The distance between them, because they are features extracted from the same modality, is calculated in... This was already taken into account at the time, and the same applies to... and ;

[0121] for and In fact, it involves three embeddings: , and ;

[0122] in, , It is from the original samples respectively , I learned it from China, and This is from augmented samples Learned;

[0123] For the same ECG sample, the representation between two modalities should be closer to the representation with respect to the enhanced sample, i.e. It should be closer to Instead ;

[0124] Therefore, under the incentive of relative relationships, the model is encouraged to learn more than Smaller ;

[0125] Inspired by triplet loss, the consistency loss term is designed as follows:

[0126]

[0127] in It is a constant, and the optimization goal of this term is to bring it closer to... and The distance, while pushing away and The distance, that is, smaller. and relatively large ;

[0128] Similarly, It should also be smaller than and ;

[0129] In summary, the electrocardiogram sample The formula for calculating multimodal consistency loss is defined as follows:

[0130]

[0131] Although the formula for calculating consistency loss does not explicitly measure different ECG signal samples (e.g.) and The loss is implied in cross-sample relationships, but the cross-sample relationship is implicitly included. and In the calculation.

[0132] Based on the above embodiments, the following evidence demonstrates the technical effects:

[0133] 1. Experimental Results and Analysis

[0134] We wanted to verify the pre-training performance of this invention on a large-scale dataset, so we chose to pre-train on the largest open-source arrhythmia dataset to date. This dataset contains 12-lead electrocardiograms from 45,152 patients, with a sampling rate of 500 Hz and a sampling length of 10 seconds for each record, and includes various common rhythms and other cardiovascular diseases.

[0135] To demonstrate the generalization ability of our method, we fine-tuned it on the ICBEB2018 dataset. The ICBEB2018 dataset contains 6,877 12-lead ECG recordings, each ranging in length from 6 to 60 seconds. The dataset considers normal arrhythmias and eight abnormal arrhythmia categories, including AF, I-AVB, LBBB, PAC, PVC, RBBB, STD, and STE.

[0136] Considering that ICBEB2018 is a multi-label dataset, we use the AUC (Area under the Curve of ROC) metric.

[0137] AUC means the probability that, for any pair of positive and negative samples, the model predicts a higher score for the positive sample than for the negative sample. It does not focus on the model's predicted score for a particular class, but only on the ranking result, and therefore is not affected by a threshold, thus better reflecting the classifier's performance. We ultimately report the macro-average of AUC across the nine classes, namely:

[0138]

[0139] We first evaluated the transferability of our method by using the obtained pre-trained model as the initialization parameters for the training target dataset. In the experiments, SeResnet18 and DNN_Hannun were used as the backbone networks for the image encoder and temporal encoder, respectively. We compared the performance of fine-tuning our pre-trained model using the full, half, one-quarter, and one-eighth of the ICBEB2018 training data, and fine-tuning the supervised pre-trained model. The fine-tuning schemes are as follows: Figure 4As shown, we also present the training results with random initialization as a baseline, and demonstrate the performance of the state-of-the-art ECG pre-trained model CLECG under the same conditions. The results are as follows. Figure 3 As shown, our method outperforms other methods regardless of the size of the training data.

[0140] In addition, we explored the timing encoder. The specific implementation impacts our ECG multimodal contrast pre-trained model. Table 1 shows the experimental results using half the data scale. As can be seen from the table, the arrhythmia diagnosis task benefits from pre-trained models trained on large-scale datasets. Although classification performance varies with network architecture, compared to random initialization, our pre-trained models consistently improve AUC on the ICBEB2018 dataset to varying degrees. This indicates that our pre-trained model is a general-purpose architecture applicable to a range of network structures and possesses strong generalization ability.

[0141] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A multimodal contrastive learning method for electrocardiogram (ECG) used in the diagnosis of single-lead arrhythmias, characterized in that: S1: Build a pre-training framework. The pre-training framework includes at least a contrastive temporal learning module and a contrastive image learning module. The pre-training framework uses the BYOL architecture and includes a weighted... Defined online networks and weights Defined target network; by weights The defined online network includes encoders. Projector head and predictor ; by weight The defined target network includes the encoder. and projector head ; S2: Perform data preprocessing, slice the signal, and randomly cut the records into segments of the same length; S3: Define and apply the comparative time series learning module, including: For a given electrocardiogram signal sample Through ECG signal enhancement library B: Generate an augmented dataset ; Enhanced View From electrocardiogram signal It is derived through enhancement transformation; For ECG signal samples Randomly select an enhanced view Then enhance the view Input target network Received a representation and mapping ; Will Enter online network Received a representation Mapping and predicted output , respectively and Perform L2 regularization to obtain , ; The representation extracted by the timing encoder Generated; Calculate loss as follows: ; Similarly, by Enter the online network and The loss is calculated by inputting the target network. ; With samples Using units, the comparative time-series loss function is obtained as follows: ; S4: Define and apply the contrastive image learning module, including: After converting ECG signal data into image data, we fully utilize the advantages of CNN in image feature extraction to perform modeling; For ECG signal samples Using the Gram angle field to transmit electrocardiogram signals Convert to image For images The enhanced view is generated by randomly selecting an enhancement method from the image enhancement library. The image enhancement library includes at least random cropping, color jittering, and rotation; By Enter online network Received a representation Mapping and predicted output = ; At the same time, Input target network Received a representation and mapping ; , The representation extracted by the image encoder Generated; With samples The mean squared error loss is obtained by setting the unit as follows: ; By Input target network ,Will Enter online network Loss Therefore, the contrast image loss is defined as follows: ; S5: Design a multimodal consistency loss to measure the distance between time-based embeddings and image-based embeddings. The calculation formula is as follows: ; in, ,express and The distance between them can be calculated similarly. sim() represents the cosine similarity, and τ represents the temperature parameter; It is an index function that equals 0 when i = k and equals 1 when i ≠ k; z k ∈D represents the embedding generated from different ECG signal samples or their enhanced samples; δ>0 is a constant.

2. The ECG multimodal contrastive learning method for diagnosing single-lead arrhythmias according to claim 1, characterized in that: Apart from the predictor, the online network and the target network have the same architecture, but use different sets of weights; The target network provides an online network for training regression targets, and the parameters of the target network are... The exponential moving average formula is used for updating, given... , The updated formula is: 。 3. The ECG multimodal contrastive learning method for diagnosing single-lead arrhythmias according to claim 2, characterized in that: The random cropping in S2 is a random data transformation, which involves constructing an ECG signal enhancement transformation library. It includes four different data transformation methods, which include: Signal scaling The electrocardiogram (ECG) signal is multiplied by a random scalar in a Gaussian distribution, thereby changing the magnitude of the signal value. Let represent the mean and variance of the Gaussian distribution, respectively. Segmented random cropping First, the signal is divided into k equal segments, each containing n signal points. Then, within each segment, consecutive signals are randomly selected... Set each signal point to zero, among which The proportion of random cropping; Random permutation To scramble the signal's position, the signal is first sliced ​​into n equal-length segments, and then these segments are rearranged to create a new signal. Add Gaussian noise Standard deviation of noise It is a hyperparameter that needs to be determined in advance.

Images