Contrast-enhanced fusion ECG large model training deployment method and system

Abstract

The present invention provides a method and system for training and deploying a large-scale ECG model with contrast enhancement and fusion, comprising step S1: collecting ECG data and query information; step S2: preprocessing the ECG data to obtain preprocessed ECG data; step S3: extracting structured text labels from the preprocessed ECG data to construct an enhanced ECG dataset; step S4: training the model using the enhanced ECG dataset for contrast learning; step S5: analyzing the preprocessed ECG data and query information using the trained model and outputting the query results; and step S6: deploying the multimodal ECG basic model to a cloud server equipped with a GPU. By adopting a signal-language-label contrastive learning paradigm, the present invention addresses the problem that traditional contrastive learning, in multi-label classification tasks, introduces noisy positive examples that affect representation learning accuracy. This enhances the model's performance in complex classification tasks and offers promising application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of electrocardiogram (ECG) data processing, and in particular to a method and system for training and deploying a large-scale ECG model with contrast enhancement and fusion. Background Art

[0002] The electrocardiogram (ECG) is a crucial tool for clinical diagnosis of cardiovascular disease. By recording the heart's electrical activity, physicians can assess cardiac function and detect various heart conditions. Traditional ECG analysis relies primarily on physician experience and expertise, but this method is subject to significant subjectivity and inefficiency. In recent years, with the advancement of artificial intelligence (AI), automated ECG analysis methods based on deep learning have become a research hotspot.

[0003] However, existing deep learning methods mainly focus on single-modal data processing, ignoring the fusion of multimodal information, which limits the performance of the model. Traditional models also have problems such as difficulty in feature extraction, low accuracy, low efficiency, strong label dependence, and noisy positive examples.

[0004] Literature 1: [1] Liu C, Wan Z, Ouyang C, et al. Zero-Shot ECGClassification with Multimodal Learning and Test-time Clinical KnowledgeEnhancement[J].2024.

[0005] Literature 2: [1] Li J, Liu C, Cheng S, et al. Frozen Language Model HelpsECG Zero-Shot Learning[J].2023.

[0006] Although the two papers from Imperial College London also used ECG signals and medical text training models and achieved zero-shot diagnosis, they still used traditional self-supervised contrastive learning methods and did not propose a solution for the multi-label nature of the ECG field. This approach lacks discriminative information and also introduces noisy positive examples, which significantly reduces accuracy.

[0007] Document 3: ECG Semantic Integrator (ESI): A Foundation ECG ModelPretrainedwith LLM-Enhanced Cardiological Text

[0008] Although this paper from Rice University in the United States also proposed a knowledge enhancement method and used ECG signals and medical text to train a model, the architecture of this technology still uses the early CLIP architecture, which only performs comparative learning between the text encoder and the signal encoder, and does not build a label query. This simple architecture cannot fuse the aligned signals and disease text labels at the model level, resulting in its insufficient zero-sample diagnostic capabilities. This technology also only uses traditional comparative learning methods and lacks the introduction of discriminative information. Summary of the Invention

[0009] In view of the defects in the prior art, the purpose of the present invention is to provide a method and system for training and deploying a large ECG model with contrast enhancement and fusion.

[0010] A method for training and deploying a large ECG model with contrast enhancement and fusion according to the present invention includes:

[0011] Step S1: collecting original ECG data and querying information;

[0012] Step S2: preprocessing the original ECG data to obtain preprocessed ECG data;

[0013] The preprocessing includes denoising and normalization;

[0014] Step S3: extracting structured text labels from the preprocessed ECG data to construct an enhanced ECG dataset;

[0015] Step S4: using the enhanced ECG dataset to perform comparative learning and train a multimodal ECG basic model;

[0016] The model includes a signal encoder, a knowledge encoder, a label query network and a classification head;

[0017] Step S5: using the trained model to analyze the pre-processed ECG data and query information, and outputting the query results;

[0018] Step S6: Deploy the multimodal ECG basic model to a cloud server with a GPU.

[0019] Preferably, the original ECG data includes an ECG signal and an ECG report; and the query information includes whether a certain disease name can be derived from the ECG report.

[0020] Preferably, step S3 includes:

[0021] Structured text labels for morphology, rhythm, and diagnosis are extracted from ECG reports. Based on a large language model, medical terminology and background knowledge of disease diagnosis are added to obtain an enhanced ECG report. An enhanced ECG dataset is constructed by combining ECG signals, structured text labels, and enhanced ECG reports.

[0022] Preferably, the signal encoder encodes the original signal into a specific dimensional representation vector; the knowledge encoder encodes the electrocardiogram report or label text into a corresponding dimensional representation vector; the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result; the classification head receives the encoding of the label query result, and outputs the probability of the label existing in the electrocardiogram report.

[0023] Preferably, the step S4 includes:

[0024] Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label:

[0025]

[0026] in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

[0027] According to the present invention, a contrast-enhanced and fused ECG large model training and deployment system is provided, comprising:

[0028] Module M1: collects raw ECG data and queries information;

[0029] Module M2: preprocessing the original ECG data to obtain preprocessed ECG data;

[0030] The preprocessing includes denoising and normalization;

[0031] Module M3: Extract structured text labels from preprocessed ECG data and construct an enhanced ECG dataset;

[0032] Module M4: Use the enhanced ECG dataset for comparative learning to train a multimodal ECG basic model;

[0033] The model includes a signal encoder, a knowledge encoder, a label query network and a classification head;

[0034] Module M5: Use the trained model to analyze the preprocessed ECG data and query information, and output the query results;

[0035] Module M6: Deploy the multimodal ECG basic model to a cloud server with a GPU.

[0036] Preferably, the original ECG data includes an ECG signal and an ECG report; and the query information includes whether a certain disease name can be derived from the ECG report.

[0037] Preferably, the module M3 includes:

[0038] Structured text labels for morphology, rhythm, and diagnosis are extracted from ECG reports. Based on a large language model, medical terminology and background knowledge of disease diagnosis are added to obtain an enhanced ECG report. An enhanced ECG dataset is constructed by combining ECG signals, structured text labels, and enhanced ECG reports.

[0039] Preferably, the signal encoder encodes the original signal into a specific dimensional representation vector; the knowledge encoder encodes the electrocardiogram report or label text into a corresponding dimensional representation vector; the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result; the classification head receives the encoding of the label query result, and outputs the probability of the label existing in the electrocardiogram report.

[0040] Preferably, the module M4 includes:

[0041] Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label:

[0042]

[0043] in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. This invention adopts a signal-language architecture and uses text as a supervisory signal, effectively solving the problem that traditional models are limited by a fixed set of labels. By leveraging the rich semantic knowledge of large language models, different text labels with similar semantic relationships show similar relationships in the representation space, thereby further improving the model's zero-sample transfer capability.

[0046] 2. This invention solves the problem of traditional methods requiring time-consuming construction and alignment of knowledge bases by directly introducing external medical knowledge through large language models. It uses external enhanced medical knowledge to guide the model's representation learning, enabling it to significantly improve its diagnostic performance on different data sets. Furthermore, the model can make accurate judgment outputs when encountering categories it has never seen before.

[0047] 3. By adopting the signal-language-label contrastive learning paradigm (AugCL), this paper solves the problem that traditional contrastive learning introduces noisy positive examples in multi-label classification tasks, thereby affecting the accuracy of representation learning. AugCL effectively reduces the impact of noisy positive examples, solves the noise problem in multi-label settings, enhances the performance of the model in complex classification tasks, and has good application prospects.

[0048] 4. The present invention uses a large language model to enhance domain knowledge of ECG signals, achieving zero-sample diagnosis performance in multiple regions and multiple diseases, and providing doctors with better decision-making recommendations.

[0049] Other beneficial effects of the present invention will be explained through the introduction of specific technical features and technical solutions in the specific implementation methods. Those skilled in the art should be able to understand the beneficial technical effects brought about by the introduction of these technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0051] Figure 1 Flow chart of the method of the present invention.

[0052] Figure 2 This is a flowchart of constructing an enhanced ECG dataset in an embodiment of the present invention.

[0053] Figure 3 This is a flowchart of constructing a signal-language-label model in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0055] Reference Figure 1As shown, a method for training and deploying a large ECG model with contrast enhancement and fusion includes:

[0056] Step S1: collecting original ECG data and querying information;

[0057] The original ECG data includes an ECG signal and an ECG report, and the query information includes whether a certain disease name can be derived from the ECG report.

[0058] Step S2: preprocessing the original ECG data to obtain preprocessed ECG data;

[0059] The preprocessing includes denoising and normalization;

[0060] Step S3: extracting structured text labels from the preprocessed ECG data to construct an enhanced ECG dataset;

[0061] Structured text labels for morphology, rhythm, and diagnosis are extracted from ECG reports. Based on a large language model, medical terminology and background knowledge of disease diagnosis are added to obtain an enhanced ECG report. An enhanced ECG dataset is constructed by combining ECG signals, structured text labels, and enhanced ECG reports.

[0062] Step S4: using the enhanced ECG dataset to perform comparative learning and train a multimodal ECG basic model;

[0063] The model includes a signal encoder, a knowledge encoder, a label query network and a classification head;

[0064] The signal encoder encodes the original signal into a specific dimensional representation vector, the knowledge encoder encodes the ECG report or label text into a corresponding dimensional representation vector, the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result, and the classification head receives the encoding of the label query result and outputs the probability of the label existing in the ECG report.

[0065] Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label:

[0066]

[0067] in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

[0068] Step S5: using the trained model to analyze the pre-processed ECG data and query information, and outputting the query results;

[0069] Step S6: Deploy the multimodal ECG basic model to a cloud server with a GPU.

[0070] This invention solves the problem of traditional methods requiring time-consuming construction and alignment of knowledge bases by directly introducing external medical knowledge through large language models. It uses external enhanced medical knowledge to guide the model's representation learning, enabling it to significantly improve its diagnostic performance on different data sets. Furthermore, the model can make accurate judgment outputs when encountering categories it has never seen before.

[0071] The above is a basic embodiment of the present invention. The technical solution of the present invention is further described below through a preferred embodiment.

[0072] Example 1

[0073] Step 1: The server receives the ECG data to be analyzed and any disease name that the client wants to query whether it exists on the ECG;

[0074] Step 2: Preprocess the ECG data to be analyzed to obtain the preprocessed ECG data to be analyzed;

[0075] Preprocessing includes denoising, standardization, and segmentation of ECG data to improve data accuracy.

[0076] Step 3: The server uses the trained multimodal ECG basic model to parse the preprocessed ECG data to be analyzed and the text-based disease name to obtain the result of whether the disease exists on the ECG;

[0077] Step 3.1: If Figure 2 As shown in the figure, structured text labels of morphology, rhythm, and diagnosis are extracted from the original ECG report. Large language models such as GPT4 and Gemini are used to enhance the ECG report by adding explanations of medical terms and background knowledge of disease diagnosis. This constructs a knowledge-enhanced ECG dataset with the model of (ECG signal, structured text label, enhanced ECG report).

[0078] Step 3.2: If Figure 3 As shown, a signal-language-label model is constructed.

[0079] The model contains:

[0080] 1. ECG signal encoder, which encodes the original signal into a specific dimensional representation vector;

[0081] 2. Knowledge encoder, which encodes the report or label text into the corresponding dimensional representation vector;

[0082] 3. Label Query Network (LQN), which receives the label and ECG encoded representation vector and outputs the encoding of the label query result;

[0083] 4. Classification head, which receives the encoding of the label query result and outputs the probability of the label existing in the ECG.

[0084] Step 3.3: Train the model built in step 3.2 via enhanced signal-text-label contrastive learning.

[0085] This example proposes a new contrastive learning strategy, Augmented Signal-Text-Label Contrastive Learning (AugCL), which can integrate medical text knowledge into ECG signal learning.

[0086] The specific definition of AugCL is as follows:

[0087]

[0088] in, They are the contrast loss from signal to text and from text to signal respectively. Their specific definitions are as follows:

[0089]

[0090] in is a list of tags, is a learnable parameter, , , for Category, B is batch size, which is the batch size.

[0091] First, the ECG encoder and knowledge encoder process the ECG data and augmented reports and calculate the signal-text-label contrast loss according to formula (2). At the same time, the knowledge encoder encodes the text labels, and the encoded results serve as queries for the LQN. The ECG or report encoding is used as the key and value, and the binary cross entropy loss is calculated for each label according to formula (1).

[0092] This embodiment implements the training of a signal-language model that enhances medical knowledge by adopting two parallel loss optimization processes.

[0093] Step 3.4: Deploy the model to a cloud server with a GPU.

[0094] Step 4: The server transmits the recognition result to the terminal device.

[0095] The present invention adopts a signal-language architecture and uses text as a supervisory signal, effectively solving the problem that traditional models are limited by a fixed set of labels; utilizing the rich semantic knowledge of a large language model, different text labels with similar semantic relationships exhibit similar relationships in the representation space, thereby further improving the model's zero-sample migration capability; by directly using a large language model to introduce external medical knowledge, the problem that traditional methods require time-consuming construction and alignment of knowledge bases is solved; using external enhanced medical knowledge to guide the model's representation learning, its diagnostic performance on different data sets can be significantly improved; and the model can also make accurate judgment outputs when encountering categories it has never seen before.

[0096] The present invention also provides a contrast-enhanced fusion ECG large model training and deployment system, which can be implemented by executing the process steps of the contrast-enhanced fusion ECG large model training and deployment method. That is, those skilled in the art can understand the contrast-enhanced fusion ECG large model training and deployment method as a preferred implementation of the contrast-enhanced fusion ECG large model training and deployment system.

[0097] Specifically, a contrast-enhanced fusion ECG large model training and deployment system includes:

[0098] Module M1: collects raw ECG data and queries information;

[0099] Module M2: preprocessing the original ECG data to obtain preprocessed ECG data;

[0100] The preprocessing includes denoising and normalization;

[0101] Module M3: Extract structured text labels from preprocessed ECG data and construct an enhanced ECG dataset;

[0102] Module M4: Use the enhanced ECG dataset for comparative learning to train a multimodal ECG basic model;

[0103] The model includes a signal encoder, a knowledge encoder, a label query network and a classification head;

[0104] Module M5: Use the trained model to analyze the preprocessed ECG data and query information, and output the query results;

[0105] Module M6: Deploy the multimodal ECG basic model to a cloud server with a GPU.

[0106] The original electrocardiogram data includes an electrocardiogram signal and an electrocardiogram report; the query information includes whether a certain disease name can be derived from the electrocardiogram report.

[0107] The module M3 includes:

[0108] Structured text labels for morphology, rhythm, and diagnosis are extracted from ECG reports. Based on a large language model, medical terminology and background knowledge of disease diagnosis are added to obtain an enhanced ECG report. An enhanced ECG dataset is constructed by combining ECG signals, structured text labels, and enhanced ECG reports.

[0109] The signal encoder encodes the original signal into a specific dimensional representation vector; the knowledge encoder encodes the electrocardiogram report or label text into a corresponding dimensional representation vector; the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result; the classification head receives the encoding of the label query result and outputs the probability of the label existing in the electrocardiogram report.

[0110] The module M4 includes:

[0111] Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label:

[0112]

[0113] in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

[0114] Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, and units provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices, modules, and units provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, and units for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0115] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A method for training and deploying a large ECG model with contrast enhancement and fusion, characterized in that: include: Step S1: collecting original ECG data and querying information; Step S2: preprocessing the original ECG data to obtain preprocessed ECG data; The preprocessing includes denoising and normalization; Step S3: extracting structured text labels from the preprocessed ECG data to construct an enhanced ECG dataset; Step S4: using the enhanced ECG dataset to perform comparative learning and train a multimodal ECG basic model; The multimodal ECG basic model includes a signal encoder, a knowledge encoder, a label query network and a classification head; The signal encoder encodes the original signal into a specific dimensional representation vector; the knowledge encoder encodes the electrocardiogram report or label text into a corresponding dimensional representation vector; the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result; The classification head receives the encoding of the label query result and outputs the probability that the label exists in the electrocardiogram report; The contrastive learning training process includes: in, They are the contrastive losses of signal to text and text to signal, respectively, and are defined as follows: in is a list of tags, is a learnable parameter, , , for Category, B is batch size, which is the batch size; Step S5: using the trained multimodal ECG basic model to analyze the preprocessed ECG data and query information, and outputting the query results; Step S6: deploying the multimodal ECG basic model to a cloud server with a GPU; The step S3 comprises: Extract structured text labels for morphology, rhythm, and diagnosis from ECG reports. Based on a large language model, add explanations of medical terminology and background knowledge of disease diagnosis to obtain an enhanced ECG report. Combine the ECG signal, structured text labels, and enhanced ECG report to construct an enhanced ECG dataset. The step S4 comprises: Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label: in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

2. The contrast-enhanced fusion ECG large model training and deployment method according to claim 1 is characterized in that: The original electrocardiogram data includes an electrocardiogram signal and an electrocardiogram report; the query information includes whether a certain disease name can be derived from the electrocardiogram report.

3. A contrast-enhanced fusion ECG large model training and deployment system, characterized by: include: Module M1: collects raw ECG data and queries information; Module M2: preprocessing the original ECG data to obtain preprocessed ECG data; The preprocessing includes denoising and normalization; Module M3: Extract structured text labels from preprocessed ECG data and construct an enhanced ECG dataset; Module M4: Use the enhanced ECG dataset for comparative learning to train a multimodal ECG basic model; The multimodal ECG basic model includes a signal encoder, a knowledge encoder, a label query network and a classification head; The signal encoder encodes the original signal into a specific dimensional representation vector; the knowledge encoder encodes the electrocardiogram report or label text into a corresponding dimensional representation vector; the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result; The classification head receives the encoding of the label query result and outputs the probability that the label exists in the electrocardiogram report; The contrastive learning training process includes: in, They are the contrastive losses of signal to text and text to signal, respectively, and are defined as follows: in is a list of tags, is a learnable parameter, , , for Category, B is batch size, which is the batch size; Module M5: Use the trained multimodal ECG basic model to analyze the preprocessed ECG data and query information, and output the query results; Module M6: Deploy the multimodal ECG basic model to a cloud server with a GPU; The module M3 includes: Extract structured text labels for morphology, rhythm, and diagnosis from ECG reports. Based on a large language model, add explanations of medical terminology and background knowledge of disease diagnosis to obtain an enhanced ECG report. Combine the ECG signal, structured text labels, and enhanced ECG report to construct an enhanced ECG dataset. The module M4 includes: Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label: in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

4. The contrast-enhanced fusion ECG large model training and deployment system according to claim 3, characterized in that: The original electrocardiogram data includes an electrocardiogram signal and an electrocardiogram report; the query information includes whether a certain disease name can be derived from the electrocardiogram report.

Images