Method and system for training and deploying large electrocardio model based on contrast enhancement fusion

Abstract

The invention provides a contrast enhancement fusion electrocardio large model training deployment method and system. The method comprises the steps that S1, electrocardio data and inquiry information are collected; s2, the electrocardio data are preprocessed, and preprocessed electrocardio data are obtained; s3, extracting a structured text tag from the preprocessed electrocardiogram data, and constructing an enhanced electrocardiogram data set; s4, using an enhanced electrocardiogram data set to compare, learn and train the model; s5, analyzing the preprocessed electrocardiogram data and inquiry information by using the trained model, and outputting an inquiry result; and S6, deploying the multi-modal electrocardio basic model to a cloud server with a GPU (Graphics Processing Unit). According to the method, the signal-language-label comparative learning normal form is adopted, so that the problem that the representation learning accuracy is affected due to the fact that a noise positive example is introduced into a multi-label classification task in traditional comparative learning is solved, the representation of the model in a complex classification task is enhanced, and the method has a good application prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of electrocardiogram data processing, and in particular to a method and system for training and deploying a contrast-enhanced and fused electrocardiogram large model. Background Art

[0002] Electrocardiogram (ECG) is an important tool for clinical diagnosis of cardiovascular diseases. By recording the electrical signal activity of the heart, doctors can evaluate the functional state of the heart and detect various heart diseases. Traditional ECG analysis mainly relies on the doctor's experience and expertise, but this method has problems such as strong subjectivity and low efficiency. In recent years, with the development of artificial intelligence technology, automatic ECG analysis methods based on deep learning have gradually become a research hotspot.

[0003] However, existing deep learning methods mainly focus on single-modal data processing, ignoring the fusion of multimodal information, which limits the performance of the model. Traditional models also have problems such as difficulty in feature extraction, low accuracy, low efficiency, strong label dependence, and noisy positive examples.

[0004] Literature 1: [1] Liu C, Wan Z, Ouyang C, et al. Zero-Shot ECGClassification with Multimodal Learning and Test-time Clinical KnowledgeEnhancement[J].2024. Literature 2: [1] Li J, Liu C, Cheng S, et al. Frozen Language Model HelpsECG Zero-Shot Learning[J].2023. Although the above two papers from Imperial College London also used ECG signals and medical text training models and achieved zero-sample diagnosis, they still used traditional self-supervised contrastive learning methods and did not propose solutions for the multi-label characteristics of the ECG field. This method lacks the introduction of discriminative information and will introduce noisy positive examples, which will greatly reduce the accuracy.

[0005] Document 3: ECG Semantic Integrator (ESI): A Foundation ECG ModelPretrainedwith LLM-Enhanced Cardiological Text Although this paper from Rice University in the United States also proposed a knowledge enhancement method and trained a model using ECG signals and medical text, the architecture of this technology still uses the early CLIP architecture, which only performs comparative learning between the text encoder and the signal encoder, and does not build a label query. This simple architecture cannot fuse the aligned signals and disease text labels at the model level, resulting in its insufficient zero-sample diagnosis capability; this technology also only uses traditional comparative learning methods and lacks the introduction of discriminative information. Summary of the invention

[0006] In view of the defects in the prior art, the purpose of the present invention is to provide a method and system for training and deploying a large ECG model with contrast enhancement and fusion.

[0007] A contrast-enhanced and fused ECG large model training and deployment method provided by the present invention includes: Step S1: Collecting original ECG data and querying information; Step S2: preprocessing the original ECG data to obtain preprocessed ECG data; The preprocessing includes denoising and standardization; Step S3: extracting structured text labels from the preprocessed ECG data and constructing an enhanced ECG dataset; Step S4: using the enhanced ECG dataset to perform comparative learning and train a multimodal ECG basic model; The model includes a signal encoder, a knowledge encoder, a label query network and a classification head; Step S5: using the trained model to analyze the preprocessed ECG data and query information, and outputting the query results; Step S6: deploy the multimodal ECG basic model to a cloud server with a GPU.

[0008] Preferably, the original ECG data includes an ECG signal and an ECG report; the query information includes whether a certain disease name associated with the ECG report can be derived.

[0009] Preferably, step S3 comprises: Structured text labels of morphology, rhythm and diagnosis are extracted from the ECG report. Based on the large language model, explanations of medical terms and background knowledge of disease diagnosis are added to obtain an enhanced ECG report. The ECG signal, structured text labels and enhanced ECG report are combined to construct an enhanced ECG dataset.

[0010] Preferably, the signal encoder encodes the original signal into a specific dimensional representation vector; the knowledge encoder encodes the electrocardiogram report or label text into a corresponding dimensional representation vector; the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result; the classification head receives the encoding of the label query result, and outputs the probability of the label existing in the electrocardiogram report.

[0011] Preferably, step S4 comprises: Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label:

[0012] in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

[0013] A contrast-enhanced and fused ECG large model training and deployment system provided by the present invention comprises: Module M1: collects raw ECG data and queries information; Module M2: preprocessing the original ECG data to obtain preprocessed ECG data; The preprocessing includes denoising and standardization; Module M3: Extract structured text labels from preprocessed ECG data and construct an enhanced ECG dataset; Module M4: Use the enhanced ECG dataset to conduct comparative learning and train a multimodal ECG basic model; The model includes a signal encoder, a knowledge encoder, a label query network and a classification head; Module M5: Use the trained model to analyze the preprocessed ECG data and query information, and output the query results; Module M6: Deploy the multimodal ECG basic model to a cloud server with a GPU.

[0014] Preferably, the original ECG data includes an ECG signal and an ECG report; the query information includes whether a certain disease name associated with the ECG report can be derived.

[0015] Preferably, the module M3 includes: Structured text labels of morphology, rhythm and diagnosis are extracted from the ECG report. Based on the large language model, explanations of medical terms and background knowledge of disease diagnosis are added to obtain an enhanced ECG report. The ECG signal, structured text labels and enhanced ECG report are combined to construct an enhanced ECG dataset.

[0016] Preferably, the signal encoder encodes the original signal into a specific dimensional representation vector; the knowledge encoder encodes the electrocardiogram report or label text into a corresponding dimensional representation vector; the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result; the classification head receives the encoding of the label query result, and outputs the probability of the label existing in the electrocardiogram report.

[0017] Preferably, the module M4 includes: Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label:

[0018] in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention adopts a signal-language architecture and uses text as a supervisory signal, which effectively solves the problem that traditional models are limited by a fixed set of labels. By utilizing the rich semantic knowledge of large language models, different text labels with similar semantic relationships show similar relationships in the representation space, thereby further improving the model's zero-sample migration capability.

[0020] 2. The present invention solves the problem that traditional methods require time-consuming construction and alignment of knowledge bases by directly using a large language model to introduce external medical knowledge; it uses external enhanced medical knowledge to guide the representation learning of the model, so that its diagnostic performance on different data sets can be significantly improved; and the model can also make accurate judgment outputs when encountering categories that have never been seen before.

[0021] 3. The present invention adopts the signal-language-label contrastive learning paradigm (AugCL) to solve the problem that traditional contrastive learning will introduce noisy positive examples in multi-label classification tasks, thereby affecting the accuracy of representation learning; AugCL effectively reduces the impact of noisy positive examples, solves the noise problem in multi-label settings, enhances the performance of the model in complex classification tasks, and has good application prospects.

[0022] 4. The present invention uses a large language model to enhance the domain knowledge of ECG signals, achieves zero-sample diagnosis performance in multiple regions and multiple diseases, and provides doctors with better decision-making suggestions.

[0023] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 The figure is a flow chart of the method of the present invention.

[0025] Figure 2 The flowchart of constructing an enhanced ECG data set in an embodiment of the present invention.

[0026] Figure 3 This is a flowchart of constructing a signal-language-label model in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0028] Reference Figure 1 As shown, a method for training and deploying a large ECG model with contrast enhancement fusion includes: Step S1: Collecting original ECG data and querying information; The original ECG data includes an ECG signal and an ECG report, and the query information includes whether a certain disease name associated with the ECG report can be obtained.

[0029] Step S2: preprocessing the original ECG data to obtain preprocessed ECG data; The preprocessing includes denoising and standardization; Step S3: extracting structured text labels from the preprocessed ECG data and constructing an enhanced ECG dataset; Structured text labels of morphology, rhythm and diagnosis are extracted from the ECG report. Based on the large language model, explanations of medical terms and background knowledge of disease diagnosis are added to obtain an enhanced ECG report. The ECG signal, structured text labels and enhanced ECG report are combined to construct an enhanced ECG dataset.

[0030] Step S4: using the enhanced ECG dataset to perform comparative learning and train a multimodal ECG basic model; The model includes a signal encoder, a knowledge encoder, a label query network and a classification head; The signal encoder encodes the original signal into a specific dimensional representation vector, the knowledge encoder encodes the ECG report or label text into a corresponding dimensional representation vector, the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result, and the classification head receives the encoding of the label query result and outputs the probability of the label existing in the ECG report.

[0031] Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label:

[0032] in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

[0033] Step S5: using the trained model to analyze the preprocessed ECG data and query information, and outputting the query results; Step S6: deploy the multimodal ECG basic model to a cloud server with a GPU.

[0034] The present invention introduces external medical knowledge by directly using a large language model, thus solving the problem that traditional methods require time-consuming construction and alignment of knowledge bases. Externally enhanced medical knowledge is used to guide the representation learning of the model, so that its diagnostic performance on different data sets can be significantly improved. Moreover, the model can make accurate judgment outputs when encountering categories that it has never seen before.

[0035] The above is a basic embodiment of the present invention. The technical solution of the present invention is further described below through a preferred embodiment.

[0036] Example 1 Step 1: The server receives the ECG data to be analyzed and any disease name that the client wants to inquire whether it exists on the ECG; Step 2: preprocess the ECG data to be analyzed to obtain the preprocessed ECG data to be analyzed; Preprocessing includes denoising, standardization, and segmentation of ECG data to improve data accuracy.

[0037] Step 3: The server uses the trained multimodal ECG basic model to parse the preprocessed ECG data to be analyzed and the disease name in text form to obtain the result of whether the disease exists in the ECG; Step 3.1: If Figure 2 As shown in the figure, structured text labels of morphology, rhythm, and diagnosis are extracted from the original ECG report. Large language models such as GPT4 and Gemini are used to enhance the ECG report by adding explanations of medical terms and background knowledge of disease diagnosis, and a knowledge-enhanced ECG dataset with the (ECG signal, structured text label, enhanced ECG report) pattern is constructed.

[0038] Step 3.2: If Figure 3 As shown, a signal-language-label model is constructed.

[0039] The model contains: 1. ECG signal encoder, which encodes the original signal into a specific dimensional representation vector; 2. Knowledge encoder, which encodes the report or label text into the corresponding dimensional representation vector; 3. Label query network (LQN), which receives the label and ECG encoded representation vector and outputs the encoding of the label query result; 4. Classification head, which receives the encoding of the label query result and outputs the probability of the label existing in the ECG.

[0040] Step 3.3: Train the model built in step 3.2 via enhanced signal-text-label contrastive learning.

[0041] This embodiment proposes a new contrastive learning strategy, namely, enhanced signal-text-label contrastive learning (AugCL), which can integrate medical text knowledge into ECG signal learning.

[0042] The specific definition of AugCL is as follows:

[0043] in, They are the contrast loss from signal to text and from text to signal respectively. Their specific definitions are as follows:

[0044] in is a list of tags, is a learnable parameter, , , for Category, B is batch size, which is the batch size.

[0045] First, the ECG encoder and knowledge encoder process the ECG data and enhanced reports, and calculate the signal-text-label contrast loss according to formula (2). At the same time, the knowledge encoder encodes the text labels, and the results of these encodings are used as queries for LQN. ECG or report encoding is used as the key and value, and the binary cross entropy loss of each label is calculated according to formula (1).

[0046] This embodiment implements the training of a signal-language model for enhancing medical knowledge by adopting two parallel loss optimization processes.

[0047] Step 3.4: Deploy the model to a cloud server with a GPU.

[0048] Step 4: The server transmits the recognition result to the terminal device.

[0049] The present invention adopts a signal-language architecture and uses text as a supervisory signal, which effectively solves the problem that traditional models are limited by a fixed set of labels; it utilizes the rich semantic knowledge of a large language model to make different text labels with similar semantic relationships show similar relationships in the representation space, thereby further improving the model's zero-sample migration capability; by directly using a large language model to introduce external medical knowledge, it solves the problem that traditional methods require time-consuming construction and alignment of knowledge bases; it uses external enhanced medical knowledge to guide the representation learning of the model, so that its diagnostic performance on different data sets can be significantly improved; and the model can also make accurate judgment outputs when encountering categories that it has never seen before.

[0050] The present invention also provides a contrast-enhanced and fused ECG large model training and deployment system, which can be implemented by executing the process steps of the contrast-enhanced and fused ECG large model training and deployment method, that is, those skilled in the art can understand the contrast-enhanced and fused ECG large model training and deployment method as a preferred implementation of the contrast-enhanced and fused ECG large model training and deployment system.

[0051] Specifically, a contrast-enhanced fusion ECG large model training and deployment system includes: Module M1: collects raw ECG data and queries information; Module M2: preprocessing the original ECG data to obtain preprocessed ECG data; The preprocessing includes denoising and standardization; Module M3: Extract structured text labels from preprocessed ECG data and construct an enhanced ECG dataset; Module M4: Use the enhanced ECG dataset to conduct comparative learning and train a multimodal ECG basic model; The model includes a signal encoder, a knowledge encoder, a label query network and a classification head; Module M5: Use the trained model to analyze the preprocessed ECG data and query information, and output the query results; Module M6: Deploy the multimodal ECG basic model to a cloud server with a GPU.

[0052] The original ECG data includes an ECG signal and an ECG report; the query information includes whether a certain disease name associated with the ECG report can be obtained.

[0053] The module M3 comprises: Structured text labels of morphology, rhythm and diagnosis are extracted from the ECG report. Based on the large language model, explanations of medical terms and background knowledge of disease diagnosis are added to obtain an enhanced ECG report. The ECG signal, structured text labels and enhanced ECG report are combined to construct an enhanced ECG dataset.

[0054] The signal encoder encodes the original signal into a specific dimensional representation vector; the knowledge encoder encodes the electrocardiogram report or label text into a corresponding dimensional representation vector; the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result; the classification head receives the encoding of the label query result, and outputs the probability of the label existing in the electrocardiogram report.

[0055] The module M4 comprises: Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label:

[0056] in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

[0057] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0058] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A contrast-enhanced fusion ECG large model training and deployment method, characterized in that: include: Step S1: Collecting original ECG data and querying information; Step S2: preprocessing the original ECG data to obtain preprocessed ECG data; The preprocessing includes denoising and standardization; Step S3: extracting structured text labels from the preprocessed ECG data and constructing an enhanced ECG dataset; Step S4: using the enhanced ECG dataset to perform comparative learning and train a multimodal ECG basic model; The model includes a signal encoder, a knowledge encoder, a label query network and a classification head; Step S5: using the trained model to analyze the preprocessed ECG data and query information, and outputting the query results; Step S6: deploy the multimodal ECG basic model to a cloud server with a GPU.

2. The contrast-enhanced fusion ECG large model training and deployment method according to claim 1 is characterized in that: The original ECG data includes an ECG signal and an ECG report; the query information includes whether a certain disease name associated with the ECG report can be obtained.

3. The contrast-enhanced fusion ECG large model training and deployment method according to claim 2 is characterized in that: The step S3 comprises: Structured text labels of morphology, rhythm and diagnosis are extracted from the ECG report. Based on the large language model, explanations of medical terms and background knowledge of disease diagnosis are added to obtain an enhanced ECG report. The ECG signal, structured text labels and enhanced ECG report are combined to construct an enhanced ECG dataset.

4. The contrast-enhanced fusion ECG large model training and deployment method according to claim 1 is characterized in that: The signal encoder encodes the original signal into a specific dimensional representation vector; the knowledge encoder encodes the electrocardiogram report or label text into a corresponding dimensional representation vector; the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result; the classification head receives the encoding of the label query result, and outputs the probability of the label existing in the electrocardiogram report.

5. The method for training and deploying a large ECG model with contrast enhancement fusion according to claim 4, characterized in that: The step S4 comprises: Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label: in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

6. A contrast-enhanced fusion ECG large model training and deployment system, characterized in that: include: Module M1: collects raw ECG data and queries information; Module M2: preprocessing the original ECG data to obtain preprocessed ECG data; The preprocessing includes denoising and standardization; Module M3: Extract structured text labels from preprocessed ECG data and construct an enhanced ECG dataset; Module M4: Use the enhanced ECG dataset to conduct comparative learning and train a multimodal ECG basic model; The model includes a signal encoder, a knowledge encoder, a label query network and a classification head; Module M5: Use the trained model to analyze the preprocessed ECG data and query information, and output the query results; Module M6: Deploy the multimodal ECG basic model to a cloud server with a GPU.

7. The contrast-enhanced fusion ECG large model training and deployment system according to claim 6, characterized in that: The original ECG data includes an ECG signal and an ECG report; the query information includes whether a certain disease name associated with the ECG report can be obtained.

8. The contrast-enhanced fusion ECG large model training and deployment system according to claim 7, characterized in that: The module M3 comprises: Structured text labels of morphology, rhythm and diagnosis are extracted from the ECG report. Based on the large language model, explanations of medical terms and background knowledge of disease diagnosis are added to obtain an enhanced ECG report. The ECG signal, structured text labels and enhanced ECG report are combined to construct an enhanced ECG dataset.

9. The contrast-enhanced fusion ECG large model training and deployment system according to claim 6, characterized in that: The signal encoder encodes the original signal into a specific dimensional representation vector; the knowledge encoder encodes the electrocardiogram report or label text into a corresponding dimensional representation vector; the label query network receives the label and the representation vector encoded by the signal encoder, and outputs the encoding of the label query result; the classification head receives the encoding of the label query result, and outputs the probability of the label existing in the electrocardiogram report.

10. The contrast-enhanced fusion ECG large model training and deployment system according to claim 9, characterized in that: The module M4 comprises: Let the signal encoder and knowledge encoder process the original ECG data and the enhanced ECG report, and calculate the signal-text-label contrast loss; let the knowledge encoder encode the text label, use the encoding result as the query basis of the label query network, use the ECG or report encoding as the key and value, and calculate the binary cross entropy loss for each label: in, is a binary label 0 or 1, is the probability that the output belongs to the label, and N represents the number of groups of objects predicted by the model.

Images