Automatic electrocardiosignal report generation method, device, equipment and medium

Abstract

The invention discloses an electrocardiosignal report automatic generation method and device, equipment and a medium. The method comprises the steps that electrocardiosignal data to be processed are acquired; inputting the obtained electrocardiosignal data into a report generation model, and outputting an electrocardiograph report; the report generation model is obtained by improving an encoder-decoder architecture; the method comprises the following steps: capturing features in electrocardiosignals by adopting an ECG-lip text supervision feature extraction method; a cross-modal alignment mechanism is introduced, and electrocardiosignal features and medical text features are aligned to enhance the anomaly detection capability of the model. The invention provides an improved electrocardiosignal report generation model, aiming at key challenges in electrocardiosignal report generation, the accuracy and diversity of report generation are remarkably improved by introducing an ECG-CLIP text supervision feature extraction method, a cross-modal learning mechanism and weak supervision alignment loss. The method can be widely applied to the technical field of physiological signal processing.

Description

Technical Field

[0001] The present invention relates to the technical field of physiological signal processing, and in particular to a method, device, equipment and medium for automatically generating an electrocardiogram signal report. Background Art

[0002] Cardiovascular disease is the leading cause of death worldwide, and timely and accurate diagnosis is crucial for the treatment and prognosis of patients. As a non-invasive and low-cost detection method, electrocardiogram (ECG) is widely used in the early screening and diagnosis of cardiovascular disease. However, traditional ECG analysis relies on manual interpretation and report generation of ECG records by cardiologists, a process that is complex, time-consuming, and susceptible to subjective factors.

[0003] Currently, artificial intelligence (AI) technology has been widely used in the classification and anomaly detection of ECG signals, greatly improving the efficiency of diagnosis. However, the technology for automatically generating ECG diagnostic reports is still in its infancy. Existing technologies cannot fully capture subtle changes in ECG signals, and the generated reports have significant limitations in flexibility and clinical practicality. Existing technologies are insufficient in capturing abnormal waveforms, processing report details, and generating text consistent with clinical diagnoses. Therefore, it is crucial to develop a method that can efficiently and accurately generate ECG reports. Summary of the invention

[0004] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide a method, device, equipment and medium for automatically generating ECG reports based on weakly supervised contrastive learning and Encoder-Decoder framework.

[0005] The first technical solution adopted by the present invention is:

[0006] A method for automatically generating an electrocardiogram signal report comprises the following steps:

[0007] Obtaining the ECG signal data to be processed;

[0008] Input the acquired ECG signal data into a report generation model and output an ECG report;

[0009] The report generation model is obtained by improving the encoder-decoder architecture; an ECG-clip text-supervised feature extraction method is used to capture subtle timing features in electrocardiogram signals; a cross-modal alignment mechanism is introduced to align electrocardiogram signal features with medical text features to enhance the model's anomaly detection capability.

[0010] Further, the report generation model includes an ECG-clip module and a Transformer Decoder;

[0011] The training steps of the report generation model include:

[0012] In the text generation stage, the ECG-clip module is used to extract the electrocardiogram signal E i Extract the feature H x , and use Transformer Decoder to generate the corresponding ECG report; through the cross entropy loss L CE To optimize the matching degree between the generated report and the reference report; by introducing the weak supervision contrast loss L WCL To focus on the subtle differences between the reported text, the total loss function is:

[0013] L loss =(1-λ)L CE +λL WCL

[0014] In the formula, λ is the weight parameter;

[0015] By jointly optimizing the weakly supervised contrast loss and the cross entropy loss, the report generation model can generate more diverse and accurate reports.

[0016] Furthermore, the training step of the report generation model further includes the following steps:

[0017] In the data processing stage, the BERT model is used to process the preset ECG report Y i Semantic feature extraction is performed and semantic feature representation is obtained through [CLS] embedding; then, K-means clustering is used to group the ECG reports and generate cluster labels l j , used to improve the semantic understanding ability of the model during training.

[0018] Furthermore, the ECG-clip module works as follows:

[0019] The ECG signal data and the corresponding report text are taken as input and integrated into a multimodal contrastive learning framework, which includes a language processing component and an ECG encoder, which are used to obtain the embedding representation of the text and the embedding representation of the ECG respectively; both the language processing component and the ECG encoder map the text embedding and the ECG embedding into a vector space of the same dimension through a linear projection head;

[0020] In this vector space, by calculating the similarity between ECG embedding and text embedding, the contrastive learning loss function is used for optimization, so that the embeddings of the relevant ECG and report text are closer, thereby minimizing the contrastive loss; this process generates a pre-trained model that can effectively combine ECG signal data with text information, has rich medical knowledge, can capture abnormal change points, and learn high-level semantic feature representations.

[0021] Furthermore, to make full use of the prior clinical knowledge in the report text, the report text was input into a large frozen language model (BioinicalBERT), and the ECG encoder adopted inception time as the backbone network.

[0022] Furthermore, the cross-modal alignment mechanism works as follows:

[0023] Positive sample pair (E i ,Y i );E i is the input ECG signal, and the corresponding text report Y i Constitute positive sample pairs; by maximizing the similarity of positive sample pairs, the report generation model can better capture the semantic connection between ECG signals and text;

[0024] Negative sample pairs (E i ,Y j ): Y j is the input ECG signal E i Irrelevant reports, which constitute negative sample pairs; by minimizing the similarity of negative sample pairs, the report generation model can reduce the occurrence of incorrect pairings to improve the accuracy and robustness of the representation;

[0025] Weakly supervised contrast loss: Weakly supervised contrast loss is applied to ECG report generation, and the hidden representations of ECG and generated report are projected into the latent space:

[0026]

[0027] In the formula, and They are the hidden states h from the last layer output of Encoder-Decoder. X and h Y The average pooling result of x and φ y There are two fully connected layers with ReLU activation function; then, the model is optimized by maximizing the similarity between positive samples and minimizing the similarity between negative sample pairs.

[0028] Furthermore, the weakly supervised contrast loss L WCL The expression is:

[0029]

[0030] In the formula, α is a hyperparameter that controls the weight of weak supervision, τ is a temperature parameter, and N is the number of words. is the hidden representation of the positive sample pair, is the hidden representation of negative samples, l i is the cluster label of the positive sample pair, l j is the cluster label of negative samples.

[0031] The second technical solution adopted by the present invention is:

[0032] A device for automatically generating an electrocardiogram signal report, comprising:

[0033] A data acquisition module, used to acquire the ECG signal data to be processed;

[0034] A report generation module, used for inputting the acquired ECG signal data into a report generation model and outputting an ECG report;

[0035] The report generation model is obtained by improving the encoder-decoder architecture; an ECG-clip text-supervised feature extraction method is used to capture subtle timing features in electrocardiogram signals; a cross-modal alignment mechanism is introduced to align electrocardiogram signal features with medical text features to enhance the model's anomaly detection capability.

[0036] The third technical solution adopted by the present invention is:

[0037] An electronic device comprises a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a method for automatically generating an electrocardiogram signal report as described above.

[0038] The fourth technical solution adopted by the present invention is:

[0039] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement a method for automatically generating an electrocardiogram signal report as described above.

[0040] The fifth technical solution adopted by the present invention is:

[0041] A computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned method for automatically generating an electrocardiogram signal report.

[0042] The beneficial effects of the present invention are as follows: the present invention proposes an improved ECG signal report generation model, which targets key challenges in ECG report generation, such as the scarcity of abnormal signals, temporal dependency, and subtle differences in report text, by introducing an ECG-CLIP text supervised feature extraction method, a cross-modal learning mechanism, and a weakly supervised alignment loss (L WCL ), significantly improving the accuracy and diversity of report generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 is an overall flow chart of a report generation model in an embodiment of the present invention;

[0045] Figure 2 is a schematic diagram of the operation of the ECG-clip module in an embodiment of the present invention;

[0046] Figure 3 is a comparison diagram of the performance verification of the ECG-CLIP module in an embodiment of the present invention;

[0047] Figure 4 It is a comparison chart of the performance verification of the cross-modal alignment mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0049] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0050] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0051] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0052] Terminology explanation:

[0053] CLIP: Abbreviation for Contrastive Language–Image Pre-training. The CLIP model is a multimodal training model that can process image and text data simultaneously. Through contrastive learning and multi-task classification training, the model can perform well in different tasks. The core of the CLIP model is that it can learn the deep semantic connection between images and text, thereby achieving cross-modal understanding.

[0054] InceptionTime: is a deep learning model for time series classification. It is mainly used to process time series data and is suitable for time-sensitive applications such as health informatics, anomaly detection, and device status recognition.

[0055] Example 1

[0056] This embodiment provides a method for automatically generating an electrocardiogram signal report, comprising the following steps:

[0057] S1. Obtaining ECG signal data to be processed;

[0058] S2. Input the obtained ECG signal data into a report generation model and output an ECG report; the report generation model is obtained by improving the encoder-decoder architecture; an ECG-clip text-supervised feature extraction method is used to capture features in the ECG signal; a cross-modal alignment mechanism is introduced to align ECG signal features with medical text features to enhance the model's anomaly detection capability.

[0059] This example designs an improved model architecture based on the encoder-decoder, uses the Transformer Decoder as the language decoder, and combines two major core modules to improve the performance of ECG signal report generation. First, we use the ECG-clip text supervision feature extraction method to specifically capture the subtle timing features in the ECG signal, especially the changes in abnormal signals. Secondly, we introduce a cross-modal learning mechanism to align the ECG signal features with the medical text features, thereby enhancing the model's anomaly detection capabilities.

[0060] Next, the above model is explained in detail with reference to the accompanying drawings and specific implementation methods, explaining how the encoder-decoder architecture is applied to ECG report generation, and then the two core modules of ECG-clip and cross-modal learning are introduced.

[0061] (1) Overall description of the report generation model

[0062] The Encoder-Decoder framework can effectively capture important semantic information in images and generate reports that meet clinical needs by extracting key features from input data and generating corresponding natural language reports. In the ECG signal report generation task, we adopted this framework and used the feature extractor f to extract ECG signal data. x Extract key features (such as temporal and spatial features, local changes in waveforms, semantic features, etc.) and then pass them through the text decoder f t Decode these features into corresponding natural language reports. Specifically expressed as:

[0063]

[0064] Where E represents the input ECG signal, H x is the feature information extracted by the encoder from the input signal, is the ECG report generated by the model, and the model is optimized by calculating the cross entropy loss. Assume that the generated report and the true report Y = {y1,…,y NY},in and NY represent the number of tokens in the generated report and the real report, respectively. When generating the i-th token, the model generates tokens based on the historical tokens The probability distribution of the vocabulary V is calculated with the ECG signal E, and the probability of the generated j-th word is expressed as P i,j The model selects the word with the highest probability as the generated word Then, calculate the generated word unit With real word y i The cross entropy loss between

[0065]

[0066] in, The definition of v j =y i hour, otherwise Therefore, the cross entropy loss L of the entire generated report CE is the sum of all word losses:

[0067]

[0068] The overall process of the model is as follows Figure 1 First, in the data processing stage, we use the BERT model to analyze the ECG report Y i Semantic feature extraction is performed and semantic feature representation is obtained through [CLS] embedding. Subsequently, K-means clustering is used to group the reports and generate cluster labels l j , used to improve the semantic understanding ability of the model during training.

[0069] In the text generation stage, the model uses the ECG-clip module to extract the electrocardiogram signal E i Extract the feature H x , and use Transformer Decoder to generate the corresponding natural language report. In this process, the hidden states of the encoder and decoder of the positive and negative sample pairs are averaged and pooled. and Through the MLP layer, they are mapped to the same latent space for contrastive learning. Finally, the model is transformed into a vector by the cross entropy loss L CE To optimize the matching degree between the generated report and the reference report. At the same time, the model also introduces a weak supervision contrast loss L WCL , paying special attention to the subtle differences between report texts. By jointly optimizing the weakly supervised contrast loss and the cross entropy loss, the model is able to generate more diverse and accurate reports.

[0070] (2) ECG-clip module

[0071] The ECG-clip module of this embodiment aims to draw on the successful experience of the CLIP (Contrastive Language–Image Pre-training) model in the field of image-text cross-modal understanding and generation, and apply it to the multimodal task of electrocardiogram (ECG) and report text. Although multimodal contrastive learning (SSL) has performed well in image-text tasks in recent years, the combination of medical signals and text, especially the cross-modal generation of ECG signals and text, has not yet fully utilized the advantages of these technologies.

[0072] join Figure 2 In the ECG-clip module, the model takes ECG data and the corresponding report text as input and integrates them into a multimodal contrastive learning framework. The framework contains a language processing component and an ECG encoder, which are used to obtain embedded representations of text and ECG respectively. In order to make full use of the prior clinical knowledge in the report text, the report text is input into a large frozen language model (BioinicalBERT), while the ECG encoder uses inceptiontime as the backbone network. Both use a linear projection head to map the text and ECG embeddings into a vector space of the same dimension.

[0073] In this vector space, the model calculates the similarity between ECG embedding and text embedding, and optimizes it using the contrastive learning loss function to make the embeddings of the relevant ECG and report text closer, thereby minimizing the contrastive loss. This process generates a pre-trained model that can effectively combine ECG data with text information, has rich medical knowledge, can capture abnormal change points, and learn high-level semantic feature representations.

[0074] (3) Cross-modal alignment mechanism

[0075] Since ECG report texts are relatively narrow in distribution and full of medical-specific terms, and usually contain relatively rare abnormal information, relying solely on cross-entropy loss may cause the model to tend to generate common descriptions and ignore important abnormal findings. Therefore, we further introduce a cross-modal alignment mechanism to enhance the report generation quality of the model.

[0076] Positive sample pair (E i ,Y i );E i is the input ECG signal, and the corresponding text report Y i Constitute positive sample pairs; by maximizing the similarity of positive sample pairs, the report generation model can better capture the semantic connection between ECG signals and text.

[0077] Negative sample pairs (E i ,Y j )(i≠j):Y j is the input ECG signal E i Unrelated reports constitute negative sample pairs; by minimizing the similarity of negative sample pairs, the report generation model can reduce the occurrence of incorrect pairing and improve the accuracy and robustness of the representation.

[0078] Weakly supervised contrastive loss: Weakly supervised contrastive loss is applied to ECG report generation to project the hidden representations of ECG and generated report into the latent space.

[0079]

[0080] In the formula, and They are the hidden states h from the last layer output of Encoder-Decoder. X and h Y The average pooling result of x and φ y There are two fully connected layers with ReLU activation function; then, the model is optimized by maximizing the similarity between positive samples and minimizing the similarity between negative sample pairs.

[0081] Weakly supervised contrast loss L WCL The expression is:

[0082]

[0083] s i,j is the similarity between the positive sample pair and the negative sample pair, α is the hyperparameter that controls the weak supervision weight, and τ is the temperature parameter.

[0084] Overall, the model is optimized with a mixture of cross entropy loss and weakly supervised contrastive loss (λ is a weight parameter that balances the effects of the two losses):

[0085] L loss =(1-λ)L CE +λL WCL

[0086] (4) Experimental verification

[0087] 4.1) Experimental setup

[0088] a) Dataset

[0089] We use the PTB-XL dataset for experiments. The dataset contains 21,837 clinical 12-lead ECG records from 18,885 patients, each with a duration of 10 seconds. Among them, 67.13% of the diagnostic reports were manually interpreted by cardiologists, and 31.2% of the diagnostic reports were automatically generated by ECG devices. Among the reports generated by ECG devices, 4.45% of the reports were verified by cardiologists, but 26.75% of the report verification information was incomplete. In addition, 1.67% of the records did not generate ECG reports. The reports are displayed in the document as strings, of which 70.89% are in German, 27.9% are in English, and 1.21% are in Swedish. In order to cooperate with the clinical pre-trained BERT model, we uniformly translate all reports into English for processing. The division of the dataset is based on the general provisions of PTB-XL, with 1-8 folds of data as training sets, the 9th fold of data as validation sets, and the 10th fold as test sets.

[0090] b) Indicators

[0091] In our experiments, we used several automatic text generation metrics to measure the quality of the generated ECG reports, including the following:

[0092] BLEU and METEOR: These two indicators are mainly used to evaluate the quality of machine translation, focusing on the accuracy and fluency of the generated reports. BLEU evaluates the accuracy of the generated report by calculating the degree of n-gram matching between the generated text and the reference text; METEOR further supplements BLEU's shortcomings in evaluating fluency by considering factors such as synonyms and stems.

[0093] ROUGE-L: This metric is used to measure the fluency and structure of sentences. ROUGE-L focuses on the matching of the longest common subsequence, which is used to reflect the consistency of the sentence structure between the generated report and the reference report, and can evaluate the coherence of the generated text.

[0094] ROUGE-1 and ROUGE-2: These two indicators evaluate the match between the generated text and the reference text through the overlap of single letters (ROUGE-1) and bigrams (ROUGE-2), and are mainly used to check the word-level accuracy of the text.

[0095] CIDEr-D: This metric evaluates the relevance and uniqueness of the generated ECG report in the candidate set. CIDEr-D measures the uniqueness and consistency of the generated text by calculating the TF-IDF weighted similarity of the n-grams of the reference report, and is particularly suitable for evaluating diversity and semantic matching in generation tasks.

[0096] c) Implementation details

[0097] We use the Adam optimizer for training, and decay the learning rate by 0.8 per epoch. The number of cluster labels K is set to 5, and the weighting parameters λ and α are 0.2 and 2, respectively. To ensure the best performance of the model, we perform hyperparameter tuning on the validation set of the dataset, and perform grid search-based hyperparameter optimization for the weight factor λ∈{0.1,0.2,0.3,0.4,0.5} and the temperature parameter τ∈{0.1,1,10}. The hidden layer size of the decoder is set to 768, with 8 attention heads and a 2-layer structure. The batch size during training is 128. Finally, we use the model with the highest BLEU-4 score on the validation set to report the experimental results of the test set, and the final decoding is performed according to the greedy algorithm.

[0098] 4.2) Experimental results

[0099] a) ECG-CLIP performance verification

[0100] See also Figure 3, we make a detailed comparison of the feature extraction performance of the ECG-CLIP module by different types of feature extractors (text supervision, weak text supervision, no text supervision):

[0101] Weak text supervisor (VIT and InceptionTime): In the five-classification experiment, it is pre-trained through the anomaly classification task. Although it can extract some features, it performs poorly in complex semantic generation tasks, especially in text generation indicators such as BLEU and CIDEr. The scores are relatively low (such as BLEU-4 is 0.158 and 0.175, CIDEr is 1.352 and 1.275), proving that it is weak in adaptability to complex report generation.

[0102] Text-free supervisor (ST-MEM): This model performs spatiotemporal modeling through self-supervised learning. Although it can capture the spatiotemporal characteristics of ECG signals, it lacks language alignment capabilities and performs poorly in generating reports, especially in capturing subtle timing features and generating natural language (e.g., BLEU-4 is only 0.188 and CIDEr is 1.333).

[0103] Text-supervised feature extractors (ECG-CLIP and MERL): Through multimodal alignment, the language mapping capability of ECG signals is significantly improved. In particular, the ECG-CLIP module we proposed performs well in indicators such as BLEU-1 and CIDEr, with a BLEU-1 score of 0.420 and a CIDEr score of 1.721, which is significantly better than other models. This verifies the effectiveness of enhancing the quality of ECG report generation through text alignment, and proves that ECG-CLIP can better capture complex semantic features and abnormal signals.

[0104] b) Performance verification of cross-modal alignment mechanism

[0105] See also Figure 4 To further verify the effectiveness of the WCL (weakly supervised contrastive learning) module, we conducted an ablation experiment. The results show that after removing the WCL module, the model performance is significantly reduced, especially in terms of diversity of report generation and abnormal signal capture. Specifically, the BLEU-4 of the CLIP encoder drops from 0.254 to 0.228, and the CIDEr drops from 1.931 to 1.721. This shows that WCL is crucial to capturing complex semantic features and abnormal signals.

[0106] After removing the WCL module from other models (such as ST-MEM and MERL), various key indicators showed different degrees of decline. MERL's BLEU-4 dropped from 0.235 to 0.228, and CIDEr dropped from 1.771 to 1.685; ST-MEM's BLEU-4 dropped from 0.219 to 0.188, and CIDEr dropped from 1.700 to 1.333.

[0107] These results further prove that the WCL module effectively improves the detection accuracy of the model when processing complex semantics and rare abnormal signals through the cross-modal alignment mechanism.

[0108] In summary, the method proposed in this paper proposes an improved ECG signal report generation model, targeting the key challenges in ECG report generation, such as the scarcity of abnormal signals, timing dependencies, and subtle differences in report text. By introducing the ECG-CLIP text supervised feature extraction method, cross-modal learning mechanism and weakly supervised alignment loss (L WCL ), we significantly improved the accuracy and diversity of report generation. This method makes full use of the association between ECG signals and medical texts, and enhances the model's ability to capture abnormal signals through multimodal alignment and feature extraction. In the experiment, our model achieved excellent results in multiple evaluation indicators, verifying the effectiveness of the proposed method. Future work will further optimize the model structure and explore more methods for the fusion of medical signals and texts, in order to play a greater role in clinical diagnosis.

[0109] Example 2

[0110] This embodiment provides an automatic generation device for an electrocardiogram signal report, comprising:

[0111] A data acquisition module, used to acquire the ECG signal data to be processed;

[0112] A report generation module, used for inputting the acquired ECG signal data into a report generation model and outputting an ECG report;

[0113] The report generation model is obtained by improving the encoder-decoder architecture; an ECG-clip text-supervised feature extraction method is used to capture subtle timing features in electrocardiogram signals; a cross-modal alignment mechanism is introduced to align electrocardiogram signal features with medical text features to enhance the model's anomaly detection capability.

[0114] Since the device is an automatic ECG signal report generating device of an embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation process of the above method embodiment, and the repeated parts will not be repeated.

[0115] Example 3

[0116] An embodiment of the present invention further provides an electronic device, the electronic device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the following Figure 1 A method for automatically generating an electrocardiogram signal report is shown.

[0117] It is understood that the memory may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data created according to the use of the server, etc.

[0118] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect the various parts of the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes the operating system and application programs; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but implemented separately through a chip.

[0119] Since the electronic device is an electronic device corresponding to a method for automatically generating an electrocardiogram signal report in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0120] Example 4

[0121] The embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 1 A method for automatically generating an electrocardiogram signal report is shown.

[0122] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0123] Since the storage medium is a storage medium corresponding to a method for automatically generating an ECG signal report in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0124] Example 5

[0125] In some possible implementations, various aspects of the method of the embodiment of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the method for automatically generating an ECG signal report according to various exemplary embodiments of the present application described above in this specification. Among them, the executable computer program code or "code" for executing each embodiment may be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, structured query language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0126] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0127] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0128] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically generating an electrocardiogram signal report, characterized in that: The following steps are involved: Obtaining the ECG signal data to be processed; Input the acquired ECG signal data into a report generation model and output an ECG report; The report generation model is obtained by improving the encoder-decoder architecture; an ECG-clip text-supervised feature extraction method is used to capture features in electrocardiogram signals; a cross-modal alignment mechanism is introduced to align electrocardiogram signal features with medical text features to enhance the model's anomaly detection capability.

2. The method for automatically generating an ECG signal report according to claim 1, characterized in that: The report generation model includes an ECG-clip module and a Transformer Decoder; The training steps of the report generation model include: In the text generation stage, the ECG-clip module is used to extract the electrocardiogram signal E i Extract the feature H x , and use TransformerDecoder to generate the corresponding ECG report; through the cross entropy loss L CE To optimize the matching degree between the generated report and the reference report; by introducing the weak supervision contrast loss L WCL Let's focus on the subtle differences between the reported text, that is, the total loss function is: THE loss =(1-λ)L CE +λL WCL In the formula, λ is the weight parameter; By jointly optimizing the weakly supervised contrast loss and the cross entropy loss, the report generation model can generate more diverse and accurate reports.

3. The method for automatically generating an ECG signal report according to claim 2, characterized in that: The training step of the report generation model also includes the following steps: In the data processing stage, the BERT model is used to process the preset ECG report Y i Semantic feature extraction is performed and semantic feature representation is obtained through [CLS] embedding; then, K-means clustering is used to group the ECG reports and generate cluster labels l j , used to improve the semantic understanding ability of the model during training.

4. The method for automatically generating an ECG signal report according to claim 2, characterized in that: The ECG-clip module works as follows: The ECG signal data and the corresponding report text are taken as input and integrated into a multimodal contrastive learning framework, which includes a language processing component and an ECG encoder, which are used to obtain the embedding representation of the text and the embedding representation of the ECG respectively; both the language processing component and the ECG encoder map the text embedding and the ECG embedding into a vector space of the same dimension through a linear projection head; In this vector space, by calculating the similarity between ECG embedding and text embedding, the contrastive learning loss function is used for optimization so that the embeddings of the relevant ECG and report text are closer, thereby minimizing the contrastive loss.

5. The method for automatically generating an ECG signal report according to claim 4, characterized in that: To make full use of the prior clinical knowledge in the report text, the report text is input into a large frozen language model, and the ECG encoder adopts inception time as the backbone network.

6. The method for automatically generating an ECG signal report according to claim 1, characterized in that: The cross-modal alignment mechanism works as follows: Positive sample pair (E i ,Y i );E i is the input ECG signal, and the corresponding text report Y i Constitute a positive sample pair; by maximizing the similarity of the positive sample pair, the report generation model can better capture the semantic connection between the ECG signal and the text; the negative sample pair (E i ,Y j ): Y j is the input ECG signal E i Irrelevant reports, which constitute negative sample pairs; by minimizing the similarity of negative sample pairs, the report generation model can reduce the occurrence of wrong pairings; Weakly supervised contrast loss: Weakly supervised contrast loss is applied to ECG report generation, and the hidden representations of ECG and generated report are projected into the latent space: In the formula, and They are the hidden states h from the last layer output of Encoder-Decoder. X and h Y The average pooling result of x and φ y There are two fully connected layers with ReLU activation function; then, the model is optimized by maximizing the similarity between positive samples and minimizing the similarity between negative sample pairs.

7. The method for automatically generating an ECG signal report according to claim 6, characterized in that: The weakly supervised contrast loss L wCL The expression is: In the formula, α is a hyperparameter that controls the weight of weak supervision, τ is a temperature parameter, and N is the number of words. is the hidden representation of the positive sample pair, is the hidden representation of negative samples, l i is the cluster label of the positive sample pair, l j is the cluster label of negative samples.

8. An automatic generation device for an electrocardiogram signal report, characterized in that: include: A data acquisition module, used to acquire the ECG signal data to be processed; A report generation module, used for inputting the acquired ECG signal data into a report generation model and outputting an ECG report; The report generation model is obtained by improving the encoder-decoder architecture; the ECG-clip text supervision feature extraction method is used to capture the subtle timing features in the electrocardiogram signal; The cross-modal alignment mechanism aligns ECG signal features with medical text features to enhance the model's anomaly detection capability.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

Images