A method and related device for electrocardiogram representation learning and report generation

Abstract

The application belongs to the technical field of time sequence signal analysis, and aims at the problems that in existing electrocardio analysis, local features are not corresponding to real cardiac cycles, sparse abnormal heart beat information is easy to be weakened, and electrocardio report generation is not fully utilized for local abnormal information, provides an electrocardio representation learning and report generation method and related device, acquires electrocardio signals to be analyzed and encodes to obtain electrocardio time feature sequence; R peak detection is performed on the electrocardio signals, heart beat query is generated based on the R peak position, and attention pooling is performed on the electrocardio time feature sequence by using the heart beat query to obtain heart beat level representation; the heart beat importance weight is determined according to the heart beat level representation, and each time feature is modulated by using the heart beat importance weight; the modulated electrocardio time feature sequence is used as the cross attention context of the language decoder to generate the electrocardio report. The application can extract heart beat features around the real cardiac cycle, and enhance the utilization of heart beat information with diagnostic value in report generation.

Description

Technical Field

[0001] This application belongs to the field of time series signal analysis technology, specifically relating to an electrocardiogram representation learning and report generation method and related apparatus. Background Technology

[0002] Electrocardiography (ECG) records cardiac electrical activity and is an important diagnostic tool for screening, diagnosing, and following up on cardiovascular diseases in clinical practice. ECG signals typically include multiple leads and continuous-time sampling data. Local waveforms such as the QRS complex, ST segment, and T wave within different cardiac cycles, as well as the overall rhythm information formed between multiple heartbeats, can reflect different cardiac electrophysiological states. When interpreting ECGs, clinicians usually first observe the local heartbeat morphology and then combine the relationships between multiple heartbeats to form an overall diagnostic conclusion. Therefore, intelligent ECG analysis needs to consider both local heartbeat information and overall temporal information in continuous ECG signals. With the development of deep learning technology, using neural networks for feature extraction, disease classification, and diagnostic report generation of ECG signals has become an important direction for intelligent ECG analysis. Since some abnormal waveforms with diagnostic value may only appear in a few heartbeats, if the ECG representation process mainly focuses on the overall information of the entire signal, the morphological features contained in local abnormal heartbeats are easily weakened during the global feature aggregation process. Meanwhile, ECG reports need to reflect diagnostic information corresponding to the actual waveforms, thus the report generation process places higher demands on the preservation and utilization of local abnormal information in ECG representation.

[0003] Existing deep learning-based ECG analysis methods typically begin by temporally encoding continuous ECG signals, then perform classification, representation learning, or report generation based on the encoded ECG features. Some methods compress the entire ECG signal into a global representation using average pooling or global pooling; others, to obtain local information, divide the ECG signal into fixed-length time segments or extract local features using random learnable queries or uniform time grids. For ECG report generation, existing schemes usually employ an encoder-decoder structure, using the ECG encoding results as input or context for a language model, which then generates the corresponding clinical report. Furthermore, self-supervised learning and ECG-language multimodal learning have also been used for ECG representation learning, reducing reliance on manually labeled data and utilizing diagnostic information from clinical texts. However, in existing local feature extraction methods, there is a lack of clear correspondence between random queries, fixed-length time segments, or uniform time grids and the actual cardiac cycle. The extracted local features may not correspond to the actual heartbeat location, making it difficult to form a heartbeat-level representation around real cardiac physiological events. On the other hand, different heartbeats contribute differently to ECG diagnosis. Some abnormalities may only exist in a small number of heartbeats, and when using average pooling, equal-weighted aggregation, or relying mainly on global representation, the information contained in abnormal heartbeats is easily diluted by a large number of normal heartbeats. For ECG report generation, if the language decoding process mainly relies on globally or uniformly processed ECG features, it may be difficult to fully utilize the few diagnostically valuable local heartbeat information, thus easily producing overly generalized descriptions or omitting sparse abnormalities. At the same time, existing models often fail to clearly reflect the contribution of different heartbeats and different time regions to the report generation results, leading to insufficient physiological interpretability of ECG representation and report generation processes. Summary of the Invention

[0004] This application addresses the technical problems in existing electrocardiogram (ECG) analysis, such as insufficient correspondence between local features and the actual cardiac cycle, and the easy weakening of sparse abnormal cardiac information, which leads to insufficient utilization of local abnormal information in ECG report generation. It provides an ECG representation learning and report generation method and related apparatus.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, this application proposes a method for ECG representation learning and report generation, including: The electrocardiogram (ECG) signal to be analyzed is acquired, encoded, and the ECG time feature sequence is obtained. R-peak detection is performed on the ECG signal to be analyzed to obtain multiple R-peak positions. Based on each R-peak position, a corresponding heartbeat query is generated. Attention pooling is then performed on the ECG time feature sequence using the heartbeat query to obtain the heartbeat level representation corresponding to each R-peak. The corresponding cardiac importance weights are determined based on the cardiac grade characteristics described above; Based on the correspondence between each time feature in the ECG time feature sequence and the position of the R peak, each time feature is modulated using the corresponding heartbeat importance weight to obtain the modulated ECG time feature sequence. The modulated ECG time feature sequence is used as the cross-attention context of the language decoder, and the language decoder generates the ECG report corresponding to the ECG signal to be analyzed.

[0006] Furthermore, the encoding of the electrocardiogram signal to be analyzed includes: The ECG signal to be analyzed is input into the ECG encoder to obtain the ECG time feature sequence with time structure preserved. The ECG encoder includes a convolutional network and a Transformer encoder; the convolutional network is used to extract local waveform features of the ECG signal to be analyzed, and the Transformer encoder is used to perform contextual modeling on the local waveform features to obtain an ECG time feature sequence.

[0007] Furthermore, the step of generating the corresponding heart rate query based on the position of each R peak includes: The positions of each R-peak are normalized based on the signal length of the ECG signal to be analyzed, and the normalized positions of each R-peak are obtained. Position encoding is performed on each normalized position to obtain the heart rate query corresponding to each R peak.

[0008] Furthermore, the attention pooling of the ECG time feature sequence using heartbeat queries includes: Using each heartbeat query as the query and the ECG time feature sequence as the key and value for attention calculation, we obtain the heartbeat level representation corresponding to each R peak. The attention calculation employs either a single-head attention mechanism or a multi-head attention mechanism.

[0009] Furthermore, the modulation of each time feature using the corresponding cardiac rhythm importance weights includes: For each time feature in the electrocardiogram time feature sequence, determine the heartbeat corresponding to the R peak that is closest to that time feature; The time feature is linearly projected and layered normalized, and the time feature after linear projection and layered normalization is weighted using the heartbeat importance weights corresponding to the determined heartbeats to obtain the corresponding modulated time feature.

[0010] Further, based on each of the aforementioned cardiac grade characteristics, the corresponding cardiac importance weights are determined, including: The importance calculation module determines the corresponding importance score based on each cardiac level representation, and normalizes each importance score to obtain the cardiac importance weight corresponding to each cardiac level representation. The importance calculation module employs one of the following: multilayer perceptron, gating network, attention network, graph neural network, or scoring network incorporating clinical priors.

[0011] Furthermore, after determining the corresponding cardiac importance weights based on each of the aforementioned cardiac level characteristics, the process also includes: Output the heart rate importance weights and R-peak-guided attention maps corresponding to each heart rate level representation; wherein the R-peak-guided attention map is generated based on the attention calculation results obtained when the heart rate query performs attention pooling on the ECG time feature sequence.

[0012] Secondly, this application proposes a method for ECG representation learning and report generation, including: The encoding module is used to acquire the electrocardiogram (ECG) signal to be analyzed, encode the ECG signal to be analyzed, and obtain the ECG time feature sequence. The pooling module is used to detect R-peaks in the ECG signal to be analyzed, obtain multiple R-peak positions, generate corresponding heartbeat queries based on each R-peak position, and use the heartbeat queries to perform attention pooling on the ECG time feature sequence to obtain the heartbeat level representation corresponding to each R-peak. The weighting module is used to determine the corresponding cardiac importance weight based on each cardiac level characterization. The modulation module is used to modulate each time feature according to the correspondence between each time feature and the position of the R peak in the ECG time feature sequence, and to obtain the modulated ECG time feature sequence by using the corresponding heartbeat importance weight. The decoding module is used to use the modulated ECG time feature sequence as the cross-attention context of the language decoder, and the language decoder generates the ECG report corresponding to the ECG signal to be analyzed.

[0013] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described electrocardiogram representation learning and report generation method.

[0014] Fourthly, this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described electrocardiogram representation learning and report generation method.

[0015] Compared with the prior art, this application has the following beneficial effects: This application proposes an electrocardiogram (ECG) representation learning and report generation method. It encodes the ECG signal to be analyzed to preserve its temporal structure and generates corresponding heartbeat queries based on the R-peak positions, establishing a correspondence between the heartbeat queries and the actual cardiac cycle. Then, it uses these heartbeat queries to perform attention pooling on the ECG time feature sequence to obtain heartbeat-level representations corresponding to each R-peak. This enables the extraction of local heartbeat features around real physiological events, reducing local representation bias caused by inconsistencies between random queries, fixed time segments, or uniform time grids and the actual cardiac cycle. Furthermore, it determines the corresponding heartbeat importance weights based on each heartbeat-level representation and, according to the correspondence between each time feature and the R-peak position, assigns the corresponding heartbeat importance weights to the appropriate heartbeats. The importance weighting of beats is applied to ECG time features, so that beats with different diagnostic values ​​have different degrees of influence on subsequent report generation. This helps to highlight a few abnormal beats with diagnostic significance and reduce the problem of abnormal information being diluted by a large number of normal beats. On this basis, the modulated ECG time feature sequence is used as the cross-attention context of the language decoder, which enables the language decoder to make more use of beat segments with high diagnostic value when generating ECG reports. This reduces the interference of irrelevant normal beats on report generation and the omission of local abnormal information, making the generated ECG report more consistent with the actual ECG waveform and improving the physiological consistency and interpretability of ECG representation and report generation process.

[0016] This application also proposes an electrocardiogram (ECG) representation learning and report generation system, an electronic device, and a computer-readable storage medium, which possess all the advantages of ECG representation learning and report generation methods. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating one of the electrocardiogram representation learning and report generation methods of this application; Figure 2 This is a general architecture diagram of one embodiment of this application; Figure 3 This is a schematic diagram of R-peak-guided heart rate query in an embodiment of this application; Figure 4 This is a schematic diagram of importance-aware rhythm aggregation in the embodiments of this application; Figure 5 This is a schematic diagram of the ECG representation learning and report generation system of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] This application proposes an electrocardiogram (ECG) representation learning and report generation method and related apparatus. For the ECG signal to be analyzed, while preserving the continuous time structure, the method guides the extraction of local heartbeat information by using the actual cardiac cycle position corresponding to the R peak. Furthermore, the method modulates the time features used for ECG report generation according to the relative importance of different heartbeats, thereby improving the utilization of local abnormal heartbeat information in the report generation process.

[0022] like Figure 1 The diagram shown is a flowchart of one method for ECG representation learning and report generation according to this application, which may include: S101, acquire the ECG signal to be analyzed, encode the ECG signal to be analyzed, and obtain the ECG time feature sequence.

[0023] The encoding result retains multiple temporal features along the time direction, allowing local waveform information at different time points to continue to correspond to the corresponding time regions in the original ECG signal, rather than being directly compressed into a single global representation during the encoding stage. Therefore, subsequent processing can further establish the relationship between heartbeat location, local heartbeat representation, and the report generation context within a continuous temporal structure.

[0024] S102, R-peak detection is performed on the ECG signal to be analyzed to obtain multiple R-peak positions. Based on each R-peak position, a corresponding heartbeat query is generated, and the heartbeat query is used to perform attention pooling on the ECG time feature sequence to obtain the heartbeat level representation corresponding to each R-peak.

[0025] It should be noted that each R-peak position corresponds to a real physiological event in the continuous electrocardiogram (ECG) signal and can be used to characterize the position of the corresponding cardiac cycle. A corresponding heartbeat query is generated based on each R-peak position, so that different heartbeat queries carry positional guidance information related to the corresponding R-peak. Attention pooling is performed on the ECG time feature sequence using each heartbeat query. Each heartbeat query aggregates time features related to the corresponding R-peak and its corresponding cardiac cycle from the ECG time feature sequence to obtain the heartbeat-level representation corresponding to that R-peak. By guiding the heartbeat query with the R-peak position, local representation extraction can be made to revolve around the real cardiac cycle, reducing the inadequacy of the correspondence between local features and the real cardiac cycle when using fixed time divisions or local feature extraction methods lacking physiological position constraints.

[0026] S103, determine the corresponding cardiac importance weight based on each cardiac level characterization.

[0027] Different heartbeat levels reflect local morphological information within different cardiac cycles. The determined heartbeat importance weights are used to distinguish the relative contributions of different heartbeats to the current ECG signal analysis and subsequent ECG report generation. For abnormal waveforms that appear only in a few heartbeats or local morphologies with high diagnostic value, the corresponding heartbeats can obtain a representation intensity commensurate with their contribution through heartbeat importance weights, reducing the possibility of them being weakened by a large amount of normal heartbeat information.

[0028] S104. Based on the correspondence between each time feature in the ECG time feature sequence and the position of the R peak, the corresponding cardiac importance weights are used to modulate each time feature to obtain the modulated ECG time feature sequence.

[0029] This application preserves the organizational relationship of temporal features on a continuous time axis, while mapping the importance of cardiac beats back to the corresponding time regions, so that cardiac beats with different diagnostic contributions can affect the subsequent language decoding process to different degrees.

[0030] S105, the modulated ECG time feature sequence is used as the cross-attention context of the language decoder, and the language decoder generates the ECG report corresponding to the ECG signal to be analyzed.

[0031] Because the context received by the language decoder retains both the temporal structure and is modulated by the importance weight of the heartbeat, it can improve the attention to the time region related to local abnormal heartbeats while utilizing the overall ECG information.

[0032] In summary, a continuous information transmission relationship is formed between the R-peak position, heartbeat query, heartbeat level representation, heartbeat importance weight, modulated ECG time feature sequence, and language decoder. This provides the report generation process with clear local physiological location basis, thereby addressing the problems of insufficient correspondence between local features and the real cardiac cycle, the easy weakening of sparse abnormal heartbeat information, and the resulting insufficient utilization of local abnormal information in ECG reports. This improves the ability to utilize key local waveform information in the ECG report generation process.

[0033] The present application will be further described in detail below through some more detailed embodiments.

[0034] In one embodiment of this application, a physiologically-aware ECG-language joint learning framework, referred to as PACE, can be used to complete ECG representation learning and ECG report generation. Clinical ECG interpretation typically relies on both local beat morphology and the overall rhythmic relationship between multiple beats. For example, local morphologies such as QRS complexes, ST segments, and T waves are observed first, and then continuous beats are combined to determine the overall rhythm, conduction abnormalities, myocardial ischemia, or infarction. Therefore, this embodiment explicitly introduces the physiological location corresponding to the R peak into the continuous temporal features, enabling the model to extract local beat information around the real cardiac cycle, and further modulates the temporal context used for ECG report generation using the importance of different beats.

[0035] like Figure 2 As shown, in a preferred training implementation, the overall process of the ECG representation learning and report generation method of this application includes ECG input, ECG encoding, R-peak detection, R-peak-guided heartbeat query generation, heartbeat-level attention pooling, heartbeat importance calculation, optional importance-aware rhythm aggregation, optional hierarchical text encoding, optional cross-modal alignment, and heartbeat-guided ECG report generation. During the training phase, pairs of multi-lead ECG signals and clinical reports can be input, and relevant model parameters are optimized through multi-granularity training relationships at the rhythm, heartbeat, and word levels. In the report generation phase after training is completed, ECG encoding, R-peak detection, heartbeat-level representation extraction, heartbeat importance calculation, temporal feature modulation, and language decoding are performed for new ECG signals to be analyzed; it is not required to input the corresponding artificial clinical report for the ECG signal to be analyzed.

[0036] In a preferred embodiment of this application, the input electrocardiogram signal is denoted as:

[0037] in, C Indicates the number of leads. TThis indicates the sampling length. As an example, the input can be a 12-lead, 10-second ECG recording with a 500 Hz sampling rate; the number of leads, sampling rate, and recording duration can be adjusted according to the actual acquisition equipment and application scenario. The clinical report paired with the training ECG signal is denoted as:

[0038] In the data preparation phase of model training, the input ECG signals and paired clinical reports can optionally be preprocessed. For ECG signals, missing or null values ​​can be removed, and amplitude normalization, necessary filtering, resampling, or segment truncation can be performed according to the network input requirements. For clinical reports, reports that are too short or lack effective diagnostic information can be filtered, and case uniformity, punctuation, and special character cleanup can be performed. Figure 2 One example of text preprocessing shown also includes chapter detection, sentence segmentation, lexicalization, and medical terminology normalization. These preprocessing methods are optional data preparation approaches and can be selected based on the quality of the input data and the training task.

[0039] The ECG signal to be analyzed is input into the ECG encoder. The time-preserving ECG time feature sequence was obtained:

[0040] in, N Indicates the number of time stamps. d The feature dimension is represented. In one embodiment of this application, the ECG encoder includes a convolutional network and a Transformer encoder. The convolutional network is used to extract local waveform features from the ECG signal to be analyzed, so that the local morphology of the QRS complex, ST segment, T wave and its adjacent regions can be mapped to continuous local features; the Transformer encoder further performs contextual modeling on the local waveform features, captures the rhythm dependence between different time positions, and forms an ECG time feature sequence.

[0041] Alternatively, the ECG encoder can also employ Wav2Vec 2.0, a state-space model, a temporal convolutional network, or other temporal coding networks. A combination of convolutional networks, Transformers, state-space models, or other temporal modeling units can also be used. When using an alternative coding structure, the processing method of outputting the ECG time feature sequence along the time direction is still retained, so that the corresponding cardiac level representation can be extracted subsequently based on the R-peak position.

[0042] R-peak detection is performed on the ECG signal to be analyzed to obtain the set of cardiac cycle positions:

[0043] in, Indicates the firstr The time index of the R-peak. R-peak detection can employ the Pan-Tompkins algorithm, wavelet-based detection methods, traditional signal processing methods, or trained neural network detectors. Alternatively, traditional thresholding methods, template matching methods, deep neural network detectors, or multi-lead joint detectors can also be used, as long as the location of the R-peak for locating the cardiac cycle can be obtained.

[0044] The positions of each R-peak are normalized based on the signal length of the ECG signal to be analyzed, resulting in the normalized positions corresponding to each R-peak:

[0045] Subsequently, position encoding was performed on each normalized position to obtain the heart rate query corresponding to each R peak. In a preferred embodiment, sine and cosine position encoding is used. For dimension indexing... j ,when j When it is even:

[0046] when j When it is an odd number:

[0047] The query matrix consists of all heart rate queries:

[0048] Through the above processing, each heartbeat query corresponds to an actual R-peak position from the moment it is generated. Compared with random initialization queries or uniform time grid initialization, R-peak-guided heartbeat queries have a clear basis for cardiac cycle location, making it easier for subsequent attention pooling to focus on the corresponding QRS complex and its adjacent P wave, ST segment, and T wave regions.

[0049] like Figure 3 As shown, six R peaks, R1 to R6, were continuously detected in the example ECG segment, and Query 1 to Query 6 were formed by the corresponding R peak positions. Figure 3 The horizontal axis corresponds to time or ECG time markers, while the vertical axis corresponds to different heartbeat queries. Attention weights range from 0 to 1, representing different levels of attention. The high-attention areas for each heartbeat query are concentrated near the corresponding R-peak and are distributed sequentially along the continuous cardiac cycle, thus reflecting the correspondence between the R-peak position, the heartbeat query, and the local time region.

[0050] Position coding is not limited to sine and cosine position coding. Alternatively, learnable position coding, relative position coding, radial basis function coding, or position coding fused with lead information can be used to enable heartbeat queries to carry physiological position information of the corresponding R peak.

[0051] Get the query matrix Q Then, each heartbeat is used as the query, and the electrocardiogram time characteristic sequence is used as the query. H Attention is calculated using the keys and values, and cardiac-level attention pooling is performed to obtain the cardiac-level representations corresponding to each R-peak:

[0052] And there are:

[0053] Each This corresponds to the local morphological features of the cardiac cycle near an R-peak. Attention calculation can employ either a single-head attention mechanism or a multi-head attention mechanism. The results of attention calculation reflect the degree of attention different cardiac queries pay to each time marker in the ECG time feature sequence. Since the cardiac query is guided by the R-peak position, the local representation can be formed around the corresponding cardiac cycle, thereby improving the physiological consistency of cardiac-level representation.

[0054] The importance weight of each heartbeat level is determined based on its representation. In a preferred embodiment, the importance calculation module employs a two-layer multilayer perceptron to calculate the importance weight of each heartbeat level. i Individual cardiac level characteristics Calculate importance score:

[0055] Expect, This represents the Sigmoid function. and These are learnable parameters. For all Normalization is performed to obtain the importance weights of heartbeats. The importance weights of all heartbeats are summed to 1. By weighting the importance of heartbeats, the diagnostic contribution of different heartbeats can be distinguished based on their level of representation, allowing heartbeats with abnormalities, key morphological changes, or rhythm inflection points to receive higher contributions in subsequent processing, rather than treating all heartbeats equally.

[0056] The importance calculation module is not limited to a two-layer multilayer perceptron. Optionally, a gating network, attention network, graph neural network, or a scoring network combined with clinical priors can be used to obtain the corresponding importance score based on the characteristics of each cardiac beat, and then further normalized to obtain the cardiac beat importance weight.

[0057] In a further embodiment of this application, the importance weight of heartbeats can be used for importance-perceived rhythm aggregation to construct a rhythm-level representation. R :

[0058] Through weighted aggregation, the cardiac-level representations with higher importance are compared to the rhythm-level representations. R This represents a significant contribution. The rhythm-level representation can be used for rhythm-level ECG-text alignment during the training phase, and also for downstream tasks requiring a holistic ECG representation.

[0059] like Figure 4 As shown, multiple consecutive heartbeats are scored and normalized to form different heartbeat importance weights, and participate in rhythm-level aggregation according to their respective weights. Figure 4 In the examples, the normalized importance weights corresponding to B1 to B7 are 0.07, 0.11, 0.26, 0.30, 0.14, 0.07, and 0.05, respectively. The regions corresponding to B3 and B4 are marked as abnormal heartbeat regions, with weights of 0.26 and 0.30 higher than most other example heartbeats. The above values ​​are used to illustrate the relationship between different heartbeat importance weights and the higher contribution of abnormal heartbeats, and do not constitute fixed thresholds or fixed weights in practical applications.

[0060] In a further training embodiment of this application, the paired clinical report Y can be decomposed into rhythmic-level text. Heartbeat-level text and word-level text Rhythm-level text is used to preserve overall diagnostic or rhythm conclusions; beat-level text is used to preserve local waveform or morphological descriptions, such as ST segment changes, QRS complex abnormalities, and T wave abnormalities; lexical-level text preserves the complete report sequence for autoregressive ECG report generation.

[0061] Text decomposition can be assisted by rule pipelines, medical dictionaries, diagnostic entity recognition models, syntactic analysis, or large language models. Optionally, keyword extraction or medical knowledge graphs can also be used for text layering. Each text view is input into the medical text encoder. This yields the corresponding text vector:

[0062] As an example, Figure 2 The text shown can be extracted using the MedCPT text encoder; this text encoder is an optional implementation and is not limited to a single structure.

[0063] In rhythm-level global alignment, ECG rhythm representation R The learning process involves comparing and contrasting the data with the rhythmic text vectors of the corresponding clinical reports. For a training batch, the... i The ECG and the first j The similarity between texts is calculated using the following formula:

[0064] in, The temperature parameter is learnable. A bidirectional InfoNCE loss is used for training, which ensures paired ECG-text samples are close to each other in a shared space, while keeping unpaired samples far apart.

[0065] In cardiac-level local alignment, the cardiac-level representation set B and the cardiac-level text representation are calculated. Attention matching scores are used to form an intra-batch pairing similarity matrix U, which is then trained using symmetric cross-entropy loss to establish a connection between local waveform anomalies and corresponding local descriptions in clinical reports. Thus, cross-modal correspondences are established at the corresponding granularities between overall rhythm conclusions and rhythm-level ECG representations, and between local morphological descriptions and heartbeat-level representations.

[0066] Optionally, the cross-modal alignment loss can be adopted or extended to include triplet loss, supervised contrast loss, matching classification loss, optimal transport alignment loss, or masked language modeling loss. The above-mentioned cross-modal training is a further implementation method in the model parameter learning stage. When using the trained model to generate the ECG report corresponding to the ECG signal to be analyzed, it is not necessary to obtain manually paired clinical reports again and perform the above-mentioned alignment training.

[0067] After obtaining the cardiac stroke importance weights corresponding to each cardiac stroke level, a time mapping is performed on these weights. This is done for each time feature in the ECG time feature sequence. Based on this time characteristic and the distance between the positions of each R peak, the nearest R peak is determined, and the corresponding heart rate index of the R peak is determined. Subsequently, the time feature is linearly projected and layer normalized, and the processed time feature is weighted using the heartbeat importance weights corresponding to the determined heartbeats. According to the representation method used in this embodiment, the time feature modulation relationship is as follows:

[0068] in, It is a linear projection matrix. LN Representation layer normalization. The above relationship remaps the importance of heartbeat levels to continuous time features, so that the time regions corresponding to heartbeats with high diagnostic value have corresponding feature contributions in the language decoding process.

[0069] After performing the above modulation on each time feature in the ECG time feature sequence, the modulated ECG time feature sequence is obtained. The modulated ECG time feature sequence As the cross-attention context for the language decoder, the language decoder generates ECG report terms in an autoregressive manner. Language modeling loss is used during training.

[0070] This processing enables the language decoder to make greater use of the time regions corresponding to heartbeats with high diagnostic value when generating report content, and reduces the interference of a large number of irrelevant normal heartbeats on sparse abnormal information.

[0071] Language decoders can employ Transformer decoders, medical language models, lightweight autoregressive models, or retrieval-enhanced generative models. Different language decoders can utilize modulated ECG time feature sequences as cross-attention context or conditional context corresponding to cross-attention functions.

[0072] In a further training embodiment of this application, the overall training objective is composed of rhythm-level alignment loss, heartbeat-level alignment loss, and report generation loss:

[0073] in, and This refers to the weighting coefficient. As a preferred setting, , It can also be adjusted according to the scale of training data and task objectives. Through end-to-end training, the parameters related to rhythm-level representation, heartbeat-level representation, cross-modal alignment, and ECG report generation can be optimized simultaneously, enabling the same model to acquire discriminative representation capabilities, cross-modal alignment capabilities, and generative ECG report generation capabilities.

[0074] In one embodiment of this application, the importance weights of each heartbeat level and an R-peak-guided attention map can also be output. The R-peak-guided attention map is generated based on the attention calculation results obtained when the heartbeat query performs attention pooling on the ECG time feature sequence, and is used to demonstrate the degree of attention different heartbeat queries pay to each time region in the ECG time feature sequence; the heartbeat importance weights are used to demonstrate the relative contribution of different heartbeats in the rhythm-level aggregation or ECG report generation process. The combination of these two methods provides interpretable output from two levels: the query's area of ​​interest and the relative contribution of heartbeats.

[0075] Figure 2 The data relationships between the ECG side, text side, cross-modal alignment, and heartbeat-guided ECG report generation are further illustrated: On the ECG side, the ECG time feature sequence is obtained from the ECG input, and a heartbeat-level representation is formed through R-peak detection, R-peak-guided heartbeat query, and attention pooling. Heartbeat importance weights are then calculated. Optionally, the heartbeat-level representations are weighted and aggregated to form a rhythm-level representation. On the text side, the clinical report undergoes hierarchical processing to form rhythm-level, heartbeat-level, and word-level text features. Rhythm-level features are used for global alignment, heartbeat-level features for local alignment, and word-level text features participate in report generation training. The heartbeat importance weights are also mapped back to the ECG time feature sequence to modulate the cross-attention context of the language decoder. Thus, Figure 2The modules shown correspond to the continuous processing chain of local heartbeat extraction, importance calculation, multi-granularity training, and ECG report generation.

[0076] In practical applications, after training, ECG encoding is performed on new ECG signals to be analyzed to obtain ECG time feature sequences; R-peak detection is performed and heartbeat queries corresponding to each R-peak are generated; attention pooling is performed using the heartbeat queries to obtain heartbeat-level representations corresponding to each R-peak; heartbeat importance weights are calculated based on each heartbeat-level representation; based on the correspondence between each time feature and the R-peak position, the heartbeat importance weights are mapped back to the corresponding time features, and linear projection, layer normalization, and weighted modulation are performed; the modulated ECG time feature sequences are used as the cross-attention context of the language decoder to generate ECG reports corresponding to the ECG signals to be analyzed. This inference process does not rely on paired clinical reports used in the training phase.

[0077] In addition to 12-lead resting electrocardiograms, the above processing can optionally be used for single-lead wearable electrocardiograms, Holter monitoring, and cardiac monitoring. Furthermore, for heart sounds or other medical time-series signals with periodic physiological events, when location anchors corresponding to the periodic physiological events can be obtained, appropriate local event-guided methods can be used to form local representations and perform importance modulation.

[0078] In one example of model training and validation, a publicly available ECG dataset is used to pre-train the aforementioned framework, perform downstream classification, zero-shot classification, cross-domain transfer, ablation, and ECG report generation validation. The data settings and corresponding results are described below.

[0079] The pre-training data used MIMIC-IV-ECG v1.0, and after quality control, 745,929 valid ECG-text samples were obtained for training. Downstream classification used PTB-XL, CPSC2018, and CSN, achieving robust leadership in multiple linear detection tasks, with significant advantages in low-label data scenarios.

[0080] Zero-shot classification, using PTBXL-Super, PTBXL-Sub, PTBXL-Form, PTBXL-Rhythm, CPSC2018, and CSN, achieved an average performance of 78.57, higher than MERL, K-MERL, and MELP. Cross-domain migration, employing multiple source-to-target domain combinations, achieved an average performance of 81.63, outperforming the strongest baseline by approximately 2.00 percentage points.

[0081] In the CPSC2018 ablation experiment, the complete model achieved scores of 84.59, 86.93, 90.75, and 92.32 at zero-sample, 1%, 10%, and 100% linear detection settings, respectively. ECG report generation was performed using PTB-XL, and the complete model outperformed the comparative methods on BLEU-1, BLEU-4, METEOR, ROUGE-L, and BERTScore.

[0082] Ablation experiments also showed that removing R-peak-guided heartbeat query, importance-aware rhythm aggregation, hierarchical text semantic decomposition, or heartbeat-guided ECG report generation all resulted in a decrease in model performance; among them, the decrease was most significant in the zero-sample setting when hierarchical text semantic decomposition was removed. Replacing importance-aware rhythm aggregation with average pooling also reduced performance. These results indicate that each key processing step contributes to the overall model performance.

[0083] In the ECG report generation instance, the full model on PTB-XL outperformed ECGGFM, MERL, MELP, and variants using only token-level ECG context on metrics such as BLEU, METEOR, ROUGE-L, and BERTScore, demonstrating that mapping the importance weights of heartbeats back to ECG time feature sequences and using them in the language decoder context can improve the quality of ECG report generation.

[0084] like Figure 5 The diagram shown is a schematic representation of the ECG representation learning and report generation system of this application, which may include: The encoding module is used to acquire the electrocardiogram (ECG) signal to be analyzed, encode the ECG signal to be analyzed, and obtain the ECG time feature sequence. The pooling module is used to detect R-peaks in the ECG signal to be analyzed, obtain multiple R-peak positions, generate corresponding heartbeat queries based on each R-peak position, and use the heartbeat queries to perform attention pooling on the ECG time feature sequence to obtain the heartbeat level representation corresponding to each R-peak. The weighting module is used to determine the corresponding cardiac importance weight based on each cardiac level characterization. The modulation module is used to modulate each time feature according to the correspondence between each time feature and the position of the R peak in the ECG time feature sequence, and to obtain the modulated ECG time feature sequence by using the corresponding heartbeat importance weight. The decoding module is used to use the modulated ECG time feature sequence as the cross-attention context of the language decoder, and the language decoder generates the ECG report corresponding to the ECG signal to be analyzed.

[0085] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of each block is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple blocks may be combined or integrated into another device, or some features may be ignored or not executed. The modules described as separate components may or may not be physically separated. The components shown as modules may be one or more physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0086] Furthermore, the modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0087] This application also provides an electronic device, which may include one or more processors, memory and communication interfaces.

[0088] The memory, communication interface, and processor are coupled together. For example, the memory, communication interface, and processor can be coupled together via a bus.

[0089] The communication interface is used for data transmission with other devices. The memory stores computer program code. This computer program code includes computer instructions, which, when executed by the processor, cause the electronic device to perform the steps of the aforementioned ECG representation learning and report generation method.

[0090] The processor can be a processor or controller, such as a Central Processing Unit (CPU), a general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The processor can be used to support an electronic device in performing the method steps provided in the above embodiments.

[0091] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. These buses can be categorized as address buses, data buses, control buses, etc.

[0092] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the above-described electrocardiogram representation learning and report generation method.

[0093] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage media known in the art.

[0094] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for learning and generating electrocardiogram (ECG) representations, characterized in that, include: The electrocardiogram (ECG) signal to be analyzed is acquired, encoded, and the ECG time feature sequence is obtained. R-peak detection is performed on the ECG signal to be analyzed to obtain multiple R-peak positions. Based on each R-peak position, a corresponding heartbeat query is generated. Attention pooling is then performed on the ECG time feature sequence using the heartbeat query to obtain the heartbeat level representation corresponding to each R-peak. The corresponding cardiac importance weights are determined based on the cardiac grade characteristics described above; Based on the correspondence between each time feature in the ECG time feature sequence and the position of the R peak, each time feature is modulated using the corresponding heartbeat importance weight to obtain the modulated ECG time feature sequence. The modulated ECG time feature sequence is used as the cross-attention context of the language decoder, and the language decoder generates the ECG report corresponding to the ECG signal to be analyzed.

2. The method for learning and generating electrocardiogram representations according to claim 1, characterized in that, The encoding of the electrocardiogram signal to be analyzed includes: The ECG signal to be analyzed is input into the ECG encoder to obtain the ECG time feature sequence with time structure preserved. The ECG encoder includes a convolutional network and a Transformer encoder; the convolutional network is used to extract local waveform features of the ECG signal to be analyzed, and the Transformer encoder is used to perform contextual modeling on the local waveform features to obtain an ECG time feature sequence.

3. The ECG representation learning and report generation method according to claim 1, characterized in that, The generation of corresponding heart rate queries based on the position of each R peak includes: The positions of each R-peak are normalized based on the signal length of the ECG signal to be analyzed, and the normalized positions of each R-peak are obtained. Position encoding is performed on each normalized position to obtain the heart rate query corresponding to each R peak.

4. The method for ECG representation learning and report generation according to claim 1, characterized in that, The attention pooling of the ECG time feature sequence using heartbeat query includes: Using each heartbeat query as the query and the ECG time feature sequence as the key and value for attention calculation, we obtain the heartbeat level representation corresponding to each R peak. The attention calculation employs either a single-head attention mechanism or a multi-head attention mechanism.

5. The ECG representation learning and report generation method according to claim 1, characterized in that, The modulation of each time feature using the corresponding cardiac rhythm importance weights includes: For each time feature in the electrocardiogram time feature sequence, determine the heartbeat corresponding to the R peak that is closest to that time feature; The time feature is linearly projected and layered normalized, and the time feature after linear projection and layered normalization is weighted using the heartbeat importance weights corresponding to the determined heartbeats to obtain the corresponding modulated time feature.

6. The ECG representation learning and report generation method according to claim 1, characterized in that, The corresponding cardiac importance weights are determined based on the cardiac grade characteristics described above, including: The importance calculation module determines the corresponding importance score based on each cardiac level representation, and normalizes each importance score to obtain the cardiac importance weight corresponding to each cardiac level representation. The importance calculation module employs one of the following: multilayer perceptron, gating network, attention network, graph neural network, or scoring network incorporating clinical priors.

7. The method for learning and generating electrocardiogram representations according to claim 1, characterized in that, After determining the corresponding cardiac importance weights based on the cardiac grade characteristics described above, the method further includes: Output the heart rate importance weights and R-peak-guided attention maps corresponding to each heart rate level representation; wherein the R-peak-guided attention map is generated based on the attention calculation results obtained when the heart rate query performs attention pooling on the ECG time feature sequence.

8. A method for learning and generating electrocardiogram representations, characterized in that, include: The encoding module is used to acquire the electrocardiogram (ECG) signal to be analyzed, encode the ECG signal to be analyzed, and obtain the ECG time feature sequence. The pooling module is used to detect R-peaks in the ECG signal to be analyzed, obtain multiple R-peak positions, generate corresponding heartbeat queries based on each R-peak position, and use the heartbeat queries to perform attention pooling on the ECG time feature sequence to obtain the heartbeat level representation corresponding to each R-peak. The weighting module is used to determine the corresponding cardiac importance weight based on each cardiac level characterization. The modulation module is used to modulate each time feature according to the correspondence between each time feature and the position of the R peak in the ECG time feature sequence, and to obtain the modulated ECG time feature sequence by using the corresponding heartbeat importance weight. The decoding module is used to use the modulated ECG time feature sequence as the cross-attention context of the language decoder, and the language decoder generates the ECG report corresponding to the ECG signal to be analyzed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the ECG representation learning and report generation method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the ECG representation learning and report generation method as described in any one of claims 1-7.

Images