An electrocardiogram multi-label classification method based on knowledge coding and related devices

Abstract

Embodiments of the present application disclose an electrocardiogram multi-label classification method based on knowledge coding and related devices. Embodiments of the present application include preprocessing the collected multi-lead electrocardiogram signal to obtain a preprocessed multi-lead electrocardiogram signal; constructing knowledge coding; based on a signal and knowledge embedding module, merging the preprocessed multi-lead electrocardiogram signal, lead knowledge coding, and time sequence knowledge coding into a signal data block; based on an encoder composed of a multi-head self-attention layer, performing feature learning on the signal data block to obtain a signal feature block; based on a knowledge learning module, classifying and identifying the signal feature block according to the category knowledge coding. The present application not only solves the problem of too few labels on data in the existing method, but also can be compatible with different lead formats. In addition, the present application also obtains high interpretability for each electrocardiogram label category, has good classification effect, and improves the accuracy of electrocardiogram multi-label classification.

Description

Technical Field

[0001] This application relates to the field of computer science and technology, specifically to a knowledge-encoding-based multi-label classification method for electrocardiograms and related devices. Background Technology

[0002] An electrocardiogram (ECG) is a graphical representation of the electrical activity changes of the heart during each cardiac cycle, recorded from the body surface using an electrocardiograph. This data is then analyzed by a computer to detect arrhythmias and myocardial ischemia that are not easily detected during routine surface ECG examinations. This provides important objective evidence for clinical diagnosis, treatment, and evaluation of treatment efficacy.

[0003] In the development of multi-label ECG diagnostic models, deep learning-based methods have achieved significant results. However, currently published multi-label ECG diagnostic methods still have some shortcomings before clinical application, or insufficient research is needed in certain aspects. For example, currently published methods are mainly based on a complete 10-second 12-lead ECG. While the original ECG acquisition in clinical diagnosis includes 10 seconds of 12 leads, the most commonly used format in ECG reports is the 3×4 format, which is a 2.5-second 12-lead ECG plus a 10-second lead II format. This is also the diagnostic format commonly used by doctors and the data storage format typically used in hospitals. Because the 3×4 format significantly reduces the data volume compared to a complete 10-second 12-lead ECG, most current methods are not backward compatible with this format. In addition, existing technologies not only have too few data labels, but also, for multi-label tasks, a single result may have multiple classifications, making it difficult to distinguish the key areas corresponding to each classification from a single heatmap, resulting in poor multi-label ECG classification performance. Summary of the Invention

[0004] In view of the above problems, this specification is proposed to provide a knowledge-encoding-based electrocardiogram multi-label classification method and related apparatus to overcome or at least partially solve the above problems.

[0005] Other features and advantages disclosed herein will become apparent from the following detailed description, or may be learned in part by practice of this disclosure.

[0006] In a first aspect, embodiments of this application provide a knowledge-encoding-based electrocardiogram (ECG) multi-label classification method, comprising: preprocessing the acquired multi-lead ECG signals to obtain preprocessed multi-lead ECG signals; constructing knowledge encoding, which includes lead knowledge encoding, time-series knowledge encoding, and category knowledge encoding; merging the preprocessed multi-lead ECG signals, lead knowledge encoding, and time-series knowledge encoding into a signal data block based on a signal and knowledge embedding module; performing feature learning on the signal data block based on an encoder composed of multi-head self-attention layers to obtain signal feature blocks; and classifying and recognizing the signal feature blocks according to the category knowledge encoding based on a knowledge learning module to obtain the label type corresponding to the multi-lead ECG signals.

[0007] In some embodiments, preprocessing the acquired multi-lead electrocardiogram (ECG) signals to obtain preprocessed multi-lead ECG signals includes: acquiring multi-lead ECG signals based on a preset sampling frequency, wherein the multi-lead ECG signals include a first signal matrix and a second signal matrix; converting the second signal matrix to the same dimension as the first signal matrix to obtain a dimension-transformed second signal matrix; merging the dimension-transformed second signal matrix with the first signal matrix to obtain a merged multi-lead ECG signal; linearly transforming the merged multi-lead ECG signal into sequence blocks based on a preset feature dimension of the model, wherein the sequence blocks are three-dimensional matrices; and performing dimension transformation on the sequence blocks to obtain a dimension-transformed two-dimensional matrix, wherein the dimension-transformed two-dimensional matrix is ​​the preprocessed multi-lead ECG signal.

[0008] In some embodiments, constructing knowledge encoding includes: constructing lead knowledge encoding based on the model's preset feature dimensions and the number of leads, wherein the lead knowledge encoding is L. k The parameter matrix of ×D, L k The number of leads is represented by D, and the model's preset feature dimension is represented by t. A time-series knowledge encoding is constructed based on the model's preset feature dimension, the signal length with a time period of t, and the length of each block in the sequence block. The time-series knowledge encoding is as follows: The parameter matrix, L t L represents the length of a signal with a time period of t. npatch This represents the length of each block in the sequence; a category knowledge encoding is constructed based on the model's preset feature dimensions and the number of label categories, and the category knowledge encoding is L. type The parameter matrix of ×D, L type This indicates the number of label categories.

[0009] In some embodiments, constructing the knowledge encoding further includes training the knowledge encoding using a two-stage training method; using the signal and knowledge embedding module and encoder as the first-stage model, in the first stage of the two-stage training method, the first-stage model is used to train the lead knowledge encoding and the time-series knowledge encoding, including: acquiring training multi-lead ECG signals, wherein the training multi-lead ECG signals are preprocessed multi-lead ECG signals collected during the training of the knowledge encoding; randomly initializing the lead knowledge encoding and the time-series knowledge encoding to obtain initialized lead knowledge encoding and initialized time-series knowledge encoding; inputting the initialized lead knowledge encoding and the initialized time-series knowledge encoding together with the training multi-lead ECG signals into the first-stage model to obtain the first-stage signal feature block. The initialized lead knowledge encoding and the first-stage signal feature blocks are input into the knowledge learning module. The knowledge learning module outputs the predicted time point and predicted lead for each signal feature block in the preprocessed multi-lead ECG signal. Based on the trained multi-lead ECG signal, the predicted leads are linearly projected onto the corresponding lead signals to obtain the projected lead signals. The lead loss value between the projected lead signals and the original lead signals is calculated. Based on the position of each signal feature block in the preprocessed multi-lead ECG signal within the signal length of time period t, the predicted time points are compared to obtain the time point loss value. Based on the lead loss value and the time point loss value, the initialized lead knowledge encoding and the initialized time series knowledge encoding are adjusted respectively to obtain the trained lead knowledge encoding and the trained time series knowledge encoding.

[0010] In some embodiments, a two-stage training method is used to train the knowledge encoding, further comprising using a knowledge learning module as a second-stage model. In the second stage of the two-stage training method, the second-stage model is used to train the category knowledge encoding, including: initializing the category knowledge encoding to obtain initialized category knowledge encoding; inputting the trained lead knowledge encoding, the trained time series knowledge encoding, and the trained multi-lead ECG signal into the first-stage model, and the first-stage model outputting a training signal feature sequence block; inputting the initialized category knowledge encoding and the training signal feature sequence block into the knowledge learning module, and the knowledge learning module outputting a predicted category knowledge encoding; inputting the predicted category knowledge encoding into a fully connected layer, and the fully connected layer outputting the classification result corresponding to the predicted category knowledge encoding; calculating the loss value between the classification result and the classification label based on the classification label; and adjusting the initialized category knowledge encoding based on the loss value to obtain the trained category knowledge encoding.

[0011] In some embodiments, based on the signal and knowledge embedding module, the preprocessed multi-lead ECG signal, lead knowledge encoding, and time series knowledge encoding are merged into a signal data block. This includes: merging the preprocessed multi-lead ECG signal, lead knowledge encoding, and time series knowledge encoding into a signal encoding block containing lead and time series knowledge, wherein the signal encoding block containing lead and time series knowledge is a three-dimensional matrix; and performing low-dimensional feature mapping on the signal encoding block containing lead and time series knowledge through a multilayer perceptron to obtain a signal data block. The signal data block is the signal encoding block containing lead and time series knowledge after low-dimensional feature mapping. The multilayer perceptron includes a fully connected layer, a GELU activation function, and a signal block random discarding layer.

[0012] In some embodiments, the knowledge learning module includes several knowledge learning layers connected in sequence and a last knowledge learning output layer. Each knowledge learning layer includes a knowledge attention layer and a knowledge self-attention layer. The last knowledge learning output layer is a knowledge attention layer. The knowledge attention layer uses a knowledge attention module, the knowledge self-attention layer uses a self-attention module, and the multi-head self-attention layer uses multiple self-attention modules.

[0013] In some embodiments, the knowledge-encoding-based ECG multi-label classification method further includes: both the self-attention module and the knowledge attention module employ the same attention architecture, which includes an attention module and a multilayer perceptron layer; the output of the attention architecture is z. l =LN(MLP(z′) l ))+z′ l LN represents normalization, MLP represents a multilayer perceptron, and z′ l This represents the residual output of the attention module, z′. l =LN(ATTEN(z) l-1 ))+z l-1 ATTEN represents an attention module, which can be a self-attention module or a knowledge-based attention module; z l-1 The input sequence represents the attention architecture.

[0014] In some embodiments, an encoder based on a multi-head self-attention layer performs feature learning on a signal data block to obtain a signal feature block, including: performing a linear projection transformation on the signal data block to obtain three output matrices, namely a query matrix Q, a key matrix K, and a value matrix V; splitting the query matrix Q, the key matrix K, and the value matrix V into several groups of matrices on a preset feature dimension of the model, with each group of matrices corresponding to a self-attention module, and the several groups of matrices corresponding to a multi-head self-attention layer; obtaining the output matrices of the self-attention modules corresponding to the several groups of matrices according to the multi-head self-attention layer; merging the output matrices of the self-attention modules corresponding to the several groups of matrices to obtain a merged self-attention module matrix; and performing a linear projection on the merged self-attention module matrix to obtain a signal feature block, the dimension of which is consistent with the dimension of the output matrix.

[0015] In some embodiments, based on the multi-head self-attention layer, several sets of self-attention module output matrices corresponding to different sets of matrices are obtained, including: the i-th set of key matrices K i Transpose the matrix to obtain the i-th transpose key matrix K. i T ; Transpose the i-th group of transpose key matrix K i T With the i-th query matrix Q i Perform dot product processing to obtain the first dot product value Q. i K i T Based on scaling factor The dot product value Q i K i T Multiply by the scaling factor to obtain the multiplication matrix. Multiply matrices With the i-th value matrix V i Perform dot product processing to obtain the second dot product value. Calculate the second dot product value using the softmax function. The corresponding softmax value yields the output matrix of the self-attention module corresponding to the i-th group of matrices.

[0016] In some embodiments, based on the knowledge learning module, signal feature blocks are classified and identified according to category knowledge encoding to obtain the label type corresponding to the multi-lead ECG signal. This includes: processing the signal feature blocks using any one of the knowledge attention modules in the knowledge learning module, including: performing a linear projection transformation on the knowledge sequence block to obtain a linearly projected knowledge sequence block q; performing two linear projection transformations on the signal feature blocks to obtain a first signal feature output matrix k and a second signal feature output matrix v; performing attention calculations on each knowledge block in the linearly projected knowledge sequence block on the first signal feature output matrix k and the second signal feature output matrix v in sequence to obtain several knowledge attention matrices; merging the several knowledge attention matrices to obtain a merged knowledge attention matrix; calculating the softmax value corresponding to the merged knowledge attention matrix based on the softmax function; and performing a dot product operation between the softmax value corresponding to the merged knowledge attention matrix and the second signal feature output matrix v to obtain the output of the knowledge attention module.

[0017] In some embodiments, attention calculations are performed sequentially on the first signal feature output matrix k and the second signal feature output matrix v for each knowledge block in the linearly projected knowledge sequence block to obtain several knowledge attention matrices, including: the j-th knowledge block q in the linearly projected knowledge sequence block. j Broadcast computation is performed by adding the first signal feature output matrix k to obtain the broadcast computation matrix; the tanh value corresponding to the broadcast computation matrix is ​​calculated using the tanh activation function; based on a learnable parameter matrix w... j , combine the tanh value and the parameter matrix w j Perform dot product processing to obtain the j-th knowledge block q. j The corresponding attention matrix; traverse the knowledge sequence blocks after linear projection transformation to obtain several knowledge attention matrices corresponding to each knowledge block.

[0018] Secondly, embodiments of this application provide a knowledge-encoding-based electrocardiogram (ECG) multi-label classification device, comprising: a preprocessing module for preprocessing acquired multi-lead ECG signals to obtain preprocessed multi-lead ECG signals; an encoding construction module for constructing knowledge encoding, including lead knowledge encoding, time-series knowledge encoding, and category knowledge encoding; a signal merging module for merging the preprocessed multi-lead ECG signals, lead knowledge encoding, and time-series knowledge encoding into signal data blocks based on a signal and knowledge embedding module; a feature learning module for performing feature learning on the signal data blocks based on an encoder composed of multi-head self-attention layers to obtain signal feature blocks; and a classification module for classifying and recognizing the signal feature blocks according to the category knowledge encoding based on the knowledge learning module to obtain the label type corresponding to the multi-lead ECG signals.

[0019] Thirdly, embodiments of this application provide an electronic device, including a memory storing multiple instructions; a processor loads instructions from the memory to execute steps in any of the knowledge-encoding-based electrocardiogram multi-label classification methods provided in embodiments of this application.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium storing multiple instructions adapted for loading by a processor to execute steps in any of the knowledge-encoding-based electrocardiogram multi-label classification methods provided in embodiments of this application.

[0021] In this embodiment, the acquired multi-lead ECG signals are first preprocessed to obtain preprocessed multi-lead ECG signals; then, knowledge encoding is constructed, including lead knowledge encoding, time series knowledge encoding, and category knowledge encoding; then, based on the signal and knowledge embedding module, the preprocessed multi-lead ECG signals, lead knowledge encoding, and time series knowledge encoding are merged into signal data blocks; next, based on an encoder composed of multi-head self-attention layers, feature learning is performed on the signal data blocks to obtain signal feature blocks; finally, based on the knowledge learning module, the signal feature blocks are classified and identified according to the category knowledge encoding to obtain the label type corresponding to the multi-lead ECG signals.

[0022] This invention is designed for complete multi-label scenarios, utilizing a large number of labels to address the problem of insufficient labels in existing methods, while also being compatible with different lead formats. Furthermore, by combining the temporal signal attributes of the ECG signal itself, the spatial angular mapping attributes of the human heart between leads, and the classification characteristics of the task, this invention proposes a knowledge-encoding-based computation framework, specifically composed of lead knowledge encoding, time-series knowledge encoding, and category knowledge encoding. Another aspect is that the knowledge encoding captures corresponding attention from signal features through a knowledge attention layer. Since the focus differs between categories, this invention treats each category as a category knowledge encoding, allowing each category to learn its own unique attention and feature capture methods. Moreover, the lead encoding and time-series encoding of this invention consider the spatial location of the human heart and the temporal signal attributes of the ECG signal, respectively, enhancing the signal expression in both aspects. Additionally, through category knowledge encoding, based on the proposed knowledge attention layer, this invention can obtain an attention map on the ECG signal corresponding to each category. Thus, this invention achieves high interpretability and good classification results for each ECG label category, thereby improving the accuracy of multi-label ECG classification. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of a scenario for the knowledge-encoding-based electrocardiogram multi-label classification method provided in an embodiment of this application;

[0025] Figure 2a This is a flowchart illustrating the knowledge-encoding-based electrocardiogram multi-label classification method provided in this application embodiment;

[0026] Figure 2b This is a schematic diagram of a 3×4 format sample of a clinical electrocardiogram provided in the embodiments of this application;

[0027] Figure 2c This is a schematic diagram of the classification model structure provided in the embodiments of this application;

[0028] Figure 2d This is a schematic diagram of the attention architecture provided in an embodiment of this application;

[0029] Figure 2e This is a schematic diagram of the multi-head self-attention layer structure provided in the embodiments of this application;

[0030] Figure 2f This is a schematic diagram of the knowledge attention module structure provided in an embodiment of this application;

[0031] Figure 3 This is a schematic diagram of the structure of the knowledge-encoding-based electrocardiogram multi-label classification device provided in the embodiments of this application;

[0032] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] This application provides a knowledge-encoding-based electrocardiogram multi-label classification method and related apparatus.

[0035] Specifically, the knowledge-encoding-based electrocardiogram multi-label classification device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, or personal computer (PC); the server can be a single server or a server cluster consisting of multiple servers.

[0036] In some embodiments, the knowledge-encoding-based electrocardiogram (ECG) multi-label classification device can also be integrated into multiple electronic devices. For example, the knowledge-encoding-based ECG multi-label classification device can be integrated into multiple servers, and the knowledge-encoding-based ECG multi-label classification method of this application can be implemented by multiple servers.

[0037] In some embodiments, the server may also be implemented as a terminal.

[0038] For example, refer to Figure 1 This is a schematic diagram of a scenario for a knowledge-encoding-based electrocardiogram multi-label classification method provided in the embodiments of this application. It may include a server 100 and a storage terminal 110. The server 100 and the storage terminal 110 can communicate with each other, which will not be described in detail here.

[0039] Storage terminal 110 can store acquired multi-lead ECG signals, etc. Server 100 can first preprocess the acquired multi-lead ECG signals to obtain preprocessed multi-lead ECG signals; then construct knowledge encoding, including lead knowledge encoding, time series knowledge encoding, and category knowledge encoding; then, based on the signal and knowledge embedding module, merge the preprocessed multi-lead ECG signals, lead knowledge encoding, and time series knowledge encoding into signal data blocks; next, based on an encoder composed of multi-head self-attention layers, perform feature learning on the signal data blocks to obtain signal feature blocks; finally, based on the knowledge learning module, classify and identify the signal feature blocks according to the category knowledge encoding to obtain the label type corresponding to the multi-lead ECG signals, etc.

[0040] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0041] In this embodiment, a knowledge-encoding-based multi-label electrocardiogram classification method is provided, such as... Figure 2a As shown, when applied to a server, the specific process of this knowledge-encoding-based multi-label ECG classification method can be as follows:

[0042] 200. The acquired multi-lead ECG signals are preprocessed to obtain preprocessed multi-lead ECG signals.

[0043] The 12-lead electrocardiogram commonly used in clinical practice is a multi-lead electrocardiogram, which is divided into limb leads and precordial leads. By placing electrodes in different parts of the body, it reflects the electrical activity of the heart and is the most widely used examination method in clinical practice.

[0044] Multi-lead electrocardiogram (ECG) signals are collected from the body surface by placing probes at any two points on the body surface at a certain distance apart. Currently, the internationally accepted ECG standard is composed of several leads. By acquiring signals from multiple leads, the origin and abnormality of heart disease can be indirectly inferred. For example, in the most common case of myocardial infarction, different lead combinations can indirectly indicate the location of myocardial ischemia or damage; for instance, it can be determined whether it is an inferior, anterior, or posterior myocardial infarction. Therefore, ECG is currently the simplest and most convenient examination for diagnosing heart disease.

[0045] In some embodiments, preprocessing the acquired multi-lead electrocardiogram (ECG) signals to obtain preprocessed multi-lead ECG signals includes: acquiring multi-lead ECG signals based on a preset sampling frequency, wherein the multi-lead ECG signals include a first signal matrix and a second signal matrix; converting the second signal matrix to the same dimension as the first signal matrix to obtain a dimension-transformed second signal matrix; merging the dimension-transformed second signal matrix with the first signal matrix to obtain a merged multi-lead ECG signal; linearly transforming the merged multi-lead ECG signal into sequence blocks based on a preset feature dimension of the model, wherein the sequence blocks are three-dimensional matrices; and performing dimension transformation on the sequence blocks to obtain a dimension-transformed two-dimensional matrix, wherein the dimension-transformed two-dimensional matrix is ​​the preprocessed multi-lead ECG signal.

[0046] In the embodiments of this application, such as Figure 2b As shown, the model input can be the clinically common 3×4 ECG format, consisting of a 2.5-second 12-lead input plus a 10-second II-lead input.

[0047] Specifically, the preset sampling frequency used in this embodiment can be a 500Hz sampling rate, corresponding to a 12×1250 dimension first signal matrix and a 1×5000 dimension second signal matrix as the actual ECG signal input. In data preprocessing, this embodiment can convert the 1×5000 dimension second signal matrix into a 4×1250 dimension matrix, then merge the 4×1250 dimension matrix and the 12×1250 dimension first signal matrix, and discard the repeated 2.5-second lead II signal, thereby forming a 15×1250 matrix as the signal input, thus obtaining the preprocessed multi-lead ECG signal.

[0048] In this embodiment, the acquired multi-lead electrocardiogram signals are preprocessed and then linearly transformed into sequence blocks, i.e., the sequence blocks can be based on the length L of each block.npatch (For example, set to 25) Divide the signal into a sequence block, and then perform self-attention calculation between blocks.

[0049] Specifically, for example, embodiments of this application can be based on the length L of each block in the sequence block. npatch =25 division of multi-lead ECG signals, after linear transformation the matrix size of the sequence block is 15×50×D, where D is the preset feature dimension of the model. The knowledge encoding dimension and the input dimension of each module in the model are consistent. In this embodiment, it can be set to 256. Then, the sequence block with dimension 15×50×D is flattened and transformed into a signal encoding block with a matrix size of dimension 750×D, that is, the two-dimensional matrix after dimension transformation is obtained. The two-dimensional matrix after dimension transformation is the preprocessed multi-lead ECG signal.

[0050] 210. Construct knowledge coding, which includes lead knowledge coding, time series knowledge coding, and category knowledge coding.

[0051] In some embodiments, constructing knowledge encoding includes: constructing lead knowledge encoding based on the model's preset feature dimensions and the number of leads, wherein the lead knowledge encoding is L. k The parameter matrix of ×D, L k The number of leads is represented by D, and the model's preset feature dimension is represented by t. A time-series knowledge encoding is constructed based on the model's preset feature dimension, the signal length with a time period of t, and the length of each block in the sequence block. The time-series knowledge encoding is as follows: The parameter matrix, L t L represents the length of a signal with a time period of t. npatch This represents the length of each block in the sequence; a category knowledge encoding is constructed based on the model's preset feature dimensions and the number of label categories, and the category knowledge encoding is L. type The parameter matrix of ×D, L type This indicates the number of label categories.

[0052] In this embodiment of the application, the knowledge encoding is specifically composed of a set of L k The parameter matrix of ×D is composed of L k This refers to the corresponding knowledge length. In the lead knowledge encoding of this application embodiment, L k The length can be 12, corresponding to 12 leads. In time series knowledge encoding, L k The length can be Here L 10s This is a 10-second signal length, corresponding to 5000 signal points. In classification knowledge encoding, L... k The length is the number of categories for the classification task. In this embodiment, 111 categories can be used, and the length is set to 111.

[0053] In some embodiments, constructing the knowledge encoding further includes training the knowledge encoding using a two-stage training method; using the signal and knowledge embedding module and encoder as the first-stage model, in the first stage of the two-stage training method, the first-stage model is used to train the lead knowledge encoding and the time-series knowledge encoding, including: acquiring training multi-lead ECG signals, wherein the training multi-lead ECG signals are preprocessed multi-lead ECG signals collected during the training of the knowledge encoding; randomly initializing the lead knowledge encoding and the time-series knowledge encoding to obtain initialized lead knowledge encoding and initialized time-series knowledge encoding; inputting the initialized lead knowledge encoding and the initialized time-series knowledge encoding together with the training multi-lead ECG signals into the first-stage model to obtain the first-stage signal feature block. The initialized lead knowledge encoding and the first-stage signal feature blocks are input into the knowledge learning module. The knowledge learning module outputs the predicted time point and predicted lead for each signal feature block in the preprocessed multi-lead ECG signal. Based on the trained multi-lead ECG signal, the predicted leads are linearly projected onto the corresponding lead signals to obtain the projected lead signals. The lead loss value between the projected lead signals and the original lead signals is calculated. Based on the position of each signal feature block in the preprocessed multi-lead ECG signal within the signal length of time period t, the predicted time points are compared to obtain the time point loss value. Based on the lead loss value and the time point loss value, the initialized lead knowledge encoding and the initialized time series knowledge encoding are adjusted respectively to obtain the trained lead knowledge encoding and the trained time series knowledge encoding.

[0054] In this embodiment, for the training methods of three types of knowledge encoding—lead knowledge encoding, time series knowledge encoding, and category knowledge encoding—this invention uses a two-stage training method to train them separately. In the first stage, lead knowledge encoding and time series knowledge encoding are trained first. Both encodings are randomly initialized and then input into the first-stage model along with the training multi-lead ECG signal. After passing through the signal and knowledge embedding module and the encoder, signal feature sequence blocks are obtained. Then, the lead knowledge encoding and signal feature sequence blocks are input together into the knowledge learning module. The knowledge learning module outputs the lead knowledge encoding, which is then linearly projected onto the corresponding lead signal. The projected lead signal is compared with the original signal to calculate the L1 loss function. Additionally, the signal feature blocks are input into the time header and lead header of the knowledge learning module, and the output is the time point and lead corresponding to each signal feature block. In this embodiment, the label of the time point is the position of the current signal block within a signal length of time period t (e.g., 10 seconds). Thus, after the first stage of training, this embodiment obtains the trained lead knowledge encoding and the trained time series knowledge encoding.

[0055] In some embodiments, a two-stage training method is used to train the knowledge encoding, further comprising using a knowledge learning module as a second-stage model. In the second stage of the two-stage training method, the second-stage model is used to train the category knowledge encoding, including: initializing the category knowledge encoding to obtain initialized category knowledge encoding; inputting the trained lead knowledge encoding, the trained time series knowledge encoding, and the trained multi-lead ECG signal into the first-stage model, and the first-stage model outputting a training signal feature sequence block; inputting the initialized category knowledge encoding and the training signal feature sequence block into the knowledge learning module, and the knowledge learning module outputting a predicted category knowledge encoding; inputting the predicted category knowledge encoding into a fully connected layer, and the fully connected layer outputting the classification result corresponding to the predicted category knowledge encoding; calculating the loss value between the classification result and the classification label based on the classification label; and adjusting the initialized category knowledge encoding based on the loss value to obtain the trained category knowledge encoding.

[0056] In this embodiment, the second stage of training the classification knowledge encoding first initializes the classification knowledge encoding, which is then input along with the already trained lead knowledge encoding, temporal knowledge encoding, and signal. Similar to the first stage, the lead knowledge encoding, temporal knowledge encoding, and signal are processed by a signal and knowledge embedding module and an encoder to obtain a signal feature sequence block. Then, the classification knowledge encoding and the signal feature sequence block are input to a knowledge learning module, which outputs the classification knowledge encoding. Finally, the classification knowledge encoding is passed through a fully connected layer to output the corresponding classification result. The output classification result and the label are then used to calculate the loss value using the cross-entropy loss function.

[0057] In some embodiments, the knowledge learning module includes several knowledge learning layers connected in sequence and a last knowledge learning output layer. Each knowledge learning layer includes a knowledge attention layer and a knowledge self-attention layer. The last knowledge learning output layer is a knowledge attention layer. The knowledge attention layer uses a knowledge attention module, the knowledge self-attention layer uses a self-attention module, and the multi-head self-attention layer uses multiple self-attention modules.

[0058] In this embodiment, a classification model is used to classify multi-lead electrocardiogram (ECG) signals. The classification model mainly includes an encoder composed of multi-head self-attention layers and a knowledge learning module. The multi-head self-attention layer comprises several self-attention layers, each corresponding to a self-attention module. The knowledge learning module consists of multiple knowledge attention layers and multiple knowledge self-attention layers. For example... Figure 2cAs shown, the knowledge learning module in this embodiment can be configured with two knowledge learning layers and one knowledge learning output layer, namely, a first knowledge attention layer, a first knowledge self-attention layer, a second knowledge attention layer, a second knowledge self-attention layer, and a third knowledge attention layer. The third knowledge attention layer corresponds to the last knowledge learning output layer.

[0059] The knowledge attention layer in this embodiment uses each lead or classification code to capture corresponding features from the signal feature block. In the subsequent knowledge self-attention layer, self-attention calculation is performed between the knowledge codes after capturing signal attention. Because different knowledge does not exist independently but has a mutual dependency relationship, different knowledge codes can share knowledge. This process is repeated multiple times, and finally, the knowledge code is the task output.

[0060] In this embodiment, each lead knowledge is used to reconstruct the corresponding original signal in the first stage of self-supervised learning, and each classification knowledge is used to predict the corresponding anomaly in the second stage of supervised learning. Furthermore, in the first stage, signal blocks are fed into the lead header and time-series signal header in the knowledge learning module to predict the lead and time corresponding to each signal feature block. Specifically, in this embodiment, each signal feature block can be input into a lead header composed of a 3-layer MLP to predict the corresponding lead. The first stage is a multi-classification task, while each signal feature block can be input into a time-series signal header composed of a 3-layer MLP to predict the corresponding time node, which is a regression task. In this embodiment, lead knowledge encoding and time-series knowledge encoding are generated through the self-supervised learning method in the first stage, while category knowledge encoding is generated through supervised learning in the second stage. This embodiment, through the knowledge learning module, unifies the self-supervised learning and supervised learning tasks with a single structure, improving the model's prediction efficiency.

[0061] 220. Based on the signal and knowledge embedding module, the preprocessed multi-lead ECG signal, lead knowledge encoding, and time series knowledge encoding are merged into a signal data block.

[0062] In some embodiments, based on the signal and knowledge embedding module, the preprocessed multi-lead ECG signal, lead knowledge encoding, and time series knowledge encoding are merged into a signal data block. This includes: merging the preprocessed multi-lead ECG signal, lead knowledge encoding, and time series knowledge encoding into a signal encoding block containing lead and time series knowledge, wherein the signal encoding block containing lead and time series knowledge is a three-dimensional matrix; and performing low-dimensional feature mapping on the signal encoding block containing lead and time series knowledge through a multilayer perceptron to obtain a signal data block. The signal data block is the signal encoding block containing lead and time series knowledge after low-dimensional feature mapping. The multilayer perceptron includes a fully connected layer, a GELU (Gaussian Error Linear Unit) activation function, and a signal block random discarding layer.

[0063] In this embodiment, the signal coding block and the corresponding lead knowledge coding and time series knowledge coding are merged and mapped into a new signal coding block containing spatial and temporal attributes. First, the signal coding block after linear transformation (corresponding to the preprocessed multi-lead ECG signal) is registered with the corresponding lead knowledge coding and time series knowledge coding. The registration rule is as follows: each individual signal coding block, with a size of 1×D, is matched with a corresponding lead knowledge coding, also with a size of 1×D, based on its lead, and a corresponding time series knowledge coding, also with a size of 1×D, based on its position on the time axis. These are then matrix-merged, resulting in a 3×D coding block.

[0064] In this embodiment, the input signal encoding block of 750×D is registered, and the output is a 750×3D signal encoding block containing lead and temporal knowledge. Then, the signal encoding block undergoes low-dimensional feature mapping through a multilayer perceptron (MLP) in the neural network, i.e., the features are recompressed to 750×D. In this embodiment, the MLP includes a fully connected layer and a GELU activation function, and finally a signal block random discarding layer. This embodiment randomly discards integers of length α from the 750-digit signal encoding block by setting a discarding ratio α. In this embodiment, α can be set to 0.85 in the first stage of unsupervised learning training and 0.65 in the second stage of multi-label classification training. In the second stage of verification, the discarding ratio α is not 0, meaning that in actual clinical prediction, this embodiment will not discard any signal blocks, thus improving classification accuracy.

[0065] This application's embodiment uses a random dropout layer to increase the model's generalization and robustness, and mitigate overfitting, by masking random features during training. This application chooses a random dropout strategy instead of feature masking or occlusion because masking or occlusion does not reduce the number of parameters of the overall features involved in the calculation; it only sets the features that need to be masked to 0 to prevent them from taking effect. This application, through a signal block random dropout layer, significantly reduces the computational load of training for self-attention models where the computational cost increases exponentially with the length of the input sequence, increasing the model's training speed while simultaneously mitigating overfitting and increasing the model's generalization ability.

[0066] 230. Based on an encoder composed of multi-head self-attention layers, feature learning is performed on signal data blocks to obtain signal feature blocks.

[0067] like Figure 2c As shown, the encoder in this embodiment of the application consists of a multi-head self-attention layer, wherein the multi-head self-attention layer includes several self-attention modules connected in sequence. Signal data blocks are input to the multi-head self-attention layer, and the multi-head self-attention layer outputs signal feature blocks.

[0068] In some embodiments, an encoder based on a multi-head self-attention layer performs feature learning on a signal data block to obtain a signal feature block, including: performing a linear projection transformation on the signal data block to obtain three output matrices, namely a query matrix Q, a key matrix K, and a value matrix V; splitting the query matrix Q, the key matrix K, and the value matrix V into several groups of matrices on a preset feature dimension of the model, with each group of matrices corresponding to a self-attention module, and the several groups of matrices corresponding to a multi-head self-attention layer; obtaining the output matrices of the self-attention modules corresponding to the several groups of matrices according to the multi-head self-attention layer; merging the output matrices of the self-attention modules corresponding to the several groups of matrices to obtain a merged self-attention module matrix; and performing a linear projection on the merged self-attention module matrix to obtain a signal feature block, the dimension of which is consistent with the dimension of the output matrix.

[0069] In some embodiments, based on the multi-head self-attention layer, several sets of self-attention module output matrices corresponding to different sets of matrices are obtained, including: the i-th set of key matrices K i Transpose the matrix to obtain the i-th transpose key matrix K. i T ; Transpose the i-th group of transpose key matrix K i T With the i-th query matrix Q i Perform dot product processing to obtain the first dot product value Q. i K i T Based on scaling factor The dot product value Qi K i T Multiply by the scaling factor to obtain the multiplication matrix. Multiply matrices With the i-th value matrix V i Perform dot product processing to obtain the second dot product value. Calculate the second dot product value using the softmax function. The corresponding softmax value yields the output matrix of the self-attention module corresponding to the i-th group of matrices.

[0070] like Figure 2e As shown, in this embodiment, the input to the multi-head self-attention layer is a sequence block, i.e., a 2D matrix. l is the sequence length, and d is the dimension of each sequence block. First, the 2D matrix x is projected through three independent linear projections, outputting three matrices, each with a size of l×d. These three output matrices are the query matrix Q, the key matrix K, and the value matrix V. Then, the query matrix Q, key matrix K, and value matrix V are evenly divided into n groups along dimension d, meaning each group has a dimension of d / n. Here, n groups are called multiple groups. In this embodiment, n can be 8, d can be 256, and the dimension of each group is 32. This embodiment selects each group... Where K i Transpose it to become Then with Q i Perform the dot product calculation and multiply by the scaling factor. Here d k That is, the dimension d of the query matrix Q, the calculated output matrix is ​​in l×l format and softmax is performed to obtain the attention matrix in l×l format, and then the attention matrix and Perform a dot product operation to obtain the output matrix of the current head, in the following format: In this embodiment, each head is calculated in parallel to obtain n formats. The outputs are then combined to obtain an l×d output. Finally, a linear projection is performed to obtain a signal feature block in l×d format. The l×d format signal feature block maintains the same format as the input sequence block. The calculation formula for each header is:

[0071] 240. Based on the knowledge learning module, the signal feature blocks are classified and identified according to the category knowledge encoding to obtain the label type corresponding to the multi-lead ECG signal.

[0072] In some embodiments, the knowledge-encoding-based ECG multi-label classification method further includes: both the self-attention module and the knowledge attention module employ the same attention architecture, which includes an attention module and a multilayer perceptron layer; the output of the attention architecture is z. l =LN(MLP(z′) l ))+z′ l LN represents normalization, MLP represents a multilayer perceptron, and z′ l This represents the residual output of the attention module, z′. l =LN(ATTEN(z) l-1 ))+z l-1 ATTEN represents an attention module, which can be a self-attention module or a knowledge-attention module; zl -1 zl represents the sequence block input of the attention architecture, and zl represents the final output of the attention layer.

[0073] In this embodiment, both the knowledge attention layer and the knowledge self-attention layer use the same attention architecture design, but the key attention modules differ. The knowledge self-attention layer employs a multi-head self-attention layer, which can include multiple self-attention modules. The knowledge attention layer, on the other hand, uses a single knowledge attention module. Specifically, as... Figure 2d As shown, the attention architecture mainly consists of an attention module and an MLP layer, both of which are followed by a normalization module (Norm1 and Norm2, respectively), and both adopt a residual structure, that is, the input and output of the layer are added together.

[0074] In some embodiments, based on the knowledge learning module, signal feature blocks are classified and identified according to category knowledge encoding to obtain the label type corresponding to the multi-lead ECG signal. This includes: processing the signal feature blocks using any one of the knowledge attention modules in the knowledge learning module, including: performing a linear projection transformation on the knowledge sequence block to obtain a linearly projected knowledge sequence block q; performing two linear projection transformations on the signal feature blocks to obtain a first signal feature output matrix k and a second signal feature output matrix v; performing attention calculations on each knowledge block in the linearly projected knowledge sequence block on the first signal feature output matrix k and the second signal feature output matrix v in sequence to obtain several knowledge attention matrices; merging the several knowledge attention matrices to obtain a merged knowledge attention matrix; calculating the softmax value corresponding to the merged knowledge attention matrix based on the softmax function; and performing a dot product operation between the softmax value corresponding to the merged knowledge attention matrix and the second signal feature output matrix v to obtain the output of the knowledge attention module.

[0075] In some embodiments, attention calculations are performed sequentially on the first signal feature output matrix k and the second signal feature output matrix v for each knowledge block in the linearly projected knowledge sequence block to obtain several knowledge attention matrices, including: the j-th knowledge block q in the linearly projected knowledge sequence block. j Broadcast computation is performed by adding the first signal feature output matrix k to obtain the broadcast computation matrix; the tanh value corresponding to the broadcast computation matrix is ​​calculated using the tanh activation function; based on a learnable parameter matrix w... j , combine the tanh value and the parameter matrix w j Perform dot product processing to obtain the j-th knowledge block q. j The corresponding attention matrix; traverse the knowledge sequence blocks after linear projection transformation to obtain several knowledge attention matrices corresponding to each knowledge block.

[0076] Specifically, in this embodiment, the knowledge attention module has two inputs: the first is a sequence block of signal features. (Corresponding to the signal feature block), the second is the corresponding knowledge sequence block. Among them l s and l k These represent the lengths of the corresponding blocks, and d is the dimension of each block. For example... Figure 2f As shown, firstly, the knowledge sequence block x k After a linear projection, the knowledge sequence block is obtained after the linear projection transformation. Signal feature block x s The output after two linear projections is the first signal feature output matrix. Second signal characteristic output matrix Then, for each knowledge block in the knowledge sequence block after linear projection transformation, such as the j-th knowledge block q... j ∈R 1×d Attention calculations will be performed sequentially on the first signal feature output matrix k and the second signal feature output matrix v, specifically on the j-th knowledge block q. j ∈R 1×d and the first signal feature output matrix Perform a broadcast calculation of matrix addition to obtain an output format of l. s ×d, then calculate the tanh value corresponding to the broadcast-calculated matrix using the tanh activation function, and then combine the tanh value corresponding to the broadcast-calculated matrix with a learnable parameter matrix w. j ∈R d×1 The output format obtained by performing dot product calculation is l sA multiplier matrix of 1 (each knowledge block has its own independent parameter matrix, which is not shared between knowledge blocks). The specific formula is as follows: attentionmatrix(q j ,k)=tanh(q j +k)*w j Each signal feature block undergoes the above calculation sequentially to obtain l. k Each l s A ×1 attention matrix. Let l k Each l s After merging the ×1 attention matrices, we get l k ×l s The merged knowledge attention matrix, after being processed by the softmax activation function, is then combined with the second signal feature output matrix. Perform a dot product operation to obtain the final output format l k ×d, this output format is the input knowledge sequence block x k The format is consistent.

[0077] The network module in this embodiment mainly consists of three parts. The first part fuses the multi-lead ECG signal with lead knowledge encoding and time series knowledge encoding to obtain a spatial and temporal representation of the multi-lead ECG signal. The second part is an encoder composed of multiple consecutive multi-head self-attention layers that performs feature learning on the fused ECG signal. The third part uses a cross-attention knowledge learning module to perform attention learning on the corresponding category knowledge encoding and the ECG signal features output by the second encoder.

[0078] In this embodiment, each individual knowledge encoding captures the required unique features by performing attention calculations on all ECG signal features. The knowledge encoding here involves lead knowledge encoding in the first stage of training to reconstruct lead signals, and category knowledge encoding in the second stage to predict the positive or negative status of each category. In practical use, each abnormality category can have its corresponding attention matrix calculated to visualize the location of the corresponding ECG signal abnormality. This embodiment provides high interpretability for each category.

[0079] Specifically, this application embodiment used clinical electrocardiograms from a hospital for the experiment, wherein the experimental data, data preprocessing, experimental parameters, and experimental results are as follows:

[0080] (I) Experimental Data

[0081] The dataset used was derived from clinical electrocardiograms (ECGs) collected from hospitals between January 1, 2012, and December 31, 2021, comprising 946,373 ECGs from 603,144 patients. All ECGs were stored in a 3x4 format. The sampling rate was 500 Hz per second. A total of 111 categories were included, all derived from ECG diagnoses in clinical ECG reports.

[0082] (II) Data Preprocessing

[0083] The data input in this embodiment is 2.5 seconds of data from a 12-lead ECG and 10 seconds of data from lead II, with a sampling rate of 500 Hz per second and a signal unit of mV. These correspond to ECG signal matrices of sizes 12×1250 and 1×5000, respectively. In this embodiment, the 1×5000 matrix from lead II is transformed into a 4×1250 format (i.e., four 2.5-second signals), then combined with the 2.5-second data from the 12-lead ECG, and the duplicate 0-2.5 second lead II signals are removed, resulting in a 15×1250 matrix. The ECG signals are then clipped with upper and lower thresholds of -3 and 3, respectively; data exceeding 3mV is set to 3, and data below -3mV is set to -3. Finally, the clipped 15×1250 matrix is ​​input into the model.

[0084] In addition, experiments were also conducted on the publicly available dataset PTB-XL. PTB-XL contains 21,837 data points from 18,885 patients, in a complete 12-lead x 10-second format. To match the 3×4 clinical format, this embodiment of the application performed a segmentation, selecting only the corresponding 3×4 format data for the experiment. Specifically, the 0-2.5 second time intervals of leads I, II, and III; the 2.5-5 second time intervals of leads aVR, aVL, and aVF; the 5-7.5 second time intervals of leads V1, V2, and V3; the 7.5-10 second time intervals of leads V4, V5, and V6; and the complete 0-10 second time interval of lead II were selected. The data sampling rate was 500 Hz per second, containing a total of 70 categories. The training and validation sets were allocated in an 80% and 20% ratio, respectively. Furthermore, the segmentation was based on patient data, rather than electrocardiogram segmentation, to avoid multiple data points from the same patient appearing in both training and validation sets.

[0085] (III) Experimental Parameters

[0086] The experimental hardware environment consisted of one compute server with three NVIDIA GeForce RTX 3090 graphics cards, two Intel Xeon E5 CPUs, 128GB of RAM, and a 20TB hard drive. The software environment used the PyTorch deep learning framework for model building. The AdamW optimizer was used for training, with an initial learning rate of 0.0002 and a cosine decay curve as the learning rate decay strategy. In the first training round, the number of epochs was set to 300, and the batch size was 512. In the second training round, the number of epochs was set to 100, and the batch size was 128.

[0087] (IV) Experimental Results

[0088] In 111 classes of clinical data, four metrics were calculated: AUC, precision, recall, and F1 score, using both micro and macro statistical methods. Micro averaging involves calculating the metric for each class individually and then averaging the results. Macro averaging directly calculates the metrics needed for the sample without calculating them for each class individually. Experiments have shown that the micro AUC (area under the ROC curve) obtained by classifying the 111 classes of clinical data using the embodiments of this application reaches 99.39%, and the macro AUC reaches 96.23%. Furthermore, in the 70-class PTB-XL public dataset, experiments have shown that the micro AUC (area under the ROC curve) obtained by classifying the 70-class PTB-XL public dataset using the embodiments of this application reaches 93.74%, and the macro AUC reaches 85.92%.

[0089] This invention is designed for complete multi-label scenarios, utilizing a large number of labels to address the problem of insufficient labels in existing methods, while also being compatible with different lead formats. Furthermore, this invention combines the temporal signal attributes of the ECG signal itself, the spatial angular mapping attributes of the human heart between leads, and the classification characteristics of the task. On one hand, it proposes a knowledge-encoding-based computation framework, specifically composed of lead knowledge encoding, time-series knowledge encoding, and category knowledge encoding. On the other hand, it proposes that knowledge encoding captures corresponding attention from signal features through a knowledge attention layer. Since the focus differs between categories, this invention treats each category as a category knowledge encoding, allowing each category to learn its own unique attention and feature capture methods. Moreover, the lead encoding and time-series encoding of this invention consider the spatial location of the human heart and the temporal signal attributes of the ECG signal, respectively, enhancing the signal expression in these two aspects. Additionally, through category knowledge encoding, based on the proposed knowledge attention layer, this invention can obtain an attention map on the ECG signal corresponding to each category. Thus, this invention achieves high interpretability for each ECG label category, improving the accuracy of ECG multi-label classification.

[0090] In existing technologies, most methods focus on a few dozen categories. These methods either use overly generalized labels or only cover specific categories, while ECG clinical diagnosis encompasses hundreds of categories, making comprehensive diagnosis impossible with current methods. This application, in contrast to existing technologies, is designed for a complete multi-label scenario. Its design not only accommodates various lead formats but also enables comprehensive diagnosis.

[0091] Furthermore, existing technologies primarily rely on convolutional neural networks (CNNs), recurrent neural networks (RNNs), manual feature extraction (such as morphological feature extraction through wavelet transform), machine learning, or combinations of these methods. These existing technologies generally exhibit poor classification performance, and there is a lack of exploration of purely attention-based ECG models on ultra-large ECG datasets. Compared to existing technologies, this application achieves superior performance compared to CNNs and RNNs on massive datasets exceeding one million records.

[0092] Furthermore, existing technologies lack refinement in interpretability. Current conventional model heatmaps, i.e., attention and weight visualization maps (CAMs), are averages across all categories, rather than category-specific attention heatmaps. For multi-label tasks, where a result may fall into multiple categories, it's difficult to distinguish the key regions corresponding to each category from a single heatmap, as the key regions differ between categories. In contrast to existing technologies, this application is designed based on a pure attention mechanism. The model primarily includes a self-attention module and a knowledge-learning attention module (excluding CNNs, RNNs, and handcrafted features used in existing technologies). Addressing the temporal signal attributes of ECG signals and the spatial angular mapping between leads, as well as the classification characteristics of the task, this application proposes the following methods: a) a framework based on knowledge encoding, specifically composed of lead knowledge encoding, time-series knowledge encoding, and category knowledge encoding; b) knowledge encoding captures corresponding attention from signal features through a knowledge attention layer. This application targets multi-label and clinical 3x4 formats. Each category serves as a category knowledge encoding, allowing each category to learn its own unique attention and feature capture methods, as the focus differs between categories. Furthermore, lead coding and time-series coding consider the spatial location of the human heart on the electrocardiogram (ECG) and its temporal signal attributes, respectively, enabling the enhancement of the signal's expression in both aspects. Moreover, through category-based knowledge coding, this application, based on the proposed knowledge-based attention layer, can obtain an attention map on the ECG signal corresponding to each category, thus achieving high interpretability for each category.

[0093] To better implement the above methods, this application also provides a knowledge-encoding-based electrocardiogram (ECG) multi-label classification device. This knowledge-encoding-based ECG multi-label classification device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster consisting of multiple servers.

[0094] For example, in this embodiment, the method of this application embodiment will be described in detail by taking the integration of a knowledge-encoding-based electrocardiogram multi-label classification device into an electronic device as an example.

[0095] For example, such as Figure 3As shown, the knowledge-encoding-based electrocardiogram (ECG) multi-label classification device may include: a preprocessing module 300, an encoding construction module 310, a signal merging module 320, a feature learning module 330, and a classification module 340. The preprocessing module 300 is configured to preprocess the acquired multi-lead ECG signals to obtain preprocessed multi-lead ECG signals; the encoding construction module 310 is configured to construct knowledge encoding, which includes lead knowledge encoding, time-series knowledge encoding, and category knowledge encoding; the signal merging module 320 is configured to merge the preprocessed multi-lead ECG signals, lead knowledge encoding, and time-series knowledge encoding into signal data blocks based on a signal and knowledge embedding module; the feature learning module 330 is configured to perform feature learning on the signal data blocks based on an encoder composed of multi-head self-attention layers to obtain signal feature blocks; and the classification module 340 is configured to classify and identify the signal feature blocks according to the category knowledge encoding based on the knowledge learning module to obtain the label type corresponding to the multi-lead ECG signals.

[0096] In some embodiments, the preprocessing module 300 includes a preprocessing submodule, which is configured to: acquire multi-lead electrocardiogram (ECG) signals based on a preset sampling frequency, the multi-lead ECG signals including a first signal matrix and a second signal matrix; convert the second signal matrix to the same dimension as the first signal matrix to obtain a dimension-transformed second signal matrix; merge the dimension-transformed second signal matrix with the first signal matrix to obtain a merged multi-lead ECG signal; linearly transform the merged multi-lead ECG signal into a sequence block based on a preset feature dimension of the model, the sequence block being a three-dimensional matrix; and perform a dimension transformation on the sequence block to obtain a dimension-transformed two-dimensional matrix, the dimension-transformed two-dimensional matrix being the preprocessed multi-lead ECG signal.

[0097] In some embodiments, the encoding construction module 310 includes an encoding construction submodule, configured to: construct lead knowledge encoding based on the model's preset feature dimensions and the number of leads, wherein the lead knowledge encoding is L. k The parameter matrix of ×D, L k The number of leads is represented by D, and the model's preset feature dimension is represented by t. A time-series knowledge encoding is constructed based on the model's preset feature dimension, the signal length with a time period of t, and the length of each block in the sequence block. The time-series knowledge encoding is as follows: The parameter matrix, L t L represents the length of a signal with a time period of t. npatch This represents the length of each block in the sequence; a category knowledge encoding is constructed based on the model's preset feature dimensions and the number of label categories, and the category knowledge encoding is L. type The parameter matrix of ×D, L type This indicates the number of label categories.

[0098] In some embodiments, the encoding construction module 310 further includes a first training module, configured to: use the signal and knowledge embedding module and the encoder as a first-stage model; in the first stage of the two-stage training method, train the lead knowledge encoding and time-series knowledge encoding using the first-stage model, including: acquiring training multi-lead ECG signals, wherein the training multi-lead ECG signals are preprocessed multi-lead ECG signals collected during the training of the knowledge encoding; randomly initializing the lead knowledge encoding and time-series knowledge encoding to obtain initialized lead knowledge encoding and initialized time-series knowledge encoding; and inputting the initialized lead knowledge encoding and initialized time-series knowledge encoding together with the training multi-lead ECG signals into the first-stage model to obtain a first-stage signal feature block. The initialized lead knowledge encoding and the first-stage signal feature blocks are input into the knowledge learning module. The knowledge learning module outputs the predicted time point and predicted lead for each signal feature block in the preprocessed multi-lead ECG signal. Based on the trained multi-lead ECG signal, the predicted leads are linearly projected onto the corresponding lead signals to obtain the projected lead signals. The lead loss value between the projected lead signals and the original lead signals is calculated. Based on the position of each signal feature block in the preprocessed multi-lead ECG signal within the signal length of time period t, the predicted time points are compared to obtain the time point loss value. Based on the lead loss value and the time point loss value, the initialized lead knowledge encoding and the initialized time series knowledge encoding are adjusted respectively to obtain the trained lead knowledge encoding and the trained time series knowledge encoding.

[0099] In some embodiments, the encoding construction module 310 further includes a second training module, configured to: use the knowledge learning module as a second-stage model, and in the second stage of the two-stage training method, train the category knowledge encoding using the second-stage model, including: initializing the category knowledge encoding to obtain initialized category knowledge encoding; inputting the trained lead knowledge encoding, the trained time series knowledge encoding, and the trained multi-lead ECG signal into the first-stage model, and the first-stage model outputting a training signal feature sequence block; inputting the initialized category knowledge encoding and the training signal feature sequence block into the knowledge learning module, and the knowledge learning module outputting a predicted category knowledge encoding; inputting the predicted category knowledge encoding into a fully connected layer, and the fully connected layer outputting the classification result corresponding to the predicted category knowledge encoding; calculating the loss value between the classification result and the classification label based on the classification label; and adjusting the initialized category knowledge encoding based on the loss value to obtain the trained category knowledge encoding.

[0100] In some embodiments, the signal merging module 320 includes a signal merging submodule, which is configured to: merge the preprocessed multi-lead ECG signal, lead knowledge encoding, and time series knowledge encoding into a signal encoding block containing lead and time series knowledge, wherein the signal encoding block containing lead and time series knowledge is a three-dimensional matrix; and perform feature low-dimensional mapping on the signal encoding block containing lead and time series knowledge through a multilayer perceptron to obtain a signal data block, wherein the signal data block is the signal encoding block containing lead and time series knowledge after feature low-dimensional mapping, and the multilayer perceptron includes a fully connected layer, a GELU activation function, and a signal block random discarding layer.

[0101] In some embodiments, the knowledge learning module includes several knowledge learning layers connected in sequence and a last knowledge learning output layer. Each knowledge learning layer includes a knowledge attention layer and a knowledge self-attention layer. The last knowledge learning output layer is a knowledge attention layer. The knowledge attention layer uses a knowledge attention module, the knowledge self-attention layer uses a self-attention module, and the multi-head self-attention layer uses multiple self-attention modules.

[0102] In some embodiments, the knowledge-encoding-based ECG multi-label classification device further includes: both the self-attention module and the knowledge attention module employ the same attention architecture, which includes an attention module and a multilayer perceptron layer; the output of the attention architecture is z. l =LN(MLP(z′) l ))+z′ l LN represents normalization, MLP represents a multilayer perceptron, and z′ l This represents the residual output of the attention module, z′. l =LN(ATTEN(z) l-1 ))+z l-1 ATTEN represents an attention module, which can be a self-attention module or a knowledge-based attention module; z l-1 The input sequence represents the attention architecture.

[0103] In some embodiments, the feature learning module 330 includes a feature learning submodule, which is configured to: perform a linear projection transformation on the signal data block to obtain three output matrices, namely a query matrix Q, a key matrix K, and a value matrix V; split the query matrix Q, the key matrix K, and the value matrix V into several groups of matrices on a preset feature dimension of the model, with each group of matrices corresponding to a self-attention module, and the several groups of matrices corresponding to a multi-head self-attention layer; obtain the output matrices of the self-attention modules corresponding to the several groups of matrices according to the multi-head self-attention layer; merge the output matrices of the self-attention modules corresponding to the several groups of matrices to obtain a merged self-attention module matrix; and perform a linear projection on the merged self-attention module matrix to obtain a signal feature block, the dimension of which is consistent with the dimension of the output matrix.

[0104] In some embodiments, the feature learning submodule includes a multi-head self-attention layer processing module, which is configured to: process the i-th key matrix K i Transpose the matrix to obtain the i-th transpose key matrix K. i T ; Transpose the i-th group of transpose key matrix K i T With the i-th query matrix Q i Perform dot product processing to obtain the first dot product value Q. i K i T Based on scaling factor The dot product value Q i K i T Multiply by the scaling factor to obtain the multiplication matrix. Multiply matrices With the i-th value matrix V i Perform dot product processing to obtain the second dot product value. Calculate the second dot product value using the softmax function. The corresponding softmax value yields the output matrix of the self-attention module corresponding to the i-th group of matrices.

[0105] In some embodiments, the classification module 340 includes a classification submodule, which is configured to: process signal feature blocks using any one of the knowledge attention modules in the knowledge learning module, including: performing a linear projection transformation on the knowledge sequence block to obtain a linearly projected knowledge sequence block q; performing two linear projection transformations on the signal feature block to obtain a first signal feature output matrix k and a second signal feature output matrix v; performing attention calculations on each knowledge block in the linearly projected knowledge sequence block on the first signal feature output matrix k and the second signal feature output matrix v in sequence to obtain a plurality of knowledge attention matrices; merging the plurality of knowledge attention matrices to obtain a merged knowledge attention matrix; calculating the softmax value corresponding to the merged knowledge attention matrix based on the softmax function; and performing a dot product operation between the softmax value corresponding to the merged knowledge attention matrix and the second signal feature output matrix v to obtain the output of the knowledge attention module.

[0106] In some embodiments, the classification submodule includes an attention calculation module, which is configured to: calculate the j-th knowledge block q in the linearly projected knowledge sequence block. j Broadcast computation is performed by adding the first signal feature output matrix k to obtain the broadcast computation matrix; the tanh value corresponding to the broadcast computation matrix is ​​calculated using the tanh activation function; based on a learnable parameter matrix w... j , combine the tanh value and the parameter matrix w j Perform dot product processing to obtain the j-th knowledge block q. j The corresponding attention matrix; traverse the knowledge sequence blocks after linear projection transformation to obtain several knowledge attention matrices corresponding to each knowledge block.

[0107] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0108] As can be seen from the above, the ECG multi-label classification device based on knowledge encoding in this embodiment can first preprocess the acquired multi-lead ECG signals to obtain preprocessed multi-lead ECG signals; construct knowledge encoding, which includes lead knowledge encoding, time series knowledge encoding, and category knowledge encoding; based on the signal and knowledge embedding module, merge the preprocessed multi-lead ECG signals, lead knowledge encoding, and time series knowledge encoding into signal data blocks; based on the encoder composed of multi-head self-attention layers, perform feature learning on the signal data blocks to obtain signal feature blocks; based on the knowledge learning module, classify and identify the signal feature blocks according to the category knowledge encoding to obtain the label type corresponding to the multi-lead ECG signals.

[0109] This invention is designed for complete multi-label scenarios, utilizing a large number of labels to address the problem of insufficient labels in existing methods, while also being compatible with different lead formats. Furthermore, this invention combines the temporal signal attributes of the ECG signal itself, the spatial angular mapping attributes of the human heart between leads, and the classification characteristics of the task. On one hand, it proposes a knowledge-encoding-based computation framework, specifically composed of lead knowledge encoding, time-series knowledge encoding, and category knowledge encoding. On the other hand, it proposes that knowledge encoding captures corresponding attention from signal features through a knowledge attention layer. Since the focus differs between categories, this invention treats each category as a category knowledge encoding, allowing each category to learn its own unique attention and feature capture methods. Moreover, the lead encoding and time-series encoding of this invention consider the spatial location of the human heart and the temporal signal attributes of the ECG signal, respectively, enhancing the signal expression in these two aspects. Additionally, through category knowledge encoding, based on the proposed knowledge attention layer, this invention can obtain an attention map on the ECG signal corresponding to each category. Thus, this invention achieves high interpretability for each ECG label category, improving the accuracy of ECG multi-label classification.

[0110] This application also provides an electronic device, which can be a terminal, a server, or other similar device. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc. In some embodiments, the knowledge-encoding-based electrocardiogram (ECG) multi-label classification device can also be integrated into multiple electronic devices. For example, the knowledge-encoding-based ECG multi-label classification device can be integrated into multiple servers, with multiple servers implementing the knowledge-encoding-based ECG multi-label classification method of this application.

[0111] In this embodiment, a server will be used as an example for detailed description. For example, ... Figure 4 As shown, it illustrates a schematic diagram of the server structure involved in an embodiment of this application. Specifically:

[0112] The server may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input module 404, and a communication module 405. Those skilled in the art will understand that... Figure 4 The server architecture shown does not constitute a limitation on the server and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. Wherein:

[0113] The processor 401 is the control center of the server, connecting various parts of the server through various interfaces and lines. It performs various server functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, thereby providing overall monitoring of the server. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401. The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. Furthermore, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 402 may also include a memory controller to provide processor 401 with access to memory 402.

[0114] The server also includes a power supply 403 that supplies power to the various components. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The server may also include an input module 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. The server may also include a communication module 405. In some embodiments, the communication module 405 may include a wireless module, through which the server can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 405 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0115] Although not shown, the server may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the server loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402, thereby realizing various functions in the knowledge-encoding-based electrocardiogram multi-label classification device. In some embodiments, a computer program product is also proposed, including a computer program or instructions, which, when executed by the processor, implement the steps in any of the above-described knowledge-encoding-based electrocardiogram multi-label classification methods.

[0116] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0117] As can be seen from the above, the embodiments of this application are designed for complete multi-label scenarios, using a large number of labels, solving the problem of insufficient labels in existing methods, and are also compatible with different lead formats. Furthermore, this invention, by combining the temporal signal attributes of the ECG signal itself, the spatial angular mapping attributes of the human heart between leads, and the classification characteristics of the task, proposes a knowledge-encoding-based computation framework, specifically composed of lead knowledge encoding, time-series knowledge encoding, and category knowledge encoding. Another aspect is that the knowledge encoding captures corresponding attention from signal features through a knowledge attention layer. Since the focus of attention differs between different categories, this invention treats each category as a category knowledge encoding, thus allowing each category to learn its own unique attention and feature capture method. Moreover, the lead encoding and time-series encoding of this invention consider the spatial location of the human heart in the ECG and its temporal signal attributes, respectively, which can enhance the expression of the signal in these two aspects. Additionally, through category knowledge encoding, based on the proposed knowledge attention layer, this invention can obtain the attention map on the ECG signal corresponding to each category. In this way, the present invention achieves high interpretability for each ECG label category, thereby improving the accuracy of ECG multi-label classification.

[0118] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0119] To this end, embodiments of this application provide a computer-readable storage medium storing multiple instructions that can be loaded by a processor to execute steps in any of the knowledge-encoding-based electrocardiogram (ECG) multi-label classification methods provided in this application. For example, the instructions can execute the following steps: preprocessing the acquired multi-lead ECG signals to obtain preprocessed multi-lead ECG signals; constructing knowledge encoding, including lead knowledge encoding, time-series knowledge encoding, and category knowledge encoding; merging the preprocessed multi-lead ECG signals, lead knowledge encoding, and time-series knowledge encoding into a signal data block based on a signal and knowledge embedding module; performing feature learning on the signal data block based on an encoder composed of multi-head self-attention layers to obtain signal feature blocks; and classifying and identifying the signal feature blocks according to the category knowledge encoding based on a knowledge learning module to obtain the label type corresponding to the multi-lead ECG signals. The storage medium may include: read-only memory (ROM), random access memory (RAM), a disk, or an optical disk, etc.

[0120] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the knowledge-encoded electrocardiogram (ECG) multi-label classification method described in the foregoing embodiments. Since the instructions stored in the storage medium can execute the steps of any of the knowledge-encoded ECG multi-label classification methods provided in the embodiments of this application, the beneficial effects achievable by any of the knowledge-encoded ECG multi-label classification methods provided in the embodiments of this application can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0121] The foregoing has provided a detailed description of a knowledge-encoding-based electrocardiogram multi-label classification method, apparatus, server, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A multi-label classification method for electrocardiograms based on knowledge encoding, characterized in that, include: The acquired multi-lead electrocardiogram (ECG) signals were preprocessed to obtain preprocessed multi-lead ECG signals; A knowledge encoding scheme is constructed, comprising lead knowledge encoding, time-series knowledge encoding, and category knowledge encoding. Specifically, lead knowledge encoding is constructed based on the model's preset feature dimensions and the number of leads. The parameter matrix, This indicates the number of leads, and D represents the preset feature dimension of the model. A time-series knowledge encoding is constructed based on the model's preset feature dimensions, the signal length with a time period of t, and the length of each block in the sequence block. The time-series knowledge encoding is as follows: The parameter matrix, This represents the length of a signal with a time period of t. Indicates the length of each block in the sequence block; A category knowledge encoding is constructed based on the model's preset feature dimensions and the number of label categories. The category knowledge encoding is as follows: The parameter matrix, The number of label categories is indicated; the knowledge encoding is trained using a two-stage training method. Based on the signal and knowledge embedding module, the preprocessed multi-lead ECG signal, lead knowledge encoding, and time series knowledge encoding are merged into a signal data block; Based on an encoder composed of multi-head self-attention layers, feature learning is performed on the signal data block to obtain a signal feature block; Based on the knowledge learning module, the signal feature blocks are classified and identified according to the category knowledge encoding to obtain the label type corresponding to the multi-lead ECG signal; The training of the knowledge encoding using a two-stage training method includes: Using the signal and knowledge embedding module and the encoder as a first-stage model, in the first stage of the two-stage training method, the first-stage model is used to train the lead knowledge encoding and the time-series knowledge encoding, including: acquiring training multi-lead ECG signals, wherein the training multi-lead ECG signals are preprocessed multi-lead ECG signals collected during the training of the knowledge encoding; randomly initializing the lead knowledge encoding and the time-series knowledge encoding to obtain initialized lead knowledge encoding and initialized time-series knowledge encoding; inputting the initialized lead knowledge encoding and the initialized time-series knowledge encoding together with the training multi-lead ECG signals into the first-stage model to obtain a first-stage signal feature block; the initialized lead knowledge encoding and the first-stage time-series knowledge encoding are then used to train the lead knowledge encoding and the time-series knowledge encoding. A first-stage signal feature block is input to the knowledge learning module, which outputs the predicted time point and predicted lead for each signal feature block in the preprocessed multi-lead ECG signal. Based on the trained multi-lead ECG signal, the predicted lead is linearly projected onto the corresponding lead signal to obtain the projected lead signal. The lead loss value between the projected lead signal and the original lead signal is calculated. Based on the position of each signal feature block in the preprocessed multi-lead ECG signal within the signal length of time period t, the predicted time point is compared to obtain the time point loss value. Based on the lead loss value and the time point loss value, the initialized lead knowledge encoding and the initialized time series knowledge encoding are adjusted respectively to obtain the trained lead knowledge encoding and the trained time series knowledge encoding. The knowledge learning module is used as the second-stage model. In the second stage of the two-stage training method, the second-stage model is used to train the category knowledge encoding, including: initializing the category knowledge encoding to obtain initialized category knowledge encoding; inputting the trained lead knowledge encoding, the trained time series knowledge encoding, and the trained multi-lead ECG signal into the first-stage model, and the first-stage model outputs a training signal feature sequence block; inputting the initialized category knowledge encoding and the training signal feature sequence block into the knowledge learning module, and the knowledge learning module outputs a predicted category knowledge encoding; inputting the predicted category knowledge encoding into a fully connected layer, and the fully connected layer outputs the classification result corresponding to the predicted category knowledge encoding; calculating the loss value between the classification result and the classification label based on the classification label; and adjusting the initialized category knowledge encoding based on the loss value to obtain the trained category knowledge encoding.

2. The knowledge-encoding-based electrocardiogram multi-label classification method as described in claim 1, characterized in that, The preprocessing of the acquired multi-lead ECG signals to obtain preprocessed multi-lead ECG signals includes: Multi-lead electrocardiogram (ECG) signals are acquired based on a preset sampling frequency. The multi-lead ECG signals include a first signal matrix and a second signal matrix. The second signal matrix is ​​transformed to have the same dimensions as the first signal matrix to obtain the second signal matrix after dimension transformation. The second signal matrix after dimensional transformation is merged with the first signal matrix to obtain the merged multi-lead electrocardiogram signal; Based on the model's preset feature dimensions, the merged multi-lead ECG signal is linearly transformed into a sequence block, which is a three-dimensional matrix. The sequence block is dimensionally transformed to obtain a dimensionally transformed two-dimensional matrix, which is the preprocessed multi-lead electrocardiogram signal.

3. The knowledge-encoding-based electrocardiogram multi-label classification method as described in claim 1, characterized in that, The signal and knowledge embedding module merges the preprocessed multi-lead ECG signal, lead knowledge encoding, and time-series knowledge encoding into a signal data block, including: The preprocessed multi-lead ECG signal, lead knowledge encoding, and time series knowledge encoding are merged into a signal encoding block containing lead and time series knowledge, and the signal encoding block containing lead and time series knowledge is a three-dimensional matrix; The signal coding block containing lead and timing knowledge is mapped in a low dimension using a multilayer perceptron to obtain a signal data block. The signal data block is the signal coding block containing lead and timing knowledge after low-dimensional feature mapping. The multilayer perceptron includes a fully connected layer, a GELU activation function, and a signal block random discarding layer.

4. The knowledge-encoding-based electrocardiogram multi-label classification method as described in claim 1, characterized in that, The knowledge learning module includes several knowledge learning layers connected in sequence and a last knowledge learning output layer. Each knowledge learning layer includes a knowledge attention layer and a knowledge self-attention layer. The last knowledge learning output layer is a knowledge attention layer. The knowledge attention layer uses a knowledge attention module, the knowledge self-attention layer uses a self-attention module, and the multi-head self-attention layer uses multiple self-attention modules.

5. The knowledge-encoding-based electrocardiogram multi-label classification method as described in claim 4, characterized in that, The method further includes: both the self-attention module and the knowledge attention module adopt the same attention architecture, which includes an attention module and a multi-layer perceptron layer; the output of the attention architecture is... , Indicates normalization, This represents a multi-layer perceptron. This represents the residual output of the attention module. , This refers to an attention module, which can be either the self-attention module or the knowledge attention module. The sequence block input represents the attention architecture.

6. The knowledge-encoding-based electrocardiogram multi-label classification method as described in claim 5, characterized in that, The encoder based on a multi-head self-attention layer performs feature learning on the signal data block to obtain a signal feature block, including: The signal data block is subjected to linear projection transformation to obtain three output matrices, namely query matrix Q, key matrix K, and value matrix V. The query matrix Q, the key matrix K, and the value matrix V are each equally divided into several groups of matrices along the preset feature dimensions of the model. Each group of matrices corresponds to a self-attention module, and the several groups of matrices correspond to a multi-head self-attention layer. Based on the multi-head self-attention layer, several sets of matrices are obtained, each corresponding to the output matrix of the self-attention module. The output matrices of the self-attention modules corresponding to the several sets of matrices are merged to obtain the self-attention module merge matrix; The self-attention module merge matrix is ​​linearly projected to obtain a signal feature block, the dimension of which is the same as the dimension of the output matrix.

7. The ECG multi-label classification method based on knowledge encoding as described in claim 6, characterized in that, The step of obtaining several sets of self-attention module output matrices corresponding to the multi-head self-attention layer includes: The i-th group of key matrices Transpose the matrix to obtain the i-th transpose key matrix. ; The i-th transpose key matrix With the i-th query matrix Perform dot product processing to obtain the first dot product value. ; Based on scaling factor The dot product value Multiplying the product by the scaling factor yields a multiplication matrix. ; The multiplication matrix With the i-th set of value matrices Perform dot product processing to obtain the second dot product value. ; The second dot product value is calculated based on the softmax function. The corresponding softmax value yields the output matrix of the self-attention module corresponding to the i-th group of matrices. .

8. The knowledge-encoding-based electrocardiogram multi-label classification method as described in claim 7, characterized in that, The knowledge-based learning module classifies and identifies the signal feature blocks according to the category knowledge encoding to obtain the label type corresponding to the multi-lead ECG signal, including: Processing the signal feature block using any one of the knowledge attention modules in the knowledge learning module includes: Perform a linear projection transformation on the knowledge sequence block to obtain the linearly projected knowledge sequence block q. The signal feature blocks are subjected to two linear projection transformations to obtain the first signal feature output matrix k and the second signal feature output matrix v. Each knowledge block in the knowledge sequence block after linear projection transformation sequentially performs attention calculations on the first signal feature output matrix k and the second signal feature output matrix v to obtain several knowledge attention matrices; The aforementioned knowledge attention matrices are merged to obtain the merged knowledge attention matrix; The softmax value corresponding to the merged knowledge attention matrix is ​​calculated based on the softmax function. The softmax value corresponding to the merged knowledge attention matrix is ​​multiplied by the second signal feature output matrix v to obtain the output of the knowledge attention module.

9. The knowledge-encoding-based electrocardiogram multi-label classification method as described in claim 8, characterized in that, The process involves sequentially performing attention calculations on the first signal feature output matrix k and the second signal feature output matrix v for each knowledge block in the knowledge sequence block after linear projection transformation, resulting in several knowledge attention matrices, including: The j-th knowledge block in the knowledge sequence block after linear projection transformation A broadcast calculation is performed by adding the first signal feature output matrix k to obtain the broadcast calculated matrix; The tanh value corresponding to the matrix after the broadcast calculation is calculated using the tanh activation function; Based on a learnable parameter matrix The tanh value and the parameter matrix Perform dot product processing to obtain the j-th knowledge block. The corresponding attention matrix; The knowledge sequence blocks after the linear projection transformation are traversed to obtain several knowledge attention matrices corresponding to each knowledge block.

10. A multi-label electrocardiogram classification device based on knowledge encoding, characterized in that, include: The preprocessing module is used to preprocess the acquired multi-lead ECG signals to obtain preprocessed multi-lead ECG signals; The encoding construction module is used to construct knowledge encoding, which includes lead knowledge encoding, time series knowledge encoding, and category knowledge encoding. Specifically, lead knowledge encoding is constructed based on the model's preset feature dimensions and the number of leads. The parameter matrix, This indicates the number of leads, and D represents the preset feature dimension of the model. A time-series knowledge encoding is constructed based on the model's preset feature dimensions, the signal length with a time period of t, and the length of each block in the sequence block. The time-series knowledge encoding is as follows: The parameter matrix, This represents the length of a signal with a time period of t. Indicates the length of each block in the sequence block; A category knowledge encoding is constructed based on the model's preset feature dimensions and the number of label categories. The category knowledge encoding is as follows: The parameter matrix, The number of label categories is indicated; the knowledge encoding is trained using a two-stage training method. The signal merging module is used to merge the preprocessed multi-lead ECG signal, lead knowledge encoding, and time series knowledge encoding into a signal data block based on the signal and knowledge embedding module. The feature learning module is used to perform feature learning on the signal data block based on the encoder composed of multi-head self-attention layers to obtain the signal feature block; The classification module is used to classify and identify the signal feature blocks based on the category knowledge encoding, according to the knowledge learning module, to obtain the label type corresponding to the multi-lead ECG signal. The training of the knowledge encoding using a two-stage training method includes: Using the signal and knowledge embedding module and the encoder as a first-stage model, in the first stage of the two-stage training method, the first-stage model is used to train the lead knowledge encoding and the time-series knowledge encoding, including: acquiring training multi-lead ECG signals, wherein the training multi-lead ECG signals are preprocessed multi-lead ECG signals collected during the training of the knowledge encoding; randomly initializing the lead knowledge encoding and the time-series knowledge encoding to obtain initialized lead knowledge encoding and initialized time-series knowledge encoding; inputting the initialized lead knowledge encoding and the initialized time-series knowledge encoding together with the training multi-lead ECG signals into the first-stage model to obtain a first-stage signal feature block; the initialized lead knowledge encoding and the first-stage time-series knowledge encoding are then used to train the lead knowledge encoding and the time-series knowledge encoding. A first-stage signal feature block is input to the knowledge learning module, which outputs the predicted time point and predicted lead for each signal feature block in the preprocessed multi-lead ECG signal. Based on the trained multi-lead ECG signal, the predicted lead is linearly projected onto the corresponding lead signal to obtain the projected lead signal. The lead loss value between the projected lead signal and the original lead signal is calculated. Based on the position of each signal feature block in the preprocessed multi-lead ECG signal within the signal length of time period t, the predicted time point is compared to obtain the time point loss value. Based on the lead loss value and the time point loss value, the initialized lead knowledge encoding and the initialized time series knowledge encoding are adjusted respectively to obtain the trained lead knowledge encoding and the trained time series knowledge encoding. The knowledge learning module is used as the second-stage model. In the second stage of the two-stage training method, the second-stage model is used to train the category knowledge encoding, including: initializing the category knowledge encoding to obtain initialized category knowledge encoding; inputting the trained lead knowledge encoding, the trained time series knowledge encoding, and the trained multi-lead ECG signal into the first-stage model, and the first-stage model outputs a training signal feature sequence block; inputting the initialized category knowledge encoding and the training signal feature sequence block into the knowledge learning module, and the knowledge learning module outputs a predicted category knowledge encoding; inputting the predicted category knowledge encoding into a fully connected layer, and the fully connected layer outputs the classification result corresponding to the predicted category knowledge encoding; calculating the loss value between the classification result and the classification label based on the classification label; and adjusting the initialized category knowledge encoding based on the loss value to obtain the trained category knowledge encoding.

11. An electronic device, characterized in that, It includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to execute the steps in the knowledge-encoding-based electrocardiogram multi-label classification method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores multiple instructions adapted for loading by a processor to perform the steps in the knowledge-encoding-based electrocardiogram multi-label classification method according to any one of claims 1-9.

Images