ECG signal classification method based on TS-BERT neural network model architecture
By using the TS-BERT neural network model architecture, combined with data augmentation and self-supervised signal processing, efficient and accurate classification of ECG signals is achieved. This solves the problems of large data requirements and computational complexity in existing ECG detection algorithms, and improves the efficiency and accuracy of ECG diagnosis.
Patent Information
- Application Number
- CN202310148218.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-15
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-02-15
AI Technical Summary
Existing ECG detection algorithms are computationally complex and require large amounts of data, making it difficult to classify ECGs efficiently and accurately. Traditional methods rely on feature engineering, while deep learning algorithms require a large amount of data annotation, resulting in low efficiency.
The TS-BERT neural network model architecture is adopted, and wavelet transform and median filtering are combined for data augmentation and denoising. Temporal convolutional layers, multi-head self-attention mechanism and point-level feedforward network are used for pre-training. Self-supervised signals are generated through vector quantization module and fine-tuned by average pooling and linear classifier to achieve end-to-end ECG signal classification.
It reduces the need for ECG training data, improves the efficiency and accuracy of ECG classification, reduces the diagnostic burden on doctors, and increases the sensitivity of arrhythmia detection.
Smart Images

Figure CN116172514B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer applications and ECG signal classification technology, specifically to an ECG signal classification method based on the TS-BERT neural network model architecture. Background Technology
[0002] An electrocardiogram (ECG) records electrical signals through the heart and is commonly used clinically to further diagnose arrhythmias and related diseases. Traditional ECG testing methods involve using a hospital electrocardiograph to collect short-term ECG signals from the patient, which are then manually diagnosed by a cardiologist through visual examination.
[0003] However, with the number of cardiovascular patients in my country continuing to rise, now exceeding 300 million, this means that each electrocardiologist needs to diagnose a large number of electrocardiograms every day. These doctors are working under immense pressure, objectively increasing the treatment costs and life-threatening risks for patients. Furthermore, because arrhythmias occur intermittently, especially in the early stages of a problem, it is difficult to detect them from short-term ECG signals, thus missing the optimal treatment window for heart disease patients.
[0004] To alleviate the aforementioned problems, numerous ECG detection devices with integrated detection capabilities have been invented in recent years. These devices can provide users with real-time heart health advice while recording abnormal ECG signals, thereby reducing the burden on doctors and detecting potential heart disease patients. Among these, the most mainstream ECG detection algorithms fall into two main categories. One is feature-based classification methods, which can be divided into two independent parts: a feature extractor and a classifier. However, this structure leads to a problem where the overall performance of the method depends heavily on the quality of the features extracted by the feature extractor, resulting in an over-reliance on feature engineering and limitations imposed by human knowledge and experience. The other type, deep learning-based algorithms, mostly require large amounts of real ECG data for training, meaning more manpower is needed for data collection and manual annotation. Despite this, these deep learning algorithms have not achieved satisfactory results in actual ECG detection.
[0005] In summary, the main challenges of current ECG detection algorithms are: 1) Traditional algorithms are difficult to design, have mediocre classification results, and may even lead to excessive computational complexity; 2) Although deep learning algorithms have improved performance, they are still not satisfactory and require a large amount of data collection and manual annotation, which is time-consuming and labor-intensive. Summary of the Invention
[0006] The purpose of this invention is to provide an ECG signal classification method based on the TS-BERT neural network model architecture. This method can solve the problems of computational complexity and excessive data requirements in existing ECG detection and classification algorithms, and achieve efficient and accurate detection and classification of patient electrocardiograms.
[0007] To achieve the above objectives, this invention provides an ECG signal classification method based on the TS-BERT neural network model architecture, the method comprising:
[0008] Step 1: Obtain the ECG dataset and perform data augmentation and denoising using wavelet transform and median filtering methods; then, divide it into training, validation, and test sets according to the proportions.
[0009] Step 2: Pre-train the TS-BERT neural network model architecture on the dataset obtained in Step 1;
[0010] Step 3: Fine-tune the data pre-trained in Step 2 using the downstream ECG signal classification task to obtain the final model.
[0011] Preferably, in step two, TS-BERT divides the time-series ECG signal into multiple small segments, integrates each small segment into a temporal convolutional layer (TCN) as input, and passes it through a 12-layer encoder based on a multi-head self-attention mechanism and a point-level feedforward network.
[0012] The calculation formula for the multi-head self-attention mechanism is as follows:
[0013] MultoHeadAttn(F l = [head1, head2, ..., head h W o ;
[0014] head i =Attention(F l W i Q ,F l W i K ,FlW i V );
[0015]
[0016] Among them, F l It is the input of the l-th layer network; when l=0, the framework takes the timing ECG signal as the input, W Q W K W V W O ∈Rd*d Here, is the projection matrix parameter, and d is the vector dimension of the ECG signal. It is a proportional factor to prevent the product of attention mechanisms from becoming too large;
[0017] The calculation formula for the point-level feedforward network is as follows:
[0018]
[0019] FFN(x)=(ReLU(xW1+b1))W2+b2;
[0020] Where W1, b1, W2, b2 are learnable parameters;
[0021] Simultaneously, a vector quantization module is used to generate self-supervised signals for the ECG data, including:
[0022] Initialize a feature space e containing K vectors, then encode the ECG signal F using a temporal convolutional network, and construct a discrete representation z for each small ECG segment, as shown in the following formula:
[0023] z i =q(F) i =argmin j ||Encoder(F) i -e j ||2;
[0024] Furthermore, two mean squared error loss functions are used to optimize the vector space and the quantization encoder respectively, so that there is a discriminable boundary between the discrete representations:
[0025]
[0026]
[0027] Where sg represents gradient stopping, the quantized code z can be used as the self-supervised signal of BERT.
[0028] Preferably, step three includes:
[0029] Average pooling is applied to the output of the last layer of BERT to obtain the overall representation of the time-series ECG signal. Then, this vector is input into a linear classifier to calculate the probability of each class, and supervised fine-tuning is performed using manually labeled classes. The formula is as follows:
[0030]
[0031] P = softmax(OW) O +b O );
[0032] Among them, WO Let b be a linear mapping matrix. O This is to offset the category shift; ultimately, the fine-tuned model can accurately diagnose and analyze patient electrocardiograms.
[0033] According to the above technical solution, the present invention utilizes an ECG signal classification method based on the TS-BERT neural network model architecture. This method can reduce the need for ECG training data and accurately classify the patient's electrocardiogram.
[0034] Other features and advantages of the present invention will be described in detail in the following detailed description section. Attached Figure Description
[0035] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof. In the drawings:
[0036] Figure 1 This is a schematic diagram of the ECG signal classification method based on the TS-BERT neural network model architecture provided by the present invention. Detailed Implementation
[0037] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.
[0038] This invention provides an ECG signal classification method based on the TS-BERT neural network model architecture, the method comprising:
[0039] Step 1: Obtain the ECG dataset and perform data augmentation and denoising using wavelet transform and median filtering methods; then, divide it into training, validation, and test sets according to the proportions.
[0040] Step 2: Pre-train the TS-BERT neural network model architecture proposed in this invention on the obtained dataset.
[0041] Specifically, TS-BERT uses a training framework similar to BERT, which is a text encoder commonly used in natural language processing. In this invention, as... Figure 1 As shown, BERT divides the temporal ECG signal into many small segments, passes each segment through a temporal convolutional layer (TCN), and then integrates them as input. This is then processed by a 12-layer encoder based on a multi-head self-attention mechanism and a point-level feedforward network. The calculation formula for the multi-head self-attention mechanism is as follows:
[0042] MultiHeadAttn(F l= [head1, head2, ..., head h W O ;
[0043]
[0044]
[0045] Among them, F l It is the input of the W-th layer network; when l=0, the framework takes the timing ECG signal as the input, W Q W K W V W O ∈R d*d Here, is the projection matrix parameter, and d is the vector dimension of the ECG signal. It is a proportional factor to prevent the product of attention mechanisms from becoming too large;
[0046] The calculation formula for the point-level feedforward network is as follows:
[0047]
[0048] FFN(x)=(ReLU(xW1+b1))W2+b2;
[0049] Where W1, b1, W2, b2 are learnable parameters.
[0050] In natural language processing (NLP) techniques, a portion of the input sequence of BERT is typically masked, and the predicted real words corresponding to the masked positions are then used as an upstream task for pre-training to learn the potential connections between words. However, for time-series ECG signals, since there is no discrete word representation like in text data, this invention further employs a vector quantization module to generate self-supervised signals for the ECG data. Specifically, a feature space e containing K vectors is first initialized, then a temporal convolutional network is used to encode the ECG signal F, and a discrete representation z is constructed for each small ECG segment, as shown in the following formula:
[0051] z i =q(F) i =argmin j ||Encoder(F) i -e j ||2;
[0052] Furthermore, two mean squared error loss functions are used to optimize the vector space and the quantization encoder respectively, so that there is a discriminable boundary between the discrete representations:
[0053]
[0054]
[0055] Here, sg represents gradient stopping. Therefore, the quantized encoding z can serve as the self-supervised signal for BERT. The TS-BERT proposed in this invention employs an end-to-end pre-training approach, combining BERT and a vector quantization module to pre-train the model.
[0056] Step 3: After completing the TS-BERT pre-training, this invention will use a downstream ECG signal classification task for fine-tuning. Specifically, average pooling is applied to the output of the last layer of BERT to obtain the overall representation of the time-series ECG signal. Then, this vector is input into a linear classifier to calculate the probability of each category, and supervised fine-tuning training is performed using manually labeled categories.
[0057]
[0058] P = softmax(OW) O +b O )
[0059] Among them, W O Let b be a linear mapping matrix. O This is for class shift. Ultimately, the fine-tuned model can accurately diagnose and analyze patient electrocardiograms.
[0060] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solution of the present invention, and these simple modifications all fall within the protection scope of the present invention.
[0061] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not describe the various possible combinations separately.
[0062] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed by the present invention.
Claims
1. A method for ECG signal classification based on TS-BERT neural network model architecture, characterized in that, The method includes: Step 1: Obtain the ECG dataset and perform data augmentation and denoising using wavelet transform and median filtering methods; then, divide it into training, validation, and test sets according to the proportions. Step 2: Pre-train the dataset obtained in Step 1 using the TS-BERT neural network model architecture. TS-BERT segments the temporal ECG signal into multiple small segments, integrates each segment into a temporal convolutional layer (TCN) as input, and then uses a 12-layer encoder based on a multi-head self-attention mechanism and a point-level feedforward network. Simultaneously, a vector quantization module is used to generate a self-supervised signal for the ECG data, including: initializing a feature space e containing K vectors, then encoding the ECG signal F using a temporal convolutional network, constructing a discrete representation z for each ECG segment, and using two mean squared error loss functions to optimize the vector space and the quantization encoder respectively to ensure a discernible boundary between the discrete representations. Step 3: Fine-tune the data pre-trained in Step 2 using the downstream ECG signal classification task to obtain the final model; in this step, average pooling is applied to the output of the last layer of TS-BERT to obtain the overall representation of the time-series ECG signal; then, this overall representation is input into a linear classifier to calculate the probability of each category, and supervised fine-tuning training is performed using manually labeled categories.
2. The ECG signal classification method based on TS-BERT neural network model architecture according to claim 1, wherein, In step two, the multi-head self-attention mechanism is calculated using the following formula: ; ; ; wherein, is the input of the first layer network; when = 0, the framework takes the ECG signal as input, is the projection matrix parameter, is the vector dimension of the ECG signal, is the scaling factor to prevent the attention mechanism product from being too large; The calculation formula for the point-level feedforward network is as follows: ; ; wherein, are learnable parameters; The formula for constructing the discrete representation z is as follows: ; The two-part mean squared error loss function is: ; ; wherein, represents a gradient stop, then the discrete representation i.e., as a self-supervised signal for BERT.
3. The method of claim 1, wherein the TS-BERT neural network model architecture is based on a Transformer architecture. Step three includes: The formulas for the average pooling operation and the linear classifier are as follows: ; ; wherein, is a linear mapping matrix, is a class offset; finally, the model after fine-tuning can accurately diagnose the patient's electrocardiogram.
