A heartbeat classification method and device based on electrocardiogram diagnosis report

Abstract

The present invention provides a heartbeat classification method and device based on an electrocardiogram diagnosis report, the method comprising: obtaining electrocardiogram data; inputting the electrocardiogram data into a pre-trained heartbeat classification model to obtain a heartbeat classification result; wherein the heartbeat classification model is a six-layer convolutional neural network; and the heartbeat classification model is trained by a rhythm network and electrocardiogram-level labeled data. Currently, the prior art can only solve two problems: how to train an electrocardiogram classifier with electrocardiogram-level labeled data and how to train a heartbeat classifier with heartbeat-level labeled data. The present application realizes the function of training a heartbeat classifier with electrocardiogram-level labeled data by combining a rhythm network model and a heartbeat network model.

Description

Technical Field

[0001] The present application belongs to the field of medical equipment technology, and specifically, relates to a heartbeat classification method and device based on an electrocardiogram diagnosis report. Background Art

[0002] Arrhythmia corresponds to abnormal heart rate or abnormal waveform in the electrocardiogram, and the latter is more difficult to diagnose. Heartbeat is the basic component of the electrocardiogram, so diagnosis at the heartbeat level is more interpretable than diagnosis at the electrocardiogram level, and is therefore more clinically needed.

[0003] Training a heartbeat-level classifier usually requires annotated data at the heartbeat level, which is rare in reality and can only be specially annotated for this purpose. At the same time, the cost of this annotation is huge. Therefore, the bottleneck limiting the heartbeat classifier lies in the annotation of heartbeat-level data. On the other hand, although heartbeat-level annotation is very rare in reality, there is no shortage of ECG-level annotation in reality: almost every patient's routine ECG examination will have a diagnosis report, and the annotation can be extracted from the diagnosis report. Figure 1 It is the relationship between the rhythm-level label and the heartbeat-level label. Summary of the invention

[0004] The present application provides a heartbeat classification method and device based on an electrocardiogram diagnostic report, so as to at least solve the problem that the heartbeat classification cannot be completed only by data with electrocardiogram level annotations due to the lack of heartbeat level annotations on the market.

[0005] According to a first aspect of the present application, a heartbeat classification method based on an electrocardiogram diagnosis report is provided, comprising:

[0006] Obtain ECG data;

[0007] The electrocardiogram data of a single heartbeat is input into a pre-trained heartbeat classification model to obtain a heartbeat classification result; wherein the heartbeat classification model is a six-layer convolutional neural network; the heartbeat classification model is trained through a rhythm network and electrocardiogram-level labeled data.

[0008] In one embodiment, a method for training a heartbeat classification model includes:

[0009] Input the collected conventional ECG training samples into the pre-trained rhythm network model to obtain the ECG thermogram;

[0010] Label each heartbeat in the electrocardiogram with a pseudo label and label confidence;

[0011] Generate a pseudo heartbeat level training set based on the annotated heartbeats;

[0012] The heartbeat classification model is trained using a pseudo heartbeat level training set with confidence.

[0013] In one embodiment, the training method of the rhythm network model includes:

[0014] Input regular ECG training samples with global category annotations into the rhythm network model;

[0015] The rhythm network model infers a global class label for the input conventional electrocardiogram.

[0016] In one embodiment, the rhythm network adds a U-shaped structure and an attention pooling layer based on the public ResNet34.

[0017] According to another aspect of the present application, a heartbeat classification device based on an electrocardiogram diagnosis report is also provided, comprising:

[0018] An acquisition unit, used for acquiring electrocardiogram data;

[0019] The heartbeat classification unit is used to input the electrocardiogram data of a single heartbeat into a pre-trained heartbeat classification model to obtain a heartbeat classification result; wherein the heartbeat classification model is a six-layer convolutional neural network; the heartbeat classification model is trained through a rhythm network and electrocardiogram level labeled data.

[0020] In one embodiment, the heartbeat classification device based on the electrocardiogram diagnosis report further includes a heartbeat classification model training device, specifically including:

[0021] An electrocardiogram thermodynamic map acquisition unit is used to input the collected conventional electrocardiogram training samples into a pre-trained rhythm network model to obtain an electrocardiogram thermodynamic map;

[0022] A labeling unit, used to label each heart beat in the electrocardiogram with a pseudo label and label confidence;

[0023] A heartbeat training set generating unit, used for generating a pseudo heartbeat level training set according to the annotated heartbeats;

[0024] The first training unit is used to train a heartbeat classification model using a pseudo heartbeat level training set with confidence.

[0025] In one embodiment, the heartbeat classification device based on the electrocardiogram diagnosis report also includes a rhythm network model training device, specifically including:

[0026] A training sample input unit, used for inputting conventional electrocardiogram training samples into the rhythm network model;

[0027] The global category labeling unit is used for the rhythm network model to label the electrocardiogram with global categories;

[0028] In one embodiment, the rhythm network adds a U-shaped structure and an attention pooling layer based on the public ResNet34.

[0029] Currently, the existing technology can only solve two problems: how to train an ECG classifier with ECG-level labeled data and how to train a heartbeat classifier with heartbeat-level labeled data. This application combines the rhythm network model with the heartbeat network model to achieve the function of training a heartbeat classifier with ECG-level labeled data. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 It is the relationship between the rhythm-level label and the heartbeat-level label.

[0032] Figure 2 A flow chart of a heartbeat classification method based on an electrocardiogram diagnostic report provided in this application.

[0033] Figure 3 Flow chart of the training method of the heartbeat classification model in the embodiment of the present application.

[0034] Figure 4 This is a flow chart of the training method of the rhythmic network model in the embodiment of the present application.

[0035] Figure 5 This is the specific structure of the rhythmic network in the embodiment of this application.

[0036] Figure 6 This is the specific structure of the heartbeat classification network in the embodiment of the present application.

[0037] Figure 7 This is a structural block diagram of a heart beat classification device based on an electrocardiogram diagnostic report provided in this application.

[0038] Figure 8 It is a structural block diagram of the training device of the heartbeat classification model in the embodiment of the present application.

[0039] Fig. 9This is a structural block diagram of a training device for a rhythmic network model in an embodiment of the present application.

[0040] Fig.10 This is a specific implementation of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Arrhythmia corresponds to abnormal heart rate or abnormal waveform in the electrocardiogram, and the latter is more difficult to diagnose. Heartbeat is the basic component of the electrocardiogram, so diagnosis at the heartbeat level is more interpretable than diagnosis at the electrocardiogram level, and is therefore more clinically needed.

[0043] Training a heartbeat-level classifier usually requires annotated data at the heartbeat level, which is rare in reality and can usually only be specially annotated for this purpose. At the same time, the cost of this annotation is huge. Therefore, the bottleneck that limits the heartbeat classifier lies in the annotation of the heartbeat-level data. On the other hand, although the annotation of the heartbeat level is very rare in reality, there is no shortage of electrocardiogram-level annotations in reality: almost every patient's routine electrocardiogram examination will have a diagnosis report, and the annotation can be extracted from the diagnosis report. Most of the current existing technologies study the above two problems in isolation, namely: how to train an electrocardiogram classifier with electrocardiogram-level annotated data and how to train a heartbeat classifier with heartbeat-level annotated data. Therefore, the problem to be solved in this application is how to complete the training of a heartbeat-level classifier using only data with electrocardiogram-level annotations.

[0044] According to one aspect of the present application, a heartbeat classification method based on an electrocardiogram diagnosis report is provided, such as Figure 2 As shown, including:

[0045] S201: Obtain electrocardiogram data.

[0046] S202: Input the electrocardiogram data of a single heartbeat into a pre-trained heartbeat classification model to obtain a heartbeat classification result. The heartbeat classification model is a six-layer convolutional neural network; the heartbeat classification model is trained by a rhythm network and electrocardiogram level labeled data.

[0047] In one embodiment, if Figure 3 As shown, the training method of the heartbeat classification model includes:

[0048] S301: Collect conventional electrocardiogram training samples and input them into a pre-trained rhythm network model to obtain an electrocardiogram thermogram.

[0049] S302: Annotate each heartbeat in the electrocardiogram with a pseudo label and label confidence.

[0050] S303: Generate a pseudo heartbeat level training set based on the labeled heartbeats.

[0051] S304: Train a heartbeat classification model using the pseudo heartbeat level training set with confidence.

[0052] In one embodiment, if Figure 4 As shown, the training method of the rhythm network model includes:

[0053] S401: Inputting conventional electrocardiogram training samples with global category annotations into the rhythm network model.

[0054] S402: The rhythm network model infers a global category label of the input conventional electrocardiogram.

[0055] In one embodiment, the rhythmic network adds a U-shaped structure and an attention pooling layer.

[0056] In a specific embodiment, a segment of electrocardiogram is diagnosed as abnormal if and only if it contains at least one abnormal heartbeat. Conversely, a segment of electrocardiogram is diagnosed as normal if and only if all the heartbeats therein are normal. This is consistent with the multiple instance learning modeling in machine learning, and therefore, the method design is based on multiple instance learning. This application will train two neural network models: a rhythm-level net and a heartbeat-level net, the latter being the final target model, and the former providing support for the training of the latter.

[0057] In a specific embodiment, the structure of the rhythm network is based on the Attention U-net, which is an extended version of the convolutional neural network. It accepts a segment of electrocardiogram as input. During training, it requires the corresponding global category annotation of the electrocardiogram; during inference, it can not only infer the global category of the electrocardiogram, but also generate a heat map for locating the corresponding area of ​​the abnormal category. In the convolutional neural network, each position of the intermediate feature layer corresponds to a segment of continuous data (i.e., receptive field) of the original input. The present application models such continuous data as a single instance, so that the overall model can be modeled using multi-instance learning. Compared with ordinary convolutional neural networks, it includes two improvements: (1) the introduction of the U-shaped structure, which is a structure commonly used in semantic segmentation networks, and its purpose is to generate fine-grained heat maps. (2) The introduction of the attention pooling layer, which is a novel integrated function of multi-instance learning, can be used to explicitly model the weight of each position on the final classification, which will also greatly affect the generated heat map. Using the heat map generated by the rhythm network, a pseudo label and its label confidence can be added to each heartbeat. The pseudo label is defined as follows: for any abnormal electrocardiogram, all heartbeats are labeled with the corresponding abnormal pseudo label; the confidence is defined as the normalized value (in the range of 0-1) of the area of ​​the heartbeat in the heat map. In this way, a pseudo heartbeat level training set can be obtained for subsequent training.

[0058] Figure 5 For the specific structure of the rhythm network, each convolutional layer is followed by a batch normalization layer.

[0059] In a specific embodiment, the structure of the heartbeat network is based on a 6-layer convolutional neural network. It accepts a heartbeat as input. During training, it uses the pseudo heartbeat level training set obtained above for training; during inference, it infers the category to which the new heartbeat belongs like a normal classifier. The technical key here is how to use a data set with confidence to train the classifier. This application proposes a weighted cross entropy loss function. Suppose the confidence of a sample is s, and the original cross entropy loss value is CE, then the weighted loss function is defined as s α CE, where α is a non-negative hyperparameter used to control the scaling of confidence.

[0060] Figure 6 For the specific structure of the heartbeat network, each convolutional layer is followed by a layer normalization layer.

[0061] In a specific embodiment, the hyperparameter α used in training the heartbeat network is set to 1.0 to obtain the best performance, but we have proved through experiments that the actual effect is not sensitive to the selection of this hyperparameter, and the value of α in [0.5, 1000] does not lead to significant changes in performance. When training the rhythm network, the Adam optimizer is used, in which the built-in parameter is the default value β1 =0.9,β 2 =0.999), the learning rate is 0.0003, and the batch size is 64. When training the heartbeat network, the Adam optimizer is used, and the built-in parameters are the default values ​​(β 1 =0.9,β 2 =0.999), the learning rate is 0.0003, and the batch size is 128.

[0062] Normally, there are four types of arrhythmias: supraventricular arrhythmia, ventricular arrhythmia, right bundle branch block, and left bundle branch block. These four types of arrhythmias and the normal class together constitute five categories. This application uses about 60,000 24-second electrocardiograms (with electrocardiogram-level annotations only) for model training, and uses about 20,000 heartbeats (with heartbeat-level annotations) for model testing. The final average F1 score of the test on the heartbeat network is 0.807. In comparison, the same model was used for five-fold cross-validation training on the above-mentioned heartbeat test set, and the final average F1 score was 0.739. Taking into account the high difficulty of obtaining 20,000 heartbeat annotations in reality, this result shows that the method provided in this application can make full use of the data existing in reality and train a model with excellent results.

[0063] Based on the same inventive concept, the embodiments of the present application also provide a kind of heartbeat classification device based on electrocardiogram diagnostic report, which can be used to implement the method described in the above embodiments, as described in the following embodiments. Since the principle of solving the problem by the heartbeat classification device based on electrocardiogram diagnostic report is similar to the heartbeat classification method based on electrocardiogram diagnostic report, the implementation of the heartbeat classification device based on electrocardiogram diagnostic report can refer to the implementation of the heartbeat classification method based on electrocardiogram diagnostic report, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can implement a combination of software and / or hardware for a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0064] According to another aspect of the present application, a heartbeat classification device based on an electrocardiogram diagnosis report is also provided. Figure 7 As shown, including:

[0065] An acquisition unit 701 is used to acquire electrocardiogram data;

[0066] The heartbeat classification unit 702 is used to input the electrocardiogram data of a single heartbeat into a pre-trained heartbeat classification model to obtain a heartbeat classification result; wherein the heartbeat classification model is a six-layer convolutional neural network; the heartbeat classification model is trained through a rhythm network and electrocardiogram level labeled data.

[0067] In one embodiment, the heartbeat classification device based on the electrocardiogram diagnosis report also includes a training device for a heartbeat classification model, such as Figure 8 As shown, specifically including:

[0068] The electrocardiogram acquisition unit 801 is used to input the collected conventional electrocardiogram training samples into the pre-trained rhythm network model to obtain the electrocardiogram;

[0069] A labeling unit 802 is used to label each heartbeat in the electrocardiogram with a pseudo label and a label confidence;

[0070] A heartbeat training set generating unit 803, configured to generate a pseudo heartbeat level training set according to the annotated heartbeats;

[0071] The first training unit 804 is used to train the heartbeat classification model using the pseudo heartbeat level training set with confidence.

[0072] In one embodiment, the heartbeat classification device based on the electrocardiogram diagnosis report also includes a training device for the rhythm network model, such as Fig. 9 As shown, specifically including:

[0073] The training sample input unit 901 is used to input the conventional electrocardiogram training sample into the rhythm network model;

[0074] A global category labeling unit 902 is used for the rhythm network model to label the electrocardiogram with global categories;

[0075] In one embodiment, the rhythm network adds a U-shaped structure and an attention pooling layer based on the public ResNet34.

[0076] Currently, the existing technology can only solve two problems: how to train an ECG classifier with ECG-level labeled data and how to train a heartbeat classifier with heartbeat-level labeled data. This application combines the rhythm network model with the heartbeat network model to achieve the function of training a heartbeat classifier with ECG-level labeled data.

[0077] The embodiments of the present application also provide a specific implementation of an electronic device that can implement all the steps in the method in the above embodiments, see Fig.10 , the electronic device specifically includes the following contents:

[0078] Processor 1001, memory 1002, communication interface 1003, bus 1004 and non-volatile memory 1005;

[0079] The processor 1001, memory 1002, communication interface 1003 and non-volatile memory 1005 communicate with each other via the bus 1004;

[0080] The processor 1001 is used to call the computer program in the memory 1002 and the non-volatile memory 1005. When the processor executes the computer program, all the steps in the method in the above embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0081] S201: Obtain electrocardiogram data.

[0082] S202: Input the electrocardiogram data into a pre-trained heartbeat classification model to obtain a heartbeat classification result; wherein the heartbeat classification model is a six-layer convolutional neural network; the heartbeat classification model is trained by a rhythm network and electrocardiogram level labeled data.

[0083] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the method in the above embodiments. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, all the steps of the method in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0084] S201: Obtain electrocardiogram data.

[0085] S202: Input the electrocardiogram data into a pre-trained heartbeat classification model to obtain a heartbeat classification result; wherein the heartbeat classification model is a six-layer convolutional neural network; the heartbeat classification model is trained by a rhythm network and electrocardiogram level labeled data.

[0086] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Although the embodiment of this specification provides the method operation steps described in the embodiment or flow chart, more or less operation steps can be included based on conventional or non-creative means. The order of steps listed in the embodiment is only one way of executing the order of many steps, and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, product or device. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or device including the elements. For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the embodiments of this specification, the functions of each module can be implemented in the same one or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The present invention is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process Figure 1 A process or multiple processes and / or boxes Figure 1A device for specifying functions in a box or multiple boxes. It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment. In the description of this specification, the description of the reference term "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction. The above description is only an embodiment of the embodiment of this specification and is not intended to limit the embodiment of this specification. For those skilled in the art, the embodiment of this specification may have various changes and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the embodiment of this specification shall be included in the scope of the claims of the embodiment of this specification.

Claims

1. A heartbeat classification method based on an electrocardiogram diagnosis report, characterized in that: include: Obtain ECG data; Inputting the electrocardiogram data of a single heartbeat into a pre-trained heartbeat classification model to obtain a heartbeat classification result; wherein the heartbeat classification model is a six-layer convolutional neural network; the heartbeat classification model is trained by a rhythm network and electrocardiogram level labeled data; The training method of the heartbeat classification model includes: Input the collected conventional ECG training samples into the pre-trained rhythm network model to obtain the ECG thermogram; Labeling each heartbeat in the electrocardiogram with a pseudo label and label confidence; Generate a pseudo heartbeat level training set based on the annotated heartbeats; The heartbeat classification model is trained using the pseudo heartbeat level training set with confidence.

2. The heartbeat classification method based on the electrocardiogram diagnosis report according to claim 1, characterized in that: The training methods of the rhythm network model include: Inputting the conventional electrocardiogram training samples with global category annotations into the rhythm network model; The rhythm network model infers a global class label for the input conventional electrocardiogram.

3. The heartbeat classification method based on the electrocardiogram diagnosis report according to claim 1, characterized in that: The rhythm network adds a U-shaped structure and an attention pooling layer based on the public ResNet34.

4. A heartbeat classification device based on an electrocardiogram diagnosis report, characterized in that: include: An acquisition unit, used for acquiring electrocardiogram data; A heartbeat classification unit, used for inputting the electrocardiogram data of a single heartbeat into a pre-trained heartbeat classification model to obtain a heartbeat classification result; wherein the heartbeat classification model is a six-layer convolutional neural network; the heartbeat classification model is trained by a rhythm network and electrocardiogram level annotation data; Also included is a training device for a heartbeat classification model, specifically comprising: An electrocardiogram acquisition unit is used to input the collected conventional electrocardiogram training samples into a pre-trained rhythm network model to obtain an electrocardiogram; a labeling unit, used for labeling a pseudo label and a label confidence level for each heartbeat in the electrocardiogram; A heartbeat training set generating unit, used for generating a pseudo heartbeat level training set according to the annotated heartbeats; The first training unit is used to train the heartbeat classification model using the pseudo heartbeat level training set with confidence.

5. The heart beat classification device based on the electrocardiogram diagnosis report according to claim 4, characterized in that: Also included is a training device for a rhythmic network model, specifically comprising: A training sample input unit, used for inputting the conventional electrocardiogram training sample into the rhythm network model; The global category labeling unit is used for the rhythm network model to label the electrocardiogram with global categories.

6. The heart beat classification device based on electrocardiogram diagnosis report according to claim 4, characterized in that: The rhythm network adds a U-shaped structure and an attention pooling layer based on the public ResNet34.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the heartbeat classification method based on the electrocardiogram diagnosis report according to any one of claims 1 to 3 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the heartbeat classification method based on the electrocardiogram diagnosis report described in any one of claims 1 to 3 are implemented.

Images