Heartbeat clustering

Abstract

A method includes associating heartbeats with respective initial heartbeat classifications using a trained machine learning model and based on electrocardiogram data. A potential spatial representation of electrocardiogram data is generated for the heartbeat associated with the initial heartbeat classification using an encoder machine learning model. Heartbeats of similar shapes are associated with each other based on the potential spatial representations.

Description

[0001] CROSS REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Provisional Application No. 63 / 413,442 filed on October 5, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to devices, methods and systems for analyzing cardiac activity. Background Art

[0004] Monitoring devices used to collect biometric data are increasingly common in diagnosing and treating patients' medical conditions. For example, mobile devices can be used to monitor a patient's cardiac data. Such cardiac monitoring can provide doctors with valuable information about the occurrence and regularity of a patient's various cardiac conditions and irregularities. Classifying individual heart beats can help accurately identify and classify cardiac events, such as abnormal heart rhythms, so that critical alerts can be provided to the patient, doctor or other care provider, and treatment can be administered to the patient. Summary of the invention

[0005] In Example 1, a method includes associating heartbeats with corresponding initial heartbeat classifications based on electrocardiogram (ECG) data using a trained machine learning model. The initial heartbeat classification includes a first classification and a second classification. The method also includes generating a first latent space representation of the ECG data of the heartbeat associated with the first classification using a first encoder machine learning model. The method also includes generating a second latent space representation of the ECG data of the heartbeat associated with the second classification using a second encoder machine learning model. Based on the first latent space representation and the second latent space representation, heartbeats of similar shapes are associated with each other.

[0006] In Example 2, the method according to Example 1 also includes: using a trained machine learning model to generate separate clips of ECG data, each of the separate clips representing a separate heartbeat.

[0007] In Example 3, according to the method of Example 2, wherein the separate segments include separate T waves for each heartbeat.

[0008] In Example 4, the method according to any one of Examples 2 and 3 further includes: generating a first latent space representation based on separate segments associated with the first category; and generating a second latent space representation based on separate segments associated with the second category.

[0009] In Example 5, according to any of the preceding examples, the method wherein the first latent space representation comprises 8-15 data points for each heartbeat associated with the first classification, and wherein the second latent space representation comprises 8-15 data points for each heartbeat associated with the second classification.

[0010] In Example 6, according to the method of any of the preceding examples, wherein the ECG data for each heartbeat includes a first number of data points, wherein the number of data points represented by the first latent space includes 1-2% of the first number of data points.

[0011] In Example 7, according to the method of any of the preceding examples, associating the heartbeats of similar shapes comprises assigning a same value to each group of heartbeats of similar shapes.

[0012] In Example 8, according to the method of any of the preceding examples, associating heartbeats of similar shapes comprises grouping the heartbeats of similar shapes using a k-means clustering algorithm.

[0013] In Example 9, according to the method of any of the preceding examples, the first encoder machine learning model and the second machine learning model are neural networks trained using unbalanced encoders and decoders.

[0014] In Example 10, according to the method of Example 9, wherein the encoder includes more node layers than the decoder.

[0015] In Example 11, according to the method of any of the preceding examples, the first encoder machine learning model and the second machine learning model are deep neural networks.

[0016] In Example 12, according to the method of any of the foregoing examples, the first classification is a normal heartbeat classification, the second classification is a ventricular heartbeat classification, and the initial heartbeat classification also includes a supraventricular heartbeat classification.

[0017] In Example 13, a computer program product includes instructions for causing one or more processors to perform the steps of the method of Examples 1-12.

[0018] In Example 14, a computer readable medium has stored thereon the computer program product of Example 13.

[0019] In Example 15, a computer includes the computer readable medium of Example 14.

[0020] In Example 16, a method includes using a trained machine learning model and associating heartbeats with corresponding initial heartbeat classifications based on electrocardiogram data. Using the encoder machine learning model, a latent space representation of the electrocardiogram data is generated for the heartbeats associated with the initial heartbeat classification. Based on the latent space representation, heartbeats of similar shapes are associated with each other.

[0021] In Example 17, according to the method of Example 16, wherein the one or more encoder machine learning models include a first encoder machine learning model and a second encoder machine learning model, wherein the initial heartbeat classification includes a first classification and a second classification, wherein the latent space representation includes a first latent space representation of the heartbeat associated with the first classification and a second latent space representation of the heartbeat associated with the second classification. The method also includes: generating the first latent space representation using the first encoder machine learning model; and generating the second latent space representation using the second encoder machine learning model.

[0022] In Example 18, according to the method of Example 17, it also includes: using the trained machine learning model to generate separate segments of ECG data, each of the separate segments representing a separate heartbeat.

[0023] In Example 19, according to the method of Example 18, wherein the separate segments include separate T waves for each heartbeat.

[0024] In Example 20, according to the method of Example 18, it also includes: generating a first latent space representation based on the separate segments associated with the first category; and generating a second latent space representation based on the separate segments associated with the second category.

[0025] In Example 21, according to the method of Example 18, the first latent space representation includes 8-15 data points for each heartbeat associated with the first classification, and the second latent space representation includes 8-15 data points for each heartbeat associated with the second classification.

[0026] In Example 22, according to the method of Example 17, the first classification is a normal heartbeat classification, wherein the second classification is a ventricular heartbeat classification, and wherein the initial heartbeat classification further includes a supraventricular heartbeat classification.

[0027] In Example 23, the method of claim 16, wherein the ECG data for each heartbeat comprises a first number of data points, wherein the number of data points represented by the latent space comprises 1-2% of the first number of data points.

[0028] In Example 24, according to the method of Example 16, wherein associating the heartbeats of similar shapes comprises grouping the heartbeats of similar shapes using a k-means clustering algorithm.

[0029] In Example 25, according to the method of Example 16, wherein associating the heartbeats of similar shapes comprises assigning a same value to each group of heartbeats of similar shapes.

[0030] In Example 26, the method according to Example 25 also includes: receiving ECG data, initial heartbeat classifications, and values ​​assigned to heartbeats through a computing system; displaying the ECG data in a user interface (UI); receiving a command to change at least some of the initial heartbeat classifications to subsequent heartbeat classifications; and automatically modifying the values ​​to different values ​​associated with subsequent heartbeat classifications.

[0031] In Example 27, a system includes a server having: a first trained machine learning model programmed to generate a first latent space representation of heartbeats associated with a first heartbeat classification; a second trained machine learning model programmed to generate a second latent space representation of heartbeats associated with a second heartbeat classification; a third trained machine learning model programmed to generate a third latent space representation of heartbeats associated with a third heartbeat classification; and one or more processors programmed to apply a clustering algorithm to the first latent space representation, the second latent space representation, and the third latent space representation to associate heartbeats of similar shapes with each other.

[0032] In Example 28, the system of Example 27, wherein the ECG data for each heartbeat comprises a first number of data points, wherein the number of data points for the first latent space representation, the second latent space representation, and the third latent space representation comprises 1-2% of the first number of data points.

[0033] In Example 29, the system of Example 27, wherein the first trained machine learning model, the second trained machine learning model, and the third trained machine learning model are deep learning neural networks.

[0034] In Example 30, the system according to Example 27, wherein the clustering algorithm is a k-means clustering algorithm.

[0035] In Example 31, the system according to Example 27, wherein the first heartbeat classification is a normal heartbeat classification, wherein the second heartbeat classification is a ventricular heartbeat classification, and wherein the third heartbeat classification is a supraventricular heartbeat classification.

[0036] In Example 32, the system of Example 27, wherein the first trained machine learning model, the second trained machine learning model, and the third trained machine learning model are neural networks trained using unbalanced encoders and decoders.

[0037] In Example 33, a method for training an encoder neural network includes inputting data into an encoder neural network, the encoder neural network including a first number of node layers. The method also includes generating a latent space representation of the data by the encoder neural network, and inputting the latent space representation into a decoder having a second number of node layers, the second number being less than the first number. The method also includes training the encoder neural network based on an output of the decoder in response to the input latent space representation.

[0038] In Example 34, according to the method of Example 33, wherein the data is ECG data, and further comprising: inputting metadata associated with the ECG data into the encoder neural network.

[0039] In Example 35, according to the method of Example 33, the first number is 4-8 times the second number.

[0040] Although multiple examples are disclosed, other examples of the present disclosure will become apparent to those skilled in the art from the following detailed description, which shows and describes illustrative examples of the present disclosure. Therefore, the drawings and detailed description should be regarded as illustrative in nature, rather than restrictive. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A cardiac monitoring system according to certain examples of the present disclosure is shown.

[0042] Figure 2 A server, a remote computer, and a user interface are shown according to certain examples of the present disclosure.

[0043] Figure 3 An example of heartbeats that have been grouped together and displayed in a user interface is shown in accordance with certain examples of the present disclosure.

[0044] Figure 4-8 Various views of a report building user interface according to certain examples of the present disclosure are shown.

[0045] Fig. 9 and Fig.10 A block diagram depicting an illustrative method according to certain examples of the present disclosure is shown.

[0046] Fig.11 is a block diagram depicting an illustrative computing device according to examples of the present disclosure.

[0047] Although the disclosed subject matter may be subjected to various modifications and alternative forms, specific examples are shown by way of example in the drawings and are described in detail below. However, the purpose is not to limit the disclosure to the specific examples described. On the contrary, the disclosure is intended to cover all modifications, equivalents and substitutes falling within the scope of the disclosure defined by the appended claims. DETAILED DESCRIPTION

[0048] The present disclosure relates to devices, methods and systems for classifying heartbeats (hereinafter referred to as "heartbeats") and grouping heartbeats of similar shapes together into clusters. Such classification and clustering are helpful for analyzing heart activity.

[0049] Electrocardiogram (ECG) data of a patient can be used to identify whether the patient has experienced a cardiac event and what type of cardiac event has occurred. One input to determining the type of cardiac event includes the type (or classification) of heartbeats experienced during the cardiac event. For example, an ECG analysis system can automatically determine that a certain type of cardiac event has occurred based, inter alia, on how the system classifies the heartbeats that occurred during the event. However, if the heartbeats were initially misclassified, the type of cardiac event determined may also be misclassified and, therefore, may need to be reclassified. Therefore, examples of the present disclosure are directed to systems, methods, and devices for classifying and grouping heartbeats and additionally for facilitating analysis and reclassification of heartbeats.

[0050] Figure 1Patient 10 and example system 100 are shown. System 100 includes a monitor 102 (e.g., a pacemaker, ICD, CRT, or ICM) attached to or implanted in patient 10 to detect cardiac activity of patient 10. Monitor 102 can generate an electrical signal representing cardiac activity of patient 10. For example, monitor 102 can detect the heart beat of the patient (e.g., using infrared sensors, electrodes, heart sounds) and convert the detected heart beat into an electrical signal representing ECG data. In some cases, monitor 102 stores ECG data (e.g., one or more days of ECG data) of a patient study, and then transmits the ECG data to another device or system, such as a server. Additionally or alternatively, monitor 102 transmits ECG data to mobile device 104 (e.g., a mobile phone). In this case, mobile device 104 may include a program (e.g., a mobile phone application) that receives, processes, and analyzes ECG data. For example, the program can analyze ECG data and detect or mark cardiac events (e.g., irregular cardiac activity periods) contained in the ECG data. Mobile device 104 can periodically transmit ECG data blocks to another device or system (such as a server), which can process, attach and archive ECG data blocks and metadata associated with ECG data blocks (e.g., time, duration, cardiac events detected / marked). In some cases, monitor 102 can be programmed to directly transmit ECG data to other devices or systems without using mobile device 104. In addition, monitor 102 and / or mobile device 104 include buttons or touch screen icons that allow patient 10 to initiate events. Such instructions can be recorded and communicated to other devices or systems. In other cases involving multi-day studies, ECG data and related metadata are transmitted in larger blocks (e.g., ECG data of the entire study).

[0051] Cardiac Event Server

[0052] The ECG data (and associated metadata, if any) is transmitted to and stored by a cardiac event server 106 (hereinafter referred to as "server 106"). Server 106 includes a number of models, platforms, layers, or modules that work together to process and analyze ECG data so that cardiac events can be detected, filtered, prioritized, and ultimately reported to the patient's physician for analysis and treatment. Figure 1 In the example of FIG. 1 , server 106 includes one or more machine learning models 108A, 108B, and 108C, a clustering algorithm module 109, a cardiac event router 110, a reporting platform 112, and a notification platform 114. Figure 1Only one server 106 is shown, but server 106 may include multiple separate physical servers, and various models / platforms / modules / layers may be distributed among multiple servers. Each model / platform / module / layer may represent a separate program, application, and / or code block, where the output of one model / platform / module / layer is the input of another model / platform / module / layer. Each model / platform / module / layer may communicate between or among other models / platforms / modules / layers and systems and devices external to server 106 using an application programming interface.

[0053] In some cases, once the ECG data is processed by the machine learning models 108A-C and the clustering algorithm module 109, the ECG data (and associated metadata) is available to a reporting platform 112. As will be described in more detail below, the reporting platform 112 can be accessed by a user of a clinic or laboratory 118 via a remote computer 116 (e.g., a client device such as a laptop, mobile phone, desktop computer, etc.). In other cases, the cardiac event router 110 is used to determine which platform further processes the ECG data based on the classification associated with the cardiac event. For example, if the identified cardiac event is critical or severe, the cardiac event router 110 can mark or send the ECG data, etc. to a notification platform 114. The notification platform 114 can be programmed to immediately send a notification (and associated ECG data and associated metadata) to the patient's physician / care team remote computer 116 and / or the patient 10 (e.g., to their computer system, email, mobile phone application).

[0054] Figure 2 A server 106 is shown communicatively coupled to a remote computer 116 (eg, via a network). Figure 2 In the example of , remote computer 116 includes a monitor displaying a user interface 122 (hereinafter referred to as "UI 122"), which displays features of reporting platform 112 hosted by server 106. UI 122 includes multiple pages or screens for tracking and facilitating analysis of patient ECG data.

[0055] In some cases, reporting platform 112 is a software as a service (SaaS) platform hosted by server 106. To access reporting platform 112, a user (e.g., a technician) interacts with UI 122 to log into reporting platform 112 via a web browser so that the user can use and interact with reporting platform 112.

[0056] Machine Learning Models

[0057] Server 106 applies one or more machine learning models 108A-C to the ECG data to analyze and classify the heartbeats and cardiac activity of patient 10.

[0058] As described in more detail below, the first machine learning model 108A and the second machine learning model 108B are programmed to, among other things, compare the ECG data to the labeled ECG data to determine which labeled ECG data the ECG data is most similar to. The labeled ECG data can identify specific cardiac events, including but not limited to ventricular tachycardia, bradycardia, atrial fibrillation, pause, normal sinus rhythm, or artifact / noise, and specific heartbeat classifications, including but not limited to ventricular, normal, or supraventricular. In addition to identifying heartbeat classifications and event classifications (and generating related metadata), the first machine learning model 108A and the second machine learning model 108B can also determine and generate metadata about the heart rate, duration, and heartbeat count of the patient 10 based on the ECG data. As a specific example, the first machine learning model 108A and / or the second machine learning model 108B can identify the start, center, and end of an individual heartbeat (e.g., an individual T wave) so that the individual heartbeat can be extracted from the ECG data. Each individual heartbeat can be assigned a value (eg, a unique identifier) ​​so that throughout the processing and analysis of ECG data, individual heartbeats can be identified and associated with metadata.

[0059] ECG data (e.g., ECG data associated with individual heartbeats) and certain outputs of the first machine learning model 108A and the second machine learning model 108B can be input to a third machine learning model 108C. Although two machine learning models are shown and described, a single machine learning model can be used to generate the metadata described herein, or additional machine learning models can be used.

[0060] The first machine learning model 108A and the second machine learning model 108B may include the neural network described in 16 / 695,534, the entire contents of which are incorporated herein by reference. The first neural network may be a deep convolutional neural network, and the second neural network is a deep fully connected neural network - although other types and combinations of machine learning models may also be implemented. The first machine learning model 108A receives one or more groups of heartbeats (e.g., a heartbeat sequence with 3-10 heartbeats), which are processed by a series of layers in a deep convolutional neural network. The series of layers may include a convolutional layer for convolving the time series data in the heartbeat sequence, a batch normalization layer for normalizing the output from the convolutional layer (e.g., concentrating the results around the origin), and a nonlinear activation function layer for receiving the normalized values ​​from the batch normalization layer. The heartbeat sequence then passes through a set of repeated layers, such as another convolutional layer, a batch normalization layer, and a nonlinear activation function layer. This set of layers may be repeated multiple times.

[0061] The second machine learning model 108B receives the RR interval data (e.g., the time interval between adjacent heartbeats) and processes the RR interval data through a series of layers: a fully connected layer, a nonlinear activation function layer, another fully connected layer, another nonlinear activation function layer, and a regularization layer. The outputs from both paths are then provided to a fully connected layer. The resulting values ​​pass through a fully connected layer and a softmax layer to produce a probability distribution of heartbeat categories.

[0062] The third machine learning model 108C (e.g., one or more trained encoder machine learning models) is programmed to generate a latent space representation of the ECG data such that the ECG data is represented by fewer data points than the original ECG data. The latent space representation can be used as an approximation of the original ECG data for each heartbeat. Although the inputs to the third machine learning model 108C are described as (1) ECG data, such as groups of individual T waves, and (2) certain outputs of the first machine learning model 108A and the second machine learning model 108B, the third machine learning model 108C can be programmed to generate the latent space representation without input from the first machine learning model 108A and / or the second machine learning model 108B.

[0063] In some cases, the server 106 includes separate machine learning models for each type of heartbeat classification (e.g., normal heartbeat, ventricular heartbeat, and supraventricular heartbeat), rather than a single third machine learning model 108C. Figure 1As shown, server 106 may include three third machine learning models (108C-N, 108C-V, and 108C-S) instead of a single third machine learning model. In some cases, heartbeats that were not initially classified (e.g., unclassified heartbeats) may be processed by different third machine learning models, or the step of generating a latent space representation and clustering with heartbeats of similar shapes may be skipped.

[0064] exist Figure 1 In the example of , one machine learning model 108C-N is used for heartbeats classified as normal heartbeats, another machine learning model 108C-V is used for heartbeats classified as ventricular heartbeats, and another machine learning model 108C-S is used for heartbeats classified as supraventricular heartbeats. Therefore, only ECG data (e.g., T waves) for heartbeats that were initially classified as normal heartbeats by the first machine learning model 108A and / or the second machine learning model 108B and metadata generated by such machine learning models are input to the machine learning models 108C-N, and so on. It has been found that using a machine learning model that is trained to focus only on analyzing certain types of heartbeats can improve the performance of the third machine learning model compared to using a single third machine learning model. In addition, using three machine learning models to process ECG data in parallel can reduce the time required to generate a latent space representation. In some cases, a study may contain hundreds of thousands to millions of individual heartbeats.

[0065] Each third machine learning model (108C-N, 108C-V, 108C-S) receives ECG data associated with a separate heartbeat (e.g., a separate ECG data segment for each heartbeat) and generates a latent space representation of such ECG data. For example, each separate heartbeat is processed by one of the third machine learning models (depending on the classification of each separate heartbeat) so that the ECG data is refined into (or represented as) a small number of separate data points. The raw ECG data for a separate heartbeat may include around 500 data points, and each third machine learning model may refine the ECG data for a given heartbeat into 4-16 data points. In other words, each third machine learning model may generate a latent space representation for a given heartbeat that includes 4-16 data points. It has been found that this range balances the accuracy of the heartbeat representation and the effectiveness of clustering (described further below). In some cases, the latent space representation includes 7, 8, or 9 (e.g., 7-9) data points for a given heartbeat. Compared to the raw ECG data for each heartbeat, the latent space representation includes 1-2% of the data points. Each latent space can be represented by a vector (eg, a latent vector).

[0066] The resulting data points are representations of the amplitude of the ECG signal at different relative time points. These limited data points are the ones generated by the trained machine learning model so that different heartbeat shapes can be identified and heartbeats of similar shapes can be grouped together. In other words, these data points may be the ones that are most likely to help distinguish heartbeat shapes. The third machine learning model can omit data point representations that are unlikely to help distinguish individual heartbeats. Figure 3 A set of example heartbeats that have been grouped or clustered together are shown, and non-limiting examples of points 123 within a heartbeat ECG signal are also shown, which can be used to distinguish heartbeat shapes. For example, points 123 can be located at the beginning and end of each heartbeat, apex (e.g., QRS peak), lowest point, etc.

[0067] exist Figure 1 In the example of , the third machine learning model (108C-N, 108C-V, 108C-S) generates corresponding separate latent space representations for the heartbeat groups initially classified as normal heartbeats, ventricular heartbeats, and supraventricular heartbeats. In some cases, heartbeats that cannot be initially classified (or ECG data containing artifacts due to noise) are not processed by any third machine learning model.

[0068] One or more outputs of the one or more third machine learning models 108C are processed by a clustering algorithm module 109. The clustering algorithm module 109 receives the latent space representations of the individual heartbeats and is programmed to associate heartbeats of similar shapes into different groups. Figure 3 A set of example heartbeats that have been grouped or clustered together are shown. Figure 3 As shown, the ECG waveforms (e.g., T waves) of individual heartbeats are superimposed on each other. Each cluster or group may include hundreds or thousands of heartbeats grouped together by the clustering algorithm module 109. It can be seen that these heartbeats all have similar profiles relative to each other. Each heartbeat is aligned with the other heartbeats so that the corresponding QRS peak is centered on the graph.

[0069] In some cases, the clustering algorithm module 109 is programmed to apply a clustering algorithm (such as a k-means clustering algorithm) or a derivative or variant thereof to the latent space representation. In some cases, the same clustering algorithm module 109 and the same algorithm are used to process the latent space representation of each third machine learning model (108C-N, 108C-V, 108C-S). In some cases, the output of the clustering algorithm module 109 includes assigning a value (e.g., an identifier, such as a number) to each heartbeat that indicates the group selected by the clustering algorithm module 109. For example, if the clustering algorithm module 109 clusters the heartbeats into eight different groups, all heartbeats selected in the first group may be assigned a value of "1", and all heartbeats selected in the second group may be assigned a value of "2", and so on. Other types of values ​​may be used. These group values ​​may be added to the metadata associated with each heartbeat.

[0070] As described in more detail below, the heartbeat groups are ultimately presented to the end user in an ECG analysis tool and are used for efficient review of large amounts of ECG data (eg, one or more days of ECG data).

[0071] Report Build Page

[0072] The server 106 (e.g., via programming associated with the reporting platform 112) can initiate the process of sending data to the remote computer 116. The data includes ECG data and metadata associated with the ECG data. Accessing, processing, and displaying one or more days of ECG data and metadata consumes a large amount of computing resources, network bandwidth resources, and human resources. To help reduce the burden on these resources, the server 106 (e.g., via the reporting platform 112) can selectively transmit the ECG data and metadata packages to the remote computer 116.

[0073] The initial data packet may include: (1) a short strip of ECG data surrounding a detected cardiac event (e.g., a 60 second strip), (2) metadata associated with the strip (e.g., heartbeat metadata), and (3) executable code (e.g., JavaScript code). In some cases, only ECG data associated with the highest priority cardiac event is initially delivered. After the initial data packet is transmitted from the server to the remote computer 116, additional data packets may be transmitted in response to selections made by the user in the UI 122.

[0074] Through these initial data packets, the user can access (via remote computer 116 and UI 122) Figure 4-81. The report construction page 200 is shown. The report construction page 200 facilitates the analysis of cardiac events and metadata associated with the cardiac events. The pages are generated at the remote computer 116 based on the data packets, and they are selectively displayed via the UI 122. As will be described in more detail below, when the user interacts with the report construction page 200, the metadata is updated in real time at the remote computer 116.

[0075] Figure 4 A screen shot of a report construction page 200 is shown. The report construction page 200 is used by a user to view and analyze a patient's ECG data and metadata. The report construction page 200 includes a plurality of windows for displaying data, plots, icons, links, tags, indicators, and the like.

[0076] Window 202 displays a heart rate graph of multiple days of ECG data. This window 202 provides an initial visual insight into which time periods appear to contain abnormal heart rate activity. Figure 4 In the example shown, window 202 displays four days of ECG data, although a shorter or longer time period may be displayed by default or by user modification.

[0077] Window 204 allows the user to view a shorter graph of ECG data. For example, window 204 can display ECG data associated with a detected cardiac event and ECG data before and after the detected event. This window 204 provides visual insight into the start of the detected event and whether the detected event may be an artifact, a subsequent event, etc. When the user scrolls through window 204, window 202 can display an indicator (e.g., a vertical line) that shows the location of the ECG data of window 204 within the heart rate graph of window 202.

[0078] Window 208 shows a shorter graph of ECG data (e.g., approximately 10 heart beats) than the graphs of windows 202 and 204. Window 208 displays a close-up view of a portion of the ECG data of windows 202 and 204. A user may use window 204 to select which shorter ECG data set to display in window 208. Each of windows 202, 204, and 208 may include markers, indicators, icons, etc. to visually indicate the location of a detected cardiac event within a strip of ECG data.

[0079] Heartbeat pattern

[0080] exist Figure 4On the left side of the report building page 200 in FIG. 2 is a heartbeat morphology window 210 (hereinafter referred to as "morphology window 210"), which includes multiple individual sub-windows that display one or more individual heartbeat graphs. The user can use the morphology window 210 to select a heartbeat, which can cause the windows 202, 204, and 208 to be updated. For example, windows 202, 204, and 208 can be updated to display ECG data around the heartbeat selected in the morphology window 210.

[0081] Figure 5 and Figure 6 A close-up alternative view of the morphology window 210 is shown. Figure 5 A morphology window 210 is shown displaying a plurality of sub-windows. Each sub-window may display a plurality of heartbeat graphs that are characterized (e.g., by the clustering algorithm module 109) as having a similar shape (or morphology). A user may select a type of cardiac event (e.g., atrial fibrillation, ventricular tachycardia) from a menu button 212 to display a group of heartbeats associated with the selected cardiac event in a sub-window.

[0082] The user can then select individual or multiple groups of heartbeat graphs in the subwindow and, if desired, change the cardiac event type or heartbeat classification associated with the selected heartbeat. In addition, instead of selecting individual heartbeats, the user can select all heartbeats associated with a given cardiac event and change the given cardiac event (or heartbeat classification) to a different type of cardiac event (or heartbeat classification). Because the machine learning model 108A assigns an initial heartbeat classification to each heartbeat, the report construction page 200 can be used to perform large-scale updates to metadata associated with similarly characterized heartbeats. For example, in Figure 6 , the cardiac event types associated with the sub-windows are shown as being associated with 2607 heartbeats. Using the morphology window 210 of the report construction page 200, the user can recharacterize each of the 2607 heartbeats with one operation. The user can also merge the sub-windows using the merge button 214 so that the heartbeats in two or more sub-windows are characterized as the same cardiac event type.

[0083] Figure 7 and Figure 8 Other views showing how the user can make changes to the heartbeat using the UI. Figure 7 In , the user can select a window showing groups of similarly classified or grouped heartbeats, and then select a drop-down menu of options for reclassifying the selected heartbeat group. Figure 7 As shown, potential options for classification include normal beat, ventricular beat, supraventricular beat, and unclassified beat. Figure 8 Another view is shown in which the user can select a heartbeat or group of heartbeats and use buttons to change the classification associated with the selected heartbeat or heartbeats.

[0084] Using one or more of the above methods, metadata for thousands to hundreds of thousands (or millions, for long-term studies) of heartbeats can be updated in bulk through the UI. Because a set of ECG data may represent tens, hundreds of thousands, or even millions of individual heartbeats, this ability to perform large-scale updates of heartbeats saves users time analyzing ECG data and ultimately building reports.

[0085] To conserve processing and network resources, and to allow these changes to metadata to occur in real time, the calculation and modification of cardiac event classifications and the automatic updating of heartbeat classifications may be performed locally on the remote computer 116, rather than sending data back and forth between the server 106 and the remote computer 116. For example, a cache memory 124 (e.g., Figure 1 The reporting platform 112 may utilize the processing power (e.g., one or more microprocessors) of the remote computer 116 to perform the reclassification. To enable local processing and updating, the reporting platform 112 may send code to the remote computer 116 for local execution. The code uses (or operates on) the output of the machine learning model 108, such as heartbeat classification and rhythm classification (as opposed to the underlying or raw ECG data), which reduces the computing resources required to process changes made by the user locally on the remote computer 116. In some embodiments, the code is executed by an Internet browser running on the remote computer 116.

[0086] In some cases, once the final report is built and completed, the remote computer 116 can send any changes to the metadata (e.g., subsequent heartbeat classifications and rhythm classifications) to the server 106 and its database. The server 106 can then replace the metadata originally created by the machine learning model (and saved to the database) with the metadata generated by the remote computer as the user views and edits the metadata. Thus, if the ECG and metadata need to be accessed again, the server's database has the latest version of the metadata. In addition, the machine learning model 108 can be further trained on the metadata generated by the user at the remote computer.

[0087] method

[0088] Fig. 9 1 shows an overview of the steps of a method 300 for clustering heartbeat groups. The method 300 may include the various steps and / or functions described above and is not necessarily limited to Fig. 9 Follow the steps shown in .

[0089] Method 300 includes associating heartbeats with corresponding initial heartbeat classifications using one or more trained machine learning models ( Fig. 9 302 of the drawings). An initial heartbeat classification may be selected from a list of classifications, which may include a first classification (eg, normal heartbeat) and a second classification (eg, ventricular heartbeat).

[0090] The method 300 also includes generating, using the first encoder machine learning model, a first latent space representation of the ECG data of the heartbeat associated with the first classification ( Fig. 9 The method 300 also includes generating, using a second encoder machine learning model, a second latent space representation of the ECG data for the heartbeat associated with the second classification ( Fig. 9 306).

[0091] The method 300 also includes associating heartbeats of similar shapes with each other based on the first latent space representation and the second latent space representation ( Fig. 9 308).

[0092] train

[0093] Fig.10 An overview of the steps of a method 400 for training a machine learning model 108C (including models 108C-N, 108C-V, 108C-S) is shown. In some cases, the machine learning model 108 is an autoencoder neural network (e.g., an autoencoder DNN). The autoencoder neural network can be trained to take a large amount of input data (e.g., ECG data of hundreds of thousands or millions of heart beats) and extract features into a smaller latent space representation that can be used as an approximation of the original input data.

[0094] The autoencoder neural network can be trained using an unbalanced or asymmetric approach. For example, one or more machine learning models 108C can be an autoencoder trained with a decoder, which is a simpler neural network than the neural network of the autoencoder. More specifically, the autoencoder neural network can include more node layers (e.g., hidden layers or intermediate layers of nodes) than the number of node layers of the decoder. In some cases, the number of layers of the autoencoder neural network is 4-8 times that of the decoder. For example, the autoencoder can include 12 layers, while the decoder includes 2 layers.

[0095] Due to the different number of corresponding layers in the autoencoder and decoder neural networks during training, the autoencoder neural network is forced to learn more and generate better latent space representations that the weaker decoder can use for accurate decoding. For example, because the autoencoder neural network is trained by trying to match the output of the decoder with what is input to the autoencoder, the autoencoder is forced to train more rigorously to provide the decoder with high-quality latent space representations. In some cases, the autoencoder neural network is trained using ECG data and corresponding metadata from millions of individual heartbeats. In this case, the autoencoder neural network is trained using ECG data and the output of a trained machine learning model (e.g., a model such as machine learning models 108A and 108B). In addition, in the case where multiple machine learning models 108C are dedicated to a specific type of heartbeat to be processed, the machine learning model 108C can be trained only on ECG data associated with heartbeats with a specific heartbeat classification. Once the autoencoder neural network is trained, it can be implemented as one or more third machine learning models 108C.

[0096] Method 400 includes inputting ECG data and metadata into an encoder neural network, which includes a first number of node layers ( Fig.10 402 in FIG. 4 ). The ECG data may include individual segments of ECG data associated with corresponding heartbeats, and the metadata may include outputs of one or more other machine learning models.

[0097] The method 400 also includes generating a latent space representation of the ECG data by an encoder neural network, and inputting the latent space representation into a decoder having a second number of node layers, the second number being less than the first number ( Fig.10 404 in FIG.

[0098] The method 400 also includes training the encoder neural network ( Fig.10 406 in FIG.

[0099] Although the paragraphs in this section describe an encoder neural network for generating a latent space representation of a heartbeat, the unbalanced or asymmetric training methods described can be used for encoder neural networks used in different applications and with different types of input data. Thus, the present disclosure describes methods for training encoder neural networks for a variety of applications and data by utilizing an encoder neural network that has more node layers that the decoder uses during training. This method forces the encoder neural network to work harder (and get better trained) than training methods for encoder / decoders with the same number of node layers.

[0100] Computing devices and systems

[0101] Fig.11 is a block diagram depicting an illustrative computing device 500 according to examples of the present disclosure. Computing device 500 may include any type of computing device suitable for implementing aspects of examples of the disclosed subject matter. Examples of computing devices include dedicated computing devices or general-purpose computing devices, such as workstations, servers, laptops, desktops, tablets, handheld devices, smartphones, and general-purpose graphics processing units (GPGPUs). Each of the various components shown and described in the figure may include their own dedicated set of computing device components, such as Fig.11 For example, mobile device 104, server 106, and remote computer 116 may each include their own set of components, such as Fig.11 shown and described below.

[0102] In some cases, computing device 500 includes a bus 510 that directly and / or indirectly couples one or more of the following devices: processor 520, memory 530, input / output (I / O) ports 540, I / O components 550, and power supply 560. Any number of additional components, different components, and / or combinations of components may also be included in computing device 500.

[0103] Bus 510 represents one or more buses (such as, for example, an address bus, a data bus, or a combination thereof). Similarly, in an example, computing device 500 may include multiple processors 520, multiple memory components 530, multiple I / O ports 540, multiple I / O components 550, and / or multiple power supplies 560. Furthermore, any number or combination of these components may be distributed and / or replicated across multiple computing devices.

[0104] In an example, memory 530 includes computer-readable media in the form of volatile and / or non-volatile memory, and may be removable, non-removable, or a combination thereof. Examples of media include random access memory (RAM); read-only memory (ROM); electrically erasable programmable read-only memory (EEPROM); flash memory; optical or holographic media; magnetic cassettes, tapes, disk storage, or other magnetic storage devices; data transmission; and / or any other medium that can be used to store information and can be accessed by a computing device. In an example, memory 530 stores computer-executable instructions 570 for causing processor 520 to implement aspects of examples of components discussed herein and / or perform aspects of examples of methods and procedures discussed herein. Memory 530 may include non-transitory computer-readable media stored in computer-executable instructions 570.

[0105] Computer-executable instructions 570 may include, for example, computer code, machine usable instructions, etc., such as, for example, program components that are executable by one or more processors 520 (e.g., microprocessors) associated with computing device 500. Program components may be programmed using any number of different programming environments, including various languages, development toolkits and / or frameworks, etc. Some or all of the functionality contemplated herein may also or alternatively be implemented in hardware and / or firmware.

[0106] According to an example, for example, the instructions 570 may be configured to be executed by the processor 520 and, after execution, cause the processor 520 to perform certain processes. In some cases, the processor 520, the memory 530, and the instructions 570 are part of a controller, such as an application specific integrated circuit (ASIC) and / or a field programmable gate array (FPGA). Such devices may be used to perform the functions and steps described herein.

[0107] The I / O components 550 may include presentation components configured to present information to a user, such as, for example, a display device, a speaker, and / or a printing device, and / or input components, such as, for example, a microphone, a joystick, a satellite dish, a scanner, a printer, a wireless device, a keyboard, a pen, a voice input device, a touch input device, a touch screen device, an interactive display device, and / or a mouse, and / or the like.

[0108] The devices and systems described herein may be communicatively coupled via a network, which may include a local area network (LAN), a wide area network (WAN), a cellular data network, the Internet via the use of an Internet service provider, and the like.

[0109] Various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, systems, and computer program products. It should be understood that each block of the flowcharts and / or block diagrams and combinations of blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions.

[0110] Various modifications and additions may be made to the exemplary examples discussed without departing from the scope of the disclosed subject matter. For example, while the examples described above refer to specific features, the scope of the present disclosure also includes examples with different combinations of features and examples that do not include all of the described features. Therefore, the scope of the disclosed subject matter is intended to include all such substitutions, modifications, and variations that fall within the scope of the claims, as well as all equivalents thereof.

Claims

1. A method comprising: Associating heartbeats with corresponding initial heartbeat classifications based on electrocardiogram (ECG) data using the trained machine learning model, the initial heartbeat classifications comprising a first classification and a second classification; generating, using a first encoder machine learning model, a first latent space representation of the ECG data for heartbeats associated with the first classification; generating, using a second encoder machine learning model, a second latent space representation of the ECG data for heartbeats associated with the second classification; as well as Based on the first latent space representation and the second latent space representation, heartbeats of similar shapes are associated with each other.

2. The method according to claim 1, further comprising: Individual segments of the ECG data are generated using the trained machine learning model, each of the individual segments representing an individual heartbeat.

3. The method according to claim 2, wherein: The individual segments include individual T waves for each heartbeat.

4. The method according to any one of claims 2 and 3, further comprising: generating the first latent space representation based on the individual segments associated with the first classification; as well as The second latent space representation is generated based on the individual segments associated with the second classification.

5. A method according to any one of the preceding claims, wherein: The first latent space representation includes 4-16 data points for each heartbeat associated with the first classification, wherein the second latent space representation includes 4-16 data points for each heartbeat associated with the second classification.

6. A method according to any one of the preceding claims, wherein: The ECG data for each heartbeat includes a first number of data points, wherein the number of data points represented by the first latent space includes 1-2% of the first number of data points.

7. A method according to any one of the preceding claims, wherein: The associating the heartbeats of similar shapes includes assigning a same value to each group of heartbeats of similar shapes.

8. A method according to any one of the preceding claims, wherein: The associating heartbeats of similar shapes includes grouping the heartbeats of similar shapes using a k-means clustering algorithm.

9. A method according to any one of the preceding claims, wherein: The first encoder machine learning model and the second machine learning model are neural networks trained using unbalanced encoders and decoders.

10. The method according to claim 9, wherein: The encoder includes more node layers than the decoder.

11. A method according to any one of the preceding claims, wherein: The first encoder machine learning model and the second machine learning model are deep neural networks.

12. A method according to any one of the preceding claims, wherein: The first classification is a normal heartbeat classification, wherein the second classification is a ventricular heartbeat classification, wherein the initial heartbeat classification further includes a supraventricular heartbeat classification.

13. A computer program product comprising instructions for causing one or more processors to execute the steps of the method of claims 1-12.

14. A computer readable medium having stored thereon the computer program product of claim 13.

15. A computer comprising the computer-readable medium of claim 14.

Images