Electrocardiogram processing system for depiction and classification

Abstract

It is desirable to analyze ECG data using machine learning algorithms and medical-grade artificial intelligence with improved accuracy and efficiency. [Solution] This system receives ECG data from one or more sensing devices positioned on the patient, such as ECG leads. This system includes an application that communicates with an ECG platform running on a server, which processes and analyzes the ECG data using a neural network for depicting cardiac signals and classifying various abnormalities, conditions, and / or descriptors. The ECG platform further processes and analyzes the ECG data using a neural network and / or algorithms for embedding and grouping. The processed ECG data is used to generate a graphical user interface that is communicated from the server to a computer for display in a user-friendly, two-way format with improved accuracy.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims priority to U.S. patent application Ser. No. 16 / 267,380, filed Feb. 4, 2019, published as U.S. Patent Application Publication No. 2019 / 0167143, the entire contents of which are incorporated herein by reference. This application also claims priority to U.S. patent application Ser. No. 14 / 924,239, filed Oct. 27, 2015, published as U.S. Patent Application Publication No. 2017 / 0112401, now U.S. Pat. No. 10,426,364, and European application Ser. No. 15191769.7, filed Oct. 27, 2015, which is the national phase of PCT / EP16 / 075972, published as WO2017 / 072250, the entire contents of which are incorporated herein by reference. This application is also related to U.S. patent application Ser. No. 15 / 771,807, filed Apr. 27, 2018, published as U.S. Patent Application Publication No. 2019 / 0223739, the entire contents of which are incorporated herein by reference. This application also claims priority to U.S. Provisional Application Ser. No. 62 / 549,994, filed Aug. 25, 2017, the entire contents of which are incorporated herein by reference, and is related to PCT / EP2018 / 072912, filed Aug. 24, 2018, published as WO2019 / 038435.

[0002] (Technical Field) The present disclosure generally relates to electrocardiogram (ECG) processing systems, for example, ECG systems with artificial intelligence, having rendering, classification, embedding, and clustering machine learning functionality, and facilitating visualization of large amounts of ECG - related data.

Background Art

[0003] (Background) An electrocardiogram (ECG) receives electrical signals from the heart, which can be digitized and recorded by a computing device. An ECG typically consists of cardiac signals sensed by several electrodes placed within a specific area on the patient's body. It is a simple, non-invasive tool that can be used by virtually any healthcare professional. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] U.S. Publication No. 8,903,479 [Patent Document 2] U.S. Publication No. 8,332,017 [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] A cardiac signal consists of one or more synchronized time signals. Figure 1A illustrates a recording of a standard 12-lead resting ECG. As shown in Figure 1A, each lead generates an electrical signal, resulting in 12 electrical signals. The ECG illustrated in Figure 1A has 12 leads, resulting in 12 recordings, but some ECGs may have fewer leads, resulting in fewer recordings. As shown in Figure 1A, a cardiac signal typically displays a repeating pattern including a P wave, a QRS complex, and a T wave. As the name suggests, the QRS complex includes a Q wave, an R wave, and an S wave. Exemplary P waves, QRS complex, and T waves are illustrated in Figure 1B, focusing on two to three pulses within a single lead signal, representing one RR interval.

[0006] To make a diagnosis, trained healthcare professionals can analyze ECG records and identify any abnormalities and / or seizures. It is estimated that approximately 150 measurable abnormalities can currently be identified on ECG records. However, specific expertise and / or training are required to identify abnormalities from ECGs. ECG analysis is available only to patients who can afford the cost of access to a healthcare professional with the appropriate expertise, or otherwise have access to such professionals.

[0007] Remote cardiology centers have been developed to provide ECG analysis to patients who would otherwise not have access to these trained healthcare professionals. Typically, ECG records are generated remotely by non-specialists and sent to a remote cardiology center for analysis by a cardiologist or a specialized ECG technician. While the results are generally of high quality, the process can be slow and expensive.

[0008] Software systems have also been developed as an alternative to analysis by trained professionals. Current software systems often provide low-quality interpretations that lead to false positives. Currently, these interpretation systems can generate two types of information about cardiac signals: (1) temporal and spatial information for each wave, referred to as description, and (2) global information, referred to as classification, which provides a classification of cardiac signals or indicates abnormalities in them.

[0009] Regarding the description, two main approaches are used to identify the waves of cardiac signals. The first approach is based on multiscale wavelet analysis. This approach searches for wavelet coefficients that reach a predetermined threshold on a given scale. (See Martinez et al., A wavelet-based ECG delineator: evaluation on standard databases, IEEE transactions on biomedical engineering, Vol. 51, No. 4, April 2004, pp. 570-58; Almeida et al., IEEE transactions on biomedical engineering, Vol. 56, No. 8, August 2009, pp. 1996-2005; Boichat et al., Proceedings of Wearable and Implantable Body Sensor Networks, 2009, pp. 256-261; see U.S. Patent No. 8,903,479 to Zoica et al.). The usual process involves identifying the QRS complex, then the P wave, and finally the T wave. This approach is made unstable by the use of thresholds and is unable to distinguish between multiple P waves and "hidden" P waves.

[0010] The second description approach is based on a Hidden Markov Model (HMM). This machine learning approach treats the current state of a signal as a hidden variable that is desired to be extracted (Coast et al., IEEE Transactions on Biomedical Engineering, Vol.37, No.9, September 1990, pp826-836; Hughes et al., Proceedings of Neural Information Processing Systems, 2004, pp611-618; U.S. Patent No. 8,332,017 to Trassenko et al.). This approach is an improvement over the first description approach described above, except that the representation of the signal must be designed using handcrafted "features," and the mathematical model must be fitted wave by wave based on these features. Based on a sufficient number of examples, the algorithm can learn to recognize each wave. However, this process can be cumbersome and inaccurate due to its dependence on handcrafted features. Specifically, handcrafted features are always suboptimal because they are not mastered, and the process of crafting them may ignore or exclude important information. Furthermore, models, typically Gaussian models, do not fit well. Also, current models cannot account for hidden P waves.

[0011] Regarding classification, the current system performs analysis only on the QRS complex. For example, analysis of the QRS complex may detect ventricular or paced pulses. Training involves a handcrafted set of features and corresponding pulse labels (Chazal et al., IEEE Transactions on Biomedical Engineering, 2004, vol.51, pp.1196-1206). As analyzed above, handcrafted features are always suboptimal because they are not mastered and the process of handcrafting the features may ignore or exclude important information.

[0012] To address the above problems, recent research (Kiranyaz et al., IEEE Transactions on Biomedical Engineering, 2016, Vol.63, pp664-675) has focused on a novel architecture called neural networks, which has been extensively studied and yielded significant results in the field of imaging (Russakovsky et al., arXiv: 1409.0575v3, 30 January 2015). Neural networks learn from raw or lightly pre-processed data, thus bypassing the need for hand-crafted features. While the application of neural networks is an improvement over the depiction and classification approaches described above, current systems have some drawbacks. For example, current neural networks have been developed solely for QRS characterization. Furthermore, current neural networks process information in a pulse-by-pulse manner, unable to capture contextual information from surrounding pulses.

[0013] Regarding the identification of abnormalities and / or cardiovascular disease, most algorithms use rules based on temporal and morphological indicators (e.g., PR interval, RR interval, QT interval, QRS width, ST segment level, T wave slope) calculated using depictions. Often, these algorithms are designed by cardiologists (Prineas et al., The Minnesota Code Manual of Electrocardiographic Findings, Springer, ISBN 978-1-84882-777-6, 2009). However, current algorithms are crude simplifications that do not reflect how cardiologists analyze ECGs. For example, the University of Glasgow algorithm does not reflect how cardiologists analyze ECGs (Statement of Validation and Accuracy for the Glasgow 12-Lead ECG Analysis Program, Physio Control, 2009).

[0014] More advanced methods using learning algorithms have also been developed. For example, in Shen et al., Biomedical Engineering and Informatics (BMEI), 2010, vol.3, pp.960-964, the authors used a support vector machine to detect leg blocks. However, these methods again require representing the raw data in a way that reserves invariance and stability qualities.

[0015] While more complex neural network architectures have been developed, limitations have emerged when they are applied to ECGs. One team (Jin and Dong, Science China Press, Vol.45, No.3, 2015, pp 398-416, CN104970789) proposed binary classification on complete ECGs, thus providing a single classification for any analyzed ECG. The proposed architecture used convolutional layers that processed the leads independently before mixing them into fully connected layers. The authors also describe multi-class analysis, which aims to pick one class out of several classes, in contrast to binary analysis. However, they did not consider multi-label classification, where multiple labels (e.g., anomalies) are assigned to cardiac signals.

[0016] In light of the aforementioned limitations of previously known systems and methods, it would be desirable to process ECG data accurately and efficiently and present this information in an easily understandable manner. For example, it may be desirable to obtain a description and classification of ECG signals in a manner that does not require feature extraction, to identify hidden P waves, to analyze ECG signals across multiple pulses, and to achieve multi-label classification for cardiac signals. [Means for solving the problem]

[0017] (Summary of the invention) Provided herein are systems and methods for analyzing ECG data with improved accuracy and efficiency using machine learning algorithms and medical-grade artificial intelligence. Specifically, the systems and methods are provided for analyzing a patient's electrocardiogram (ECG) data using artificial intelligence and a substantial amount of ECG data. The system receives ECG data from one or more sensing devices positioned on the patient, such as ECG leads. The system may include an application that communicates with an ECG platform running on a server, which processes and analyzes the ECG data, for example, using a neural network for depicting cardiac signals and classifying various anomalies, conditions, and / or descriptors. The ECG platform may be a cloud-based ECG platform that processes and analyzes ECG data in the cloud. The processed ECG data is communicated from the server for display with improved accuracy and in a user-friendly, two-way format. Together, the ECG application and ECG platform implement an ECG processing system, receive ECG data, process and analyze ECG data, display ECG data on system devices, and generate reports containing the ECG data.

[0018] A computerized system is provided herein for analyzing patient ECG data, which is generated by one or more electrodes across multiple time points and includes multiple pulses. The computerized system may be designed to analyze the ECG data using a depiction algorithm to generate wave information corresponding to the possibility of the presence of at least one wave at multiple time points, and further to determine pulse onset and pulse disappearance information for pulses among the multiple pulses, which are determined to have at least one wave and to generate multiple pulse onsets and pulse disappearances. The computerized system may further be designed to extract multiple pulse portions of the ECG data based on the multiple pulse onsets and pulse disappearances, each pulse portion of the multiple pulse portions of the ECG data corresponding to a pulse among the multiple pulses, and to determine, based on the multiple pulse portions of the ECG data, that at least two pulses among the multiple pulses should be grouped together, so that at least two pulses form a cluster. Determining that at least two pulses among the multiple pulses should be grouped together may involve determining that the group data meets a threshold.

[0019] The computerized system may further be designed to use an embedding algorithm to analyze multiple portions of ECG data and generate embedding data representing multiple pulses, and a grouping algorithm to analyze the embedding data and generate group data. At least two of the multiple pulses may be determined to be grouped together based on the group data. The group data may correspond to the distance between the two pulses. The drawing algorithm may utilize a first neural network, and the embedding algorithm may utilize a second neural network. The grouping algorithm may utilize a third neural network. The computerized system may further be designed to receive user input data from an input device regarding inaccuracies corresponding to displayed data related to the ECG data. The computerized system may further be designed to adjust one or more of the drawing algorithm, embedding algorithm, or grouping algorithm based on the user input data.

[0020] The computerized system may further be designed to modify the displayed data based on user input data. User input data may correspond to adding, deleting, or splitting one or more QRS clusters, PVC clusters, or PAC clusters. The embedded data may include a data vector for each pulse among multiple pulses. The computerized system may further be designed to transmit information indicating the clusters to a computer for display on a graphical user interface. The computerized system may further be designed to display at least one overlay containing at least two pulses among multiple pulses that generate information and are overlaid across each other. The computerized system may further be designed to use a classification algorithm to analyze the pulses within the clusters and determine the likelihood of the presence of one or more anomalies, conditions, or descriptors associated with a cardiac event related to the patient.

[0021] The computerized system may further be designed to use a classification algorithm to analyze the wave information from the characterization algorithm and determine the likelihood of the presence of one or more abnormalities, conditions, or descriptors associated with cardiac events related to the patient. The wave information may be input into the classification algorithm and used separately to determine that at least two of the plurality of heartbeats should be grouped together. The computerized system may further be designed to preprocess the ECG data and remove noise from the ECG data using a characterization algorithm prior to analyzing the ECG data. The computerized system may assign the ECG data and information based on the ECG data to a user account for review. The computerized system may receive user input data regarding the ECG data and information based on the ECG data from the user account based on the review.

[0022] A method for analyzing electrocardiogram (ECG) data of a patient, generated by one or more electrodes across a plurality of time points and including a plurality of heartbeats, is described herein. The method involves using a characterization algorithm to analyze the ECG data and generate wave information corresponding to the likelihood of the presence of at least one wave at a plurality of time points, and determining heartbeat onset information and heartbeat offset information regarding the heartbeats of the plurality of heartbeats, where it is determined that at least one wave is present and that a plurality of heartbeat onsets and heartbeat offsets are generated. The method further involves extracting a plurality of heartbeat portions of the ECG data based on the plurality of heartbeat onsets and heartbeat offsets, where each heartbeat portion of the plurality of heartbeat portions of the ECG data corresponds to a heartbeat of the plurality of heartbeats, and determining that at least two of the plurality of heartbeats should be grouped together based on the plurality of heartbeat portions of the ECG data, where the at least two heartbeats form a cluster.

[0023] The method may further involve using an embedding algorithm to analyze multiple portions of the ECG data to generate embedded data representing multiple heartbeats, and using a clustering algorithm to analyze the embedded data to generate cluster data. At least two of the multiple heartbeats may be determined to be clustered together based on the cluster data. The method may further involve assigning the ECG data and information based on the ECG data to a user account for scrutiny of the ECG data. The method may further involve submitting the ECG data and information based on the ECG data for quality scrutiny by one or more reviewers. The method may further involve receiving quality control inputs generated by one or more reviewers. The method may further involve causing display of the quality control inputs for additional quality control scrutiny. The method may further involve receiving user input data from an input device regarding inaccuracies corresponding to information based on the ECG data. The method may further involve adjusting one or more of a depiction algorithm, an embedding algorithm, or a clustering algorithm based on the user input data. The method may further involve assigning the displayed data to a user account for quality scrutiny.

[0024] In one embodiment, a system for analyzing patient ECG data may include a first set of instructions, which, when executed, are designed to acquire patient ECG data over a multi-point timescale, and may further cause the transmission of ECG data to at least one server. The ECG data may be sampled at a predetermined sampling rate, such as at least 20 samples per second. The system for analyzing ECG data may further include a second set of instructions, which, when executed, are designed to cause at least one server to receive patient ECG data, analyze the patient ECG data using at least one algorithm trained on multiple ECG datasets from different patients, and transmit information corresponding to the presence of one or more anomalies, conditions, or descriptors, or any combination thereof, from at least one server to a remote computer in order to quantify and display the likelihood of the presence of one or more anomalies, conditions, or descriptors, or any combination thereof.

[0025] The system for analyzing ECG data may also include a third set of instructions, which, when executed by a computer, are designed to cause the computer to display information corresponding to the presence of one or more anomalies, conditions, or descriptors, or any combination thereof, based on information transmitted from at least one server. It should be understood that each set of multiple ECG datasets from different patients may occur at a sampling rate equal to the rate used to acquire the ECG data. Furthermore, it should be understood that the computer executing the third set of instructions may also execute the first set of instructions.

[0026] The second set of instructions, when executed, may cause at least one server to preprocess the ECG data, which may involve removing noise from the ECG data or representing the ECG data at a pre-determined reference frequency. Furthermore, the second set of instructions, when executed, may analyze the patient's ECG data using at least one algorithm that applies the ECG data to a first neural network for depiction, and may further quantify the probability of the presence of at least one of the P waves, QRS complex, or T waves at each of several time points. The second set of instructions may further calculate at least one occurrence and at least one disappearance for at least one of the P waves, QRS complex, or T waves, and / or calculate at least one measurement from one or more occurrences, disappearances, or outputs of the first neural network.

[0027] Furthermore, it should be understood that the second set of instructions, once executed, may analyze a patient's ECG data using at least one algorithm that applies the ECG data to a second neural network for classification. Specifically, the second set of instructions may quantify the likelihood of the presence of one or more anomalies, conditions, or descriptors, applying a threshold to at least one value in the output of the second neural network, and if the value exceeds the threshold, assigning at least one marker corresponding to one or more anomalies, conditions, or descriptors. The second set of instructions may also post-process the ECG data by removing redundant markers.

[0028] The system may further include a fourth and / or fifth set of instructions. The fourth set of instructions, when executed, may cause at least one server to generate a report containing at least transmitted information corresponding to the presence of one or more anomalies, conditions, or descriptors. The fifth set of instructions, when executed, may cause a computer to receive user input related to ECG data and transmit the user input to at least one server so that at least one server uses the user input and generates a report. The report may include at least one heart rate density plot representing the patient's heart rate density as a function of time. It should be understood that the third set of instructions, when executed by the computer, is further configured to cause the computer to display a heart rate density plot representing the patient's heart rate density as a function of time.

[0029] In another embodiment, a system for analyzing patient ECG data may be accompanied by instructions stored on at least one server, which, when executed, are designed to cause at least one server to receive sets of patient ECG data over multiple time points. The sets of ECG data may be sampled at a predetermined sampling rate, such as at least 20 samples per second. The instructions may further be designed to cause at least one server to transmit information to a computer to quantify and display the probability of the presence of one or more anomalies, conditions, or descriptors, or any combination thereof, using at least one algorithm to analyze the sets of patient ECG data. At each of the multiple time points, the probability of the presence of one or more anomalies, conditions, or descriptors may be quantified and displayed. At least one algorithm may be trained using multiple sets of ECG data generated from different patients at a sampling rate of at least 20 samples per second.

[0030] A computerized method for analyzing patient ECG data may similarly involve receiving sets of patient ECG data across multiple time points, sampled at a sample rate, and analyzing sets of patient ECG data using at least one algorithm trained with multiple sets of ECG data. Each set in the multiple sets of ECG data may originate from different patients at the same sample rate. The computerized method for analyzing ECG data may further involve identifying one or more anomalies, conditions, or descriptors, or any combination thereof, at each time point, and further involve transmitting information, including one or more anomalies, conditions, or descriptors, or any combination thereof, to a computer for display. It should be understood that the computerized method may involve analyzing the entire sampled set of ECG data without discarding any data from the set of ECG data. In one embodiment, the computerized method may involve at least 20 samples per second.

[0031] The computerized method may further involve assigning a set of ECG data and information based on that set of ECG data to a user account for scrutiny of the ECG data. The computerized method may further involve submitting a set of ECG data and information based on that set of ECG data for quality scrutiny by one or more scrutinizers. The computerized method may further involve receiving quality control inputs generated by one or more scrutinizers. The method may further involve triggering a display of quality control inputs for additional quality control scrutiny.

[0032] The above summary is illustrative and not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by referring to the following drawings and modes for carrying out the invention. [Brief explanation of the drawing]

[0033] [Figure 1A] Figure 1A shows a standard 12-lead resting ECG recording. [Figure 1B] Figure 1B shows exemplary P-wave, QRS complex, and T-wave recordings. [Figure 2] Figure 2 is a schematic diagram illustrating exemplary components for implementing the system and method according to aspects of this disclosure. [Figure 3A] Figures 3A and 3B are schematic diagrams of exemplary hardware and software components for an exemplary system device and an exemplary server, respectively. [Figure 3B] Figures 3A and 3B are schematic diagrams of exemplary hardware and software components for an exemplary system device and an exemplary server, respectively. [Figure 4] Figure 4 is a flowchart illustrating an exemplary method for processing ECG data, displaying ECG data, and generating a report containing ECG data. [Figure 5A] Figures 5A and 5B are line graphs representing the exemplary ECG signal and the exemplary output of the first neural network for each wave type being analyzed, respectively. [Figure 5B] Figures 5A and 5B are line graphs representing the exemplary ECG signal and the exemplary output of the first neural network for each wave type being analyzed, respectively. [Figure 6A] Figures 6A and 6B are illustrative representations of classification neural networks in the form of convolutional neural networks and regressive neural networks, respectively. [Figure 6B] Figures 6A and 6B are illustrative representations of classification neural networks in the form of convolutional neural networks and regressive neural networks, respectively. [Figure 7] Figure 7 is an illustrative representation of a variable number of wire entries and a fixed number of outputs. [Figure 8] Figure 8 is an exemplary user interface with an RR plot generated according to aspects of recent disclosures. [Figure 9] Figure 9 is a zoomed-in view of the RR plot shown in Figure 8. [Figure 10] Figure 10 is an exemplary user interface having a heart rate density plot generated in accordance with aspects of this disclosure. [Figure 11] Figure 11 is a flowchart illustrating an exemplary approach to generating a heart rate density plot. [Figure 12] Figure 12 is an exemplary heart rate density plot generated in accordance with aspects of this disclosure. [Figure 13] Figure 13 shows an exemplary user interface with a zoomed-in heart rate density plot. [Figure 14A] Figures 14A-14E show a comparative analysis of various RR plots and heart rate density plots generated from the same cardiac signal. [Figure 14B] Figures 14A-14E show a comparative analysis of various RR plots and heart rate density plots generated from the same cardiac signal. [Figure 14C] Figures 14A-14E show a comparative analysis of various RR plots and heart rate density plots generated from the same cardiac signal. [Figure 14D] Figures 14A-14E show a comparative analysis of various RR plots and heart rate density plots generated from the same cardiac signal. [Figure 14E] Figures 14A-14E show a comparative analysis of various RR plots and heart rate density plots generated from the same cardiac signal. [Figure 15A] Figures 15A-15D are exemplary reports generated by an ECG processing system, displaying heart rate density plots and ECG strips, containing information corresponding to the patient and processed ECG data. [Figure 15B] Figures 15A-15D are exemplary reports generated by an ECG processing system, displaying heart rate density plots and ECG strips, containing information corresponding to the patient and processed ECG data. [Figure 15C]Figures 15A-15D are exemplary reports generated by an ECG processing system, displaying heart rate density plots and ECG strips, containing information corresponding to the patient and processed ECG data. [Figure 15D] Figures 15A-15D are exemplary reports generated by an ECG processing system, displaying heart rate density plots and ECG strips, containing information corresponding to the patient and processed ECG data. [Figure 16A] Figure 16A is a flowchart illustrating an exemplary method for processing ECG data, displaying ECG data, and generating a report containing ECG data. [Figure 16B] Figure 16B is an illustrative data flow illustrating the embedding. [Figure 17] Figure 17 shows an exemplary process for analyzing ECG data and grouping similar pulses using a grouping algorithm. [Figure 18] Figure 18 is an illustrative graphic user interface illustrating multiple configurations. [Figure 19] Figure 19 illustrates the third graphic window of the graphic user interface. [Figure 20A] Figures 20A and 20B illustrate exemplary forms and pulse overlays. [Figure 20B] Figures 20A and 20B illustrate exemplary forms and pulse overlays. [Figure 21] Figure 21 is an illustrative graphic user interface illustrating an ectopic interval. [Figure 22A] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22B] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22C] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22D] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22E] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22F] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22G] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22H] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22I] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22J] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22K] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22L] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22M] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22N] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22O] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 22P] Figures 22A-22P show exemplary reports generated by the ECG processing system. [Figure 23A] Figures 23A-23D illustrate a user interface for assigning ECG data and information based on ECG data to healthcare professionals. [Figure 23B] Figures 23A-23D illustrate a user interface for assigning ECG data and information based on ECG data to healthcare professionals. [Figure 23C]Figures 23A-23D illustrate a user interface for assigning ECG data and information based on ECG data to healthcare professionals. [Figure 23D] Figures 23A-23D illustrate a user interface for assigning ECG data and information based on ECG data to healthcare professionals. [Figure 24A] Figures 24A-24F illustrate the user interface for quality control and quality review of ECG data and information based on ECG data. [Figure 24B] Figures 24A-24F illustrate the user interface for quality control and quality review of ECG data and information based on ECG data. [Figure 24C] Figures 24A-24F illustrate the user interface for quality control and quality review of ECG data and information based on ECG data. [Figure 24D] Figures 24A-24F illustrate the user interface for quality control and quality review of ECG data and information based on ECG data. [Figure 24E] Figures 24A-24F illustrate the user interface for quality control and quality review of ECG data and information based on ECG data. [Figure 24F] Figures 24A-24F illustrate the user interface for quality control and quality review of ECG data and information based on ECG data. [Modes for carrying out the invention]

[0034] The aforementioned and other features of the present invention will become apparent from the following description and appended claims, to be interpreted in conjunction with the accompanying drawings. Understanding that these drawings depict only some embodiments of the present disclosure and are therefore not to be considered as limiting its scope, the present disclosure will be described with additional specificities and details through the use of the accompanying drawings.

[0035] (Detailed description of the invention) The present invention relates to an electrocardiogram (ECG) processing system with medical-grade artificial intelligence, comprising an ECG application launched on a system device and an ECG platform launched on a server. The ECG application and ECG platform implement the ECG processing system by processing and analyzing ECG data using machine learning algorithms to achieve visualization of cardiac signals and classification of various anomalies, conditions, and descriptors. The server may be located in a different location from the system device and does not need to be in the same physical location as the system device (for example, the server may be a remote server). Alternatively, the server and system device may be located within the same general area (for example, on a local area network (LAN)). The ECG platform may be a cloud-based ECG platform that can implement the ECG processing system by processing and analyzing ECG data in the cloud.

[0036] To implement an ECG processing system, an ECG application running on a system device may receive ECG data (i.e., heart signals) from a sensing device and transmit the ECG data to an ECG platform running on a server. The ECG platform may run first and second neural networks and apply the ECG data to the first and second neural networks. The first neural network may be a depicting neural network with machine learning capabilities. The second neural network may be a classification neural network with machine learning capabilities. The outputs of the first and / or second neural networks may be processed by the ECG platform to achieve depiction and classification of the ECG data. The ECG data and / or data generated by the ECG platform may be communicated from the ECG platform to the ECG application. The ECG application may display the ECG data and / or data generated by the ECG platform in a bidirectional manner. The ECG platform may generate a report containing the ECG data and / or data generated by the ECG platform and communicate the report to the ECG application.

[0037] Referring now to Figure 2, exemplary components for running the electrocardiogram (ECG) processing system 10 are illustrated. Figure 2 shows the ECG sensing device 13, the system device 14, and the server 15, as well as the drive 16.

[0038] The ECG sensing device 13 is designed to sense the electrical activity of the heart in order to generate ECG data. For example, the sensing device 13 may be one or more electrodes arranged on one or more wires. The ECG sensing device 13 may be a dedicated ECG sensing device such as a conventional 12-wire array, or it may be a multipurpose device with sensing hardware for sensing the electrical activity of the heart for ECG generation, such as the Apple Watch available from Apple, Inc. (Cupertino, California). The sensing device 13 may be placed on the surface of the patient's chest and / or limbs. The sensing device 13 may communicate telecommunications with a system device 14 that invokes the ECG application 29 so that the electrical signals sensed by the sensing device 13 can be received by the ECG application 29. The ECG application 29 may include commands to the sensing device 13 to sense or otherwise acquire ECG data.

[0039] The system device 14 is preferably one or more computing devices (e.g., laptops, desktops, tablets, smartphones, smartwatches, etc.) having the components described below with reference to Figure 3A and the functionality described herein. The system device 14 that launches the ECG application 29 may be connected to the server 15 that launches the ECG platform 37 via any well-known wired or wireless connection. For example, the system device 14 may be connected to the Internet using well-known technologies (e.g., WiFi, cellular, cable / coaxial, and / or DSL) and may communicate with the server 15 via the Internet.

[0040] Server 15 is preferably one or more servers having the components described below with reference to Figure 3B and the functionality described herein. Server 15 preferably has better processing power than system device 14 so that it can process and analyze cardiac signals having a sampling rate above a predetermined threshold, such as at least 20 samples per second, at least 250 samples per second, or at least 1,000 samples per second. As will be readily apparent to those skilled in the art, Server 15 may include multiple servers located in a common physical location or in different physical locations. In a preferred embodiment, Server 15 is located in a remote location different from the system device 14 (e.g., on the cloud), but Server 15 and system device 14 may be located in a common location (e.g., on a local area network (LAN)).

[0041] Server 15 may optionally communicate with a drive 16, which may be one or more drives having memory directed to store digital information specific to a particular patient, specialist, equipment, and / or device. For example, drive 16 may include, but is not limited to, volatile (e.g., random access memory (RAM)), non-volatile (e.g., read-only memory (ROM)), flash memory, or any combination thereof. Drive 16 may be integrated into server 15, or it may be separate and distinct from server 15 and communicate with server 15 via any well-known wireless or wired connection.

[0042] Referring here to Figures 3A-3B, exemplary functional blocks representing the hardware and software components of system device 14 and server 15 are shown. Referring here to Figure 3A, the hardware and software components of system device 14 may include one or more processing units 21, memory 22, storage device 27, communication unit 23, power supply 24, input device 25, and output device 26.

[0043] The processing unit 31 may be one or more processors configured to start the cooperative operating system 28 and the ECG application 29 and to perform the tasks and operations of the system device 14 described herein. The memory 22 may include, but is not limited to, volatile (e.g., random access memory (RAM)), non-volatile (e.g., read-only memory (ROM)), flash memory, or any combination thereof. The communication unit 23 may receive and / or transmit information to, but is not limited to, the sensing device 13 and the server 15, and other components within the ECG processing system 10. The communication unit 23 may also include any well-known communication infrastructure facilitating communication via any well-known wired or wireless connection, including via any well-known standard such as any IEEE 802 standard. The power supply 24 may be a battery, or the system device 14 may be connected to a wall outlet or any other external power supply. The storage device 27 may include, but is not limited to, removable and / or non-removable storage devices such as magnetic disks, optical disks, or tapes.

[0044] The input device 25 may be one or more devices that are coupled to or incorporated into the system device 14 for inputting data into the system device 14. The input device 25 may further include, for example, a keyboard, mouse, pen, voice input device (e.g., microphone), touch input device (e.g., touchpad or touchscreen), location sensor, and / or camera. The output device 26 may be any device that is coupled to or incorporated into the system device 14 for outputting or otherwise displaying data, and may include at least a display 17. The output device 26 may further include, for example, a speaker and / or printer.

[0045] The ECG application 29 may be stored in a storage device 27 and executed on the processing unit 21. The ECG application 29 may also be a software application and / or software module having one or more sets of instructions suitable for performing the operations of the system device 14 described herein, including facilitating the exchange of information with the sensing device 13 and the server 15. For example, the ECG application 29 may cause the system device 14 to receive ECG data from the sensing device 13, to record the ECG data from the sensing device 13, to communicate the ECG data to the server 15, to instruct the server 15 to process and analyze the ECG data, to receive the processed and / or analyzed ECG data from the server 15, to communicate user input regarding the occurrence of a report to the server, and to generate a graphic user interface suitable for displaying the raw, analyzed, and / or processed ECG data and related data.

[0046] The operating system 28 may be stored in the storage device 27 and executed on the processing unit 21. The operating system 28 may be suitable for controlling the general operation of the system device 14 and may work in conjunction with the ECG application 29 to achieve the functionality of the system device 14 as described herein. The system device 14 may also optionally launch a graphics library, another operating system, and / or any other application program. Naturally, it should be understood that the system device 14 may contain additional or fewer components than those illustrated in Figure 3A, and may contain more than one of each type of component.

[0047] Referring here to Figure 3B, the hardware and software components of server 15 may include one or more processing units 31, memory 32, storage device 35, power supply 33, and communication unit 34. Each processing unit 31 may be one or more processors configured to boot the operating system 36 and ECG platform 37 and perform the tasks and operations of server 15 as described herein. It should be understood that, given the amount of data and processing tasks assigned to each processing unit 31, each processing unit 31 has superior processing capabilities compared to each processing unit 21.

[0048] The memory 32 may include, but is not limited to, volatile (e.g., random access memory (RAM)), non-volatile (e.g., read-only memory (ROM)), flash memory, or any combination thereof. The storage device 35 may include, but is not limited to, removable and / or non-removable storage devices such as magnetic disks, optical disks, or tapes. The communication unit 34 may receive information from and / or transmit information to other components of the ECG processing system 10, including system devices 14 and / or drives 16. The communication unit 34 may also be any well-known communication infrastructure facilitating communication via any well-known wired or wireless connection. The power supply 33 may be a battery, or the server 15 may be connected to a wall outlet or any other external power supply.

[0049] The operating system 36 and the ECG platform 37 may be stored in the storage device 35 and executed on the processing unit 31. The operating system 36 may be suitable for controlling the general operation of the server 15. The ECG platform 37 may be a software application and / or software module having one or more sets of instructions. The ECG platform 37 may also process and analyze ECG data received from the system device 14, facilitate and monitor report generation, and may otherwise be suitable for performing the operations of the server 15 described herein.

[0050] The ECG platform 37 may include several submodules and / or applications, including, but not limited to, a preprocessor 38, a delineator 39, a classifier 41, a clustering device 42 which may include an embedding device 48 and a grouping device 49, a postprocessor 43, a report generator 44, and a recomputer 40. Each submodule and / or application may be a separate software application and / or module having one or more sets of instructions. The preprocessor 38 may preprocess the raw ECG data; the delineator 39 may run a first neural network to achieve a depiction; the classifier 41 may run a second neural network to achieve classification; the clustering device 42 may identify clusters in the data processed by the first neural network; the postprocessor 43 may postprocess the data processed by the second neural network; the embedding device 48 may run one or more algorithms and / or a third neural network to achieve embedding; the grouping device 49 may run one or more algorithms and / or a fourth neural network to generate cluster groups; the report generator 44 may generate a report based on the raw ECG data and the ECG data processed by the ECG platform 37; and the recomputer 40 may recalculate and / or adjust the embedding device 48 and / or the grouping device 49 based on user input data. For example, the recomputer 40 may recalculate seizures based on corrected wave information. The ECG platform 37 may also perform a variety of other functions, including, but is not limited to, receiving requests from system device 14 to process and / or analyze ECG data, communicating processed and / or analyzed ECG data to system device 14, receiving requests to generate reports, requesting and / or receiving user interactions and / or commands from system device 14, receiving user input data and / or command information from system device 14 regarding report generation, and / or communicating reports to system device 14.

[0051] Server 15 may also optionally launch graphics libraries, other operating systems, and / or any other application programs. Naturally, it should be understood that Server 15 may contain additional or fewer components than those illustrated in Figure 3B, and may contain more than one of each type of component.

[0052] Figure 4 illustrates an exemplary process for implementing the ECG processing system 10, receiving and recording ECG data, processing and analyzing ECG data, and generating reports with ECG data, and further shows the flow of information between the front end 45 and back end 46 of the ECG processing system 10. The front end 45 includes at least an ECG application 29 that runs on the system device 14. The back end 46 includes at least an ECG platform 37 that runs on the server 15.

[0053] As shown in Figure 4, in step 51, the ECG application 29 may cause the system device 14 to receive and / or otherwise acquire raw ECG data 52 from the sensing device 13. For example, the ECG application 29 may cause the sensing device 13 to sense cardiac signals and communicate the cardiac signals sensed by the sensing device 13 to the system device 14. Raw ECG data is the cardiac signals sensed by the sensing device 13. Raw ECG data 52 has not been processed or analyzed by the ECG processing system 10. Raw ECG data 52 preferably consists of data sampled multiple times per heartbeat across multiple heartbeats. It should be understood that the sensing device 13 may convert analog cardiac signals to digital signals, different components not shown in Figure 2 may convert analog cardiac signals to digital signals, or the ECG application 29 may cause the system device 14 to convert analog cardiac signals to digital signals. Raw ECG data in both analog and digital forms is referred to herein as raw ECG data 52.

[0054] Upon receiving the raw ECG data 52, the ECG application 29 may cause the system device 14 to record the raw ECG data 52, and may optionally store some or all of the raw ECG data 52 in the system device 14. As described above, the signal may correspond to one or more leads. When multiple leads are used, all leads may be processed simultaneously. It should be understood that the cardiac signals generated by each lead may have varying lengths. Furthermore, it should be understood that the cardiac signals can be short-term (e.g., 10 seconds for a standard ECG) or long-term (several days for a Holter monitor). The system device 14 may optionally display the raw ECG data 52 or a portion of it on the display 17.

[0055] As shown in Figure 4, the raw ECG data 52 may be transmitted from the front-end 45 to the back-end 46. Specifically, the ECG application 29 may cause the system device 14 to communicate the raw ECG data 52 to the ECG platform 37 running on the server 15. Upon receiving the raw ECG data 52, the ECG platform 37 may cause the server 15 to store some or all of the raw ECG data 52 on the server 15. Furthermore, after receiving the raw ECG data 52, the ECG platform 37 has the raw ECG data 52 preprocessed by the preprocessor 38 in step 54. It should be understood that the preprocessor 38 may be an independent component of the ECG platform 37 or a subcomponent of the delineator 39.

[0056] The preprocessor 38 may process the raw ECG data 52 or a portion thereof by removing interference elements from the cardiac signal, such as noise, from the raw ECG data. A multivariable function data analysis approach may be used for noise filtering (Pigoli and Sangalli. Computational Statistics and Data Analysis, Vol.56, 2012, pp1482-1498). As the signal sensed by the sensing device 13 may fluctuate due to patient movement, the reference frequency of the raw ECG data 52 may be removed by the preprocessor 38, and the cardiac signal may be represented at a selected frequency. The frequency of the signal corresponding to patient movement may be removed using median filtering (Kaur et al., Proceedings published by International Journal of Computer Applications, 2011, pp30-36). Applying the raw ECG data 52 to the preprocessor 38 generates preprocessed ECG data 55. At this point, the ECG platform 37 may optionally communicate the preprocessed ECG data 55 to an ECG application 29 running on the system device 14 for display on the display 17. Alternatively, or in addition, the ECG platform 37 may allow the preprocessed ECG data 55 to be used as input in the classification step 58, which will be discussed in more detail.

[0057] In step 56, the ECG platform 37 applies the preprocessed ECG data 55 to the delineator 39 for drawing. The delineator 39 applies a first neural network, which is a drawing neural network, to the preprocessed ECG data 55. A neural network refers to a mathematical structure or algorithm that can take an object (e.g., a matrix or vector) as input and, through a set of linear and nonlinear operations called layers, produce another object as output. For example, the input to the first neural network may be one or more multi-wire heart signals that are preprocessed to remove noise and / or reference strays.

[0058] To apply the preprocessed ECG data 55 to the first neural network, the delimiter 39 may cause some or all of the raw ECG data 52 to be represented as a matrix X, which may be a matrix of real numbers. For example, matrix X may be a matrix of size m × n at the frequency used to train the network, as described in more detail below. The constant "m" may be the number of wires in the sensing device 13, typically 12, but any number of wires may be used. In this embodiment, the number of samples "n" provides the duration of the cardiac signal "n / f", where f is the sampling frequency of the cardiac signal. The sampling rate is above a predetermined rate and preferably relatively high, for example, at least 20, at least 250, at least 500, or at least 1,000 samples per second. In one embodiment, all of the sampled ECG data is transferred to a server for input into the processing algorithm, without any ECG data being excluded. The ECG data applied to the first neural network is preferably pre-processed ECG data 55, but it should be understood that unprocessed cardiac signals (i.e., raw ECG data 52 or a portion thereof) can also be applied to the first neural network.

[0059] The first neural network may provide as an output value corresponding to the probability of the presence of one or more waves at multiple time points in the cardiac signal. The time points may be determined by raw ECG data, selected by the user of the system device 14, or pre-programmed. The first neural network may be a convolutional neural network, preferably a fully convolutional neural network. A convolutional neural network is a specific type of neural network in which one or more matrices learned do not encode a complete linear combination of input elements, but through convolution encode the same local linear combination for all elements of a structured signal such as a cardiac signal (Fukushima, Biol. Cybernetics, Vol.36, 1980, pp193-202, LeCun et al., Neural Computation, Vol.1, 1989, pp541-551). A network containing only convolutional networks is called a fully convolutional neural network.

[0060] Therefore, in step 56, the delineator 39 causes the first neural network to read each time point of the cardiac signal, analyze each time point of the cardiac signal spatiotemporally, and assign a score to each time point corresponding to one or more types of waves. In this way, all types of waves in the cardiac signal may be analyzed in a single step, and the probability of their presence at each time point may be quantified. Thus, each score generated by the delineator 39 indicates the probability of the presence of a particular wave type in the cardiac signal at a given time point. The wave types may be any well-known wave types, such as P waves, Q waves, R waves, S waves, Q waves, R waves, S waves, QRS complex, and / or T waves. In this way, the delineator 39 may process data sampled multiple times per heartbeat across multiple heartbeats.

[0061] The output of the first neural network may be a matrix of real numbers, or it may be matrix Y. For example, matrix Y may be a matrix of size p × n. Matrix Y may contain a score for each wave type at each time point in the cardiac signal. In matrix Y, "n" is the number of samples, as discussed above with respect to matrix X, and "p" is the number of wave types plus the number of wave characterizations. As will be explained in more detail below, wave characterizations may correspond to, for example, the conductivity, prematureness, ectopicity, and / or origin of waves in the cardiac signal. In one embodiment, the wave types include (1) P waves, (2) QRS complex, and (3) T waves, and the wave characterizations include (1) early waves, (2) paced waves, (3) ventricular QRS complex, (4) junctional QRS complex, (5) ectopic P waves, and (6) nonconductive P waves. Thus, in this embodiment, p = 3 + 6 = 9. Each wave type can be represented according to certain characteristics of its wave, such as its start and end points (i.e., onset and disappearance).

[0062] Referring now to Figures 5A and 5B, the exemplary output of the first neural network is plotted graph by wave type to illustrate the value of generating a score at each time point corresponding to multiple wave types. Specifically, Figure 5A illustrates the exemplary output when the depicting neural network processes a normal (abnormal) cardiac signal, and Figure 5B illustrates the exemplary output when the depicting neural network processes a cardiac signal with a "hidden" P wave, for example, due to atrioventricular block.

[0063] Referring to Figure 5A, four line graphs are illustrated, with each graph showing time on the x-axis. Line graph 71 represents the cardiac signal over multiple pulses. The plotted signal reflects a well-known ECG waveform, having a P wave (point 75), QRS complex (point 76), and T wave (point 77). Line graph 72 is a graph of the P wave score over the same time point in the cardiac signal. Similarly, line graphs 73 and 74 are graphs of the QRS score and T wave score over the same time point, respectively. The y-axis for each line graph 72-74 is a score assigned at each time point, ranging from 0 to 1, where 0 indicates a low probability of the presence of a particular wave, and 1 indicates a high probability of the presence of a particular wave. For example, line graph 72 shows a very high probability of the presence of a P wave at score 78, corresponding to a time point near point 75; line graph 73 shows a very high probability of the presence of a QRS complex at score 79, corresponding to a time point near point 76; and line graph 74 shows a very high probability of the presence of a T wave at score 80, corresponding to a time point near point 77.

[0064] Figure 5B illustrates four line graphs, namely line graphs 81-82, similar to line graphs 71-74, as in Figure 5A. Specifically, line graph 81 represents the cardiac signal over several pulses, line graph 82 represents the P-wave score over the cardiac signal, line graph 83 represents the QRS score over the cardiac signal, and line graph 84 illustrates the T-wave score over the cardiac signal. Unlike Figure 5A, the ECG signal in line graph 81 includes hidden P-waves, such as the hidden P-wave shown at point 85. A hidden P-wave is a P-wave that occurs between another wave or group, such as a T-wave. As the cardiac signal processed by the drawing network is accompanied by a high sample rate and the drawing network generates data for each wave type at each time point, the output extracted is robust enough (i.e., contains enough sample points) to distinguish between two simultaneously occurring waves, such as those accompanied by hidden P-waves. For example, line graph 82 shows a very high probability of the presence of a P wave at score 86, corresponding to a time point near point 85. Therefore, it should be understood that the depicting neural network is not limited to extracting only one wave at each time point, and thus can identify several waves at any given time point. Furthermore, it should be understood that signals from one or more wires can be processed simultaneously by the first neural network.

[0065] Using scores assigned to each time point corresponding to each wave type (e.g., P wave, QRS complex, T wave, etc.), the delineator 39 may post-process the cardiac signal. Post-processing involves assigning one or more waves to each time point, where no wave is assigned, calculating the onset and disappearance of each identified wave, and optionally determining the characterization of the waves. Waves may be assigned to each time point by determining that a wave exists at that time point if a certain value is achieved. Calculating the “onset” and “disappearance” of each wave involves calculating the start and end times of each wave in the cardiac signal, where the start is referred to as “onset” and the end as “disappearance.” This may involve analyzing the time points corresponding to the start and end of the highest value for each wave type. The delineator 39 may characterize the waves by identifying prematureness, conductivity, and ectopicity. Wave characterization leverages contextual information between each wave and / or each pulse. For example, if a certain threshold is achieved at a certain point in time, or if an average value is achieved over several points in time, early markers may be applied to the wave.

[0066] After calculating the occurrence and disappearance of each wave type in the cardiac signal, the delineator 39 may calculate global measurements. Global measurements are derived from the occurrence and disappearance of each wave type and may relate to the characteristics and properties of the cardiac signal, such as the interval between waves and the duration of waves. For example, global measurements may include, but are not limited to, PR interval, P wave duration, QRS group duration, QRS axis, QT interval, corrected QT interval (Qtc), T wave duration, JT interval, corrected JT interval, heart rate, ST elevation, Sokolov index, number of ventricular premature contractions, number of atrial premature contractions (PACs), ratio of nonconductive P waves, and / or ratio of paced waves.

[0067] Delineator 39 may further infer labels solely from the information it generates. For example, the following labels may be inferred by delineator 39: short PR intervals (i.e., PR interval < 120 milliseconds), first-degree AV block (e.g., PR interval > 200 milliseconds), axis deviation, long QTc, short QTc, broad compound tachycardia, and / or intraventricular conduction block. Labels determined solely from the information generated by delineator 39 are referred to as picture-based labels.

[0068] Referring again to Figure 4, the ECG platform 37 may communicate or otherwise apply the output of step 56 (e.g., wave information 62) and the preprocessed ECG data 55 to the clustering device 42 for clustering in step 63. The wave information 62 may include scores for PVC and PAC waves, including the occurrence and disappearance, and the relevant duration. The clustering device 42 may process the wave information 62 and identify clusters of PAC or PAV waves during the duration of the cardiac signal. Once identified, the clustering device 42 may assign cluster markers 64 to one or more time windows, identifying either PVC or PAC clusters for each time window. A time window is defined by two points in time within the cardiac signal.

[0069] Referring again to Figure 4, the ECG platform 37 may also communicate or otherwise apply the output of step 56 (e.g., wave information 57) and the preprocessed ECG data 55 to the classifier 41 for classification in step 58. The classification in step 58 involves applying a second neural network (i.e., a classification neural network) to the preprocessed ECG data 55. Thus, in one embodiment, the input to the second neural network may be one or more multi-wire cardiac signals with variable length to be preprocessed. The classifier 41 may process the wave information 57 and / or other information such as patient-specific information including the patient's age or any relevant clinical information. As described above, the ECG platform 37 may optionally communicate the preprocessed ECG data 55 directly to the classifier 41 and have the classifier 41 process it if depiction in step 56 is not required. Thus, the classifier 41 may process data that is sampled multiple times per heartbeat across multiple heartbeats.

[0070] The second neural network generates an output having values ​​corresponding to the possibility of the presence of one or more anomalies, conditions, and / or descriptors at each time point in the cardiac signal. If a time point or time window is determined to correspond to a certain anomaly, condition, and / or descriptor, a marker corresponding to that anomaly, condition, and / or descriptor will be assigned to that time point or window. In one embodiment, one or more markers 59 may be assigned to a time point or time window if the score reaches a pre-determined threshold. Thus, multi-marker localization can be achieved with respect to anomalies, conditions, and / or descriptors by generating multiple values ​​at each time point and assigning one or more markers at each time point.

[0071] The classifier 41 may extract the output of the classification neural network as a vector of size q. The values ​​in the vector correspond to the presence of each marker at each time point or time window. For example, the output of the classification neural network may be a vector [0.98:0.89;0.00] with corresponding markers for each element of the vector, i.e., right bundle branch block, atrial fibrillation, and normal ECG. The score may be between 0 and 1. With respect to the above vector, a threshold of 0.5 will cause the classifier 41 to assign the markers "right bundle branch block" and "atrial fibrillation" to the time points or time windows corresponding to the scores. It should be understood that the threshold may be pre-programmed and / or selected by the user and may be modified to provide a degree of sensitivity and specificity that varies. By assigning one or more markers to each time point, the occurrence and disappearance corresponding to each marker may be calculated to identify the duration of a seizure (e.g., an abnormal seizure).

[0072] Abnormalities and conditions may include any physiological abnormalities or conditions that may be identifiable on cardiac signaling. Currently, approximately 150 measurable abnormalities can be identified on cardiac signaling recordings. Abnormalities and conditions may include, but are not limited to, sinoatrial block, paralysis or arrest, atrial fibrillation, atrial flutter, atrial tachycardia, junctional tachycardia, supraventricular tachycardia, sinus tachycardia, ventricular tachycardia, pacemaker, ventricular premature contractions, atrial premature contractions, first-degree atrioventricular block (AVB), second-degree Mobitz type 1 AVB, second-degree Mobitz type 2 AVB, third-degree AVB, Wolff-Parkinson-White syndrome, left bundle branch block, right bundle branch block, intraventricular conduction delay, left ventricular hypertrophy, right ventricular hypertrophy, acute myocardial infarction, old myocardial infarction, ischemia, hyperkalemia, hypokalemia, Brugada syndrome, and / or long QTc. The descriptor may include a description of the cardiac signal quality, such as "normal" or "noisy ECG."

[0073] Depending on the application of the second neural network in step 58, the classifier 41 may read each time-point and global measurement of the cardiac signal, analyze each time-point and global measurement of the cardiac signal, calculate a time window by aggregating at least two time points, and calculate a score for each time window, the score corresponding to multiple non-exclusive markers.

[0074] A classification neural network may be a convolutional neural network or a regressive neural network. Referring here to Figure 6A, a classification neural network in the form of a convolutional neural network is illustrated applied to an ECG signal. Most convolutional neural networks implement several convolutional layers, then standard layers, to provide classification. The ECG signal is given as input to the network, which aggregates the information locally and then combines it layer by layer to generate a multi-level, multi-labeled classification of the ECG. A score is provided for each label. The labels for the convolutional neural network shown in Figure 6 include atrial fibrillation (AFIB), right bundle branch block (RBBB), and ventricular premature contractions (PVCs).

[0075] Referring to Figure 6B, a classification neural network in the form of a recurrent convolutional neural network is illustrated. Similar to Figure 6A, the ECG signal is given as input to the network. A recurrent convolutional neural network refers to a specific convolutional neural network structure that can maintain memory of the object it was previously applied to. A recurrent convolutional neural network consists of two subnetworks: a convolutional neural network that extracts features and computes them at all points in time of the cardiac signal, and a neural network on top of it that accumulates the output of the convolutional neural network over time to provide a refined output. In this way, the convolutional neural network acts as a pattern detector, its output of which will be accumulated over time by the recurrent neural network.

[0076] As shown in Figure 6B, the output of the convolutional neural network identified four labels at various time points, including ventricular premature contractions (PVCs) and normal. These labels were then applied to a second neural network, which generated a refined output, "ventricular premature contraction." In this embodiment, the network correctly recognized ventricular premature contractions (PVCs, the fifth maximum pulse) in the first portion of the signal, while the second portion of the signal was considered normal. If the cardiac signal contains an abnormality, it cannot therefore be considered normal, and the accumulated output is therefore a PVC.

[0077] The first neural network (i.e., the depiction neural network) and the second neural network (i.e., the classification neural network) must be trained to achieve the behavior and functionality described herein. In both the depiction and classification embodiments, the networks may be represented using open-source software such as Tensorflow, Theano, Caffe, or Torch. These tools provide functionality for calculating the network's output and updating its parameters through gradient descent.

[0078] Training a neural network involves applying a large number of datasets containing cardiac signals and known outputs to the neural network. Using the systems and methods described herein, a database of datasets containing cardiac signals collected across multiple patients may be stored on server 15 and / or drive 16 (e.g., in the cloud). The datasets in the database may be used by server 15 to analyze new cardiac signals input to the system for processing. In a preferred embodiment, any cardiac signals applied to the trained neural network will have the same sampling rate and / or frequency as the cardiac signals in the dataset used to train the neural network. For example, training a classification neural network begins with a dataset containing cardiac signals and their known depictions. As described above, the cardiac signals are represented as a matrix of size m × n at a given frequency. For example, the network may be trained at 250 Hz, 500 Hz, or 1,000 Hz, but any frequency can be used. The depictions are then represented in the form of a matrix Y of size p × n, where p is the number of wave types. Each wave is represented with its start and end points, for example, (P, 1.2 sec, 1.3 sec), (QRS, 1.4 sec, 1.7 sec), (T, 1.7 sec, 2.1 sec), (P, 2.2 sec, 2.3 sec). In this embodiment, the first row of matrix Y corresponds to the P wave and will have a value of 1 at times 1.2 sec and 1.3 sec, and 2.2 sec and 2.4 sec, and 0 otherwise. The second row of matrix Y corresponds to the QRS group and will have a value of 1 at times 1.4 sec and 1.7 sec, and 0 otherwise. Finally, the third row of matrix Y corresponds to the T wave and will have a value of 1 at times 2.2 sec and 2.3 sec, and 0 otherwise. The network parameters may then be modified to reduce the cost function, comparing the network output to a known depiction. The entropy cross-error function is used to allow for multi-signaling (i.e., enabling multiple waves at a given moment).This minimization can be performed through gradient steps, repeating the aforementioned steps at least once for each heart signal in the dataset. It should be understood that a similar approach can be used to train a depicting neural network (i.e., a second neural network).

[0079] Furthermore, it should be understood that the ECG platform 37 can cause the neural network described herein to process cardiac signals having different numbers of input conductors. For example, the neural network may have a set of layers at the beginning of the network to obtain a network that is independent of the number of input conductors and therefore capable of processing cardiac signals using any number of conductors m. For example, Figure 7 illustrates two input conductors (m=2) and three output signals (k=3). However, the same structure can process any number of input conductors m and still provide the same number of output signals that can be fed to the rest of the network where a fixed number of input signals are required. For this reason, the number of input conductors can vary and does not need to be fixed.

[0080] As shown in Figure 7, to obtain k signals from m input wires, the wires may be convolved using k filters and wire-by-wire convolution. The signals may then be grouped by convolution filters to obtain k groups of m wires, and a mathematical function is applied to each group to ultimately obtain k wires. The mathematical function may be the maximum value at each point in time, or any other function known to those skilled in the art.

[0081] Referring again to Figure 4, in step 61, the ECG platform 37 may aggregate the time window labels (i.e., labels) by the post-processor 43 to generate the processed labels 60. The labels may be derived from global measurements based on the drawing. For example, the label corresponding to first-degree atrioventricular block may be derived from a PR interval longer than 200 milliseconds. As described above, the PR interval is a global measurement based on the drawing. The post-processor 43 may also aggregate the drawing-based labels with classification labels corresponding to the same time period.

[0082] The postprocessor 43 may also filter the labels to remove redundant labels, assemble the labels according to a known hierarchy of labels, or ignore labels that are known to be less important according to hierarchy or weighting values. The postprocessor 43 may also aggregate the labels over time to calculate the start (appearance) and end (disappearance) times of each anomaly. It should be understood that the postprocessor 43 may be a standalone component or a subcomponent of the classifier 41.

[0083] As shown in Figure 4, the information generated on the backend 46 by the ECG platform 37 in steps 54, 56, 58, and 61, and optionally in step 63, may be communicated by the ECG platform 37 to the ECG application 29 on the frontend 45. The ECG application 29 may display the aforementioned information on the display 17 of the system device 14 in step 65. The information generated on the backend 46 may be transmitted automatically by the ECG platform 37, or the ECG platform 37 may store the information on the server 15 until requested by the ECG application 29. In response to generating data, the ECG platform 37 may transmit a message to the ECG application 29 informing it that the data is available from the ECG platform 37.

[0084] The ECG application 29 may receive and display data (e.g., raw ECG data, pre-processed ECG data, wave information, labels, and any other data generated during steps 54, 56, 58, 61, and / or 63) on the system device 14, as described in PCT / EP2018 / 072912 (the entirety of which is incorporated herein by reference). Specifically, application '912 describes how the ECG signal, the characteristics of the ECG signal, and / or the descriptor of the ECG signal may be displayed in a bidirectional manner within multiple field displays.

[0085] Referring here to Figure 8, an exemplary display, namely a bidirectional display 101, is illustrated. The bidirectional display 101 includes a first side 102 and a second side 103. The first side 102 further includes a second graphic window 105 and a first graphic window 104, which have a plot 110 containing data corresponding to the ECG signal. The first graphic window 104 includes a plot 110 that provides a global view of the ECG signal.

[0086] Referring here to Figure 9, a zoomed-in version of the first graphic window 104 is illustrated. In this exemplary display, plot 110 is an RR interval plot, which is a plot of RR intervals (the interval between two QRS waves) over time. As shown in Figure 9, the upper area of ​​the first graphic window 104 comprises several indicator buttons 109. Each indicator button 109 displays text adjacent to it that describes the indicator it is associated with. Each indicator button 109 is associated with a color such that when the indicator button 109 is selected by the user, the graphic portion 111 is displayed on the plot 110 to visually indicate the presence of seizures and / or events corresponding to the indicator associated with the indicator button 109. This provides a visual reference for the user, allowing for easy identification of specific categories of events and / or seizures along with cardiac signals. In the exemplary display illustrated in Figure 9, a secondary indicator 112 is included. In this illustrative representation, secondary markers 112 include pulse-marked PVCs (ventricular premature contractions) and PSVCs (supraventricular premature contractions), but it should be understood that other secondary markers may also be included. Points in plot 110 associated with marked PVCs and PSVCs are colored as shown in Figure 9 by the presence of black and points of different colors.

[0087] The first graphic window 104 further includes a time bar 115 parallel to the time axis of the plot 110. The time bar 115 provides a linear representation of the total ECG acquisition time, where time periods associated with seizures or events are represented as colored segments. As shown in Figure 9, darker gray zones on the time bar 115 correspond to time periods of noisy signals (e.g., when the signal contains excessive artifacts and the analysis algorithm cannot suggest depiction and appropriate detection). The first graphic window 104 further includes a bidirectional cursor 116. A user of the ECG application 29 may move the bidirectional cursor 116 along the time bar 115 to enable navigation of the plot 110 along the total ECG acquisition time. In the lower right corner of the first graphic window 104, the first graphic window 104 includes a second bidirectional means 117 configured to zoom in and out of the plot 110.

[0088] Referring again to Figure 8, the second aspect 103 includes multiple seizure plots 106. Each seizure plot 106 displays at least one segment of an ECG strip corresponding to a detected seizure and may include text regarding the duration (e.g., "Duration: 1 hour 38 minutes") and / or the seizure's onset time (e.g., "Day 3 / 09:39:30"). Each seizure plot 106 includes a third bidirectional icon 108 for selecting the corresponding seizure plot to include in the report. Each seizure plot 106 further includes a fourth bidirectional icon 107, which allows the user to remove individual ECG plots from the bidirectional display 101. The second aspect 103 may further include text describing one or more of the seizure plots 106.

[0089] The bidirectional display 101 further includes a graphics window 105 which includes an ECG strip 118 in a second time window that starts from the point selected by the cursor 116. The second graphics window 105 further includes an ECG strip 119 in a third time window that is larger than the second time window and includes the second time window. The third time window includes a shaded portion corresponding to the second time window.

[0090] Referring now to Figure 10, a similar display, namely a bidirectional display 121, is illustrated. The bidirectional display 121 includes a first side 122 and a second side 123. The first side 122 further includes a first graphic window 124 and a second graphic window 125. The second side 113 has the same functionality as the second side 103 described above and includes a seizure plot 126 similar to the seizure plot 106. Furthermore, the second graphic window 125 has the same functionality as the second graphic window 105 and includes ECG strips 138 and 139 similar to ECG strips 118 and 119.

[0091] The first graphic window 124 is similar to the first graphic window 104 except for plot 130. Like the first graphic window 104, the first graphic window 124 includes multiple indicator buttons 129 having the same functionality as multiple indicator buttons 109, a secondary indicator 132 having the same functionality as secondary indicator 112, a time bar 135 and cursor 136 having the same functionality as time bar 115 and cursor 116, and a second bidirectional means 137 having the same functionality as second bidirectional means 117. Unlike plot 110, plot 130 is a heart rate density plot, which is a projection of a histogram of heart rate density as a function of time onto a bivariate intensity plot.

[0092] Referring here to Figure 11, steps are provided for generating and plotting heart rate density plots such as plot 130. In step 141, the ECG platform 37 calculates the RR interval in the cardiac signal (i.e., ECG data). For example, the ECG platform 37 may apply the cardiac signal to a depicting neural network to determine the RR interval as described above. In step 142, the ECG platform 37 may generate a heart rate plot over time. An exemplary heart rate plot, i.e., HRDP 150, is shown in Figure 12.

[0093] As shown in Figure 12, time is projected along the x-axis, and heart rate (e.g., pulses per minute) is projected along the y-axis. In one embodiment, both time and heart rate are scaled linearly. However, time and / or heart rate may be scaled logarithmically or using other well-known scales. For simplicity, only four heart rates are shown in Figure 12.

[0094] Referring again to Figure 11, in step 143, the ECG platform 37 may divide the y and x axes into basic elements called HR bins and time bins, respectively. For example, in Figure 12, HR bin 151 and time bin 152 are illustrated. HR bin 151 contains the first and second heart rate values ​​(e.g., hb 1 and hb 2 ) is defined by the same. Similarly, the time bin 152 is defined by the first and second time values ​​(e.g., tb 1 and tb 2 ) is defined by the intersection of the HR bin and the time bin. The intersection of the HR bin and the time bin will be called a bin. In other words, a bin will be defined by the first and second heart rate values ​​and the first and second time values. In Figure 12, bin 153 is shown and is defined by the HR bin 151 and the time bin 152.

[0095] Referring again to Figure 11, in step 144, the ECG platform 37 will assign each heartbeat to a bin. Specifically, heartbeats (e.g., QRS complexes) occurring within a given time window of a time bin are included in the calculation of the column corresponding to that time bin. Furthermore, the heart rate corresponding to that heartbeat determines the HR bin to which it belongs within the column defined by the time bin. For example, in Figure 12, heartbeats 154 and 155 have corresponding time and heart rate values ​​that fall within time bin 152 and HR bin 151, respectively. Conversely, heartbeats 156 and 157 each have time values ​​that fall outside time bin 151, and therefore neither is included in bin 153.

[0096] Referring again to Figure 11, in step 145, the ECG platform 47 will calculate the heart rate density for each time bin. With respect to a given bin, the area defined by the individual time bins and heart rate bins will be represented according to the density of heartbeats contained in the bin (i.e., the number of heartbeats in the bin). Each bin may then be color-coded according to its density. For example, each bin may have a shade or pattern such as, for example, a gray level. In the embodiment of Figure 12, the bins may be represented as gray levels that become darker as the heart rate density increases. As shown in Figure 12, a bin 153 containing two heartbeats may be represented by a darker shade of gray than a bin with only one heartbeat, but by a lighter shade of gray than a bin with three or more heartbeats.

[0097] In a preferred embodiment, density is calculated as a function of the number of R waves in a bin divided by the heart rate of the HR bin (e.g., the average of the minimum and maximum boundaries of the time window). This preferred calculation of density takes into account the time spent in a particular bin. For example, in a 3-minute time bin, if there are 100 pulses at a heart rate of 50 bpm (pulses per minute) in the first HR bin and 100 pulses at 100 bpm in the second HR bin, then there will be the same number of pulses in each bin, but 2 minutes will be spent at 50 bpm and only 1 minute at 100 bpm. Thus, the bins will have the same density representation if only the number of pulses is considered. However, when considering the pulse count divided by the heart rate, the first bin corresponding to the 50 bpm heart rate bin will be denser than the bin corresponding to the 100 bpm heart rate bin, because dividing by the heart rate gives a higher weight to a lower heart rate value. A preferred embodiment therefore captures this time information better than considering only pulse counting.

[0098] Referring again to Figure 11, in step 146, the ECG platform 37 will plot the heart rate density for each bin. It should be understood that capturing the time information within the columns (time bins), in addition to the time information naturally given as a function of the x-axis, facilitates a representation of density in a better manner than other aggregated representations of the ECG signal, such as RR plots in plot 110.

[0099] It should be understood that the x-axis boundary of the HR density plot can be the start and end of the signal. However, in a preferred embodiment, the x-axis boundary can vary bidirectionally with zoom-in and zoom-out actions performed by the user. The y-axis boundary remains fixed when this action is performed. Referring again to Figure 10, plot 130 includes a bidirectional means 137 that can be used to zoom in on the heart rate density plot. The zoom action can only change the size of the plot display. Alternatively, zooming in and out changes the size of the time window corresponding to the time bins. Using the zoom-in action, bins represented with the same number of pixels cover a shorter time window. Zooming in, therefore, leads to a finer time division, and consequently, a new calculation of the histogram with finer time information. This allows for a representation of the ECG signal that shows a fluctuating level of information aggregation as a function of the time series which is chosen to display so that the histogram remains both readable and useful at any level of zoom. Referring now to Figure 13, a bidirectional display, i.e., a bidirectional display 170 similar to the bidirectional display in Figure 10, is illustrated. The bidirectional display 170 is zoomed in, resulting in a plot 159 having a zoomed-in portion 158.

[0100] Figures 14A-E illustrate the superiority of HRDP compared to a typical RR plot. Referring to Figure 14A, the Holter-generated signal with a very large number of PVCs accompanied by fluctuating couplings is illustrated as RR plot 161 and density plot 162. In density plot 162, the fundamental tuning is clearly visible as line 171. Furthermore, compensatory rests are illustrated below as line 172. In RR plot 161, this pattern is not as clear. Referring to Figure 14B, the Holter-generated signal with fewer premature contractions than that in Figure 14A is illustrated as RR plot 163 and density plot 164. The primary tuning is clearly illustrated in density plot 164 and not as clear in RR plot 163. Referring to Figure 14C, the Holter-generated signal with fluctuating conducted flutter is illustrated as RR plot 165 and density plot 166. As shown in Figure 14C, conduction flutter is highlighted by four distinct black lines in density plot 166 rather than by four spread clouds appearing in RR plot 165. Referring now to Figure 14D, the signals generated by Holter with persistent atrial fibrillation are illustrated as RR plot 167 and density plot 168. As shown in this figure, density plot 168 provides more precise information about the variation in heart rate within the fibrillation. Specifically, the denser lower half 173 indicates that more time is spent at lower heart rates than at higher heart rates. Density plot 168 further illustrates a surge, with the upper half becoming slightly denser in response to an increasing heart rate. These nuances are not visible in RR plot 167. Referring now to Figure 14E, the signals generated by Holter with paroxysmal atrial fibrillation, otherwise having a regular rhythm, are illustrated as RR plot 174 and density plot 175. The pattern of regular rhythm is more visible in density plot 175, where a clear black line appears. Also, the pattern of atrial fibrillation is more contrasting in density plot 175 than in RR plot 174, as the color changes (density decreases, making the plot thinner).

[0101] Referring again to Figure 4, in step 66, a user of the ECG application 29 may use the input device 25 to interact with the two-way active display described above, request a report, and / or customize the report. The report may typically include portions of the cardiac signal, information about abnormalities and / or seizures (e.g., seizure plots), and / or other information generated during pre-processing (step 54), depiction (step 56), classification (step 58), clustering (step 63), and / or post-processing (step 61). The report may further include patient-specific medical data such as the patient's name, age, medical history, and / or other medical information. It should be understood that any individually identifiable health information and / or protected health information may be encrypted when communicated between the ECG application 29 and the ECG platform 37.

[0102] As explained above, the bidirectional icons within the bidirectional display may be involved in incorporating data and images to be displayed in the report. For example, a third bidirectional icon 108 may be selected by the user using the ECG application 29 to include a seizure plot corresponding to the report. Thus, in step 66, the user may request a report and select customized features such as certain data (e.g., abnormal data, seizure data, seizure plot, etc.) that should be included in the report.

[0103] In step 67, the ECG application 29 may transmit a request for a report and selected customizable features (e.g., ECG data to be included in the report) to the ECG platform 37, which may receive the request and information. The ECG platform 37 may record the request and store the information received from the ECG application 29. In step 68, the ECG platform 37 may generate a report in the report generator 44 according to the information received from the system ECG application 29.

[0104] Referring here to Figures 15A-15D, an exemplary report generated in step 68 is illustrated. The first page of the exemplary report is illustrated in Figure 15A. The first page may be presented in several sections, such as the first section 181, the second section 182, the third section 183, the fourth section 184, the fifth section 185, and the sixth section 186. The first section 181 may include patient-specific information such as the patient's name, primary indication, whether the patient has a pacemaker, the patient's date of birth, sex, and / or patient ID. The second section 182 may include clinician information such as the supervising physician, name of the institution, date of analysis, and / or signature.

[0105] A third section, Section 183, may include plots of ECG data. In Figure 15A, Section 183 includes a heart rate density plot similar to that shown in Figure 12. The time window shown may be a default time or a user-defined time window. As with the heart rate density plot in Figure 12, certain indicators may be selected to show the occurrence of anomalies on the density plot. The time window is usually selected according to the relevant seizures and / or events. However, it should be understood that other plots, such as RR plots, may also be included in the report.

[0106] Section 4, 184 may include metrics from cardiac signal recordings. For example, Section 4, 184 may include recording duration, maximum, minimum, and mean heart rates, supraventricular premature contractions and any patient-induced events, and / or any other metrics relating to cardiac signals. Section 5, 185 may include information corresponding to any seizures detected. For example, Section 5, 185 may include pause information (counting and / or longest RR interval), atrioventricular block information, atrial fibrillation / flutter information, ventricular tachycardia information, other supraventricular tachycardia information, and / or any other information relating to any seizure or abnormality. Section 6, 186 may include result information such as a summary of seizures and / or abnormalities, a diagnosis, and / or any other information analyzed, aggregated, calculated, determined, identified, or otherwise detected from cardiac signals. For example, Section 6, 186 may identify sinus rhythm with paroxysmal atrial fibrillation.

[0107] Figures 15B-D illustrate the second, third, and fourth pages of an exemplary report. As shown in Figures 15B-D, the report may further include ECG strips pre-selected by the user or selected under default settings. For example, the user may select a maximum HR strip 191, a minimum HR strip 192, an atrial fibrillation / flutter strip 193, another SVT strip 194, a PSVC strip 195, and a PVC strip 196. The maximum HR strip 191 may be an ECG strip showing the maximum heart rate during a given cardiac signal recording. Similarly, the minimum HR strip 191 may be an ECG strip showing the minimum heart rate during a given cardiac signal recording. The atrial fibrillation / flutter strip 193 may be an ECG strip showing each episode of atrial fibrillation / flutter. The other SVT strip 194 may be an ECG strip showing each episode of supraventricular tachycardia. The PSVC strip 195 may be an ECG strip showing an episode of supraventricular premature contractions. The PVC strip 197 may be an ECG strip indicating an episode of ventricular premature contractions. The ECG strip may be displayed with relevant and associated metrics and comments, such as those added by the user. It should be understood that the reports shown in Figures 15A–B are illustrative only, and reports generated in step 68 may have a different structure or configuration and / or may include different ECGs and patient-related information discussed herein.

[0108] Referring here to Figure 16A, an ECG processing system 10' is illustrated, which is substantially similar to the ECG processing system 10 but includes steps 202-205. Like the ECG processing system 10, the ECG processing system 10' may receive raw ECG data 52 in the backend 46 so that it is preprocessed in step 54 by the preprocessor 38, and generate preprocessed ECG data 55 which can be applied to the delineator 39 for depiction in step 56 to generate wave information 201. The wave information 201 may include wave information 57 (e.g., scores corresponding to the likelihood of the presence of T waves, P waves, QRS groups, etc.) and / or wave information 62 (e.g., scores relating to PVC waves and PAC waves, etc.). As described above, the output of the delineator 39 may be one or more vectors and / or matrices.

[0109] Wave information 201 may further include information about the appearance and disappearance of one or more waves and / or pulses in the ECG data. For example, based on the likelihood of the presence of a certain type of wave at a given point in time throughout the ECG data, the delineator 39 may determine the appearance and disappearance of one or more waves and / or pulses. In step 200, the clustering device 42 may use the appearance and disappearance of pulses in the wave information 201 to extract pulses from the ECG data (e.g., pre-processed ECG data). This may determine portions of the ECG data that begin with the appearance of a pulse and end with its disappearance. These portions may be extracted from the ECG data and designated as pulses. The portions of the ECG data corresponding to pulses may be used as input in step 202.

[0110] Furthermore, in step 200, the clustering device 42 may monitor and / or implement an embedding device 48 and a grouping device 49. Specifically, step 200 may include substeps 202 and 204, in which the clustering device 42 may execute one or more algorithms, which may be one or more trained neural networks, to generate embedding data 203, and ultimately group data 205. Specifically, in substep 202, the embedding device 48 may execute one or more algorithms, which may be trained neural networks, to generate embedding data 203. The portion of the ECG data extracted in step 200 may be input to the embedding device 48. The embedding data 203 may be one or more vectors representing the pulse portion of the ECG data used as input for the embedding device 48. Thus, pulses identified within the ECG data may be extracted from the ECG data and embedded using the embedding device 48. In one embodiment, the embedding data 203 may be one or more vectors and / or matrices corresponding to and representing one or more pulses. In substep 204, the grouping device 49 may run one or more grouping algorithms, which may be trained neural networks, on the embedded data 203 to generate group data 205.

[0111] The process of extracting a portion of the ECG data, inputting that portion into the embedding device 48 to generate the embedding data 203, and inputting the embedding data 203 into the grouping device 49 is illustrated in more detail in Figure 16B. As shown in Figure 16B, a portion of the ECG data 206 may be used as input for the embedding device 48 for embedding. The portion of the ECG data 206 may be a portion of the ECG data (e.g., pre-processed ECG data) over a time period that begins when the pulse appears and ends when the pulse disappears. For example, the embedding device 48 may process the portion of the ECG data 206 using one or more algorithms to produce the embedding data 203. The one or more algorithms may be one or more neural networks (e.g., embedding neural networks). The embedding data 203 may be one or more pulse vectors 207. The pulse vectors 207 may be vectors containing data and / or values ​​representing the portion of the ECG data 206. As shown in Figure 16B, the pulse vector 207 may be input to the grouping device 49 for grouping.

[0112] Referring again to Figure 16A, the grouping device 49 may use one or more algorithms to process the embedding data 203 corresponding to pulses, yielding group data 205 corresponding to the similarity between two or more pulses. The one or more algorithms may be one or more neural networks (e.g., grouping neural networks). The group data 205 may include one or more scores or values ​​corresponding to the similarity and / or distance between vectors of the embedding data 203. The similarity may be accompanied by similarity properties and / or features, such as similar T waves, P waves, QRS complex, or other waves, or similar distance between any of the aforementioned. In this way, the grouping device 49 may quantify the similarity between two or more pulses in the ECG data. Based on the group data 205, the clustering device 49 may determine whether the group data 205 (e.g., scores or values) are within a threshold difference or satisfy a threshold, and therefore whether the corresponding pulses are sufficiently similar to each other (e.g., have similarity properties and / or features) so that they should be grouped or matched together. Groups of similar pulses may be referred to herein as morphologies and / or clusters. As shown in Figure 16A, the group data 205 may optionally be applied to the classifier 41 for classification in step 58.

[0113] As shown in Figures 3A, 3B, and 16, the group data 205 and / or a graphical user interface based on or otherwise incorporating the group data 205 may be communicated by the ECG platform 37 on the backend 46 to the ECG application 29 on the frontend 45. The ECG application 29 may display information about pulse grouping and / or the group data 205 on the display 17 of the system device 14 in step 65. For example, information about pulses associated with seizure types (e.g., QRS clusters, PVC clusters, and / or PAC clusters) may be displayed on the display 17. Information generated on the backend 46 may be transmitted automatically by the ECG platform 37, or the ECG platform 37 may store the information on the server 15 until requested by the ECG application 29. In response to generating data, the ECG platform 37 may transmit a message to the ECG application 29 informing it that data is available from the ECG platform 37. The ECG application 29 receives data (e.g., raw ECG data, preprocessed ECG data, wave information, labels, clusters, and / or morphology, as well as any other data generated between steps 54, 56, 58, 61, 63, and 200, including substeps 202 and / or 204), and may cause the system device 14 to display information and / or data as described herein and in WO2019 / 038435 (whose entire contents are incorporated herein by reference).

[0114] A user of the ECG application 29 may view on the display 17 wave information, markers, seizures, clusters, pulse groupings, and / or morphology, as well as any other data generated between steps 54, 56, 58, 61, and 200. In step 66, a user of the ECG application 29 may use an input device 25 to generate user input data 69 that can be communicated from the ECG application 29 on the front end 45 to the ECG platform 37 on the back end 46, interacting with the two-way display described above. For example, a user may view QRS clusters, PVC clusters, and / or PAC clusters on the display 17. Using the input device 25, a user may interact with the display 17 and generate user input data 69 related to wave information, markers, seizures, clusters, pulse groupings, and / or morphology, as well as any other data presented on the display 17. For example, the user may determine or identify additional clusters and / or pulse groups, identify, associate, and / or merge similar clusters and / or pulse groups, delete inaccurate, irrelevant, or mislabeled clusters and / or pulse groups, and / or split clusters and / or pulse groups into different subsets. In addition, user input data 69 may be subject to deletions and / or modifications as described below.

[0115] In response to receiving user input 69, the ECG platform 37 on the backend 46 may, in step 70, cause the recomputer 40 to use the user input data 69 to retrain, correct, and / or adjust the delineator 39, the embedding device 48, and / or the grouping device 49. For example, the user input data 69 may indicate that the grouping data 205 was inaccurate. The recomputer 40 may use this information to retrain and / or adjust the delineator 39, the embedding device 48, and / or the grouping device 49 to generate more accurate grouping data 205. Depending on the retraining of the delineator 39, implantation device 48, and / or grouping device 49, one or more of steps 200, substep 202, and / or substep 204 may be repeated, and the generated information (e.g., group data 205, seizures, and / or clusters or morphology) may be transmitted to the ECG application 29 on the front end 45 as described above, displayed on the display 17, and viewed by the user. The user may adjust and fine-tune the processing system 10' in a different way by recalculating the group data, seizures, and / or clusters or morphology and presenting the same content to the user via the display 17. For example, more accurate seizures may be identified, and the quality of wave information 201, implantation data 203, and / or group data 205 may be improved.

[0116] Referring here to Figure 17, the process of analyzing ECG data and grouping similar pulses is illustrated. This process generally involves receiving patient ECG data in step 211. This may also involve receiving raw ECG data 52 in the ECG platform 37. The raw ECG data may optionally be preprocessed as described above (e.g., to remove noise). In response to receiving the ECG data, in step 212, the ECG data may be analyzed and / or processed using at least one drawing algorithm to generate wave information. The embedding algorithm may be one or more algorithms and / or a neural network. The generated wave information may be wave information 201. In step 213, pulse onset and disappearance may be determined from the wave information. Based on pulse onset and disappearance, a portion of the ECG data starting at the point corresponding to disappearance and ending at the point corresponding to onset may be extracted from the ECG data and associated with that particular pulse. Multiple pulse portions of the ECG data may be extracted in step 213. In step 214, the portion of the ECG data corresponding to the pulse may be analyzed and / or processed using an embedding algorithm so that embedding data is generated. The embedding data may be embedding data 203. The embedding algorithm may be one or more algorithms and / or a neural network. The embedding data generated by the embedding algorithm may represent a portion of the ECG data corresponding to the pulse.

[0117] In step 215, the embedding data may be analyzed and / or processed using a grouping algorithm to generate group data. The group data may be group data 205. The grouping algorithm may be one or more algorithms (e.g., kmeans, dbscan) and / or a trained neural network. The grouping algorithm is preferably trained from multiple ECG datasets and / or portions of ECG data corresponding to various pulses. In step 216, the group data may be analyzed to determine whether two or more pulses are similar (e.g., share certain features and / or properties). For example, the group data values ​​may be compared to a threshold range to determine the similarity between pulses. If the group data values ​​indicate similarity between pulses (e.g., the group data values ​​meet the threshold), the similar pulses may be grouped together.

[0118] In step 217, ECG data, wave information, labels, embedding data, group data, pulse groupings, similar pulses, clusters, seizures, morphology information, and / or other information based on or indicating any of the aforementioned (e.g., analyzed ECG data) may be transmitted from the ECG platform 37 to the ECG application 29 for display. Some or all of the analyzed ECG data may be displayed on a graphic user interface. For example, a graphic user interface similar to the bidirectional display 220 illustrated in Figure 18 may display the morphology 240 and other analyzed ECG data. The graphic user interface may include pulse overlay images of sample pulses within its morphology, as illustrated in Figure 20B.

[0119] In an optional step 218, the ECG platform 37 may receive a request to assign the analyzed ECG data to one or more user accounts associated with one or more healthcare professionals, and the ECG platform 37 may assign the analyzed ECG data to those user accounts. In addition to assigning the analyzed ECG data to healthcare professionals (e.g., technicians, cardiologists, etc.), the system may also inform the healthcare professionals that the ECG data has been assigned to them for further review. For example, a message and / or alert may be sent to the healthcare professionals. The analyzed ECG data may also be unassigned and / or reassigned. It should be understood that the optional step 218 may, as an alternative, occur after or between steps 219, 220, and / or 221, or at any other point. In an optional step 219, the analyzed ECG data may be submitted for quality control, the request for quality control may be received by the ECG platform 37, and / or the analyzed ECG data may be placed in a quality control queue for review by one or more quality inspectors (e.g., engineers).

[0120] In an optional step 220, the ECG platform 37 may receive quality control inputs from one or more quality inspectors (e.g., engineers). The quality control inputs may include portions of the analyzed ECG data that have been deleted and / or modified. In an optional step 221, the ECG platform 37 may transmit the quality control inputs to the ECG application 29 for display. The ECG application may display the analyzed ECG data that has been selected for deletion and / or modification. Steps 220 and / or 221 may be repeated multiple times if multiple quality control inspectors are inspecting the analyzed ECG data. For example, a first quality control inspector may indicate that the analyzed ECG data requires further inspection, and a second quality control inspector may analyze the analyzed ECG data and / or quality control inputs generated during the first quality control inspection.

[0121] In step 222, the ECG platform 37 may receive user input data from the ECG application 29. For example, a healthcare professional (e.g., an assigned healthcare professional) may scrutinize the analyzed ECG data and / or quality control inputs and generate user input data regarding the analyzed ECG data and / or quality control inputs (e.g., regarding the inaccuracy of the analyzed ECG data). The user input data may involve the deletion and / or modification of some or all of the analyzed ECG data. The user input data may be generated by the input device 25 of the system device 14 and transmitted from the ECG application 29 to the ECG platform 37. For example, the user input data may correspond to the accuracy of the embedded data and / or group data, or to the accuracy of the graphic user interface. As described below with respect to Figure 20A, the user input data may correspond to deleted forms. For example, after a clinician scrutinizes a displayed form, if a healthcare provider or technician determines that the sample pulses in that form are normal and not abnormal, the clinician may use the input device 25 to delete that form.

[0122] In an optional step 223, the received user input data and / or quality control input may be used (e.g., using the recomputer 40) to train and / or modify the drawing algorithm, classification algorithm, grouping algorithm, and / or embedding algorithm. For example, user input data may inform the ECG platform 37 that certain seizures and / or pulse groups should be removed and / or otherwise inaccurate. This information may be used to retrain or otherwise modify the drawing algorithm, classification algorithm, grouping algorithm, and / or embedding algorithm to improve the quality of the wave information, marking, embedding data, and / or group data generated by these algorithms.

[0123] In an optional step 224, the ECG data (e.g., preprocessed ECG data) may be reanalyzed and / or recalculated based on a modified drawing algorithm, classification algorithm, grouping algorithm, and / or embedding algorithm to generate reanalyzed ECG data (e.g., wave information, markers, embedding data, group data, pulse grouping, similar pulses, clusters, seizures, morphology, and / or other information based on or indicating any of the foregoing). In step 225, the reanalyzed ECG data and / or other information indicating or based on the reanalyzed ECG data may be transmitted from the ECG platform 37 to the ECG application 29 for display. The data and / or information may be displayed on a graphical user interface. For example, the graphical user interface may be updated to remove morphology as instructed in the user input data. In addition, or alternatively, the recalculated seizures or clusters may be displayed.

[0124] Referring here to Figure 18, a bidirectional display 220 is illustrated. The bidirectional display 220 includes a first graphic window 221 similar to the first graphic window 124 and a second graphic 222 similar to the second graphic window 125. The bidirectional display 220 may also include a report creation button 225 for generating a report. The first graphic window 221 may illustrate multiple pulses over a certain time period and may include a cursor 224 similar to the cursor 136. The second graphic window 222 may include a zoomed-in view of the region of multiple pulses selected by the cursor 224. The bidirectional display 220 may further include a third graphic window 223 which may include analysis within the first graphic window 221 or additional information about the multiple pulses. For example, the third graphic window 223 may illustrate one or more forms, each containing multiple pulse strips matched together or grouped differently based on some similarity, as described above. In some embodiments, a pulse strip is a series of pulses in time. Generally, multiple pulse strips, either matched together or grouped separately, are from different time intervals and are not a series of pulse strips in time.

[0125] Referring here to Figure 19, a third graphic window 223 is illustrated. As shown, the third graphic window 223 may include a category tab 231 for selecting information to be displayed in the third graphic window 223, such as morphological analysis, ventricular activity, or any other category listed within the navigation window 259 (shown in Figure 21). The third graphic window 223 may also further include subtabs 232 for further refining the category tab 231. For example, the subtabs 232 may include types of abnormalities such as PVC and PSVC, or general tabs such as "Other pulses". When the "Morphological Analysis" window tab is selected, the morphs 233 may be viewed using the two-way display 220. As described above, each morph of the morphs 233 may include a group of pulse strips grouped together based on group data. For example, each morph may include pulses having one or more similar characteristics or qualities.

[0126] Referring here to Figure 20A, an example of one of the forms 233, form 240, is illustrated. Form 240 may include a form name 241, a form navigator 242, a pulse overlay 243, a button to add to report 244, various information 245, a time identifier 246, a report button 247, a delete button 248, and a pulse window 249. The form name 241 may identify the form. The form navigator 242 may be used to select pulse strips within form 240 to be viewed in the pulse window 249. The pulse strips that can be displayed in the form navigator 242 may be a subset of pulse strips from all pulse strips within that form. The subset of pulse strips may be randomly selected from all pulse strips, or, in a preferred embodiment, selected based on the pulse strips within that form that have the greatest differences from each other. For example, a subset may be generated by comparing group data and / or embedded data and determining pulse strips within that morphology that have vector and / or matrix values ​​with the greatest difference from each other. In this way, the morphology navigator 242 displays pulse strips that are relevant with respect to the morphology based on the score. The Add to Report button 244 may be used to add morphology 240 to a report to be generated by the ECG platform 37 using the Report Generation button 225. The Various Information 245 may include some information about the patient or morphology. For example, the Various Information 245 may identify that the patient has a pacemaker. The Time Identifier 246 may identify the time at a point in the pulse window 249. For example, the user may place the cursor over points in the pulse window 249, and the Time Identifier 246 may identify the corresponding time. The Report button 247 may be used to open a report or report template. The Delete button 248 may delete morphology 240. For example, after examining morphology 240, a healthcare provider or technician may determine that the pulses within the morphology are not similar, are normal, and / or should not be grouped together in any other way, and therefore may decide to delete the morphology.As explained above with respect to Figures 16 and 17, removing the shapes may modify the graphic user interface (e.g., the bidirectional display 220) and / or modify or otherwise alter the grouping algorithm to retrain the neural network.

[0127] Referring here to Figure 20B, the pulse overlay 243 is enlarged. As shown, the pulse overlay 243 may display some or all similar pulses within morphology 240 on a single plot. For example, the pulse overlay 243 may display a single pulse from each of the pulse strips of morphology navigator 242 that are overlaid across each other. In a preferred embodiment, the pulses displayed in the pulse overlay 243 from each of the pulse strips are pulses that the system identifies as abnormal in the individual pulse strips. The ECG platform 37 may match the pulses according to wave or plot features (P wave, T wave, QRS complex). A healthcare provider or technician may use the pulse overlay 243 to determine whether the morphology is abnormal, similar, and / or precisely categorized within the appropriate subtab. If a healthcare provider or technician determines that the pulses within a morph are not similar, are normal, and / or should not be grouped together in any other way, the healthcare provider or technician may delete the morph using the delete button 248 as described above.

[0128] Referring here to Figure 21, a bidirectional display 250 is illustrated. The bidirectional display 250 includes a first graphic window 251 similar to the first graphic window 221 and a second graphic 252 similar to the second graphic window 222. The first graphic window 251 may illustrate multiple pulses over a certain time period and may include a cursor 254 similar to the cursor 224. The second graphic window 252 may include a zoomed-in view of a region of multiple pulses selected by the cursor 254. The bidirectional display 250 may further include a third graphic window 253 illustrating one or more categories. The bidirectional display 250 may also include a navigation window 254 and a report creation button 255 for generating a report. The navigation window 259 may be used to select various categories to be viewed within the third graphic window 253. For example, the navigation window 259 may be used to select categories, but are not limited to, maximum HR, minimum HR, AV block, other pulses, morphological analysis such as PSVC, other supraventricular activity such as SVT, ventricular activity such as VT, and / or sinus.

[0129] The bidirectional display 250 may also display an ectopic interval among a plurality of pulses contained within the first graphic window 251. An ectopic interval is three or more abnormal pulses that are continuous in time. Thus, the system described herein may detect an ectopic interval when three or more abnormal pulses in a row are identified, for example, using the identifier described above. The first graphic window 251 may identify an ectopic interval using an identification bar 256 at the point of the ectopic interval. The second graphic window 252 may identify the area of ​​the ectopic interval using an ectopic section 257 which may be a different color from the second graphic window 252. Each ectopic pulse within the ectopic section 257 may be identified using an ectopic marker 258 that indicates the precise location of each ectopic pulse.

[0130] A report may be created with respect to ECG data illustrated in the two-way display 250 by using the report creation button 255 (similar to the report creation button 225 in Figure 18). As described above, a healthcare provider or technician using the two-way display 250 may select a morphology, or various other pulse strips, or a group of pulse strips, displayed using the navigation window 259, to include in the report. Figures 22A–22P illustrate report 260, which is an exemplary report that may be generated using the ECG platform 37 and the ECG application 29.

[0131] Referring here to Figure 22A, report 260 may include a cover page 261 which includes information about the patient, information about the healthcare provider or technician, a report summary, information about the patient's heart rate (e.g., maximum, minimum, mean), ectopic information (e.g., supraventricular premature contractions, ventricular premature contractions), information about patient events, information about paced pulse, and a findings section which identifies that the report includes the most prominent findings, outcomes, and / or analyses. Cover page 261 may further include prominent pulse strips corresponding to the patient's ECG data. For example, prominent pulse strips may include, but are not limited to, atrial fibrillation / flutter, other SVT, pause, AV block, and VT.

[0132] Referring here to Figure 22B, the report 260 may include a heart rate trend display 262 that can illustrate the patient's heart rate over a certain time period (e.g., several hours). The heart rate trend display 261 may also include one or more anomaly identifiers 263 for identifying abnormalities in the pulse displayed over time. The heart rate trend display may also identify PTE and / or atrial fibrillation / flutter. Referring here to Figure 22C, the report 260 may include an atrial fibrillation / flutter display 264 that can display the patient's heart rate over a certain time period (e.g., several hours). The atrial fibrillation / flutter display 264 may be used to display the amount of time the patient is suffering from atrial fibrillation and / or flutter. The atrial fibrillation / flutter display 264 may further display the burden percentage, the length of the longest episode, and the maximum heart rate.

[0133] Referring here to Figure 22D, report 260 may include a PSVC display 265 that can display the patient's heart rate over a time period (e.g., several hours) and can be used to display the time the patient is experiencing a PSVC. ​​PSVC display 265 may further display the PSVC value and the number and / or percentage of caplets. Referring here to Figure 22E, report 260 may include a PVC display 266 that can display the patient's heart rate over a time period (e.g., several hours) and can be used to display the time the patient is experiencing a PVC. PVC display 266 may further display the PVC value and / or percentage, the number of forms associated with the PVC, the number and / or percentage of caplets, the bigeminythm and / or percentage, and the trigeminythm and / or percentage.

[0134] Referring here to Figure 22F, report 260 may include a patient symptom display 267 that can identify the patient symptoms detected and may include the total number of patient events and the number of symptomatic patient events. The patient symptom display 267 may be divided into palpitations, chest pain, dizziness, syncope, and other symptoms. Referring here to Figure 22G, report 260 may include a heart rate variability display 268 and a QT analysis display 269. The heart rate variability display 268 may identify the heart rate distribution and variability and display heart rate parameters such as mean, SDNN, SDANN, ASDNN, NN50, pNN50, RMSSD, VLF, LF, and HF. The QT analysis display 269 may analyze and display the QT analysis over time and identify parameters such as minimum QT, mean QT, maximum QT, minimum QTcB, mean QTcB, maximum QTcB, and QTcB > 450 milliseconds.

[0135] Referring here to Figure 22H, report 260 may include a strip index 271. The strip index 271 may include parameters such as ID, date and time, category, heart rate, patient event, comments, and page. The categories may include items such as maximum HR, minimum HR, sinus, atrial fibrillation / flutter, VT, patient event, PVC, ventricular caplet, ventricular bigeminy, PSVC, supraventricular caplet, etc. The index strip 271 may be used to find the report page number from which a pulse strip displaying a particular category can be found.

[0136] Referring here to Figure 22I, report 260 may include pulse strip displays such as maximum HR display 272, minimum HR display 273, and sinus display 274. Maximum HR display 272 may display a pulse strip with the maximum heart rate. Minimum HR display 273 may display a pulse strip with the minimum heart rate. Sinus display 274 may display a pulse strip with sinus abnormality. Referring here to Figure 22J, report 260 may include pulse strip displays such as sinus display 274 and atrial fibrillation / flutter display 275. Atrial fibrillation / flutter display 275 may display a pulse strip indicating atrial fibrillation and / or flutter. Referring here to Figure 22K, report 260 may include pulse strips such as atrial fibrillation / flutter display 275 and VT display 276. VT display 276 may display a pulse strip indicating VT. Referring here to Figure 22L, report 260 may include pulse strips such as VT indicator 276, patient event indicator 277, and PVC indicator 278. Patient event indicator 277 may display a pulse strip indicating a patient event. PVC indicator 278 may display a pulse strip indicating PVC. Figure 22M illustrates an additional PVC indicator 278. Referring here to Figure 22N, report 260 may include pulse strips such as ventricular caplets 279 and ventricular bigeminy 281. Ventricular caplets 279 may display a pulse strip indicating one or more ventricular caplets. Ventricular bigeminy 281 may display a pulse strip indicating ventricular bigeminy. Referring here to Figure 22O, report 260 may include pulse strips such as PSVC indicator 282 and supraventricular caplet indicator 283. PSVC indicator 272 may display a pulse strip indicating PSVC. The supraventricular caplet display 283 may display pulse strips indicating one or more supraventricular caplets. Figure 22P also illustrates the supraventricular caplet display 283.

[0137] Referring here to Figures 23A-23D, exemplary processes and user interfaces are illustrated for assigning analyzed ECG data for scrutiny. For example, Figures 23A-23D illustrate exemplary interfaces and processes for assigning, deassigning, and / or reassigning responsibility for scrutinizing an ECG dataset and / or analyzed ECG data to one or more healthcare professionals.

[0138] Referring here to Figure 23A-B, the graphical user interface displayed via the ECG application 29 (for example, on system device 14) may display a pending review window 312 which can be used to assign ECG data (e.g., a file) to one or more healthcare professionals. The pending review window 312 may include a data status identifier 304 which may correspond to ECG data, and / or wave information, labeling, group data, embedding data, classification data, depiction data, clusters, analyzed data, and / or any other data based on or corresponding to the ECG data. The data status identifier 304 may also indicate whether the corresponding data has been viewed and by whom it has been viewed. The data status identifier 304 may also indicate, where applicable, the healthcare professional to whom the data has been assigned. For example, the data status identifier 304 may include a user or data identifier, a duration value corresponding to the duration of the ECG data (e.g., 1 day), the upload date (e.g., corresponding to the date the data was uploaded to the ECG platform 37), a scrutiny status corresponding to the scrutiny status (e.g., returned for analysis, completed, quality control pending), and / or information about the person who submitted the corresponding data for quality control or scrutiny, and / or when the data was last modified. If no status exists, the data and / or analysis may be uploaded and ready for analysis. If the status is "quality control pending," the data and / or analysis may be submitted for quality control and ready for scrutiny. If the status is "returned for analysis," the data and / or analysis may have been scrutinized but still require further scrutiny. If the status is "completed," the data and / or analysis may have been scrutinized and may be considered acceptable.

[0139] The data status identifier 304 may include an assignment menu 341 for assigning and unassigning the data status identifier 304 to one or more user accounts associated with a healthcare professional. As shown in Figures 23A-B, the assignment menu 341 may allow a user to enter the name of a healthcare professional to assign the data status identifier 304, and therefore the corresponding data, to the identified healthcare professional. Alternatively, or in addition, the assignment menu 341 may enumerate one or more names of healthcare professionals that may be selected to assign that healthcare professional to the ECG data corresponding to the data status identifier 304. The assignment menu 341 may also be used to unassign an assigned data status identifier 304. The assignment menu 341 may also enumerate healthcare professionals to whom the data status identifier may be reassigned.

[0140] Referring here to Figure 23C, the pending review window 312 may include an organization bar 351 and a data selector 352. The data selector 352 may be involved in selecting one or more data status identifiers 304. The organization bar 351 may be used to organize and / or assign the data status identifiers 304. For example, a change button 353 on the organization bar 351 may be involved in assigning, unassigning, and / or reassigning all selected data status identifiers to a particular healthcare professional. In this way, multiple data status identifiers 304 may be assigned, unassigned, and / or reassigned simultaneously.

[0141] Referring here to Figure 23D, the pending review window 312 may include a filter button 361. The filter button 361 may be used to filter one or more data status identifiers 304. As shown in Figure 23D, the filter button 361 may be used by a healthcare professional to filter data status identifiers 304 within the pending review window 312. For example, if healthcare professional "Stan Dupp" is selected, only the data status identifiers 304 assigned to Stan Dupp will be displayed in the pending review window 312. It should be understood that other filters (date, patient, quality review status, etc.) may also be selected.

[0142] Referring here to Figures 24A-F, an exemplary process and user interface are illustrated for submitting ECG data and / or analyzed ECG data for quality review by one or more quality reviewers (e.g., engineers). As shown in Figures 24A-B, the toolbar 401 may be incorporated into the user interface generated by the ECG application 29 and / or the ECG platform 37 and may be used (e.g., using the quality control button 402) to submit ECG data and / or analysis of ECG data for quality control (QC). For example, the ECG platform 37 may generate group data, embedding data, classification data, depiction data, and / or analyzed ECG data, and the quality control button 402 may be involved in submitting the aforementioned data for quality control.

[0143] Referring here to Figure 24B, interface 403 is illustrated. Interface 403 displays navigation columns 406, which may include, for example, a Worklist Interface tab, a Quality Control tab 405, Archives, and Favorites tabs. Interface 403 may be used, for example, by a Quality Control reviewer. When the Quality Control tab 405 is involved, one or more data status identifiers 404 may be displayed. The data status identifier 404 may be identical to the data status identifier 304 and / or may include information about quality control (e.g., who submitted it for quality control, submission date, last modification date, quality control status, etc.).

[0144] Referring here to Figure 24C, the ECG application 29 may cause the system device 14 to display an interface 411 for quality control review. The interface 411 may display a pending window 412 and a review window 413. The pending window 412 and the review window 413 may each contain one or more data status identifiers 404. The pending window 412 may contain one or more data status identifiers corresponding to uploaded data that is pending for quality control review. The reviewed window 413 may contain one or more data status identifiers 404 corresponding to uploaded data that is being reviewed. A quality control reviewer (e.g., initial reviewer, secondary reviewer, etc.) may use the interface 411 to determine data that still needs to be reviewed, and / or otherwise determine whether certain data should be reviewed or has been reviewed.

[0145] A data status identifier 404 may be selected for further quality control analysis of the data associated with the selected data status identifier 404. Depending on the selection of a data status identifier 404, the ECG application 29 may cause the system device 14 to display Figure 24D-E. Figure 24D-E illustrates analysis interfaces 421 and 422, respectively, illustrating the ECG data strips 424 corresponding to the data status identifier 404. The ECG data strips 424 may be organized according to tabs 423 that can identify groups of ECG data strips (e.g., SVTs, couplets, etc.). Analysis interfaces 421 and 422 may be used to view ECG data strips removed during quality control scrutiny, as well as remaining (i.e., not removed) ECG data strips. The removal status 425 indicates whether the ECG data strip has been removed or whether the ECG data strip is a remaining one. For example, user interface 421 illustrates the remaining ECG data strips, and user interface 422 illustrates the removed ECG data strips. In this way, changes made during quality review may be stored in an archive and reviewed. A restore button 426 may be involved in undoing the removal of an ECG data strip so that it is recategorized as a remaining ECG data strip.

[0146] Referring here to Figure 24F, interface 431 may be used to generate a report, which is illustrated and includes quality control information (e.g., pending, passed, returned for analysis). Interface 431 may include a reporting options window 432, a patient information window 433, a clinical information window 434, a results window 435, a quality control window 436, a patient metrics window 437, and / or a window 438 for generating a report. The reporting options window 432 may include several options for generating a report and may further allow the selection of one or more leads for selecting the ECG data to be included in the report. Patient information 433 may include identification and other information corresponding to the patient from whom the ECG data was generated (e.g., name, patient identifier, primary indication, date of birth, presence of pacemaker, sex, etc.). The clinical information window 434 may include clinical information such as the name of the healthcare professional who performed the analysis, the date, and / or the name of the institution. The results window 435 may be used to select a results template and / or metrics. The quality control window 436 may be used to select the status of quality control (e.g., pending, passed, returned for analysis). The patient metrics window 437 may display patient metrics such as the duration of monitoring and / or the time to be analyzed, heart rate information, information about premature contractions (e.g., supraventricular premature contractions, ventricular premature contractions), patient events, patient seizures, and / or pause occurrences.

[0147] It should be understood that any of the operations described herein above may be implemented, at least in part, as computer-readable instructions stored in computer-readable memory. In response to the execution of a computer-readable instruction by a processor, the computer-readable instruction may cause a node to perform an operation. Naturally, it should be understood that the embodiments described herein are illustrative, and the components may be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which have been considered and fall within the scope of this disclosure.

[0148] The foregoing description of illustrative embodiments is provided for illustrative and explanatory purposes only. It is not intended to be comprehensive or limiting with respect to the precise forms disclosed, and modifications and variations may be conceivable in light of the foregoing teachings or obtained from the practice of the disclosed embodiments. The scope of the invention is intended to be defined by the claims.

Claims

1. A computerized system for analyzing a patient's electrocardiogram (ECG) data, wherein the ECG data is generated by one or more electrodes across multiple time points and includes multiple pulses, and the computerized system, Using a depiction algorithm, the ECG data is analyzed to generate wave information corresponding to the possibility of the presence of at least one wave at the multiple time points, Determining pulse onset information and pulse disappearance information for pulses among multiple pulses, where at least one wave exists and it is determined that multiple pulse onsets and pulse disappearances occur. Based on the aforementioned multiple pulse onsets and pulse disappearances, the method involves extracting multiple pulse portions from the ECG data, wherein each pulse portion among the multiple pulse portions of the ECG data corresponds to one of the pulses. Based on multiple pulse portions of the ECG data, it is determined that at least two of the multiple pulses should be grouped together, and that the at least two pulses form a cluster. A computerized system configured to perform the following actions.

2. The computerized system further, Using an embedding algorithm, multiple portions of the ECG data are analyzed to generate embedding data representing the multiple pulses, Using a grouping algorithm, the embedded data is analyzed and group data is generated. It is configured to do the following: The computerized system according to claim 1, wherein at least two of the plurality of pulses are determined to be grouped together based on the group data.

3. The computerized system according to claim 2, wherein the group data corresponds to the distance between two pulses.

4. The computerized system according to claim 2, wherein the drawing algorithm utilizes a first neural network, and the embedding algorithm utilizes a second neural network.

5. The computerized system according to claim 4, wherein the grouping algorithm utilizes a third neural network.

6. The computerized system according to claim 2, further configured to receive user input data from an input device regarding the inaccuracy corresponding to the ECG data.

7. The computerized system according to claim 6, further configured to adjust one or more of the drawing algorithm, embedding algorithm, or grouping algorithm based on the user input data.

8. The computerized system according to claim 7, further configured to modify the displayed data based on the user input data.

9. The computerized system according to claim 8, wherein the user input data corresponds to adding, deleting, or splitting one or more QRS clusters, PVC clusters, or PAC clusters.

10. The computerized system according to claim 7, wherein the embedded data includes a data vector for each pulse among the plurality of pulses.

11. The computerized system according to claim 2, wherein determining that at least two of the plurality of pulses should be grouped together further includes determining that the group data satisfies a threshold.

12. The computerized system according to claim 1, further configured to transmit information indicating the cluster to a computer for display on a graphic user interface.

13. The computerized system according to claim 1, further configured to generate information and display at least one overlay including at least two of the plurality of pulses that are overlaid on each other.

14. The computerized system according to claim 1, further configured to use a classification algorithm to analyze the pulses within the cluster and determine the likelihood of the presence of one or more abnormalities, conditions, or descriptors associated with a cardiac event relating to the patient.

15. The computerized system according to claim 1, further configured to use a classification algorithm to analyze the wave information from the depiction algorithm and determine the likelihood of the presence of one or more abnormalities, conditions, or descriptors associated with a cardiac event relating to the patient.

16. The computerized system according to claim 15, wherein the wave information is input to the classification algorithm and used separately to determine that at least two of the plurality of pulses should be grouped together.

17. The computerized system according to claim 1, further configured to preprocess the ECG data and remove noise from the ECG data prior to analyzing the ECG data using the drawing algorithm.

18. The computerized system according to claim 1, further configured to assign the ECG data and information based on the ECG data to user accounts for scrutiny.

19. The computerized system according to claim 18, further configured to receive user input data relating to the ECG data and information based on the ECG data from the user account based on the scrutiny.

20. A method for analyzing electrocardiogram (ECG) data of a patient, wherein the ECG data is generated by one or more electrodes across multiple time points and includes multiple pulses, and the method is Using a depiction algorithm, the ECG data is analyzed to generate wave information corresponding to the possibility of the presence of at least one wave at the multiple time points, Determining pulse onset information and pulse disappearance information for pulses among multiple pulses, where at least one wave exists and it is determined that multiple pulse onsets and pulse disappearances occur. Based on the aforementioned multiple pulse onsets and pulse disappearances, the method involves extracting multiple pulse portions from the ECG data, wherein each pulse portion among the multiple pulse portions of the ECG data corresponds to one of the pulses. Based on multiple pulse portions of the ECG data, it is determined that at least two of the multiple pulses should be grouped together, and that the at least two pulses form a cluster. Methods that include...

21. Using an embedding algorithm, multiple portions of the ECG data are analyzed to generate embedding data representing the multiple pulses, Using a grouping algorithm, the embedded data is analyzed and group data is generated. It further includes, The method according to claim 20, wherein at least two of the plurality of pulses are determined to be grouped together based on the group data.

22. The method according to claim 20, further comprising assigning the ECG data and information based on the ECG data to a user account for the purpose of scrutinizing the ECG data.

23. The method according to claim 20, further comprising submitting the ECG data and information based on the ECG data for quality review by one or more reviewers.

24. The method according to claim 23, further comprising receiving quality control inputs generated by one or more of the aforementioned inspectors.

25. The method according to claim 24, further comprising causing a display of the quality control input for additional quality control scrutiny.

26. The method according to claim 20, further comprising receiving user input data from an input device regarding the inaccuracy corresponding to the ECG data.

27. The method according to claim 20, further comprising adjusting one or more of the drawing algorithm, embedding algorithm, or grouping algorithm based on the user input data.

28. A system for analyzing a patient's electrocardiogram (ECG) data, wherein the system is A first set of instructions, configured to acquire ECG data of the patient over multiple points in time when executed, wherein the ECG data is sampled at a rate of at least 20 samples per second, and the first set of instructions is further configured to cause the transmission of the ECG data to at least one server when executed. When executed, at least one of the servers will Receiving the ECG data of the aforementioned patient, Analyzing the patient's ECG data using at least one algorithm trained on multiple ECG datasets from different patients, Based on the analysis of the aforementioned ECG data, the probability of the presence of one or more anomalies, conditions, or descriptors, or any combination thereof, is quantified. To display, information corresponding to the presence of one or more anomalies, conditions, or descriptors, or any combination thereof, is transmitted from the at least one server to the computer. A second set of instructions configured to perform the following actions: A third set of instructions, when executed by the computer, is configured to cause the computer to display information corresponding to the presence of one or more anomalies, conditions, or descriptors, or any combination thereof, based on the information transmitted from the at least one server. A system equipped with these features.

29. The system according to claim 28, wherein the second set of instructions, when executed, are configured to cause at least one server to preprocess the ECG data, which involves at least one of removing noise from the ECG data or representing the ECG data at a pre-determined reference frequency.

30. The system according to claim 28, wherein the second plurality of instructions, when executed, are configured to analyze the patient's ECG data using at least one algorithm that applies the ECG data to a first neural network for depiction.

31. The system according to claim 30, wherein analyzing the patient's ECG data using at least one algorithm that applies the ECG data to a first neural network generates wave information.

32. The system according to claim 31, wherein the second set of instructions, when executed, are configured to analyze the wave information using one or more clustering algorithms and to identify one or more clusters corresponding to the wave information.

33. The system according to claim 32, wherein the one or more clusters include one or more ventricular premature contraction (PVC) clusters or atrial premature contraction (PAC) clusters.

34. The system according to claim 30, wherein, when the second set of instructions is executed, it quantifies the probability of the presence of at least one of a P wave, a QRS complex, or a T wave at each of the set of instructions.

35. The system according to claim 34, wherein the second set of instructions, when executed, are configured to calculate at least one occurrence and at least one disappearance with respect to at least one of the P wave, QRS complex, or T wave.

36. The system according to claim 35, wherein the second set of instructions, when executed, are further configured to calculate at least one measurement from one or more of the emergences, disappearances, or outputs of the first neural network.

37. The system according to claim 28, wherein the second set of instructions, when executed, are configured to analyze the patient's ECG data using at least one algorithm that applies the ECG data to a second neural network for classification.

38. The system according to claim 37, wherein the second set of instructions, when executed, quantify the likelihood of the presence of one or more anomalies, conditions, or descriptors.

39. The system according to claim 38, wherein the second plurality of instructions, when executed, are configured to apply a threshold to at least one value in the output of the second neural network, and if the value exceeds the threshold, to assign at least one marker corresponding to the one or more anomalies, conditions, or descriptors.

40. The system according to claim 39, wherein the second set of instructions is further configured to post-process the ECG data by removing redundant indicators when executed.

41. The system according to claim 40, wherein the computer that executes the third plurality of instructions is configured to execute the first plurality of instructions.

42. The system according to claim 38, further comprising a fourth set of instructions, which, when executed, are configured to cause at least one server to generate a report containing at least the transmitted information corresponding to the presence of one or more anomalies, conditions, or descriptors.

43. The system according to claim 42, further comprising a fifth set of instructions, which, when executed, are configured to receive user input related to the ECG data and to cause the computer to transmit the user input to the at least one server so that the at least one server uses the user input and generates the report.

44. The system according to claim 42, wherein the report includes at least one heart rate density plot representing the patient's heart rate density as a function of time.

45. The system according to claim 38, wherein the third set of instructions, when executed by the computer, is configured to cause the computer to display at least one heart rate density plot representing the patient's heart rate density as a function of time.

46. The system according to claim 38, wherein each set of the plurality of ECG datasets from different patients is generated at a sampling rate equal to the rate used to acquire the ECG data.

47. The system according to claim 38, wherein the second set of instructions is further configured to generate a set of wave information corresponding to a set of pulses in the ECG data.

48. The system according to claim 47, wherein the second set of instructions is further configured to cause the at least one server to analyze the set of wave information using an embedding algorithm and generate a set of embedding data corresponding to a comparison of two or more pulses among the set of pulses.

49. The system according to claim 48, wherein each of the plurality of wave information is a vector.

50. The system according to claim 48, wherein each of the plurality of embedded data is a vector.

51. The system according to claim 48, wherein the embedding algorithm is trained from a plurality of ECG datasets.

52. The system according to claim 48, wherein the second set of instructions further causes the set of embedded data to be analyzed by a grouping algorithm to generate group data, the group data matching at least two of the set of pulses.

53. The system according to claim 52, wherein the second plurality of instructions are further configured to generate a graphic user interface based on the group data.

54. The system according to claim 53, wherein the second plurality of instructions further cause the at least one server to transmit the graphic user interface to the computer.

55. The system according to claim 53, wherein the graphic user interface comprises at least one pulse overlay, which includes at least two of the plurality of pulses in the ECG data that are overlaid on each other.

56. A system for analyzing a patient's electrocardiogram (ECG) data, wherein the system includes instructions stored on at least one server, and when an instruction is executed, the at least one server, Receiving a set of ECG data from the patient at multiple points in time, wherein the set of ECG data is sampled at a rate of at least 20 samples per second, Analyzing a set of ECG data from a patient using at least one algorithm trained with multiple sets of ECG data generated from different patients at a sampling rate of at least 20 samples per second, At each of the aforementioned multiple time points, the probability of the presence of one or more anomalies, conditions, or descriptors, or any combination thereof, based on the analysis of the set of ECG data, To display, information corresponding to the possibility of the existence of one or more anomalies, conditions, or descriptors, or any combination thereof, is transmitted to the computer. A system configured to perform a certain action.

57. A computerized method for analyzing a patient's electrocardiogram (ECG) data, wherein the method is: Receiving a set of ECG data from the patient at multiple points in time, wherein the set of ECG data is sampled at a sample rate, Analyzing a set of ECG data from a patient using at least one algorithm trained on multiple sets of ECG data, wherein each set of ECG data is generated from a different patient at the given sample rate. At each point in time, based on the analysis of the set of ECG data, one or more anomalies, conditions, or descriptors, or any combination thereof, are identified. To display, information including one or more anomalies, conditions, or descriptors, or any combination thereof, is transmitted to a computer. Methods that include...

58. The computerized method according to claim 57, wherein analyzing the set of ECG data of the patient includes analyzing the entire sampled set of ECG data without discarding any data from the set of ECG data.

59. The computerization method according to claim 57, wherein the sample rate is at least 20 samples per second.

60. The computerization method according to claim 57, further comprising assigning a set of ECG data and information based on the set of ECG data to a user account for the purpose of examining the ECG data.

61. The computerized method according to claim 57, further comprising submitting a set of ECG data and information based on the set of ECG data for quality review by one or more reviewers.

62. The computerized method according to claim 61, further comprising receiving quality control inputs generated by the one or more examiners.

63. The computerized method according to claim 62, further comprising causing a display of the quality control input for additional quality control scrutiny.

Images