Electrocardiogram processing system for detecting and / or predicting cardiac events

Abstract

Systems and methods are provided for analyzing a patient's electrocardiogram (ECG) data using large amounts of ECG data. The system receives ECG data from a sensing device positioned on the patient, such as one or more ECG leads / electrodes that can be integrated into a smart device. The system may include an application communicating with an ECG platform running on one or more servers, which processes and analyzes the ECG data, for example using neural networks, to detect and / or predict various abnormalities, conditions, and / or descriptors. The system may also determine confidence scores corresponding to the abnormalities, conditions, and / or descriptors. The processed ECG data is used to generate a graphical user interface, which is transmitted from the one or more servers to a computer for display in a user-friendly and interactive manner with enhanced accuracy.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to European Patent Application No. EP20306566.9, filed December 15, 2020, and U.S. Provisional Patent Application Serial No. US 63 / 063899, filed August 10, 2020, the entire contents of each of which are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to electrocardiogram (ECG) processing systems, such as ECG systems with artificial intelligence and machine learning capabilities, for detecting and / or predicting cardiac events such as arrhythmias and abnormalities. Background Technology

[0004] An electrocardiogram (ECG) receives electrical signals from the heart, which can be digitized and recorded by a computing device. An ECG is typically generated based on heart signals sensed by multiple electrodes placed in specific areas on the patient. It is a simple, non-invasive tool that can be used by most healthcare professionals.

[0005] Cardiac signals consist of one or more synchronized time signals. Figure 1A illustrates a recording of a standard 12-lead resting ECG. As shown in Figure 1A, each lead generates one electrical signal, resulting in 12 electrical signals. While the ECG illustrated in Figure 1A involves 12 leads, resulting in 12 recordings, some ECGs may involve fewer leads, resulting in fewer recordings. As shown in Figure 1A, cardiac signal displays typically include repeating patterns of P waves, QRS complexes, and T waves. As the name suggests, QRS complexes include Q waves, R waves, and S waves. Figure 1B illustrates exemplary P waves, QRS complexes, and T waves, focusing on a pair of beats in a single lead signal, showing an RR interval.

[0006] To make a diagnosis, trained healthcare professionals can analyze ECG records to identify any abnormalities and / or disease onset. It is estimated that approximately 150 measurable abnormalities can be identified on ECG records today. However, identifying abnormalities from an ECG requires specific expertise and / or training. ECG analysis should only be available to patients who can afford the fees of healthcare professionals with the appropriate expertise and who have access to such professionals through other means.

[0007] Telecardiology centers have been developed to provide ECG analysis to patients who may not otherwise have access to these trained healthcare professionals. Typically, ECG records are generated off-site by non-experts and sent to the telecardiology center for analysis by a cardiologist or a specialized ECG technician. While the results are generally high-quality, the process can be slow and expensive.

[0008] Software systems have also been developed as an alternative to analysis by trained professionals. Current software systems provide low-quality interpretations, which often result in false positives. Today, these interpretation systems can generate two types of information about cardiac signals: (1) temporal location information for each wave, called delineation, and (2) global information that provides classification of cardiac signals or marks their abnormalities, called classification.

[0009] Regarding delineation, two main schemes are used to find the wavelets in the cardiac signal. The first scheme is based on multi-scale wavelet analysis. This scheme searches for wavelet coefficients that reach a predefined threshold at a specified scale. (See: Martinez et al., A wavelet-based ECG delineator: evaluation on standard databases, IEEE Transactions on Biomedical Engineering, Vol. 51, No. 4, April 2004, pp. 570-58; Almeida et al., IEEE Transactions on Biomedical Engineering, Vol. 56, No. 8, August 2009, pp. 1996-2005; Boichat et al., Proceedings of Wearable and Implantable BodySensor Networks, 2009, pp. 256-261; US ​​Patent No. 8903479 to Zoicas et al.). The typical process involves identifying the QRS complex, then the P wave, and finally the T wave. This scheme is unstable due to the use of thresholds, and it fails to identify multiple P waves and “hidden” P waves.

[0010] The second delimitation scheme is based on Hidden Markov Models (HMMs). This machine learning scheme treats the current state of the signal as a hidden variable that one wants to recover (Coast et al., IEEE Transactions on Biomedical Engineering, Vol. 37, No. 9, September 1990, pp. 826-836; Hughes et al., Proceedings of Neural Information Processing Systems, 2004, pp. 611-618; US Patent 8332017 to Trassenko et al.). While this scheme is an improvement on the first delimitation scheme described above, it requires the use of handcrafted “features” to design the signal representation, and a mathematical model must be fitted for each wave based on these features. Based on a sufficient number of examples, the algorithm can learn to identify each wave. However, this process can be cumbersome and inaccurate due to its dependence on handcrafted features. Specifically, the handcrafted features will always be suboptimal because they are not learned, and the process of handcrafting features may ignore or eliminate crucial information. Furthermore, the model (usually a Gaussian model) does not tune well. Furthermore, the current model fails to account for hidden P waves.

[0011] Regarding classification, in the current system, analysis is performed only on QRS complexes. For example, analysis of QRS complexes can detect ventricular or pacemaker beats. Training involves manually crafting a set of features and corresponding beat labels (Chazal et al., IEEE Transactions on Biomedical Engineering, 2004, Vol. 51, pp. 1196-1206). As explained above, manually crafted features will always be suboptimal because they are not learned, and the process of crafting features may ignore or eliminate crucial information.

[0012] To address these issues, recent work (Kiranyaz et al., IEEE Transactions on Biomedical Engineering, 2016, Vol. 63, pp. 664-675) has turned to a novel architecture called neural networks, which have been extensively studied and have achieved significant results in the field of imaging (Russakovsky, arXiv: 1409.0575v3, January 30, 2015). Neural networks learn from raw or moderately preprocessed data, thus bypassing the need for handcrafted features. While the application of neural networks represents an improvement over the aforementioned delineation and classification schemes, existing systems have certain limitations. For example, current neural networks have only been developed for QRS representation. Furthermore, current neural networks process information in a pulsatile manner, which fails to capture contextual information from surrounding pulsations.

[0013] Regarding the identification of abnormalities and / or cardiovascular disease detection, most algorithms use rules based on time and morphological indicators, which are calculated using defined parameters (e.g., PR interval, RR interval, QT interval, QRS width, ST segment level, T wave slope). Often, these algorithms are designed by cardiologists (Prineas et al., The MinnesotaCode Manual of Electrocardiographic Findings, Springer, ISBN 978-1-84882-777-6, 2009). However, current algorithms do not reflect how cardiologists analyze ECGs and are coarse simplifications. For example, the University of Glasgow's algorithm does not reflect how cardiologists analyze ECGs (Statement of Validation and Accuracy for the Glasgow 12-Lead ECG Analysis Program, Physio Control, 2009).

[0014] More advanced methods using learning algorithms have also been developed. For example, in Shen et al., Biomedical Engineering and Informatics (BMEI), 2010, Vol. 3, pp. 960-964, the authors used support vector machines to detect bundle branch block. However, in these methods, again, it is necessary to represent the original data in a way that preserves invariance and stability properties.

[0015] While more complex neural network architectures have been proposed, limitations have emerged when applying them to ECG. A team (Jin and Dong, Science China Press, Vol. 45, No. 3, 2015, pp. 398-416; CN104970789) proposed a binary classification for the complete ECG, thus providing one and only one classification for any analyzed ECG. The proposed architecture uses convolutional layers that process the leads independently before mixing them into fully connected layers. The authors also mention multi-class analysis (as opposed to binary analysis), which aims to recover one of several classes. However, they did not consider multi-label classification, where multiple labels (e.g., abnormality) are assigned to cardiac signals.

[0016] Other algorithms and neural network architectures have been proposed to detect the risk of atrial fibrillation. However, such algorithms and neural networks often require 12 leads and recordings over long periods. Moreover, such predictions are typically long-term (e.g., one year). In Attia et al., “An artificial intelligence-enabled ECG algorithm for the identification of patients with atrial fibrillation during sinus rhythm: atrospective analysis of outcome prediction,” The Lancet, Vol. 394, No. 10201, pp. 861-867, September 7, 2019 (the entire contents of which are incorporated herein by reference), the authors describe the use of artificial intelligence and convolutional neural networks to detect asymptomatic atrial fibrillation. In Christopoulos et al., “Artificial Intelligence-Electrocardiography to Predict Incident Atrial Fibrillation,” *Circ Arrhythm Electrophysiol*, Vol. 13, No. 12, December 2020 (the entire contents of which are incorporated herein by reference), the authors describe the use of artificial intelligence algorithms applied to electrocardiograms during sinus rhythm to determine the probability of atrial fibrillation. In Raghunath et al., “Deep Neural Networks Can Predict New-Onset Atrial Fibrillation From the 12-Lead ECG and Help Identify Those at Risk of Atrial Fibrillation-Related Stroke,” *Circulation*, Vol. 143, No. 13, March 30, 2021 (the entire contents of which are incorporated herein by reference), the authors describe the determination of the onset of atrial fibrillation from a resting 12-lead ECG.

[0017] Given the aforementioned limitations of previously known systems and methods, it will be desirable to process ECG data accurately and efficiently, and to present that information in an easily understandable manner. For example, it will be desirable to use enhanced computational techniques to analyze ECG data sampled from patients to accurately and efficiently detect and / or predict cardiac events, for instance, using artificial intelligence and / or machine learning techniques specifically designed for ECG analysis. Summary of the Invention

[0018] This document provides systems and methods for analyzing ECG data with enhanced accuracy and efficiency using machine learning algorithms and medical-grade artificial intelligence. Specifically, systems and methods are provided for analyzing patient ECG data using artificial intelligence and large amounts of electrocardiogram (ECG) data. The system receives ECG data from sensing devices positioned on the patient, such as one or more ECG leads / electrodes, which may be integrated into smart technologies (e.g., smartwatches). The system can analyze the ECG data sampled from the patient to accurately and efficiently detect and / or predict cardiac events, such as arrhythmias and / or abnormalities, including atrial fibrillation (AFib). The system may include an application communicating with an ECG platform running on a server, which, for example, uses neural networks for characterizing cardiac signals and classifying various abnormalities, conditions, and / or descriptors to process and analyze the ECG data. The ECG platform may be a cloud-based ECG platform that processes and analyzes ECG data in the cloud. Processed ECG data is passed from the server for display in a user-friendly and interactive manner with enhanced accuracy. The ECG application and ECG platform work together to implement the ECG processing system, which receives ECG data, processes and analyzes ECG data, displays ECG data on system devices, and generates reports containing ECG data.

[0019] This document provides a computerized system for analyzing ECG data of a patient, the ECG data being generated by one or more electrodes across multiple time points and including multiple pulsations. The computerized system can be designed to analyze the ECG data using a delineation algorithm to generate wave information corresponding to the probability of the presence of at least one wave at the multiple time points, and is also designed to determine pulsation initiation and pulsation termination information of pulsations among the multiple pulsations, wherein the presence of at least one wave is determined to generate multiple pulsation initiations and pulsation terminations. The computerized system can also be designed to extract multiple pulsation segments of the ECG data based on the multiple pulsation initiations and pulsation terminations, each of the multiple pulsation segments of the ECG data corresponding to a pulsation among the multiple pulsations, and to determine that at least two pulsations among the multiple pulsations should be grouped together based on the multiple pulsation segments of the ECG data, the at least two pulsations forming a cluster. Determining that the at least two pulsations among the multiple pulsations should be grouped together may involve determining that the grouped data meets a threshold.

[0020] The computerized system can also be designed to use an embedding algorithm to analyze the plurality of portions of ECG data to generate embedded data representing the plurality of beats, and to use a grouping algorithm to analyze the embedded data to generate group data. Based on the group data, it can be determined that at least two beats among the plurality of beats are grouped together. The group data can correspond to the distance between the two beats. The delineation algorithm can utilize a first neural network, and the embedding algorithm can utilize a second neural network. The grouping algorithm can utilize a third neural network. The computerized system can also be designed to receive user input data from an input device regarding inaccuracies corresponding to data displayed related to the ECG data. The computerized system can also be designed to adjust one or more of the following based on the user input data: the delineation algorithm, the embedding algorithm, or the grouping algorithm.

[0021] The computerized system can also be designed to modify the displayed data based on the user input data. The user input data may correspond to adding, deleting, or splitting one or more QRS clusters, PVC clusters, or PAC clusters. The embedded data may involve vectors of data for each of the plurality of beats. The computerized system can also be designed to transmit information indicating the clusters to a computer for display on a graphical user interface. The computerized system can also be designed to generate information to display at least one superposition, the at least one superposition comprising at least two beats superimposed on each other among the plurality of beats. The computerized system can also be designed to use classification algorithms to analyze the beats in the clusters to determine the likelihood of the presence of one or more abnormalities, conditions, or descriptors associated with the patient's cardiac events.

[0022] The computerized system can also be designed to use a classification algorithm to analyze the wave information from the delineation algorithm to determine the likelihood of the presence of one or more abnormalities, conditions, or descriptors associated with the patient's cardiac events. The wave information can be input into the classification algorithm and used individually to determine that at least two of the plurality of beats should be grouped together. The computerized system can also be designed to preprocess the ECG data before using the delineation algorithm to remove noise from the ECG data. The computerized system can assign the ECG data and information based on the ECG data to a user account for review. The computerization can receive user input data from the user account regarding the ECG data and information based on the ECG data based on the review.

[0023] This document describes a method for analyzing electrocardiogram (ECG) data of a patient, the ECG data being generated by one or more electrodes across multiple time points and including multiple beats. The method may involve using a delineation algorithm to analyze the ECG data to generate wave information corresponding to the probability of the presence of at least one wave at the multiple time points, and determining beat initiation and beat termination information for beats among the multiple beats, wherein the presence of at least one wave is determined to generate multiple beat initiations and beat terminations. The method may also involve: based on the multiple beat initiations and beat terminations; extracting multiple beat portions of the ECG data, each beat portion corresponding to a beat among the multiple beats; and determining that at least two beats among the multiple beats should be grouped together based on the multiple beat portions of the ECG data, the at least two beats forming a cluster.

[0024] The method may also involve using an embedding algorithm to analyze the plurality of portions of ECG data to generate embedded data representing the plurality of beats, and using a grouping algorithm to analyze the embedded data to generate group data. It can be determined, based on the group data, that at least two beats among the plurality of beats are grouped together. The method may also involve assigning the ECG data and information based on the ECG data to a user account for review of the ECG data. The method may also involve submitting the ECG data and information based on the ECG data for quality review by one or more reviewers. The method may also involve receiving quality control input generated by the one or more reviewers. The method may also involve displaying the quality control input for further quality control review. The method may also involve receiving user input data from an input device regarding inaccuracies corresponding to information based on the ECG data. The method may also involve adjusting one or more of the following based on the user input data: the delimitation algorithm, the embedding algorithm, or the grouping algorithm. The method may also involve assigning displayed data to a user account for quality checks.

[0025] In one example, a system for analyzing a patient's ECG data may involve a first plurality of instructions designed, when executed, to acquire the patient's ECG data at multiple time points and to transmit the ECG data to at least one server. The ECG data may be sampled at a predetermined sampling rate (such as at least 20 samples per second). The system for analyzing ECG data may also involve a second plurality of instructions designed, when executed, to cause the at least one server to: receive the patient's ECG data; analyze the patient's ECG data using at least one algorithm trained based on multiple ECG datasets from different patients; quantify the probability of the presence of one or more anomalies, conditions, or descriptors; and transmit information corresponding to the presence of the one or more anomalies, conditions, or descriptors, or any combination thereof, to a computer remotely located from the at least one server for display.

[0026] The system for analyzing ECG data may also involve a third plurality of instructions designed, when executed by the computer, to cause the computer to display information corresponding to the presence of one or more abnormalities, conditions, or descriptors, or any combination thereof, based on information transmitted from the at least one server. It should be understood that each of the plurality of sets of ECG data from the different patients can be generated at a sampling rate equal to the rate used to acquire the ECG data. It should also be understood that the computer executing the third plurality of instructions can also run the first plurality of instructions.

[0027] When executed, the second plurality of instructions can also cause the at least one server to preprocess the ECG data. Preprocessing the ECG data may involve removing noise from the ECG data or representing the ECG data at a predetermined baseline frequency. Furthermore, when executed, the second plurality of instructions can analyze the patient's ECG data using at least one algorithm that applies the ECG data to a first neural network for delineation, and can also quantify the probability of the presence of at least one of the P wave, QRS complex, or T wave at each of the plurality of time points. The second plurality of instructions can also calculate at least one start and at least one end of at least one of the P wave, QRS complex, or T wave, and / or calculate at least one measurement result based on one or more of the start, the end, or the output of the first neural network.

[0028] It should also be understood that, when executed, the second plurality of instructions can analyze the patient's ECG data using at least one algorithm that applies the ECG data to a second neural network for classification. Specifically, the second plurality of instructions can quantify the likelihood of the presence of one or more abnormalities, conditions, or descriptors, and can apply a threshold to at least one value in the output of the second neural network, assigning at least one label corresponding to the one or more abnormalities, conditions, or descriptors if the value exceeds the threshold. The second plurality of instructions can also post-process the ECG data by removing redundant labels.

[0029] The system may further include a fourth and / or a fifth plurality of instructions. The fourth plurality of instructions, when executed, may cause the at least one server to generate a report, the report including at least transmitted information corresponding to the presence of the one or more anomalies, conditions, or descriptors. The fifth plurality of instructions, when executed, may receive user input related to the ECG data and cause the computer to transmit the user input to the at least one server, causing the at least one server to use the user input to generate the report. The report may include at least one heart rate density graph, the at least one heart rate density graph representing the density of the patient's heart rate as a function of time. It should be understood that the third plurality of instructions are also configured, when executed by the computer, to cause the computer to display a heart rate density graph, the heart rate density graph representing the density of the patient's heart rate as a function of time.

[0030] In another example, a system for analyzing a patient's ECG data may involve instructions stored on at least one server, the instructions being designed, when executed, to cause the at least one server to receive a set of the patient's ECG data at multiple time points. The set of ECG data may be sampled at a predetermined sampling rate (such as at least 20 samples per second). The instructions may also be designed to cause the at least one server to analyze the set of the patient's ECG data using at least one algorithm, quantifying the probability of the presence of one or more anomalies, conditions, or descriptors, or any combination thereof, at each of the multiple time points, and transmitting information corresponding to the probability of the presence of the one or more anomalies, conditions, or descriptors to a computer for display. The at least one algorithm may be trained using multiple sets of ECG data generated from different patients at a sampling rate of at least 20 samples per second.

[0031] A computerized method for analyzing a patient's ECG data may similarly involve: receiving a collection of the patient's ECG data sampled at a sample rate at multiple time points, and analyzing the collection of the patient's ECG data using at least one algorithm trained on multiple collections of ECG data. Each of the multiple collections of ECG data may be generated from different patients at the same sample rate. The computerized method for analyzing ECG data may also involve identifying one or more abnormalities, conditions, or descriptors, or any combination thereof, at each time point, and may further involve transmitting information including the one or more abnormalities, conditions, or descriptors, or any combination thereof, to a computer for display. It should be understood that the computerized method may involve analyzing the entire collection of sampled ECG data without discarding data from the collection of ECG data. In one example, the computerized method may involve a sample rate of at least 20 samples per second.

[0032] The computerized method may also involve assigning the set of ECG data and information based on the set of ECG data to a user account for review of the ECG data. The computerized method may also involve submitting the set of ECG data and information based on the set of ECG data for quality review by one or more reviewers. The computerized method may also involve receiving quality control input generated by the one or more reviewers. The method may also involve displaying the quality control input for additional quality control review.

[0033] In another example, a computerized system for analyzing a patient's electrocardiogram (ECG) data may include a computerized system for analyzing the ECG data to determine the presence of a cardiac event. If the analysis of the ECG data determines that a cardiac event is present, the computerized system may generate information to identify the presence of the cardiac event for display. If the analysis of the ECG data determines that a cardiac event is not present, the computerized system may further analyze the ECG data to determine a risk score indicating the future risk of a cardiac event for display. The cardiac event may be atrial fibrillation.

[0034] The computerized system can also perform segmentation on ECG data to identify multiple beats, extract multiple first features from the multiple beats, and determine a first risk score based on the multiple first features. The first risk score can indicate the patient's future atrial fibrillation. Performing segmentation on the ECG data can determine multiple QRS initiation values. The computerized system can also perform classification on the ECG data to classify beats among the multiple beats as normal, atrial premature beat complex (PAC), or ventricular premature beat complex (PVC), determine multiple timestamps corresponding to the multiple beats, determine multiple heart rate values ​​based on the multiple timestamps, determine a matrix based on the multiple timestamps and the multiple heart rate values, generate a graphical representation of the matrix, and process the graphical representation of the matrix using at least one neural network to determine a second risk score indicating the patient's future atrial fibrillation.

[0035] At least one neural network is at least one deep neural network (DNN). The computerized system may further: determine patient information corresponding to the patient, determine multiple second features of ECG data indicating atrial fibrillation, and apply the multiple second features to a classifier to determine a third risk score, the third risk score indicating future atrial fibrillation for the patient. The patient information may be one or more of the patient's age and sex. The classifier is one or more of the following: DNN, logistic regression, and random forest. The computerized system may determine the average of a first risk score, a second risk score, and a third risk score to determine a risk score. The computerized system may also be configured to apply the first risk score, the second risk score, and the third risk score to a first neural network trained to determine the risk score.

[0036] In another example, a computerized system for analyzing a patient's ECG data may use a delineation algorithm to analyze the ECG data to determine the likelihood of the presence of at least one wave, and may use a classification algorithm to analyze the ECG data to extract multiple feature maps corresponding to the ECG data. The computerized system may also apply multiple feature maps to a recurrent neural network and use the recurrent neural network to analyze the multiple feature maps to determine a sequence label corresponding to the first beat, at least in part, based on feature maps indicative of a second beat occurring immediately preceding the first beat. The sequence label may be one of ectopic, supraventricular, or PVC.

[0037] In another example, a computerized system for analyzing a patient's ECG data can use a delineation algorithm to analyze the ECG data to determine wave information indicating the likelihood of the presence of at least one wave, and uses a baseline classification algorithm to analyze the ECG data and wave information. The computerized system can also use the baseline classification algorithm to determine a first value indicating the presence of at least one cardiac event, and can use a desensitized classification algorithm with reduced sensitivity compared to the baseline classification algorithm to analyze the ECG data and wave information. Furthermore, the computerized system can use the desensitized classification algorithm to determine a second value, use a sensitive classification algorithm to analyze the ECG data and wave information with increased sensitivity compared to the baseline classification algorithm, use the sensitive classification algorithm to determine a third value, and can determine, based on the second and third values, that the baseline classification is affirmative, indicating the presence of at least one cardiac event. The computerized system can also automatically generate a report corresponding to the presence of at least one cardiac event.

[0038] In another example, a computerized system for analyzing a patient's ECG data can upload ECG data from a database of ECG data to the computerized system, assign profiles to the ECG data, determine instructions to associate predetermined tags with the ECG data, assign predetermined tags to profiles associated with the ECG data, and determine instructions for filtering multiple ECG profiles based on predetermined tags. The computerized system can also analyze the ECG data to determine the presence of cardiac events and assign second tags to profiles associated with the ECG data based on the presence of cardiac events.

[0039] In another example, a computerized system for analyzing a patient's ECG data may: determine a plurality of ECG data, the plurality of ECG data including first ECG data corresponding to a first lead and second ECG data corresponding to a second lead; cause an ECG interface to display a first graphical representation of at least a portion of the first ECG data; determine instructions for displaying a second graphical representation of at least a portion of the second ECG data in addition to the first graphical representation; and cause the ECG interface to simultaneously display a second graphical representation that is time-synchronized with the first graphical display. The computerized system may also determine third ECG data corresponding to a third lead, the plurality of ECG data also including the third ECG data; determine instructions for displaying a third graphical representation of at least a portion of the second and third ECG data; and cause the ECG interface to simultaneously display a third graphical representation that is time-synchronized with the second graphical representation.

[0040] In another example, a computerized system for analyzing a patient's ECG data can analyze the ECG data using a delineation algorithm to determine first information indicating the probability of the presence of at least one wave, and can analyze the ECG data and the first information using multiple classification neural networks. Each of the multiple classification neural networks can utilize a weighting value unique to its classification neural network. The computerized system can also use the multiple classification neural networks to determine multiple outputs. Each of the multiple outputs can correspond to a classification neural network among the multiple classification neural networks. The computerized system can also use a combiner to analyze the multiple outputs to determine the probability of atrial fibrillation and a confidence score of the accuracy of the probability of atrial fibrillation. The combiner can determine an average value by averaging the multiple outputs. Alternatively, the combiner can determine the minimum value among the multiple outputs. In another example, the combiner can determine the maximum value among the multiple outputs.

[0041] In another example, a computerized system for analyzing a patient's ECG data can use a delimiting algorithm to analyze the ECG data to determine first information indicating the probability of the presence of at least one wave; an input transformer can be used to analyze the ECG data and the first information to modify the ECG data and generate multiple inputs; and a classification neural network can be used to analyze the multiple inputs. Furthermore, the computerized system can use the classification neural network to determine multiple outputs. Each of the multiple outputs can correspond to an input among the multiple inputs. Additionally, the computerized system can use a combiner to analyze the multiple outputs to determine the probability of atrial fibrillation and a confidence score for the accuracy of the probability indicating atrial fibrillation. The combiner can determine an average value by averaging the multiple outputs. The combiner can determine a minimum value among the multiple outputs. The combiner can determine a maximum value among the multiple outputs. The input transformer can perform an amplification transformation to amplify the ECG data using floating values. The input transformer can perform an expansion transformation to distort the ECG data in time.

[0042] The above overview is merely illustrative and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, other aspects, embodiments, and features will become apparent from the following drawings and detailed description. Attached Figure Description

[0043] Figure 1A is a recording of a standard 12-lead resting ECG, and Figure 1B is a recording of an exemplary P wave, QRS complex, and T wave.

[0044] Figure 2 This is a diagram illustrating exemplary components for performing systems and methods according to various aspects of this disclosure.

[0045] Figures 3A-3B These are schematic views of exemplary hardware and software components of an exemplary system device and an exemplary server, respectively.

[0046] Figure 4 This is a flowchart of an exemplary method for using processed ECG data, displaying the ECG data, and generating a report that includes the ECG data.

[0047] Figures 5A-5B These are line graphs representing exemplary ECG signals and exemplary outputs of the first neural network for each waveform being analyzed.

[0048] Figures 6A-6B These are exemplary representations of classification neural networks that employ convolutional neural networks and recurrent neural networks, respectively.

[0049] Figure 7 This is an exemplary representation of a variable number of lead inputs and a constant number of outputs.

[0050] Figure 8 It is an exemplary user interface with RR curves generated based on various aspects of recently published content.

[0051] Figure 9 yes Figure 8 An enlarged view of the RR curve shown in the figure.

[0052] Figure 10 It is an exemplary user interface having a heart rate density curve generated according to various aspects of this disclosure.

[0053] Figure 11 This is a flowchart illustrating an exemplary scheme for generating a heart rate density curve.

[0054] Figure 12 This is an exemplary heart rate density curve generated based on various aspects of this disclosure.

[0055] Figure 13 This is a flowchart of an exemplary method for processing ECG data, performing sequence identification, displaying ECG data, and generating a report that includes ECG data.

[0056] Figure 14 This is an exemplary representation of the process of applying a classification neural network and extracting feature maps for analysis by a sequence analyzer.

[0057] Figures 15A-15E An exemplary method is illustrated for analyzing ECG data and determining the certainty and / or confidence level of the probability of the existence of anomalies and / or conditions.

[0058] Figure 16This is an exemplary procedure for analyzing ECG data and determining the presence of asymptomatic atrial fibrillation.

[0059] Figure 17 This is an exemplary representation of the process of applying labels to ECG data.

[0060] Figures 18A-18H The diagram illustrates the user interface used to assign labels and other information to ECG data.

[0061] Figures 19A-19B The diagram illustrates the user interface for selecting and displaying ECG data from multiple leads.

[0062] Figure 20 The illustration shows an exemplary process for determining an atrial fibrillation risk score.

[0063] Figure 21 An exemplary graphical representation of the heart rate-time matrix is ​​illustrated.

[0064] The foregoing and other features of the invention will become apparent from the accompanying drawings, the following description, and the appended claims. It should be understood that these drawings depict only a few embodiments according to this disclosure and are therefore not intended to limit its scope, which will be described with additional specificity and detail using the drawings. Detailed Implementation

[0065] This invention relates to an electrocardiogram (ECG) processing system with medical-grade artificial intelligence, comprising an ECG application running on a system device and an ECG platform running on one or more servers. The ECG application and ECG platform implement the ECG processing system by using machine learning algorithms to process and analyze ECG data to detect and / or predict cardiac events, such as arrhythmias and / or abnormalities including atrial fibrillation (AFib). The system can perform the delineation of cardiac signals and the classification of various abnormalities, conditions, and descriptors. The one or more servers may be located in a different location than the one or more system devices, and the servers do not need to be in the same physical location as each other (e.g., the one or more servers may be remote servers). Alternatively, the one or more servers and the one or more system devices may be located in the same general area (e.g., on a local area network (LAN)). The ECG platform may be a cloud-based ECG platform, which implements the ECG processing system by processing and analyzing ECG data in the cloud.

[0066] To implement the ECG processing system, an ECG application running on the system device can receive ECG data (i.e., heart signals) from sensing devices and can transmit the ECG data to an ECG platform running on a server. The ECG platform can execute first and second neural networks and can apply ECG data to these neural networks. The first neural network can be a delineation neural network with machine learning capabilities. The second neural network can be a classification neural network with machine learning capabilities. The outputs of the first and / or second neural networks can be processed by the ECG platform to delineate and classify the ECG data. The data and / or ECG data generated by the ECG platform can be transferred from the ECG platform to the ECG application. The ECG application can interactively display the data and / or ECG data generated by the ECG platform. The ECG platform can generate reports including the data and / or ECG data generated by the ECG platform and can transfer these reports to the ECG application.

[0067] Now for reference Figure 2 The illustration shows exemplary components for performing electrocardiogram (ECG) processing in the system 10. Figure 2 ECG sensing device 13, system device 14, server 15, and driver 16 are shown.

[0068] ECG sensing device 13 is designed to sense the electrical activity of the heart for the purpose of generating ECG data. For example, sensing device 13 may be one or more electrodes that can be deployed on one or more leads. ECG sensing device 13 may be an ECG-specific sensing device (such as a conventional 12-lead arrangement) or a multi-purpose device with sensing hardware for sensing the electrical activity of the heart for ECG generation (such as the Apple Watch available from Apple Inc. in Cupertino, California). Sensing device 13 may be placed on the surface of a patient's limbs and / or chest. Sensing device 13 may be in electrical communication with system device 14 running ECG application 29, such that electrical signals sensed by sensing device 13 can be received by ECG application 29. ECG application 29 may include instructions to cause sensing device 13 to sense or otherwise acquire ECG data.

[0069] System device 14 preferably has the following reference Figure 3AThe components described herein include one or more computing devices (e.g., laptop computers, desktop computers, tablet computers, smartphones, smartwatches, etc.). System device 14 running ECG application 29 can connect to server 15 running ECG platform 37 via any known wired or wireless connection. For example, system device 14 can connect to the Internet using known technologies (e.g., WiFi, cellular networks, cable / coaxial cables, and / or DSL) and communicate with server 15 via the Internet.

[0070] Server 15 preferably has the following references Figure 3B The components described herein include one or more servers having the functions described herein. Server 15 preferably has processing power superior to system device 14, enabling server 15 to process and analyze cardiac signals with sampling rates above a predetermined threshold, such as at least 20 samples per second, at least 250 samples per second, or at least 1000 samples per second. As will be readily apparent to those skilled in the art, server 15 may comprise multiple servers located in a common physical location or in different physical locations. In a preferred embodiment, server 15 is located in a remote location different from system device 14 (e.g., in the cloud), but server 15 and system device 14 may be located in a common location (e.g., on a local area network (LAN)).

[0071] Server 15 may optionally communicate with drive 16, which may be one or more drives having memory dedicated to storing digital information specific to a particular patient, professional, facility, and / or device. For example, drive 16 may include, but is not limited to, volatile (e.g., random access memory (RAM)), non-volatile (e.g., read-only memory (ROM)), flash memory, or any combination thereof. Drive 16 may be incorporated into server 15, or it may be separate and distinct from server 15 and may communicate with server 15 via any known wireless or wired connection.

[0072] All aspects of the ECG processing system 10 and / or any other ECG processing system described herein may be identical or similar to the ECG processing system described in WO2020161605A1, which is the published application of PCT / IB2020 / 050850 filed on February 3, 2020 (corresponding to U.S. series application US17 / 390714), which claims priority to U.S. Patent US10,959,660, each of which is incorporated herein by reference in its entirety.

[0073] Now for reference Figures 3A-3BThis illustrates exemplary function blocks representing the hardware and software components of system device 14 and server 15. Now refer to... Figure 3A The hardware and software components of system device 14 may include one or more processing units 21, memory 22, storage device 27, communication unit 23, power supply 24, input device 25 and output device 26.

[0074] Processing unit 31 may be one or more processors configured to run cooperative operating system 28 and ECG application 29, and perform the tasks and operations of system device 14 as set forth herein. Memory 22 may include, but is not limited to, volatile (e.g., random access memory (RAM)), non-volatile (e.g., read-only memory (ROM)), flash memory, or any combination thereof. Communication unit 23 may transmit and / or receive information to and from other components in ECG processing system 10 (including, but not limited to, sensing device 13 and server 15). Communication unit 23 may be any known communication infrastructure that facilitates communication via any known wired or wireless connection (including via any known standard such as any IEEE 802 standard). Power supply 24 may be a battery, or may allow system device 14 to be connected to a wall outlet or any other external power source. Storage device 27 may include, but is not limited to, removable and / or non-removable storage devices, such as, for example, disks, optical discs, or magnetic tapes.

[0075] Input device 25 may be one or more devices coupled to or incorporated into system device 14 for inputting data into system device 14. Input device 25 may also include, for example, a keyboard, mouse, pen, voice input device (e.g., microphone), touch input device (e.g., touchpad or touchscreen), position sensor, and / or camera device. Output device 26 may be any device coupled to or incorporated into system device 14 for outputting or otherwise displaying data, and includes at least display 17. Output device 26 may also include, for example, speakers and / or printers.

[0076] ECG application 29 may be stored in storage device 27 and executed on processing unit 21. ECG application 29 may be a software application and / or software module having one or more instruction sets adapted to perform the operations of system device 14 as set forth herein, including facilitating the exchange of information with sensing device 13 and server 15. For example, ECG application 29 may enable system device 14 to receive ECG data from sensing device 13, record ECG data from sensing device 13, transmit ECG data to server 15, instruct server 15 to process and analyze ECG data, receive processed and / or analyzed ECG data from server 15, transmit user input regarding report generation to server, and generate a graphical user interface suitable for displaying raw, analyzed, and / or processed ECG data and related data.

[0077] An operating system 28 may be stored in storage device 27 and executed on processing unit 21. Operating system 28 may be adapted to control the general operation of system device 14 and may work with ECG application 29 to perform the functions of system device 14 as described herein. System device 14 may also optionally run graphics libraries, other operating systems, and / or any other applications. Of course, it should be understood that, in addition to… Figure 3A In addition to the components illustrated herein, system device 14 may include additional or fewer components, and system device 14 may include more than one component of each type.

[0078] Now for reference Figure 3B The hardware and software components of server 15 may include one or more processing units 31, memory 32, storage devices 35, power supply 33, and communication unit 34. Processing unit 31 may be one or more processors configured to run operating system 36 and ECG platform 37 and perform the tasks and operations of server 15 as described herein. Considering the amount of processing tasks and data allocated to processing unit 31, it should be understood that processing unit 31 has superior processing capabilities compared to processing unit 21.

[0079] Memory 32 may include, but is not limited to, volatile (e.g., random access memory (RAM)), non-volatile (e.g., read-only memory (ROM)), flash memory, or any combination thereof. Storage device 35 may include, but is not limited to, removable and / or non-removable storage devices, such as, for example, disks, optical discs, or magnetic tapes. Communication unit 34 may transmit and / or receive information to and from other components of ECG processing system 10 (including, but not limited to, system device 14 and / or drive 16). Communication unit 34 may be any known communication infrastructure that facilitates communication via any known wired or wireless connection. Power supply 33 may be a battery, or server 15 may be connected to a wall socket or other external power source.

[0080] An operating system 36 and an ECG platform 37 may be stored in storage device 35 and run on processing unit 31. The operating system 36 may be adapted to control the general operation of server 15. The ECG platform 37 may be a software application and / or software module having one or more instruction sets. The ECG platform 37 may facilitate and supervise the processing and analysis of ECG data received from system device 14, report generation, and in other cases may be adapted to perform the operations of server 15 as described herein.

[0081] ECG platform 37 may include several sub-modules and / or applications, including but not limited to preprocessor 38, delimiter 39, classifier 41, clusterer 42 which may include embedder 48 and grouper 49, postprocessor 43, report generator 44, recalculator 40 and / or sequence analyzer 50. Each sub-module and / or application may be a separate software application and / or module with one or more sets of instructions. The preprocessor 38 can preprocess the raw ECG data; the delimiter 39 can execute a first neural network to perform delimitation; the classifier 41 can execute a second neural network to perform classification; the clusterer 42 can identify clusters in the data processed by the first neural network; the postprocessor 43 can postprocess the data processed by the second neural network; the embedder 48 can execute one or more algorithms and / or a third neural network to perform embedding; the grouper 49 can execute one or more algorithms and / or a fourth neural network to generate cluster groups; the report generator 44 can generate a report based on the raw ECG data and the ECG data processed by the ECG platform 37; and the recalculator 40 can recalculate and / or adjust the embedder 48 and / or the grouper 49 based on user input data. For example, the recalculator 40 can recalculate the attack based on corrected wave information. The sequence analyzer 50 can be one or more algorithms and / or a third neural network, which can be a recurrent neural network. The sequence analyzer 50 can analyze feature maps to determine one or more sequence labels, thereby achieving sequence identification as described below. ECG platform 37 may also perform various other functions, including but not limited to receiving requests from system device 14 to process and / or analyze ECG data, transmitting processed and / or analyzed ECG data to system device 14, receiving requests to generate reports, requesting and / or receiving user interactions and / or instructions from system device 14, receiving user input data and / or instruction information from system device 14 regarding report generation, and / or transmitting reports to system device 14.

[0082] Server 15 can also optionally run graphics libraries, other operating systems, and / or any other applications. Of course, it should be understood that, in addition to... Figure 3BIn addition to the components illustrated herein, server 15 may include additional or fewer components, and server 15 may include more than one component of each type.

[0083] Figure 4 The illustration depicts an exemplary process for implementing an ECG processing system 10 to receive and record ECG data, process and analyze ECG data, and generate reports relating to ECG data. It also shows the information flow between the front-end 45 and back-end 46 of the ECG processing system 10, as described, for example, in U.S. Patents US10,959,660, US10,779,744, and US10,426,364, the entire contents of each of which are incorporated herein by reference. The front-end 45 includes at least an ECG application 29 running on system device 14. The back-end 46 includes at least an ECG platform 37 running on server 15.

[0084] like Figure 4 As shown, at step 51, ECG application 29 can cause system device 14 to receive and / or otherwise acquire raw ECG data 52 from sensing device 13. For example, ECG application 29 can cause sensing device 13 to sense cardiac signals and transmit the cardiac signals sensed by sensing device 13 to system device 14. The raw ECG data is the cardiac signals sensed by sensing device 13. The raw ECG data 52 has not yet been processed or analyzed by ECG processing system 10. The raw ECG data 52 preferably involves data sampled multiple times per heartbeat across multiple heartbeats. It should be understood that sensing device 13 can convert analog cardiac signals into digital signals. Figure 2 Different components, not shown, can convert analog heart signals into digital signals, or ECG application 29 can enable system device 14 to convert analog heart signals into digital signals. The raw ECG data in both analog and digital forms is referred to herein as raw ECG data 52.

[0085] Upon receiving raw ECG data 52, ECG application 29 enables system device 14 to record the raw ECG data 52, and may optionally save some or all of the raw ECG data 52 to system device 14. As explained above, the signal may correspond to one or more leads. When multiple leads are used, all leads can be processed simultaneously. It should be understood that the cardiac signal generated by each lead may have varying lengths. It should also be understood that the cardiac signal may be short-lived (e.g., 10 seconds in a standard ECG) or long-lived (several days in a Holter monitor). System device 14 may optionally display the raw ECG data 52 or a portion thereof on display 17.

[0086] like Figure 4As shown, raw ECG data 52 can be transmitted from front-end 45 to back-end 46. Specifically, ECG application 29 enables system device 14 to transmit raw ECG data 52 to ECG platform 37 running on server 15. Upon receiving raw ECG data 52, ECG platform 37 enables server 15 to save some or all of the raw ECG data 52 to server 15. Furthermore, after receiving raw ECG data 52, ECG platform 37 enables preprocessing of raw ECG data 52 by preprocessor 38 at step 54. It should be understood that preprocessor 38 can be a standalone component of ECG platform 37 or a sub-component of delimiter 39.

[0087] The preprocessor 38 can process the raw ECG data 52 or a portion thereof by removing interfering elements of the cardiac signal, such as noise from the raw ECG data. For noise filtering, a multivariate data analysis scheme can be used (Pigoli and Sangalli, Computational Statistics and Data Analysis, Vol. 56, 2012, pp. 1482-1498). Since the signal sensed by the sensing device 13 may vary with patient movement, the preprocessor 38 can remove the baseline frequency of the raw ECG data 52, and selected frequencies can be used to represent the cardiac signal. Median filtering can be used to remove frequencies of the signal corresponding to patient movement (Kaur et al., Proceedings of the International Journal of Computer Applications, 2011, pp. 30-36). The raw ECG data 52 is applied to the preprocessor 38 to generate preprocessed ECG data 55. At this point, the ECG platform 37 allows the preprocessed ECG data 55 to be optionally passed to the ECG application 29 running on the system device 14 for display on the display 17. ECG platform 37 may alternatively or additionally use preprocessed ECG data 55 as input at the classification step 58, which is discussed in more detail.

[0088] At step 56, the ECG platform 37 applies preprocessed ECG data 55 to the delimiter 39 for delimitation. The delimiter 39 applies a first neural network (which is a delimiting neural network) to the preprocessed ECG data 55. A neural network is a mathematical structure or algorithm that takes an object (e.g., a matrix or vector) as input and produces another object as output through a set of linear and nonlinear operations called layers. For example, the input to the first neural network may be one or more multi-lead cardiac signals, which are preprocessed to remove noise and / or baseline drift.

[0089] To apply the preprocessed ECG data 55 to the first neural network, the delimiter 39 can represent some or all of the raw ECG data 52 as a matrix X, which can be a real matrix. For example, matrix X can be an m×n matrix at the frequency used to train the network, as described in more detail below. The constant “m” can be the number of leads in the sensing device 13, which is typically 12, but any number of leads can be used. In this example, the number of samples “n” provides the duration of the cardiac signal “n / f”, where f is the sampling frequency of the cardiac signal. The sample rate is higher than a predetermined rate and is preferably relatively high, such as, for example, at least 20, at least 250, at least 500, or at least 1000 samples per second, etc. In one embodiment, all sampled ECG data is transferred to a server for input into the processing algorithm without filtering the ECG data. While the ECG data applied to the first neural network is preferably preprocessed ECG data 55, it should be understood that unprocessed cardiac signals (i.e., raw ECG data 52 or a portion thereof) can be applied to the first neural network.

[0090] The first neural network can provide as output values ​​corresponding to the probability of the presence of one or more waves in the cardiac signal at multiple time points. The time points may depend on the raw ECG data, may be selected by the user of system device 14, or may be pre-programmed. The first neural network may be a convolutional neural network, and preferably a fully convolutional neural network. A convolutional neural network is a specific type of neural network in which one or more matrices being learned do not encode a complete linear combination of input elements, but rather encode the same local linear combination at all elements of a structured signal (such as a cardiac signal) through convolution (Fukushima, Biol. Cybernetics, Vol. 36, 1980, pp. 193-202; LeCun et al., Neural Computation, Vol. 1, 1989, pp. 541-551). A network containing only convolutional networks is called a fully convolutional neural network.

[0091] Accordingly, at step 56, the delimiter 39 causes the first neural network to read each time point of the cardiac signal, perform spatiotemporal analysis on each time point of the cardiac signal, and assign a score corresponding to one or more types of waves at each time point. In this way, all types of waves in the cardiac signal can be analyzed, and the probability of their presence at each time point is quantified in a single step. Accordingly, each score generated by the delimiter 39 indicates the probability of a specific wave pattern being present at a given time point of the cardiac signal. The wave pattern can be any well-known wave pattern, such as, for example, P wave, Q wave, R wave, S wave, multiple Q waves, multiple R waves, multiple S waves, QRS complex, and / or multiple T waves. In this way, the delimiter 39 can process data sampled multiple times per heartbeat across multiple heartbeats.

[0092] The output of the first neural network can be a matrix Y, which can be a real matrix. For example, matrix Y can be a matrix of size p×n. Matrix Y can include a score for each type of wave at each time point of the cardiac signal. In matrix Y, “n” is the number of samples as discussed above relative to matrix X, and “p” is the number of wave types plus the number of wave representations. As explained in more detail below, wave representations can correspond to, for example, the conduction rate, prematureness, ectopicness, and / or origin of the wave in the cardiac signal. In one example, wave types include (1) P wave, (2) QRS complex, and (3) T wave, and wave representations include (1) premature wave, (2) pacing wave, (3) ventricular QRS complex, (4) junctional QRS complex, (5) ectopic P wave, and (6) nonconduction P wave. Accordingly, in this example, p = 3 + 6 = 9. Each wave type can be described according to certain characteristics of the wave, such as the start and end points (i.e., the beginning and the end).

[0093] Now for reference Figure 5A and 5B For each wave type, an exemplary output of the first neural network is represented graphically to illustrate the generated score values ​​corresponding to multiple wave types at each time point. Specifically, Figure 5A The illustration shows an exemplary output of a defined neural network processing a normal cardiac signal (where there are no abnormalities), and Figure 5B The illustration shows an exemplary output in which a defined neural network processes cardiac signals, such as those with "hidden" P waves due to atrioventricular block.

[0094] Now for reference Figure 5AThe diagram illustrates four line graphs, each showing time on the x-axis. Line graph 71 represents the cardiac signal across multiple beats. The plotted signal reflects a known ECG waveform with a P wave (point 75), QRS complex (point 76), and T wave (point 77). Line graph 72 is a graph of the P wave score at the same time point in the cardiac signal. Similarly, line graphs 73 and 74 are graphs of the QRS and T wave scores at the same time points, respectively. The y-axis of each line graph 72-74 is a score assigned to each time point in the range of 0 to 1, where 0 indicates a low probability of the presence of a particular wave, and 1 indicates a high probability of the presence of a particular wave. For example, line graph 72 indicates an extremely high probability of the presence of a P wave at score 78, corresponding to a time point near point 75; line graph 73 indicates an extremely high probability of the presence of a QRS complex at score 79, corresponding to a time point near point 76; and line graph 74 indicates an extremely high probability of the presence of a T wave at score 80, corresponding to a time point near point 77.

[0095] Figure 5B Similar to Figure 5A The diagram illustrates four line graphs (line graphs 81-82), similar to line graphs 71-74. Specifically, line graph 81 shows the cardiac signal over several beats, line graph 82 shows the P-wave score on the cardiac signal, line graph 83 shows the QRS score on the cardiac signal, and line graph 84 illustrates the T-wave score on the cardiac signal. Figure 5A Unlike other ECG signals, the line graph 81 includes a hidden P wave, such as the one shown at point 85. A hidden P wave is a P wave that occurs during another wave or complex wave, such as the T wave. Because the cardiac signals processed by the delineation network involve a high sample rate, and because the delineation network generates data for each waveform at each time point, the recovered output is robust enough (e.g., including enough sample points) to identify two waves occurring at the same time, such as those with a hidden P wave. For example, line graph 82 indicates a very high probability of the presence of a P wave at score 86, which corresponds to a time point near point 85. Accordingly, it should be understood that the delineation neural network is not limited to recovering only one wave at each time point and is therefore able to identify several waves at any given time point. It should also be understood that signals from one or more leads can be processed simultaneously by the first neural network.

[0096] Using a score assigned to each time point corresponding to each waveform (e.g., P wave, QRS complex, T wave, etc.), delimiter 39 can post-process the cardiac signal. Post-processing involves: assigning no wave to each time point, assigning one wave, or assigning several waves; calculating the start and end of each identified wave; and optionally determining the wave's characterization. Waves can be assigned to each time point by determining the presence of a wave at that time point if a certain value is reached. Calculating the "start" and "end" of each wave involves calculating the start and end time points of each wave in the cardiac signal; the start is referred to as the "start," and the end as the "end." This can involve analyzing the start and end time points of the highest value corresponding to each waveform. Delimiter 39 can characterize waves by identifying premature, conduction, and ectopic waves. Wave characterization utilizes contextual information between each wave and / or each beat. For example, a premature label can be applied to a wave if it reaches a certain threshold at a certain time point or as an average across several time points.

[0097] After calculating the start and end of each waveform in the cardiac signal, the delimiter 39 can calculate global measurements. Global measurements are derived from the start and end of each waveform and can be related to characteristics and properties of the cardiac signal, such as the intervals between waves and wave duration. For example, global measurements may include, but are not limited to, PR interval, P wave duration, QRS complex duration, QRS axis, QT interval, corrected QT interval (Qtc), T wave duration, JT interval, corrected JT interval, heart rate, ST segment elevation, Sokolov index, number of ventricular premature beats (PVCs), number of atrial premature beats (PACs), ratio of nonconduction P waves, and / or ratio of pacing waves.

[0098] The delimiter 39 can also infer tags solely from the information generated by the delimiter 39. For example, the delimiter 39 can infer tags such as: short PR interval (i.e., PR interval < 120 ms), first-degree AV block (i.e., PR interval > 200 ms), axis deviation, long QTc, short QTc, wide complex wave tachycardia, and / or intraventricular conduction block. Tags determined solely from the information generated by the delimiter 39 are referred to as delimited tags.

[0099] Refer again Figure 4The ECG platform 37 allows the output of step 56 (e.g., wave information 62) and preprocessed ECG data 55 to be passed to or otherwise applied to clusterer 42 for clustering at step 63. Wave information 62 may include scores for PVC and PAC waves (including the start and end of their generation and their associated duration). Clusterer 42 processes wave information 62 and identifies clusters of PAC or PAV waves during the duration of the cardiac signal. Once identified, clusterer 42 can assign cluster labels 64 to one or more time windows, thereby identifying PVC or PAC clusters for each time window. The time window is defined by two time points in the cardiac signal.

[0100] Refer again Figure 4 The ECG platform 37 can also allow the output of step 56 (e.g., wave information 57) and the preprocessed ECG data 55 to be passed to or otherwise applied to classifier 41 for classification at step 58. Classification at step 58 involves applying a second neural network (i.e., a classification neural network) to the preprocessed ECG data 55. Accordingly, in one example, the input to the second neural network could be one or more multi-lead cardiac signals of variable length that have been preprocessed. Classifier 41 can also process wave information 57 and / or other information (such as patient-specific information, including the patient's age or any relevant clinical information). As explained above, the ECG platform 37 can optionally allow the preprocessed ECG data 55 to be passed directly to classifier 41 and processed by classifier 41 if classification at step 56 is not necessary. In this way, classifier 41 can process data sampled multiple times per heartbeat across multiple heartbeats.

[0101] The second neural network generates outputs with values ​​corresponding to the probability of the presence of one or more abnormalities, conditions, and / or descriptors at each time point of the cardiac signal. If a time point or time window is determined to correspond to a certain abnormality, condition, and / or descriptor, a label corresponding to that abnormality, condition, and / or descriptor is assigned to that time point or window. In one example, if the score reaches a predetermined threshold, one or more labels 59 can be assigned to a time point or time window. Accordingly, multi-label localization of abnormalities, conditions, and / or descriptors can be achieved by generating multiple values ​​at each time point and assigning one or more labels at each time point.

[0102] Classifier 41 can recover the output of the classification neural network as a vector of size q. The values ​​in the vector correspond to the presence of each label at each time point or time window. For example, the output of the classification neural network could be a vector [0.98: 0.89; 0.00] with a corresponding label for each element of the vector: right bundle branch block; atrial fibrillation; normal ECG. The score can be between 0 and 1. For the above vector, a threshold of 0.5 will cause classifier 41 to assign the labels “right bundle branch block” and “atrial fibrillation” to the time point or time window corresponding to the score. It should be understood that the threshold can be pre-programmed and / or selected by the user, and can be modified to provide different levels of sensitivity and specificity. By assigning one or more labels to each time point, the start and end points corresponding to each label can be calculated to identify the duration of an episode (e.g., an abnormal episode).

[0103] Abnormalities and conditions can include any physiological abnormality or condition that can be identified on cardiac signals. Currently, approximately 150 measurable abnormalities can be identified on cardiac signal recordings. Abnormalities and conditions may include, but are not limited to: sinoatrial block, paralysis or arrest, atrial fibrillation, atrial flutter, atrial tachycardia, junctional tachycardia, supraventricular tachycardia, sinus tachycardia, ventricular tachycardia, pacemaker, ventricular premature beats complex waves, atrial premature beats complex waves, first-degree atrioventricular block (AVB), second-degree AVB Mobitz I, second-degree AVB Mobitz II, third-degree AVB, Wolf-Parkinson-White syndrome, left bundle branch block, right bundle branch block, intraventricular conduction delay, left ventricular hypertrophy, right ventricular hypertrophy, acute myocardial infarction, old myocardial infarction, ischemia, hyperkalemia, hypokalemia, ventricular fibrillation, and / or long QTc. Descriptors can include descriptive quality of cardiac signals, such as “normal” or “noisy ECG”.

[0104] When the second neural network is applied at step 58, the classifier 41 can read each time point of the heart signal and each global measurement result, analyze each time point of the heart signal and each global measurement result, calculate a time window by aggregating at least two time points, and calculate a score for each time window, the score corresponding to multiple non-exclusive labels.

[0105] Classification neural networks can be either convolutional neural networks or recurrent neural networks. (See now for reference.) Figure 6AThe figure illustrates a classification neural network in the form of a convolutional neural network applied to ECG signals. Most convolutional neural networks implement several convolutional layers and then standard layers to provide classification. The ECG signal is given as input to the network, which aggregates information locally and then combines the information layer by layer to produce a high-level multi-label classification of the ECG. A score is provided for each label. The labels of the convolutional neural network shown in Figure 6 include atrial fibrillation (AFIB), right bundle branch block (RBBB), and premature ventricular contractions (PVC).

[0106] Now for reference Figure 6B The diagram illustrates a classification neural network employing a recurrent convolutional neural network. (And...) Figure 6A Similarly, the ECG signal is given as input to the network. A recurrent convolutional neural network (RCN) is a specific convolutional neural network structure that can retain a memory of previous objects it has been applied to. An RCN consists of two sub-networks: a convolutional neural network that extracts features and is computed at all time points of the cardiac signal, and a neural network built on top of the convolutional neural network that accumulates the output of the convolutional neural network over time to provide a refined output. In this way, the convolutional neural network acts as a pattern detector, and its output is accumulated in a timely manner by the recurrent neural network.

[0107] like Figure 6B As shown, the output of the convolutional neural network identified four labels at various time points, including ventricular premature beat complex (PVC) and normal. Those labels were then applied to a second neural network, which produced the refined output "ventricular premature beat complex". In this example, the network correctly identified the PVC (the fifth and largest beat) in the first part of the signal, while the second part of the signal was considered normal. Because cardiac signals can contain abnormalities, it cannot be considered normal, and the cumulative output is therefore PVC.

[0108] The first neural network (i.e., the delimiting neural network) and the second neural network (i.e., the classification neural network) must be trained to achieve the behavior and functionality described herein. In both the delimiting and classification embodiments, the networks can be described using open-source software such as, for example, Tensorflow, Theano, Caffe, or Torch. These tools provide the functionality to compute one or more outputs of the network and to update its parameters via gradient descent.

[0109] Training a neural network involves applying a large dataset containing cardiac signals and known outputs to the neural network. A database containing a dataset of cardiac signals collected across multiple patients using the systems and methods described herein can be stored on server 15 and / or driver 16 (e.g., in the cloud). The dataset in the database can be used by server 15 to analyze new cardiac signals input into the system for processing. In a preferred embodiment, any cardiac signal applied to the trained neural network will have the same sampling rate and / or frequency as the cardiac signals in the dataset used to train the neural network. For example, training of a classification neural network begins with a dataset containing cardiac signals and their known delineations. As explained above, the cardiac signals are represented as a matrix of size m×n at predefined frequencies. For example, the network can be trained at 250Hz, 500Hz, or 1000Hz, but any frequency can be used. The delineation is then represented in the form of a matrix Y of size p×n, where p is the number of wave types. Each wave is represented by its start and end points, such as, for example: (P, 1.2s, 1.3s), (QRS, 1.4s, 1.7s), (T, 1.7s, 2.1s), (P, 2.2s, 2.3s). In this example, the first row of matrix Y corresponds to the P wave and will have values ​​of 1 at times 1.2s and 1.3s, and also 2.2s and 2.4s, and values ​​of 0 in other cases. The second row of matrix Y corresponds to the QRS complex wave and will have values ​​of 1 at times 1.4s and 1.7s, and values ​​of 0 in other cases. Finally, the third row of matrix Y corresponds to the T wave and will have values ​​of 1 at times 2.2s and 2.3s, and values ​​of 0 in other cases. Subsequently, the network parameters can be modified to reduce the cost function compared to the network output and a known delineation. A cross-entropy error function is used to allow for multiple labels (i.e., allowing multiple waves at a given instant). This minimization can be accomplished via a gradient step, which repeats the aforementioned steps at least once for each cardiac signal in the dataset. It should be understood that a similar approach can be used to train the defined neural network (i.e., the second neural network).

[0110] It should also be understood that the ECG platform 37 enables the neural network described herein to process cardiac signals with varying numbers of leads as input. For example, the neural network may include a series of layers at the beginning of the network to obtain a network that is independent of the number of input leads and thus capable of processing cardiac signals with any number of leads m. For example, Figure 7 The diagram illustrates two input leads (m=2) and three output signals (k=3). However, the same structure can handle any number of input leads m and will still provide the same number of output signals, which can be fed to the rest of the network that requires a fixed number of input signals. For this reason, the number of input leads can vary and does not need to be fixed.

[0111] like Figure 7 As shown, to obtain k signals from m input leads, the leads can be convolved using lead-by-lead convolution through k filters. The signals can then be grouped by convolution filters to obtain k groups of m leads, and a mathematical function is finally applied to each group to obtain k leads. The mathematical function can be the maximum value at each time point, or it can be any other function known to those skilled in the art.

[0112] Refer again Figure 4 At step 61, the ECG platform 37 can aggregate the labels (i.e., tags) for each time window by the post-processor 43 to generate processed labels 60. These labels can be derived from defined global measurements. For example, a label corresponding to first-degree atrioventricular block can be derived from a PR interval longer than 200 ms. As explained above, the PR interval is based on defined global measurements. The post-processor 43 can also aggregate the defined labels with classification labels corresponding to the same time point.

[0113] The post-processor 43 can also filter labels to remove redundant labels, assemble labels based on their known hierarchy, or ignore labels that are known to be less important based on hierarchy or weighting. The post-processor 43 can also aggregate labels by time to calculate the start (beginning) and end (termination) time of each anomaly. It should be understood that the post-processor 43 can be a standalone component or a sub-component of the classifier 41.

[0114] like Figure 4 As shown, the information generated by the ECG platform 37 on the backend 46 in steps 54, 56, 58, and 61, and optionally in step 63, can be transmitted by the ECG platform 37 to the ECG application 29 on the frontend 45. The ECG application 29 can display the aforementioned information on the display 17 of the system device 14 at step 65. The information generated on the backend 46 can be automatically transmitted by the ECG platform 37, or the ECG platform 37 can store the information on the server 15 before it is requested by the ECG application 29. When data is generated, the ECG platform 37 can transmit a message to the ECG application 29, thereby notifying the ECG application 29 that data is available from the ECG platform 37.

[0115] ECG application 29 can receive data (e.g., raw ECG data generated during steps 54, 56, 58, 61, and / or 63, preprocessed ECG data, wave information, tags, and any other data) and cause system device 14 to display it as described in U.S. Patent Publication US2020 / 0022604, which is incorporated herein by reference in its entirety. Specifically, the '604 publication explains that ECG signals, characteristics of ECG signals, and / or descriptors of ECG signals can be displayed interactively in a multi-field display.

[0116] Now for reference Figure 8 An exemplary display (interactive display 101) is illustrated. Interactive display 101 includes a first side 102 and a second side 103. The first side 102 also includes a second graphics window 105 and a first graphics window 104, which have a graph 110 including data corresponding to the ECG signal. The first graphics window 104 includes the graph 110 providing a global view of the ECG signal.

[0117] Now for reference Figure 9 The illustration shows an enlarged version of the first graphics window 104. In this exemplary display, graph 110 is an RR interval graph, which is a graph of the RR interval (the interval between two QRS waves) over time. Figure 9 As shown, the upper area of ​​the first graphical window 104 includes a plurality of label buttons 109. Each label button 109 has text displayed nearby describing a label associated with it. Each label button 109 is associated with a color such that when a label button 109 is selected by the user, a graphical portion 111 is displayed on a graph 110 to visually indicate the presence of an attack and / or event corresponding to the label associated with the label button 109. This provides the user with a visual reference, thereby allowing easy identification of specific categories of events and / or attacks along cardiac signals. Figure 9 The exemplary display illustrated includes secondary labels 112. In this exemplary display, secondary labels 112 include pulsation labels PVC (ventricular premature beat complex) and PSVC (supraventricular premature beat complex), but it should be understood that other secondary labels may be included. The points in graph 110 associated with labels PVC and PSVC are colored by the presence of dots with a color different from black, as shown below. Figure 9 As shown in the image.

[0118] The first graphics window 104 also includes a time bar 115 parallel to the time axis of the graph 110. The time bar 115 provides a linear representation of the total ECG acquisition time, where the time periods associated with an attack or event are represented as colored segments. Figure 9As shown, the darker gray areas on time bar 115 correspond to periods of noisy signals (e.g., when the signal has too many artifacts and the analysis algorithm cannot propose delineation and correct detection). The first graphics window 104 also includes an interactive cursor 116. A user of the ECG application 29 can move the interactive cursor 116 along time bar 115 to allow navigation of the graph 110 along the total ECG acquisition time. In the lower right corner of the first graphics window 104, the first graphics window 104 includes a second interactive component 117 configured to zoom in and out of the graph 110.

[0119] Refer again Figure 8 The second side 103 includes multiple seizure graphs 106. Each seizure graph 106 displays at least one segment of the ECG band corresponding to a detected seizure and may include text about the duration (e.g., “Duration: 1 hour 38 minutes”) and / or the seizure start time (e.g., “Day 3 / 09:39:30”). Each seizure graph 106 includes a third interactive icon 108 for selecting the corresponding seizure graph for inclusion in a report. Each seizure graph 106 also includes a fourth interactive icon 107 that allows the user to remove the corresponding ECG graph from the interactive display 101. The second side 103 may also include text describing one or more of the seizure graphs 106.

[0120] The interactive display 101 also includes a graphics window 105, which includes an ECG bar 118 in a second time window starting from the time point selected by the cursor 116. The second graphics window 105 also includes an ECG bar 119 in a third time window that is larger than the second time window and includes the second time window. The third time window includes a shaded portion corresponding to the second time window.

[0121] Now for reference Figure 10 A similar display (interactive display 121) is illustrated. Interactive display 121 includes a first side 122 and a second side 123. The first side 122 also includes a first graphics window 124 and a second graphics window 125. The second side 113 has the same function as the second side 103 described above, and includes a seizure curve 126 similar to the seizure curve 106. Furthermore, the second graphics window 125 has the same function as the second graphics window 105, and includes ECG bands 138 and 139 similar to ECG bands 118 and 119.

[0122] Except for graph 130, the first graphics window 124 is similar to the first graphics window 104. Like the first graphics window 104, the first graphics window 124 includes multiple label buttons 129 having the same function as the multiple label buttons 109, secondary labels 132 having the same function as secondary labels 112, a time bar 135 and a cursor 136 having the same function as a time bar 115 and a cursor 116, and a second interactive component 137 having the same function as a second interactive component 117. Unlike graph 110, graph 130 is a heart rate density graph, which is a projection onto a bivariate intensity graph of a histogram of heart rate density as a function of time.

[0123] Now for reference Figure 11 The document provides steps for generating and plotting a heart rate density curve, such as graph 130. At step 141, the ECG platform 37 calculates the RR intervals in the cardiac signal (i.e., ECG data). For example, as described above, the ECG platform 37 can apply the cardiac signal to a delineated neural network to determine the RR intervals. At step 142, the ECG platform 37 can generate a heart rate curve over time. Figure 12 The figure shows an exemplary heart rate curve HRDP 150.

[0124] like Figure 12 As shown, time is projected along the x-axis, and heart rate (e.g., beats per minute) is projected along the y-axis. In one embodiment, both time and heart rate are scaled linearly. However, time and / or heart rate may be scaled logarithmically or using other known scales. For simplicity, Figure 12 The image shows only four heart rates.

[0125] Refer again Figure 11 In step 143, the ECG platform 37 allows the y-axis and x-axis to be divided into basic elements, referred to as HR bins and time bins, respectively. For example, in Figure 12 The diagram illustrates HR bin 151 and time bin 152. HR bin 151 consists of first and second heart rate values ​​(e.g., h...). b 1 and h b 2 The time bin 152 is defined by first and second time values ​​(e.g., t). b 1 and t b 2 Definition. The intersection of the HR bin and the time bin will be called a bin. In other words, a bin will be defined by the first and second heart rate values ​​and the first and second time values. Figure 12 The diagram illustrates bin 153, which is defined by HR bin 151 and time bin 152.

[0126] Refer again Figure 11 At step 144, the ECG platform 37 assigns each heartbeat to a bin. Specifically, heartbeats occurring within the time window of a given time bin (e.g., QRS complex) are included in the calculation of the column corresponding to that time bin. Furthermore, the heart rate corresponding to that heartbeat determines which HR bin it belongs to in the column defined by the time bins. For example, in Figure 12 In this context, heartbeats 154 and 155 each have corresponding time and heart rate values ​​that fall within time bin 152 and HR bin 151, respectively. Conversely, heartbeats 156 and 157 each have time values ​​that fall outside time bin 151 and are therefore not included in bin 153.

[0127] Refer again Figure 11 At step 145, the ECG platform 47 calculates heart rate density for each time frame. For a given frame, the region defined by the corresponding time frame and heart rate frame is represented based on the density of heartbeats included in the frame (i.e., the number of heartbeats within the frame). Each frame can then be color-coded based on density. For example, each frame can have certain shades of color or pattern, such as grayscale levels. Figure 12 In the example, binning can be represented as gray levels that become darker as the density of the heart rate increases. For example... Figure 12 As shown, a bin 153 containing two heartbeats can be represented by a darker gray shading than a bin with only one heartbeat, but a lighter gray shading than a bin with three or more heartbeats.

[0128] In a preferred embodiment, density is calculated as a function of the number of R waves in a bin divided by the heart rate in the HR bin (e.g., the average of the minimum and maximum limits of the time window). This preferred calculation of density takes into account the time spent in a particular bin. For example, in a 3-minute time bin, if 100 beats occur at a heart rate of 50 bpm in the first HR bin and 100 beats occur at 100 bpm in the second HR bin, there will be the same number of beats in each bin, but it will take 2 minutes at 50 bpm and only one minute at 100 bpm. Therefore, if only the number of beats is considered, this bin will have the same density representation. However, when considering the number of beats divided by the heart rate, the first bin corresponding to the 50 bpm heart rate bin will be darker than the bin corresponding to the 100 bpm heart rate bin, because dividing by the heart rate gives higher weight to lower heart rate values. The preferred embodiment therefore captures this time information better than considering only the number of beats.

[0129] Refer again Figure 11At step 146, the ECG platform 37 will plot the heart rate density for each bin. It should be understood that, in addition to the time information naturally given as a function of the x-axis, the time information in the capture column (time bin) facilitates a more efficient representation of the density than other forms of aggregated representation of ECG signals (such as the RR curve in Figure 110).

[0130] It should be understood that the x-axis boundaries of the HR density curve can be the start and end of the signal. However, in a preferred embodiment, the x-axis boundaries can interactively change with zooming in and out performed by the user. During this action, the y-axis boundaries remain fixed. (See again...) Figure 10 The graph 130 includes an interactive component 137 that can be used to zoom in on the heart rate density graph. The zooming action can simply change the size of the graph display. Alternatively, zooming in and out changes the size of the time window corresponding to the time bins. With the zooming action, bins represented by the same number of pixels cover a shorter time window. Zooming in thus allows for new calculations of the histogram with finer time divisions and therefore finer time information. This allows for a representation of the ECG signal showing varying levels of information aggregation (which are functions of the time scale we choose to display) so that the histogram remains both readable and informative at any zoom level.

[0131] Now for reference Figure 13 The illustration depicts an exemplary process for implementing an ECG processing system 500 to receive and record ECG data, process and analyze ECG data and ECG data sequences, and generate reports relating to them. Figure 13 ,like Figure 4 This illustrates the information flow between the front end 45 and the back end 46 of the ECG processing system 10. As described above, the front end 45 includes at least an ECG application 29 running on the system device 14, while the back end 46 includes at least an ECG platform 37 running on the server 15.

[0132] Also with Figure 4 Similarly, at step 51, ECG application 29 may cause system device 14 to receive and / or otherwise acquire raw ECG data 52 from sensing device 13. Upon receiving raw ECG data 52, ECG application 29 may optionally cause system device 14 to record raw ECG data 52, and may optionally save part or all of the raw ECG data 52 to system device 14. As described above, the signal may correspond to one or more leads. System device 14 may optionally display the raw ECG data 52 or a portion thereof on display 17.

[0133] like Figure 13 As shown and as above regarding Figure 4As explained, raw ECG data 52 can be transmitted from front-end 45 to back-end 46 via ECG application 29, thereby enabling system device 14 to pass raw ECG data 52 to ECG platform 37 running on server 15. Upon receiving raw ECG data 52, ECG platform 37 can cause server 15 to save some or all of the raw ECG data 52 to server 15. Furthermore, after receiving raw ECG data 52, ECG platform 37 can optionally cause raw ECG data 52 to be preprocessed by preprocessor 38 at step 54. Applying raw ECG data 52 to preprocessor 38 can generate preprocessed ECG data 55. Preprocessor 38 can process data as described above regarding... Figure 4 The raw ECG data 52 is interpreted. At this point, the ECG platform 37 may optionally transfer the preprocessed ECG data 55 to the ECG application 29 running on the system device 14 for display on the display 17. The ECG platform 37 may alternatively or additionally use the preprocessed ECG data 55 as input at the classification step 58. Alternatively, the raw ECG data 52 may be used as input at the classification step 58.

[0134] At step 56, the ECG platform 37 applies preprocessed ECG data 55 or raw ECG data 52 to the delimiter 39 for delimitation. As described above, the delimiter 39 applies a first neural network (which is a delimiting neural network) to the preprocessed ECG data 55 or raw ECG data 52, and some or all of the ECG data can be expressed as a matrix X, which can be a matrix of real numbers. While the ECG data applied to the first neural network is preferably preprocessed ECG data 55, it should be understood that unprocessed cardiac signals (i.e., raw ECG data 52 or a portion thereof) can be applied to the first neural network. The first neural network can provide, as output, values ​​corresponding to the probability of the presence of one or more waves at multiple time points in the cardiac signal. The first neural network can be a convolutional neural network, and preferably a fully convolutional neural network. Therefore, at step 56, the delimiter 39 causes the first neural network to read each time point of the cardiac signal, perform spatiotemporal analysis on each time point of the cardiac signal, and assign a score at each time point corresponding to one or more types of waves.

[0135] The output of the first neural network can be a matrix Y, which can also be a matrix of real numbers. For example, matrix Y can be a p×n matrix as described above. Matrix Y can include a score for each type of wave at each time point of the cardiac signal, and each wave type can be expressed according to certain characteristics of the wave, such as the start and end points (i.e., beginning and termination). The output of the first neural network can detect pulsations in the input ECG data, or can be used to detect pulsations in the ECG data. ECG platform 37 can allow the output of step 56 (e.g., wave information 57) and the preprocessed ECG data 55 or the raw ECG data 52 to be passed to or otherwise applied to classifier 41 for classification at step 58.

[0136] As mentioned above Figure 4 As explained, the classification at step 58 involves applying a second neural network (i.e., a classification neural network) to the preprocessed ECG data 55 or the raw ECG data 52. The classifier 41 may also process wave information 57 and / or other information, such as patient-specific information including patient age or any relevant clinical information. The ECG platform 37 may optionally pass the preprocessed ECG data 55 or the raw ECG data 52 directly to the classifier 41, and process it if the classification at step 56 is unnecessary. It should be understood that the first neural network and the second neural network may alternatively be a single neural network performing the functions and operations of both.

[0137] As mentioned above Figure 4 As explained, at step 58, the second neural network can generate an output having values ​​corresponding to the probability of the presence of one or more abnormalities, conditions, and / or descriptors at each time point of the cardiac signal. If a time point or time window is determined to correspond to a certain abnormality, condition, and / or descriptor, a label corresponding to that abnormality, condition, and / or descriptor will be assigned to that time point or window as label 59. In one example, if the score reaches a predetermined threshold, one or more labels 59 can be assigned to a time point or time window. Furthermore, at step 58, the second neural network can determine a feature map 501 corresponding to the original ECG data 52, ECG data 54, and / or wave information 57 input into the second neural network.

[0138] At step 502, a feature map 501 can be extracted using a sequence analyzer 50, which can be one or more algorithms and / or a third neural network. For example, the third neural network might be a recurrent neural network. The sequence analyzer 50 can analyze the feature map 501 to determine one or more sequence labels 503, and thus achieve sequence identification of the ECG data.

[0139] Now for reference Figure 14 The diagram illustrates steps 56, 58, and 502. For example... Figure 14 As shown, the raw ECG data 52 is shown as the first neural network (i.e., the delineated neural network) that enters the output wave information 57. Figure 14 The diagram also illustrates classifier 41 and sequencer analyzer 50. ECG data 52 and / or wave information 57 can be applied by classifier 41 as input to a second neural network (i.e., a classification neural network), as described above. Figure 13 As described, a classification neural network can be a convolutional neural network with a small number of convolutional layers and then standard layers, thereby providing classification. ECG signals are given as input to the network, information is aggregated locally, and then combined layer by layer to produce a high-level multi-label classification of ECG signals.

[0140] like Figure 14 As shown, the units of a layer in a neural network can be segmented into feature maps 501. Feature maps 501 can represent the output of filters applied to ECG data and can be used to identify certain features in the input of the neural network (e.g., ECG data). Feature maps 504 can include high-level information (e.g., information about ventricular tachycardia (PVCs) – such as three consecutive PVCs). Therefore, classifier 41 can project the beats determined by the delineated neural network onto the feature maps to create a fixed-size feature for each beat.

[0141] At step 502, the feature map 501 determined by classifier 41 can be extracted by sequence analyzer 50. Sequence analyzer 50 can also extract or otherwise decompose the feature map corresponding to the coordinates of each beat, thereby obtaining feature map portion 505. Feature map portion 505 can be organized sequentially according to the position of each beat in the ECG data. Sequence analyzer 50 can feed the sequence of feature map portion 505 into a third neural network (e.g., long short-term memory or simple recurrent neural network). The third neural network, as a recurrent neural network, can analyze each feature map portion 505 with respect to adjacent feature map portions and / or earlier sequence map portions to extract information about the sequence of feature map portion 505. The third neural network, as a recurrent neural network, can be trained end-to-end. The third neural network can output the probability of the feature map portion 505 corresponding to a specific sequence label 503.

[0142] Sequence labels 503 can be assigned to a pulsation associated with a pulse map portion 505 and can indicate information determined based on surrounding pulsations and / or pulsation sequences. For example, sequence labels 503 may include ectopic, supraventricular, PVC, or normal labels. It should also be understood that thresholds can be used to process the probability of each feature map portion 505 to determine the presence of a sequence label. Figure 14As shown, the pulse plot portion corresponding to the PVC (e.g., the red pulse plot portion) is assigned "1" for meeting the PVC threshold, and the other pulse plot portions of the ECG data band (e.g., the green pulse plot portion) are assigned "0" for not meeting the threshold.

[0143] Refer again Figure 13 After determining feature map 501 and tag 59 at step 58, and after determining sequence identification 502 and sequence tag 503, as per... Figure 4 The startup step 61. Specifically, at step 61, the ECG platform 37 may have the labels (i.e., tags) of each time window aggregated by the post-processor 43 to generate processed labels 60. The labels may be labels 59 and / or sequence labels 53. The post-processor 43 may also filter the labels to remove redundant labels, assemble the labels according to a known label hierarchy, or ignore labels known to have less importance based on hierarchy or weighting. The post-processor 43 may also aggregate the labels by time to calculate the start (start) and end (termination) times of each anomaly. It should be understood that the post-processor 43 may be a standalone component or a sub-component of the classifier 41.

[0144] As mentioned above Figure 4 As explained, the information generated by the ECG platform 37 on the backend 46 in steps 54, 56, 58, 502, and 61 can be transmitted from the ECG platform 37 to the ECG application 29 on the frontend 45. The ECG application 29 can then display this information on the display 17 of the system device 14 at step 65. Furthermore, at step 66, the user can request a report and select customizable features, such as certain data to be included in the report (e.g., anomaly / condition data, event data, episodes, etc.). At step 67, the ECG application 29 can transmit the report request and selectable customizable features (e.g., ECG data to be included in the report) to the ECG platform 37, and the ECG platform 37 can receive the request and information. The ECG platform 37 can record the request and save the information received from the ECG application 29. At step 68, the ECG platform 37 can cause the report generator 44 to generate a report 69 based on the information received from the system ECG application 29.

[0145] Now for reference Figure 15A The diagram illustrates an exemplary process for determining certainty in a classification system and automatically generating reports on the presence of anomalies and / or conditions. Steps 511-513 are the same as described above. Figure 4Steps 52 and 56 are similar. Specifically, at step 511, the system may receive, determine, or otherwise acquire the patient's ECG data. At step 512, the ECG data (e.g., raw or preprocessed ECG data) may be analyzed using a delineation algorithm to generate wave information (e.g., a score corresponding to the likelihood of the presence of T waves, P waves, QRS complexes, etc.). Furthermore, at an optional step 513, the onset and termination of the pulse may be optionally determined (e.g., based on wave information), and the pulsatile portion of the ECG data may be extracted based on the onset and termination of the pulse.

[0146] At step 514, a classification algorithm with a baseline setting can be used to analyze the ECG data (e.g., pulsatility and / or wave information). The baseline setting may include a baseline sensitivity determined by training the classification algorithm. Step 514 can be substantially the same as described above. Figure 4 Step 58 is similar. At step 515, the classification algorithm processes the ECG data to determine the presence of abnormalities, labels, and / or conditions (e.g., atrial fibrillation, AV block, normal). For example, the classification algorithm may determine the probability of the presence of one or more abnormalities, labels, and / or conditions. A threshold may be applied to the probability of the presence of one or more abnormalities, labels, and / or conditions to determine the presence of the abnormality, label, and / or condition based on whether the probability meets the threshold.

[0147] At step 516, the ECG data analyzed in step 514 can be analyzed with reduced sensitivity by the classification algorithm compared to the classification algorithm in step 514. The classification algorithm with reduced sensitivity can be determined by training the algorithm. For example, the classification algorithm in step 516 can be trained to identify the presence of one or more anomalies, labels, and / or conditions more frequently than the baseline classification algorithm in step 514 given the same ECG data. At step 517, the presence of anomalies, labels, and / or conditions corresponding to those in step 516 can be determined similarly to step 515.

[0148] At step 518, the ECG data and / or corresponding wave information analyzed in step 514 can be analyzed with increased sensitivity by the classification algorithm compared to the classification algorithm in step 514. A classification algorithm with increased sensitivity can be determined by training the algorithm. For example, the classification algorithm in step 516 can be trained to identify the presence of one or more anomalies, labels, and / or conditions less frequently than the baseline classification algorithm in step 514 given the same ECG data. At step 519, the presence of the anomaly, label, and / or condition corresponding to step 518 can be determined similarly to step 515. It should be understood that steps 516-517 and steps 518-519 can occur in parallel or in series.

[0149] At decision 520, the ECG system (e.g., ECG platform 37) can determine whether the anomaly / condition / tag identified in steps 514 and 515 was also identified as present in steps 516 and 517 and 518 and 519. Therefore, at decision 520, the system determines whether the reduced sensitivity classification algorithm and the increased sensitivity classification algorithm also lead to the determination of the presence of anomalies, tags, and / or conditions.

[0150] If it is determined that no anomaly, label, or condition exists based on one or more of the reduced sensitivity classification algorithm and the increased sensitivity classification algorithm, then the determination of the presence of the anomaly, label, and / or condition by the baseline classification algorithm (e.g., at steps 514 and 515) is identified as uncertain, and at optional step 521, the system may suggest and / or recommend that the ECG data requires further analysis (e.g., by the user).

[0151] Alternatively, if an anomaly, label, and / or condition is determined to exist based on a reduced-sensitivity classification algorithm and an increased-sensitivity classification algorithm, then the determination of the presence of the anomaly, label, and / or condition by the baseline classification algorithm (e.g., at steps 514 and 515) is identified as definitive, and at optional step 522, a report based on the presence of the anomaly, label, and / or condition can be generated. For example, this can be done according to the description above. Figure 4 Steps 68 and 69 generate the report. In one example, the report may be fully automated and therefore can be generated without any human interaction. It should be understood that the processes described in sections 516-520 can be a standalone set of algorithms or can be one or more layers on top of a classification neural network. It should also be understood that... Figure 15A The steps described herein can be performed in conjunction with or supplement to other operations described herein, including those mentioned above. Figure 4 Those described.

[0152] It should also be understood that the determinism of a classification algorithm can be alternatively determined by determining the area under the receiver operating characteristic (ROC) curve (AUC), or it can also be indicated by determining the area under the ROC curve (AUC). Specifically, the sensitivity of a neural network can be plotted against 1-specificity, and the area under the curve can indicate the model's accuracy. Specificity can be determined by dividing the number of true positives by the sum of true positives and false negatives. Furthermore, "1-specificity" can be determined by the number of false positives as a whole by the sum of false positives and true negatives. An AUC value close to 1 may indicate an accurate model.

[0153] Now for reference Figure 15B The diagram illustrates the robust classification system 565. (This is related to the previous section on...) Figure 15A The described process is similar, and the robust classification system 565 can be a component of the ECG processing system described above (e.g., ECG processing system 10, ECG processing system 10', and / or ECG processing system 500). For example, the ECG processing system 565 can replace or otherwise supplement the classification step 58. Specifically, the robust classification module 568 can replace the classification step 58. The robust classification module 568 can include a classifier 569, which can be the same as or similar to the classifier 41 described above. Specifically, the classifier 569 can execute a neural network to perform classification. The classifier 569 can receive preprocessed ECG data 566 and RR data 567 as input. The preprocessed ECG data 566 can be the same as the preprocessed ECG data 56 described above. The additional RR data 567 can be the output of a delimiting neural network, such as the output of delimiting step 56 described above. For example, the RR data 567 can be the same as the wave information 57 described above.

[0154] like Figure 15B As shown, classifier 569 can process preprocessed ECG data 566 and RR data 567 to determine the AFib probability 571, similar to classifier 41 in classification step 58 described above. The AFib probability 571 is the likelihood that the preprocessed ECG data 566 includes an atrial fibrillation episode. The output of classifier 569, the preprocessed ECG data 566, and / or RR data 567 can also be applied to and processed by confidence estimator 570 to determine a confidence score 572, which may correspond to a confidence score 572 for the probability that the AFib probability 571 is accurate and / or determined. Confidence estimator 570 can be one or more algorithms (e.g., neural networks) and can have the same classifier structure (e.g., architecture) as classifier 569, but can be trained to be more specific. For example, confidence estimator 570 can perform the same operations as described above regarding... Figure 15A Steps 516-517 are similar to those described.

[0155] Now for reference Figure 15CThe diagram illustrates a robust classification system 573, which is similar to robust classification system 565 but includes a robust classification module 574. Also similar to robust classification system 565, robust classification system 573 may replace or otherwise supplement the classification step 58 described above. Robust classification module 574 may receive ECG preprocessed data 566 and RR data 567 and may output AFib probabilities 571 and confidence scores 572, similar to robust classification module 568. Unlike robust classification module 568, robust classification module 574 may include a confidence classifier 575 instead of classifiers 569 and confidence estimators 570. The confidence classifier 575 may be a neural network that performs the same functions and operations as classifiers 569 and confidence estimators 570 to output AFib probabilities 571 and confidence scores 572.

[0156] Now for reference Figure 15D The diagram illustrates a robust classification system 576. Similar to robust classification systems 565 and 573, robust classification system 576 includes preprocessed ECG data 566 and RR data 567 as input, and may replace or otherwise supplement the classification step 58 described above. ECG classification system 576 may include a robust classification module 577.

[0157] The robust classification module 577 may include a classifier 580 that is the same as or similar to the classifier 41 described above. Specifically, the classifier 580 may execute a neural network to achieve classification. The robust classification module 577 may also include an input transformer 578 and a combiner 582. The input transformer 578 may process the inputs to the robust classification module 577 (e.g., preprocessed ECG data 566 and / or RR data 567) and may apply transformations to such inputs. For example, the transformations may be amplification, expansion, and / or lead selection. Amplification transformations may use floating-point values ​​to amplify the signal (e.g., preprocessed ECG data 566). Expansion transformations may involve the signal (e.g., preprocessed ECG data 566) being distorted in time, and the RR data 567 being adjusted and / or otherwise modified as a result. If the ECG data received by the ECG system corresponds to more than one lead, a subset of the data (e.g., preprocessed ECG data and / or RR data) may be preserved and / or maintained and may be retrieved. By applying preprocessed ECG data 566 and RR data 567 to input converter 578, a transformed input 579 can be generated. The transformed input 579 can include several (e.g., four) transformed copies of the same signal. Although in Figure 15CThe diagram illustrates four transformed inputs 579, but it should be understood that any number of transformed inputs can be generated. It should also be understood that amplification, expansion, lead selection, and / or any other transformation techniques can be used to generate the transformed inputs 579.

[0158] like Figure 15D As shown, the transformed input 579 can be applied to and processed by classifier 580. Similar to classifier 569, classifier 580 can be the same as or similar to classifier 41 described above. Specifically, classifier 580 can execute a neural network to perform classification. Classifier 580 can generate output 581. Classifier 580 can process each copy of the transformed input 579 and can generate an output for each copy. Therefore, if the transformed input 579 comprises four transformed copies of the signal, classifier 580 can generate four outputs.

[0159] The robust classification module 577 may also include a combiner 582. Output 581 can be applied to and processed by the combiner 582. The combiner 582 can take the outputs of classifier 580, which have the same format (e.g., size N), and combine the outputs. For example, the combiner 582 can combine outputs 581 by taking the average of the outputs 581, which may result in a single output possibly having a size N. Alternatively, to achieve greater sensitivity, the combiner may alternatively determine the maximum value in outputs 581. Similarly, to achieve lower sensitivity, the combiner may alternatively determine the minimum value in outputs 581. The combiner 582 can output AFib 571 and confidence score 572. For example, classifier 580 may be similar to confidence classifier 575 and can generate a set of outputs 581 corresponding to AFib probability 571 in addition to a second set of outputs 581 corresponding to confidence score 572. The combiner can process different sets of outputs separately to obtain AFib probability 571 and confidence score 572. Alternatively, combiner 582 and / or classifier 580 may be similar to confidence classifier 575 and may together determine AFib probability 571 and confidence score 572.

[0160] Now for reference Figure 15E The diagram illustrates a robust classification system 583. Similar to robust classification systems 565, 573, and 576, robust classification system 583 includes preprocessed ECG data 566 and RR data 567 as input and may replace or otherwise supplement classification step 58, as described above. ECG classification system 583 may include a robust classification module 584.

[0161] The robust classification module 584 may include classifiers 585, which may be several different classifiers, each of which may be the same as or similar to classifier 41 described above. Specifically, each classifier 581 may execute a neural network to perform classification. Each classifier may be trained differently (e.g., using different datasets and / or each with different weights). For example, classifiers 581 may all be the same structure (e.g., architecture) with different weights, thus producing different classifiers. Figure 15E As shown, classifier 381 may include four classifiers, each trained using four different training datasets, and therefore with different sets of weights. This may result in classifiers having different levels of sensitivity (e.g., some classifiers having lower and / or higher sensitivity than others). The same preprocessed ECG data 566 and / or RR data 567 can be input and processed by each classifier in classifier 585. Each classifier in classifier 585 can then generate a corresponding output. Together, all classifiers 585 generate output 586.

[0162] The robust classification module 584 may also include a combiner 587. Output 586 can be applied to and processed by the combiner 587. For example, the combiner 587 can take the outputs of classifier 586 with the same format (e.g., size N) and combine the outputs. The combiner can combine outputs 586 by taking the average of the outputs 586, which may result in a single output of size N. Alternatively, to achieve greater sensitivity, the combiner can alternatively determine the maximum value of output 586. Similarly, to achieve lower sensitivity, the combiner can alternatively determine the minimum value among the outputs 586. The combiner 587 can output AFib 571 and a confidence score 572. For example, classifier 585 can be similar to confidence classifier 575, and for each classifier, in addition to a second output in output 581 corresponding to confidence score 572, one output in output 586 corresponding to AFib probability 571 can be generated. The combiner can process different sets of outputs separately to obtain AFib probability 571 and confidence score 572. Alternatively, combiner 587 and / or classifier 586 can be similar to confidence classifier 575 and can together determine AFib probability 571 and confidence score 572.

[0163] Based on the AFib probability 571 and confidence score 572, the ECG system can determine with high confidence that atrial fibrillation is absent, with low confidence that atrial fibrillation is absent, with high confidence that atrial fibrillation is present, or with low confidence that atrial fibrillation is present. If the confidence level is low, the corresponding ECG signal will be identified as indeterminate. If the confidence level is high, the ECG signal will be appropriate as, and / or otherwise identified as having or not having atrial fibrillation. If the ECG signal is appropriate as having or not having atrial fibrillation, the ECG system will generate a report. For example, the report may include daily heart rate trends, the most relevant ECG bands based on certain criteria (e.g., the onset of an atrial fibrillation episode, the fastest atrial fibrillation episode), and / or conclusions and / or determinations regarding the presence and / or determination of atrial fibrillation. If the ECG signal is appropriate as indeterminate, the ECG signal may be flagged and / or submitted for human review.

[0164] It should be understood that the methods illustrated in 15D and 15E and described above can be combined to include a robust classification system comprising an input transformer and multiple classifiers, each trained differently. For example, the methods can be combined into a robust classification module that takes a signal and transforms it into N modified signals, each signal being input into M classifiers, which provide N×M outputs that can be applied to a combiner to determine the AFib probability and confidence score.

[0165] It should also be understood that when the preprocessed ECG data 566 is described as input to classifier 569 and / or confidence estimator 570, the raw ECG data can alternatively be input to... Figure 15B -The robust classification module in E. It should also be understood that although the above description of atrial fibrillation... Figure 15A -E, but the same system can be trained to detect any other conditions and / or anomalies that can be detected using ECG data and corresponding confidence scores.

[0166] Now for reference Figure 16 The illustration depicts an exemplary procedure for determining the risk of cardiac events (such as atrial fibrillation) in asymptomatic ECG data (e.g., without an episode of atrial fibrillation). This procedure can be performed by an ECG platform 37. While the procedure is illustratively described as determining the risk of atrial fibrillation, it can be used to determine the risk of other cardiac events. Figure 15A resemblance, Figure 16 The exemplary process begins at step 531, which is similar to step 511, by receiving or obtaining the patient's ECG data; at step 532, which is similar to step 512, by analyzing the ECG data using a delineation algorithm to generate wave information; and at step 533, which is similar to step 514, by analyzing the ECG data (and / or wave information) using a classification algorithm. Figure 16 The process also includes step 534, which involves determining the likelihood of atrial fibrillation. Similar to... Figure 15A In step 515, at step 534, the classification algorithm can process the ECG data to determine the presence of atrial fibrillation. For example, the classification algorithm can determine the probability of the presence of atrial fibrillation. A threshold can be applied to the probability of the presence of atrial fibrillation to determine the presence of atrial fibrillation based on whether the probability meets the threshold.

[0167] At decision 535, the ECG platform 37 can determine whether atrial fibrillation exists in the ECG data based on the determination made at step 534. If atrial fibrillation is determined to be present in the ECG data, then at step 536, the ECG data can be identified as having atrial fibrillation. For example, corresponding to atrial fibrillation... Figure 4 Tag 59 can be assigned to ECG data and can be started. Figure 4 One or more of steps 61, 65, 66, and 68, as described above.

[0168] Alternatively, if at decision 535, based on the determination made at step 534, it is determined that atrial fibrillation is not present in the ECG data, then an atrial fibrillation (AF) risk analyzer can be used to analyze the ECG data analyzed at step 533. The AF risk analyzer can be a standalone set of algorithms (e.g., a neural network), or it can be one or more layers above a classification neural network.

[0169] The AF risk analyzer can output one or more values ​​indicating the likelihood that a patient corresponding to ECG data has atrial fibrillation, even if atrial fibrillation is not detected at step 534. For example, the AF risk analyzer can be a convolutional neural network that can analyze ECG data and / or wave information, such as morphological and temporal features. The AF risk analyzer can include several blocks (e.g., one or more layers). For example, the AF risk analyzer can include batch normalization layers to account for the normalization of the data distribution and a nonlinear corrected linear unit (ReEU) function. The ReEU function can output zero for negative inputs and output an identification output for positive inputs to facilitate feature extraction. Blocks can include shortcut links that allow gradient propagation between layers and / or components of the neural network, and may also include pooling layers, one or more convolutional layers, dropout layers, and / or output layers. It should be understood that the AF risk analyzer can be trained using ECG data from patients diagnosed with atrial fibrillation but in which no atrial fibrillation episodes were present in the ECG data, to identify or otherwise determine features in ECG data associated with patients having atrial fibrillation.

[0170] At step 538, the output of the AF risk analyzer can be used to determine the atrial fibrillation risk value. For example, a classification algorithm can determine the probability of atrial fibrillation. A threshold can be applied to the probability of atrial fibrillation to determine the presence of atrial fibrillation based on whether the probability meets the threshold.

[0171] At decision 539, the system (e.g., ECG platform 37) can determine whether a marker of atrial fibrillation is present in the ECG data. For example, if a threshold is met (e.g., the probability of atrial fibrillation being present exceeds or equals the threshold), a marker of atrial fibrillation can be determined to be present in the ECG data. If a marker of atrial fibrillation is determined to be present or absent in the ECG data and / or the threshold is not met, then at optional step 540, the system may do nothing or may mark the ECG data as not having atrial fibrillation. Alternatively, if a marker of atrial fibrillation is determined to be present in the ECG data and / or the threshold is not met, then at step 541, an atrial fibrillation risk can be identified. For example, an atrial fibrillation label or an atrial fibrillation risk label can be assigned to the ECG data.

[0172] At optional step 542, the system may display the risk of atrial fibrillation on display 17. The risk may be displayed as a risk level score or some indication. For example, the risk may be displayed as a calculated risk value on a predetermined scale (e.g., 75 out of 100). The risk may also be categorized based on where the calculated risk value falls within a predetermined threshold risk range (e.g., 0 to 35 out of 100 corresponds to "low", 35 to 70 out of 100 corresponds to "moderate", and 70 to 100 out of 100 corresponds to "high"). The categorization may be displayed with or without a calculated risk value. Furthermore, at optional step 543, P-wave morphology analysis, PAC analysis, and / or other information corresponding to the risk of atrial fibrillation may be determined as described herein and may optionally be displayed on display 17. It should be understood that, in addition to including information about... Figure 4 The operations described herein, in addition to or in conjunction with other operations, can be performed. Figure 16 The steps described herein. It should also be understood that a risk analyzer can be trained to detect the presence of another asymptomatic abnormality and / or condition (e.g., cardiac event) in the analysis of ECG data.

[0173] Now for reference Figure 17 The diagram illustrates the labeled ECG data. For example... Figure 17As shown, upon determining or otherwise obtaining ECG data 551, the system (e.g., ECG platform 37 and / or ECG application 29) may assign a profile 552 to ECG data 551. Profile 552 may be displayed on display 17 and may include certain information about ECG data 551. For example, profile 552 may include review status 553, header 554, and ECG information 555. Review status 553 may include information about whether ECG data 551 has been reviewed. For example, review status 553 might be “pending review” or “reviewed.” Header 554 may include letters and numbers associated with ECG data 551 and is used to identify ECG data 551. ECG information 555 may include information about ECG data 551, such as recording duration (e.g., 1 day and 23 hours) and / or upload date (e.g., June 19, 2020).

[0174] Profile 552 may include one or more tags assigned to ECG data 551. For example, ECG tag 558 and ECG tag 559 may be added to profile 552. ECG tag 558 may identify the technician or healthcare provider assigned to ECG data 551. ECG tag 559 may identify an abnormality, condition, tag, and / or sequence tag assigned to analyze ECG data 551. It should be understood that ECG data may be processed automatically by delimiting and / or classifying neural networks, and therefore the system may automatically determine the presence of abnormalities, episodes, and / or conditions in the ECG data and may assign ECG tags to profile 552 of ECG data 551 to associate the ECG data with that abnormality, episode, or condition. For example, ECG tag 559 may say “Abnormal Y”. Alternatively, the tag may be added manually. Profile 552 may include an add button 556 to add a tag and / or a delete button 557 to delete a tag from profile 552. Therefore, ECG tag 552 may be generated automatically or by the user.

[0175] Various ECG data, determined by the system or otherwise obtained, can be categorized and / or filtered based on the information in profile 552. For example, the labels assigned to ECG data 551 in profile 552 can be used to filter the ECG data. Figure 17In the illustrated example, ECG data 552 can fall into three different groups of ECG data. First, ECG data 551 can be included in group 561, which can include ECG data with anomaly Y. Second, ECG data 563 can be included in group 563, which can include ECG data assigned to technician X. Furthermore, ECG data 551 can even be grouped into group 562 with two labels (technician X and anomaly Y). These groups can be used to view other ECG data associated with a given label in the corresponding group. This can be a useful tool for technicians to find the ECG data assigned to them and / or for users to view ECG data with similar labels.

[0176] It should be understood that a profile may include information about ECG data that is not embedded in the ECG data but may affect how the ECG data is managed within the ECG system. Tags may be visible to all users within the same organization, or may be limited to certain users within the organization in other cases.

[0177] Now for reference Figure 18A The diagram illustrates interface 601. Interface 601 may include a processing window 602, a pending review window 603, and a reviewed window 604. Processing window 602 may include an upload button 607, which can be engaged to upload ECG signals. For example, upload button 607 may cause ECG platform 37 to retrieve ECG signals from a database, or it may cause electrodes to acquire ECG signals. Window 603 may include ECG profiles of ECG signals that have been uploaded but not yet reviewed. For example, profile 605 may be included in pending window 603 and may be similar to profile 552. Furthermore, reviewed window 604 may include ECG profiles of ECG signals that have been uploaded and reviewed. For example, profile 606 may be included in reviewed window 606 and may be similar to profile 552.

[0178] Now for reference Figure 18B The illustration shows an exemplary profile 611. Exemplary profile 611 can be included in a processing window 602 or an reviewed window 603. Exemplary profile 611 can include profile 612, which can be similar to profile 552 and can include a title, ECG information, and at least one label. Figure 18B As shown, profile 552 may include notification 613 stating that ECG data "has been returned for analysis." As mentioned above, this can occur if ECG platform 37 fails to analyze or determine a high degree of confidence in the condition and / or anomaly. Also... Figure 18BAs shown, profile 612 may also include a label 614 such that label 614 can specify "stat" to indicate that ECG data analysis is required. Furthermore, an add label button 617 can be engaged to add more labels to profile 612. For example, engaging label button 617 may result in a label window 615, which may display available labels and the selected label profile 612. In this example, the label "stat" is selected, and the label "notification" is available but not selected. Additionally, a create new label button 618 can be included in label window 615 to create a new label for profile 612. The newly created label can be used by any ECG profile. It should be understood that some labels may be available by default (e.g., stat, notification) and / or may not be available for editing.

[0179] Now for reference Figure 18C The illustration shows profile 625, which may be similar to profile 611. Profile 625 may include profile 621, which may be similar to profile 611, as it may include a title, ECG information, and at least one label. For example, profile 621 may include label 622, which may indicate that a patient corresponding to the ECG data in profile 621 has a pacemaker. Label window 626 may be included in profile 621 and may be similar to label window 615. Label window 626 may also include an edit button 623 and a delete button 624. Edit button 623 may be engaged to edit the name of the label in label window 622. Furthermore, delete button 624 may be engaged to delete the label in label window 626.

[0180] Now for reference Figure 18D Interface 631 illustrates: a window to be reviewed 632, which may be similar to a window to be reviewed 603; and a review window, which may be similar to a review window 633. For example... Figure 18D As shown, interface 631 may include a filter window 634, which can be used to filter ECG system profiles. Filter window 634 may include a category section 636 and a label section 637. Category section 636 may include special types of labels corresponding to conditions, anomalies, or events. Label section 637 may include labels indicating information about the corresponding ECG profile, such as "stat" and "notification." Notifications may correspond to highlighted or marked portions of ECG data, and / or may include specific information about the ECG data. Categories and / or labels in filter window 634 can be selected to display only ECG profiles with the selected categories and / or labels. Clear button 635 can be used to reset the filter.

[0181] Now for reference Figure 18E The diagram illustrates interface 641, which can be used to upload new ECG signals. When in... Figure 18AWhen the upload button 607 is engaged, an upload interface 641 is generated. Interface 641 may include an upload window 642, which can be used to upload ECG data to the ECG system. When an ECG file is selected for upload, the ECG file is placed in an upload queue 643, which may display the selected file and also display a download progress chart 644 visually indicating the progress of the upload to the ECG system. From the upload queue, a tag window 646 can be displayed to add tags to the ECG file as it is uploaded to the ECG system. The upload interface may also include a tag window 645, which may display the tags selected for the ECG signal in the upload queue 643.

[0182] Now for reference Figure 18F The illustration shows a window 651 to be reviewed, which is similar to window 603 to be reviewed, and may include an ECG profile 652 to be reviewed. Figure 18F As shown, two or more ECG profiles 652 can be selected simultaneously by engaging the selection button 654 next to the corresponding ECG profile. When multiple ECG profiles 652 are selected, a tab window 653 can be displayed to add one or more tabs and / or categories to the selected ECG profiles. For example, the tab window 653 may include multiple tab options and a "Create New Tab" option, which can be selected to add a new tab to the selected ECG profiles.

[0183] Now for reference Figure 18G The illustration shows a window 661 to be reviewed, which is similar to window 603 and may include an ECG profile 662. ECG profile 662 may be similar to profile 611, as it may include a title, ECG information, and at least one label. For example, label 663 may indicate that a notification has been added to the ECG data corresponding to profile 662. The label identified in label 663 may also be associated with the ECG data when the ECG data is displayed as pulsating bands in different interfaces. For example, now refer to... Figure 18H Interface 671 can display various pulsation bands, such as pulsation band 672. Pulsation band 672 may correspond to ECG data associated with ECG profile 662. Additional pulsation bands 672 may include notifications 673, which may be the same as label 663 and / or may include a marker icon. Notification 673 may indicate the same notification as label 663. In one example, the notification may indicate the presence of an episode in the ECG band. Notifications may be user-generated (e.g., physician notifications) or may be automatically generated (e.g., when an abnormality, episode, condition, etc., is detected) and assigned to the ECG profile. Adding notifications to ECG data may cause marked ECG bands or portions thereof to be automatically added to the final generated report on the ECG data.

[0184] Now for reference Figure 19A The illustration shows an interactive display 701. Interactive display 701 includes a first graphics window 702 similar to a first graphics window 124 and a second graphics window 703 similar to a second graphics window 125. Interactive display 701 may also include a third graphics window 704, which may include analysis or additional information about multiple pulsations in the first graphics window 702. For example, the third graphics window 704 may illustrate one or more morphologies, each morphology including multiple pulsation bands that have been combined based on some similarity matching or otherwise, as described above. The interactive display may include a lead button 706, which opens a lead window 705 when engaged. Lead window 705 may include a list of available leads to be selected and viewed in the second graphics window 703 and / or the third graphics window 704. Figure 19A As shown, bipolar lead 1 can be selected, and bipolar lead 2 and biopolar lead 3 can also be used for display in the second graphics window 703 and / or the third graphics window 704, but can be left unselected. The "Add Lead" button 707 can be engaged to simultaneously display ECG data from the second lead.

[0185] Now for reference Figure 19B The interactive display 701 is illustrated as having two leads that are selected to be displayed in a second graphics window 703 and / or a third graphics window 704. Specifically, in Figure 19A After lead button 707 is engaged, a second lead button 711 may appear in addition to lead window 712. Lead window 712 may be similar to lead window 705 and may include the same list of available leads to be selected and displayed in the second graphics window 703 and / or the third graphics window 704. Using lead buttons 706 and 711, two different leads can be selected from the list of available leads for display in the second graphics window 703 and / or the third graphics window 704. These two different leads may correspond to the same time frame and may have the same sampling frequency. The leads may be two different leads placed at different locations on the patient's body, so even if the cardiac activity detected by these two different leads is the same, they may detect slightly different ECG data.

[0186] like Figure 19BAs shown, the second graphics window 703 and the third graphics window 704 can display simultaneous ECG data (e.g., strips) from two different leads, such that the detection data are shown simultaneously along the horizontal "x" axis. Specifically, ECG strips 713 and 714 can be displayed in the third graphics window 704, and ECG strips 715 and 716 can be displayed in the second graphics window 703. ECG strip 713 can correspond to ECG strip 715 and ECG strip 714 can correspond to ECG strip 716. It should be understood that displaying two different ECG strips simultaneously may facilitate a better understanding of ECG events, episodes, abnormalities, irregularities, and / or conditions. For example, noise may be detected in one lead, and if the noise is not detected in the other lead, it can be ignored. In another example, due to the position of the second lead relative to the first lead, an abnormality may be weak in one ECG strip but much more pronounced in the second ECG strip. It should also be understood that, using lead window 712, different leads other than the two shown can be selected for further comparison. It should also be understood that, while two leads are shown in the second graphics window 703 and the third graphics window 704, more than two leads can be displayed (e.g., 3 leads, 4 leads, 5 leads, etc.).

[0187] Now for reference Figure 20 The illustration shows an exemplary process for determining an atrial fibrillation risk score. Figure 20 The procedure illustrated in the diagram can be used to determine the risk of atrial fibrillation in the near future (e.g., within the next 7 days) based on ECG data that does not involve any atrial fibrillation events. It should be understood that ECG data can be generated from fewer than 12 leads, such as 1-3 leads, and / or the ECG recording device can be a portable device. Figure 20 Some or all of the steps in the defined process can be executed in a distributed manner across any number of devices (e.g., computing devices and / or servers). For example, Figure 20 The process illustrated in the figure can be derived from the above text regarding... Figure 2 The described server 15 executes the procedure. Some or all of the steps in the procedure may be optional and may be executed in a different order.

[0188] To start Figure 20The process described herein may involve determining an ECG signal including ECG data at step 802. For example, this may involve receiving raw ECG data at an ECG platform. The raw ECG data may optionally be preprocessed as described above (e.g., to remove noise). The ECG data may be generated within a specific time period (e.g., 24 hours) and / or may be generated from a 24-hour Holter monitor. Alternatively, the ECG data may be a portion (e.g., 24 hours) of a longer recording (e.g., a 7-day Holter monitor recording). It should be understood that any other recording time period may be used.

[0189] Upon receiving and / or determining ECG data, at step 804, at least one delineation algorithm may be used to perform delineation to analyze and / or process the ECG data. For example, one or more algorithms and / or neural networks may be trained to perform delineation and process the ECG data to determine QRS initiation. After step 804, steps 806-812, 814-828, and / or 830-836 may be performed sequentially or simultaneously. At step 806, a beat sequence may be determined. For example, based on the QRS initiation, a beat sequence may be identified, and based on this information, the ECG data may be segmented, and / or the segments of the ECG data may be stacked or otherwise grouped together. At step 808, features from each beat may be extracted. For example, each beat (e.g., a segment of ECG data) may be processed by one or more neural networks (e.g., a deep neural network) to extract features from each beat. A model (e.g., a neural network) may be trained to determine features that can indicate and / or predict or otherwise inform the risk of atrial fibrillation.

[0190] At step 810, beats can be compared based on features extracted from each beat. For example, beats with certain features can be identified, trends or patterns of beats with certain features can be determined, and / or beats with similar features can be grouped together. At step 812, an atrial fibrillation risk score can be generated based on the features extracted at step 808 and / or the comparison of beats at step 810. For example, the presence of certain features determined at step 808 and / or trends or patterns of beats with certain features identified at step 810 can be used to determine the atrial fibrillation risk score at step 812. The atrial fibrillation risk score can indicate the risk of an atrial fibrillation event in the near future (e.g., in the next 7 days).

[0191] Referring again to step 804, after the delineation is performed, at step 814 at least one classification algorithm can be used to perform classification to analyze and / or process one or more outputs of the ECG data and / or the delineation algorithm. For example, the ECG data and / or one or more outputs of the delineation algorithm (e.g., QRS initiation) can be analyzed to classify the ECG data or the portion corresponding to the pulsation as normal, PAC, or PVC. After step 804, steps 816-828 and 830-836 can be performed sequentially or simultaneously.

[0192] At step 816, timing data for the pulses detected in the ECG data can be determined. For example, a timestamp or other time data can be determined for each pulse. The timestamp can be the time from the start of the recording when the pulse occurs. At step 818, heart rate data based on the identified pulses can be determined. Heart rate data can be determined by identifying the RR interval for each pulse based on the timestamp determined for each pulse. At step 820, a heart rate-time (HR-time) matrix can be determined. The HR-time matrix can be formed from the time data determined at step 816 and the heart rate data determined at step 818. For example, each row in the HR-time matrix can represent a bin of heart rate values, and each column can represent a bin of timestamps, as described above regarding... Figure 11 and Figure 12 The methods described are similar.

[0193] At step 824, a graphical representation of the HR-time matrix can be generated. This graphical representation can be constructed using the pulsation categories identified at step 814 (e.g., normal, PAC, and PVC). For example, Figure 21 The diagram below illustrates a graphical representation of the HR-time matrix. (For example...) Figure 21 As shown, the graphical representation 850 may include time along the x-axis and may include multiple heart rate points plotted relative to time. The graphical representation 850 may include identifiers 852 and 853, which can identify a PAC or PVC relative to time on the graphical representation 850. It should be understood that similar graphical representations may be generated using only the beats in certain categories (such as PAC, PVC, etc.).

[0194] Refer again Figure 20At step 826, the graphical representation generated at step 824 can be processed by one or more neural networks trained to determine the risk or likelihood of atrial fibrillation. For example, the one or more neural networks can be DNNs and / or CNNs. At step 828, a score indicating the risk of atrial fibrillation can be generated by the one or more neural networks used to process the graphical representation. In one example, the one or more neural networks can additionally or alternatively process the graphical representation using only beats classified as PACs and / or PVCs.

[0195] Furthermore, following step 814, patient information can be determined at step 830. For example, information about the patient corresponding to the ECG data determined at step 802, such as age, sex, patient history, family history, etc., can be determined. At step 832, features such as the number of PACs, the number of PVCs, heart rate statistics, and age can be determined. For example, heart rate statistics could be maximum heart rate, minimum heart rate, and / or average heart rate. In one example, one or more neural networks can be used to extract and / or determine features associated with the risk of atrial fibrillation. For example, one or more neural networks can be used to process ECG datasets known to have atrial fibrillation at a later time (e.g., several days later) but not in the ECG signal to determine features (e.g., the number of PACs, the number of PVCs, HR statistics, age, and / or sex) associated with ECG data of patients who do not have atrial fibrillation but are known to have experienced atrial fibrillation at a later time (e.g., within the next 7 days).

[0196] At step 834, the features identified in step 832 can be applied to a classifier (e.g., one or more neural networks) to ultimately determine a risk score for this ECG data and patient information. For example, the classifier may identify the presence of one or more features that can be used to inform the patient of the likelihood or risk of experiencing atrial fibrillation in the future (e.g., within the next 7 days). In one example, the classifier could be a DNN, logistic regression, and / or random forest.

[0197] At step 838, the atrial fibrillation risk score determined in steps 812, 828, and / or 836 can be used to determine a final atrial fibrillation risk score, which can indicate whether the patient will experience one or more atrial fibrillation events in the near future (e.g., in the next 2-3 days, the next 7 days, next month, etc.). It should be understood that the ECG data determined at step 802 may not include any atrial fibrillation events, but the scores determined in steps 812, 828, 836, and 838 can indicate the likelihood or risk of future atrial fibrillation events.

[0198] The atrial fibrillation risk scores determined at steps 812, 828, and / or 836 can be input into a model (e.g., one or more neural networks) that is trained to determine the final atrial fibrillation risk score. In another example, the risks determined at steps 812, 828, and 836 can be averaged, or alternatively, a weighted average or combined score can be determined. Alternatively, these risks can be combined in any other way to determine the final atrial fibrillation risk score at step 838.

[0199] At optional step 839, the duration of ECG monitoring can be adjusted based on the ultimate atrial fibrillation risk. For example, if the ultimate atrial fibrillation risk score indicates a high risk of developing atrial fibrillation in the near future (e.g., the ultimate atrial fibrillation risk meets and / or exceeds a certain threshold indicating a high risk of atrial fibrillation within the next 7 days), the ECG processing system can enable the ECG recorder to set, extend, and / or adjust ECG monitoring to capture predicted atrial fibrillation events (e.g., within the next 7 days). For example, the ECG processing system can enable a dynamic recording device to record ECG data over a period of time, thus capturing predicted atrial fibrillation events.

[0200] It should be understood that any operation described above herein can be implemented, at least in part, as computer-readable instructions stored on a computer-readable storage medium. When executed by a processor, the computer-readable instructions cause the node to perform the operation. It should be understood, of course, that the embodiments described herein are illustrative, and components can be arranged, replaced, combined, and designed in a wide variety of different configurations, all of which are contemplated and fall within the scope of this disclosure.

[0201] The foregoing description of illustrative embodiments has been presented for purposes of illustration and description. This document is not intended to be exhaustive or limiting with respect to the precise forms disclosed, and modifications and variations are possible in light of the foregoing teachings, or may be obtained from practice of the disclosed embodiments. It is intended that the scope of the invention be defined by the claims.

Claims

1. A computerized system for analyzing electrocardiogram (ECG) data of a patient, said computerized system being configured to: Analyze the ECG data to determine the presence of atrial fibrillation; If the atrial fibrillation is determined to be present based on the analysis of the ECG data, information for identifying the presence of the atrial fibrillation is generated for display. and If the analysis of the ECG data determines that the atrial fibrillation does not exist, the ECG data is further analyzed to determine a risk score indicating the future risk of the atrial fibrillation for display. The computerized system is further configured as follows: Perform a delineation (804) on the ECG data to determine multiple pulses (806); Extract multiple first features (808) from the multiple pulsations; A first risk score (812) is determined based on the plurality of first characteristics, the first risk score indicating future atrial fibrillation in the patient; The ECG data is classified (814) to classify the multiple beats as normal, atrial premature beat complex (PAC) or ventricular premature beat complex (PVC). Determine multiple timestamps corresponding to the multiple pulsations (816); Multiple heart rate values ​​are determined based on the multiple timestamps (818); The matrix (820) is determined based on the multiple timestamps and the multiple heart rate values. Generate a graphical representation of the matrix (824); and At least one neural network is used to process the graphical representation of the matrix (826) to determine a second risk score indicating future atrial fibrillation for the patient.

2. The computerized system according to claim 1, wherein, Multiple QRS initial values ​​are determined for the ECG data.

3. The computerized system according to claim 1, wherein, The at least one neural network is at least one deep neural network (DNN).

4. The computerized system according to claim 1, wherein the computerized system is further configured to: Determine patient information corresponding to the patient, the patient information including one or more of the patient's age and gender; To identify multiple second features indicative of atrial fibrillation in the ECG data; The multiple second features are applied to a classifier to determine a third risk score indicating future atrial fibrillation for the patient.

5. The computerized system according to claim 4, wherein, The classifier is one or more of the following: DNN, logistic regression, and random forest.

6. The computerized system according to claim 4, wherein, The risk score is based on one or more of the following: the first risk score, the second risk score, and the third risk score.

7. The computerized system according to claim 4, further configured to: The average of the first risk score, the second risk score, and the third risk score is determined to determine the risk score.

8. The computerized system according to claim 4, further configured to: The first risk score, the second risk score, and the third risk score are applied to a first neural network trained to determine the risk score.

9. The computerized system according to claim 1, wherein, The risk score indicates the likelihood of atrial fibrillation within a set time period, and the computerized system is further configured to: The ECG recorder captures a second ECG data within the set time period.

10. A non-transient computer-readable storage medium configured to store instructions thereon, the instructions causing the at least one processor to perform the following operations when loaded by the at least one processor: Analyze the patient's electrocardiogram (ECG) data to determine the presence of atrial fibrillation; If the atrial fibrillation is determined to be present based on the analysis of the ECG data, information for identifying the presence of the atrial fibrillation is generated for display. and If the analysis of the ECG data determines that the atrial fibrillation does not exist, the ECG data is further analyzed to determine a risk score indicating the future risk of the atrial fibrillation for display. The instructions are further configured to cause the at least one processor to perform the following operations: Perform a delineation (804) on the ECG data to determine multiple pulses (806); Extract multiple first features (808) from the multiple pulsations; A first risk score (812) is determined based on the plurality of first characteristics, the first risk score indicating future atrial fibrillation in the patient; The ECG data is classified (814) to classify the multiple beats as normal, atrial premature beat complex (PAC) or ventricular premature beat complex (PVC). Determine multiple timestamps corresponding to the multiple pulsations (816); Multiple heart rate values ​​are determined based on the multiple timestamps (818); The matrix (820) is determined based on the multiple timestamps and the multiple heart rate values. Generate a graphical representation of the matrix (824); and At least one neural network is used to process the graphical representation of the matrix (826) to determine a second risk score indicating future atrial fibrillation for the patient.

11. The non-transient computer-readable storage medium according to claim 10, wherein, Multiple QRS initial values ​​are determined for the ECG data.

12. The non-transient computer-readable storage medium according to claim 10, wherein, The at least one neural network is at least one deep neural network (DNN).

13. The non-transient computer-readable storage medium of claim 10, further configured to cause the at least one processor to perform the following operations: Determine patient information corresponding to the patient, the patient information including one or more of the patient's age and gender; To identify multiple second features indicative of atrial fibrillation in the ECG data; The multiple second features are applied to a classifier to determine a third risk score indicating future atrial fibrillation for the patient.

14. The non-transient computer-readable storage medium according to claim 13, wherein, The classifier is one or more of the following: DNN, logistic regression, and random forest.

15. The non-transient computer-readable storage medium according to claim 13, wherein, The risk score is based on one or more of the following: the first risk score, the second risk score, and the third risk score.

16. The non-transient computer-readable storage medium of claim 13, further configured to cause the at least one processor to perform the following operations: The average of the first risk score, the second risk score, and the third risk score is determined to determine the risk score.

17. The non-transient computer-readable storage medium of claim 13, further configured to cause the at least one processor to perform the following operations: The first risk score, the second risk score, and the third risk score are applied to a first neural network trained to determine the risk score.

18. The non-transient computer-readable storage medium according to claim 10, wherein, The risk score indicates the likelihood of atrial fibrillation within a set time period, and the non-transient computer-readable storage medium is also configured to cause the at least one processor to perform the following operations: The ECG recorder captures a second ECG data within the set time period.

Images