Transforming time series data for application to machine learning models

Abstract

Techniques are described for converting time series data, such as electrocardiogram (ECG) data, into a format suitable for application to a machine learning model and applying the converted data as input to the machine learning model to determine a health status of a subject. In various embodiments, a two-dimensional image is generated (601) based on vectorcardiogram (VCG) data. The VCG data is based on electrocardiogram (ECG) data measured directly or from the subject. The two-dimensional image is applied (612) as input to a machine learning model to generate an output. The machine learning model is configured for use in processing the two-dimensional image. A health status of the subject is determined (614) based on the output.

Description

[Technical Field]

[0001]

[0001] Various embodiments described herein are generally directed to medicine and / or artificial intelligence. More specifically, but not exclusively, various methods and apparatus disclosed herein relate to converting time series data, such as electrocardiogram (ECG) waveforms, into a different format suitable for application to machine learning models. [Background technology]

[0002]

[0002] Modern artificial intelligence techniques, such as deep learning, have many applications. Image processing is one of the most developed. Many machine learning models (e.g., convolutional neural networks) already exist that perform image processing on digital images to perform tasks such as segmentation, object recognition, handwriting recognition, and face recognition. In their most basic and / or common form, many of these models do not operate on time-series data, i.e., a sequence of inputs each corresponding to a point in time. This makes them adaptable for use across a variety of domains.

[0003]

[0003] Techniques exist for processing time series data, such as electrocardiogram (ECG) waveforms, to perform tasks such as clinical decision-making, diagnosis, interpretation, reducing medical errors, and facilitating patient care processes. However, artificial intelligence models that process time series data, such as recurrent neural networks, are more complex than models that process single-shot data, such as feedforward neural networks. This increased complexity makes them more difficult to train and less adaptable across domains. Summary of the Invention [Problem to be solved by the invention]

[0004]

[0004] The present disclosure is directed to methods and apparatus for converting time series data, such as electrocardiogram (ECG) data, into a form suitable for application to machine learning models. [Means for solving the problem]

[0005] For example, in various embodiments, ECG data is acquired from a subject using various numbers of leads (12 leads or another number of leads). This ECG data is converted using techniques described herein into a format suitable for application to a machine learning model, such as a convolutional neural network ("CNN"), that preserves both the temporal and spatial information contained in the ECG data. That is, the converted format preserves both the rhythm and morphology of the ECG waveform.

[0006] For example, ECG data (e.g., waveforms) are used to generate a two-dimensional digital image that preserves both the rhythm and morphology of the ECG data. In some cases, the two-dimensional image may include multiple layers or channels, e.g., corresponding to different colors. For example, one type of digital image may include a red channel, a green channel, and a blue channel (RGB) for each pixel. Other combinations of channels are also contemplated. As described in more detail herein, various spatial and / or temporal aspects of the ECG data can be encoded into these different channels of the two-dimensional data. The two-dimensional digital image can then be applied as input to an appropriately trained machine learning model (e.g., a CNN) to generate an output indicative of the health status of the subject from whom the ECG data was obtained. Furthermore, the machine learning model can be trained using similar two-dimensional data that has been labeled by a medical professional before the machine learning model is trained.

[0007] In some implementations, transfer learning is used to expedite the training process of a machine learning model by leveraging at least a portion of an existing, pre-trained machine learning model. A variety of pre-trained machine learning models are widely available for tasks such as image and / or object classification. For example, some existing models are trained to identify and / or classify various types of objects, such as animals, furniture, tools, and vehicles, while others are trained to perform tasks such as handwriting recognition. In some embodiments, these existing models, particularly their “upstream” hidden layers that identify / classify relatively abstract visual features, can be leveraged to train additional downstream hidden layers that identify aspects of two-dimensional digital images generated from time-series data using the techniques described herein.

[0008] Various techniques can be used to convert time-series data, such as ECG data, into two-dimensional image data. For example, in some embodiments, the ECG data is in the form of 12 waveforms corresponding to each of the 12-lead ECG data. Vectorcardiogram (VCG) data can be generated from these ECG data or from an intermediate form of these data. In addition to calculating VCG leads from a standard ECG lead system, VCG leads can also be recorded directly. In a standard 12-lead ECG, four waveforms may be derived from the other eight waveforms, so in some embodiments, VCG leads are derived using only eight waveforms. The VCG can also be derived from other ECG lead systems with fewer or more than eight leads. In some embodiments, each of the 12 waveforms is first converted into a single representative beat (e.g., average or mean). The 12 representative beats are then converted into three VCG beats, each corresponding to a cardiac vector in one dimension of three-dimensional (3D) space. Three VCG projections are then determined, each representing a corresponding one of the three VCG beats on the spatial plane. These projections are then combined into a 2D image, e.g., each VCG projection is stored in a corresponding channel (e.g., RGB) of the 2D image.

[0009]

[0008] Generally, in one aspect, a method implemented using one or more processors includes generating a two-dimensional image based on VCG data, where the VCG data is either directly recorded or is based on ECG data measured from the subject; applying the two-dimensional image as an input to a machine learning model to generate an output, where the machine learning model is configured for use in processing the two-dimensional image; and determining a health status of the subject based on the output.

[0010] In various embodiments, the ECG data includes a plurality of waveforms corresponding to a plurality of ECG leads. In various embodiments, the method further includes converting each of the plurality of waveforms into a corresponding representative beat. In various embodiments, the method further includes converting the plurality of representative beats into three VCG beats, each VCG beat corresponding to a cardiac vector in one dimension of 3D space. In various embodiments, the method further includes upsampling the three VCG beats.

[0011] In various embodiments, the method further includes determining three VCG projections, each VCG projection representing a corresponding one of the three VCG beats on a spatial plane corresponding to a corresponding dimension of the 3D space. In various embodiments, the method further includes encoding the three VCG projections into three corresponding layers of a two-dimensional image. In various embodiments, the three corresponding layers include red, green, and blue. In various embodiments, the ECG data includes single-lead data obtained from a wearable device worn by the subject.

[0012]

[0011] Additionally, some implementations include one or more processors of one or more computing devices, the one or more processors operable to execute instructions stored in associated memory, the instructions causing any of the above methods to be performed. Some implementations also include one or more non-transitory computer-readable storage media having stored thereon computer instructions executable by the one or more processors to perform any of the above methods.

[0013] It should be understood that all combinations of the aforementioned concepts with the additional concepts described below (where such concepts are not mutually inconsistent) are considered part of the inventive subject matter disclosed herein. In particular, all combinations of the subject matter of the back claims set forth at the end of this disclosure are considered part of the inventive subject matter disclosed herein. It should also be understood that terms explicitly used in this specification that are also included in the disclosures incorporated by reference have a meaning most consistent with the specific concepts disclosed herein. [Brief explanation of the drawings]

[0014] In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis generally being placed upon illustrating various principles of the embodiments described herein.

[0015] [Figure 1] FIG. 1 illustrates an example environment in which selected aspects of the present disclosure may be implemented. [Figure 2] FIG. 2 shows an example of 12-lead ECG data and a representative beat generated therefrom. [Figure 3]

[0016] FIG. 3 shows an example of how the representative beat from FIG. 2 can be transformed into several (eg, three) VCG beats. [Figure 4]

[0017] FIG. 4 shows an example of how the VCG beats from FIG. 3 can be used to generate a multi-channel two-dimensional image. [Figure 5]

[0018] Figure 5 shows one example of how transfer learning can be used to train aspects of an existing machine learning model to detect health conditions. [Figure 6]

[0019] FIG. 6 illustrates an example of a method for practicing selected aspects of the present disclosure. [Figure 7]

[0020] FIG. 7 shows an example of a computer architecture. [Figure 8]

[0021] FIG. 8 shows a confusion matrix revealing the accuracy rate obtained using the techniques described herein. DETAILED DESCRIPTION OF THE INVENTION

[0016]

[0022] Modern artificial intelligence (AI) techniques, such as deep learning, have numerous applications, with image processing being one of the most developed. While these deep learning models are relatively adaptable across domains, they may not process time series data, such as electrocardiogram (ECG) waveforms. Furthermore, AI models that process time series data are more complex, are not readily available, and, even when available, are not easily adaptable to new domains. In light of the above, various embodiments and implementations of the present disclosure are directed to converting time series data, such as ECG waveforms, into a format suitable for application to non-time series machine learning models.

[0017]

[0023] 1 illustrates an example environment in which selected aspects of the present disclosure may be implemented, according to various embodiments. The computing devices illustrated in FIG. 1 include, for example, one or more of a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device in a user's vehicle (e.g., an in-vehicle communication system, an in-vehicle entertainment system, an in-vehicle navigation system), a standalone interactive speaker (which may in some cases include a vision sensor), a smart appliance such as a smart television (or a standard television equipped with a network dongle with automated assistant functionality), and / or a user wearable device that includes a computing device (e.g., a user's watch with a computing device, a user's glasses with a computing device, a virtual reality or augmented reality computing device, etc.). Additional and / or alternative computing devices may be provided.

[0018]

[0024] In FIG. 1 , a first subject 1001 is monitored with an ECG device 102, e.g., in a hospital, to acquire time-series data in the form of ECG data. For example, the ECG data may include 12-lead ECG data as shown in FIG. 1 . The ECG data may be provided to and / or stored in a hospital information system (HIS) 104 or another similar medical system, e.g., as part of an electronic health record (EHR) for the first subject 1001. While FIG. 1 illustrates the ECG data being provided directly to the HIS 104, this is not intended to be limiting. In various embodiments, the ECG data may be provided to the HIS 104 via one or more networks 108. The networks 108 may include one or more local area networks and / or one or more wide area networks, such as the Internet.

[0019]

[0025] The techniques described herein are not limited to 12-lead ECG data (or any ECG data for that matter). For example, a second subject 1002 is monitored by a wearable ECG device in the form of a smartwatch 114. The smartwatch 114 provides ECG data for a reduced number of leads, such as one lead, to an intermediate computing device, such as a laptop computer 115 operated by the second user 1002. The laptop computer 115 provides this ECG data over the network 108 to the HIS 104 (or in some cases the smartwatch 114 may provide the ECG data directly to the HIS 104 over the network 108).

[0020]

[0026] Training system 120 and inference system 124 are implemented using any combination of hardware and software to create, manage, and / or apply machine learning models stored in machine learning (“ML”) model database 122. Training system 120 applies training data, such as two-dimensional images generated using the techniques herein, as input to one or more of the models in database 122 to generate outputs. To determine errors associated with the models, outputs generated using the training data are compared with labels associated with the training data. The labels of the training examples indicate, for example, the presence or absence and / or probability of a health condition for the subject from whom the training examples were generated. For example, training system 120 uses these errors to train models using techniques such as backpropagation and gradient descent (stochastic or otherwise).

[0021]

[0027] The inference system 124 uses the trained machine learning models in the database 122 to infer the health status of a subject based on the two-dimensional images generated using the techniques described herein. In some embodiments, the training system 120 and / or the inference system 124 may be implemented as part of a distributed computing system, sometimes referred to as the "cloud," although this is not required.

[0022]

[0028] 1 also illustrates a medical professional, such as a physician 112, operating a computing device 110 to infer health conditions of subjects (e.g., 1001, 1002) as described herein. In particular, the computing device 110 is connected to a network 108 and interacts with an inference system 124 to infer health conditions as described herein. For example, the physician 112 can infer a health condition of a first subject 1001 based on 12-lead ECG data obtained from the first subject 1001. Similarly, the physician 112 can make similar inferences about a second subject 1002 based on single-lead ECG data obtained by a smartwatch 114.

[0023]

[0029] In some embodiments, the functionality for making these inferences is provided as part of a software application that assists the physician 112 in making a diagnosis, such as a clinical decision support (CDS) application. In some such embodiments, the physician 112 can rely on the inference as a “second opinion” to strengthen or challenge his or her own medical findings. Alternatively, the inference can be used as an ECG screening test to separate normal from abnormal ECG signals, for example, in a presumably healthy population (e.g., students, athletes, soldiers), thereby allowing further investigation to be performed on subjects with abnormal ECG data signals. Additionally or alternatively, the techniques described herein can be incorporated into medical devices that incorporate ECG signals, such as exercise stress testing machines, defibrillators, electrocardiographs, bedside monitors, etc.

[0024]

[0030] In some embodiments, subjects 1001, 1002 themselves can utilize the disclosed techniques to determine the likelihood of having a health condition, e.g., whether their heart beat is normal or abnormal. For example, second subject 1002 operates laptop computer 115 to interface with inference system 124 and receive changed state inferences from inference system 124. In some embodiments, one or more trained machine learning models are distributed from database 122 to a remote computing device (e.g., laptop computer 115), e.g., to comply with privacy laws and regulations. This allows the remote computing device to perform selected aspects of the present disclosure on locally acquired time series data (e.g., ECG data) without transferring the ECG data to, for example, the "cloud."

[0025]

[0031] 2-4 show an example of how ECG data (e.g., generated by ECG device 102 of FIG. 1) can be converted into a format suitable for use with an image processing machine learning model such as a CNN, i.e., a two-dimensional digital image having three channels (e.g., red, green, and blue). In this example, the ECG data is in the form of 12-lead ECG data, although this is not intended to be limiting. ECGs with fewer leads, such as single-lead ECG data generated by smartwatch 114, can also be converted into a VCG and then converted into a two-dimensional image.

[0026]

[0032] In FIG. 2, 12-lead ECG data 240 is shown on the left and includes I, II, III, aVR, aVL, aVF, and V1-V6. In this example, 10 seconds of ECG waveforms are collected, although other time intervals are possible. At 242 on the right side of FIG. 2, each lead waveform is converted to one "representative" beat per lead. A representative beat is, for example, the average of all beats in that lead's waveform. This process attempts to preserve all the details of the ECG signal while mitigating random fluctuations caused by, for example, muscle sounds.

[0027]

[0033] In FIG. 3 , twelve representative beats 242 are converted into several VCG beats 244. In FIG. 3 , three VCG beats 244 are generated, one for each dimension in 3D space, but this is not intended to be limiting. In some embodiments, a three-dimensional VCG is synthesized from the 12-lead ECG 242 using matrix multiplication. Intuitively, converting to VCG beats 244 involves recording the direction and magnitude of the cardiac electrical forces using a continuous series of vectors that form a curve around a center in three-dimensional space, as shown at 246 on the right side of FIG. 3 . Thus, each VCG beat represents a cardiac vector in one of the three dimensions. In some embodiments, the VCG leads are recorded directly. In some embodiments, to improve spatial resolution, the VCG beats can be upsampled, for example, to 1000 samples / second, although this is not required.

[0028]

[0034] FIG. 4 illustrates how a VCG vector in three-dimensional space 246 can be converted into a two-dimensional digital image 250 with three channels. As an example, an RGB image is stored as an m×n×3 data array that defines the red, green, and blue color components of each pixel. The color of each pixel is determined by the combination of the red, green, and blue intensities stored at that pixel's location in each color plane. In some embodiments, the three-dimensional VCG signal is projected onto three spatial planes, such that each projection can be used as a layer of the RGB image. For example, projection 248R on the XY plane can be used as the red channel, projection 248G on the XZ plane can be used as the green channel, and projection 248B on the ZY plane can be used as the blue channel. When encoding these projections into RGB layers, in some embodiments, the projections are rotated to reduce overlap in digital image 250. To maintain consistency between different recordings, the rotation angle of all recordings analyzed by the system is fixed.

[0029]

[0035] Additionally, in some embodiments, the orientation of the individual VCG projections is preserved, as indicated by the arrows (visible in two-dimensional image 250) shown as part of each projection 248R-248B, and these orientations are utilized as additional features to provide as input to a CNN to infer the subject's medical condition.

[0030]

[0036] By performing the transformations shown in Figures 2-4, the entire dataset of ECG signals can be converted into a dataset of VCG images. This new dataset can then be used to fine-tune a pre-trained neural network, such as a CNN, to perform classification of new images in a process known as "transfer learning." Transfer learning is often faster and easier than training a machine learning model from scratch, for example, with randomly initialized weights. Transfer learning also requires less training data to train a model to an acceptable level, particularly compared to the amount of training data required to train a model from scratch.

[0031]

[0037] For example, some machine learning models have already been trained on millions of images and can classify images into thousands of object categories (e.g., keyboards, coffee mugs, pencils, various animals, etc.). Such networks have learned rich feature representations for a wide range of images. Therefore, such networks can be fine-tuned to classify, for example, ECG signals as "normal" or "abnormal."

[0032]

[0038] An example of this is shown in Figure 5. At the top of Figure 5, a trained network 560 is divided into two portions: an upstream portion 562A and a downstream portion 562B. The upstream portion 562A includes any number of hidden layers or other machine learning model components that have been pre-trained to identify and / or classify relatively abstract visual features (e.g., curves, lines, patterns, etc.). The downstream portion 562B includes hidden layers or other components that identify and / or classify less abstract features and / or features that are readily understood by humans (e.g., cats, dogs, chairs, keyboards, handwritten characters (in a handwriting recognition scenario), etc.).

[0033]

[0039] As shown in the center of FIG. 5 , downstream layer 562B may be replaced with a new layer 564. The new layer 564 is initially unshaded to indicate that it will be trained to classify two-dimensional images (representing ECG data) created using the techniques described herein. In various embodiments, at this point, a training dataset created from ECG signals of a population of subjects is created and / or acquired. Specifically, using the techniques described herein, ECG signals known to be abnormal and normal can be converted into two-dimensional images as described above. These two-dimensional images are labeled as normal or abnormal, for example, based on a medical professional's diagnosis of the original ECG signals. These two-dimensional images can then be used to train a new network 560 (562A+564) using the training dataset to classify images, for example, as normal or abnormal. As shown at the bottom of FIG. 5 , the new layer 564′ is shaded to indicate that it has been trained using the ECG training dataset.

[0034]

[0040] Below is one non-limiting example of how transfer learning can be applied to leverage existing models to classify ECG data (which has been converted into a two-dimensional image as described herein), for example, as normal or abnormal, or to infer other health conditions, such as atrial fibrillation, heart murmurs, hypertrophy, etc.

[0035]

[0041] First, a pre-trained neural network 560 is obtained. As shown in FIG. 5, this pre-trained network 560 includes three final layers 562B that were trained to identify 1000 different object classes. These three layers are then fine-tuned and / or replaced for a new classification problem (e.g., classifying a 12-lead, 10-second ECG recording as having a "normal" or "abnormal" morphology). All layers except the last three are extracted from the pre-trained network. These extracted layers are combined with three new final layers, as shown at 564 in FIG. 5. These new layers 564 may include a fully connected layer, a softmax layer, and a classification output layer with two nodes corresponding to normal and abnormal.

[0036]

[0042] In some embodiments, training options and / or parameters are set to allow the new layer 564 to learn much faster than the transferred layer 562A. This can be achieved, for example, by setting the initial learning rate of the transferred layer 562A to a small value compared to the last three newly added layers 564. When performing transfer learning, it is not necessary to train for the same number of epochs (an epoch is a full training cycle across the entire training data set) because most of the network (the extracted layer 562A) has already been trained using a much larger training data set.

[0037]

[0043] 6 illustrates a flowchart of an example method 600 for practicing selected aspects of the present disclosure. The operations of FIG. 6 are performed by one or more processors, such as one or more processors of the various computing devices / systems described herein. For convenience, the operations of method 600 are described as being performed by a system configured with selected aspects of the present disclosure. Other implementations may include additional operations other than those shown in FIG. 6, may perform the steps of FIG. 6 in a different order and / or in parallel, and / or may omit one or more of the operations of FIG. 6.

[0038]

[0044] In 601 (including steps 602-610), the system generates a two-dimensional image based on ECG data measured from a subject. In some embodiments, the operations of step 601 are performed by inference system 124 or training system 120, as appropriate. In other embodiments, the operations may be performed by another component not shown in FIG. 1, such as a transformation system.

[0039]

[0045] In step 602, the system converts each of several waveforms of the multi-lead ECG data into a corresponding representative beat, as shown in FIG. 2. As previously mentioned, this conversion can be performed in various ways, such as averaging beats over a 10-second time interval for each lead. In step 604, the system converts the (e.g., 12) representative beats converted in step 602 into several (e.g., 3) VCG beats. In various embodiments, each VCG beat corresponds to a one-dimensional cardiac vector in 3D space, as shown in FIG. 3.

[0040]

[0046] In some embodiments, in step 606, the system upsamples the (e.g., three) VCG beats converted in step 604, e.g., to improve spatial resolution. In other embodiments, this upsampling may be omitted. In step 608, the system determines several (e.g., three) VCG projections based on the VCG beats. Each VCG projection represents a corresponding one of the VCG beats on a spatial plane corresponding to a corresponding dimension in 3D space. This is illustrated in FIG. 4.

[0041]

[0047] In step 610, the system encodes the three VCG projections into three corresponding layers or channels of a two-dimensional image, for example, one VCG projection in one spatial plane is encoded into the red channel of an RGB digital image, another VCG projection is encoded into the green channel of the RGB image, and another VCG projection is encoded into the blue channel of the RGB image.

[0042]

[0048] After steps 602-610, the system is ready to use the generated multi-layered 2D image to make inferences (if a model has already been trained) or to train a model. For example, in step 612, the system applies the multi-layered 2D image as input to a machine learning model, e.g., using inference system 124 or training system 120, to generate an output. In step 614 of FIG. 6, the system determines the subject's health status based on the output. For example, the health status could be a "normal" or "abnormal" heart rate, e.g., if the system is performing screening on a population presumed to be healthy.

[0043]

[0049] Alternatively, machine learning may provide more than a binary output. For example, in some embodiments, the machine learning model generates as output multiple probabilities, each associated with a different health state. In some such embodiments, the n(n >The health status of 1) is presented to medical personnel, for example, as part of a graphical user interface, as natural language output over a speaker, or as part of a report about the subject.

[0044]

[0050] In other embodiments in which a machine learning model is trained, the output generated in step 612 is compared to a label associated with the input data. For example, the input data may be ECG data that has been pre-labeled by a medical professional as, for example, "normal," "abnormal," or indicative of one or more health conditions. The difference between the output generated using the machine learning model and the label, i.e., the error, is used to train the machine learning model, for example, by training system 120.

[0045]

[0051] 7 is a block diagram of an example computing device 710 that is optionally used to perform one or more aspects of the techniques described herein. The computing device 710 typically includes at least one processor 714 that communicates with several peripheral devices via a bus subsystem 712. These peripheral devices may include a storage subsystem 724, including a memory subsystem 725 and a file storage subsystem 726, a user interface output device 720, a user interface input device 722, and a network interface subsystem 716. The input and output devices allow a user to interact with the computing device 710. The network interface subsystem 716 provides an interface to an external network and is coupled to corresponding interface devices of other computing devices.

[0046]

[0052] The user interface input devices 722 may include a keyboard, a pointing device (such as a mouse, trackball, touchpad, or graphics tablet), a scanner, a touchscreen integrated into a display, a voice recognition system, an audio input device such as a microphone, and / or other types of input devices. In general, use of the term "input device" is intended to include all possible types of devices and ways of inputting information into the computing device 710 or a communications network.

[0047]

[0053] The user interface output devices 720 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or other mechanism for creating a visible image. The display subsystem also provides non-visual displays, such as an audio output device. In general, use of the term "output device" is intended to include all possible types of devices and ways of outputting information from the computing device 710 to a user or to another machine or computing device.

[0048]

[0054] Storage subsystem 724 stores programming and data structures that provide the functionality of some or all of the modules described herein, for example, storage subsystem 724 includes logic for performing selected aspects of the method of Figure 6 or implementing various components shown in Figure 1.

[0049]

[0055] These software modules typically execute on the processor 714 alone or in combination with other processors. The memory 725 used within the storage subsystem 724 may include a number of memories, including a main random access memory (RAM) 730 used to store instructions and data during program execution, and a read-only memory (ROM) 732 where fixed instructions are stored. The file storage subsystem 726 provides persistent storage for program and data files and may include a hard disk drive, a floppy disk drive with associated removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. Modules that implement the functionality of a particular implementation may be stored on the storage subsystem 724 by the file storage subsystem 726, or on another machine accessible to the processor 714.

[0050]

[0056] Bus subsystem 712 provides a mechanism that allows the various components and subsystems of computing device 710 to communicate with each other as intended. Although bus subsystem 712 is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0051]

[0057] Computing device 710 can be of various types, such as a workstation, a server, a computing cluster, a blade server, a server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, the description of computing device 710 shown in Figure 7 is intended only as a specific example to illustrate some implementations. Many other configurations of computing device 710 are possible, having more or fewer components than the computing device shown in Figure 7.

[0052]

[0058] Figure 8 shows a confusion matrix for a "normal" vs. "abnormal" classifier trained using the techniques described herein. In this matrix, rows correspond to predicted classes and columns correspond to true classes (e.g., ground truth labels). Diagonal cells correspond to correctly classified observations. Off-diagonal cells correspond to incorrectly classified observations. Each cell shows both the number of observations and the percentage of the total number of observations.

[0053]

[0059] The rightmost column of the matrix shows the proportion of all examples predicted to belong to each class that were correctly classified (in bold) and the proportion of those that were incorrectly classified (in italics). These metrics are often called precision (or positive predictive value) and false discovery rate. The bottom row of the plot shows the proportion of all examples that belong to each class that were correctly classified and the proportion that were incorrectly classified. These metrics are often called recall (or true positive rate or sensitivity) and false negative rate. The bottom right cell of the plot shows the overall accuracy rate.

[0054]

[0060] The confusion matrix in Figure 8 reveals that applying the techniques described herein to real-world data achieved a sensitivity of 82.4% and a positive predictive value of 93.7% for detecting normal ECGs. Abnormal cases in this dataset included a variety of abnormalities in ECG morphology, including abnormal T waves, bundle branch block, PR interval shortening or prolongation, left or right ventricular hypertrophy, axis deviation, ST depression or elevation, low voltage, abnormal or absent P waves, infarction, left atrial enlargement, atrioventricular block, conduction delay, QT interval shortening or prolongation, poor R-wave enhancement, ventricular preexcitation, and other abnormal morphologies. This accuracy rate was achieved without the use of signal processing or clinical rule implementation.

[0055]

[0061] While several inventive embodiments have been described and illustrated herein, those skilled in the art will readily envision numerous other means and structures for performing the functions or obtaining the results or one or more advantages described herein. Each of these variations and modifications is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily recognize that all parameters, dimensions, materials, and configurations described herein are exemplary, and that the actual parameters, dimensions, materials, and / or configurations will depend on the particular application in which the teachings of the present invention are being used. Those skilled in the art will recognize or be able to ascertain, using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. Accordingly, the foregoing embodiments are presented by way of example only, and it should be understood that, within the scope of the appended claims and equivalents, inventive embodiments may be practiced otherwise than as specifically described and claimed. The inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. Furthermore, any combination of two or more such features, systems, articles, materials, kits, and / or methods is within the inventive scope of the present disclosure, provided that such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent.

[0056]

[0062] All definitions defined and used herein should be understood to supersede dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms. The singular forms "a," "an," and "the" should be understood to mean "at least one" when used in this specification and claims.

[0057]

[0063] The term "and / or," as used in the specification and claims, should be understood to mean "either or both" of the elements so conjunctively (i.e., elements that may be present conjunctively or disjunctively). Multiple elements listed with "and / or" should be construed in the same manner, i.e., "one or more" of the elements so conjunctively. Other elements other than the elements specifically identified by the "and / or" clause may optionally be present, whether related to the elements specifically identified. Thus, as a non-limiting example, when used in conjunction with open-ended language such as "comprising," a reference to "A and / or B" could, in one embodiment, refer to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); and so forth.

[0058]

[0064] As used in this specification and the claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as inclusive; that is, the inclusion of at least one element or list of elements, but also including two or more elements, and optionally additional unlisted items. Only terms clearly indicating the contrary (e.g., "only one of" or "exactly one of," or, when used in the claims, "consisting of") mean the inclusion of exactly one element or list of elements. In general, as used herein, the term "or" should only be interpreted as indicating exclusive alternatives (i.e., "either," "one of," "only one of," or "exactly one of") when preceded by terms of exclusivity (e.g., "either," "one of," "only one of," or "exactly one of"). "Consisting essentially of," when used in the claims, has its ordinary meaning as used in the field of patent law.

[0059]

[0065] As used in this specification and claims, the phrase "at least one," in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but does not necessarily include at least one of each element specifically listed in the list of elements, nor does it exclude any combination of elements in the list of elements. By this definition, elements other than those identified in the list of elements to which the phrase "at least one" refers may optionally be present, whether related to the identified elements or not. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B," or, equivalently, "at least one of A and / or B") can refer, in one embodiment, to at least one A (optionally including more than one A) (optionally including elements other than B) in the absence of any B; in another embodiment, to at least one B (optionally including more than one B) (optionally including elements other than A) in the absence of any A; in yet another embodiment, to at least one A (optionally including more than one A) and at least one B (optionally including more than one B) (optionally including other elements); and so on.

[0060]

[0066] It should also be understood that, unless otherwise specified, in any method claimed herein that includes more than one step or action, the order of the method steps or actions is not necessarily limited to the order in which the method steps or actions are described.

[0061]

[0067] In the claims, as well as in the above specification, all transitional phrases such as "have," "include," "carry," "have," "contain," "accompany," "hold," "consisting of," and the like, are to be understood to be open-ended, meaning including but not limited to. As set forth in Section 2111.03 of the United States Patent Office Manual of Patent Examining Procedures, only the transitional phrases "consisting of" and "consisting essentially of" are closed or semi-closed transitional phrases. Pursuant to Rule 6.2(b) of the Patent Cooperation Treaty ("PCT"), certain phrases and reference signs used in the claims are to be understood not to limit the scope.

Claims

1. 1. A method of operating a device including one or more processors, the method comprising: generating a two-dimensional image based on vector electrocardiogram (VCG) data, the VCG data being based on electrocardiogram (ECG) data directly recorded or measured from a subject, the ECG data including multiple waveforms corresponding to multiple ECG leads; applying, by the processor, the two-dimensional image as an input to a machine learning model to generate an output, the machine learning model being configured for use in processing two-dimensional images; the processor determining a health status of the subject based on the output; Including, The generating step includes: The processor converts each of the plurality of waveforms into a corresponding representative beat; the processor converting the plurality of representative beats into three VCG beats, each VCG beat corresponding to a cardiac vector in one dimension of three-dimensional (3D) space; determining, by the processor, three VCG projections, each VCG projection representing a corresponding one of the three VCG beats on a spatial plane corresponding to a corresponding dimension of the 3D space; the processor encoding the three VCG projections into three corresponding layers of the two-dimensional image; A method of operation comprising:

2. The operating method described in claim 1, further comprising a step in which the processor upsamples the three VCG beats.

3. The method of claim 1 , wherein the three corresponding layers include red, green, and blue.

4. The method of claim 1 , wherein the ECG data comprises single-lead data obtained from a wearable device worn by the subject.

5. A device including a processor and a memory, the memory storing instructions, the instructions in response to execution of the instructions by the processor causing the device to: generating a two-dimensional image based on vectorcardiogram (VCG) data, the VCG data being based on electrocardiogram (ECG) data measured directly or from a subject, the ECG data including multiple waveforms corresponding to multiple ECG leads; applying the two-dimensional image as an input to a machine learning model to generate an output, the machine learning model being configured for use in processing two-dimensional images; determining a health status of the subject based on the output; and Execute The generating step comprises: converting each of the plurality of waveforms into a corresponding one representative beat; converting the plurality of representative beats into three VCG beats, each VCG beat corresponding to a cardiac vector in one dimension of three-dimensional (3D) space; determining three VCG projections, each VCG projection representing a corresponding one of the three VCG beats on a spatial plane corresponding to a corresponding dimension of the 3D space; encoding the three VCG projections into three corresponding layers of the two-dimensional image; further comprising instructions to execute device.

6. The device of claim 5 , comprising a wearable device worn by the subject.

7. The device of claim 5 , further comprising instructions for upsampling the three VCG beats.

8. At least one non-transitory computer-readable medium containing instructions, which, upon execution of the instructions by one or more processors, cause the one or more processors to: generating a two-dimensional image based on vectorcardiogram (VCG) data, the VCG data being based on electrocardiogram (ECG) data directly recorded or measured from a subject, the ECG data including multiple waveforms corresponding to multiple ECG leads; applying the two-dimensional image as an input to a machine learning model to generate an output, the machine learning model being configured for use in processing two-dimensional images; determining a health status of the subject based on the output; Run The generating step may include: converting each of the plurality of waveforms into a corresponding one representative beat; converting the plurality of representative beats into three VCG beats, each VCG beat corresponding to a cardiac vector in one dimension of three-dimensional (3D) space; determining three VCG projections, each VCG projection representing a corresponding one of the three VCG beats on a spatial plane corresponding to a corresponding dimension of the 3D space; encoding the three VCG projections into three corresponding layers of the two-dimensional image; A non-transitory computer-readable medium for executing the method.

Images