Method and computing system for enhancing images using source position
Through a modeled output system and model library generation system based on machine learning, the electromagnetic state of the heart is dynamically simulated, and the classifier is trained to identify the source of arrhythmia, solving the problems of complexity and high cost of identifying arrhythmia in the existing technology, achieving high efficiency, Accurate source localization and treatment of arrhythmia.
Patent Information
- Application Number
- CN201980088940.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-01-14
- Filing Date
- 2019-10-25
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2039-10-25
AI Technical Summary
Existing methods for identifying the source of arrhythmia are complex, expensive and at risk of complications, making it difficult to accurately locate the source location of the heart disease, especially in the ventricular and atrial septums.
Using a machine learning-based modeled output system (MLMO) and model library generation system (MLG), the classifier is trained to identify the source of arrhythmia and utilize patient-specific The model is accurately classified.
It achieves efficient and accurate identification of the source of arrhythmia, reduces treatment costs, reduces complication risks, and improves the accuracy and safety of treatment.
Smart Images

Figure CN113286557B_ABST
Abstract
Description
Background Art
[0001] Many cardiac conditions may cause symptoms, morbidity (e.g., syncope or stroke), and death. Common cardiac conditions caused by arrhythmias include inappropriate sinus tachycardia ("IST"), atrial ectopic rhythm, junctional rhythm, ventricular escape rhythm, atrial fibrillation ("AF"), ventricular fibrillation ("VF"), focal atrial tachycardia ("focal AT"), atrial microreentry, ventricular tachycardia ("VT"), atrial flutter ("AFL"), premature ventricular contractions ("PVC"), premature atrial contractions ("PAC"), atrioventricular nodal reentrant tachycardia ("AVNRT"), atrioventricular reciprocating tachycardia ("AVRT"), permanent junctional reciprocating tachycardia ("PJRT"), and junctional tachycardia ("JT"). The sources of arrhythmias can include electrical rotors (e.g., ventricular fibrillation), recurrent electrically focused sources (e.g., atrial tachycardia), anatomically based reentry (e.g., ventricular tachycardia), etc. These sources are important drivers of sustained or clinically significant episodes. Arrhythmias can be treated by ablation using different techniques directed at the source of the cardiac disorder, including radiofrequency energy ablation, cryoablation, ultrasound ablation, laser ablation, external radiation sources, targeted gene therapy, etc. Because the source and source location of cardiac disorders vary from patient to patient, targeted treatment requires identification of the source of the arrhythmia even for common cardiac disorders.
[0002] Unfortunately, current methods for reliably identifying the source location of cardiac disorders can be complex, cumbersome, and expensive. For example, one method uses an electrophysiology catheter with a multi-electrode basket catheter that is inserted intravascularly into the heart (e.g., the left ventricle) to collect measurements of cardiac electrical activity from within the heart, such as during an induced episode of VF. The measurements can then be analyzed to help identify possible source locations. Currently, electrophysiology catheters are expensive (and are typically limited to single use) and can cause serious complications, including cardiac perforation and tamponade. Another method uses an external body surface vest with electrodes to collect measurements from the patient's body surface, which can be analyzed to help determine the location of the arrhythmia source. Such body surface vests are expensive, complex, and difficult to manufacture, and can interfere with the placement of defibrillator pads that are needed to collect measurements after VF is induced during an arrhythmia. In addition, vest analysis requires a computed tomography ("CT") scan and is unable to sense the ventricular and atrial septums, where approximately 20% of arrhythmia sources may reside. BRIEF DESCRIPTION OF THE DRAWINGS
[0003] This application contains at least one drawing executed in color. Copies of this application with color drawing(s) will be provided by the Patent and Trademark Office upon request and payment of the necessary fee.
[0004] Figure 1 is a block diagram illustrating the overall processing of an MLMO system in some embodiments.
[0005] Figure 2 is a flow chart illustrating the overall process of generating a classifier by a MLMO system in some embodiments.
[0006] Figure 3 is a block diagram illustrating training and classification using a convolutional neural network in some embodiments.
[0007] Figure 4 is a flow chart illustrating the processing of the generate classifier component of the MLMO system in some embodiments.
[0008] Figure 5 is a flow chart illustrating the processing of components of an MLMO system that generates a simulated VCG in some embodiments.
[0009] Figure 6 is a flow chart illustrating processing of components of an MLMO system for periodically generating training data in some embodiments.
[0010] Figure 7 is a flow chart illustrating processing of components of an identification cycle of an MLMO system in some embodiments.
[0011] Figure 8 is a block diagram illustrating the processing of a component that normalizes periods of an MLMO system in some embodiments.
[0012] Fig. 9 is a flow chart illustrating processing of a component of an MLMO system in some embodiments that generates training data for a sequence of similar periods.
[0013] Fig.10 is a flow chart illustrating the processing of the classification component of the MLMO system in some embodiments.
[0014] Fig.11 is a block diagram illustrating components of an MLG system in some embodiments.
[0015] Fig.12 is a block diagram illustrating the generation of a simulated anatomical structure from a seed anatomical structure.
[0016] Fig.13 is a display page illustrating a user experience for viewing a simulated anatomical structure in some embodiments.
[0017] Fig.14 is a flow chart illustrating the processing of the generate model library component of the MLG system in some embodiments.
[0018] Fig.15 is a flow chart illustrating processing of components of the MLG system for generating multiple simulated anatomical structures in some embodiments.
[0019] Fig.16 is a flow chart illustrating the processing of components of the MLG system to generate simulated anatomical structures in some embodiments.
[0020] Fig.17 is a flow chart illustrating the processing of components of a generate source configuration of an MLG system in some embodiments.
[0021] Fig.18 is a flow chart illustrating the processing of components of a generative model of an MLG system in some embodiments.
[0022] Fig.19 is a flow chart illustrating processing of components of the MLG system for displaying simulated anatomical structures in some embodiments.
[0023] Fig. 20 is a block diagram illustrating the process of bootstrapping a simulation based on a previously simulated EM mesh with similar anatomical parameters.
[0024] Fig.21 is a flow chart illustrating processing of components of the MLG system in some embodiments to generate voltage solutions for a set of representative arrhythmia models.
[0025] Fig. 22 is a flow chart illustrating processing of components of the MLG system in some embodiments to generate a set of voltage solutions for arrhythmia models based on representative voltage solutions.
[0026] Fig.23 is a block diagram illustrating a process of approximating a cardiogram based on a voltage solution of an arrhythmia model with different anatomical parameters.
[0027] Fig.24 is a flow chart illustrating processing of VCG-approximate components of an MLG system in some implementations.
[0028] Fig.25 is a block diagram illustrating a process of converting an arrhythmia model based on a first polyhedron into an arrhythmia model based on a second polyhedron in some embodiments.
[0029] Fig.26 is a flow chart illustrating the processing of the components of the MLG system that convert polyhedral models in some embodiments.
[0030] Fig. 27 is a flow chart illustrating the processing of components of an identification attribute of a PM system in some embodiments.
[0031] Fig.28 is a flow chart illustrating the processing of components of a PM system that identifies a matching VCG in some embodiments.
[0032] Fig.29 is a flow chart illustrating the processing of components of a PM system to identify categories based on clustering in some embodiments.
[0033] Fig.30 is a flow chart illustrating the processing of components of a PM system that generates a cluster classifier in some embodiments.
[0034] Fig.31 is a block diagram illustrating the overall processing of the patient classifier system of the MLCD system in some embodiments.
[0035] Fig.32 is a flow chart illustrating processing of a component of a patient classifier generating a patient classifier of a patient classifier system in some embodiments.
[0036] Fig.33 is a flow chart illustrating the process of generating a clustered patient classifier of the patient classifier system in some embodiments.
[0037] Fig.34 is a block diagram illustrating components of a patient-specific model classifier system of an MLCD system in some embodiments.
[0038] Fig.35 is a flow chart illustrating the processing of components of the PSMC system that generate a patient-specific model classifier in some embodiments.
[0039] Fig.36 is a flow chart illustrating the processing of components of the PSMC system that identify similar models in some embodiments.
[0040] Fig.37 is a block diagram illustrating the overall processing of a patient-specific model display system in some embodiments.
[0041] Fig.38 is a flow chart illustrating processing of components of a PSMD system that generates a display of a patient's heart in some embodiments.
[0042] Fig.39 is a flow chart illustrating the processing of the calculate display value component of the PSMD system in some embodiments.
[0043] Fig.40 Various surface representations of vectorcardiograms are shown.
[0044] Fig.41 is a flow chart illustrating processing of a component of an EFD system that visualizes a VCG in some embodiments.
[0045] Fig.42 is a flow chart illustrating processing by components of an EFD system that display a VCG surface representation in some embodiments.
[0046] Fig.43 is a flow chart illustrating the processing of components of a CSC system that identifies a simulated echocardiogram in some embodiments.
[0047] Fig.44 is a block diagram illustrating components of a CSC system in some embodiments.
[0048] Fig.45 is a flow chart illustrating a process performed by an electrophysiologist ("EP") to collect patient data.
[0049] Fig.46 is a flow chart illustrating the processing of the calibration simulated echocardiogram component of the CSC system in some embodiments.
[0050] Fig.47 is a flow chart illustrating processing of components of a CSC system that generates vectors in some embodiments.
[0051] Fig.48 is a flow chart illustrating a process of identifying similarly oriented components of a CSC system in some embodiments.
[0052] Fig.49 is a flow chart illustrating the processing of the calibrated heart geometry component of a CSC system in some embodiments.
[0053] Fig.50 is a flow chart illustrating the processing of components of a CSC system that calibrates action potentials in some embodiments.
[0054] Fig.51 is a flow chart illustrating processing of a component of a CSC system that generates a mapping function in some embodiments.
[0055] Fig.52 is a flow chart illustrating the processing of the transform simulated cardiogram component of the CSC system in some embodiments.
[0056] Fig.53 A user interface for manually calibrating orientation is shown.
[0057] Fig.54 is a flow chart illustrating the overall processing of the API system in some embodiments.
[0058] Fig.55 is a flow chart illustrating the processing of a generate mapping function of an API system in some embodiments.
[0059] Fig.56 is a block diagram illustrating processing of a method for treating a patient during ablation in some embodiments.
[0060] Fig.57 is a block diagram illustrating components of an API system in some embodiments.
[0061] Fig.58 is a flow chart illustrating processing of components of an API system that generates ablation pattern simulations in some embodiments.
[0062] Fig.59 is a block diagram illustrating processing of components of the guided ablation of an API system in some embodiments.
[0063] Fig.60 is a flow chart illustrating processing of a component of an API system that identifies potential ablation patterns in some embodiments.
[0064] Fig.61 is a flow chart illustrating the processing of the component of the API system that selects the actual ablation pattern in some embodiments.
[0065] Fig.62 is a block diagram illustrating components of a DLSR system in some embodiments.
[0066] Fig.63 is a block diagram illustrating the structure of a block and records within a block in some embodiments.
[0067] Fig.64 is a flow chart illustrating processing of a research system in some embodiments.
[0068] Fig.65 is a flow diagram illustrating processing of a transaction generator system in some embodiments.
[0069] Fig.66 is a flow chart illustrating the processing of a miner system in some embodiments.
[0070] Fig.67 is a flow chart illustrating the overall process of the AISL system in some embodiments.
[0071] Fig.68 is a block diagram illustrating components of an AISL system in some embodiments.
[0072] Fig.69 is a flow chart of the generate calibration library component of the AISL system in some embodiments.
[0073] Fig.70 is a flow chart illustrating a process of identifying matching ECG components.
[0074] Fig.71 is a flow chart illustrating the process of adding a source location component in some embodiments.
[0075] Fig.72 is a flow chart illustrating the processing of the capture 4D CT image component in some embodiments. DETAILED DESCRIPTION
[0076] Methods and systems are provided for generating data representing the electromagnetic state (e.g., normal sinus rhythm versus ventricular fibrillation) of an electromagnetic source (e.g., a heart) within a body for various purposes (such as medical, scientific, research, and engineering purposes). The system generates data based on a source configuration of an electromagnetic source (e.g., the size of the heart, and the location of scars or fibrosis or arrhythmogenic substrates within the heart) and a computational model of the electromagnetic output of the electromagnetic source. The system can dynamically generate source configurations to provide representative source configurations that can be found in a population. For each source configuration of an electromagnetic source, the system runs a simulation of the function of the electromagnetic source to generate a modeled electromagnetic output for that source configuration. The system generates derived electromagnetic data (e.g., a vector cardiogram) for that source configuration from the modeled electromagnetic output of each source configuration. The system can then use the modeled electromagnetic output for various purposes, such as identifying the location of a condition within an electromagnetic source in a patient's body, guiding a procedure to repair (e.g., directed gene therapy) or modify (e.g., ablate) an electromagnetic source, predicting the outcome of a procedure on an electromagnetic source, analyzing genetic defects, and the like. The system may include a machine learning-based modeling output system and a model library generation system, which will be described below.
[0077] Model-based output system based on machine learning
[0078] A method and system for generating a classifier for classifying electromagnetic data obtained from an in-vivo electromagnetic source is provided. The body may be, for example, a human body, and the electromagnetic source may be a heart, a brain, a liver, a lung, a kidney, or another body part that generates an electromagnetic field that can be measured from outside the body or inside the body. The electromagnetic field may be measured by various measuring devices (e.g., electrocardiographs and electroencephalographs) using, for example, one or more (e.g., 12) leads connected to electrodes attached to or adjacent to (e.g., via a smartwatch device) the patient's body, a surface vest worn by the patient, an in-source device (e.g., a basket catheter), a cap worn by the patient, and the like. The measurement results may be represented by a cardiogram (such as an electrocardiogram ("ECG") and a vectorcardiogram ("VCG")), an electroencephalogram ("EEG"), and the like. In some embodiments, a machine learning based modeled output ("MLMO") system is provided for generating a classifier by modeling the electromagnetic output of an electromagnetic source for a variety of source configurations, and using machine learning to train the classifier using derived electromagnetic data derived from the modeled electromagnetic output as training data. The MLMO system is described below primarily as being used to generate a classifier for cardiac electromagnetic data.
[0079] In some embodiments, the MLMO system uses a computational model of an electromagnetic source to generate training data for training a classifier. The computational model models the electromagnetic output of the electromagnetic source over time based on the source configuration of the electromagnetic source. The electromagnetic output may represent a potential, a current, a magnetic field, etc. When the electromagnetic ("EM") source is the heart, the source configuration can include any subset of the group consisting of information about: the geometry and muscle fibers of the heart, trunk anatomy, normal and abnormal cardiac anatomy, normal and abnormal cardiac tissue, scars, fibrosis, inflammation, edema, accessory pathways, congenital heart disease, malignancies, previous ablation sites, previous surgical sites, external radiation therapy sites, pacing leads, implantable cardioverter-defibrillator leads, cardiac resynchronization therapy leads, pacemaker pulse generator locations, implantable cardioverter-defibrillator pulse generator locations, subcutaneous defibrillator lead locations, subcutaneous defibrillator pulse generator locations, leadless pacemaker locations, other implanted hardware (e.g., right or left ventricular assist devices), external defibrillation electrodes, surface ECG leads, surface mapping leads, mapping vests, other normal and pathological physiological characteristic distributions, etc., and the EM output is a collection of electrical potentials at various cardiac locations over time. To generate the EM output, a simulation may be performed for a simulation step of a certain step size (e.g., 1 ms) to generate an EM mesh for that step. The EM mesh may be a finite element mesh that stores the potential values at each cardiac location within that step. For example, the left ventricle may be defined as having approximately 70,000 cardiac locations, and the EM mesh stores the electromagnetic values for each cardiac location. If so, a three second simulation with a step size of 1 ms will generate 3,000 EM meshes, each including 70,000 values. The collection of EM meshes is the EM output for the simulation. A computational model is described in CT Villongco, DE Krummen, P. Stark, J H Omens, and AD M McCulloch, "Patient-specific modeling of ventricular activation pattern using surface ECG-derived vector cardiogram in bundle branch block," Progress in Biophysics and Molecular Biology, Vol. 115, No. 2-3, August 2014, pp. 305-313, which is hereby incorporated by reference. In some embodiments, the MLMO system may generate the values of the points between the vertices as a mesh, rather than just the values at the vertices. For example, the MLMO system may use Gaussian quadrature techniques to calculate the values of such points.
[0080] In some embodiments, the MLMO system generates training data by running many simulations, each based on a different source configuration, which is a set of different values of the configuration parameters of the computational model. For example, the configuration parameters of the heart can be heart geometry, rotor position, focal source position, orientation of the ventricle in the chest, orientation of ventricular muscle fibers, generation and propagation of intracellular potential of myocardial cells, etc. Each configuration parameter can have a set or range of possible values. For example, the rotor position can be 78 possible parameter sets corresponding to different positions in the ventricle. Since the MLMO system can run simulations for each combination of possible values, the number of simulations may be millions of times.
[0081] In some embodiments, the MLMO system uses simulated EM outputs to train a classifier for generating classifications based on EM data collected from a patient. The MLMO system may generate derived EM data for each EM output of the simulation. The derived EM data corresponds to EM data generated based on measurements collected by an EM measurement device, a body surface vest, an electromagnetic source internal device, etc. that will generate ECG or VCG using, for example, 12 leads. ECG and VCG are equivalent source representations of EM outputs. Then, MLMO generates one or more tags for each derived EM data to specify its corresponding classification. For example, the MLMO system may generate a tag, which is the value of a configuration parameter (e.g., rotor position) used when generating an EM output from which EM data is derived. The set of derived EM data corresponding to the feature vector and its tag constitutes training data for training a classifier. Then, the MLMO system trains the classifier. The classifier may be any one of a variety of classifiers or classifier combinations including a neural network (such as a fully connected neural network, a convolutional neural network, a recursive neural network), an autoencoder or a restricted Boltzmann machine, a support vector machine, a Bayesian classifier, etc. When the classifier is a deep neural network, training generates a set of weights for the activation function of the deep neural network. The classifier selected may be based on the type of condition to be identified. For example, certain types of neural networks may be able to be effectively trained based on focal sources rather than rotor sources.
[0082] In some embodiments, the MLMO system may augment the training data with additional features from the configuration parameters of the source configuration used to generate the training data. For example, the MLMO system may generate additional features to represent the geometry of the heart, the orientation of the heart, one or more scars or fibrosis or arrhythmogenic substrate locations, ablation locations, ablation shapes, etc. The MLMO system may input these additional features into the fully connected layer together with the output generated by the layer before the fully connected layer of the convolutional neural network ("CNN"), which is described below. The output of the layer before the fully connected layer (e.g., the pooling layer) may be "flattened" into a one-dimensional array, and the MLMO system may add additional features as other elements of the one-dimensional array. The output of the fully connected layer may provide the probability of each tag used in the training data. Therefore, the probability will be based on a combination of the derived EM data and the additional features. Even when the derived EM data is the same or similar, thereby reflecting that, for example, the same or similar EM data can be generated for patients with different heart geometries and different scars or fibrosis or arrhythmogenic substrate locations, the classifier will also be able to output different probabilities. Alternatively, the MLMO system can use an additional classifier that (1) inputs the probabilities generated by the CNN based on derived EM data only, and (2) inputs the additional features, and then outputs the final probability of each classification taking the additional features into account. The additional classifier can be, for example, a support vector machine. The CNN and the additional classifier can be trained in parallel.
[0083] In some embodiments, the MLMO system normalizes the VCG of each cycle of the training data on the voltage and time axis. A cycle can be defined as a time interval (e.g., start time to end time) that defines a single unit or beat of periodic electrical activity during a normal or abnormal rhythm. The cycle facilitates beat-by-beat analysis of the evolution of the source configuration over time, and enables subsequent voltage and time normalization within each cycle. Normalization retains the significant features of voltage-time dynamics and improves the generalization of training data to changes in source configuration parameters (e.g., trunk conductivity, lead placement and resistance, myocardial conduction velocity, action potential dynamics, overall heart size, etc.) expected in actual patients. The MLMO system can normalize voltage to a range between -1 and 1, and normalize time to a fixed range of 0 to 1, with increments of milliseconds or percentages. In order to normalize the voltage of the cycle, the MLMO system can identify the maximum size of the vector across the axis. The MLMO system divides each voltage by the maximum size. To normalize the time axis, the MLMO system performs interpolation from the number of points in the VCG (which may be greater or less than 1000) to 1000 points of the normalized period.
[0084] In some embodiments, after training the classifier, the MLMO system is ready to generate classifications based on EM and other routinely available clinical data collected from the patient. For example, an ECG may be collected from the patient, and a VCG may be generated from the ECG. The VCG is input to the classifier to generate a classification indicating, for example, the patient's rotor position. Thus, the MLMO system can be used to generate classifications even if the geometry of the patient's heart is unknown or there is no simulation based on the same geometry as the patient's heart. If other patient measurements are available, such as heart size and orientation, scar or fibrosis, or arrhythmogenic substrate configuration, etc., the other patient measurements may be included as EM data to improve accuracy. This allows the classifier to effectively learn complex hidden features in a variety of clinical data that are not directly represented by the training data.
[0085] In some embodiments, by generating training data based on a sequence of continuous cycles with similar EM features, the MLMO system can classify source stability (i.e., the beat-to-beat consistency of a dominant arrhythmia source confined to a specific region in the heart). The technique for determining the stability of arrhythmia sources is described in Krummen, D. et al., Rotor Stability Separates Sustained Ventricular Fibrillation from Self-Terminating Episodes in Humans, Journal of American College of Cardiology, Vol. 63, No. 23, 2014, which is hereby incorporated by reference. This reference demonstrates the effectiveness of targeted ablation at a stable source site for preventing recurrent arrhythmia attacks. For example, in view of VCG, the MLMO system can identify cycles, and then identify sequences of two consecutive cycles, three consecutive cycles, four consecutive cycles, etc., where all VCG cycles in the sequence have similar morphologies to each other. Each identified sequence can be marked based on the value of the parameter of the source configuration used to generate the VCG. The MLMO system may then train a separate classifier for each sequence length (e.g., 2, 3, 4, etc.) using the training data for sequences of that sequence length. For example, the MLMO system may train a classifier for sequences of two cycles and a separate classifier for sequences of three cycles. To generate a classification for a patient, the MLMO system may identify sequences of similar cycles with different sequence lengths in the patient's VCG and input these sequences into a classifier for the appropriate sequence length. The MLMO system may then combine the classifications from all classifiers to derive a final classification, or may simply output all classifications.
[0086] Although classifiers can be trained using actual patient ECGs or VCGs and corresponding intracardiac basket catheter measurements of source locations, the cost of collecting, preparing, and labeling sufficient amounts of data can currently be prohibitively expensive, although technological advances may significantly reduce costs. Moreover, although advances in collecting and storing medical records may result in more efficient use of actual patient data, training data based on actual patients may currently be too sparse and noisy to be particularly effective in training classifiers for large populations. In some embodiments, a combination of actual patient VCGs and VCGs derived from simulations may be used to train the MLMO system. Training data collected from actual patients may be collected using various devices such as flexible electronic devices (e.g., epidermal electronic systems), smart clothing, and the like.
[0087] In some embodiments, a combination of actual patient VCGs and VCGs derived from simulations may be used to train the MLMO system.
[0088] Figure 11 is a block diagram illustrating the overall processing of the MLMO system in some embodiments. The MLMO system includes a classifier generation component 110 and a classification component 120. The computational model of the heart is a heart model, which may include data and code stored in a heart model data storage area 111. The component 112 that generates the simulation inputs the heart model and a parameter set for the simulation. The parameter set (also referred to as a source configuration) may include a parameter set for each combination of possible values of the parameter, or may specify how to generate (e.g., by computer code) the parameter set. For example, for the rotor position parameter, the computer code may include a list of possible rotor positions, and for the ventricular orientation parameter, the computer code may dynamically generate values from the basic orientation axis and the code for generating possible tilt angles (such as x-axis and j︿ axis increments) from that basic orientation. The output of the component that generates the simulation is stored in a voltage solution data storage area 113, where the voltage solution is an EM output. The voltage solution is an example of an EM grid. The component 114 that generates the VCG generates the VCG from the voltage solution and stores the VCG in the VCG data storage area 115. The component generating the VCG may generate the ECG from the voltage solution and then generate the VCG from the ECG. The generation of the VCG from the ECG is described in "Reconstruction of the Frank vector cardiogram from standard electrocardiographic leads: diagnostic comparison of different methods," European Heart Journal, Vol. 11, No. 12, Dec. 1, 1990, pp. 1083-1092, by J. A. Kors, G. Van Herpen, A. S. Sittig, and J. H. Van Bemmel, which is hereby incorporated by reference. The component generating the training data 116 inputs the VCG and marks each VCG with one or more marks that can be derived from the parameter set, and stores the training data in the training data storage area 117. The mark can be, for example, the value of a parameter in a parameter set for generating an EM output from which the VCG can be derived. The component training classifier 118 inputs the training data, trains the classifier, and stores the weights (e.g., the weights of the activation function of the convolutional neural network) in the classifier weight data storage area 119. To generate the classification, the ECG collecting component 121 inputs the ECG collected from the patient. The VCG generating component 122 generates the VCG from the ECG. The classification component 123 inputs the VCG and generates the classification using the classifier weights of the trained classifier.
[0089] The computing system (e.g., a network node or a collection of network nodes) on which the MLMO system and other described systems can be implemented may include a central processing unit, an input device, an output device (e.g., a display device and a speaker), a storage device (e.g., a memory and a disk drive), a network interface, a graphics processing unit, a cellular wireless link interface, a global positioning system device, etc. The input device may include a keyboard, a pointing device, a touch screen, a gesture recognition device (e.g., for air gestures), a head and eye tracking device, a microphone for voice recognition, etc. The computing system may include a high-performance computing system, a cloud-based server, a desktop computer, a laptop computer, a tablet computer, an e-reader, a personal digital assistant, a smart phone, a gaming device, a server, etc. For example, a high-performance computing system may be used to perform simulations and training, and classification may be performed by a tablet computer. The computing system may access computer-readable media including computer-readable storage media and data transmission media. Computer-readable storage media are tangible storage devices that do not include transient propagation signals. Examples of computer-readable storage media include memories such as main memory, cache memory, and auxiliary memory (e.g., DVD) and other storage devices. Computer-readable storage media may have recorded thereon or may be encoded with computer-executable instructions or logic that implement the MLMO system and other described systems. Data transmission media are used to transmit data through a wired or wireless connection through a transient propagating signal or carrier wave (e.g., electromagnetic). The computing system may include a secure cryptographic processor as part of a central processing unit that is used to generate and securely store cryptographic keys and to encrypt and decrypt data using the cryptographic keys.
[0090] The MLMO system and other described systems may be described in the general context of computer-executable instructions (such as program modules and components) executed by one or more computers, processors, or other devices. Typically, a program module or component includes a routine, program, object, data structure, etc. that performs the tasks of the MLMO system and other described systems or implements its data types. Typically, in various examples, the functionality of the program modules may be combined or distributed as needed. Aspects of the MLMO system and other described systems may be implemented in hardware using, for example, an application specific integrated circuit ("ASIC") or a field programmable gate array ("FPGA").
[0091] Figure 22 is a flow chart showing the overall processing of generating a classifier by an MLMO system in some embodiments. Component 200 of generating a classifier is executed to generate a classifier. In box 201, the component accesses the computational model to be used for running a simulation. In box 202, the component selects the next source configuration (i.e., parameter set) to be used in the simulation. In decision box 203, if all source configurations have been selected, the component continues at box 205, otherwise the component continues at box 204. In box 204, the component runs the simulation using the selected source configuration to generate the EM output of the simulation, and then loops to box 202 to select the next source configuration. In box 205, the component selects the next EM output generated by the simulation. In decision box 206, if all EM outputs have been selected, the component continues at box 210, otherwise the component continues at box 207. In box 207, the component derives EM data from the EM output. For example, the EM output can be a collection of EM grids, and the EM data can be an ECG or VCG derived from the electromagnetic values of the EM grid. In some embodiments, the component may additionally identify cycles (periodic intervals of arrhythmic activity) within an ECG or VCG. A cycle may be defined by continuous crossings from negative voltage to positive voltage ("positive crossings") or continuous crossings from positive voltage to negative voltage ("negative crossings") relative to a spatial direction or a set of directions comprising a reference frame or a set of reference frames. The reference frame may coincide with an anatomical axis (e.g., from left to right on the x-axis, from top to bottom on the y-axis, from front to back on the z-axis), an imaging axis (e.g., a CT, MR, or x-ray coordinate system), a surface lead vector, a principal axis calculated by principal component analysis of measured or simulated EM source configurations and outputs, or a user-defined direction of interest. For example, a three-second VCG may have three cycles, and each cycle may be defined by the time of a positive crossing along the x-axis. Alternatively, a cycle may be defined by a crossing along the y-axis or z-axis. Additionally, a cycle may be defined by a negative crossing. Thus, in some embodiments, a component may generate training data from a single VCG based on various cycle definitions, which are various combinations of positive and negative crossings, where the cycle for all axes is defined by a crossing on one of the x-axis, y-axis, and z-axis, or the cycle for each axis is defined by a crossing on that axis. In addition, the training data may include cycles identified based on all possible cycle definitions or subsets of cycle definitions. For example, the training data may include cycles defined by positive crossings of the x-axis, negative crossings of the y-axis, and positive crossings of that axis itself for each axis. The cycle definitions may also be defined by the timing of electrical events derived from values stored in the EM grid. For example, a point or group of points in the grid may periodically cross voltage thresholds representing electrical activation and deactivation. Thus, a cycle may be defined by activation-deactivation, or consecutive activation-activation or deactivation-deactivation intervals corresponding to a point or group of points within the grid. The resulting timing of these intervals may be co-located to the ECG or VCG for cycle identification.In block 208, the component labels the EM data based on the source configuration (e.g., source position). When identifying cycles, the component may label each cycle with the same label. For example, the component may label the identified cycles with the same rotor position. In block 209, the component adds the EM data along with the label to the training data and then loops to block 205 to select the next EM output. In block 210, the component trains the classifier using the training data and then completes.
[0092] Figure 3 is a block diagram illustrating training and classification using a convolutional neural network in some embodiments. A convolutional neural network can be one-dimensional in the sense that the convolutional neural network input is an image of a single row of pixels, each pixel having a red, green, and blue ("RGB") value. The MLMO system sets the pixel value based on the voltage of the VCG of the training data. The image has the same number of pixels as the vector of the VCG of the training data. The MLMO system sets the red, green, and blue values of the image pixels to the x, y, and z values of the corresponding vector of the VCG. For example, if the period of the VCG is 1 second long and the VCG has one vector per millisecond, the image is 1 by 1000 pixels. A one-dimensional convolutional neural network ("1D CNN") trainer 310 uses training data 301 to learn the weights of the activation function of the convolutional neural network. In order to generate a classification of a patient, the MLMO system provides the patient's VCG 302 as a one-dimensional image. Then, the 1D CNN 320 classifies the VCG based on the weights and outputs classification information, such as rotor position.
[0093] CNN is a type of neural network developed specifically for processing images. CNN can be used to input an entire image and output a classification of the image. For example, CNN can be used to automatically determine whether a patient's scan indicates the presence of an abnormality (e.g., a tumor). The MLMO system treats the exported EM data as a one-dimensional image. CNN has multiple layers, such as convolutional layers, rectified linear unit (ReLU) layers, pooling layers, fully connected (FC) layers, etc. Some more complex CNNs may have multiple convolutional layers, ReLU layers, pooling layers, and FC layers.
[0094] A convolutional layer may include multiple filters (also called kernels or activation functions). The filter inputs a convolution window of the image, applies weights to each pixel of the convolution window, and outputs an activation value for that convolution window. For example, if the image is 256 by 256 pixels, the convolution window may be 8 by 8 pixels. The filter may apply different weights to each of the 64 pixels in the convolution window to generate an activation value, also called a eigenvalue. Assuming a stride of 1 with appropriate padding, for each filter, the convolutional layer may include a node (also called a neuron) for each pixel of the image. Each node outputs a eigenvalue based on the set of weights for the filter learned during the training phase for that node. Continuing with the example, for each filter, the convolutional layer may have 65,566 nodes (256 * 256). The eigenvalues generated by the nodes of the filter can be considered to form a convolutional feature map with a height and width of 256. If it is assumed that the eigenvalues calculated for the convolution window at one location to identify a feature or characteristic (e.g., an edge) will help identify that feature at a different location, all nodes of the filter can share the same set of weights. By sharing weights, training time and storage requirements can be significantly reduced. If each pixel of the image is represented by multiple colors, the convolution layer can include another dimension to represent each individual color. Moreover, if the image is a 3D image, the convolution layer can include another dimension for each image within the 3D image. In this case, the filter can input a 3D convolution window.
[0095] For each node of the convolutional layer, the ReLU layer may have a node that generates an eigenvalue. The generated eigenvalues form a ReLU feature map. The ReLU layer applies a filter to each eigenvalue of the convolutional feature map to generate an eigenvalue of the ReLU feature map. For example, a filter such as max(0, activation value) may be used to ensure that the eigenvalues of the ReLU feature map are not negative.
[0096] The pooling layer can be used to reduce the size of the ReLU feature map by downsampling the ReLU feature map to form a pooled feature map. The pooling layer includes a pooling function that inputs a set of eigenvalues of the ReLU feature map and outputs an eigenvalue. For example, the pooling function can generate an eigenvalue that is the average of multiple sets of 2 by 2 eigenvalues of the ReLU feature map. Continuing the above example, for each filter, the pooling layer will have a 128 by 128 pooled feature map.
[0097] The FC layer includes a certain number of nodes, each of which is connected to each feature value of the pooled feature map. For example, if you want to classify an image as a cat, dog, bird, mouse, or ferret, the FC layer may include five nodes whose feature values provide scores indicating the likelihood that the image contains one of these animals. Each node has a filter with its own set of weights appropriate for the type of animal the filter is trying to detect.
[0098] In the following, the MLMO system is described with reference to the following data structure. Brackets represent arrays. For example, VCG[2].V[5].x represents the voltage of the x-axis of the fifth time interval in the second VCG.
[0099] The data structures are further described below when first referenced.
[0100]
[0101] Figure 4 4 is a flowchart of the detailed processing of the components of the generation classifier of the MLMO system in some embodiments. The component 400 of the generation classifier is called to generate the classifier. In box 401, the component calls the component that generates the simulated VCG to simulate the VCG (VCG[]) for multiple parameter sets. In boxes 402-405, the component loops, thereby generating training data for each simulation. In box 402, the component sets the index i to 1 for indexing the parameter set. In decision box 403, if the index i is equal to the number of parameter sets, all training data have been generated, and the component continues at box 406, otherwise the component continues at box 404. In box 404, the component calls the component that generates the training data, thereby passing the indication of the index parameter set. In box 405, the component increments the index i, and then loops to box 403. In box 406, the component calls the component of the training classifier to train the classifier based on the generated training data, and then completes.
[0102] Figure 5is a flowchart illustrating the processing of the components for generating simulated VCGs of the MLMO system in some embodiments. The component 500 for generating simulated VCGs is called to generate simulated VCGs for each parameter set. In box 501, the component sets index i to 1 for indexing through these parameter sets. In decision box 502, if index i is greater than the number of parameter sets, the component is completed, otherwise the component continues at box 503. In box 503, the component sets index j to 1 for indexing through simulation steps. In decision box 504, if index j is greater than the number of simulation steps, the simulation for the indexed parameter set is completed, and the component continues at box 507, otherwise the component continues at box 505. In box 505, the component applies a computational model based on the indexed parameter set and the indexed simulation step to generate a voltage solution (VS[j]) for the indexed simulation step. In box 506, the component increments index j and then loops to box 504 to process the next simulation step. In block 507, the component generates a VCG (VCG[i]) for the indexed parameter set from the voltage solution (VS[]) calculated for the parameter set. In block 508, the component increments the index i and then loops to block 502 to process the next parameter set.
[0103] Figure 6 6 is a flowchart showing the processing of a component for generating training data for a cycle in an MLMO system in some embodiments. The component 600 generating training data is called to pass the index i, and the component 600 generating training data indexes the VCG generated for the parameter set and generates training data from the VCG. In box 601, the component calls the component identifying the cycle, thereby passing the indication of the index VCG (VCG [i]), and receives the normalized VCG (nVCG []) for each cycle together with the count (#C) of the identified cycle. In box 602, the component sets the index k to 1 for indexing through these cycles. In decision box 603, if the index k is greater than the count of the cycle, the training data of all cycles of the index VCG has been generated and the component is completed, otherwise the component continues at box 604. In box 604, the component increments the running count (#TD) of the training data (TD) (which is used as an index into the training data). In box 605, the component sets the normalized nVCG (TD [#TD]. nVCG) of the index training data to the part specified by the index cycle of the VCG. The component extracts the portion from the x-axis, y-axis, and z-axis, as defined by the start and end of the cycle. In block 606, the component sets one or more flags that index the training data based on a function of the index parameter set (e.g., rotor position). In block 607, the component increments the index k to index to the next cycle, and then loops to block 603 to process the next cycle.
[0104] Figure 7700 is a flowchart illustrating the processing of a component for identifying a cycle of an MLMO system in some embodiments. A component 700 for identifying a cycle is called to identify a cycle within a VCG and to provide a normalized VCG (nVCG[]) for these cycles. In box 701, the component initializes index j to 2 for indexing through the VCG, and sets index k to 0 to index through the identified cycle. In decision box 702, if index j is greater than the size of the VCG, the component has identified all cycles and the component is complete, thereby providing a normalized nVCG, otherwise the component continues at box 703. In box 703, if the previous voltage of the x-axis of the VCG (VCG.V[j-1].x) is greater than or equal to zero and the index voltage of the x-axis of the VCG (VCG.V[j].x) is less than zero (i.e., a negative crossing of the x-axis), the start of a possible cycle has been identified, and the component continues at box 704 to identify the cycle, otherwise the component continues at box 709. In block 704, the component sets the start of the indexed cycle (C[k].start) equal to index j. In decision block 705, if at least one cycle has been identified, the end of the previous cycle is known, and the component increments index k and continues at block 706, otherwise the component increments index k and continues at block 709. In block 706, the component sets the end of the previous cycle to index j-1. In block 707, the component extracts a VCG (eVCG) for the previous indexed cycle defined by the start and end of the previous cycle. In block 708, the component calls a component that normalizes the cycle, passing an indication of the extracted VCG (eVCG), and receiving a normalized cycle (nVCG). In block 709, the component increments index j for indexing through the VCGs and loops to block 702.
[0105] Figure 8800 is a block diagram illustrating the processing of a component for normalizing a cycle in an MLMO system in some embodiments. The component 800 for normalizing a cycle is called to pass an indication of a VCG of the cycle and normalize the cycle. In box 801, the component identifies the maximum vector size V' of the vector in the cycle. For example, the vector size of the vector can be calculated by taking the square root of the square sum of the x, y and z values of the vector. In box 802, the component sets index i to index the next axis of the VCG. In decision box 803, if all axes have been selected, the component is completed, thereby providing a normalized VCG, otherwise the component continues at box 804. In box 804, the component initializes index j to 1 for indexing vectors through the normalized cycle. In decision box 805, if index j is greater than the number of vectors of the normalized cycle, the component loops to box 802 to select the next axis, otherwise the component continues at box 806. In block 806, the component sets the normalized VCG for the index vector for the index axis to an interpolation of the passed VCG, the index vector, and the maximum vector size V'. The interpolation effectively compresses or expands the VCG to the number of vectors in the normalized VCG and divides the x, y, and z values of the vector by the maximum vector size V. In block 807, the component increments the index j and then loops to block 805.
[0106] Fig. 9Flowchart showing the processing of components for generating training data for a sequence of similar cycles of an MLMO system in some embodiments. Component 900 for generating training data for a sequence of similar cycles is called to identify a sequence of two consecutive similar cycles of a VCG indexed by a passed index i, and training data is generated based on the sequence of similar cycles identified. The cycles in the sequence are similar according to a similarity score reflecting the stability of the cycle. In box 901, the component calls a component for identifying a cycle to identify a cycle of the VCG (nVCG[]). In box 902, the component sets index j to 2 for indexing through the identified cycle. In decision box 903, if index j is greater than the number of cycles identified, all cycles have been indexed, and then the component is completed, otherwise the component continues at box 904. In box 904, the component generates similarity scores for cycles indexed by j-1 and j. Similarity scores can be based on, for example, cosine similarity, Pearson correlation, etc. In decision block 905, if the similarity score is above a similarity score threshold (T) indicating similar cycles, then a sequence of similar cycles has been identified and the component continues at block 906, otherwise the component continues at block 909. In block 906, the component increments a running count (#TD) of the training data. In block 907, the component sets the training data to the sequence of similar cycles. In block 908, the component sets the tag of the training data to a tag derived from the parameter set (PS[i]) used to generate the VCG and then continues at block 909. In block 909, the component increments index i to select the next cycle sequence and loops to block 903.
[0107] Fig.10 1 is a flowchart illustrating the processing of the classification component of the MLMO system in some embodiments. The classification component 1000 is called to pass the VCG obtained from the patient and output the classification. In box 1001, the component calls the component that identifies the cycle, thereby passing the indication of the VCG, and receives the normalized VCG for the cycle and the count of the cycle. In box 1002, the component sets the index k to 1 for indexing through the cycle. In decision box 1003, if the index k is greater than the number of cycles, the component completes the classification, otherwise the component continues at box 1004. In box 1004, the component applies a classifier to the index cycle to generate a classification. In box 1005, the component increments the index and then loops to box 1003 to process the next cycle. For each cycle, different classifications (e.g., different rotor positions) can be generated. In this case, the overall classification can be derived from a combination of different classifications (e.g., the average value of the rotor position).
[0108] Model library generation system
[0109] Methods and systems for generating a model library of models of in vivo EM sources are provided. In some embodiments, a model library generation ("MLG") system generates a model based on a source configuration having configuration parameters, the configuration parameters including anatomical parameters and electrophysiological parameters of the EM source. The anatomical parameters specify the size or overall geometry of the EM source. When the EM source is a heart, the model can be an arrhythmia model based on anatomical parameters including any subset of the group consisting of: thickness of the heart wall (e.g., thickness of the endocardium, myocardium, and epicardium), size of the chamber, diameter, orientation of the ventricle in the chest, trunk anatomy, fiber structure, scar, location of fibrosis and arrhythmogenic substrate, scar shape, etc. The geometry of the heart can be measured at maximum chamber volume and minimum wall thickness and when activation occurs. In the case of normal sinus, activation occurs at the time of the end-diastolic portion of the beat. In the case of arrhythmias, activation may occur at different times during the beat. Therefore, the MLG system can allow the geometry to be specified at times other than the end-diastolic portion. The torso anatomy can be used to adjust the EM output based on the size, shape, composition, etc. of the torso. Electrophysiological parameters are non-anatomical parameters that may include any subset of the group consisting of: inflammation, edema, accessory pathways, congenital heart disease, malignancy, previous ablation sites, previous surgical sites, external radiation therapy sites, pacing leads, implantable cardioverter defibrillator leads, cardiac resynchronization therapy leads, pacemaker pulse generator locations, implantable cardioverter defibrillator pulse generator locations, subcutaneous defibrillator lead locations, subcutaneous defibrillator pulse generator locations, leadless pacemaker locations, other implanted hardware (e.g., right or left ventricular assist devices), external defibrillation electrodes, surface ECG leads, surface mapping leads, mapping vests, and other normal and pathological physiological characteristics distribution, action potential dynamics, conductivity, arrhythmia source locations, etc. The configuration parameters selected for the simulation may be based on the machine learning algorithm employed. For example, fiber structure parameters may be selected for a convolutional neural network but not for other types of neural networks. The MLG system generates a source configuration from which an arrhythmia model can be generated. For each source configuration, the MLG system generates an arrhythmia model, which includes a grid and model parameters, such as variable weights of the arrhythmia model. The MLG system generates a grid based on anatomical parameters. After generating the computational grid, MLG generates model parameters of the arrhythmia model at points within the grid based on the electrophysiological parameters of that source configuration. The electrophysiological parameters control, for example, the modeling of electromagnetic propagation at that point based on the electrophysiological parameters. The collection of arrhythmia models forms an arrhythmia model library. The MLG system can then generate a modeled EM output for each arrhythmia model and use the modeled EM output to train a classifier. The arrhythmia model library can be used for other purposes, such as studying the efficacy of different types of ablations.
[0110] In some embodiments, the MLG system generates an anatomical parameter set for the heart, wherein each set has a value (e.g., a scalar, vector, or tensor) for each anatomical parameter. The anatomical parameter set specifies the anatomical structure of the heart. The MLG system generates a simulated anatomical structure based on a seed anatomical structure set (a value specifying the size of the EM source) and a weight set including a weight for each seed anatomical structure. The seed anatomical structure may also include feature values derived from specified values (such as the mass, volume, aspect ratio, sphericity, etc. of the chamber). The seed anatomical structure may represent an extreme anatomical structure found in a patient. For example, a seed anatomical structure may be generated from ultrasound, computed tomography, and magnetic resonance imaging scans of a patient with extreme cardiac disease (such as right ventricular enlargement, very thick or very thin endocardium, etc.). The MLG system generates a simulated anatomical structure for each weight set. Each weight in the weight set indicates the contribution of the anatomical parameter of the seed anatomical structure to the simulated anatomical structure. For example, if four seed anatomical structures are specified, the weights may be 0.4, 0.3, 0.2, and 0.1. The MLG system sets the value of each anatomical parameter of the simulated anatomical structure to the weighted average of the value of each anatomical parameter of the seed anatomical structure for the weight set. The MLG system can use various techniques to generate the weight sets, such as defining a fixed interval (e.g., 0.001), randomly selecting weights that add up to 1.0, using experimental design techniques, etc. The MLG system can also validate the simulated anatomical structure based on a comparison of the anatomical parameters with the actual anatomical parameters of the actual patient. For example, if the combination of the height, width, and depth of the heart chamber results in a volume that is not seen in the actual patient (e.g., not found in the patient population), the MLG system can discard that simulated anatomical structure because it is unlikely to occur in the actual patient.
[0111] In some embodiments, the MLG system may employ a bootstrapping technique to accelerate the generation of an arrhythmia model based on the modeled EM outputs. Since a mesh for the left ventricle may have 70,000 vertices, a simulation to generate the modeled EM outputs for the arrhythmia model of the left ventricle will require the calculation of 70,000 values per EM mesh. If the simulation lasts three seconds and the step size is 1 ms, then 70×10 vertices will need to be calculated. 6 If the arrhythmia model library includes one million arrhythmia models, the number of values that need to be calculated will be 70×10 12. In addition, the calculation of a single value may involve many mathematical operations, and multiple values may be calculated for each vertex. To help reduce the number of values that need to be calculated, the MLG system effectively shares some values of the EM mesh generated for one simulation based on one arrhythmia model with another simulation based on another arrhythmia model. For example, when a simulation is started, it may take about one second before some values of the EM mesh have a noticeable effect on these values. For example, at the one-second point in the simulation, the EM meshes of simulations based on arrhythmia models with the same source configuration except for the location of scar or fibrosis or arrhythmogenic substrate may have very similar values. In order to reduce the number of values that need to be calculated, the MLG system groups together arrhythmia models with the same or nearly the same source configuration except for the location of scar or fibrosis or arrhythmogenic substrate. The grouping may also not take into account conduction velocity and action potential parameters. The MLG system then runs a simulation for a representative arrhythmia model of this group (e.g., a model without scar or fibrosis or arrhythmogenic substrate location). To run simulations for other arrhythmia models, the MLG system sets the values of the initial EM grid to the values of the EM grid at the one-second point in the simulation for the representative arrhythmia model. The MLG system then runs the other simulations for two seconds, starting with the values of the initial EM grid. The modeled EM output for each other arrhythmia model includes the EM grid for the first second of the representative arrhythmia model and the EM grid for the two seconds of that other arrhythmia model. In this way, the MLG system can significantly reduce the number of values that need to be calculated during some simulations (e.g., by about one-third), which can significantly speed up the generation of model EM outputs or voltage solutions for the arrhythmia model library.
[0112] In some embodiments, the MLG system may employ other bootstrapping techniques to accelerate the generation of modeled EM outputs. One bootstrapping technique may allow for the rapid generation of different configuration parameters, such as different geometries, different action potentials, and different conductivity parameters. For example, for a given focal or rotor source location and a set of other configuration parameter sets, a simulation may be run for two seconds. Then, the EM grid from that simulation is modified based on different geometries, or the model parameters are adjusted based on different action potentials or conductivity parameters. Then the simulation is continued. Based on different configuration parameters, it may take about a second for the activation potential to stabilize. Another bootstrapping technique may accelerate the generation of rotors anchored to different scar locations. For example, for a given anchor scar location, a simulation may be run for two seconds. After the first second, the rotor may be stably anchored to the scar location. During the second second, sufficient modeled EM output is simulated to generate an echocardiogram. Then the EM grid from that simulation is modified to move the anchor scar location nearby, and possibly modified based on different geometries. Then the simulation is continued. The simulation will allow the rotor to detach from the previous anchor scar location and attach to the new anchor scar location. Once attached, the next second or so of modeled EM output can be used to generate an ECG or VCG. Likewise, instead of modifying the anchor scar position, an ablation shape and pattern can be added to the EM mesh to simulate the effect of that ablation, or the configuration of the anchor scar can be modified to quickly simulate the impact of that configuration.
[0113] In some embodiments, the MLG system can accelerate the generation of derived EM data derived from the modeled EM output generated for an arrhythmia model of an arrhythmia model library. The derived EM data can be a VCG (or other electrocardiogram) generated from the modeled EM output (e.g., 3,000 EM grids). The MLG system can group together arrhythmia models whose source configurations have similar anatomical parameters. Therefore, each arrhythmia model in a group will have similar electrophysiological parameters and different anatomical parameters. The MLG system then runs simulations for the representative arrhythmia model of this group. However, the MLG system does not need to run simulations for other arrhythmia models in this group. In order to generate a VCG for one of the other arrhythmia models, the MLG system inputs the modeled EM output of the representative arrhythmia model and the anatomical parameters of this other arrhythmia model. The MLG system then calculates a VCG value for this other arrhythmia model based on the value of the modeled EM output by adjusting based on the difference in anatomical parameters between the representative arrhythmia model and this other source configuration. In this way, the MLG system avoids running any simulations other than a representative arrhythmia model for each group of arrhythmia models.
[0114] In some embodiments, the MLG system converts an arrhythmia model based on one polyhedral model into an arrhythmia model based on another polyhedral model. Different finite grid problem solvers may be based on different polyhedral models. For example, one polyhedral model may be a hexahedral model, while another polyhedral model may be a tetrahedral model. For the hexahedral model, the mesh is filled with hexahedrons. For the tetrahedral model, the mesh is filled with tetrahedrons. If an arrhythmia model library is generated based on a hexahedral model, that arrhythmia library cannot be input into the tetrahedral problem solver. A separate arrhythmia model library based on a tetrahedral model may be generated based on a source configuration, but doing so would be computationally expensive. In order to reduce the computational overhead of generating a tetrahedral arrhythmia model library, the MLG system converts a hexahedral arrhythmia model into a tetrahedral arrhythmia model. In order to convert a tetrahedral arrhythmia model, the MLG system generates a surface representation of the tetrahedral arrhythmia model, for example, based on the faces of the mesh of the hexahedral arrhythmia model. The MLG system then fills the volume formed by the surface representation with tetrahedrons to generate a tetrahedral mesh. The MLG system then generates a value for each vertex of the tetrahedral mesh by interpolating the values of the vertices of the hexahedral mesh that are close to that vertex of the tetrahedral mesh. In this way, the MLG system can use a library of arrhythmia models of one type of polyhedron to generate a library of arrhythmia models of another type of polyhedron and avoid the computational overhead of generating a library of arrhythmia models of another type of polyhedron from a source configuration. The MLG system can also convert an arrhythmia model from one polyhedral model to another polyhedral model for displaying electromagnetic sources. For example, the MLG system can run a simulation using a hexahedral ventricle mesh and then convert to a surface triangular ventricle mesh for display to provide a more realistic ventricle display.
[0115] Fig.111 is a block diagram showing the components of the MLG system in some embodiments. The MLG system 1100 includes a component 1101 for generating a model library, a component 1102 for generating multiple simulated anatomical structures, a component 1103 for generating simulated anatomical structures, a component 1104 for generating source configurations, a component 1105 for generating models, a component 1106 for displaying simulated anatomical structures, a component 1107 for generating representative voltage solutions, a component 1108 for generating group voltage solutions, a component 1109 for approximating VCGs, and a component 1110 for converting polyhedral models. The MLG system also includes a model library 1111 for storing arrhythmia models, a seed anatomical structure library 1112 for storing seed anatomical structures (e.g., cardiac), and a simulated anatomical structure library 1113 for storing simulated anatomical structures (e.g., cardiac). The component for generating a model library controls the overall generation of the model library. The component for generating multiple simulated anatomical structures generates simulated anatomical structures for various weight sets by calling the component for generating simulated anatomical structures for each weight set. The component for generating source configurations generates various source configurations based on the simulated anatomical structures and possible values of electrophysiological parameters. A component for generating a model generates a model based on a source configuration. A component for displaying a simulated anatomical structure may provide an application programming interface or user experience for specifying a weight set and viewing the seed and simulated anatomical structure. A component for generating a representative voltage solution generates a voltage solution for a representative source configuration of a group. A component for generating a voltage solution for a group generates a voltage solution for a source configuration in a group based on bootstrapping using the voltage solution for the representative source configuration. A component for approximating a VCG generates an approximate VCG based on a voltage solution for a power configuration that is similar but has different anatomical parameters. A component for converting a polyhedral model converts a hexahedral model into a tetrahedral model.
[0116] Fig.12 is a block diagram showing the generation of simulated anatomical structures from seed anatomical structures. In this example, simulated anatomical structures 1210 are generated using seed anatomical structures 1201-1206. Each seed anatomical structure has an associated weight that specifies the contribution of the seed anatomical structure to the simulated anatomical structure. The sum of the weights may be 1.0. In some embodiments, the MLG system may adopt a weight for each anatomical parameter of the seed geometry, rather than a single weight for each seed anatomical structure. For example, the weights may include the weight for the size of the heart chamber, the weight for the thickness of the heart wall, etc. The sum of the weights of the anatomical parameters of the seed anatomical structures may be 1.0. For example, for the size of the left ventricle, the weights of the seed anatomical structures 1201-1206 may be 0.1, 0.1, 0.1, 0.3, and 0.3, respectively, and for the thickness of the endocardium, myocardium, and epicardium, they may be 0.2, 0.2, 0.2, 0.2, 0.1, and 0.1, respectively. Alternatively, the weights of the seed anatomical structures or anatomical parameters of the seed anatomical structures do not need to sum to 1.0.
[0117] Fig.131300 includes a graphics area 1310, a weight area 1320, and an options area 1330. The graphics area displays a graphic and weight of each seed anatomical structure of the heart, as well as a graphic of the simulated anatomical structure of the heart based on these weights. The user can use the weight area to specify the weight of each seed anatomical structure. The user can use the options area to add additional seed anatomical structures, delete seed anatomical structures, and specify weight rules based on a user interface (not shown) for each option. The MLG system can specify a data structure for storing anatomical parameters of seed anatomical structures. When adding a new seed anatomical structure, the user can upload the data structure. The MLG system can also allow users to create a seed anatomical structure library that can be used to generate simulated anatomical structures. The weight rule specifies how to generate a weight set for the simulated anatomical structure library. For example, the weight rule may specify that each combination of a set of non-normalized weights in the range between 0.0 and 1.0 and in increments of 0.2 is generated, and then the weights are normalized so that the sum of the weights in each weight set is 1.0. In view of this weight rule and six seed anatomical structures, 5×10 6 The graphics can be generated using a 3D computer graphics software toolset such as Blender.
[0118] Fig.14 1400 is a flowchart illustrating the processing of a component of a generated model library of an MLG system in some embodiments. A component 1400 for generating a model library is called to generate a model library (such as an arrhythmia model library) based on a seed anatomical structure, and a weight set of the seed anatomical structure is generated for each model to generate anatomical parameters and electrophysiological parameter descriptions that specify the range of values for each electrophysiological parameter. The component generates a source configuration from a combination of anatomical parameters and electrophysiological parameters. In box 1401, the component calls a component for generating multiple simulated anatomical structures to generate a simulated anatomical structure from a seed anatomical structure. In box 1402, the component calls a component for generating a source configuration to generate a source configuration based on the anatomical parameters and electrophysiological parameter descriptions of the simulated anatomical structure. In boxes 1403-1406, the component loops to generate a model for each source configuration. In box 1403, the component selects the next source configuration i. In decision box 1404, if all source configurations have been selected, the component is completed, otherwise the component continues at box 1405. In block 1405, the component calls the component that generates models, passing an indication of source configuration i to generate a model for that source configuration (model[i]). In block 1406, the component adds the generated model to the model library and then loops to block 1403 to select the next source configuration.
[0119] Fig.15is a flow chart illustrating the processing of a component for generating multiple simulated anatomical structures of an MLG system in some embodiments. Component 1500 for generating multiple simulated anatomical structures is called to generate simulated anatomical structures based on a seed anatomical structure and an associated weight set. In box 1501, the component selects the next weight set i. In decision box 1502, if all weight sets have been selected, the component is complete, otherwise the component continues at box 1503. In box 1503, the component calls the component for generating simulated anatomical structures, thereby passing an indication of the selected weight set i, and receiving an indication of anatomical parameters of the simulated anatomical structures. In box 1504, the component stores the anatomical parameters and then loops to box 1501 to select the next weight set.
[0120] Fig.16 1 is a flow chart illustrating the processing of a component of the MLG system that generates a simulated anatomical structure in some embodiments. Component 1600 that generates a simulated anatomical structure is called, thereby passing an indication of a weight set, and generating a simulated anatomical structure based on that weight set. In box 1601, the component selects the next anatomical parameter i. In decision box 1602, if all anatomical parameters have been selected, the component returns an indication of the anatomical parameters of the simulated anatomical structure, otherwise the component continues at box 1603. In box 1603, the component initializes the selected anatomical parameter i. In boxes 1604-1606, the component loops, thereby adjusting the anatomical parameter based on the weight and the value of the selected anatomical parameter of each seed anatomical structure. In box 1604, the component selects the next seed anatomical structure j. In decision box 1605, if all seed anatomical structures have been selected, the component loops to box 1601 to select the next anatomical parameter, otherwise the component continues at box 1606. In block 1606, the component sets the selected anatomical parameter i to the sum of the selected anatomical parameter i and that of the selected seed anatomical structure j divided by the weight of the selected seed anatomical structure. The component then loops to block 1604 to select the next seed anatomical structure.
[0121] Fig.171700 is a flowchart illustrating the processing of the component generating source configuration of the MLG system in some embodiments. The component generating source configuration 1700 is called to generate a source configuration for generating a model library. The source configuration is specified by a combination of simulated anatomical structures and values of anatomical parameters. In box 1701, the component selects the next simulated anatomical structure. In decision box 1702, if all simulated anatomical structures have been selected, the component is completed, otherwise the component continues at box 1703. In boxes 1703-1707, the component loops to select an anatomical parameter set for the selected simulated anatomical structure. In box 1703, the component selects the next value of the fiber structure anatomical parameter. In decision box 1704, if all values of the fiber structure anatomical parameter have been selected for the selected simulated anatomical structure, all source configurations of the selected simulated anatomical structure have been generated, and the component loops to box 1701 to select the next simulated anatomical structure, otherwise the component continues to select the value of each of the other anatomical parameters, as shown by the ellipsis. In box 1705, the component selects the next source position for the selected simulated anatomical structure and the value set of other anatomical parameters. In decision block 1706, if all source locations have been selected for the selected simulated anatomical structure and value sets for other anatomical parameters, the component loops to block 1705 to select a new value set. In block 1707, the component stores the indication of the selected simulated anatomical structure and the set of selected values for the anatomical parameters as a source configuration and then loops to block 1705 to select the next source location.
[0122] Fig.18 1 is a flow chart illustrating the processing of a component of a generation model of an MLG system in some embodiments. Component 1800 of the generation model is called, thereby passing an indication of a source configuration i, and generating a model. The model includes one or more model parameters for each vertex of a mesh of a simulated anatomical structure representing the source configuration. In box 1801, the component generates a mesh based on the anatomical parameters. The mesh may be represented by a data structure that includes, for each vertex, a reference to its neighboring vertices, which is used explicitly or implicitly together with the value of one or more model parameters to generate, for example, a voltage at that vertex. In box 1802, the component selects the next vertex j. In decision box 1803, if all vertices have been selected, the component is complete, otherwise the component continues at box 1804. In box 1804, the component selects the next model parameter k of the computational model for the selected vertex. In decision box 1805, if all model parameters have been selected, the component loops to box 1802 to select the next vertex, otherwise the component continues at box 1806. In block 1806, the component computes a selected model parameter k based on the source configuration i for the selected vertex j and then loops to block 1804 to select the next model parameter.
[0123] Fig.191 is a flowchart illustrating the processing of a component of the MLG system displaying a simulated anatomical structure in some embodiments. Component 1900 for displaying a simulated anatomical structure is called to display a simulated anatomical structure that has been generated based on a seed anatomical structure. In box 1901, the component displays a simulated representation of the simulated anatomical structure, such as graphic 1311. In boxes 1902-1906, the component loops to display a seed representation of each seed anatomical structure. In box 1902, the component selects the next seed anatomical structure for generating the simulated anatomical structure. In decision box 1903, if all seed anatomical structures have been selected, the component continues at box 1907, otherwise the component continues at box 1904. In box 1904, the component displays a seed representation of the selected seed anatomical structure, such as graphics 1301-1306. In box 1905, the component displays an arrow from the seed representation to the simulated representation. In box 1906, the component displays a weight associated with the seed anatomical structure adjacent to the arrow, and then loops to box 1902 to select the next seed anatomical structure. In decision block 1907, if the user indicates to change the weights, the component continues at block 1908, otherwise the component completes. In block 1908, the component displays a new simulated representation of the simulated anatomical structure based on the changed weights. The component may call a component to generate a simulated anatomical structure to generate the simulated anatomical structure, and call a component to generate a graphic of the simulated anatomical structure. In block 1909, the component displays the changed weights near an arrow from the seed representation of the seed anatomical structure whose weights have been changed, and then completes.
[0124] Fig. 20is a block diagram illustrating a process of bootstrapping a simulation based on an EM mesh of a previous simulation having similar anatomical parameters. A simulation may be bootstrapped based on a previous simulation, which in turn is based on an arrhythmia model of the same source configuration except for a different scar or fibrosis or arrhythmogenic substrate location. To generate an initial EM mesh, a component 2020 of generating a simulation inputs an arrhythmia model 2010 having a first scar or fibrosis or arrhythmogenic substrate location and outputs a voltage solution 2030. The component 2020 of generating a simulation includes a component 2021 of initializing a mesh to a default and a component 2022 of running a simulation. The component of initializing an EM mesh initializes the EM mesh to a default value of voltages having vertices. The component 2022 of running a simulation runs a simulation (e.g., for 3 seconds) based on the initialized EM mesh and the arrhythmia model 2010 to generate a voltage solution (e.g., 3,000 EM meshes) and outputs a voltage solution 2030. To bootstrap the simulation, the component 2060 generating the simulation inputs the arrhythmia model 2050 with the second scar or fibrosis or arrhythmogenic substrate location and outputs a voltage solution 2070. The component 2060 generating the simulation includes a component 2061 initializing the EM grid based on the voltage solution and a component 2062 running the simulation. The component 2061 initializing the EM grid based on the voltage solution inputs the voltage solution 2030 and initializes the EM grid to, for example, an EM grid corresponding to a one-second point of the voltage solution 2030. The component 2062 running the simulation runs the simulation based on the initialized EM grid. For example, if the simulation is three seconds and the initialized EM grid is based on the EM grid at the one-second point, the component 2062 running the simulation runs the simulation for another two seconds. The component 2062 running the simulation stores the voltage solution in the voltage solution storage area 2070.
[0125] Fig.21 is a flow chart illustrating the processing of a component of the MLG system that generates a voltage solution for a representative arrhythmia model of a group in some embodiments. The component 2100 that generates a representative voltage solution generates a voltage solution for an arrhythmia model i. In box 2101, the component initializes an index j to track the number of simulation steps. In decision box 2102, if index j is greater than the number of simulation steps, the component is complete, indicating a voltage solution, otherwise the component continues at box 2103. In box 2103, the component applies a computational model based on the arrhythmia model and the selected simulation step to generate a voltage solution for the selected simulation step. In box 2104, the component increments to the next simulation step and loops to box 2102.
[0126] Fig. 222204-2206, the component loops, thereby running the simulation starting from simulation step 1001 and continuing until the end of the simulation. In box 2204, the component selects the next arrhythmia model i. In decision box 2202, if all arrhythmia models have been selected, the component is completed, otherwise the component continues at box 2203. In box 2203, the component initializes the voltage solution of the selected arrhythmia model to the representative voltage solution of the first 1000 simulation steps. In boxes 2204-2206, the component loops, thereby running the simulation starting from simulation step 1001 and continuing until the end of the simulation. In box 2204, the component selects the next simulation step j starting from simulation step 1001. In decision block 2205, if the current simulation step is greater than the number of simulation steps, the component loops to block 2201 to select the next arrhythmia model, otherwise the component continues at block 2206. In block 2206, the component applies the arrhythmia model based on the simulation step to generate a voltage solution for the simulation step, and then loops to block 2204 to select the next simulation step.
[0127] Fig.23 23 is a block diagram showing a process of approximating a vector cardiogram based on a voltage solution of an arrhythmia model with different anatomical parameters. To generate a voltage solution of an arrhythmia model 2301 based on a first set of anatomical parameters, a component 2302 for generating simulations inputs the arrhythmia model and outputs a voltage solution 2303. A component 2304 for generating VCGs then generates a VCG from the voltage solution and stores it in a VCG storage area 2305. To approximate the VCG of an arrhythmia model based on a source configuration that is similar but has different anatomical parameters, a component 2314 for approximating VCGs inputs an arrhythmia model 2311, generates an approximated VCG, and outputs a VCG 2315.
[0128] Fig.242400 . The component 2400 of the approximate VCG of the MLG system in some embodiments inputs a voltage solution ("VS1") generated based on a first arrhythmia model having first anatomical parameters, and approximates the VCG of a second arrhythmia model having second anatomical parameters. In box 2401, the component sets an index i for indexing through simulation steps. In decision box 2402, if index i is greater than the number of simulation steps, the component continues at box 2408, otherwise the component continues at box 2403. In box 2403, the component sets an index j for indexing through a point (e.g., a vertex or Gaussian point) of the mesh of the arrhythmia model. In decision box 2404, if index j is greater than the number of points, the component continues at box 2407, otherwise the component continues at box 2405. In box 2405, the component sets the value of the voltage solution of the second arrhythmia model at the indexed simulation step and point ("VS2") to the corresponding value of the voltage solution of the first arrhythmia model. In block 2405, the component increments index j and loops to block 2404. In block 2407, the component increments index i and loops to block 2402. In block 2408, the component generates a VCG for a second arrhythmia model based on the generated voltage solution VS2 and is done.
[0129] Fig.25 1 is a block diagram illustrating a process of converting an arrhythmia model based on a first polyhedron into an arrhythmia model based on a second polyhedron in some embodiments. A first polyhedral arrhythmia model 2501 is input to a component for extracting surfaces 2502. The component for extracting surfaces extracts the surface of the mesh of the arrhythmia model. A component for filling with second polyhedrons 2303 inputs the surface and fills the volume within the surface with a second type of polyhedron. A component for interpolating model parameters 2504 inputs the second polyhedral mesh and the first polyhedral model and generates model parameters of the second polyhedral model, and outputs the second polyhedral model 2505.
[0130] Fig.2626 is a flowchart illustrating the processing of the component for converting a polyhedral model of the MLG system in some embodiments. The component 2600 for converting a polyhedral model is called to convert a hexahedral model into a tetrahedral model. In box 2601, the component extracts a surface from the hexahedral model. In box 2602, the component generates a tetrahedral mesh based on the surface of the hexahedral model. In box 2603, the component maps the vertices of the tetrahedral mesh to the space of the hexahedral mesh of the hexahedral model, for example, setting the position of the vertex relative to the same origin. In box 2604, the component selects the next vertex j of the tetrahedral mesh. In decision box 2605, if all such vertices have been selected, the component is completed, otherwise the component continues at 2606. In box 2606, the component selects the next model parameter k. In decision box 2607, if all model parameters have been selected for the selected vertex, the component loops to box 2604 to select the next vertex, otherwise the component continues to box 2608. In block 2608, the component sets the selected model parameter based on interpolating the values of the selected model parameter for adjacent vertices in the hexahedral model. The component then loops to block 2606 to select the next model parameter.
[0131] Patient matching system
[0132] Methods and systems are provided for identifying attributes or classifications of EM sources in vivo based on patient matching, which is matching a patient source configuration to a model source configuration of an EM source. In some embodiments, a patient matching ("PM") system uses matching model source configurations to accelerate the process of identifying attributes of a patient or using a classifier to provide a more accurate classification of a patient. In order to identify an attribute of a patient (e.g., an arrhythmia source location), the PM system generates a mapping of each model source configuration used to generate a simulated (or modeled) EM output to derived EM data derived from the simulated EM output generated based on that model source configuration. Each derived (or modeled) EM data is also mapped to an attribute associated with the EM source having the model source configuration. For example, when the EM source is a heart, the model source configuration includes configuration parameters, such as anatomical parameters and electrophysiological parameters. The configuration parameters may include a source location of an arrhythmia, a rotor type, a type of disease or condition, and the like. One of the configuration parameters (such as an arrhythmia source location) may be designated as an attribute parameter representing an attribute to be identified for a patient. The PM system generates a mapping of the model source configuration to the modeled VCG, and generates a mapping of the modeled VCG to its corresponding arrhythmia source location. The mapping of the modeled VCG to the source location can be generated in a manner similar to the manner in which the MLMO system generates training data using attributes corresponding to the markers. In some embodiments, the mapping of the derived electromagnetic data to the source configuration and attributes can be based on data collected from actual patients, rather than input and data generated by simulation.
[0133] In some embodiments, the PM system uses a mapping to identify the attributes of the patient based on the combination of the patient's patient source configuration and the patient's patient VCG. The attributes can be identified by comparing the patient VCG with each modeled VCG to identify the most similar modeled VCG, and identifying the attributes of the most similar modeled VCG as the patient's patient attributes. However, comparing the patient VCG with each modeled VCG is very expensive in computation because the number of modeled VCGs may be millions. The PM system can use various techniques to reduce computational expenses. By one technique, the PM system uses the patient source configuration to reduce the number of modeled VCGs that need to compare the patient VCG with. The PM system may allow the user to provide the patient's patient source configuration. It would be preferred to provide the value of each configuration parameter of the patient source configuration. However, in reality, the patient may only know the values of certain configuration parameters. In this case, the PM system performs comparisons based on known configuration parameter values rather than all configuration parameters. The patient's anatomical parameters can be calculated based on imaging scans. The action potential can be calculated based on the output of the basket catheter inserted into the patient's heart or the patient's history of antiarrhythmic drugs or gene therapy, and the conductivity can be calculated based on the analysis of the patient's ECG. The configuration parameters may also include electrophysiological parameters that indicate whether the action potential and / or conductivity is indicative of a disease state. In this case, if the patient's action potential or conductivity is not available, the configuration parameters indicate whether the action potential or conductivity is indicative of a disease state. The PM system may use various techniques to evaluate the similarity between the patient source configuration and the model source configuration, such as techniques based on least squares, cosine similarity, etc. The PM system may generate a similarity score for each model source configuration and identify those with similarity scores above a matching threshold as matching model source configurations.
[0134] In some embodiments, after identifying a matching model source configuration, the PM system compares the patient VCG to the modeled VCG to which the matching model source configuration is mapped. The PM system may generate a similarity score for each modeled VCG (e.g., using a Pearson correlation technique) and identify those with similarity scores above a threshold similarity as matching modeled VCGs. The PM system then identifies the patient's attributes based on the attributes of the matching modeled VCG. For example, if the attribute is the source location of an arrhythmia, the PM system may generate an average of the source locations weighted based on the similarity scores of the matching model VCGs. In this way, the PM system avoids the computational overhead of comparing the patient VCG to each modeled VCG.
[0135] In some embodiments, the PM system may employ other techniques to reduce the computational overhead of comparing the patient VCG to each modeled VCG. For example, the PM system may identify features of the VCG, such as area, maximum size, etc. The PM system may then generate an index that maps the values of the features to modeled VCGs having those values. In order to identify a modeled VCG that matches a patient VCG, the PM system identifies the features of the patient VCG and uses the index to identify a matching modeled VCG. For example, the PM system may identify a set of modeled VCGs that match that feature of the patient VCG for each feature. The PM system may then identify the modeled VCGs common to each set (e.g., the intersection of the sets) or the most common modeled VCG for the set as matching modeled VCGs. The PM system may then compare those matching modeled VCGs with the patient VCGs to identify the attributes as described above.
[0136] In some embodiments, the PM system may use patient matching to improve the classification of VCGs based on the trained classifier. The PM system may use various clustering techniques to generate clusters of similar model source configurations. Clustering techniques may include centroid-based clustering techniques (e.g., k-means clustering), supervised or unsupervised learning clustering techniques (e.g., using neural networks), etc. Through centroid-based clustering techniques, the PM system generates clusters by sequentially adding each model source configuration to the current cluster of the model source configuration that is most similar to it. The PM system can dynamically combine and split clusters, for example, based on the number of model source configurations in each cluster, the similarity of the model source configuration of one cluster to another cluster, etc. After identifying the cluster, the PM system can use components of the MLMO system to generate a classifier for each cluster. The PM system trains the classifier of that cluster based on the modeled VCG of the model source configuration of the cluster as the training data of the classifier.
[0137] In some embodiments, to generate a classification for a patient, the PM system identifies a cluster whose model source configuration best matches the patient source configuration. For example, the PM system may generate a similarity score for each cluster based on the similarity between the representative model source configuration of each cluster (e.g., having the average value of the cluster) and the patient source configuration. The PM system then selects the classifier for the cluster with the highest similarity score and applies that classifier to the patient VCG to generate a classification for the patient. Because each classifier is trained based on a cluster of similar model source configurations, each classifier is suitable for generating classifications based only on the differences (which may be subtle) in those similar model source configurations. In contrast, a classifier trained based on all model source configurations may not be able to take into account subtle differences between similar model source configurations. Therefore, a classifier trained based on a cluster of similar model source configurations can provide more accurate classifications than a classifier trained based on all model source configurations.
[0138] In some embodiments, the PM system may generate a modeled EM output for a model source configuration that assumes a standard heart orientation. However, if the orientation of the patient's heart is slightly different, the matching modeled VCG may not have properties or classifications that will be applicable to the patient due to the difference in orientation. In this case, the PM system may perform a VCG rotation before comparing the patient VCG to the modeled VCG. The PM system may rotate each modeled VCG or patient VCG. The PM system may generate a rotation matrix based on the difference in orientation. The PM system then performs matrix multiplication between each point of the VCG (e.g., x, y, and z values) and the rotation matrix (e.g., a 3×3 matrix) to generate a rotation point for the rotated VCG. The MLMO system may also rotate the VCG based on the difference in orientation.
[0139] Figures 27 to 30 is a flow chart illustrating processing of components of a PM system in some embodiments. Fig. 27 2700 is a flow chart illustrating the processing of the component for identifying attributes of the PM system in some embodiments. The component for identifying attributes 2700 identifies the patient attributes of the patient based on the patient source configuration and the patient's cardiogram (e.g., VCG). In box 2701, the component calls the component for generating simulated VCGs of the MLMO system to generate modeled VCGs, and then maps the modeled VCGs to the model source configurations from which they are generated. In box 2702, the component retrieves the patient source configuration and the patient VCG. In box 2703, the component calls the component for identifying matching VCGs, thereby passing an indication of the patient source configuration and the patient VCG to identify the patient attributes based on the matching VCG. In box 2704, the component presents the patient attributes to inform the treatment of the patient, and then completes.
[0140] Fig.282801. 2802. 2803. 2804. 2805. 2806. 2807. 2808. 2809. 2900. 3011. 312. 324. 335. 346. 357. 361. 3700. 3801. 3902. 408. 413. 425. 436. 447. 458. 468. 479. 481. 4903. 491. 409. 501. 414. 426. 437. 448. 459. 469. 4704. 481. 495. 496. 497. 498. 509. 401. 415. 498. 499. 516. 499. 527. 499. 538. 499. 540. 551. 561. 5704. 581. 599. 601. 617. 618. 629. 639. 640. 650. 661. 671. 681. 690. 701. 719. 720. 731. 742. 759. 763. 779. 781. 799. 801. In some embodiments, the PM system may include components for identifying other types of matching measurements, such as EEG and measurements corresponding to those collected via a body surface vest, a basket catheter, a cap worn by the patient, etc.
[0141] Fig.292900 is a flow chart illustrating the processing of the components of the PM system in some embodiments that identify categories based on clustering. The components 2900 that identify categories based on clustering identify the categories of patients based on the classifier trained using the cluster of similar model source configurations. In box 2901, the component calls the component that generates the cluster classifier to generate the classifier for the cluster of similar model source configurations. In box 2902, the component receives the patient source configuration and the patient VCG. In boxes 2903-2908, the component loop selects the cluster to identify the cluster that the model source configuration is most similar to the patient source configuration. In box 2903, the component initializes the variable to track the maximum similarity calculated so far. In box 2904, the component selects the next cluster. In decision box 2905, if all clusters have been selected, the component continues at box 2909, otherwise the component continues at box 2906. In box 2906, the component calls the component that calculates similarity to calculate the similarity between the model source configuration of the selected cluster and the patient source configuration. In decision box 2907, if the similarity is greater than the maximum similarity calculated so far, the component continues at box 2908, otherwise the component loops to box 2904 to select the next cluster. In box 2908, the component sets the maximum similarity to the similarity calculated for the selected cluster and sets a variable to indicate the index of the cluster with the maximum similarity calculated so far. The component then loops to box 2904 to select the next cluster. In box 2909, the component calls the component that classifies based on clusters to identify the category of the patient's VCG based on the cluster with the maximum similarity. The component that classifies based on clusters may correspond to a classification component of an MLMO system that has been adapted to input a classifier to be used. The component then completes the indicated category.
[0142] Fig.30 Flowchart showing the processing of the component generating cluster classifier of the PM system in some embodiments. Call the component generating cluster classifier 3000 to cluster the model source configuration and generate a classifier based on each cluster. In box 2001, the component generates a cluster of model source configurations. In box 3002, the component selects the next cluster. In decision box 3003, if all clusters have been selected, the component completes the indication classifier, otherwise the component continues at box 3004. In box 3004, the component calls the component generating classifier of the MLMO system, thereby passing the indication of the model source configuration of the selected cluster to generate a classifier for that cluster. Then, the component loops to box 3002 to select the next cluster.
[0143] Machine learning based on clinical data
[0144] Methods and systems are provided for adapting an MLMO system to generate a classifier based on actual patient data. In some embodiments, a machine learning based on clinical data ("MLCD") system is provided, the MLCD system (1) generating a patient classifier based on patient training data associated with an actual patient using transfer of model classifier weights, and (2) generating a patient-specific model classifier based on the model training data selected based on similarity to the patient. The patient classifier is generated by a patient classifier system of the MLCD system, and the patient-specific model classifier is generated by a patient-specific model classifier system. The term "patient classifier" refers to a classifier generated based on patient training data generated based on the patient's data, and the term "model classifier" refers to a classifier generated based on model training data generated based on a computational model of an EM source.
[0145] Patient Classifier System
[0146] In some embodiments, a patient classifier ("PC") system generates a patient classifier for classifying derived EM data derived from the EM output of an in vivo EM source. For example, the patient classifier classifies VCG derived from an ECG. The PC system accesses a model classifier, such as a model classifier generated using an MLMO system. The model classifier is generated using model training data generated using a computational model of the EM source. The model classifier includes model classifier weights learned when training the model classifier, such as weights of an activation function of a CNN. The PC system also accesses patient training data, which includes, for each patient, patient-derived EM data (e.g., VCG) and patient classifications for that patient, such as rotor position and previous ablation procedure results. An exemplary previous ablation procedure result may be that the patient had no arrhythmias for a certain period of time after ablation at a certain location with a certain burn pattern. To train the patient classifier, the PC system initializes the patient classifier weights of the patient classifier based on the model classifier weights of the model classifier, and then trains the patient classifier using the patient training data and the initialized patient classifier weights. Typically, the weights of a classifier are initialized to default values, such as all initialized to a certain value (e.g., 0.0, 0.5, or 1.0) or random values. Therefore, the learning of the classifier weights is considered to be "from scratch". The process of initializing weight values based on previously learned weights is called the "transfer" of knowledge. The knowledge gained through the previous training of the previous classifier is transferred to the training of the new classifier. The goal of the transfer is to speed up the training of the new classifier and improve its accuracy.
[0147] In some embodiments, the PC system uses a patient classifier to classify patients. For example, when the EM source is a heart, the PC system may receive a cardiogram (e.g., ECG or VCG) of the patient and apply the patient classifier to that cardiogram. Depending on the patient selected for training the patient classifier, the patient classifier may be a more accurate classifier than a model classifier trained based on model training data generated using a computational model. In addition, if the patient is similar to the patient used to train the patient classifier, the accuracy of the classifier may be even more accurate. The PC system may also identify clusters of similar patients and train a separate patient classifier (referred to as a cluster patient classifier) for each cluster. The similarity of the patients may be determined in various ways, such as based on a comparison of various characteristics, such as one or more of the derived EM data collected from the patient (e.g., cardiogram), the patient source configuration of the patient (e.g., anatomical parameters, electrodynamic characteristics), patient demographic information, etc. The cluster patient classifier for that cluster may be trained based on the VCG and corresponding markers of the patient in each cluster. When a target patient is to be classified, the PC system identifies the cluster of patients that are most similar to the target patient. The PC system then applies the cluster patient classifier of the identified cluster to the VCG of the target patient.
[0148] Patient-specific model classifier system
[0149] In some embodiments, a patient-specific model classifier ("PSMC") system generates a patient-specific model classifier for classifying derived EM data of an in vivo EM source. The PSMC system identifies a model similar to a patient. The PSMC system identifies similar models based on patient-model similarity. Patient-model similarity may be based on similarity between a model source configuration of the model and a patient source configuration of the patient and / or similarity between modeled derived EM data (e.g., VCG) of the model and corresponding patient derived EM data of the patient. For example, the PSMC system may base similarity on anatomical parameters (e.g., size of the right ventricle), certain electrophysiological parameters, etc. The PSMC system may then use the MLMO system to generate a patient-specific model classifier using the model source configuration of the similar model. The PSMC system may generate a patient-specific model classifier by first applying a computational model of the EM source to generate a modeled EM output of the EM source based on the model source configuration of the similar model. The PSMC system then generates model training data, the model training data comprising modeled derived EM data (e.g., VCG) from the generated modeled EM output and a label of the model. Alternatively, if model training data has been generated for a similar model, the PSMC system does not need to regenerate the model training data. The PSMC system then trains a patient-specific model classifier based on the model training data.
[0150] In some embodiments, after generating the patient-specific model classifier, the PSMC system applies the patient-specific model classifier to the patient's exported EM data (e.g., VCG) to generate a classification for the patient. Because the patient-specific model classifier is trained using model training data selected based on the patient, the classification provided by the patient-specific model classifier is more accurate than a model classifier trained based on a set of model training data that is not specific to the patient.
[0151] In some embodiments, the PSMC system can generate a cluster-specific model classifier for a cluster of target patients. In order to generate a cluster-specific model classifier, the PSMC system identifies a model that is generally similar to the target patient of the cluster, and then trains the cluster-specific model classifier based on the similar model. Then, the PSMC system can apply the cluster-specific model classifier to generate the classification of the target patient of the cluster. The PSMC system can also generate a cluster of target patients, and generate a cluster-specific model classifier for each cluster. Then, the PSMC system can use the cluster-specific model classifier of the cluster of which the target patient is a member to generate the classification of each target patient. The PSMC system can even use the cluster-specific model classifier to generate the classification of a new target patient. The PSMC system can identify the cluster that is most similar to the new target patient, and apply the cluster-specific model classifier of that identified cluster to the patient-derived EM data of the new target patient to generate the classification of the new target patient.
[0152] Fig.31 is a block diagram illustrating the overall processing of the patient classifier system of the MLCD system in some embodiments. The classifier component 3110 (i.e., components 3111-3119) and Figure 1 The classifier component 110 of the present invention is similar. The term "model" has been inserted into various components to emphasize that these components are used to generate classifiers based on models represented by simulated source configurations or parameter sets. Component 3120 includes a patient data storage area 3121, a component 3122 that generates patient training data, a patient training data storage area 3123, a component 3124 for training patient classifiers, and a patient classifier weight storage area 3125. The patient data storage area may include an ECG collected from a patient and corresponding tags, such as the location of a cardiac disease. For example, by generating VCG from ECG and marking VCG and storing the training data in the patient training data storage area, the component that generates patient training data generates patient training data from patient data. The component that trains the patient classifier inputs the model classifier weights from the model classifier weight storage area 3119 as a transfer of knowledge from the model classifier, and trains the patient classifier based on the patient training data. Then, the training patient classifier stores the patient classifier weights in the patient classifier weight storage area.
[0153] Fig.323200 is a flowchart illustrating the processing of the components of the patient classifier system generating patient classifiers in some embodiments. The component 3200 generating the patient classifier uses the transfer of knowledge from the model classifier to generate the patient classifier for the set of patients. In box 3201, if the model classifier has not yet been generated, the component calls the component generating the classifier to generate the model classifier based on the model training data. The component generating the classifier generates the model classifier weights for the model classifier. In box 3202, the component extracts the model classifier weights of the model classifier. In box 3203, the component generates patient training data, for example, by generating VCG and marking VCG. In box 3204, the patient classifier weights of the patient classifier are initialized to the model classifier weights by the component. In box 3205, the component calls the component training classifier to train the patient classifier based on the patient training data and the initialized patient classifier weights. Then the component is completed.
[0154] Fig.33 3300 generates a cluster patient classifier for a patient cluster. In box 3301, a component generates a patient cluster based on the similarity of the patient's clinical characteristics, source configuration, or VCG. In box 3302, the component selects the next cluster. In decision box 3303, if all clusters have been selected, the component is completed, otherwise the component continues at box 3304. In box 3304, the component calls the component that generates the patient classifier, thereby passing an indication of the selected cluster to generate a cluster patient classifier for the patients in the selected cluster, and then loops to box 3302 to select the next cluster. When calling the component that generates the patient classifier, it is not necessary to generate a model classifier for each call because it can reuse the same model classifier weights for each call.
[0155] Fig.34 is a block diagram showing components of a patient-specific model classifier system of an MLCD system in some embodiments. The PSMC system includes Figure 1110 (components 3411-3419) similar to the component 110 of the PSMC system. The PSMC system also includes a component 3430 for identifying similar cardiac configurations and a component 3440 for identifying similar VCGs, which represent two different embodiments of the PSMC system. In the first embodiment, the component for identifying similar cardiac configurations inputs the patient's cardiac configuration and the model cardiac configuration, and identifies the model cardiac configuration similar to the patient's cardiac configuration. Then, the similar model cardiac configuration is input to the component for generating simulations to generate voltage solutions and ultimately train the patient-specific model classifier. In the second embodiment, the component for identifying similar VCGs inputs the patient's VCG and training data, and identifies the VCG of the training data similar to the patient's VCG. The training data of the similar VCGs is the input to the component 3418 of the training PSMC classifier to train the patient-specific model classifier. Although not shown, the training PSMC classifier can use transfer to initialize the PSMC classifier weights. Moreover, both the first embodiment and the second embodiment can be used to generate training data based on similar cardiac configurations, and then select similar VCGs for training.
[0156] Fig.35 3500 is a flowchart illustrating the processing of a component of a PSMC system in some embodiments that generates a patient-specific model classifier. The component 3500 that generates a patient-specific model generates a patient-specific model classifier for a target patient. In box 3501, the component calls a component that identifies a similar model, thereby passing an indication of the target patient to identify a similar model. In box 3502, the component generates model training data based on the identified similar model. The component may employ an MLMO system to generate training data based on a model heart configuration of the identified similar model, or, in the case where training data has been generated, retrieve training data based on a similar VCG. In box 3503, the component calls a component that trains a classifier to train a patient-specific model classifier based on the model training data of a similar model, and then completes.
[0157] Fig.36 3600 is a flowchart illustrating the processing of the component of the PSMC system of some embodiments for identifying similar models. The component 3600 for identifying similar models identifies a model similar to a patient. In box 3601, the component selects the next model. In decision box 3602, if all models have been selected, the component completes indicating similar models, otherwise the component continues at box 3603. In box 3603, the component generates a similarity score between the selected model and the patient. The patient-model similarity score can be based on heart configuration, cardiogram, or both. In decision box 3604, if the similarity score is higher than the similarity threshold, the component continues at box 3605, otherwise the component loops to box 3601 to select the next model. In box 3605, the component designates the model as similar to the patient and loops to box 3601 to select the next model.
[0158] Patient-specific model display
[0159] Provided are methods and systems for adapting an MLMO system to support generating and displaying a representation of an EM source of a patient, the representation being based on a modeled EM output of a model similar to the patient. In some embodiments, a patient-specific model display ("PSMD") system identifies a model of an EM source that is considered similar to the patient's EM source. The PSMD system then generates a graphical representation of the patient's EM source based on the patient's clinical parameters (such as infarction and medication history). For example, if the EM source is a heart, the PSMD system may generate a graph representing an anatomical model of the patient's heart. The PSMD system then populates the graph with display values derived from the modeled EM output of a similar model. For example, the PSMD system may select an EM mesh of that modeled EM output and set the value of each vertex of the graph based on the corresponding voltage of that EM mesh. The PSMD system may also map the modeled EM output from a polyhedral mesh used in a simulation to another polyhedral mesh to generate a more realistic display, such as mapping a hexahedral mesh to a surface triangle mesh. The PSMD system may set the values corresponding to high voltages to varying intensities of red (or shades of gray) and values corresponding to low voltages to varying intensities of green. As another example, the PSMD system may select a cycle of the model EM output and set a value based on the difference or increment between the voltage of the first EM grid in the cycle and the last EM grid in the cycle. The PSMD system may also set a value based on the accumulation of increments within consecutive EM grids in the cycle (which may be weighted). The PSMD system then displays a map (e.g., using a rasterization technique) as a representation of the activity of the patient's EM sources. The PSMD system may also display a contour image of the heart based on the patient's anatomical parameters. The contour image may show the boundaries of the chambers of the heart.
[0160] In some embodiments, the PSMD system can identify a model similar to a patient by comparing the model source configuration and modeled derived EM data with the patient source configuration and patient derived EM data. Fig.28 The process shown in the figure may be a process of identifying a component that matches a VCG. However, that component may be adapted to generate a similarity score for each matching VCG so that the PSMD system may select the VCG that is most similar to the patient, and therefore the model that is most similar to the patient. Alternatively, the PSMD system may generate a value for a pixel of the map based on a weighted combination (e.g., an average) of the values of the EM grids of all matching models. The PSMD system may also weight the values based on the similarity scores of the matching models.
[0161] In some embodiments, the PSMD system may generate a sequence of graphs for display as a video representation of the activation of the patient's EM source over time. For example, the PSMD may divide the cycle into 30 display intervals and generate a graph for each display interval based on the value of the EM grid corresponding to that display interval. The PSMD system may then display the 30 graphs sequentially over the time of the cycle to provide a video based on the actual cycle time, or display the 30 graphs sequentially over a time longer than the cycle to provide a slow motion effect. The PSMD system may also achieve a slow motion effect by simulating more graphs per second than the maximum frame rate of the display. For example, if the maximum frame rate is 60 frames per second, simulating 120 graphs per second will result in one second of simulation time being displayed in two seconds. The slowest and smoothest slow motion effect may be achieved by generating one graph from the voltage solution for each simulation time.
[0162] Fig.37 3710 (i.e., components 3711-3715) and Figure 1 Component 3720 includes a component 3721 for collecting ECG, a component 3722 for generating VCG, a component 3723 for identifying similar VCGs, a component 3724 for generating display representations, and a display device 3725. The component for collecting ECGs receives the ECG of the patient whose heart will be represented by the output of the PSMD system. The component for generating VCGs receives the ECG of the patient and generates VCGs for the patient. The component for identifying similar VCGs compares the patient's VCG with the modeled VCGs in the VCG storage area to identify the modeled VCGs similar to the patient's VCGs. Similar VCGs can be identified based on a comparison of the cycles of the VCGs. Therefore, the component for identifying similar VCGs can call the component for identifying cycles to identify the cycles of each VCG. The component for generating display representations inputs similar modeled VCGs and generates a display representation of the patient's heart based on the voltage solution from which the similar modeled VCGs are derived. Then, the component for generating display representations outputs the display representations to the display device.
[0163] Fig.383800 is a flowchart illustrating the processing of a component that generates a patient heart display of a PSMD system in some embodiments. A patient VCG and a patient heart configuration are provided to a component 3800 that generates a patient heart display, and the component generates an output representation of the patient's heart. In box 3801, the component calls a component of a patient matching system that identifies a matching VCG, thereby passing an indication of the patient VCG and the patient's heart configuration to identify one or more matching VCGs. In box 3802, the component identifies the closest match of the VCG. In box 3803, the component identifies the cycle within the closest matching VCG. In box 3804, the component calls a component that calculates display values, thereby passing an indication of the identified cycle to generate display values of a graph. In box 3805, the component generates a display representation by storing the display values in a graph. In box 3805, the component displays in the graph a value that represents the outline of the patient's heart based on the anatomical parameters of that patient. In box 3807, the component outputs the graph and is completed.
[0164] Fig.39 3900 is a flowchart illustrating the processing of a component of a PSMD system that calculates display values in some embodiments. An indication of a cycle is passed to a component 3900 that calculates display values, and the component generates a display value based on the EM grid of that cycle. In box 3901, the component selects the first voltage solution of the cycle. In box 3902, the component selects another voltage solution of the cycle, such as the last voltage solution. In box 3903, the component selects the next value of the voltage solution. In decision box 3904, if all values have been selected, the component continues at box 3906, otherwise the component continues at box 3905. In box 3905, the component sets the incremental value of the selected value to the difference between the value of the first voltage solution and the last voltage solution of the cycle. Alternatively, the incremental value can be a weighted accumulation of the difference within the cycle. Then, the component loops to box 3903 to select the next value of the voltage solution. In box 3906, the component selects the next display value of the graph. In decision box 3907, if all display values have been selected, the component completes the indication display value, otherwise the component continues at box 3908. In block 3908, the component identifies adjacent incremental values that are close to the displayed value. In block 3910, the component sets the displayed value to a function of the adjacent incremental values and then loops to block 3906 to select the next displayed value. The function may be an average of the adjacent incremental values, a weighted average based on the distance between the location of the voltage solution and the location represented by the displayed value and the patient's heart, etc.
[0165] Display of electromagnetic force
[0166] Provided are methods and systems for generating visual representations of electromagnetic forces generated by an electromagnetic source in the body. In some embodiments, an electromagnetic force display ("EFD") system generates a "surface representation" of electromagnetic forces from a vector sequence representing the magnitude and direction of the electromagnetic force over time. For example, when the electromagnetic source is a heart, the vector sequence can be a vector cardiogram. The vector is relative to an origin that can be located within the electromagnetic source. In order to generate a surface representation, the EFD system identifies regions based on the origin and vector pairs for temporally adjacent vector pairs. For example, if a vector has values (1.0, 2.0, 2.0) for its x, y, and z coordinates and an adjacent vector has values (1.1, 2.0, and 2.0), then the region is defined by a triangle with vertices located at (0.0, 0.0, 0.0), (1.0, 2.0, 2.0), and (1.1, 2.0, and 2.0). Then, the EFD system displays a representation of each region to form a surface representation of the electromagnetic force. Since these regions are unlikely to be located in one plane, the EFD system can provide shading or coloring to help show that the regions are located in different planes. The EFD system can also display a representation of the electromagnetic source so that the surface representation visually emanates from the electromagnetic source. For example, the EFD system can display a heart based on the anatomical parameters of the patient from which the VCG is collected. When the electromagnetic force has a cycle, the area of the cycle forms a surface representation of that cycle. The EFD system can display surface representations of multiple cycles simultaneously. For example, when the electromagnetic source is a heart, the cycle can be based on an arrhythmia. The EFD system can also display a surface representation of each cycle in sequence (e.g., centered at the same location on the display) to show changes in the electromagnetic force over time. The EFD system can display each area of the surface representation in sequence to show the time associated with the vector. The EFD system can be used to display a simulated VCG or a VCG collected from a patient.
[0167] Fig.40 Various surface representations of vector cardiograms are shown. Image 4010 shows a display of a surface representation of a vector cardiogram based on a perspective view. Image 4020 shows a display of a surface representation of a vector cardiogram shown emanating from the heart. Images 4031-4034 show a display of a surface representation over time to show changes in electromagnetic forces.
[0168] In some embodiments, the EFD system may use a variety of animation techniques to assist the user in analyzing the VCG. For example, the EFD system may animate the display of the surface representation by displaying each region in sequence with the same timing as the actual timing of the VCG. If the period is 1000ms, the EFD system will display the region in sequence within 1000ms. When the surface representation of multiple cycles is displayed in sequence and the next region to be displayed will overlap with the previously displayed region, the EFD system may first delete the overlapping region and then display the next region. The EFD system may also allow the user to specify a portion of the surface representation of only a cycle. For example, the user may specify to display a portion corresponding to 250ms of the surface representation. In this case, the EFD system may animate the display of the portion by deleting the tail region when adding the head region. The EFD system may also allow the user to speed up or slow down the display of the surface representation.
[0169] Fig.41 4104. Detailed Description of the Invention 4105 is a flow chart illustrating the processing of a component that visualizes a VCG of an EFD system in some embodiments. A VCG is passed to a component that visualizes a VCG 4100, and the component generates a surface representation for each portion (e.g., cycle) of the VCG. In block 4101, the component displays a representation of the heart. In block 4102, the component selects the next cycle of the VCG. In decision block 4103, if all cycles have been selected, the component is complete, otherwise the component continues at block 4104. In block 4104, the component calls a component that displays a surface representation of the VCG to display a surface representation of the selected cycle, and then loops to block 4102 to select the next cycle.
[0170] Fig.42 4204, the component fills the VCG triangle with a shading that may vary based on the plane of the triangle. In box 4205, the component displays the filled VCG triangle. In box 4206, the component increments index t. In decision box 4207, if index t is greater than the number of time intervals, the component is complete, otherwise the component continues at box 4203. In box 4203, the component generates a VCG triangle based on the origin, index interval t, and previous interval t-1. In box 4204, the component fills the VCG triangle with a shading that may vary based on the plane of the triangle. In box 4205, the component displays the filled VCG triangle. In box 4206, the component increments index t. In decision box 4207, if index t is greater than the display span t 跨度 +1, then components continue at block 4208, otherwise components loop back to block 4202. In block 4208, components remove the VCG triangle at the beginning of the currently displayed span from the display and loop back to block 4202.
[0171] Simulated Calibration
[0172] Provided are methods and systems for generating a calibration set of simulated cardiograms (also referred to as model cardiograms) based on morphological similarities (such as orientation similarities and / or electrophysiological similarities) with a patient's cardiograms, performed by a computing system. Morphological similarity can be determined based on an analysis of "raw trace" data (e.g., voltage vs. time) representing the patient's ECG and / or VCG data. A calibration set can be further generated based on the similarity of additional configuration parameters used to generate the simulated cardiograms with the patient's configuration parameters. In some embodiments, a system for calibrating simulated cardiograms ("CSC") generates a calibration set. Because the CSC system generates a calibration set based on similarities with the patient's cardiograms, the calibration set represents a patient-specific simulated cardiogram set. The simulated cardiograms in the calibration set can be used to train a patient-specific model classifier with each simulated cardiogram, and each simulated cardiogram is labeled with the configuration parameters used to generate that simulated cardiogram. The simulated cardiogram can be labeled with configuration parameters that are not collected from the patient (such as the source position of the rotor). Then, the patient-specific model classifier can be used to determine the patient's configuration parameters. Like the PSMC system, the CSC system generates patient-specific model classifiers, but uses various similarity measures, such as morphological similarity.
[0173] In some embodiments, the CSC system inputs the simulated cardiograms and the simulation configuration parameters used to generate each simulated cardiogram. The simulated cardiograms can be generated by the MLMO system, such as Figure 1 The CSC system also inputs a patient electrocardiogram and patient configuration parameters. The CSC system identifies a simulated electrocardiogram with a pacing similar to the patient pacing used when collecting the patient's electrocardiogram. In order to collect the patient's electrocardiogram, the electrophysiologist (or other care provider) may place a catheter at different locations in the patient's heart, such as near the coronary sinus. For each location, the patient's heart is paced, for example at 1 Hz for 10 beats, followed by an additional stimulation beat after a certain delay. The same process of pacing at 1 Hz is then repeated multiple times at different delays in the additional stimulation beat. The same pacing process is then repeated, but at different pacing rates (e.g., 2 Hz and 4 Hz). The result is a collection of patient electrocardiograms collected based on different locations, pacing rates, and additional stimulation delays. Electrocardiograms of normal sinus rhythm beats may also be collected.
[0174] In some embodiments, the pacing positions include at least two locations (which may be distant locations) in the target chamber that is the target of the pacing or clinical procedure. For example, if the target chamber is a ventricle, the pacing positions may include a location in the right posterior ventricle and a pacing location in the left outer ventricle. Such pacing locations are generally accessible and can be easily accessed by a catheter. However, other pacing locations may be used. For example, if the only target chamber is the left ventricle, the pacing positions may include a pacing location in the left outer ventricle wall near the base (at the level of the heart valve) and a pacing location in the left ventricular septal wall near the apex. As another example, if the target chamber is the atrium, only one pacing location may be used, such as a pacing location in the coronary sinus in the left atrium. This pacing location can be paired with a normal P wave at the sinoatrial node located in the right atrium. Generally, the effectiveness of calibration increases with more pacing locations and more distant pacing locations.
[0175] The CSC system identifies simulated cardiograms that are morphologically similar to the patient's cardiogram. Morphological similarity includes both orientation similarity and electrophysiological similarity, and electrophysiological similarity includes both action potential and conduction velocity similarity. The CSC system identifies simulated cardiograms with similar orientation to simulated hearts with similar orientation to the patient's heart from cardiograms with similar pacing. In order to determine the orientation similarity, the CSC system generates QRS complexes and T wave vectors of the patient's ventricular beats and P wave vectors of the patient's atrial beats. The vector may represent the average spatial orientation of the total electric dipole at that cycle phase (e.g., QRS complex) of the cardiac cycle. Alternatively, the CSC system may use a time series of vectors (i.e., the original VCG signal) instead of an average vector. The CSC system marks each vector with its pacing (e.g., pacing position and pacing rate) and cycle phase. The CSC system generates corresponding simulated vectors from simulated cardiograms of a simulated set with similar pacing. Then, the CSC system calculates the orientation difference between the patient vector and each simulated vector of the same pacing and cycle phase. For example, the CSC system may calculate the dot product between vectors at the left ventricle ("LV") apex, the LV lateral wall, the LV posterior wall, and the LV anterior wall during the QRS complex and the T wave. The CSC system then identifies a set of simulated cardiograms with the smallest total orientation difference (e.g., a weighted average of the dot products). To calculate the orientation differences, the CSC system generates a rotation matrix that represents the orientation difference between the patient orientation and each simulated orientation in the identified set. To generate the rotation matrix, the CSC system may apply a least squares fit of the patient vectors to the simulated vectors in the set. The CSC system selects the simulated cardiogram with the smallest simulated orientation difference (as indicated by the rotation matrix) as the simulated cardiogram with similar orientation. In some embodiments, the CSC system may determine orientation similarity based on features of the raw trace data (e.g., voltage versus time) in addition to vectors derived from the QRS complex, T wave, and P wave of the ECG or VCG.
[0176] The CSC system identifies a simulated cardiogram with similar action potential from a simulated cardiogram with similar morphology, and the simulated cardiogram with similar potential represents an action potential similar to the action potential represented by the patient cardiogram. In order to determine the action potential similarity, the CSC system normalizes the ECG of the simulated cardiogram and the patient cardiogram in time and size. For example, each ECG can be normalized to one second and normalized to a certain peak QRS complex wave or P wave signal. The CSC system estimates the action potential parameters that control the duration of activated myocardial tissue (action potential duration), which change the relative duration of the ECG and / or VCG cycle phases. The action potential characteristics are heterogeneous throughout the anatomical structure and may depend on various properties of the cardiac tissue (e.g., location in the anatomical structure, type of cardiac cells, and health / disease classification). The duration of the cardiac cycle in the ECG and / or VCG represents the overall impact of the local distribution of action potential parameters on the overall cardiac dipole. The CSC system can compare the patient's VCG with the simulated VCG under the same pacing rate and pacing position. Based on the similarity of the relative sizes of the X, Y, and Z values of the QRS complex, T wave, and P wave and the deviations therebetween, the action potentials are considered similar even if the absolute timing of the cycles is different. Relative timing indicates the shape and duration (wave front, wave back, and wavelength) of the electrical waves in the myocardium. The CSC system can adjust the timing of the simulated VCG based on one or more of the following: 1) selecting a geometry that better matches the patient; 2) adjusting the overall myocardial conductivity parameters; 3) adjusting the parameters in the ionic model that determine the duration of the action potential (i.e., time constant). The latter two parameters can be changed so that the error between the absolute timing and the relative timing is below a certain threshold (e.g., 10ms or 5%). The CSC system then calculates the action potential similarity between each simulated cardiogram and each patient's cardiogram (e.g., using Pearson correlation). The CSC system then selects the most similar simulated cardiogram (e.g., action potential similarity above a threshold) as the simulated cardiogram with similar action potentials.
[0177] The CSC system identifies simulated cardiograms with similar conduction velocities from simulated cardiograms with similar action potentials, wherein the simulated cardiograms with similar conduction velocities represent conduction velocities similar to the conduction velocities represented by the patient's cardiogram. The CSC system identifies simulated cardiograms with similar conduction velocities in a manner similar to identifying simulated cardiograms with similar action potentials, except that the cardiograms are normalized in magnitude rather than in time.
[0178] The CSC system may also identify a simulated cardiogram based on heart geometry similarity and disease substrate similarity. Heart geometry similarity may be based on structural disease similarity based on the extent of structural disease in a chamber (e.g., severe cardiomyopathy in the ventricles) and measurement similarity based on a comparison of measurements of the patient's heart (e.g., collected by CT or ultrasound) with the configuration parameters of the simulated cardiogram. The disease substrate may be based on the location and size of the patient's myocardial scar. The size and location of the patient's scar may be determined by a person specifying a scar portion of the patient's heart on the geometry of the patient's heart (based on the voltage map). Disease substrate similarity is based on the degree of overlap of the patient's scar with the scar of the simulated cardiogram.
[0179] The CSC system may determine cardiac geometry similarity (e.g., structural disease similarity and measurement similarity), orientation similarity, electrophysiological similarity (e.g., action potential similarity and conduction velocity similarity), and disease substrate similarity, in any order. Specifically, given an order for determining similarity, the CSC system may determine similarity based on a set of simulated cardiograms that were determined to be similar based on a previous determination of similarity - effectively continuously filtering the set of simulated cardiograms. The CSC system may also identify, from the set of simulated cardiograms, the most similar sets for each similarity metric, and then identify the most similar simulated cardiograms from those sets. For example, the CSC system may combine the similarity scores into a final similarity score for each simulated cardiogram using a weighted average.
[0180] The CSC system may also use various machine learning ("ML") techniques (such as linear regression techniques, neural networks, etc.) to identify a calibration set of simulated cardiograms. To generate the ML calibration set, the CSC system determines a training set of simulated cardiograms that match the patient's cardiogram based on similarity of configuration parameters (e.g., morphology parameters) to corresponding parameters of the patient and based on similarity of pacing positions. For example, for each pacing position of the patient's cardiogram, the CSC identifies a set of simulated cardiograms whose configuration parameters are similar to the patient's parameters from those simulated cardiograms having that pacing position.
[0181] The CSC system then trains a mapping function that maps the simulated cardiogram to the patient's cardiogram. For example, the mapping function can be a neural network that learns its weights based on a training set. The weights represent a nonlinear transformation of the simulated cardiogram to the patient's cardiogram, which tends to minimize the difference between the configuration parameters and the patient's parameters. In order to train the mapping function, the CSC system can derive various features from the simulated cardiogram and use these features as training data. These features can be based on the magnitude of the voltage of the heart segment, the timing of the heart segment, etc. Once the mapping function is trained, the CSC system applies the mapping function to a set of simulated cardiograms that match the patient's cardiogram to generate an ML transformed simulated cardiogram. The ML transformed simulated cardiogram forms an ML calibration set of simulated cardiograms.
[0182] The ML calibration set can then be used to generate a patient-specific model classifier. The mapping function can be used in conjunction with the various techniques described above to identify simulated cardiograms that match the patient's cardiogram. For example, matching simulated cardiograms can be identified based on orientation similarity and electrophysiological similarity.
[0183] In some embodiments, the CSC system may also generate a calibration set of simulated cardiograms based on patient-specific source configurations (e.g., geometry, disease substrate or scar, action potential). In order to identify the calibration set, the CSC system identifies the simulation based on the source configuration and / or cardiogram (such as VCG) being similar to the source configuration and / or cardiogram of the patient. Then, the CSC system adopts a bootstrap technique similar to the bootstrap technique described above for accelerating the generation of the model library. For each identified simulation, the CSC system uses the value of the EM grid at a certain point of the simulation to bootstrap the patient-specific simulation. Then, the CSC system continues the simulation using the patient-specific source configuration of the patient instead of the source configuration of the identified simulation. For example, if the simulation lasts for four seconds, the CSC system can initialize the patient-specific simulation using the value from the three-second point and run the patient-specific simulation for one second. Then, the CSC system generates a patient-specific simulation VCG from the patient-specific simulation. The CSC system identifies the patient-specific simulation with the corresponding source configuration as a calibration set, and identifies the patient-specific VCG as a calibration set of simulated cardiograms. The calibration set can be used as training data for a patient-specific classifier, or can be used for other purposes, such as directly identifying source locations using the simulated source configurations in the set.
[0184] In some embodiments, the CSC system may also identify a calibration set of simulated cardiograms based on patient-specific simulated cardiograms such as VCG. To identify the calibration set, the CSC system identifies the simulation based on the source configuration being similar to the source configuration of the patient. For example, the similarity may be based on geometry and scar location. The CSC system then uses the EM data or EM output of each identified simulation and the geometry of the patient's heart to generate a patient-specific VCG for that simulation. To generate the VCG, the CSC system effectively assumes that the EM data or EM output is generated using a geometry that matches the geometry of the patient when generating the patient-specific VCG. The CSC system then identifies the simulation that best matches the VCG with the patient-specific VCG as the calibration set of the simulated cardiogram. The calibration set may be used as training data for a patient-specific classifier, or may be used for other purposes, such as directly using the source configuration of the simulation in the set to identify the source location.
[0185] In some embodiments, the CSC system may perform additional calibration based on an identified simulated VCG that best matches a patient-specific VCG by linear or nonlinear transformation of paired simulated and patient-specific VCGs. Paired simulated and patient-specific VCGs include two VCGs with similar source configurations (e.g., geometry, scar location, electrophysiological properties, source location, etc.) and electromagnetic data (e.g., electrocardiographic morphology). The CSC system may calculate a linear transformation from a least squares fit of at least one pair of simulated and patient-specific VCGs such that the transformation applied to the simulated VCG improves the similarity score with the patient-specific VCG. The CSC system may also apply a nonlinear transformation using a neural network or other machine learning technique trained on paired simulated and patient-specific VCGs to transform the simulated VCG to better match the patient-specific VCG.
[0186] Fig.434300 is a flowchart illustrating the processing of a component for identifying a simulated cardiogram of a CSC system in some embodiments. Patient cardiograms and simulated cardiograms are passed to a component 4300 for identifying simulated cardiograms, and the component identifies simulated cardiograms that are similar to each patient cardiogram. In box 4301, the component determines the similarity of cardiac geometry (e.g., structural disease similarity and / or measurement result similarity) between each patient cardiogram and each simulated cardiogram. In box 4302, the component determines the pacing similarity between each patient cardiogram and each simulated cardiogram. Pacing similarity may be based on the proximity of the pacing position and pacing rate of the simulated cardiogram to the pacing position and pacing rate of the patient cardiogram. In box 4303, the component determines the orientation similarity between each patient cardiogram and each simulated cardiogram. In box 4304, the component determines the electrophysiological similarity between each patient cardiogram and each simulated cardiogram. In block 4305, the component identifies the simulated cardiogram as a calibration simulated cardiogram based on the determined similarity, and then completes. Although not shown, the component may also determine disease substrate similarity for identifying similar simulated cardiograms. Configuration parameters used to generate simulated cardiograms are considered to be associated with those cardiograms. For example, a cardiogram may be considered to have a pacing location and disease substrate used to generate a simulated cardiogram or a patient cardiogram.
[0187] Fig.44 is a block diagram showing components of a CSC system in some embodiments. The CSC system 4400 includes a component 4401 for calibrating a simulated cardiogram. The CSC system includes a component for calibrating orientation, the component for calibrating orientation including a component 4402 for generating vectors and a component 4410 for determining morphological similarity, the component 4410 for determining morphological similarity having a component 4411 for determining orientation similarity and a component 4412 for determining electrophysiology. The CSC system includes a component 4403 for identifying orientation similarity and a component for calibrating electrophysiology, the component for calibrating electrophysiology including a component 4404 for calibrating action potentials and a component 4405 for calibrating conduction velocity. The CSC system includes a cardiac geometry component, the cardiac geometry component including a component 4406 for calibrating structural disease and a component 4407 for calibrating measurement results. The CSC system includes a component 4408 for calibrating disease substrate and a component 4409 for calibrating pacing. The CSC system accesses a simulated storage area 4420 and outputs a calibrated simulated cardiogram. The component of calibrating the simulated cardiogram calls the component of calibrating the orientation, the component of calibrating the electrophysiology, the component of calibrating the heart geometry, the component of calibrating the disease substrate and the component of calibrating the pacing to identify the calibrated simulated cardiogram. The component of calibrating the orientation, the component of calibrating the action potential and the component of calibrating the heart geometry are described in the flowchart.
[0188] Fig.454506 is a flow chart illustrating a process performed by an electrophysiologist ("EP") to collect patient data. In box 4501, the EP selects the next pacing location. In decision box 4502, if all pacing locations have been selected, then the process is complete, otherwise the process continues at box 4503. In box 4503, the EP places the catheter at the pacing location. In box 4504, the EP selects the next pacing rate. In decision box 4505, if all pacing rates have been selected for the selected pacing locations, then the process loops to box 4501 to select the next pacing location, otherwise the process continues at box 4506. In box 4506, the EP selects the next additional stimulation interval. In decision box 4507, if all additional stimulation intervals have been selected for the selected pacing locations and pacing rates, then the process loops to box 4504 to select the next pacing rate, otherwise the process continues at box 4508. In box 4508, the EP applies stimulation at the selected pacing rate. In block 4509, stimulation is applied at the additional stimulation interval, and processing loops to block 4506 to select the next additional stimulation interval.
[0189] Fig.46 4601 is a flowchart illustrating the processing of the component of calibrating the simulated cardiogram of the CSC system in some embodiments. The component 4600 of calibrating the simulated cardiogram is passed the patient cardiogram and the indication of the simulated cardiogram, and the component identifies the simulated cardiogram similar to the patient cardiogram. In box 4601, the component calls the component that generates the vector, thereby passing the indication of the patient cardiogram to generate the patient vector for each patient cardiogram. In box 4602, the component calls the simulated cardiogram that identifies the orientation similar, thereby passing the indication of the measured vector of the simulated cardiogram and each patient cardiogram, and receiving the indication of the simulated cardiogram with similar orientation in return. Boxes 4601 and 4602 represent the processing for determining the simulated cardiogram with similar orientation. In box 4603, the component calls the component of calibrating the action potential, thereby passing the indication of the simulated cardiogram and the patient cardiogram with similar orientation, and receiving the indication of the simulated cardiogram with similar action potential in return. In block 4604, the component invokes the component that calibrates conduction velocity, thereby delivering an indication that the simulated cardiogram and the patient cardiogram are similar in action potentials, and in return receiving an indication that the simulated cardiogram is similar in conduction velocity. Blocks 4603 and 4604 represent a process for determining electrophysiological similarity.
[0190] Fig.474705 is a flow chart illustrating the processing of a component that generates vectors of a CSC system in some embodiments. The component that generates vectors is called to generate vectors for each cardiogram that is passed. Although not shown, the processing of blocks 4701-4708 is performed for each cardiogram. In block 4701, the component selects the next cardiac cycle of the cardiogram. In decision block 4702, if all cardiac cycles have been selected, the component is complete, otherwise the component continues at block 4703. In block 4703, the component selects the next cardiac cycle phase of the selected cardiac cycle. In decision block 4704, if all cardiac cycle phases have been selected, the component loops to block 4701 to select the next cardiac cycle, otherwise the component continues at block 4705. In block 4705, the component selects the next cardiac position. In decision block 4706, if all cardiac positions have been selected, the component continues at block 4703 to select the next cardiac cycle phase, otherwise the component continues at block 4707. In block 4707, the component calculates a vector from the cardiogram for the heart position during the cardiac cycle phase. In block 4708, the component labels the vector with the cardiac cycle phase and the heart position, and then loops to block 4705 to select the next heart position.
[0191] Fig.48 4 is a flowchart illustrating the processing of components of a CSC system that identify similar orientations in some embodiments. Component 4800 that identifies similar orientations is called, thereby passing an indication of a simulated cardiogram and a patient vector for each patient cardiogram. Although not shown, blocks 4801-4807 are executed for each patient cardiogram. In block 4801, the component selects the next simulated cardiogram. In decision block 4802, if all simulated cardiograms have been selected, the component continues at block 4808, otherwise the component continues at block 4803. In block 4803, the component calls the component that generates the vector, thereby passing an indication of the simulated cardiogram, and receiving the simulated vector in return. In block 4804, the component selects the next simulated vector. In decision block 4805, if all simulated vectors have been selected, the component continues at block 4807, otherwise the component continues at block 4806. In block 4806, the component calculates the orientation difference between the selected simulated vector and the corresponding patient vector, and loops to block 4804 to select the next simulated vector. In block 4807, the component calculates the orientation similarity for the selected simulated cardiogram, and then loops to block 4801 to select the next simulated cardiogram. In block 4808, the component identifies and returns the simulated cardiogram with the highest orientation similarity, and then completes.
[0192] Fig.494904 is a flowchart illustrating the processing of the component of calibrating heart geometry of the CSC system in some embodiments. The component 4900 of calibrating heart geometry is called to identify a simulated cardiogram that is similar in heart geometry to the patient's cardiogram. In box 4901, the component receives an assessment of the patient's structural disease. In box 4902, the component identifies a simulated cardiogram with a similar structural disease. In box 4903, the component receives the patient's geometric measurements. In box 4904, the component selects the next identified simulated cardiogram. In decision box 4905, if all simulated cardiograms have been selected, the component continues at box 4907, otherwise the component continues at box 4906. In box 4906, the component calculates the geometric score between the selected simulated cardiogram and the patient, and then loops to box 4904 to select the next identified simulated cardiogram. In box 4907, the component selects the best matching simulated cardiogram as a simulated cardiogram with similar heart geometry based on the geometric score and returns it.
[0193] Fig.50 5001-5006. It is a flowchart showing the processing of the component of the calibration action potential of the CSC system in some embodiments. The component 5000 of the calibration action potential identifies the simulated cardiogram whose action potential is similar to the patient's cardiogram. Although not shown, the processing of boxes 5001-5006 is performed for each patient's cardiogram. In box 5001, the component generates a normalized ECG for the patient's cardiogram. In box 5002, the component selects the next simulated cardiogram. In decision box 5003, if all simulated cardiograms have been selected, the component continues at box 5006, otherwise the component continues at box 5004. In box 5004, the component generates a normalized simulated ECG for the selected simulated cardiogram. In box 5005, the component calculates the action potential similarity between the normalized patient ECG and the simulated ECG, and then loops to box 5002 to select the next simulated cardiogram. In box 5006, the component selects and returns the simulated cardiogram with the highest action potential similarity.
[0194] Fig.51is a flow chart illustrating the processing of a component of a CSC system that generates a mapping function in some embodiments. A component 5100 that generates a mapping function identifies a set of simulated cardiograms that match a patient's cardiogram to use as training data, and then trains the mapping function. In box 5101, the component selects the next pacing position of the patient's cardiogram. In decision box 5102, if all pacing positions have been selected, the component continues at box 5107, otherwise the component continues at box 5103. In box 5103, the component selects the next simulated cardiogram with that pacing position. In decision box 5104, if all such simulated cardiograms have been selected, the component loops to box 5101 to select the next pacing position, otherwise the component continues at box 5105. In decision box 5105, if the configuration parameters of the simulated cardiogram match the patient configuration parameters, the component continues at box 5106, otherwise the component loops to box 5103 to select the next simulated cardiogram. In block 5106, the component adds the simulated cardiogram to the training data and loops to block 5106 to select the next simulated cardiogram. In block 5107, the component trains the mapping function to map the simulated cardiogram of the training data to the patient cardiogram and is done.
[0195] Fig.52 is a flow chart illustrating the processing of a component of a CSC system that transforms a simulated cardiogram in some embodiments. The component 5200 that transforms a simulated cardiogram applies a mapping function to the simulated cardiogram to generate an ML calibration set for the simulated cardiogram. In box 5201, the component selects the next pacing position of the patient's cardiogram. In decision box 5202, if all pacing positions have been selected, the component is complete, otherwise the component continues at box 5203. In box 5203, the component selects the next simulated cardiogram with that pacing position. In decision box 5204, if all such simulated cardiograms have been selected, the component loops to box 5201 to select the next pacing position, otherwise the component continues at box 5205. In decision box 5205, if the configuration parameters of the simulated cardiogram match the patient configuration parameters, the component continues at box 5206, otherwise the component loops to box 5203 to select the next simulated cardiogram. In block 5206, the component applies the mapping function to the simulated cardiogram to generate an ML transformed simulated cardiogram. In block 5207, the component adds the ML transformed simulated cardiogram to the ML calibration set and loops to block 5203 to select the next simulated cardiogram.
[0196] Although primarily described in the context of an electromagnetic source being a heart, the CSC system can be used to calibrate the simulated electromagnetic output of other electromagnetic sources within the needle body. The CSC system and other systems described herein can be used, for example, when the body is a human body or the body of another animal and the electromagnetic source is a heart, brain, liver, lung, kidney, muscle, or another body part that generates an electromagnetic field that can be measured from outside the body or inside the body. Moreover, patient pacing can be performed using an invasive pacing device (such as a catheter) that applies electromagnetic pulses from inside the electromagnetic source. Patient pacing can be performed using a non-invasive pacing device (such as a magnetic resonance scanner) that applies electromagnetic pulses from outside the electromagnetic source. The non-invasive pacing device can generate an electromagnetic field to pace the patient's electromagnetic source.
[0197] User interface for calibrating orientation
[0198] In some embodiments, a user interface calibration ("UIC") system of the CSC system provides a user interface for manually calibrating the orientation of a patient's VCG relative to a simulated VCG. The UIC system accesses a patient's ECG collected during normal sinus rhythm and / or based on pacing at various pacing locations, which can be identified based on research imaging data (e.g., from a chest x-ray or catheter system positioning system). The UIC system then extracts segments from the ECG whose activation source locations are known and then denoises the segments. The UIC system then generates a VCG from these segments. The UIC system then selects a simulated electrocardiogram whose source location is similar to that of the accessed patient ECG.
[0199] Fig.53A user interface for manually calibrating orientation is shown. Display 5300 includes a patient representation 5310 and a simulated representation 5320. To generate the patient representation, the UIC system presents the patient VCGs 5311 and 5312 of the patient source position as a three-dimensional ("3D") surface. The patented VCGs may be superimposed on a torso model 5315 to provide an anatomical reference system that may be rotated in 3D, such as a coronal view and / or an orthogonal view. To generate the simulated representation, the UI system presents simulated VCGs 5321 and 5322 from a selected simulated cardiogram with simulated source positions similar to the patient source positions, which may also be superimposed on the torso model 5325. The UIC system may also superimpose a representation of a heart 5326 based on the heart geometry of the simulated cardiogram. The user may then rotate the heart geometry and the simulated VCGs (which rotate with the geometry of the heart) to align the simulated VCGs with the patient VCGs (based on a visual comparison). To assist in alignment, the UIC system may also generate an alignment score (e.g., based on a least squares fit) indicating how closely the rotated simulated VCG is aligned with the patient's VCG. When the user determines the VCG alignment, the UIC system then generates a rotation matrix based on the amount of rotation of the heart geometry required to align the simulated VCG with the patient's VCG. The CSC system may use the rotation matrix to identify similarly oriented echocardiograms.
[0200] Marking ablation pattern
[0201] Methods and computing systems for identifying ablation patterns and methods for treating patients based on the identified ablation patterns are provided. In some embodiments, an ablation pattern identification ("API") system identifies an ablation pattern of an EM source used to treat a patient. The API system accesses the patient's EM output of an EM source (e.g., a heart or brain). The API system identifies non-ablation pattern information of a non-ablation pattern simulation based on a simulated source configuration rather than based on an ablation pattern. For example, the non-ablation pattern information can be a simulated EM output (e.g., a simulated ECG) generated from a non-ablation pattern simulation. The API system identifies the non-ablation pattern information based on the similarity of the simulated EM output of the non-ablation pattern simulation to the patient's EM output. The API system then identifies the ablation pattern based on the identified non-ablation pattern information and the ablation pattern information associated with the ablation pattern simulation. For example, the ablation pattern can be identified based on a mapping of the simulated EM output to the ablation pattern used in the ablation pattern simulation. Each ablation pattern simulation is generated based on a simulated source configuration and an ablation pattern. The API system then outputs an indication of the identified ablation pattern as a potential ablation pattern of the patient's EM source.
[0202] In some embodiments, the API system accesses a model library of non-ablation pattern simulations and simulation-derived EM data generated from the simulated EM output of the model library. The model library may be generated by the MLMO system. This model library is called a "non-ablation pattern model library" and the simulation is called a "non-ablation pattern simulation" because the simulation is based on a simulated source configuration that does not include an ablation pattern. For example, if the EM source is a heart, the non-ablation pattern simulation may be based on a simulated source position of an arrhythmia without adopting an ablation pattern, and the simulated derived EM data may be a VCG. The API system also generates an ablation pattern model library, which may include one or more ablation pattern simulations for each simulated source configuration used to generate the non-ablation pattern model library. For each simulated source configuration, the API system performs an ablation pattern simulation for each of a set of ablation patterns. Each ablation pattern simulation is used to identify the result of that ablation pattern applied to the simulated source position of the simulated source configuration. If the ablation pattern simulation does not result in an arrhythmia, the ablation pattern is identified as successful for the arrhythmia for which the simulated source configuration was stopped. Otherwise, the ablation pattern is identified as unsuccessful. For example, if the set of ablation patterns includes 10 ablation patterns, the API system generates 10 ablation pattern simulations for each simulated source configuration. The number of successful ablation patterns for a simulated source configuration may vary from 0 to 10. For example, one of the simulated source configurations may have 2 successful ablation patterns, while another may have 7 successful ablation patterns.
[0203] In some embodiments, the API system identifies one or more potential ablation patterns for a patient by comparing the patient EM output (or patient-derived EM data) with the simulated EM output (or simulated-derived EM data) to identify a matching simulated EM output. The API system selects the simulated source configuration used in the non-ablation pattern simulation from which the matching simulated EM output is generated. The API system then identifies the ablation pattern simulation based on that selected simulated source configuration. The API system selects the successful ablation patterns of those ablation pattern simulations as potential ablation patterns. For example, if the EM source is the patient's heart and the patient-derived data is a VCG, the API system uses the patient's VCG to select the simulated source configuration for generating the non-ablation pattern simulation that best matches the simulated VCG. The API system then identifies the ablation pattern simulation based on the selected simulated source configuration that produced the successful ablation pattern. The API system then outputs the successful ablation pattern as the patient's potential ablation pattern.
[0204] In some embodiments, the API system is used to assist in a method of treating a patient, such as when treating a patient with an arrhythmia (or more generally with tachycardia). An ablation assistance ("AA") system is used to assist a physician in treating a patient. A patient's VCG of the patient collected during an arrhythmia is input into the AA system. The AA system then provides the patient's VCG to the API system. In response, the API system outputs an indication of one or more potential ablation patterns for ablation, and may output one or more target sites for ablation. The physician then selects the potential ablation pattern and the target site. To assist in the selection, the AA system may display the ablation pattern superimposed at the target site on a representation of the patient's heart. The displayed representation may be based on the anatomical characteristics of the patient's heart. When a stereotactic body radiation therapy ("SBRT") device is used, the anatomical characteristics of the patient may be determined based on images collected as part of the treatment (e.g., "scout" images). The AA system may provide the selected ablation pattern and source location to an ablation device (e.g., an SBRT device) for automatic positioning. Alternatively, the physician may manually position the ablation device (e.g., a catheter). When the patient is ready for ablation, energy is activated to perform ablation based on the selected ablation pattern.The ablation device can be a SBRT device, an ablation catheter, a cryoablation catheter, an implantable pulse generator, or the like.
[0205] Fig.54 is a flowchart illustrating the overall processing of the API system in some embodiments. A component 5400 for identifying an ablation pattern identifies an ablation pattern based on patient-derived EM data. In box 5401, the component accesses the exported EM data (e.g., VCG). In box 5402, the component identifies a non-ablation pattern simulation of a model library based on simulated exported EM data that best matches the patient-derived EM data. In box 5403, the component retrieves a simulation source configuration for generating the identified non-ablation pattern simulation. In box 5404, the component identifies an ablation pattern simulation performed using the retrieved simulation source configuration. In box 5405, the component retrieves an ablation pattern associated with the identified ablation simulation. In box 5406, the component outputs an indication of retrieving the ablation pattern and then completes.
[0206] Fig.5555 is a flowchart illustrating the processing of a function for generating a mapping of an API system in some embodiments. The function 5500 for generating a mapping generates a mapping function that can be used to identify potential source locations and potential ablation patterns. In box 5501, the component accesses an ablation pattern simulation. In box 5502, the component selects the next ablation pattern simulation. In decision box 5503, if all ablation pattern simulations have been selected, the component continues at box 5506, otherwise the component continues at box 5504. In box 5504, the component generates a feature vector based on the simulated source configuration of the selected ablation pattern simulation. The features of the vector may include all simulated source configurations or a subset of the simulated source configurations that are particularly relevant to identifying source locations and ablation patterns, as well as other information, such as simulated EM data generated from a non-ablation pattern simulation using a simulated source configuration. In box 5505, the component marks the feature vector based on the simulated source location and ablation pattern of the selected ablation pattern simulation, and then loops to box 5502 to select the next ablation pattern simulation. In block 5506, the component trains the mapping function using the feature vectors and labels and then completes.
[0207] Fig.56 5600 is a block diagram illustrating the processing of a method for treating a patient during ablation in some embodiments. As an initial step in the processing, method 5600 accesses patient-derived EM data (e.g., VCG). In step 5601, the method identifies the source location and ablation pattern to be used. This identification may be performed by generating a feature vector based on features derived from the patient's patient source configuration and the patient-derived EM data, and then applying a mapping function trained by the API system. Alternatively, the identification may be performed by identifying a non-ablation pattern simulation with simulated EM data that best matches the patient's EM data, and then using a simulated source configuration to identify an ablation pattern simulation generated using that simulated source configuration. In step 5602, the method positions or guides (i.e., aims) an ablation device (e.g., a neuromodulator) based on the identified source location. In step 5603, the method uses an ablation device (e.g., a neuromodulator) to apply or guide energy at a target site derived from the identified source location and based on the identified ablation pattern.
[0208] Fig.57is a block diagram showing components of the API system in some embodiments. A component 5702 for generating ablation pattern simulation inputs a parameter set of an EM source model 5701 and a source configuration, and performs a simulation based on the source configuration using a heart model. The ablation pattern simulation component then stores a mapping of the source configuration to the ablation pattern in a source configuration / ablation pattern mapping storage area 5703. A component 5704 for guiding ablation inputs a patient source configuration and patient exported EM data, and accesses an exported EM data / source configuration mapping storage area 5707 and a source configuration / ablation pattern mapping storage area to identify a source position and an ablation pattern, and controls the execution of ablation. A component 5705 for sending instructions inputs a source position and an ablation pattern, and sends instructions to an ablation device 5706.
[0209] Fig.58 is a flow chart illustrating the processing of a component of an API system that generates an ablation pattern simulation in some embodiments. The component 5800 that generates an ablation pattern simulation inputs a source configuration and maps the source configuration to a successful ablation pattern. The component is called for each source configuration. In box 5801, the component accesses the ablation pattern to be used in the ablation pattern simulation. The API system may use the same ablation pattern for each source configuration. Alternatively, the API system may select an ablation pattern based on the source position because different ablation patterns may be more effective at different source positions. In box 5802, the component selects the next ablation pattern and sets the variable i to the selected ablation pattern. In decision box 5803, if all ablation patterns have been selected, the component continues at box 5810, otherwise the component continues at box 5804. In box 5804, the component initializes the variable j to zero to track each step of the ablation pattern simulation. In decision box 5805, the component increments the variable j and determines whether the variable j is greater than the number of simulation steps, thereby indicating that the ablation pattern simulation is complete. If completed, the component continues at box 5809, otherwise the component continues at box 5806 to perform the next step of the simulation. In box 5806, the component applies the model to the selected ablation pattern for the next step of the ablation pattern simulation. In decision box 5807, if the ablation pattern simulation indicates that the arrhythmia has stopped, the component continues at box 5808, otherwise the component loops to box 5805 to perform the next step of the simulation. In box 5808, the component marks the ablation pattern as successful and loops to box 5802 to select the next ablation pattern. In box 5809, the component marks the ablation pattern as unsuccessful because the ablation simulation has been completed but the arrhythmia has not stopped. The component then loops to box 5802 to select the next ablation pattern. In box 5810, the component maps the source configuration to the successful ablation pattern and is done.
[0210] Fig.595 is a block diagram illustrating the processing of the guided ablation component of the API system in some embodiments. The guided ablation component 5900 is called to guide ablation for a patient. In box 5901, the component receives an image of the patient's EM source. In box 5902, the component generates a patient source configuration based on the image and other information provided about the patient (such as scar location, previous ablation, current medication, etc.). In box 5903, the component receives patient-derived EM data (e.g., VCG). In box 5904, the component calls a component that identifies potential ablation patterns. In box 5905, the component calls a component that selects an actual ablation pattern to select an ablation pattern for ablation. In box 5906, the component outputs ablation instructions generated based on the actual ablation pattern. The ablation instructions can be provided directly to the ablation device or displayed to the physician. The component is then completed.
[0211] Fig.60 6000 is a flowchart illustrating the processing of a component for identifying potential ablation patterns of an API system in some embodiments. Component 6000 for identifying potential ablation patterns is called to identify potential ablation patterns for ablation. In box 6001, the component receives a patient source configuration and patient-derived EM data. In box 6002, the component identifies one or more ablation pattern simulations based on the patient source configuration and the patient-derived EM data. In box 6003, the component selects the next matching ablation pattern simulation. In decision box 6004, if all matching ablation pattern simulations have been selected, the component is completed, otherwise the component continues at box 6005. In box 6005, the component adds the matching ablation pattern and source position of the selected matching ablation pattern simulation to the instruction set. The component can also add a target volume to the ablation instruction. The target volume can be a parameter of the source configuration used when generating a non-ablation pattern simulation. Then, the component loops to box 6003 to select the next matching ablation pattern.
[0212] Fig.61 6100 is a flowchart illustrating the processing of a component of an API system in some embodiments that selects an actual ablation pattern. Component 6100 that selects an actual ablation pattern is called to pass an indication of the patient's anatomical characteristics, potential ablation patterns, and corresponding source locations. The component identifies the actual ablation pattern to be used in ablation. In box 6101, the component displays a 3D representation of the patient's EM source based on the patient's anatomical characteristics. In box 6102, the component displays a list of potential ablation patterns and corresponding source locations. In box 6103, the component receives a selection of a potential ablation pattern. In box 6104, the component superimposes the selected partial ablation pattern and source location on the 3D representation. In decision box 6105, if the physician has been instructed to use the selected potential ablation pattern as the actual ablation pattern, the component is completed, otherwise the component loops to box 6103 to display another potential ablation pattern.
[0213] Remote collection of EM data
[0214] In some embodiments, a remote EM data collection ("REMDC") system may receive EM data from an EM source collected from a patient who is not in a clinical setting. For example, a patient may go to a clinic to be fitted with a portable EM data collection ("PEMDC") device (e.g., a Holter monitor) that collects EM data (e.g., ECG). In addition, an over-the-counter PEMDC (e.g., a wearable device such as a smartwatch; an external mobile device equipped with an electromagnetic sensor; an epidermal, dermal, or subcutaneous embedded electromagnetic sensor; smart clothing with an embedded electromagnetic sensor) may be used. When the patient leaves the clinic, the PEMDC device may periodically or continuously collect the patient's EM data. When the patient is away from the clinic, the PEMDC device allows the collected EM data to be transmitted to the REMDC system. For example, the PEMDC device may include a wireless interface to transmit the collected EM data using WiFi, cellular, Bluetooth, or other connections. When a WiFi connection is used, the PEMDC device may transmit the collected EM data directly to the REMDC system as long as the PEMDC device can connect to a WiFi network with Internet access. When using a cellular connection (e.g., a cellular transmitter / receiver embedded in the PEMDC device), the PEMDC device can transmit the collected EM data directly to the REMDC system as long as the PEMDC device is within range of the cellular network. When using a Bluetooth connection (e.g., a Bluetooth transmitter / receiver embedded in the PEMDC device), the PEMDC device can transmit the collected EM data to a smart phone, smart watch, desktop computer, or other Bluetooth-enabled device. The PEMDC device may also have a wired interface to transmit the collected EM data to a computing device (e.g., a laptop) via a wire (e.g., a universal serial bus ("USB") cable), which forwards the collected EM data to the REMDC system. The PEMDC device may transmit the collected data on a schedule (e.g., once an hour), once collected, upon request by the REMDC system, or on some other criteria (such as based on an analysis of the collected EM data performed by a program installed on the PEMDC).
[0215] When the REMDC system receives the collected EM data, the collected EM data may be processed by any system described herein. For example, when the EM data is an ECG, the REMDC system may use a classifier generated by the MLMO system to identify the source location based on the collected ECG. As another example, the REMDC system may use the PSMC system to generate a patient-specific model classifier, which may then be used to identify the source location. As another example, the REMDC system may use the CSC system to generate a simulated calibration set.
[0216] Because the REMDC system can process the collected EM data before the patient returns to the clinic, the care provider will be able to analyze the output of the various systems and determine a course of action (e.g., recommended procedures, ablations, and ablation patterns) before the patient returns. In this case, delays associated with the various systems processing the EM data after the EM data is collected while the patient is in the clinic can be avoided. In addition, when the EM data is collected before the patient returns to the clinic, the cost of the computing resources required to process the EM data can be greatly reduced. Moreover, before the patient returns to the clinic, the care provider can check the output of the various systems and require the patient to return earlier or later than the expected time based on that check. Finally, using the results of the collected EM data, the care provider can better allocate and execute the necessary time and other medical resources required to give the best patient care.
[0217] Recording patient research results on a distributed ledger
[0218] Provided are methods and systems for storing the results of a study of a procedure performed on a patient in a distributed ledger. In some embodiments, a distributed ledger for study results ("DLSR") system stores the results of a study (or procedure) of a procedure performed on a patient's electromagnetic source. A computational calibration program can be used to determine a program target for a program based on a mapping of electromagnetic data derived from an electromagnetic source to a program target. The computational calibration program can be a classifier generated using mapping training to identify program targets. An MLMO system can be used to generate such a classifier. Alternatively, the computational calibration program can identify the mapped electromagnetic data that is most similar to the patient's electromagnetic data, and identify that mapped program target for use in the program. The mapped electromagnetic data can be simulated or actual patient data. Hereinafter, an embodiment of a computational target program is described as using a classifier to identify a program target.
[0219] In some embodiments, the electromagnetic source can be a heart, and the procedure can be an ablation procedure performed by a physician based on an ablation target identified using a classifier. The DLSR system can apply a classifier to identify an ablation target of the heart based on the patient's electrocardiogram. Each classifier can be trained using training data generated from modeled electrocardiograms of multiple heart configurations. The modeled electrocardiogram can be generated using a computational model of the heart, which models the electromagnetic output of the heart over time based on the heart configuration. The DLSR system selects a classifier based on the patient's electrocardiogram that matches the modeled electrocardiogram. The DLSR system receives the results of the ablation procedure performed on the patient based on the identified ablation target. For each ablation procedure, the DLSR system generates an ablation procedure record of the results of the ablation procedure. The ablation procedure record identifies the patient and any other relevant parties in the ablation procedure (e.g., physicians, insurance companies, regulators, and healthcare providers), and includes a reference to the results of the ablation procedure. The DLSR system publishes each ablation procedure record for recording in a distributed ledger for access by licensees.
[0220] In some embodiments, an interested party may store a reference to the results of an ablation procedure and the patient's electronic medical record. The interested party storing the results of the ablation procedure may control a party's access to the results. The party may be provided with access to a classifier trained using training data generated from a modeled electrocardiogram based on the quantity and quality of the ablation procedure records. The ablation procedure may be one of several ablation procedures performed on a patient during an ablation study, and the patient's ablation procedure records may be stored in the same block of a distributed ledger. Each block of the distributed ledger may store ablation procedure records for only one patient. The computing resources required to maintain the distributed ledger may be provided by a miner of the distributed ledger. Access to a classifier trained using training data generated from modeled electrocardiograms of multiple heart configurations may be provided to the miner based on mining performed by the miner. The miner for mining blocks may be selected based on a proof-of-stake consensus algorithm.
[0221] Fig.62 62 is a block diagram illustrating components of a DLSR system in some embodiments. The DLSR system includes an MLMO system 6210, a component 6230 for collecting electrophysiological ("EP") data, a research system 6240, a transaction generator system 6250, stakeholder systems 6261-6264, a miner system 6270, a system 6280 for accessing a distributed ledger, and a distributed ledger 6290. In some embodiments, the MLMO system generates a cardiac configuration, applies a computational model of the heart to simulate the function of the heart, and generates an electrocardiogram based on the simulated function. The MLMO system stores the results in an MLMO storage area 6211. The MLMO storage area may store a mapping of the modeled electrocardiogram to a source location.
[0222] In some embodiments, the MLMO system may define tasks related to supporting the MLMO system, such as running simulations and mining distributed ledgers. Tasks may be implemented as containers, which are deployed and arranged by container-based systems (such as systems based on Amazon Web Services ("AWS") Elastic Container Repository or AWS batch processing). The MLMO system may allow a miner or other contributor to run tasks on the server of the MLMO system. For example, a miner may run a mining task. A report may be generated, which identifies each contributor running a task and the computing resources used to run the task. Contributors may be charged for the computing resources used (e.g., CPU, storage devices), and task tokens may be awarded to contributors as incentives for running tasks. For example, based on the computing resources used to run tasks, one or more task tokens recorded in a distributed ledger may be awarded to a miner running a mining task to mine blocks or a contributor running a simulation task. Then, the owner of the task token may exchange access to various resources tracked in the distributed ledger with the task token. For example, access to the results of certain studies may be exchanged with task tokens. The number of task tokens awarded to the miner can also be used in the proof-of-stake consensus algorithm to select the miner to mine blocks. In order to support the operation of the simulation, the MLMO system can maintain a queue of simulation tasks to be run or completed. When a simulation task is to be run, the code of the container removes the simulation task from the top of the queue and runs the simulation task. For example, if the simulation has not yet been completed, the queue will contain simulation tasks to complete that simulation task. Whenever a simulation task does not complete the simulation (for example, because the contributor has signed a contract to spend a certain amount of computing resources, and the amount is not enough to complete the simulation task), the container running that simulation task will add a new simulation task to the queue (for example, in front of the queue or more precisely the double-headed queue or double-ended queue) to continue the simulation. In addition to the simulation task, the MLMO system can also add pre-simulation tasks and post-simulation tasks to the queue. Pre-simulation tasks may be related to constructing a model or initializing a simulation. Post-simulation tasks may be related to training classifiers (or other machine learning), verifying simulation results, and inferring based on simulations.
[0223] The patient storage area 6220 stores the patient's electronic medical record. Although shown as a single database, the electronic medical record of a single patient may be stored by various entities, such as hospitals, physicians, laboratories, etc. In addition, the electronic medical record can be accessed through a distributed ledger system, which can be a licensed distributed ledger system, or a public distributed ledger system that complies with various regulations, such as the Health Insurance Portability and Accountability Act ("Health Insurance Portability and Accountability Act, HIPPA"). The system 6230 for collecting EP data collects the patient's electrophysiological data and stores the electrophysiological data in the EP study storage area 6231. The study system 6240 inputs the patient's electrophysiological data for the EP study and information from the patient's electronic medical record, selects a classifier from the MLMO storage area, applies the classifier to the patient data to identify ablation targets, and stores the study information in the study storage area 6241.
[0224] The transaction generator system 6250 accesses the research storage area and generates records (e.g., transactions) for storage in a distributed ledger. The transaction generator system may also receive information from an insurance company system 6261, a regulator system 6262, a healthcare provider system 6263, and a physician system 6264. Once a record is generated, the relevant party may use its private key to sign the record. In this case, the transaction generator system may send the record to the insurance company system, the regulator system, the healthcare provider system, and / or the physician system for signing. Then, the transaction generator system publishes the record to the miner system 6270. The miner system may implement a proof-of-stake consensus algorithm or some other consensus algorithm for identifying a miner system for generating a block storing a distributed ledger 6290. The distributed ledger includes a box 6291, each of which may represent a study of a single patient, including transactions for each procedure performed as part of that study. The system 6280 for accessing the distributed ledger may be accessible to the distributed ledger based on permission. For example, a certain class of systems accessing a distributed ledger may be configured with a private key for accessing certain records that have been encrypted with a public key. Alternatively, a system accessing a ledger may access a front-end system of the distributed ledger that controls access to the distributed ledger.
[0225] Fig.636310 includes a block header 6211 and a transaction 6212. The block header may include a block hash, which is a hash of the data of the block, including a hash of the root node of a Merkle tree of the previous block hash of the previous block and the hash of the transaction. Transaction 6320 may include a timestamp when the transaction was generated, a study identifier, a program identifier, a patient identifier, a physician identifier, a healthcare provider identifier, an insurance provider, and a regulator identifier. The transaction may also include signatures of various parties confirming the transaction. The transaction may also include various pointers to information related to the study, such as a pointer to a study storage area and a pointer to summary information generated from the study storage area, the patient storage area, and the EP study storage area.
[0226] Fig.64 6401, the component accesses the patient storage area and the EP study storage area to retrieve information related to the patient. In box 6402, the component selects a classifier from the MLMO storage area based on matching with the patient data and electrophysiological data. In box 6403, the component applies the classifier to the patient's electrocardiogram. In box 6404, the component presents one or more ablation targets to the physician. In box 6405, the component receives the ablation results of the procedure. In box 6406, the component stores the results in the study storage area and then completes.
[0227] Fig.65 6500 can be run periodically to generate transactions for storage in a distributed ledger. In box 6501, the component retrieves a reference to the results of the procedure from the study storage area. In box 6502, the component identifies the parties of the procedure based on the patients identified in the results. In box 6503, the component can generate a timestamp. In box 6504, the component generates a transaction, which identifies the parties, timestamps, and references to the results stored in the study storage area. In box 6505, the component can obtain signatures from the parties. The parties may need permission to record the transaction in the distributed ledger. In box 6506, the component publishes the transaction to the miner system and then completes.
[0228] Fig.666600 can be called periodically to generate one or more program blocks to be stored in a distributed ledger. In box 6601, the component executes the proof-of-stake consensus algorithm to determine whether to select the miner system to generate the next block. In decision box 6602, if selected, the component continues at box 6603, otherwise the component is completed. In box 6603, the component adds the studied transaction to the block. In box 6604, the component generates a Merkle tree and adds it to the block. In box 6605, the component adds the hash of the previous block to the block. In box 6606, the component adds the address of the miner to the block. In box 6607, the component adds the hash of the entire block to the block. In box 6608, the component adds the signature of the miner to the block. In box 6609, the component publishes the block and then completes. The component may also add a creation timestamp of the block, which may be used by other miner systems to verify that the block was created by the miner identified by the proof-of-stake algorithm.
[0229] Image enhancement using source location
[0230] Methods and systems are provided for enhancing an image of an organ using a patient source location of a condition of the patient organ. In some embodiments, an image enhanced using source location ("AISL") system receives image data including an image of a patient organ, receives derived EM data derived from an EM output of the patient organ, identifies a patient source location of the condition of the organ based on the derived EM data, and enhances the image data using the patient source location. For example, the image data may be a 3D image of a heart, the 3D image being an image file in a Digital Imaging and Communications in Medicine ("DICOM") format or other format. The derived EM data may be an echocardiogram. The received image data may also include a patient substrate location of the substrate of the patient's heart. After identifying the patient source location, the AISL system adds the patient source location to the image data. For example, the patient source location may be added to the DICOM file as metadata or superimposed on the 3D image (e.g., an indication added to the 3D image). The AILS system may then provide the enhanced DICOM file to a planning system to provide a basis for treatment of the patient (e.g., an ablation procedure). Because the enhanced DICOM file includes the source location, a more informed treatment plan may be developed.
[0231] In some embodiments, identifying the patient source location may be based on a mapping of the exported EM output to the source location. For example, the mapping may be generated by running a simulation for a variety of non-patient-specific configuration parameters or patient-specific configuration parameters. The running of the simulation may be performed by an MLMO system as described above. The exported EM data of each simulation is mapped to the source location used in the simulation. The AISL system may identify the patient source location by comparing the patient's exported EM data with the simulated exported EM data to identify a match and selecting the source location that matches the exported EM data as the patient source location. The AISL system may also identify the patient source location using a classifier trained by the MLMO system using the mapping as training data. In addition, before identifying the patient source location, the AISL system may calibrate the mapping using a CSC system as described above.
[0232] Although the AISL system is primarily described in the context of an organ that is the heart and a patient that is a human, the AISL system can be used for other organs (e.g., brain and liver) and non-humans (e.g., horses and primates). When the organ is the heart, the condition can be, for example, an arrhythmia, tachycardia, or fibrillation, and the source location can be rotor or seismic.
[0233] Fig.67 6680 generates enhanced image data. In box 6681, the component outputs image data, such as a 3D image of the patient's heart in DICOM format together with the patient's substrate position (such as scar position). The substrate position may have been provided by an electrophysiologist by manually superimposing the indication of the substrate position on the 3D image or adding the substrate position as metadata of the image data. The combination of the 3D image and the patient's substrate position may be referred to as a substrate map. In box 6682, the component may use a CSC system, for example, to generate a library of calibration maps based on a mapping associated with the substrate position and a heart geometry similar to the patient's heart geometry. In box 6683, the component identifies the ECG of the calibration map that matches the patient's ECG. In box 6684, the component adds the source position of the matching ECG as the patient's source position to the image data to generate enhanced image data. In box 6685, the component outputs the enhanced image data, and then completes.
[0234] Fig.6868 is a block diagram showing the components of the AISL system in some embodiments. The AISL system 6800 receives image data of a patient, the image data including a 3D image of the patient's heart, the patient's substrate position, and optionally the patient's ECG. The AISL component also receives a non-sinus rhythm ECG of the patient. The non-sinus rhythm ECG can be collected during arrhythmia caused clinically (e.g., caused by pacing), for example. The AISL system outputs enhanced image data, the enhanced image data including the 3D image, the patient's substrate position, and the identified patient source position. The AISL system includes an enhanced image data component 6801, a generate calibration library component 6802, an identification matching ECG component 6803, and an add source position component 6804. The AISL system also includes a patient-specific library 6805, a clinical library 6806, a model library 6807, and a calibration library 6808. The enhanced image data component controls the overall processing of the AISL system. The generate calibration library component generates a library of mappings calibrated for the patient. The identification matching component identifies the ECG of the calibration library that matches the patient's non-sinus rhythm ECG. The add source position component adds the patient source position to the image data to generate enhanced image data. The patient-specific library includes a mapping of ECGs generated from simulations based on configuration parameters derived from the patient. The clinical library includes a mapping of ECG positions derived from ECGs collected from actual patients. The model library contains a mapping of ECGs generated from simulations based on non-patient-specific configuration parameters. The calibration library includes a mapping of ECGs calibrated for the patient, the calibration being performed by, for example, making the ECG similar to the ECG provided by the image data and having generated the ECG using configuration parameters similar to the patient's configuration parameters. Each library maps its ECG to a source position associated with the ECG.
[0235] Fig.69 6900 is a flow chart of the Generate Calibration Library component of the AISL system in some embodiments. Generate Calibration Library 6900 is called to generate a calibration library in view of image data received by the AISL system. In box 6901, the component retrieves the patient substrate position from the image data. In box 6902, the component retrieves the ECG from the image data. In box 6903, the component analyzes the image to identify the geometry of the patient's heart. In box 6904, the component identifies an ECG associated with a heart geometry that is similar to the ECG of the patient's heart geometry. In box 6905, the component identifies an ECG similar to the ECG of the image data. In box 686, the component adds a mapping of the identified ECG to the source position to the calibration library. The component is then completed.
[0236] Fig.707001 , the component retrieves the non-sinus rhythm ECG. In box 7002, the component identifies the ECG in the calibration library that matches the patient's non-sinus rhythm ECG. The component then completes.
[0237] Fig.71 7100 is a flow chart illustrating the processing of the add source location component in some embodiments. The add source location component 7100 is called to add the source location to the image data to create enhanced image data. In box 7101, the component retrieves the source location of the matching ECG. In box 7102, the component adds the source location to the image data and then completes. In some embodiments, the identify matching ECG component may identify multiple matching ECGs. In this case, the component may generate a patient source location based on the source locations of multiple matching ECGs. For example, the component may take the average of the source location. As another example, the component may apply a clustering technique to the source location and take the average of the largest cluster. The component may also add multiple patient source locations to the image data, for example, adding one for each of the two largest clusters.
[0238] In some embodiments, the AISL system receives a 4D CT image of the patient captured while collecting the ECG of the image data. The image capture system may capture multiple 3D CT images during each cardiac cycle. For example, the 3D CT image may be captured at a capture interval of 25 milliseconds. The 4D CT image is a collection of 3D CT images. If the cycle is 1 second and 30 cycles of 3D CT images are captured, then the 4D CT image will be a collection of 1200 3D CT images (i.e., 30*1000 / 25). The image capture system may start capturing 3D CT images at the same starting point in each cycle (such as at the end of diastole as indicated by the P wave on the ECG). The image capture system may analyze the patient's ECG in real time to identify the starting point of each cycle. Since the cycle will not likely be an exact multiple capture interval, the image capture system may determine that the current cycle is close to the starting point of the next cycle (e.g., within 10 milliseconds) and capture the next 3D CT image at the starting point of the next cycle. The image capture system may adopt a variable rather than a fixed capture interval. The image capture system may have a variable capture interval per cycle. For example, if 40 3DCT images are to be captured per cycle and the last cycle is 0.8 seconds, the image capture system may set the capture interval to 20 milliseconds. However, if the last cycle (or the average of the last 10 cycles) is 1.2 seconds, the image capture system may set the capture interval to 30 milliseconds.
[0239] In some embodiments, the image data input to the AISL system may include a 4D CT image of one cycle in DICOM format and may include the patient's patient substrate position. An electrophysiologist may examine the 4D CT image to identify the patient substrate position. The patient substrate position may be added to the image data by the electrophysiologist by superimposing a marker of the patient substrate position on the 4D CT image or adding the coordinates of the patient substrate position as metadata to the image data. An ECG collected during the capture of the 4D CT image may be added to the image data as metadata.
[0240] In some embodiments, the patient substrate position may be defined by a set of positions each specified by a 3D coordinate in the reference system of the 3D CT image. Each dimension (e.g., x, y, z) of the 3D coordinate may have a radius to indicate the size of the substrate. The AISL system may use a set of positions to add an indication of the substrate position and substrate volume (which is defined by the radius) to the 3D CT image for each position. The indication of the substrate position and substrate volume is called a substrate mask. In order to add a substrate mask, the AISL system may identify the voxels in each volume and set the intensity of each identified voxel to the substrate intensity as an indication that the voxel is part of the substrate. The substrate intensity may be specified as a Hounsfield unit (HU), which is a unit of attenuation of x-rays when they pass through an object (e.g., bone). The substrate intensity may be an intensity that is uncommon in a 3D CT image of an organ (e.g., 9000HU). The AISL system may also add a source mask corresponding to the patient source position to the 3D CT image in a similar manner. The patient source position can be specified by a set of source positions and source position volumes derived from the mapping of the ECG to the source positions. Each ECG can be mapped to one or more source positions and source volumes. Therefore, the enhanced 3D CT image can include a substrate mask and / or a source mask. The substrate mask and the source mask can have different intensity values to distinguish them. Then, this enhanced 3D CT image can be displayed to help provide a basis for the treatment of the patient.
[0241] In some embodiments, the AISL system can more accurately locate the source location on the 3D CT image by aligning the geometry of the simulated organ associated with the matching derived EM data with the geometry of the patient's patient organ identified by the 3D CT image. To align the geometry, the AISL system can generate a transformation matrix that maps the marked locations (or more generally, marked volumes) within the simulated organ to the corresponding marked locations within the patient organ. The AISL system can then use the transformation matrix to transform the patient source location into the source location on the 3D CT image and add the transformed patient source location to the 3D CT image. The transformation can involve rotation, translation, scaling, and shearing of the 3D CT image.
[0242] When the organ is a heart, the landmark locations may be, for example, the heart / valve plane, the middle of the septal wall, the LV / RV apex, the aortic valve, and / or the mitral valve. The AISL system may employ automated techniques to identify landmark locations within a patient's organ. For example, the AISL system may employ a convolutional neural network trained using training data including 3D images labeled with landmark locations (e.g., the aortic valve). Alternatively or additionally, the AISL system may receive an indication of a landmark location on a patient's organ from a person. The landmark location acts as a "fiducial marker."
[0243] In some embodiments, the AISL system may receive a patient 4D CT image representing a 3D CT image collected over multiple cycles. The AISL system may use the patient 4D CT image when calibrating the mapping. For example, the AISL system may compare the movement of the patient's heart represented by the patient 4D CT image with the movement of the heart grid during simulation. In this case, the patient-specific library and the model library may include a grid sequence of ECGs from which the library was generated. In addition, the clinical library may include clinical 4D CT images captured when collecting ECGs from clinical patients. The AISL system may base the calibration on a comparison of the patient 4D CT image with the clinical 4D CT image.
[0244] Fig.72 7200 receives the patient's ECG, signals to capture 3D CT images, and stores each 3D CT image as part of a 4D CT image. In box 7201, the component initializes the capture time and cycle count. In box 7202, the component analyzes the ECG as it is collected to identify the starting point of the capture of the next cycle. In box 7203, the component guides the capture of the 3D CT image. In box 7204, the component stores the 3D CT image in a 4D CT data structure indexed by time and cycle count. In box 7205, the component increments the capture interval by time. In decision box 7206, if the capture of the 4D CT image is to be terminated, the component is completed, otherwise the component continues at box 7207. In box 7207, the component waits for the next capture point, which is at the expiration of the capture interval or at the starting point of the next cycle. In decision block 7208, if the capture point is the start of the next cycle, the component continues at block 7209, otherwise the component loops to block 7203 to capture the next 3D CT image. In block 7209, the component resets the time to zero and increments the cycle count, then loops to block 7203 to capture the next 3D CT image.
[0245] The following paragraphs describe various embodiments of various aspects of the MLMO system and other systems. The implementation of the system may adopt any combination of these embodiments. The processing described below may be performed by a computing system having a processor that executes computer-executable instructions stored on a computer-readable storage medium that implements the system.
[0246] In some embodiments, a method for generating a classifier performed by one or more computing systems is provided, the classifier being used to classify electromagnetic data obtained from an in vivo electromagnetic source. The method accesses a computational model of the electromagnetic source, wherein the computational model is used to model the electromagnetic output of the electromagnetic source over time based on the source configuration of the electromagnetic source. For each of a plurality of source configurations, the method uses the computational model to generate a modeled electromagnetic output of the electromagnetic source of that source configuration. For each modeled electromagnetic output, the method derives electromagnetic data of the modeled electromagnetic output and generates a label for the derived electromagnetic data based on the source configuration of the modeled electromagnetic data. The method trains a classifier with the derived electromagnetic data and then labels it as training data. In some embodiments, the modeled electromagnetic output of the source configuration includes an electromagnetic grid for each of a plurality of time intervals, the electromagnetic grid having a modeled electromagnetic value for each of a plurality of positions of the electromagnetic source. In some embodiments, for a time interval, the derived electromagnetic data is an equivalent source representation of the electromagnetic output. In some embodiments, the equivalent source representation is generated using principal component analysis. In some embodiments, the method further identifies cycles within the derived electromagnetic data of the modeled electromagnetic output. In some embodiments, the same label is generated for each cycle. In some embodiments, the method further identifies a sequence of similar cycles, wherein the same label is generated for each sequence. In some embodiments, the electromagnetic data derived from the modeled electromagnetic output includes normalizing the modeled electromagnetic output on a per-cycle basis. In some embodiments, the classifier is a convolutional neural network. In some embodiments, the convolutional neural network inputs a one-dimensional image. In some embodiments, the classifier is a recursive neural network, an autoencoder, a restricted Boltzmann machine, or other types of neural networks. In some embodiments, the classifier is a support vector machine. In some embodiments, the classifier is a Bayesian classifier. In some embodiments, the electromagnetic source is a heart, the source configuration represents the source location and other characteristics of a cardiac disease, the modeled electromagnetic output represents the activation of the heart, and the electromagnetic data is based on surface measurements, such as an electrocardiogram.In some embodiments, the cardiac condition is selected from the group consisting of inappropriate sinus tachycardia ("IST"), atrial ectopic rhythm, junctional rhythm, ventricular escape rhythm, atrial fibrillation ("AF"), ventricular fibrillation ("VF"), focal atrial tachycardia ("focal AT"), atrial microreentry, ventricular tachycardia ("VT"), atrial flutter ("AFL"), premature ventricular contractions ("PVC"), premature atrial contractions ("PAC"), atrioventricular nodal reentrant tachycardia ("AVNRT"), atrioventricular reentrant tachycardia ("AVRT"), permanent junctional reciprocating tachycardia ("PJRT"), and junctional tachycardia ("JT").
[0247] In some embodiments, a method for classifying electromagnetic output collected from a target is provided, the target being an in-vivo electromagnetic source, and is performed by a computing system. The method accesses a classifier to generate a classification of the electromagnetic output of the electromagnetic source. The classifier is trained using training data generated from modeled electromagnetic outputs of a plurality of source configurations of the electromagnetic source. The modeled electromagnetic output is generated using a computational model of the electromagnetic source, the computational model modeling the electromagnetic output of the electromagnetic source over time based on the source configuration. The method collects target electromagnetic outputs from the target. The method applies the classifier to the target electromagnetic output to generate a classification of the target. In some embodiments, the training data is generated by running a simulation for each of the source configurations, the simulation generating an electromagnetic grid for each of a plurality of simulation intervals, each electromagnetic grid having electromagnetic values for a plurality of locations of the electromagnetic source. In some embodiments, the electromagnetic source is a heart, the source configuration represents a source location of a cardiac condition, and the modeled electromagnetic output represents activation of the heart, and the classifier is trained using electromagnetic data derived from an electrocardiogram representation of the electromagnetic output.
[0248] In some embodiments, one or more computing systems for generating a classifier are provided, the classifier being used to classify the electromagnetic output of an electromagnetic source. The one or more computing systems include one or more computer-readable storage media and one or more processors, the one or more processors being used to execute computer-executable instructions stored in the one or more computer-readable storage media. The one or more computer-readable storage media store a computational model of the electromagnetic source. The computational model models the electromagnetic output of the electromagnetic source over time based on the source configuration of the electromagnetic source. The one or more computer-readable storage media store computer-executable instructions, the computer-executable instructions being used to control the one or more computing systems to: generate training data from the electromagnetic output of the computational model based on the source configuration for each of a plurality of source configurations, and train the classifier using the training data. In some embodiments, the computer-executable instructions for generating the training data for a source configuration further control the one or more computing systems to: generate derived electromagnetic data from the electromagnetic output of the source configuration, and generate a label for the electromagnetic data based on the source configuration.
[0249] In some embodiments, a method for generating a simulated anatomical structure of an in vivo electromagnetic source performed by a computing system is provided. The method accesses seed anatomical structures of the electromagnetic source. Each seed anatomical structure has a seed value for each of a plurality of anatomical parameters of the electromagnetic source. The method accesses a weight set, the weight set including a weight for each seed anatomical structure. For each of the anatomical parameters, the method generates a simulated value for that anatomical parameter by combining the seed value for that anatomical parameter, thereby taking the weight of the seed anatomical structure into account. In some embodiments, the method verifies the simulated anatomical structure based on a comparison with the value of the anatomical parameter found in a population. In some embodiments, the anatomical parameter includes the size of the electromagnetic source, and wherein the simulated value is verified when the value of the size included by the patient in the population is approximately the same as the simulated value of the size. In some embodiments, the anatomical parameters of the seed anatomical structure are collected by scanning an actual in vivo electromagnetic source. In some embodiments, the electromagnetic source is a heart. In some embodiments, the method generates a simulated value for that anatomical parameter based on a weighted average of the seed values of the anatomical parameter. In some embodiments, the method further generates a plurality of simulated anatomical structures, wherein each simulated anatomical structure is based on a different set of weights.
[0250] In some embodiments, a computing system for generating a simulated anatomical structure of a heart is provided. The computing system includes one or more computer-readable storage media, the one or more computer-readable storage media storing seed anatomical structures of the heart, each seed anatomical structure having a seed value for each of a plurality of anatomical parameters of the heart. The one or more computer-readable storage media also store weight sets, each of which includes a weight for each seed anatomical structure. The one or more computer-readable storage media store computer-executable instructions for controlling the computing system to: for each weight set and for each of the anatomical parameters of that weight set, generate a simulated value for that anatomical parameter by combining the seed value of that anatomical parameter, thereby taking the weight of the seed anatomical structure into account. The computing system also includes one or more processors, the one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage media. In some embodiments, the instructions further control the computing system to verify each simulated anatomical structure based on a comparison with the value of the anatomical parameter found in a population. In some embodiments, the anatomical parameters include the wall thickness of the heart and the size of the chamber of the heart. In some embodiments, the simulated value is verified when the value of the dimension included by the patients in the group is approximately the same as the simulated value of the dimension. In some embodiments, the seed anatomical structure represents an extreme case of the heart found in the group. In some embodiments, the anatomical parameters of the seed anatomical structure are collected by scanning the heart. In some embodiments, the simulated value of the generated anatomical parameter is a weighted average of the seed value of that anatomical parameter. In some embodiments, a method for generating an arrhythmia model library performed by a computing system is provided, and the arrhythmia model library is used to model the heart. The method accesses the simulated anatomical structure of the anatomical parameters of the heart. The simulated anatomical structure is generated based on the seed anatomical structure of the anatomical parameters of the heart and a weight set including the weight of each seed anatomical structure.The method accesses configuration parameters, the configuration parameters including one or more of: torso anatomy, normal and abnormal cardiac anatomy, normal and abnormal cardiac tissue, scar, fibrosis, inflammation, edema, accessory pathways, congenital heart disease, malignancy, previous ablation sites, previous surgical sites, external radiation therapy sites, pacing leads, implantable cardioverter defibrillator leads, cardiac resynchronization therapy leads, pacemaker pulse generator locations, implantable cardioverter defibrillator pulse generator locations, subcutaneous defibrillator lead locations, subcutaneous defibrillator pulse generator locations, leadless pacemaker locations, other implanted hardware (e.g., right or left ventricular assist devices), external defibrillation electrodes, surface ECG leads, surface mapping leads, mapping vests, and other normal and pathological physiological characteristic distributions within the heart, action potential dynamics of the heart, conductivity sets of the heart, arrhythmia source locations within the heart, etc. The method establishes source configurations, each of which is based on a combination of simulated anatomy and electrophysiological parameters. For each of a plurality of source configurations, the method generates a mesh based on the simulated anatomical structure of that source configuration, the mesh having vertices, and for each vertex of the mesh, the method generates model parameters of a computational model of the heart based on a combination of electrophysiological parameters of that source configuration. The computational model for modeling electromagnetic propagation at that vertex is based on the electrophysiological parameters of that source configuration. In some embodiments, the method generates a simulated anatomical structure by accessing seed anatomical structures of the heart, wherein each seed anatomical structure has a seed value for each of the anatomical parameters of the heart; accessing a weight set including a weight for each seed anatomical structure; and for each of the anatomical parameters, generating a simulated value of that anatomical parameter by combining the seed value of that anatomical parameter, thereby taking into account the weight of the seed anatomical structure. In some embodiments, the simulated anatomical structure is validated based on a comparison with the value of the anatomical parameter found in a population. In some embodiments, the anatomical parameters of the seed anatomical structure are collected by scanning an actual heart. In some embodiments, for each of a plurality of source configurations, the method uses the computational model of the heart to generate a modeled electromagnetic output of the heart of that source configuration. In some embodiments, for each source configuration, the method generates training data for the modeled electromagnetic output based on that source configuration, and uses the training data to train a classifier for classifying the electromagnetic output of the heart.
[0251] In some embodiments, a computing system for generating a model library of models of in vivo electromagnetic sources is provided. The computing system includes: one or more computer-readable storage media, the one or more computer-readable storage media storing computer-executable instructions; and one or more processors, the one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage media. The instructions control the computing system to: generate a simulated anatomical structure of the anatomical parameters of the electromagnetic source from a seed anatomical structure, and generate a source configuration, each of which is based on a combination of the simulated anatomical structure and the configuration parameters. The instructions control the computing system to: for each of a plurality of source configurations, generate a mesh based on the simulated anatomical structure of that source configuration, the mesh having vertices; and for each vertex of the mesh, generate model parameters of the computational model of the electromagnetic source based on a combination of the configuration parameters of that source configuration. In some embodiments, the computational model for modeling electromagnetic propagation at the vertex is based on the configuration parameters of the source configuration. In some embodiments, the simulated anatomical structure is generated based on the anatomical parameters of the seed anatomical structure and a weight set including a weight of each seed anatomical structure. In some embodiments, the electromagnetic source is a heart, and the configuration parameters include one or more of: trunk anatomy, normal and abnormal cardiac anatomy, normal and abnormal cardiac tissue, scar, fibrosis, inflammation, edema, bypass, congenital heart disease, malignancy, previous ablation site, previous surgery site, external radiation therapy site, pacing lead, implantable cardioverter defibrillator lead, cardiac resynchronization therapy lead, pacemaker pulse generator location, implantable cardioverter defibrillator pulse generator location, subcutaneous defibrillator lead location, subcutaneous defibrillator pulse generator location, leadless pacemaker location, other implanted hardware (e.g., right or left ventricular assist device), external defibrillation electrodes, surface ECG leads, surface mapping leads, mapping vest, and other normal and pathophysiological characteristics distribution within the heart, action potential dynamics of the heart, conductivity set of the heart, and arrhythmia source location within the heart. In some embodiments, the simulated anatomical structure is validated based on comparison with the values of anatomical parameters found in the population. In some embodiments, the anatomical parameters of the seed anatomical structure are collected by scanning an actual electromagnetic source. In some embodiments, the computer executable instructions further control the computing system to: for each of a plurality of source configurations, use the computational model of the electromagnetic source to generate a modeled electromagnetic output of the electromagnetic source for that source configuration. In some embodiments, the computer executable instructions further control the computing system to: for each source configuration, generate training data for the modeled electromagnetic output based on that source configuration, and use the training data to train a classifier for classifying the electromagnetic output of the electromagnetic source.
[0252] In some embodiments, a method for generating a model library of a model of an in vivo electromagnetic source performed by a computing system is provided. The method accesses a simulated anatomical structure of an anatomical parameter of the electromagnetic source. The method generates a source configuration, each of which is based on a simulated anatomical structure and a combination of configuration parameters. For each of a plurality of source configurations, the method generates a model based on the simulated anatomical structure of that source configuration, a combination of configuration parameters of that source configuration, and a computational model of the electromagnetic source. In some embodiments, the generating model includes: generating a mesh based on the simulated anatomical structure of that source configuration, and for each vertex of the mesh, generating model parameters of the computational model of the electromagnetic source based on a combination of configuration parameters of that source configuration. In some embodiments, the computational model is used to model electromagnetic propagation at a vertex based on the configuration parameters of the source configuration. In some embodiments, the electromagnetic source is a heart, and the model is an arrhythmia model. In some embodiments, the method further generates the simulated anatomical structure based on the anatomical parameters of the seed anatomical structure and a weight set including a weight of each seed anatomical structure. In some embodiments, the electromagnetic source is a heart, and the configuration parameters include one or more of: a human torso, normal and abnormal cardiac anatomy, normal and abnormal cardiac tissue, scars, fibrosis, inflammation, edema, bypass, congenital heart disease, malignancy, previous ablation sites, previous surgical sites, external radiation therapy sites, pacing leads, implantable cardioverter-defibrillator leads, cardiac resynchronization therapy leads, pacemaker pulse generator locations, implantable cardioverter-defibrillator pulse generator locations, subcutaneous defibrillator lead locations, subcutaneous defibrillator pulse generator locations, leadless pacemaker locations, other implanted hardware (e.g., right or left ventricular assist devices), external defibrillation electrodes, surface ECG leads, surface mapping leads, mapping vests, and distribution of other normal and pathological physiological characteristics within the heart, action potential dynamics of the heart, conductivity sets of the heart, and locations of arrhythmia sources within the heart, etc.
[0253] In some embodiments, a method for presenting weights of simulated anatomical structures of a body part performed by a computing system is provided. The method accesses seed anatomical structures of the body part. Each seed anatomical structure has a seed value for each of a plurality of anatomical parameters of the body part. For each of a plurality of seed anatomical structures of the body part, the method displays a seed representation of the body part based on the seed value of the anatomical parameter of that seed anatomical structure. The method accesses a weight set, the weight set including the weight of each seed anatomical structure. The method displays a simulated representation of the body part based on the simulated value of each anatomical parameter by combining, for each anatomical parameter, the seed value of the seed anatomical structure of that anatomical parameter, thereby taking the weight of the seed anatomical structure into account. In some embodiments, the seed representation is displayed in a circular arrangement, wherein the simulated representation is displayed within the circular arrangement. In some embodiments, the method further displays an indication of the weight associated with each displayed seed representation of a seed anatomical structure in association with that seed anatomical structure. In some embodiments, the method further displays a line between each displayed seed representation and the displayed simulated representation, wherein the displayed indication of the weight of the seed anatomical structure is displayed in association with the displayed line between the displayed seed representation and the displayed simulated representation of that seed anatomical structure. In some embodiments, the method further provides a user interface element for specifying the weight of each seed anatomical structure. In some embodiments, the method provides a user interface element for specifying a plurality of weight sets, wherein each set includes the weight of each seed anatomical structure. In some embodiments, the plurality of weight sets are specified by providing a range and increment of weights. In some embodiments, the body part is a heart. In some embodiments, the body part is a lung. In some embodiments, the body part is a trunk surface.
[0254] In some embodiments, a computing system for presenting a simulated anatomical structure of a body part is provided. The computing system includes: one or more computer-readable storage media storing computer-executable instructions; and one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage media. The instructions control the computing system to: for each of a plurality of seed anatomical structures of the body part, display a seed representation of the body part based on a seed value of an anatomical parameter of that seed anatomical structure; and display a simulated representation of the body part based on a simulated anatomical structure having a simulated value of the anatomical parameter derived from a weighted combination of the seed values of the seed anatomical structures of the anatomical parameters. In some embodiments, the computer-executable instructions further control the computing system to generate the simulated anatomical structure by: for each anatomical parameter, generating a simulated value of that anatomical parameter by combining the seed values of the seed anatomical structures of that anatomical parameter, thereby taking into account the weight of the seed anatomical structure. In some embodiments, the seed representation is displayed in a circular arrangement, wherein the simulated representation is displayed within the circular arrangement. In some embodiments, the computer executable instructions further control the computing system to display, in association with each displayed seed representation of a seed anatomical structure, an indication of a weight associated with that seed anatomical structure. In some embodiments, the computer executable instructions further control the computing system to provide a user interface for specifying a plurality of weight sets, wherein each set includes a weight for each seed anatomical structure. In some embodiments, the plurality of weight sets are specified by ranges and increments of weights.
[0255] In some embodiments, a method for presenting a simulated anatomical structure of a human heart performed by a computing system is provided. For each of a plurality of seed anatomical structures of the heart, the method displays a seed representation of the heart based on a seed value of an anatomical parameter of that seed anatomical structure. The method displays a simulated representation of the heart based on a simulated anatomical structure derived from a simulated value of that anatomical parameter for each anatomical parameter, the simulated value of that anatomical parameter being generated from a weighted combination of the seed values of the seed anatomical structure of that anatomical parameter. In some embodiments, the seed representation is displayed in a circular arrangement, wherein the simulated representation is displayed within the circular (e.g., bull's-eye) arrangement. In some embodiments, the method further displays an indication of a weight associated with each displayed seed representation of a seed anatomical structure in association with that seed anatomical structure.
[0256] In some embodiments, a method for converting a first polyhedral model into a second polyhedral model is provided, which is performed by a computing system. The first polyhedral model has a first polyhedral mesh having a volume containing the first polyhedron. Each vertex of the first polyhedron has model parameters. The method generates a representation of the surface of the first polyhedral model from the first polyhedron. The method generates a second polyhedral mesh of the second polyhedral model by filling the volume with a surface of the second polyhedron that is different from the first polyhedron. For each of the multiple vertices of the second polyhedron of the second polyhedron mesh, the method interpolates the model parameters of that vertex based on the parameters of the vertices of the first polyhedron adjacent to that vertex. In some embodiments, the first polyhedron is a hexahedron, and the second polyhedron is a tetrahedron. In some embodiments, the polyhedral mesh represents a body part. In some embodiments, the body part is a heart. In some embodiments, the first polyhedral mesh has an origin, and the method also includes mapping the second polyhedral mesh to the same origin before interpolating the model parameters. In some embodiments, each vertex of the first polyhedron has multiple parameters and each model parameter is interpolated. In some embodiments, the first polyhedron model and the second polyhedron model represent a computational model of an in vivo electromagnetic source, and the method further uses a problem solver suitable for operating on a mesh with a second polyhedron to generate a modeled electromagnetic output of the electromagnetic source based on the second polyhedron model. In some embodiments, the electromagnetic source is a heart, and the method further generates a vector cardiogram from the modeled electromagnetic output. In some embodiments, the first polyhedron model and the second polyhedron model are geometric models (e.g., geometric models of heart or trunk anatomical structures). In some embodiments, the first polyhedron model and the second polyhedron model represent models of in vivo electromagnetic sources, and the method further converts multiple first polyhedron models representing different source configurations.In some embodiments, the electromagnetic source is a heart, and the source configuration specifies one or more of: fibrous structure, trunk anatomy, normal and abnormal cardiac anatomy, normal and abnormal cardiac tissue, scar, fibrosis, inflammation, edema, bypass, congenital heart disease, malignancy, previous ablation site, previous surgical site, external radiation therapy site, pacing lead, implantable cardioverter defibrillator lead, cardiac resynchronization therapy lead, pacemaker pulse generator location, implantable cardioverter defibrillator pulse generator location, subcutaneous defibrillator lead location, subcutaneous defibrillator pulse generator location, leadless pacemaker location, other implanted hardware (e.g., right or left ventricular assist device), external defibrillation electrodes, surface ECG leads, surface mapping leads, mapping vest, and other normal and pathological physiological characteristics distribution within the heart, action potential dynamics, conductivity, one or more arrhythmia source locations, etc.
[0257] In some embodiments, a computing system for converting a first polyhedral model of a body part into a second polyhedral model of the body part is provided. The computing system includes: one or more computer-readable storage media, the one or more computer-readable storage media storing computer-executable instructions; and one or more processors, the one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage media. The instructions are used to control the computing system to generate a representation of a surface of the first polyhedral model. The first polyhedral model has a first polyhedral mesh based on a first polyhedron. The instructions control the computing system to generate a second polyhedral mesh of the second polyhedral model by filling the volume with a surface based on a second polyhedron different from the first polyhedron. The instructions control the computing system to interpolate model parameters of each of a plurality of vertices of the second polyhedron of the second polyhedron mesh based on the parameters of vertices of the first polyhedron of the first polyhedron mesh adjacent to that vertex. In some embodiments, the first polyhedron is a hexahedron and the second polyhedron is a tetrahedron. In some embodiments, the first polyhedral mesh has an origin, wherein the computer executable instructions further control the computing system to map the second polyhedral mesh to the same origin before interpolating the model parameters. In some embodiments, the first polyhedral model and the second polyhedral model represent a computational model of the heart, and the computer executable instructions further control the computing system to generate a modeled electromagnetic output of the electromagnetic heart based on the second polyhedral model using a problem solver suitable for operating on a mesh having a second polyhedron. In some embodiments, the computer executable instructions further control the computing system to generate a vector cardiogram from the modeled electromagnetic output. In some embodiments, the first polyhedral model and the second polyhedral model are geometric models (e.g., geometric models of heart or torso anatomical structures).
[0258] In some embodiments, a method for converting a first polyhedral model into a second model is provided, which is performed by a computing system. The first polyhedral model has a first polyhedral mesh having a volume containing the first polyhedron. Each vertex of the first polyhedron has model parameters. The method generates a representation of the surface of the first polyhedral model from the first polyhedron. For each of a plurality of points of the second model, the method interpolates the model parameters of that vertex based on the parameters of the vertices of the first polyhedron that are considered to be adjacent to that vertex. In some embodiments, the second model is a second polyhedral model, and the points are vertices of the second polyhedral model. In some embodiments, the second model is represented by regularly spaced grid points, and these points are grid points.
[0259] In some embodiments, a method for generating derived electromagnetic data of an in vivo electromagnetic source performed by a computing device is provided. The method accesses a modeled electromagnetic output over time of a first model of the electromagnetic source. The first model is based on a first source configuration that specifies a first anatomical structure. The modeled electromagnetic output is generated using a computational model of the electromagnetic source. The computational model is used to generate a modeled electromagnetic output over time of the electromagnetic source based on a model, the model being based on a source configuration. The method accesses a second model of the electromagnetic source, the second model being based on a second source configuration that specifies a second anatomical structure. The method generates derived electromagnetic data of the second model of the electromagnetic source based on the modeled electromagnetic output of the first model of the electromagnetic source, thereby taking into account differences between the first anatomical structure and the second anatomical structure. In some embodiments, the electromagnetic source is a heart, and the derived electromagnetic data is a cardiogram. In some embodiments, the cardiogram is a vector cardiogram. In some embodiments, the cardiogram is an electrocardiogram. In some embodiments, the modeled electromagnetic output of the model includes an electromagnetic grid for each of a plurality of time intervals, the electromagnetic grid having a modeled electromagnetic value for each of a plurality of vertices of the electromagnetic grid. In some embodiments, the modeled electromagnetic output is a collection of voltage solutions.
[0260] In some embodiments, a computing system for generating a cardiogram of a heart is provided. The computing system includes one or more computer-readable storage media storing modeled electromagnetic outputs of a first arrhythmia model of the heart over time. The first arrhythmia model is based on a first anatomical structure. The modeled electromagnetic outputs are generated using a computational model of the heart, the computational model being used to generate the modeled electromagnetic outputs of the heart over time based on the arrhythmia model. The one or more computer-readable storage media store a second arrhythmia model based on a second anatomical structure. The one or more computer-readable storage media store computer-executable instructions for controlling the computing system to generate a cardiogram of the second arrhythmia model based on the modeled electromagnetic outputs of the first arrhythmia model, thereby taking into account differences between the first anatomical structure and the second anatomical structure. The computing system includes one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage media. In some embodiments, the cardiogram is a vectorcardiogram. In some embodiments, the cardiogram is an electrocardiogram. In some embodiments, the modeled electromagnetic output of the arrhythmia model comprises, for each of a plurality of time intervals, an electromagnetic mesh having modeled electromagnetic values for each of a plurality of vertices of the electromagnetic mesh. In some embodiments, the modeled electromagnetic output is a set of voltage solutions.
[0261] In some embodiments, a method for bootstrapping a modeled electromagnetic output of an in vivo electromagnetic source performed by a computing system is provided. The method accesses a first modeled electromagnetic output of a first model having a first source configuration of the electromagnetic source for a first simulation interval. The first modeled electromagnetic output is generated using a computational model of the electromagnetic source. The method initializes a second modeled electromagnetic output of a second model having a second source configuration of the electromagnetic source to the first modeled electromagnetic output of one of the first simulation intervals. For each of a plurality of second simulation intervals, the method generates a second modeled electromagnetic output of the second model of the electromagnetic source based on the initialized second modeled electromagnetic output using the computational model. In some embodiments, the electromagnetic source is a heart, and the second source configuration is different from the first source configuration based on scar or fibrosis or arrhythmogenic substrate location within the heart. In some embodiments, the electromagnetic source is a heart, and the first model and the second model are arrhythmia models. In some embodiments, the modeled electromagnetic output of the model includes an electromagnetic grid for each of a plurality of time intervals, the electromagnetic grid having a modeled electromagnetic value for each of a plurality of vertices of the electromagnetic grid. In some embodiments, the method further generates the first modeled electromagnetic output of the first model for each of a plurality of the first simulation intervals using the computational model. In some embodiments, the method initializes the second modeled electromagnetic output to the first modeled electromagnetic output of the first simulation interval after the first modeled electromagnetic output of the first simulation interval has stabilized. In some embodiments, the electromagnetic source is a heart and the first modeled electromagnetic output has stabilized to a rhythm.
[0262] In some embodiments, a computing system for bootstrapping modeled electromagnetic outputs of a heart is provided. The computing system includes one or more computer-readable storage media storing a first modeled electromagnetic output of a first arrhythmia model of the heart for a first simulation interval, the first modeled electromagnetic output being generated using a computational model of the heart. The one or more computer-readable storage media also store computer-executable instructions for controlling the computing system to: initialize a second modeled electromagnetic output of a second arrhythmia model of the heart to the first modeled electromagnetic output of one of the first simulation intervals; and simulate a second modeled electromagnetic output of the second arrhythmia model based on the initialized second modeled electromagnetic output using the computational model. The computing system also includes one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage media. In some embodiments, the first arrhythmia model and the second arrhythmia model are based on different scar or fibrosis or arrhythmogenic substrate locations within the heart. In some embodiments, the modeled electromagnetic output of the arrhythmia model includes an electromagnetic grid for each of a plurality of time intervals, the electromagnetic grid having a modeled electromagnetic value for each of a plurality of vertices of the electromagnetic grid. In some embodiments, the computer executable instructions further control the computing system to generate the first modeled electromagnetic output of the first arrhythmia model using the computing model for each of a plurality of first simulation intervals. In some embodiments, the computer executable instructions control the computing system to initialize the second modeled electromagnetic output to the first modeled electromagnetic output of the first simulation interval after the first modeled electromagnetic output of the first simulation interval has stabilized. In some embodiments, the first modeled electromagnetic output has stabilized to a rhythm.
[0263] In some embodiments, a method for identifying derived electromagnetic data that matches electromagnetic data collected from a patient is provided, performed by a computing system. The electromagnetic data represents electromagnetic output from an in-vivo electromagnetic source. The method accesses, for each of a plurality of model source configurations of the electromagnetic source, a mapping of that model source configuration to electromagnetic data derived based on that model source configuration. The method accesses a patient source configuration, which represents a source configuration of the electromagnetic source in the patient's body. The method identifies a model source configuration that matches the patient source configuration. The method identifies derived electromagnetic data that matches the patient electromagnetic data from the derived electromagnetic data to which the identified model source configuration is mapped. In some embodiments, the derived electromagnetic data is derived from a modeled electromagnetic output generated based on a model source configuration using a computational model of the electromagnetic source. In some embodiments, the modeled electromagnetic data of the model source configuration includes an electromagnetic grid for each of a plurality of time intervals, the electromagnetic grid having a modeled electromagnetic value for each of a plurality of positions of the electromagnetic source. In some embodiments, the method further generates adjusted derived electromagnetic data based on the difference between the modeled electromagnetic output and the value of the anatomical parameter of the identified model source configuration and the patient source configuration when the value of the anatomical parameter does not match. In some embodiments, the source configuration includes configuration parameters, the configuration parameters including anatomical parameters and electrophysiological parameters. In some embodiments, the electromagnetic source is a heart, and the configuration parameters include scar or fibrosis or arrhythmogenic substrate location within the heart, the action potential of the heart, the conductivity of the heart, and the location of arrhythmias. In some embodiments, the electromagnetic source is a heart, and the anatomical parameters include the size of the chamber of the heart, the wall thickness of the heart, and the orientation of the heart. In some embodiments, the electromagnetic source is a heart, and the derived electromagnetic data is an electrocardiogram. In some embodiments, the model source configuration includes a disease parameter related to an attribute of the electromagnetic source, so that the derived electromagnetic data for the model source configuration is based on that attribute. In some embodiments, the electromagnetic source is a heart, and the attribute is for arrhythmias. In some embodiments, the identification of the derived electromagnetic data that matches the patient electromagnetic data is based on a Pearson correlation coefficient, a root mean square error, etc. In some embodiments, the identification of the derived electromagnetic data that matches the patient electromagnetic data is based on a root mean square error. In some embodiments, the source configuration includes configuration parameters, the model source configuration includes a value for each configuration parameter, and the patient source configuration includes values for only an appropriate subset of the configuration parameters. In some embodiments, the derived electromagnetic data is based on a model orientation of the electromagnetic source, and the method further includes: when the patient orientation of the electromagnetic source of the patient is different from the model orientation, the identification of the derived electromagnetic data takes into account the difference between the model orientation and the patient orientation.In some embodiments, the electromagnetic source is a heart and the electromagnetic data is a vector cardiogram, and identifying the derived electromagnetic data includes generating a rotation matrix based on a difference between the model orientation and the patient orientation, and rotating the vector cardiogram based on the rotation matrix.
[0264] In some embodiments, a method for generating a classification of a patient based on patient electromagnetic data performed by a computing system is provided, the patient electromagnetic data representing the electromagnetic output of an electromagnetic source in the patient. The method accesses a classifier of that cluster for each of a plurality of clusters (e.g., groups) of model source configurations of the electromagnetic source, the classifier being trained based on the model source configuration of that cluster to generate a classification of derived electromagnetic data. The method accesses a patient source configuration, the patient source configuration representing the source configuration of the electromagnetic source in the patient. The method identifies a cluster where the model source configuration matches the patient source configuration. The method applies the classifier for the identified cluster to the patient electromagnetic data to generate a classification of the patient. In some embodiments, the derived electromagnetic data is derived from a modeled electromagnetic output generated based on the model source configuration using a computational model of the electromagnetic source. In some embodiments, the source configuration includes configuration parameters, the configuration parameters including anatomical parameters and electrophysiological parameters. In some embodiments, the electromagnetic source is a heart, and the configuration parameters include scar or fibrosis or arrhythmogenic substrate location within the heart, the action potential of the heart, and the conductivity of the heart. In some embodiments, the electromagnetic source is a heart, and the anatomical parameters include the size of a chamber of the heart and the wall thickness of the heart. In some embodiments, the electromagnetic source is a heart, and the derived electromagnetic data is a cardiogram. In some embodiments, the cardiogram is a vector cardiogram. In some embodiments, the model source configuration matches the patient source configuration based on cosine similarity. In some embodiments, the method further generates a cluster of model source configurations; and for each of the clusters, for each of the model configuration sources in that cluster, generates a simulated electromagnetic output of the electromagnetic source based on that model configuration source; and generates derived electromagnetic data for that model source configuration from the simulated electromagnetic output based on that model source configuration. In some embodiments, the source configuration includes configuration parameters, the model source configuration includes a value for each configuration parameter, and the patient source configuration includes values for only an appropriate subset of the configuration parameters. In some embodiments, the derived electromagnetic data is based on a model orientation of the electromagnetic source, and the method further adjusts the patient electromagnetic data based on a difference between the model orientation and the patient orientation when the patient orientation of the electromagnetic source of the patient is different from the model orientation. In some embodiments, the electromagnetic source is a heart and the classification is based on the source location of the arrhythmia.
[0265] In some embodiments, a computing system for identifying a model cardiogram that matches a patient cardiogram collected from a patient is provided. The computing system includes one or more computer-readable storage media, the one or more computer-readable storage media: storing a modeled cardiogram of that model source configuration for each of a plurality of model source configurations of a heart; storing a patient source configuration representing the patient's heart; and storing computer-executable instructions, which when executed control the computing system to: identify a model source configuration that matches the patient source configuration; and identify those modeled cardiograms that match the patient cardiogram from the modeled cardiogram for the identified model source configuration. The computing system includes one or more processors, the one or more processors being used to execute the computer-executable instructions stored in the one or more computer-readable storage media. In some embodiments, a computational model of a heart having a model source configuration is used to generate a modeled cardiogram from a modeled electromagnetic output of the heart. In some embodiments, the cardiogram is adjusted based on the difference in the value of the anatomical parameter of the model source configuration and the patient source configuration. In some embodiments, the model source configuration includes scar or fibrosis or arrhythmogenic substrate locations within the heart, action potentials of the heart, conductivity of the heart, and locations of arrhythmias. In some embodiments, the model source configuration includes the dimensions of the chambers of the heart, the wall thickness of the heart, and the orientation of the heart. In some embodiments, the source configuration includes configuration parameters, the model source configuration includes values for each configuration parameter, and the patient source configuration includes values for only an appropriate subset of the configuration parameters. In some embodiments, the modeled cardiogram is based on a model orientation of the heart, and when the patient orientation of the heart is different from the model orientation, the identified modeled cardiogram takes into account the difference between the model orientation and the patient orientation.
[0266] In some embodiments, a method for generating a patient classifier performed by one or more computing systems is provided, the patient classifier being used to classify electromagnetic data derived from the electromagnetic output of an in-vivo electromagnetic source. The method accesses a model classifier for generating a classification of the electromagnetic output of the electromagnetic source. The model classifier has model classifier weights learned based on training data, the training data comprising modeled derived electromagnetic data and model classifications. The modeled derived electromagnetic data is derived from a modeled electromagnetic output generated using a computational model of the electromagnetic source, the computational model modeling the electromagnetic output of the electromagnetic source over time based on a source configuration. The method accesses patient training data, the data patient training comprising patient-derived electromagnetic data and a patient classification of that patient for each of a plurality of patients. The method initializes the patient classifier weights of the patient classifier based on the model classifier weights. The method trains the patient classifier with the patient training data and the initialized patient classifier weights. In some embodiments, the classifier is a convolutional neural network. In some embodiments, the convolutional neural network inputs a one-dimensional image. In some embodiments, the model classifier is trained using training data, the training data comprising modeled derived electromagnetic data derived from modeled electromagnetic outputs generated based on a source configuration identified as being similar to the source configuration of the patient. In some embodiments, the electromagnetic source is a heart, the source configuration represents anatomical parameters and electrophysiological parameters of the heart, the modeled electromagnetic output represents activation of the heart, and the derived electromagnetic data is based on body surface measurements. In some embodiments, the electrophysiological parameters are based on cardiac conditions, the cardiac conditions are selected from the group consisting of but not limited to: inappropriate sinus tachycardia, atrial ectopic rhythm, junctional rhythm, ventricular escape rhythm, atrial fibrillation, ventricular fibrillation, focal atrial tachycardia, atrial microreentry, ventricular tachycardia, atrial flutter, ventricular premature beats, atrial premature beats, atrioventricular nodal reentrant tachycardia, atrioventricular reentrant tachycardia, permanent junctional reciprocating tachycardia and junctional tachycardia. In some embodiments, the electromagnetic data is an electrocardiogram. In some embodiments, the classification is a source location. In some embodiments, the patient-derived EM data of a patient is derived from the patient EM output of an EM source for that patient. In some embodiments, the method further receives target patient-derived EM data of a target patient and applies the patient classifier to the target patient-derived EM data to generate a classification for the target patient.
[0267] In some embodiments, a method for classifying patient-derived electromagnetic data of a target patient performed by one or more computing systems is provided. The patient-derived electromagnetic data is derived from the patient electromagnetic output of an in vivo electromagnetic source of the patient. The method accesses a patient classifier to generate a classification of the patient-derived electromagnetic data of the electromagnetic source. The classifier is trained using the weights of a model classifier and patient training data, and the model classifier is trained using modeled derived electromagnetic data and model classification. The modeled derived electromagnetic data is generated from the modeled electromagnetic output. The modeled electromagnetic output is generated for multiple source configurations using a computational model of the electromagnetic source. The patient training data includes patient-derived electromagnetic data and patient classification. The method receives the patient-derived electromagnetic data of the target patient and applies the patient classifier to the received patient-derived electromagnetic data to generate a patient classification of the target patient. In some embodiments, the electromagnetic source is a heart, the source configuration represents anatomical parameters and electrophysiological parameters of the heart, the modeled electromagnetic output represents activation of the heart, and the derived electromagnetic data is based on surface measurements.
[0268] In some embodiments, a computing system for generating a patient classifier is provided, the patient classifier being used to classify a cardiogram. The computing system comprises: one or more computer-readable storage media storing computer-executable instructions; and one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage media. The instructions control the computing system to initialize the patient classifier weights of the patient classifier to the model classifier weights of the model classifier. The model classifier is trained based on a modeled cardiogram generated based on a computational model of the heart applied to a model heart configuration. The instructions control the computing system to train the patient classifier with the patient training data and the initialized patient classifier weights, the patient training data including a patient cardiogram and a patient classification for each of a plurality of patients. In some embodiments, the model classifier and the patient classifier are convolutional neural networks. In some embodiments, the convolutional neural network inputs a one-dimensional image. In some embodiments, the model classifier and the patient classifier are neural networks. In some embodiments, the model classifier is trained using a modeled cardiogram generated based on a model heart configuration similar to the patient's patient heart configuration. In some embodiments, the computer executable instructions further control the computing system to train a cluster patient classifier based on patient training data for multiple clusters of similar patients, the patient training data including the cardiogram and patient configuration of the patients in that cluster. In some embodiments, the computer executable instructions further control the computing system to identify similar patients based on a comparison of the patient's patient heart configuration. In some embodiments, the computer executable instructions further control the computing system to identify similar patients based on a comparison of the patient's cardiogram. In some embodiments, the computer executable instructions further control the computing system to: identify a cluster of similar patients similar to the target patient, and apply the cluster patient classifier of that identified cluster to the target patient's target patient cardiogram to generate a target patient classification of the target patient.
[0269] In some embodiments, a method for generating a classification of a target patient based on a target cardiogram of a target patient, performed by a computing system, is provided. The method generates a patient classifier based on patient training data including a cardiogram of the patient and based on a transfer from a model classifier, the model classifier being generated based on model training data including a modeled cardiogram. The modeled cardiogram is generated based on a computational model of the heart and a model heart configuration. The method applies the patient classifier to the target cardiogram to generate a target classification of the target patient.
[0270] In some embodiments, a method for generating a patient-specific model classifier performed by one or more computing systems is provided, the patient-specific model classifier being used to classify derived electromagnetic data derived from the electromagnetic output of an in-vivo electromagnetic source. The method identifies a model similar to a target patient. For each identified model, the method applies a computational model of the electromagnetic source to generate a modeled electromagnetic output of the electromagnetic source based on a model source configuration of that model; derives modeled derived electromagnetic data from the generated modeled electromagnetic output of that model; and generates a label for that model. The method uses the modeled derived electromagnetic data and the generated label as training data to train the patient-specific model classifier. In some embodiments, the classifier is a convolutional neural network that inputs a one-dimensional image. In some embodiments, the similarity between the model and the target patient is based on a source configuration. In some embodiments, the similarity between the model and the target patient is based on derived electromagnetic data. In some embodiments, the electromagnetic source is a heart, the source configuration represents anatomical parameters and electrophysiological parameters of the heart, the modeled electromagnetic output represents activation of the heart, and the derived electromagnetic data is based on body surface measurements. In some embodiments, the electromagnetic data is an echocardiogram. In some embodiments, the tag represents the source location of the disease of the electromagnetic source. In some embodiments, the training is based on a transfer from a model classifier, the model classifier is generated based on model training data including modeled derived electromagnetic data and model source configuration, and the modeled derived electromagnetic data is generated based on a computational model of the electromagnetic source. In some embodiments, the method further trains a cluster-specific model classifier for each of a plurality of clusters of similar target patients based on derived electromagnetic data of a model similar to the target patient in that cluster. In some embodiments, the method further identifies a cluster in which the target patient is similar to another target patient, and applies the cluster-specific model classifier of that identified cluster to the target patient derived electromagnetic data of the other target patient to generate a target patient tag for the other target patient. In some embodiments, the method further identifies the cluster of the target patient based on a comparison of the patient source configuration of the target patient. In some embodiments, the method further identifies the cluster of the target patient based on a comparison of the patient derived electromagnetic data of the target patient.
[0271] In some embodiments, a computing system for generating a patient-specific model classifier is provided, the patient-specific model classifier being used to classify a cardiogram of a target patient. The computing system comprises: one or more computer-readable storage media storing computer-executable instructions; and one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage media. The instructions control the computing system to: identify a model similar to the target patient; and train the patient-specific model classifier based on training data including a modeled cardiogram and a model classification of the identified model. The modeled cardiogram is generated based on a model heart configuration of the identified model using a computational model of the heart. In some embodiments, the model classification represents a source location of a cardiac condition. In some embodiments, the training of the patient-specific model classifier is based on a transfer from a model classifier generated based on model training data including a modeled cardiogram. In some embodiments, the computer executable instructions further control the computer system to: for each of a plurality of clusters of similar target patients, train a cluster-specific model classifier based on training data, the training data comprising a modeled cardiogram and a model classification of a model similar to the target patient in that cluster. In some embodiments, the computer executable instructions further control the computer system to: identify a cluster in which the target patient is similar to another target patient, and apply the cluster-specific model classifier of that identified cluster to the target patient cardiogram of the other target patient to generate a target patient classification of the other target patient. In some embodiments, the computer executable instructions control the computer system to identify clusters of similar target patients based on a comparison of patient cardiac configurations of the target patients. In some embodiments, the computer executable instructions control the computer system to identify clusters of similar target patients based on a comparison of patient cardiograms of the target patients.
[0272] In some embodiments, a method for generating a representation of an electromagnetic source performed by a computing system is provided. The method identifies modeled derived electromagnetic data that matches patient-derived electromagnetic data of a patient. The modeled derived electromagnetic data is derived from a modeled electromagnetic output of the electromagnetic source generated using a computational model of the electromagnetic source. The modeled electromagnetic output has an electromagnetic value for a location of the electromagnetic source for each of a plurality of time intervals. The method identifies a period within the modeled electromagnetic output from which the matching model-derived electromagnetic data is derived. For each of a plurality of display positions of the electromagnetic source, the method generates a display electromagnetic value for that display position based on the electromagnetic value of the modeled electromagnetic output for the identified period. The method generates a display representation of the electromagnetic source, the display representation including a visual representation of the display electromagnetic value for that display position for each of the plurality of display positions. In some embodiments, the display representation has a geometric shape based on an anatomical parameter of the electromagnetic source of the patient. In some embodiments, the visual representation of the display electromagnetic value is a shading based on the magnitude of the display electromagnetic value. In some embodiments, the visual representation of the display electromagnetic value is a color selected based on the magnitude of the display electromagnetic value. In some embodiments, the visual representation of the display electromagnetic value is an intensity of the color based on the magnitude of the display electromagnetic value. In some embodiments, the displayed electromagnetic value is based on the difference between the electromagnetic value at the beginning of the cycle and the electromagnetic value at the end of the cycle. In some embodiments, the method further generates and outputs a display representation of that display interval for each of a plurality of display intervals of the identified cycle. In some embodiments, the display representation is output sequentially to show the activation of the electromagnetic source over time. In some embodiments, when multiple instances of derived electromagnetic data match the electromagnetic data derived by the patient, the display electromagnetic value generated for the display position is based on a combination of the electromagnetic values of the modeled electromagnetic output of the matching instance from which the modeled derived electromagnetic data is derived. In some embodiments, the combination is an average weighted by the proximity of the match. In some embodiments, the electromagnetic source is a heart.
[0273] In some embodiments, a method for generating a representation of a heart performed by a computing system is provided. The method identifies a modeled cardiogram similar to a patient's cardiogram. For each of a plurality of display positions of the heart, the method generates a display electromagnetic value for that display position based on an electromagnetic value of a modeled electromagnetic output of the heart from which the modeled cardiogram is derived. The modeled electromagnetic output is generated using a computational model of the heart. The method generates a display representation of the heart, the display representation including a visual representation of the display electromagnetic value for that display position for each of the plurality of display positions. In some embodiments, the display representation has a geometric shape based on an anatomical parameter of the heart of the patient. In some embodiments, the visual representation of the display electromagnetic value is an intensity of a color based on the magnitude of the display electromagnetic value. In some embodiments, the displayed electromagnetic value is based on the difference between the electromagnetic value at the beginning of a model cycle within the modeled electromagnetic output and the electromagnetic value at the end of the model cycle. In some embodiments, the model cycle is selected based on similarity to a patient cycle within the cardiogram. In some embodiments, the method further generates and outputs, for each of a plurality of display intervals of the modeled electromagnetic output, a display representation of that display interval. In some embodiments, the display representations are output sequentially to show activation of the electromagnetic source over time.
[0274] In some embodiments, a computing system for displaying a representation of electrical activation of a patient's heart is provided. The computing system includes: one or more computer-readable storage media storing computer-executable instructions; and one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage media. The instructions control the computing system to identify a modeled cardiogram similar to the patient's cardiogram. The instructions control the computing system to generate a display representation of the heart, the display representation including, for each of a plurality of display positions of the heart, a visual representation of a display value of that display position. The display value is based on a modeled electromagnetic output of the heart from which the modeled cardiogram is derived. The modeled electromagnetic output is generated using a computational model of the heart. The instructions control the computing system to display the display representation. In some embodiments, the display representation has a geometric shape based on anatomical parameters of the heart of the patient.
[0275] In some embodiments, a method for generating a surface representation of an electromagnetic force generated by an electromagnetic source in the body is provided, which is performed by a computing system. The method accesses a sequence of vectors representing the magnitude and direction of the electromagnetic force over time. The vectors are relative to an origin. For each pair of adjacent vectors, the method identifies a region based on the origin and the vector pair; generates a regional representation of the region; and displays the generated regional representation of the region. In some embodiments, the method displays a representation of the electromagnetic source so that the displayed regional representation visually emanates from the electromagnetic source. In some embodiments, the origin is within the electromagnetic source. In some embodiments, the electromagnetic source is a heart, and the vector sequence is a vector cardiogram. In some embodiments, the electromagnetic force has a period, and the regional representation of the period forms a surface representation of that period, and the method further displays surface representations of multiple periods simultaneously. In some embodiments, the generated regional representations are displayed sequentially to show the changes in the electromagnetic force over time.
[0276] In some embodiments, a computing system for displaying a representation of a vector cardiogram is provided. The computing system includes: one or more computer-readable storage media storing computer-executable instructions; and one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage media. The instructions control the computing system to: generate a surface representation of a portion of the vector cardiogram, wherein the surface representation is bounded by points representing the x, y, and z values of the vector of the portion of the cardiogram; and display the generated surface representation. In some embodiments, the computer-executable instructions further control the computing system to display a representation of the heart from which the vector cardiogram is derived, so that the displayed surface representation visually emanates from the displayed representation of the heart. In some embodiments, the vector is relative to an origin within the heart. In some embodiments, the vector cardiogram has a cycle, and the computer-executable instructions further control the computing system to display surface representations of multiple cycles simultaneously. In some embodiments, the generated surface representation is displayed incrementally to show changes in the vector cardiogram over time. In some embodiments, the surface representation is displayed on a region-by-region basis.
[0277] In some embodiments, a method for identifying a simulated cardiogram similar to a patient cardiogram performed by a computing system is provided. The simulated cardiogram is generated based on simulated pacing, and the patient cardiogram is generated based on patient pacing of a patient. The method identifies a first simulated cardiogram whose simulated pacing is similar to the patient pacing. The method identifies from the first simulated cardiogram a second simulated cardiogram generated based on a simulated heart, the simulated heart having a simulated orientation similar to the patient orientation of the patient heart; the method identifies from the second simulated cardiogram a third simulated cardiogram, the third simulated cardiogram representing action potentials similar to the action potentials represented by the patient cardiogram. The method identifies from the third simulated cardiogram a calibration simulated cardiogram, the calibration simulated cardiogram representing conduction velocities similar to the conduction velocities represented by the patient cardiogram. In some embodiments, the method labels each calibration simulated cardiogram with configuration parameters used when generating that calibration simulated cardiogram. The method trains a classifier using the calibration simulated cardiogram and the labels as training data. In some embodiments, the method applies the trained classifier to the patient cardiogram to identify the configuration parameters of the patient. In some embodiments, the cardiogram is an electrocardiogram. In some embodiments, the cardiogram is a vector cardiogram. In some embodiments, pacing includes pacing location and pacing rate. In some embodiments, the similarity of action potentials is based on cardiograms normalized in size and duration. In some embodiments, the similarity of conduction velocity is based on cardiograms normalized in size. In some embodiments, the simulated cardiogram is identified from a set of simulated cardiograms generated based on simulated structural diseases similar to the patient's patient structural disease. In some embodiments, the simulated cardiogram is identified from a set of simulated cardiograms based on the similarity of the heart geometry of the simulated cardiogram to the patient's patient heart geometry. In some embodiments, the similarity of orientation is based on a simulated heart vector similar to the patient's patient heart vector of the patient, that is, the heart vector of the cardiac cycle phase at the heart position. In some embodiments, the simulated cardiogram is identified from a set of simulated cardiograms generated based on a simulated disease substrate similar to the patient's patient disease substrate.
[0278] In some embodiments, a method for identifying a simulated cardiogram similar to a patient's cardiogram performed by a computing system is provided. The method determines a morphological similarity between a simulated cardiogram and the patient's cardiogram based on morphology. The method identifies similar simulated cardiograms based on morphological similarity. In some embodiments, the determining morphological similarity includes: determining an orientation similarity between the simulated cardiogram and the patient's cardiogram based on orientation; and determining an electrophysiological similarity between the simulated cardiogram and the patient's cardiogram. In some embodiments, the method determines a pacing similarity between the simulated cardiogram and the patient's cardiogram, wherein the identification is further based on pacing similarity. In some embodiments, the electrophysiological similarity is based on action potential similarity and conduction velocity similarity. In some embodiments, the method trains a classifier using training data, the training data including the identified similar simulated cardiograms labeled with configuration parameters. In some embodiments, the method determines a cardiac geometry similarity between the simulated cardiogram and the patient's cardiogram, wherein the identification is further based on cardiac geometry similarity. In some embodiments, the cardiac geometry similarity is based on structural disease similarity and measurement result similarity. In some embodiments, the method determines a disease substrate similarity between a simulated cardiogram and the patient's cardiogram, wherein the identification is further based on the disease substrate similarity.
[0279] In some embodiments, one or more methods for identifying a simulated cardiogram similar to a patient's cardiogram are provided and performed by a computing system. The one or more computing systems include: one or more computer-readable storage media, the one or more computer-readable storage media being used to store computer-executable instructions; and one or more processors, the one or more processors being used to execute the computer-executable instructions stored in the one or more computer-readable storage media. The instructions control the one or more computing systems to: determine the orientation similarity of the simulated cardiogram to the patient's cardiogram based on orientation; determine the electrophysiological similarity of the simulated cardiogram to the patient's cardiogram; and identify similar simulated cardiograms based on orientation similarity and electrophysiological similarity. In some embodiments, the computer-executable instructions further control the one or more computing systems to: determine the disease substrate similarity of the simulated cardiogram to the patient's cardiogram, and identify similar simulated cardiograms further based on disease substrate similarity. In some embodiments, the computer-executable instructions further control the one or more computing systems to determine the pacing similarity of the simulated cardiogram to the patient's cardiogram, and identify similar simulated cardiograms further based on pacing similarity. In some embodiments, the electrophysiological similarity is based on action potential similarity and conduction velocity similarity. In some embodiments, the action potential similarity is based on cardiograms normalized in magnitude and duration, and the conduction velocity similarity is based on cardiograms normalized in magnitude. In some embodiments, the computer executable instructions further control the one or more computing systems to train a classifier using training data, the training data comprising the identified similar simulated cardiograms labeled with configuration parameters.
[0280] In some embodiments, a method for mapping a simulated cardiogram to a patient cardiogram of a patient performed by a computing system is provided. The method identifies a simulated cardiogram that matches the patient cardiogram based on configuration parameters and pacing positions. The method trains a mapping function that maps the identified simulated cardiogram to the patient cardiogram. The method applies the trained mapping function to the simulated cardiogram to generate a transformed simulated cardiogram. In some embodiments, the method trains a patient-specific model classifier based on the transformed simulated cardiogram. [0281...
Claims
1. A method for enhancing image data using a patient source location of a cardiac condition of a patient's heart, the method being performed by one or more computing systems, include: receiving image data, the image data comprising a 3D image of the patient's heart; Patient cardiograms of admitted patients; accessing information related to mapping of cardiograms to source locations of cardiac pathologies and mapping of cardiograms to morphological properties of the heart; identifying patient source locations of a cardiac condition of the patient based on the patient cardiograms and information related to a mapping of cardiograms to source locations, wherein the patient source locations are source locations mapped to cardiograms similar to the patient cardiogram, and wherein each cardiogram is associated with a 3D image; adjusting the patient source position based on a difference in a morphological characteristic of the patient's heart and a morphological characteristic associated with information related to a mapping of an echocardiogram to a morphological characteristic of the heart; as well as The patient source position is added to the image data to generate the enhanced image data.
2. The method of claim 1, wherein the identifying the patient source location include: A machine learning algorithm is applied that takes as input the electrocardiogram and outputs the source locations, the machine learning algorithm being trained based on the mapping.
3. The method of claim 1, wherein the identifying the patient source location include: The patient's cardiogram is compared to the mapped cardiogram to identify cardiograms that are similar to the patient's cardiogram.
4. A method as claimed in claim 1, wherein the mapping includes mapping based on one or more of the following: clinical data collected from an actual patient, simulation of the electrical activity of the heart based on varying configuration parameters, and simulation of the electrical activity of the patient's heart based on configuration parameters derived from the patient.
5. The method according to claim 1, further comprising: include: The map is calibrated for the patient's heart prior to identifying the patient source locations such that the identification is based on a calibration map. The method of claim 5 , wherein the calibration is based on the geometry of the patient's heart.
7. The method of claim 5, wherein the calibration is based on an echocardiogram of the patient.
8. The method of claim 1, further comprising: include: The enhanced image data is provided to a planning system for use in providing a basis for care for the patient based on the image data.
9. The method of claim 1, wherein the 3D image is included in a file in a standard format.
10. The method of claim 9, wherein the standard format is the Digital Imaging and Communications in Medicine ("DICOM") format.
11. The method of claim 10, wherein the patient source location is included as metadata in the file.
12. The method of claim 10, wherein the patient source location is external to the file.
13. The method of claim 10, wherein the source location is superimposed on the 3D image of the file.
14. The method of claim 1, further comprising: include: The 3D image is displayed, wherein the source location is superimposed on the displayed 3D image.
15. One or more computing systems for augmenting image data using a patient-source location of a condition of an organ of the patient, the one or more computing systems include: One or more computer-readable storage media storing computer-executable instructions for controlling the one or more computing systems to: receiving image data, the image data comprising a 3D image of the patient's organ, the 3D image being in a Digital Imaging and Communications in Medicine ("DICOM") format; receiving patient-derived electromagnetic ("EM") data derived from an EM output of an organ of the patient; accessing information related to mapping of derived EM data to source locations of conditions of a patient's organ and mapping of derived EM data to morphological properties of a patient's organ; identifying patient source locations of a condition of the patient organ based on the patient derived EM data and information related to a mapping of the derived EM data to source locations, wherein the patient source locations are source locations mapped to derived EM data similar to the patient derived EM data, and wherein each derived EM data is associated with a 3D image; adjusting the patient source position based on a difference in a morphological property of the patient organ and a morphological property associated with information related to a mapping of derived EM data to the morphological property of the organ; as well as adding the patient source position to the image data to generate enhanced image data; as well as One or more processors configured to execute the computer-executable instructions stored in the one or more computer-readable storage media.
16. One or more computing systems for enhancing image data using a patient-source location of a cardiac condition of a patient's heart, the one or more computing systems include: One or more computer-readable storage media for storing computer-executable instructions for controlling the one or more computing systems to: accessing image data, the image data comprising a 3D image of the patient's heart; access to patients’ patient cardiograms; identifying patient source locations of a cardiac condition of the patient based on the patient cardiograms and a mapping of the cardiograms to source locations, wherein the patient source locations are source locations mapped to cardiograms similar to the patient cardiogram, and wherein each cardiogram is associated with a 3D image; adjusting the identified patient source location based on a difference between a geometry of a heart represented by the 3D image associated with a cardiogram similar to the patient's cardiogram and a geometry of the patient's heart represented by the 3D image of the patient's heart; as well as enhancing the image data using the patient source position to generate the enhanced image data; as well as One or more processors configured to execute the computer-executable instructions stored in the one or more computer-readable storage media.
17. The one or more computing systems of claim 16, wherein the instructions to identify the patient source locations apply a machine learning algorithm that inputs an electrocardiogram and outputs source locations, the machine learning algorithm being trained based on the mapping.
18. The one or more computing systems of claim 16, wherein the instructions that identify the patient source location compare the patient cardiogram to the mapped cardiogram to identify a cardiogram that is similar to the patient cardiogram.
19. One or more computing systems as described in claim 16, wherein the mapping includes a mapping based on one or more of the following: clinical data collected from an actual patient; a simulation of the electrical activity of the heart based on varying configuration parameters; and a simulation of the electrical activity of the patient's heart based on configuration parameters derived from the patient.
20. The one or more computing systems of claim 16, wherein the instructions calibrate the map for the patient's heart prior to identifying the patient source location such that the identification is based on a calibration map.
21. The one or more computing systems of claim 16, wherein the instructions provide the enhanced image data to a planning system for use in providing a basis for care for the patient based on the image data.
22. The one or more computing systems of claim 16, wherein the instructions provide the enhanced image data to a system that supports treatment of a patient.
23. The one or more computing systems of claim 16, wherein the image data is in a Digital Imaging and Communications in Medicine ("DICOM") format.
24. The one or more computing systems of claim 16, wherein the adjusting is based on aligning a marked position of the geometric shape.
25. The one or more computing systems of claim 24, wherein the marked location of the 3D image of the patient's heart is specified by a fiducial marker added to the 3D image.
26. One or more computing systems for enhancing image data using a patient-source location of a cardiac condition of a patient's heart, the one or more computing systems include: One or more computer-readable storage media for storing computer-executable instructions for controlling the one or more computing systems to: accessing image data including a 3D image of the patient's heart and a patient substrate location, and a patient substrate volume indicating a volume of a substrate at the patient substrate location; access to patients’ patient cardiograms; identifying patient source locations of a cardiac condition of the patient based on the patient cardiograms and a mapping of the cardiograms to source locations, wherein the patient source locations are source locations mapped to cardiograms similar to the patient cardiogram, and wherein each cardiogram is associated with a 3D image; adjusting the identified patient source location based on a difference between a geometry of a heart represented by the 3D image associated with a cardiogram similar to the patient's cardiogram and a geometry of the patient's heart represented by the 3D image of the patient's heart; enhancing the image data using the patient source position to generate the enhanced image data; accessing a 4D image of the patient's heart, the 4D image comprising a sequence of 3D images collected during a cycle of the heart; as well as adding a substrate mask to the 4D image at the patient substrate location and having the substrate volume; as well as One or more processors configured to execute the computer-executable instructions stored in the one or more computer-readable storage media.
27. The one or more computing systems of claim 26, wherein the instructions to identify the patient source locations apply a machine learning algorithm that inputs an electrocardiogram and outputs source locations, the machine learning algorithm being trained based on the mapping.
28. The one or more computing systems of claim 26, wherein the instructions that identify the patient source location compare the patient cardiogram to the mapped cardiogram to identify a cardiogram that is similar to the patient cardiogram.
29. One or more computing systems as described in claim 26, wherein the mapping includes a mapping based on one or more of the following: clinical data collected from an actual patient; a simulation of the electrical activity of the heart based on varying configuration parameters; and a simulation of the electrical activity of the patient's heart based on configuration parameters derived from the patient.
30. The one or more computing systems of claim 26, wherein the instructions calibrate the map for the patient's heart prior to identifying the patient source location such that the identification is based on a calibration map.
31. The one or more computing systems of claim 26, wherein the instructions provide the enhanced image data to a planning system for use in providing a basis for care for the patient based on the image data.
32. The one or more computing systems of claim 26, wherein the instructions provide the enhanced image data to a system that supports treatment of a patient.
33. The one or more computing systems of claim 26, wherein the image data is in a Digital Imaging and Communications in Medicine ("DICOM") format.
34. The one or more computing systems of claim 26, wherein the patient-substrate location is specified by 3D coordinates and the patient-substrate volume is represented by an indication of a radius for each dimension of the 3D coordinates.
35. The one or more computing systems of claim 26, wherein the 4D image has voxels having intensity values, and the instructions further comprise instructions for setting the intensity values of voxels corresponding to the substrate mask to substrate intensity values.
36. The one or more computing systems of claim 26, wherein the patient substrate location and the patient source location are superimposed on the 3D image of the patient's heart.
37. The one or more computing systems of claim 26, the instructions further comprising instructions for controlling the one or more computing systems to perform the following steps: The 3D image of the patient's heart is displayed with the patient substrate location and the patient source location superimposed on the displayed 3D image.
38. The one or more computing systems of claim 30, wherein the instructions for calibrating the mapping for the patient's heart calibrate based on a geometry of the patient's heart.
39. The one or more computing systems of claim 30, wherein the instructions for calibrating the mapping to the patient's heart calibrate based on an echocardiogram of the patient.
40. The one or more computing systems of claim 33, the instructions further comprising instructions for controlling the one or more computing systems to perform the following steps: The patient-substrate location and patient-source location are stored as metadata in the image data.
41. The one or more computing systems of claim 33, the instructions further comprising instructions for controlling the one or more computing systems to perform the following steps: The patient-substrate position and the patient-source position are stored externally to the image data.
42. One or more computing systems for enhancing image data using a patient source location of a cardiac condition of a patient's heart, the one or more computing systems include: One or more computer-readable storage media for storing computer-executable instructions for controlling the one or more computing systems to: accessing image data, the image data comprising a 3D image of the patient's heart; access to patients’ patient cardiograms; identifying patient source locations of a cardiac condition of the patient based on the patient cardiograms and a mapping of the cardiograms to source locations, wherein the patient source locations are source locations mapped to cardiograms similar to the patient cardiogram, and wherein each cardiogram is associated with a 3D image; adjusting the identified patient source location based on a difference between a geometry of a heart represented by the 3D image associated with a cardiogram similar to the patient's cardiogram and a geometry of the patient's heart represented by the 3D image of the patient's heart; enhancing the image data using the patient source position to generate the enhanced image data; identifying a patient source volume associated with the patient source location; accessing a 4D image of the patient's heart, the 4D image comprising a sequence of 3D images collected during a cycle of the heart; as well as adding a source mask to the 4D image at the patient source location and having the patient source volume; as well as One or more processors configured to execute the computer-executable instructions stored in the one or more computer-readable storage media.
43. The one or more computing systems of claim 42, wherein the patient source location is specified by 3D coordinates and the patient source volume is represented by an indication of a radius for each dimension of the 3D coordinates.
44. The one or more computing systems of claim 43, wherein the 4D image has voxels having intensity values, and the instructions further comprise instructions for setting the intensity values of voxels corresponding to the source mask to source intensity values.
45. One or more computing systems as described in claim 42, wherein the instructions for controlling the one or more computing systems to identify the patient source location include instructions for controlling the one or more computing systems to apply a machine learning algorithm that inputs a cardiogram and outputs source locations, the machine learning algorithm being trained based on a mapping of the cardiogram to source locations.
46. One or more computing systems as described in claim 42, wherein the mapping includes a mapping based on one or more of the following: clinical data collected from an actual patient; a simulation of the electrical activity of the heart based on varying configuration parameters; and a simulation of the electrical activity of the patient's heart based on configuration parameters derived from the patient.
47. A method, performed by one or more computing systems having one or more computer-readable storage media and one or more processors, for enhancing image data using a patient source location of a cardiac condition of a patient's heart, the method include: accessing image data, the image data comprising a 3D image of the patient's heart, a patient substrate location, and a patient substrate volume indicating a volume of a substrate at the patient substrate location; access to patients’ patient cardiograms; identifying patient source locations of a cardiac condition of the patient based on the patient cardiograms and a mapping of the cardiograms to source locations, wherein the patient source locations are source locations mapped to cardiograms similar to the patient cardiogram, and wherein each cardiogram is associated with a 3D image; adjusting the identified patient source location based on a difference between a geometry of a heart represented by the 3D image associated with a cardiogram similar to the patient's cardiogram and a geometry of the patient's heart represented by the 3D image of the patient's heart; enhancing the image data using the patient source position to generate the enhanced image data; accessing a 4D image of the patient's heart, the 4D image comprising a sequence of 3D images collected during a cycle of the heart; as well as A substrate mask is added to the 4D image at the patient substrate location and having the substrate volume.
48. The method of claim 47, wherein the 3D image of the patient's heart is included in a file in a standard format, and wherein the standard format is a Digital Imaging and Communications in Medicine ("DICOM") format.
49. A method, performed by one or more computing systems having one or more computer-readable storage media and one or more processors, for enhancing image data using a patient source location of a cardiac condition of a patient's heart, the method include: accessing image data, the image data comprising a 3D image of the patient's heart; access to patients’ patient cardiograms; identifying patient source locations of a cardiac condition of the patient based on the patient cardiograms and a mapping of the cardiograms to source locations, wherein the patient source locations are source locations mapped to cardiograms similar to the patient cardiogram, and wherein each cardiogram is associated with a 3D image; adjusting the identified patient source location based on a difference between a geometry of a heart represented by the 3D image associated with a cardiogram similar to the patient's cardiogram and a geometry of the patient's heart represented by the 3D image of the patient's heart; as well as The image data is enhanced using the patient source position to generate the enhanced image data.
50. The method of claim 49, further comprising: include: accessing a 4D image of the patient's heart, a patient substrate location, and a patient substrate volume indicating a volume of substrate at the patient substrate location, the 4D image comprising a sequence of 3D images collected during a cycle of the heart; as well as A substrate mask is added to the 4D image at the patient substrate location and having the substrate volume.
51. A method, performed by one or more computing systems having one or more computer-readable storage media and one or more processors, for enhancing image data using a patient source location of a cardiac condition of a patient's heart, the method include: accessing image data, the image data comprising a 3D image of the patient's heart; access to patients’ patient cardiograms; identifying patient source locations of a cardiac condition of the patient based on the patient cardiograms and a mapping of the cardiograms to source locations, wherein the patient source locations are source locations mapped to cardiograms similar to the patient cardiogram, and wherein each cardiogram is associated with a 3D image; adjusting the identified patient source location based on a difference between a geometry of a heart represented by the 3D image associated with a cardiogram similar to the patient's cardiogram and a geometry of the patient's heart represented by the 3D image of the patient's heart; enhancing the image data using the patient source position to generate the enhanced image data; identifying a patient source volume associated with the patient source location; accessing a 4D image of the patient's heart, the 4D image comprising a sequence of 3D images collected during a cycle of the heart; as well as A source mask is added to the 4D image at the patient source location and having the patient source volume.
Citation Information
Patent Citations
Systems and methods for estimating physiological heart measurements from medical images and clinical data
CN105474219A
Computational localization of fibrillation sources
CN108431720A
Heart condition determination method, robot control and system
CN108601547A
Automated processing of electrophysiology data
JP2008523929A
Wireless telecommunication module for embedded EEG electrodes and EEG detection system including the same
KR1020170001420A
