Automatic Identification of Scar Regions in Organic Tissues Using Multiple Imaging Modalities

Abstract

To provide a system and a method for automatically identifying scar areas within organic tissue on the basis of multiple imaging modalities.SOLUTION: A method and an apparatus for implementing scar tissue identification using a processor coupled to a memory are disclosed. The method and apparatus receive a first modality and a second modality. The first modality is of a first type. The second modality is of a second type, which is different from the first type. Each of the first modality and the second modality describes organic tissue of a patient according to the first and second types. The method and apparatus cross-reference the first modality and the second modality and generate improved image data for the first modality based on the cross referencing. The image data includes higher accuracy or higher resolution than original data of the first modality.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 058,439, filed Jul. 29, 2020, which is incorporated herein by reference in its entirety as if fully set forth.

[0002] (Field of the Invention) The present invention relates to methods and systems for artificial intelligence and machine learning. More specifically, the present invention relates to systems and methods for automatically identifying scar regions within organic tissues based on multiple imaging modalities.

Background Art

[0003] For the treatment of heart diseases such as arrhythmia, cardiac imaging (i.e., imaging of heart tissues, heart cavities, veins, arteries, and / or pathways, etc., which is also known as cardiac scan or cardiac imaging) is often required. In cardiac imaging, since scar tissue can be identified, it enables the evaluation of the characteristics of the cardiac substrate and the understanding of the arrhythmia mechanism. Conventional imaging modalities including, by way of example, ultrasound imaging and magnetic resonance imaging (MRI) can be used to identify scar tissue, but are not limited thereto. Ultrasound imaging is a readily available real - time tool, but ultrasound imaging is operator - dependent (thereby exposing the results to subjective interpretation) and has lower resolution compared to other conventional imaging modalities. MRI utilizes gadolinium - delayed enhancement (LGE) technology and outputs higher resolution than ultrasound imaging.

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, MRI has no real - time availability, is relatively expensive, and the resulting voltage mapping has no ability to evaluate the entire depth of the heart tissue. Due to the limitations of conventional imaging modalities, there is a need to provide an improved method for cardiac imaging.

Means for Solving the Problems

[0005] According to one embodiment, a method and apparatus for implementing a scar tissue identifier using a processor coupled to a memory are provided. The method and apparatus receive a first modality and a second modality. The first modality is of a first type. The second modality is of a second type different from the first type. Each of the first modality and the second modality represents a patient's organic tissue according to the first type and the second type, respectively. The method and apparatus cross-reference the first modality and the second modality and generate improved image data of the first modality based on the cross-reference. The improved image data includes higher accuracy or higher resolution than the original data of the first modality.

[0006] According to one or more embodiments, the embodiments of the above method can be implemented within an apparatus, a system, and / or a computer program product. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] A more detailed understanding can be obtained from the following description taken in conjunction with the accompanying drawings, which are shown by way of example, wherein like reference numerals in the figures indicate like elements.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0008] Methods and systems of artificial intelligence and machine learning are disclosed herein. More specifically, the present disclosure relates to a system and method for automatically identifying scar regions within an organic tissue (e.g., the heart) based on multiple imaging modalities. The system and method include processor-executable code or software that is necessarily rooted in the process operation by a medical device and the processing of its hardware in order to improve one modality based on other modalities. According to one embodiment, the system and method provide a specific multi-step data operation of multiple imaging modalities, which provides real-time image data with higher accuracy and resolution than conventional imaging modalities that support better diagnosis (e.g., this real-time image data is not available with conventional imaging modalities). In this regard, during operation, the system and method also cross-reference different modalities to enhance the discovery of a single modality.

[0009] For example, a system and method operating to identify scar tissue accesses, or receives, one or more images (e.g., at least one of a first / same type of a first modality) from ultrasound of an organic tissue of a patient (e.g., the heart) and data (e.g., a second / different type of a second modality) from LGE-MRI. Next, the system and method operating to identify scar tissue cross-references those ultrasound images to the LGE-MRI data (e.g., based on voltage) by comparing and correlating each location of the ultrasound images with a corresponding location on the LGE-MRI data. Thus, the system and method operating to identify scar tissue utilizes imaging processing (and / or other methods), performs cross-referencing, and adjusts and interprets the data of the ultrasound images based on the LGE-MRI data. The adjusted and interpreted data of the ultrasound images is used by a system and method operating to identify scar tissue and automatically identify a scar region within the organic tissue.

[0010] The technical effects and advantages of a system and method operating to identify scar tissue include generating more accurate and higher resolution real-time image data of ultrasound images (e.g., higher accuracy or higher resolution than the original data of the first modality) without relying on subjective interpretation by a human operator (like conventional imaging modalities). The system and method operating to identify scar tissue and the generated real-time image data also enables evaluation of the entire depth of the organic tissue, overcomes the low resolution of conventional ultrasound imaging, and is relatively inexpensive and accessible compared to MRI. The system and method operating to identify scar tissue can be practically applied, but not limited to, ablation ultrasound technology, planning and diagnosis of lesions, and evaluation and diagnosis of magnetic resonance to address one or more medical conditions such as atrial fibrillation, atrial flutter, general electrophysiology, arrhythmia, ventricular fibrillation, ventricular tachycardia, etc.

[0011] FIG. 1 is a schematic diagram of an exemplary system 100 (e.g., a medical device apparatus) in which one or more features of the subject matter of the present disclosure may be implemented. Information of an imaging dataset (e.g., a training dataset) can be collected using all or part of the system 100, and / or the scar tissue identifier described herein can be implemented using all or part of the system 100.

[0012] The system 100 may include components such as a catheter 105 configured to image internal organs using intravascular ultrasound and / or MRI catheterization methods. The catheter 105 can also be further configured to acquire biological data including electrical signals of the heart (e.g., intracardiac signals). Although the catheter 105 is shown as a point catheter, it is understood that any shaped catheter including one or more elements (e.g., electrodes, tracking coils, piezoelectric transducers, etc.) can be used to implement the embodiments disclosed herein.

[0013] The system 100 includes a probe 110 having a shaft that a physician or medical professional 115 can navigate into a body part such as the heart 120 of a patient 125 lying on a bed (or table) 130. According to an embodiment, a plurality of probes may be provided, but for simplicity, a single probe 110 is described herein. However, it is understood that the probe 110 may represent a plurality of probes.

[0014] The exemplary system 100 can be utilized to detect, diagnose, and treat heart diseases (e.g., using a scar tissue identifier). Heart diseases such as arrhythmias (especially atrial fibrillation) continue as common and dangerous medical diseases, particularly in the elderly population. In a patient (e.g., patient 125) with normal sinus rhythm, the heart (e.g., heart 120) includes atria, ventricles, and conduction tissue, is electrically stimulated, and pulsates in a synchronized and patterned manner (note that this electrical excitation can be detected as an intracardiac signal, etc.).

[0015] In a patient having an arrhythmia (e.g., patient 125), the abnormal region of the heart tissue does not follow the synchronous beating cycle associated with normal conductive tissue as in a patient having normal sinus rhythm. Instead, the abnormal region of the heart tissue abnormally spreads to adjacent tissue, thereby disrupting the cardiac cycle into an asynchronous heart rhythm (note that this asynchronous heart rhythm can also be detected as an intracardiac signal). Such abnormal conduction has been known for a long time and occurs in various regions of the heart (e.g., heart 120) along the conduction path of the atrioventricular (AV) node, for example, within the region of the sino-atrial (SA) node or within the myocardial tissue forming the walls of the cardiac chambers of the ventricles and atria.

[0016] Furthermore, arrhythmias including atrial arrhythmias may be of the multiple wavelet reentrant type, scattered around the atria and often characterized by multiple asynchronous loops of self-propagating electrical impulses (e.g., another example of intracardiac signals). Alternatively, or in addition to the multiple wavelet reentrant type, arrhythmias may also have a local origin, such as when an isolated region of tissue within the atria spontaneously excites in a rapid repetitive pattern (e.g., another example of intracardiac signals). Ventricular tachycardia (V-tach or VT) is a tachycardia or rapid heart rhythm that occurs in one of the ventricles. This is a potentially lethal arrhythmia as it can lead to ventricular fibrillation and sudden death.

[0017] Atrial fibrillation, which is one type of arrhythmia, occurs when normal electrical impulses (e.g., another example of intracardiac signals) generated by the sinoatrial node are overwhelmed by chaotic electrical impulses (e.g., signal interference) occurring within the atria and pulmonary veins, and conduct irregular impulses to the ventricles. As a result, an irregular heartbeat occurs, which may persist for minutes to weeks, or even years. Atrial fibrillation (AF) is a chronic condition that often leads to a slight increase in the risk of death, often due to stroke. The first-line treatment for AF is pharmacotherapy to reduce the heart rate or restore normal heart rhythm. Additionally, patients with AF are often given anticoagulants to protect against the risk of stroke. The use of such anticoagulants is associated with the risk of internal bleeding on its own. In some patients, pharmacotherapy is not sufficient, and the AF in these patients is determined to be drug refractory, i.e., untreatable with standard pharmacological interventions. Synchronized electrical defibrillation can also be used to convert AF to normal sinus rhythm. Alternatively, patients with AF are treated by catheter ablation.

[0018] Catheter ablation-based treatment may include mapping the electrical properties of cardiac tissue, particularly the endocardium and cardiac volume, and selectively ablating cardiac tissue by application of energy. Cardiac mapping (an example of cardiac imaging) includes creating a map of the potential of wave propagation along cardiac tissue (e.g., a voltage map), or a map of the arrival times at points located in various tissues (e.g., a local time activation (LAT) map). Cardiac mapping (e.g., a cardiac map) can be used to detect local cardiac tissue dysfunction. Ablation, such as ablation based on cardiac mapping, can stop or alter the propagation of unwanted electrical signals from one part of the heart to another and restore normal sinus rhythm.

[0019] The ablation method destroys unwanted electrical pathways by forming non-conductive lesions. Various energy delivery modalities for forming the lesions are disclosed and include the use of microwaves, lasers, cryoablation, and more generally radiofrequency energy to create conduction blocks along the heart tissue wall. In a two-step procedure where ablation follows mapping, the electrical activity at each point within the heart is typically sensed and measured by advancing a catheter (e.g., catheter 105) containing one or more electrical sensors (e.g., at least one ablation electrode 134 of catheter 105) inside the heart (e.g., heart 120) and acquiring data at a number of points. This data (e.g., biometric data including intracardiac signals and 3D positions) is then used to identify the target region of the endocardium where ablation is to be performed. Note that the use of scar tissue identifiers employed by an exemplary system 100 (e.g., a medical device) provides more accurate and higher resolution real-time image data to support better diagnosis.

[0020] Cardiac ablation and other cardiac electrophysiological procedures when treating difficult diseases such as atrial fibrillation and ventricular tachycardia by clinicians are becoming increasingly complex. The treatment of complex arrhythmias currently relies on the use of three-dimensional (3D) mapping systems to reconstruct the anatomical structure of the heart chamber of interest. In this regard, the scar tissue identifier used by an exemplary system 100 (e.g., a medical device) herein provides underlying real-time image data so as to be able to generate improved images, scans, and / or maps for treating heart diseases.

[0021] For example, cardiologists rely on software such as the Complex Fractionated Atrial Electrograms (CFAE) module of the CARTO (registered trademark) 3 3D mapping system made by Biosense Webster Inc. (Irvine, Calif.) to generate and analyze intracardiac electrograms (EGMs). The scar tissue identifier of an exemplary system 100 (e.g., a medical device) enhances this software to generate and analyze improved intracardiac images, scans, and / or maps and enable determination of ablation points for the treatment of a wide range of heart diseases, including atypical atrial fibrillation and ventricular tachycardia.

[0022] The improved images, scans, and / or maps supported by the scar tissue identifier can provide multiple information regarding the electrophysiological properties of internal organs (e.g., the heart and / or organic tissues including scar tissue) representing the cardiac substrate (anatomical and functional) of these difficult arrhythmias.

[0023] Cardiomyopathies with different etiologies (ischemic, dilated cardiomyopathy (DCM), hypertrophic cardiomyopathy (HCM), arrhythmogenic right ventricular dysplasia (ARVD), left ventricular non-compaction (LVNC), etc.) have identifiable substrates and are characterized by unhealthy tissue regions surrounded by normally functioning cardiomyocyte regions.

[0024] Abnormal tissue generally features low-voltage EGM. However, initial clinical experience in endocardial-epicardial mapping has shown that low-voltage areas do not always exist as the sole arrhythmogenic mechanism in such patients. In fact, low or intermediate voltage areas may exhibit EGM fragmentation and delayed activity during sinus rhythm, which corresponds to significant narrowings identified during sustained and organized ventricular arrhythmias and applies, for example, only to intolerant ventricular tachycardia. Furthermore, in many cases, EGM fragmentation and delayed activity are observed in areas showing normal or near-normal voltage amplitudes (greater than 1 to 0.5 mV). The latter areas can be evaluated according to voltage amplitude but are not considered normal according to intracardiac signals and thus represent a true arrhythmogenic substrate. 3D mapping can identify the location of the arrhythmogenic substrate on the endocardial and / or epicardial layers of the right / left ventricles, which may vary in distribution due to the progression of the underlying disease.

[0025] The substrates associated with these heart diseases are related to the presence of fragmented delayed EGM in the endocardial and / or epicardial layers of the ventricular cavities (right and left). 3D mapping systems such as CARTO® 3 can identify the location of the potential arrhythmogenic substrate in cardiomyopathy with respect to abnormal EGM detection.

[0026] An electrode catheter (e.g., catheter 105) is used during medical procedures. The electrode catheter is used to stimulate and map the electrical activity within the heart and ablate the sites of abnormal electrical activity. During use, the electrode catheter is inserted into a major vein or artery, such as the femoral artery, and then guided into the heart chamber. A typical ablation procedure involves inserting a catheter having at least one electrode at its distal end into the heart chamber. A reference electrode is generally provided by taping it to the patient's skin or by a second catheter positioned within or near the heart. A radio frequency (RF) current is applied to the tip electrode of the ablation catheter, and current flows through the medium surrounding the tip electrode, i.e., blood and tissue, towards the reference electrode. The distribution of the current depends on the amount of the electrode surface in contact with the tissue when compared to blood, which has a higher conductivity than the tissue. Heating of the tissue occurs due to the electrical resistance of the tissue. When the tissue is sufficiently heated, cell destruction is caused in the heart tissue, and as a result, a damaged area is formed within the non-conductive heart tissue. During this process, the electrode is also heated by conduction from the heated tissue to the electrode itself. If the electrode temperature becomes sufficiently high, in some cases exceeding 60 degrees Celsius, a thin transparent film of dehydrated blood protein can form on the surface of the electrode. As the temperature continues to rise, this dehydrated layer can gradually become thicker and blood coagulates on the electrode surface. Since the dehydrated biological material has a higher electrical resistance than the endocardial tissue, the impedance to the flow of electrical energy into the tissue also increases. When the impedance becomes sufficiently high, an impedance rise occurs and the catheter must be removed from the body and the tip electrode cleaned.

[0027] The treatment of heart diseases such as arrhythmia often requires obtaining detailed mapping of heart tissue, heart chambers, veins, arteries, and / or electrical pathways. For example, a prerequisite for performing catheter ablation without problems is to accurately locate the cause of arrhythmia within the heart chamber. Such location can be done by electrophysiological investigation, during which the potential is spatially decomposed and detected by a mapping catheter introduced into the heart chamber. This electrophysiological examination, so-called electroanatomical mapping, provides 3D mapping data, which can be displayed on a monitor. In many cases, the mapping function and the treatment function (e.g., ablation) are provided by a single catheter or a group of catheters, and the mapping catheter also operates as a treatment (e.g., ablation) catheter at the same time. In this case, the scar tissue identifier can be directly stored and executed by the catheter 105.

[0028] Mapping of heart regions such as the heart region, tissue, veins, arteries, and / or electrical pathways of the heart (e.g., 120) can result in the identification of problem areas such as scar tissue, arrhythmia sources (e.g., electrical rotors), and healthy regions. The heart region can be mapped such that a visual rendering of the mapped heart region is provided using a display, as further disclosed herein. Additionally, heart mapping (an example of heart imaging) can include mapping based on one or more modalities such as, but not limited to, local activation time (LAT), electrical activity, topology, bipolar mapping, dominant frequency, or impedance. Data corresponding to multiple modalities can be captured using catheters inserted into the patient's body and provided for rendering simultaneously or at different times based on corresponding set values and / or the preferences of medical experts.

[0029] Cardiac mapping can be performed using one or more techniques. As an example of a first technique, cardiac mapping can be performed by sensing the electrical properties of cardiac tissue, such as the LAT, as a function of the exact location within the heart. The corresponding data can be acquired using one or more catheters advanced into the heart using a catheter having electrical and location sensors at its distal tip. As a specific example, location and electrical activity can first be measured at about 10 to about 20 points on the inner surface of the heart. These data points may generally be sufficient to generate a satisfactory quality preliminary restoration or map of the heart surface. This preliminary map can be combined with data acquired at additional points to generate a more comprehensive map of the electrical activity of the heart. In a clinical setting, it is not uncommon to accumulate data at over 1000 sites to generate a detailed and comprehensive map of the electrical activity of the cardiac chamber. Subsequently, the generated detailed map can serve as a basis for making therapeutic decisions, such as decisions regarding tissue ablation, to modify the propagation of the electrical activity of the heart and restore normal cardiac rhythm.

[0030] Returning to FIG. 1, to perform the cardiac imaging of interest, medical professional 115 can insert shaft 137 through sheath 136 while operating the distal end of shaft 137 using deflection from manipulator 138 and / or sheath 136 near the proximal end of catheter 105. As shown in insertion view 140, catheter 105 can be attached at the distal end of shaft 137. Catheter 105 may be inserted through sheath 136 in a folded state and then expanded within heart 120. As further described herein, catheter 105 can include at least one ablation electrode 134 and a catheter needle.

[0031] According to an embodiment, the catheter 105 can be configured to ablate a tissue region of a cardiac chamber of the heart 120. Insertion illustration 150 shows an enlarged view of the catheter 105 within a cardiac chamber of the heart 120. As shown in the figure, the catheter 105 can include at least one ablation electrode 134 coupled to the body of the catheter. According to other embodiments, a plurality of elements can be connected via a spline that forms the shape of the catheter 105. One or more other elements (not shown) can be provided, and they can be any element configured to perform ablation or acquire biological data, and can be electrodes, transducers, or one or more other elements.

[0032] According to the embodiments disclosed herein, ablation electrodes such as at least one ablation electrode 134 can be configured to supply energy to a tissue region of an internal organ such as the heart 120. The energy can be thermal energy, which may start from the surface of the tissue region and may cause damage to the tissue region extending into the thickness of the tissue region.

[0033] According to a plurality of embodiments disclosed herein, the biological data may include one or more of LAT, electrical activity, topology, bipolar mapping, dominant frequency, impedance, etc. This LAT can be the time point of threshold activity corresponding to local activation calculated based on a normalized initial starting point. The electrical activity can be any applicable electrical signal that can be measured based on one or more thresholds, and can be detected and / or enhanced based on a signal-to-noise ratio and / or other filters. The topology can correspond to the physical structure of a body part or a part of a body part, and can correspond to changes in the physical structure regarding different parts of the body part or regarding different body parts. The impedance can be a resistance measurement value in a given region of a body part.

[0034] As shown in FIG. 1, the probe 110 and the catheter 105 can be connected to the console 160. The console 160 may include a computing device 161 that uses the scar tissue identifier described herein. According to one embodiment, the console 160 and / or the computing device 161 includes at least a processor and a memory, the processor executes computer instructions regarding the scar tissue identifier described herein, and the memory stores instructions for execution by the processor.

[0035] The computing device 161 can be any computing device including software and / or hardware, such as a general-purpose computer, and includes appropriate front-end and interface circuits 162 for transmitting and receiving signals between the catheters 105 and controlling other components of the system 100. The computing device 161 can typically include a real-time noise reduction circuit configured as a field programmable gate array (FPGA), followed by an analog-to-digital (A / D) electrocardiogram or electromyogram (EMG) signal conversion integrated circuit. The computing device 161 can pass signals from the A / D ECG or EMG circuit to another processor and / or can be programmed to perform one or more of the functions disclosed herein. For example, the one or more functions include receiving a first modality and a second modality, cross-referencing the first modality and the second modality, and generating improved image data of the first modality based on the cross-reference. The front-end and interface circuits 162 include an input / output (I / O) communication interface that enables the console 160 to receive signals from and / or transmit signals to at least one ablation electrode 134.

[0036] In some embodiments, computing device 161 can be further configured to receive biometric data, such as electrical activity, and determine whether a given tissue region conducts electricity. According to one embodiment, computing device 161 can be external to console 160, for example, disposed within a catheter, within an external device, within a mobile device, within a cloud-based device, or can be a stand-alone processor.

[0037] As described above, computing device 161 can include a general-purpose computer that can be programmed with software to perform the functions of the scar tissue identifier described herein. The software can be downloaded, for example, in electronic form, over a network, to the general-purpose computer, or alternatively or additionally, can be provided and / or stored on a non-transitory tangible medium, such as magnetic memory, optical memory, or electronic memory (e.g., any suitable volatile and / or non-volatile memory, such as random access memory or a hard disk drive). The exemplary configuration shown in FIG. 1 can be modified to implement the embodiments disclosed herein. The embodiments of the present disclosure can be similarly applied using other system components and settings. Further, system 100 can include additional components, such as elements for detecting electrical activity, wired or wireless connectors, processing and display devices.

[0038] According to one embodiment, the display 165 is connected to the computing device 161. During the procedure, the computing device 161 can facilitate the presentation of the body part rendering to the medical professional 115 on the display 165 and store data representing the body part rendering in memory. In some embodiments, the medical professional 115 may be able to manipulate the body part rendering using one or more input devices such as a touchpad, mouse, keyboard, gesture recognition device, etc. For example, the position of the catheter 105 can be changed using the input device, and as a result, the rendering is updated. In an alternative embodiment, the display 165 can include a touch screen configured to receive input from the medical professional 115 in addition to presenting the body part rendering. Note that the display 165 may be installed in the same location, or at a remote location such as a separate hospital, or within a separate healthcare provider network. Further, the system 100 may be part of a surgical system that is configured to acquire anatomical and electrical measurements of a patient's organ such as the heart 120 and perform an ablation procedure on the heart. An example of such a surgical system is the Carto® system sold by Biosense Webster.

[0039] The console 160 can be connected to the body surface electrodes by a cable, and the body surface electrodes can include adhesive skin patches that are attached to the patient 125. The processor, in conjunction with the current tracking module, can determine the position coordinates of the catheter 105 inside the body part (e.g., the heart 120) of the patient 125. The position coordinates can be based on the impedance or electromagnetic field measured between the body surface electrodes and the electrodes of the catheter 105 or other electromagnetic components (e.g., at least one ablation electrode 134). Additionally, or alternatively, position pads may be placed on the surface of the bed 130 or separated from the bed 130.

[0040] System 100 can also, and optionally, acquire biological data such as anatomical measurements of the heart 120 using ultrasound, computed tomography (CT), magnetic resonance imaging (MRI), or other medical imaging techniques known in the art. System 100 can acquire an electrocardiogram (ECG) or electrical measurements using a catheter or other sensor that measures the electrical properties of the heart 120. The biological data, including the anatomical and electrical measurements, can then be stored in a non-transitory tangible medium of the console 160. The biological data can be transmitted from the non-transitory tangible medium to the computing device 161. Alternatively, or additionally, the biological data may be transmitted to a server, which may be local or remote, using a network as further described herein.

[0041] According to one or more embodiments, a catheter including a position sensor can be used to determine the trajectories of points on the surface of the heart. These trajectories can be used to infer motion characteristics such as the contractile force of the tissue. A map showing such motion characteristics can be constructed when the trajectory information is sampled at a sufficient number of points within the heart 120.

[0042] The electrical activity at a point within the heart 120 can typically be measured by advancing a catheter 105 that houses an electrical sensor at or near its distal tip (e.g., at least one ablation electrode 134) to that point within the heart 120, bringing the tissue into contact with the sensor, and acquiring data at that point. One drawback associated with using a catheter 105 that houses only a single distal tip electrode to map a cardiac chamber is the long period required to accumulate data for each point over the number of essential points required for a detailed map of the entire cardiac chamber. Thus, multi-electrode catheters have been developed to simultaneously measure the electrical activity at multiple points within the cardiac chamber.

[0043] The multi-electrode catheter can be implemented using any applicable shape such as a linear catheter having a plurality of electrodes, a balloon catheter including electrodes disposed on a plurality of struts forming a balloon, a lasso catheter or loop catheter having a plurality of electrodes, or any other applicable shape. The linear catheter may be wholly or partially elastic so that it can be twisted, bent, or otherwise changed in its shape based on the received signals and / or based on the application of an external force (e.g., cardiac tissue) to the linear catheter. The balloon catheter may be designed such that when deployed within a patient's body, its electrodes can be held in intimate contact with the endocardial surface. By way of example, the balloon catheter can be inserted into a lumen such as a pulmonary vein (PV). The balloon catheter can be inserted into the PV in a collapsed state such that, as a result, the balloon catheter does not occupy the maximum volume of the PV while inserted therein. The balloon catheter can be inflated such that those of its electrodes on the balloon catheter are in contact with the entire circular region of the PV while inside the PV. Such contact with the entire circular portion of the PV, or any other lumen, can enable efficient mapping and / or ablation.

[0044] According to one example, the multi-electrode catheter can be advanced into the heart chamber of the heart 120. Anteroposterior (AP) and lateral fluoroscopic images can be acquired to establish the position and orientation of each of the electrodes. EGM can be recorded from each of the electrodes in contact with the heart surface relative to a temporary reference such as the occurrence of the P wave in sinus rhythm from the body surface ECG. The systems further disclosed herein can distinguish between electrodes that record electrical activity and electrodes that do not record electrical activity by virtue of not being in proximity to the endocardial wall. After the first EGM is recorded, the catheter can be repositioned and the fluoroscopic image and EGM can be recorded again. An electrical map can then be constructed from repeated instances of the above process.

[0045] According to one example, cardiac mapping can be generated based on the detection of the intracardiac potential field. A non-contact method can be implemented to simultaneously acquire a large amount of cardiac electrical information. For example, a catheter having a distal end portion can include a series of sensor electrodes distributed over its entire surface and connected to an insulated conductor for connection to signal sensing and processing means. The size and shape of the end portion can be such that the electrodes are arranged at a large distance from the wall of the cardiac chamber. The intracardiac potential field can be detected during one heartbeat. According to one example, the sensor electrodes can be distributed on a series of circumferences spaced from each other in a plane. These planes can be perpendicular to the long axis of the end portion of the catheter. At least two additional electrodes can be provided adjacent to both ends of the long axis of the end portion. As a more specific example, the catheter may include four circumferences having eight electrodes arranged at equal angular intervals on each circumference. Thus, in this particular embodiment, the catheter can include at least 34 electrodes (32 circumferential electrodes and 2 end electrodes).

[0046] According to another example, an electrophysiological cardiac mapping system and technique based on a non-contact and non-expanding multi-electrode catheter can be implemented. EGM can be obtained using a catheter having a plurality of electrodes (e.g., 42 to 122 electrodes). According to this implementation, knowledge of the relative geometric shape of the probe and the endocardium can be obtained by an independent imaging modality such as transesophageal echocardiography. After independent imaging, non-contact electrodes can be used to measure the cardiac surface potential and construct a map therefrom. This technique may include the following steps (after the independent imaging step). That is, (a) measuring the potential using a plurality of electrodes disposed on a probe disposed within the heart 120, (b) determining the geometric relationship between the probe surface and the endocardial surface, (c) generating a matrix of coefficients representing the geometric relationship between the probe surface and the endocardial surface, and (d) determining the endocardial potential based on the electrode potential and the matrix of coefficients.

[0047] According to another example, techniques and apparatus for mapping the potential distribution of a heart chamber can be implemented. An intracardiac multi-electrode mapping catheter assembly may be inserted into a patient's heart 120. This mapping catheter assembly can include a multi-electrode array with an integral reference electrode, or preferably, a companion reference catheter. These electrodes can be deployed in the form of a substantially spherical array. The electrode array can be spatially referenced to a point on the endocardial surface by a reference electrode or by a reference catheter that contacts the endocardial surface. A preferred electrode array catheter can have a number of individual electrode sites (e.g., at least 24). In addition, the method of this example can be implemented by knowing the location of each electrode site on the array and knowing the geometric shape of the heart. These locations are preferably determined by impedance plethysmography.

[0048] According to another example, a cardiac mapping catheter assembly can include an electrode array that defines a number of electrode sites. This mapping catheter assembly can also include a lumen for receiving a reference catheter having a distal tip electrode assembly that can be used to examine the heart wall. The mapping catheter may include a braid of insulating wires, and each of the wires can be used to form an electrode site. This catheter can be easily positioned in the heart 120 to obtain electrical activity information from a first set of non-contact electrode sites and / or a second set of contact electrode sites.

[0049] According to another example, another catheter for mapping the electrophysiological activity within the heart can be implemented. The catheter body can include a distal tip adapted to supply a stimulation pulse for pacing the heart, or an ablation electrode for ablating tissue that contacts its tip. The catheter may further include at least a pair of orthogonal electrodes, and the orthogonal electrodes generate a differential signal indicative of local cardiac electrical activity adjacent to the orthogonal electrodes.

[0050] According to another embodiment, a process for measuring electrophysiological data within the heart cavity can be implemented. This method can include, in part, placing a set of active and passive electrodes within the heart 120, generating an electric field within the heart cavity by supplying current to the active electrode, and measuring the electric field at the passive electrode sites. The passive electrodes are included in an array disposed on the inflatable balloon of a balloon catheter. In a preferred embodiment, the array is said to have 60 to 64 electrodes.

[0051] According to another example, cardiac mapping can be implemented using one or more ultrasonic transducers. The ultrasonic transducers can be inserted into the patient's heart 120 and can collect a plurality of ultrasonic slices (e.g., two-dimensional or three-dimensional slices) at various locations and orientations within the heart 120. In some cases, the location and orientation of a particular ultrasonic transducer may be known, and the collected ultrasonic slices can be stored so that they can be displayed later. One or more ultrasonic slices corresponding to the position of a probe (e.g., a therapeutic catheter) can be displayed later, and the probe can be superimposed on the one or more ultrasonic slices.

[0052] According to other examples, body patches and / or body surface electrodes can also be placed on or near the patient's body. A catheter having one or more electrodes can be placed within the patient's body (e.g., within the patient's heart 120), and the position of the catheter can be determined by the system based on signals transmitted and / or received between one or more electrodes of the catheter and the body patch and / or body surface electrodes. Further, the catheter electrodes can sense biological data (e.g., LAT values) from within the patient's body (e.g., within the heart 120). The biological data can be associated with the determined position of the catheter, and as a result, a rendering of the patient's body part (e.g., the heart 120) can be displayed, showing the biological data superimposed on the shape of the body.

[0053] Referring now to FIG. 2, a block diagram of an exemplary system 200 for remotely monitoring and transmitting biometric data (i.e., patient biometric information, patient data, or patient biometric data) is shown. In the example shown in FIG. 2, system 200 includes a monitoring and processing device 202 associated with patient 204 (i.e., patient data monitoring and processing device), a local computing device 206, a remote computing system 208, a first network 210, and a second network 211. According to one or more embodiments, the monitoring and processing device 202 can be an example of the catheter 105 of FIG. 1, patient 204 can be an example of patient 125 of FIG. 1, and local computing device 206 can be an example of console 160 of FIG. 1.

[0054] The monitoring and processing device 202 includes a patient biometric sensor 212, a processor 214, a user input (UI) sensor 216, a memory 218, and a transmitter-receiver (i.e., transceiver) 222. During surgery, the monitoring and processing device 202 acquires biometric data of patient 204 (e.g., electrical signals, blood pressure, temperature, blood glucose level, or other biometric data), and / or receives at least a portion of the biometric data representing any acquired patient biometric information, as well as any acquired patient biometric information associated with any acquired patient biometric information from one or more other patient biometric information monitoring and processing devices. The additional information may be, for example, diagnostic information and / or additional information obtained from an additional device such as a wearable device. The monitoring and processing device 202 can process data including the acquired biometric data and any biometric data received from one or more other patient biometric information monitoring and processing devices using the scar tissue identifier described herein. For example, in this regard, when processing data, the scar tissue identifier includes receiving a first modality and a second modality, cross-referencing the first modality and the second modality, and generating improved image data of the first modality based on the cross-reference.

[0055] The monitoring and processing device 202 can continuously or periodically monitor, store, process, and transmit biometric information (e.g., acquired biometric data) of any number of various patients via the network 210. As described herein, examples of the patient's biometric information include electrical signals (e.g., ECG signals and brain biometric information), blood pressure data, blood glucose data, and temperature data. The patient's biometric metrics can be monitored and transmitted for treating any number of various diseases such as cardiovascular diseases (e.g., arrhythmia, cardiomyopathy, and coronary artery disease) and autoimmune diseases (e.g., type I and type II diabetes).

[0056] The patient biometric sensor 212 includes, for example, one or more transducers configured to convert one or more environmental conditions into electrical signals, resulting in the acquisition of different types of biometric data. For example, the patient biometric sensor 212 may include one or more of electrodes configured to acquire electrical signals (e.g., heart signals, brain signals, or other bioelectrical signals), temperature sensors (e.g., thermocouples), blood pressure sensors, blood glucose sensors, blood oxygen sensors, pH sensors, accelerometers, and microphones.

[0057] As described in more detail herein, the monitoring and processing device 202 may implement a scar tissue identifier to receive at least images, data, etc. (e.g., instances thereof may generally be referred to as modalities) from single and / or multiple patients. The modalities can be one or more types such as ultrasonic type, computed tomography (CT) type, MRI type, or other medical image / scan types. For example, the first modality can be of the first type, and the second modality can be of a second type different from the first type. The monitoring and processing device 202 may implement a scar tissue identifier, cross-reference the first modality and the second modality, and generate improved image data of the first modality based on the cross-reference. Note that the improved image data includes higher accuracy or higher resolution than the original data of the first modality.

[0058] In other examples, the monitoring and processing device 202 can be an ECG monitor for monitoring the ECG signals of the heart (e.g., the heart 120 of FIG. 1). In this regard, the patient biosensor 212 of the ECG monitor can include one or more electrodes (e.g., the electrodes of the catheter 105 of FIG. 1) for acquiring the ECG signals. The ECG signals can be used for the treatment of various cardiovascular diseases.

[0059] In another example, the monitoring and processing device 202 can be a continuous glucose monitor (CGM) for continuously monitoring the blood glucose level of a patient on a continuous basis for treating various diseases such as type I and type II diabetes. In this regard, the patient biosensor 212 of the CGM can include a subcutaneous electrode (e.g., the electrode of the catheter 105 of FIG. 1), which can monitor the blood glucose level from the interstitial fluid of the patient. The CGM can be a component of a closed-loop system, for example, where blood glucose data is sent to an insulin pump for calculated insulin delivery without user intervention.

[0060] The processor 214 can be configured to receive, process, and manage the biological data acquired by the patient biosensor 212, and transmit that biological data to the memory 218 for storage and / or throughout the network 210 via the transceiver 222. As described in more detail herein, data from one or more other monitoring and processing devices 202 can also be received by the processor 214 through the transceiver 222. Also, as described in more detail herein, the processor 214 can be configured to selectively respond to different tapping patterns (e.g., single tap or double tap) received from the UI sensor 216 (e.g., an internal capacitance sensor), such that different tasks of the patch (e.g., data acquisition, storage, or transmission) can be initiated based on the detected pattern. In some embodiments, the processor 214 can generate audible feedback regarding gesture detection.

[0061] The UI sensor 216 includes, for example, a piezoelectric sensor or a capacitance sensor configured to receive user inputs such as taps or touches. For example, the UI sensor 216 can be controlled to implement capacitive coupling in response to the patient 204 tapping or touching the surface of the monitoring and processing device 202. Gesture recognition can be implemented via any one of various capacitive types such as resistive capacitive, surface capacitive, projected capacitive, surface acoustic wave, piezoelectric, and infrared touch. The capacitance sensor may be arranged over a small area or length of the surface such that a tap or touch on the surface activates the monitoring device.

[0062] The memory 218 is any non - transitory tangible medium such as magnetic, optical, or electronic memory (e.g., any suitable volatile and / or non - volatile memory such as random access memory or hard disk drive).

[0063] The transceiver 222 can include a separate transmitter and a separate receiver. Alternatively, the transceiver 222 can include a transmitter and a receiver integrated into a single device.

[0064] According to an embodiment, the monitoring and processing device 202 can be a device that is inside the body of the patient 204 (e.g., implantable subcutaneously). The monitoring and processing device 202 can be inserted into the patient 204 via any applicable method including oral infusion, surgical insertion via vein or artery, endoscopic procedure, or laparoscopic procedure.

[0065] According to an embodiment, the monitoring and processing device 202 can be a device that is external to the patient 204. For example, as described in more detail herein, the monitoring and processing device 202 can include an attachable patch (e.g., attachable to the patient's skin). The monitoring and processing device 202 can also include a catheter, a probe, a blood pressure cuff, a weighing scale, a bracelet or a smartwatch biometric tracker, a glucose monitor, a continuous positive airway pressure (CPAP) device, or substantially any device that can provide inputs regarding the patient's health or biometric metrics.

[0066] According to one embodiment, the monitoring and processing device 202 may include both components inside the patient and components outside the patient.

[0067] Although a single monitoring and processing device 202 is shown in FIG. 2, an exemplary system can include a plurality of patient biometric information monitoring and processing devices. For example, the monitoring and processing device 202 may communicate with the biometric information monitoring and processing devices of one or more other patients. Additionally, or alternatively, the biometric information monitoring and processing devices of one or more other patients can communicate with the network 210 and other components of the system 200.

[0068] The local computing device 206 and / or the remote computing system 208, together with the monitoring and processing device 202, can be any combination of software and / or hardware that stores, executes, and implements the scar tissue identifier and its functions individually or collectively. Further, as described herein, the local computing device 206 and / or the remote computing system 208, together with the monitoring and processing device 202, can be an electronic computer framework that encompasses and / or uses any number and combination of computing devices and networks that utilize various communication technologies. The local computing device 206 and / or the remote computing system 208, together with the monitoring and processing device 202, can be easily scalable, extensible, and modular, and have the ability to change to different services or reconfigure some features independently of other features.

[0069] According to one embodiment, the local computing device 206 and the remote computing system 208, together with the monitoring and processing device 202, include at least a processor and a memory, the processor executes computer instructions regarding the scar tissue identifier, and the memory stores those computer instructions executed by the processor.

[0070] The local computing device 206 of the system 200 can communicate with the monitoring and processing device 202 and is configured to serve as a gateway to the remote computing system 208 through the second network 211. The local computing device 206 can be, for example, a smartphone, smartwatch, tablet, or other portable smart device configured to communicate with other devices via the network 211. Alternatively, the local computing device 206 can be, for example, a fixed base station including modem and / or router capabilities, a desktop computer or laptop computer that uses an executable program to communicate information between the processing device 202 and the remote computing system 208 via a wireless module of the PC, or a fixed or stand-alone device such as a USB dongle. The biological data can be communicated between the local computing device 206 and the monitoring and processing device 202 using short-range wireless technology standard specifications (such as Bluetooth, Wi-Fi, ZigBee, Z-wave, and other short-range wireless standard specifications) via a short-range wireless network 210 such as a local area network (LAN) (for example, a personal area network (PAN)). In some embodiments, as described in more detail herein, the local computing device 206 can also be configured to display the acquired patient electrical signals and information associated with the acquired patient electrical signals.

[0071] In some embodiments, the remote computing system 208 can be configured to receive at least one of the biometric information of the monitored patient and the information associated with the monitored patient via the network 211, which is a long-distance network. For example, if the local computing device 206 is a mobile phone, the network 211 may be a wireless cellular network, and the information can be communicated between the local computing device 206 and the remote computing system 208 via a wireless technology standard such as any of the above wireless technologies. As will be described in more detail herein, the remote computing system 208 can be configured to provide (e.g., visually display and / or provide audibly) at least one of the patient's biometric information and its related information to medical experts, physicians, health management experts, etc.

[0072] In FIG. 2, the network 210 is an example of a short-distance network (e.g., a local area network (LAN) or a personal area network (PAN)). Information can be transmitted between the monitoring and processing device 202 and the local computing device 206 via the short-distance network 210 using any one of various short-distance wireless communication protocols such as Bluetooth, Wi-Fi, ZigBee, Z-wave, near field communication (NFC), ultra-wideband wireless, Zigbee, or infrared (IR).

[0073] Network 211 may be a wired network, a wireless network, or one or more wired and wireless networks including an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or a series of connections, a cellular phone network, or any other network or medium capable of facilitating communication between the local computing device 206 and the remote computing system 208. Information may be transmitted via network 211 using any one of a variety of long-range wireless communication protocols (e.g., TCP / IP, HTTP, 3G, 4G / LTE, or 5G / New Radio). The wired connection can be implemented using Ethernet, Universal Serial Bus (USB), RJ-11, or any other wired connection commonly known in the art. The wireless connection can be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellites, or any other wireless connection technique. Further, some networks may operate alone or communicate with each other to facilitate communication within network 211. In some cases, the remote computing system 208 can be implemented as a physical server on network 211. In other cases, the remote computing system 208 can be implemented as a virtual server of a public cloud computing provider of network 211 (e.g., Amazon Web Services (AWS) (registered trademark)).

[0074] FIG. 3 shows an illustration of an artificial intelligence system 300 according to one or more embodiments. This artificial intelligence system 300 includes data 310, a machine 320, a model 330, a plurality of outcomes 340, and basic hardware 350. FIG. 4 shows a block diagram of a method 400 executed in the artificial intelligence system of FIG. 3. The descriptions of FIGS. 3 and 4 are made with reference to FIG. 2 for ease of understanding.

[0075] Generally, the artificial intelligence system 300 operates the method 400 by using data 310 for training a machine 320 (e.g., the local computing device 206 of FIG. 2) while building a model 330 for enabling a plurality of (predicted) outcomes 340. In such a configuration, the artificial intelligence system 300 can operate on the hardware 350 (e.g., the monitoring and processing device 202 of FIG. 2) to train the machine 320, build the model 330, and use algorithms to predict outcomes. Using these algorithms, the trained model 330 can be solved to predict the outcomes 340 associated with the hardware 350. These algorithms can generally be classified into classification algorithms, regression algorithms, and clustering algorithms.

[0076] In block 410, the method 400 includes collecting data 310 from the hardware 350. The machine 320 operates as a controller or data collection associated with the hardware 350 and / or is associated with that hardware. The data 310 (e.g., biometric data that may be derived from the monitoring and processing device 202 of FIG. 2) can be related to the hardware 350. For example, the data 310 may be ongoing data or output data associated with the hardware 350. The data 310 can also include currently collected data, historical data, or other data from the hardware 350. For example, the data 310 may include measurements during a surgical procedure and may be associated with the outcome of the surgical procedure. For example, the temperature of the heart (e.g., of patient 204) or the dimensions of the heart cavity may be collected and correlated with the outcome of a heart procedure.

[0077] In block 420, the method 400 includes training the machine 320 with respect to hardware 350, etc. This training can include the analysis and correlation of the data 310 collected in block 410. For example, in the case of the heart, the temperature and outcome data 310 can be trained to determine whether there is a correlation or association between the temperature of the heart (e.g., of patient 204) during a heart procedure and its outcome.

[0078] In block 430, the method 400 includes constructing the model 330 on the data 310 associated with the hardware 350. Constructing the model 330 can include modeling physical hardware or software, modeling algorithms, and / or the like. This modeling may aim to represent the collected and trained data 310. According to one embodiment, the model 330 can be configured to model the operation of the hardware 350, model the data 310 collected from the hardware 350, and predict the outcome achieved by the hardware 350. According to one or more embodiments, the model 330 receives a first modality and a second modality with respect to a scar tissue identifier, cross-references the first modality and the second modality, and generates improved image data of the first modality based on the cross-reference.

[0079] In block 440, the method 400 includes predicting a plurality of outcomes 340 of the model 330 associated with the hardware 350. This prediction for the plurality of outcomes 340 can be based on the trained model 330. For example, for purposes of enhancing the understanding of the present disclosure, in the case of the heart, if the temperature during treatment is between 36.5°C and 37.89°C (i.e., 97.7°F to 100.2°F) and results in a more positive outcome from the heart treatment, the outcome can be predicted in a given treatment based on the temperature of the heart during the heart treatment. Thus, using the predicted outcomes 340, the hardware 350 can be configured to provide a particular desired outcome 340 from the hardware 350.

[0080] Referring now to FIG. 5, a block diagram of a method 500 for automatically identifying a scar region within an organic tissue based on a plurality of imaging modalities is shown in accordance with one or more embodiments. According to one embodiment, the method 500 is implemented by a system for identifying scar tissue. The system for identifying scar tissue may be embodied as a machine learning algorithm that cross-references different modalities to enhance the discovery of a single modality. Any combination of software and / or hardware (e.g., local computing device 206 and remote computing system 208, as well as monitoring and processing device 202) can store, execute, and implement the system for identifying scar tissue and its functions individually or collectively. Generally, the system for identifying scar tissue cross-references different modalities to enhance the discovery of a single modality so as to be able to identify a scar region within an organic tissue. For example, the system for identifying scar tissue can be outcome-directed in that machine learning software / hardware performs comparisons, cross-references, and subsequent modality identifications based on previous modality comparison, cross-reference, and identification operations. Machine learning software / hardware can include, but is not limited to, neural networks, artificial neural networks, complex neural networks, encoders, decoders, tree structures, and autoencoders.

[0081] The process flow 500 begins at block 510, where a system for identifying scar tissue receives a first modality and a second modality. According to one or more embodiments, the first modality represents one or more first modalities, each of which is of a first type. For example, the first modality can be an image or data underlying an image. In some cases, the data of the first modality can be considered the original data. The second modality is of a second type that is different from the first type. Similarly, for example, the second modality can be an image or data underlying an image. The first type can be an ultrasound image or its data, while the second modality of the second type can be an LGE-MRI image or its data. A system for identifying scar tissue can receive the first modality and / or the second modality in real time from a medical device instrument, or can access the first modality and / or the second modality in memory. The reception can be automatic or user-initiated.

[0082] At block 520, a system for identifying scar tissue cross-references the first modality and the second modality. To cross-reference the first modality and the second modality, a system for identifying scar tissue compares the first modality with the second modality. In this regard, a system for identifying scar tissue aligns the location of the first modality with the corresponding location of the first modality. For example, ultrasound data can be mapped to LGE-MRI data according to voltage. According to one or more embodiments, the cross-reference can be mutual between modalities. That is, the first modality can improve the second modality, and the second modality can improve the first modality.

[0083] In block 530, a system for identifying scar tissue generates improved image data of a first modality based on cross-reference. That is, once mapped, the system for identifying scar tissue adjusts and interprets the original data of the first modality based on a second modality and generates improved image data of the first modality. Thus, once adjusted and interpreted, the improved image data is more accurate and includes a higher resolution than the original data without relying on the subjective interpretation of a human operator (such as in conventional imaging modalities). According to one or more embodiments, a hybrid modality can be generated / created that utilizes the first modality and the second modality and the reciprocal nature of the cross-reference operation of block 520. Since the hybrid modality includes a combination of two or more modalities of two or more types (e.g., a combination of improved data), the reciprocal improvement that each modality type can provide to the other modality is incorporated. The first modality and the second modality can be patient-specific, and subsequently, over time, subsequent modalities and / or secondary data (height, weight, heart disease, etc.) can be used by process 500 and contribute to the combined improved data. Further, over time, patient-specific data, modalities, and / or secondary data can be used by process 500 and contribute to the combined improved data for other patients.

[0084] FIG. 6 shows a block diagram of a method 600 for automatically identifying scar regions within an organic tissue based on a plurality of imaging modalities according to one or more embodiments. According to one embodiment, method 600 is implemented by a system for identifying scar tissue. Any combination of software and / or hardware (e.g., local computing device 206 and remote computing system 208, as well as monitoring and processing device 202) can store, execute, and implement the system for identifying scar tissue and its functions individually or collectively. Generally, the system for identifying scar tissue cross-references different modalities to enhance the discovery of a single modality. For example, with respect to a patient's heart, by enhancing the discovery of a single modality, the system for identifying scar tissue can model existing scars and other hard-to-reach regions that cannot be detected by a catheter. This modeling aids in the diagnosis of heart disease as described herein.

[0085] Method 600 begins at block 610 where a system for identifying scar tissue receives or accesses a first modality. According to one or more embodiments, the system for identifying scar tissue receives or accesses a plurality of modalities of a first type. For example, the data can be obtained from a particular type of sensor from a patient (e.g., catheter, electrode, ultrasound transducer, medical database, etc.). The first modality can be one of, or a combination of, ultrasonic data of a patient's heart, MRI data, and cardo data (e.g., heart map). According to one or more embodiments, the first modality of the first type includes ultrasonic data. The data can be stored and / or organized in any data structure (e.g., table, tree, matrix, etc.) suitable for how the data is obtained and how it functions.

[0086] In block 620, the system for identifying scar tissue receives or accesses a second modality. According to one or more embodiments, the system for identifying scar tissue receives or accesses a plurality of modalities of a second type with respect to the patient's heart (e.g., the data can be obtained from a database within the local computing device 206 or the remote computing system 208 of FIG. 2). The second modality may be of a second type different from the first type. According to one or more embodiments, the second modality of the second type includes LGE-MRI data.

[0087] In block 630, the system for identifying scar tissue accesses a third modality. According to one or more embodiments, the system for identifying scar tissue receives or accesses a plurality of modalities of a third type with respect to the patient's heart (e.g., the data can be obtained from a database within the local computing device 206 or the remote computing system 208 of FIG. 2). The third modality may be of a third type different from the first type and the second type. According to one or more embodiments, the third modality of the third type includes a heart map.

[0088] In block 640, a system for identifying scar tissue cross-references a first modality with a third modality and a second modality. The cross-reference includes comparing the first modality with the third modality and comparing the second modality with corresponding locations, thereby providing redundancy in a specific region of the heart. In this way, since the data correlates between modalities, the second and third modalities complement the first modality and provide additional data for hard-to-reach regions. For example, if ultrasound data is considered low-resolution baseline data, by correlating the ultrasound data with LGE-MRI data and a heart map, a system for identifying scar tissue creates a larger dataset for diagnosis. According to one or more embodiments, the cross-reference may be mutual between modalities. That is, the first modality, the second modality, and the third modality can improve each other. Further, as the method 600 progresses through multiple iterations (as indicated by the dashed arrows from blocks 650 and 660 to 640), the cross-reference can utilize the improved modalities of block 650 and the database of block 660 to perform machine learning.

[0089] In block 650, a system for identifying scar tissue generates a plurality of improved modalities (e.g., corresponding data structures for storing the generated data). In this regard, each original data of the first modality is improved based on the cross-reference. For example, a system for identifying scar tissue adjusts and interprets each original data of the first modality based on the second and third modalities to generate improved image data. The plurality of improved modalities includes improved image data having higher accuracy and / or higher resolution compared to the original data of the first modality. A system for identifying scar tissue can generate corresponding improved images from the improved image data. According to one or more embodiments, the first modality, the second modality, and the third modality may be utilized to generate / create a hybrid modality by taking advantage of the mutual nature of the cross-reference operation of block 640.

[0090] According to one or more embodiments, weights (e.g., alphanumeric values) can be assigned to the original data of the modality and / or the improved image data (e.g., associated within a data structure). Specifically, the maximum weight can be attributable to the actual identification of scar tissue. Specifically, in one embodiment, a plurality of weights can be used, including a first weight for indicating the severity of scar tissue and a second weight for providing the likelihood of the first weight. Collectively, the improved data is "improved" because the data collects benefits from each underlying modality (e.g., ultrasound data enables analysis of tissue depth, MRI data adds executability, and cardiac mapping generates electrocardiogram data).

[0091] In block 660, the database is constructed using the improved image data and / or improved images (including any hybrid modalities and / or weights). The improved image data can be used to create a new database as part of a training process or added to a database storing the data of the modality. According to one or more embodiments, the database can reside in a computing system (e.g., within the local computing device 206 or the remote computing system 208 of FIG. 2). The database may further include additional information from other patients, such as data collected from physicians and / or medical professionals, to support functionality / optimization.

[0092] Next, in block 670, the database and the improved image data and / or the improved images therein are provided to the physician and / or medical expert. Next, the physician and / or medical expert can utilize the improved image data and / or the improved images for one or more of ablation ultrasound techniques for treating the medical condition, planning and diagnosing the damaged area, and magnetic resonance evaluation and diagnosis. Specifically, the improved image data and / or the improved images identify scar tissue that is not otherwise identified by conventional imaging modalities, and thus the diagnostic ability is enhanced by the improved image data and / or the improved images.

[0093] The system and method generate a 3D model of the cardiac tissue including a scar region having a likelihood score based on an input including a 3D shell and at least one imaging modality. The at least one imaging modality can include one or more of a CT, MR, or ULS image set. Additional inputs can include demographic data, electroanatomical information from a 3D mapping system, and intracardiac and body surface related information disclosed in more detail below. The system and method output 3D voxels with a scar level score and a reliability of the score associated with each voxel.

[0094] FIG. 7 shows the data preparation and training of the system. Data from previous patient cases 710 are used to train the system for each such case. Data from previous patient cases 710 can include 3D mapping 735 (described in detail herein) and patient parameter 725 data (described in detail herein) such as age, gender, medical history, and 3D mapping based on catheter 715. Data from previous clinical cases 710 including 3D mapping based on catheter 715 and patient parameter 725 data may be used as system input 720 to the system described herein, and the data is used for the calculation of the output, thereby training the system in training step 750.

[0095] Data from previous clinical cases 710 can include previous clinical cases relevant to training the system and algorithms. For each such case, the available data should include the inputs described herein, as well as data describing the patient ID 725 and the output.

[0096] The data 735 used in the calculation of the desired output 740 can include a 3D model of the heart tissue, where each voxel shows the scar density at that voxel based on data from LGE-MRI readings determined by an expert or an automated tool. The data 735 used in the calculation of the desired output 740 can include a 3D model of the heart tissue, where each voxel shows the radiopacity and tissue elasticity at that voxel based on data from ultrasound measurements determined by an expert or an automated tool. The data 735 used in the calculation of the desired output 740 can include a 3D model of the heart tissue, where each voxel shows the radiopacity and tissue elasticity at that voxel based on data from delayed enhancement CT readings determined by an expert or an automated tool.

[0097] The preprocessing 730 occurs in the 3D mapping 735 and can match the desired output format. The data 735 is preprocessed to conform to the desired output 740 format, where each voxel has a value between 0 and 1, indicating the scar (1 = most severe, non-conductive, 0 = no scar) density and a confidence score. The calculation of the confidence score can depend on manual scoring by an expert in the 3D tissue model based on the ULS / MRI / CT mapping 735, and / or automated scoring by specifying a ULS / MRI / CT threshold-radiopacity scale.

[0098] According to one embodiment, data voxels with a trust score exceeding 90% can be used while excluding or weighting lower trust scores. Further, in one embodiment, data 710 is expected to include at least 85% of the voxels having a trust score exceeding 90% that should be included in data 710. Other scores with low trust scores may be weighted or excluded. In one embodiment, the trust threshold is selected by a physician during the operation.

[0099] After the preprocessing 730 is completed in the 3D mapping 735, a desired output 740 may be included and provided to train the neural network in step 750.

[0100] When the training 750 is performed, the neural network is trained at 760. The dataset may be divided into a training set, a validation set, and a test set. The training set and the validation set may be used during system development, and the test set is used to evaluate the accuracy of the system. Cross-validation can be used to improve performance.

[0101] Figures 8 and 9 show an exemplary combined architecture of the present system. The mapping can be input to the system at step 810 as described herein.

[0102] The registration of the mapping can occur at the registration stage 820. A system that can learn patterns and correctly generalize the data given from the registration of various patients is required. The mutual alignment of various inputs can be calculated, which is called "registration". The registration at the registration stage 820 can occur in various dimensions including intra-patient alignment and inter-patient alignment.

[0103] According to one embodiment, in the case of patient - specific alignment, when 3D mapping is created from catheters on both the epicardial and endocardial surfaces of the cardiac chamber, registration may be specific to the functionality of the 3D mapping system. For example, in CARTO® 3, the maps collected during the case are registered. Other images such as MRI, CT, ULS may need to be registered to the 3D catheter map and other such images. For example, when a ULS image is collected by a navigated catheter, the ULS image can be automatically registered to the 3D map. Other registration techniques already described in the literature can also be used.

[0104] According to one embodiment, for patient - to - patient alignment, the data may be normalized to a "standard model" of the cardiac chamber. Various mappings are "projected" onto this standard model. The resulting projections are then provided to the map merge stage 830.

[0105] Map merge 830 may occur and the output of the mapping of the passing electrical current may occur at step 840. Map merge 830 is used to calculate the desired output and there are many possible neural architectures as described herein. Some of those architectures are described above herein and some detailed architectures are further described below. Depending on the nature of the data, a more complex architecture may be desirable.

[0106] As discussed above with respect to the registration stage 820, the input to the map merge stage 830 is the output of the registration stage 820 that aligns the 3D input mapping to a "normalized" 3D mapping. The input to the map merge stage 830 includes N mappings, and each mapping is a 3D "volume image" of size HxWxD with voxels (3D "pixels"). Each voxel can have a plurality of associated values including, for example, voltage, elasticity, scarring by MR / CT, wall motion.

[0107] According to one embodiment, the map merge stage 830 combines the input mappings using several linear combinations. One neural layer of size HxWxD can be used where the values at each position (i, j, k) of the N volumetric images are supplied to the neuron (i, j, k). This embodiment of the map merge stage 830 can be used when each voxel within each input map presents, by itself, some reliable indicator of scar tissue and a simple linear combination (or averaging) of the input maps is sufficient for the integration. Such an embodiment of the map merge stage 830 can be beneficial as being easier to train with less data compared to more complex models. Such an embodiment of the map merge stage 830 may provide results that are not accurate enough.

[0108] To increase accuracy and increase the benefits of map merge 830, additional layers, i.e., deeper networks, can be used. The additional layers can provide the ability to represent more complex functions. Larger layers can be used. Larger layers enable the ability to capture more nuances in the data. Individual processing can also be used. Processing each input map separately before combining the information provides additional advantages. Additionally, or alternatively, different types of layer architectures, such as fully connected convolutional neural networks (CNNs), max pooling, etc., can be used as described herein.

[0109] Those techniques for increasing accuracy and increasing the benefits of the map merge stage 830 are described below. The map merge stage 830 can be enhanced by utilizing two layers including a first ("hidden") layer of size HxWxDxk (i.e., a 3D layer HxWxD with "thickness" k>1) and a second ("output") layer of size HxWxD. For each of the k neurons at position (i, j, k) in the first layer, N signals are supplied from the position (i, j, k) from each of the N input maps and its output is given to the neuron (i, j, k) of the second layer. A non-linear combination of the N values for each voxel can be represented.

[0110] When considering voxels within a given three-dimensional region as potential candidates for scar tissue, the map merging stage 830 may be enhanced, and such a determination may depend on the input mapping values of the three-dimensional region surrounding the candidate voxels rather than the entire mapping. In the calculation, standard techniques of deep learning using a CNN may be utilized to obtain the benefit of the information of this surrounding region.

[0111] The map merging stage 830 may be enhanced, for example, by providing that each of the input N images may have its own convolutional model, i.e., each input image is supplied to a separate CNN. Each convolutional filter of the CNN layer may utilize a small cube of voxels as input and output one value. The output of each CNN is a three-dimensional image that provides a preliminary output recommendation map based on only one input mapping.

[0112] Additional layers may be applied separately to one of the N map "tracks" according to deep learning for image / region processing using a CNN, max pooling, and other standard paradigms known to those skilled in the art as discussed in this specification to enhance the map merging stage 830.

[0113] The combination layer may be utilized to receive the input map and / or the output of a previous separate processing layer as described for combining the data into one representation. The combination may be performed using a simple linear combination as described in this specification, or a more complex combination, i.e., a non-linear combination, and / or using a CNN, max pooling, and other standard layers of image processing.

[0114] Figure 9 shows a possible architecture of the map merging stage 830. To clarify the graphics, the architecture shows an NN for processing 2D images, but this idea can be applied equally to 3D volume images.

[0115] The left input consists of a 2D mapping with N = 8 (each representing the output of registration 820 of input 810. Each such mapping is processed individually by a different CNN grid in layer 1 910. The output of layer 1 910 includes a complex mapping with N = 8 that is supplied to layer 2 920. Next, those mappings are supplied to combination layer 3 930 as described herein and above as a combination layer having a depth k = 5 to enable the non-linear combination also described above. The k combined layers are merged into an output map by layer 4 940. The values N = 8 and k = 5 here are merely exemplary values, and those skilled in the art will understand that other values may be used. Depending on the nature of the data and the required accuracy of the output, additional layers or fewer layers can be used compared to what is illustrated.

[0116] Patient parameters such as age, gender, medication treatment, medical history, and type of atrial fibrillation can affect the results. Separate models can be trained based on some of the input patient parameters. Alternatively, the patient input may be provided to layers within the NN architecture to help learn the differences based on these parameters.

[0117] The hearts of different patients can vary, and there may be variations in the recorded data. According to one embodiment, the system may need to be trained in batches, and each batch may be limited to the data of a single patient. The data needs to be collected from at least a certain number of patients to strengthen the training of the system.

[0118] Even after the system is prepared and deployed in a hospital, additional data may be accumulated. To continuously improve the accuracy, additional data may be added to the training dataset and the system may be retrained. Specifically, data from additional surgeries can provide feedback by considering the success rate of the ablation performed according to the system's recommendations.

[0119] Figure 10 shows a system 1000 that uses a neural network according to the embodiments described herein. System 1000 includes patient data 725, additional data 1010, and 3D mapping 735 including MRI / CT, ULS, and other 3D mapping based on, for example, catheter 715. This input data 710 can include 3D mapping and imaging from several algorithms that analyze readings from the patient. This data 710 can include data for each electrode / channel with locations in space over a period of time (3 axes to 6 axes - X, Y, Z + pitch, roll, yaw), and intracardiac electrocardiogram over a period of time (e.g., 2500 milliseconds). This data 710 can include 3D reconstructions, which can be in mesh form, and data for each cardiac chamber including at least one imaging modality 735 (CT, MR, or set of ultrasonic fans). Patient data 725 can include demographic information, age, gender, weight, height, body mass index, ethnicity, and other patient-specific details including left atrial major axis length (width, height, length), left ventricular ejection fraction, hypertension, and type 2 diabetes. Other underlying co-morbidities including sleep apnea, coronary artery disease, valvular heart disease (such as mitral regurgitation), and congestive heart disease can be included in the additional data 1010. The patient's medical history can be included in the patient data 725 and can include history of arrhythmia, symptoms, and documented methods, time since first diagnosis, medication history of anti-arrhythmic drugs (AADs), previous cardiac ablation, anticoagulation therapy, CHA2DS2-VASc score, history of thrombotic disease, New York Heart Association (NYHA) grade of cardiac function, history of bleeding disorders, HAS-BLED, respiratory pattern, ventricular cycle length, and atrial cycle length. Data for each electrode / channel 715 and the analysis performed on this data (e.g., derivative, algorithm calculation) can include local activation time, impedance over a period of time, change in impedance over time, rate of change of location (derivative of position over time), maximum peak-to-peak voltage, unipolar and bipolar measurements from a distant region (neither that point nor just the immediate surroundings), and start and end times of the period during which the catheter was placed at that point. Since various points of the 3D mapping are calculated based on measurements taken at various times, such time-tagged data can provide insights.When a particular point value is not obtained from catheter access at that point but is triangulated by an algorithm, the triangulation at the relevant time point can be used. Data for each heart chamber (which may be an epicardial map and / or an endocardial map) includes wall motion from ULS, Doppler from ULS, scar zones from MRI, chamber dimensions, cycle length maps, persistent atrial fibrillation focus source maps (e.g., CARTOFINDER, or equivalent), persistent atrial fibrillation rotation source maps (e.g., CARTOFINDER, or equivalent), reentry / fibrillation activation mapping (such as coherent or equivalent), ripple maps, CFAE, ECG fractionation, and 3D models of heart tissue. Each voxel indicates the tissue elasticity at that voxel based on data from ultrasonic measurements.

[0120] Output 840 is a set of voxels representing heart tissue, and each voxel has two values including a level of scar and a confidence value associated with the level of scar. The level of scar includes a numerical value from 0 to 1 where 1 = most severe, non-conductive, a boundary zone less than 1 and greater than 0, 0 = no scar. The confidence value can be provided between 0 and 1.

[0121] Between input 710 and output 840, 3D mapping can be registered via registration stage 820 as described above. The registered 3D mapping 735 can be input into neural network 830 along with patient data and any additional data 1010 as described above. Neural network 830 can include an input layer 910, hidden layers such as the layers described as layers 920, 930, and an output layer 940 for providing output 840 as described above with respect to FIGS. 8 and 9.

[0122] Similar to the neural network of FIG. 10, the system 1100 of FIG. 11 utilizes a neural network according to the embodiments described herein. The system 1100 includes a patient 725, additional data 1010, and 3D mapping 735 including MRI / CT, ULS, and other 3D mapping such as based on catheter 715. This input data 710 is the data input into other embodiments including the system 1000.

[0123] Between the input 710 and the output 840, the 3D mapping can be registered via the registration stage 820 as described above. The registered 3D mapping 735 can be input into the neural network 830 along with the patient data and any additional data 1010 as described above. The neural network 830 can include a first convolution and pooling 1110, followed by a second convolution and pooling 1120. The output from the convolutions and poolings 1110, 1120 can be reshaped using the reshaping 1150 and the data input reshaped in series to the dense layer 1130 and the dense output layer 1140. When the data is classified, the data reaches an output layer similar to the output layer 940. The output layer provides the output 840.

[0124] As described herein, the output 840 is a set of voxels representing heart tissue, and each voxel has two values including a level of scar and a confidence value related to the level of scar. The level of scar includes a numerical value from 0 to 1 where 1 = most severe, non-conductive, a boundary zone less than 1 and greater than 0, 0 = no scar. The confidence value can be provided between 0 and 1.

[0125] According to one or more embodiments, the technical effects and advantages of the system and method for identifying scar tissue include generating more accurate and high-resolution real-time image data of ultrasonic imaging without relying on the subjective interpretation of a human operator (e.g., the generated real-time image data enables evaluation of the entire depth of organic tissue, overcomes the low resolution of conventional ultrasonic images, and is relatively inexpensive and available compared to MRI).

[0126] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible embodiments of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions, which contains one or more executable instructions for implementing a particular logical function. In some alternative embodiments, the functions described within the block may occur out of the order described in the figures. For example, two blocks shown in succession may, in fact, be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, as well as combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs a particular function or operation, or may be implemented by a combination of dedicated hardware and computer instructions that operate or execute.

[0127] Although features and elements are described above in particular combinations, those skilled in the art will understand that each feature or element can be used alone or in combination with other features and elements. Additionally, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. The computer-readable medium as used herein should not be construed to be a signal per se that is transient, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse passing through an optical fiber cable), or an electrical signal transmitted through a bus.

[0128] Examples of computer-readable media include electrical signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, magnetic media such as registers, cache memories, semiconductor memory devices, internal hard disks, and removable disks, magneto-optical media, optical media such as compact discs (CDs) and digital versatile discs (DVDs), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), and memory sticks. A processor can be used with software to implement a radio frequency transceiver for use in a terminal, a base station, or any host computer.

[0129] The terms used in this specification are for the purpose of describing particular embodiments only and are not intended to be limiting. As used in this specification, the singular forms "a", "an", and "the" include the plural forms as well, unless the context clearly dictates otherwise. The terms "comprise" and / or "comprising", as used in this specification, indicate the presence of the recited features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0130] The descriptions of the different embodiments in this specification are presented for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used in this specification are chosen in order to best explain the principles of the embodiments, the practical application, or a technical improvement found in the marketplace as compared to the technology, or to enable others skilled in the art to understand the embodiments disclosed in this specification.

[0131] 〔Embodiment〕 (1) A method comprising: receiving a first modality and a second modality by a scar tissue identifier executed by a processor coupled to a memory, wherein the first modality is of a first type and the second modality is of a second type different from the first type, and each of the first modality and the second modality indicates the patient's organic tissue according to the first type and the second type, respectively; cross-referencing the first modality and the second modality by the scar tissue identifier; generating, by the scar tissue identifier, improved image data of the first modality based on the cross-reference, wherein the improved image data includes higher accuracy or higher resolution than the original data of the first modality. (2) The method according to embodiment 1, wherein the scar tissue identifier generates a hybrid modality using mutual improvement of the first modality and the second modality. (3) The method according to embodiment 1, wherein the scar tissue identifier is configured to automatically identify a scar region within the organic tissue using the improved image data. (4) The method according to embodiment 1, wherein the scar tissue identifier is configured to generate the improved image data by adjusting and interpreting the first modality based on the second modality using imaging processing. (5) The method according to embodiment 1, wherein the first modality of the first type includes an ultrasonic image.

[0132] (6) The method according to embodiment 1, wherein the second modality of the second type includes gadolinium-enhanced magnetic resonance image data. (7) The method according to embodiment 1, wherein the scar tissue identifier performs machine learning for subsequent cross-reference operations using the improved image data. (8) Referring the first modality and the second modality to each other includes comparing the first modality with the second modality and aligning the location of the first modality with the corresponding location of the first modality, the method according to Embodiment 1. (9) The scar tissue identifier receives a plurality of modalities of the first type and cross-references them with the second modality, the method according to Embodiment 1. (10) The scar tissue identifier generates corresponding improved image data for each of the plurality of modalities based on the cross-reference, the method according to Embodiment 9.

[0133] (11) The scar tissue identifier is configured to receive a third modality of a third type different from the first type and the second type, the method according to Embodiment 1. (12) The scar tissue identifier cross-references the first modality with the second modality and the third modality, and is configured to generate the improved image data based on the cross-reference, the method according to Embodiment 11. (13) The third modality of the third type includes a cardiac map, the method according to Embodiment 11. (14) The scar tissue identifier is incorporated into one or more of ultrasonic technology, planning and diagnosis of the damaged area, ECG measurement, 3D mapping system, and magnetic resonance evaluation and diagnosis to address the medical condition, the method according to Embodiment 1. (15) The medical condition includes atrial fibrillation, atrial flutter, general electrophysiology, arrhythmia, ventricular fibrillation, or ventricular tachycardia, the method according to Embodiment 1.

[0134] (16) A system, a memory storing processor-executable instructions of a scar tissue identifier, a processor that executes the processor-executable instructions of the scar tissue identifier and causes the system to Receiving a first modality and a second modality, wherein the first modality is of a first type and the second modality is of a second type different from the first type, and each of the first modality and the second modality shows an organic tissue of a patient according to the first type and the second type, and receiving; Cross-referencing the first modality and the second modality; Generating improved image data of the first modality based on the cross-reference, wherein the improved image data includes higher accuracy or higher resolution than the original data of the first modality, and generating, a processor configured to perform; A system including. (17) The scar tissue identifier is the system according to embodiment 16, which generates a hybrid modality using mutual improvement of the first modality and the second modality. (18) The scar tissue identifier is the system according to embodiment 16, which is configured to automatically identify a scar region in the organic tissue using the improved image data. (19) The scar tissue identifier is the system according to embodiment 16, which is configured to generate the improved image data by adjusting and interpreting the first modality based on the second modality using imaging processing. (20) The first modality of the first type includes an ultrasonic image, the system according to embodiment 16.

[0135] (21) The second modality of the second type includes gadolinium delayed contrast - magnetic resonance image data, the system according to embodiment 16. (22) The scar tissue identifier is the system according to embodiment 16, which performs machine learning for subsequent cross-reference operations using the improved image data. (23) Referring the first modality and the second modality to each other includes comparing the first modality with the second modality and aligning the location of the first modality with the corresponding location of the first modality, according to the system of embodiment 16. (24) The scar tissue identifier receives a plurality of modalities of the first type and cross-references them with the second modality, according to the system of embodiment 16. (25) The scar tissue identifier generates corresponding improved image data for each of the plurality of modalities based on the cross-reference, according to the system of embodiment 24.

[0136] (26) The scar tissue identifier is configured to receive a third modality of a third type different from the first type and the second type, according to the system of embodiment 16. (27) The scar tissue identifier cross-references the first modality with the second modality and the third modality, and is configured to generate the improved image data based on the cross-reference, according to the system of embodiment 26. (28) The third modality of the third type includes a cardiac map, according to the system of embodiment 26. (29) The scar tissue identifier is incorporated into one or more of ultrasonic technology, planning and diagnosis of the damaged area, ECG, and 3D mapping, and evaluation and diagnosis of magnetic resonance to address the medical condition, according to the system of embodiment 16. (30) The medical condition includes atrial fibrillation, atrial flutter, general electrophysiology, arrhythmia, ventricular fibrillation, or ventricular tachycardia, according to the system of embodiment 16.

Claims

1. A system, a memory storing processor-executable instructions for a scar tissue identifier of cardiac imaging, a processor that executes the processor-executable instructions of the scar tissue identifier, and causes the system to, receive ultrasonic image data, gadolinium delayed contrast-enhanced magnetic resonance (LGE-MRI) image data, and cardiac map image data respectively indicating organic tissues of a patient, when the ultrasonic image data is regarded as low-resolution basic data, compare the ultrasonic image data with the LGE-MRI image data and the cardiac map image data, match the location of the ultrasonic image data with the corresponding locations of the LGE-MRI image data and the cardiac map image data, and cross-reference the ultrasonic image data, the LGE-MRI image data, and the cardiac map image data, generate improved image data of the ultrasonic image data based on the cross-reference, the improved image data including higher accuracy or higher resolution than the original data of the ultrasonic image data, and a processor configured to cause the system to automatically identify a scar region in the organic tissue through machine learning using the improved image data. A system comprising.

2. The system according to claim 1, wherein the scar tissue identifier is configured to generate the improved image data by adjusting and interpreting the ultrasonic image data based on the LGE-MRI image data and the cardiac map image data using imaging processing.

3. The system according to claim 1, wherein the scar tissue identifier performs the machine learning for subsequent cross-reference operations using the improved image data.

4. The scar tissue identifier is the system according to claim 1, which receives a plurality of the ultrasonic image data and cross-references the LGE-MRI image data and the image data of the heart map.

5. The scar tissue identifier is the system according to claim 4, which generates corresponding improved image data for each of the plurality of the ultrasonic image data based on the cross-reference.

6. The scar tissue identifier is incorporated into one or more of ultrasonic technology, planning and diagnosis of damaged parts, ECG, and 3D mapping, and evaluation and diagnosis of magnetic resonance in order to address the medical condition, in the system according to claim 1.

7. The medical condition includes atrial fibrillation, atrial flutter, general electrophysiology, arrhythmia, ventricular fibrillation, or ventricular tachycardia, in the system according to claim 1.

8. A method comprising: receiving ultrasonic image data, gadolinium delayed contrast-enhanced magnetic resonance (LGE-MRI) image data, and image data of a heart map, each indicating organic tissues of a patient, by a scar tissue identifier for cardiac imaging executed by a processor of a system coupled to a memory; when the ultrasonic image data is regarded as low-resolution basic data, comparing the ultrasonic image data with the LGE-MRI image data and the image data of the heart map, and matching the location of the ultrasonic image data with the corresponding locations of the LGE-MRI image data and the image data of the heart map to cross-reference the ultrasonic image data with the LGE-MRI image data and the image data of the heart map; generating, by the scar tissue identifier, improved image data of the ultrasonic image data based on the cross-reference, wherein the improved image data includes higher accuracy or higher resolution than the original data of the ultrasonic image data; A method comprising automatically identifying a scar region within the organic tissue through machine learning using the improved image data. **Claim 9** The method according to claim 8, wherein the scar tissue identifier is configured to generate the improved image data by adjusting and interpreting the ultrasonic image data based on the LGE-MRI image data and the image data of the heart map using an imaging process. **Claim 10** The method according to claim 8, wherein the scar tissue identifier performs the machine learning for subsequent cross-reference operations using the improved image data. **Claim 11** The method according to claim 8, wherein the scar tissue identifier receives a plurality of the ultrasonic image data and cross-references them with the LGE-MRI image data and the image data of the heart map. **Claim 12** The method according to claim 11, wherein the scar tissue identifier generates corresponding improved image data for each of the plurality of ultrasonic image data based on the cross-reference. **Claim 13** The method according to claim 8, wherein the scar tissue identifier is incorporated into one or more of ultrasonic technology, planning and diagnosis of the damaged part, ECG measurement, 3D mapping system, and magnetic resonance evaluation and diagnosis to address the medical condition. **Claim 14** The method according to claim 8, wherein the medical condition includes atrial fibrillation, atrial flutter, general electrophysiology, arrhythmia, ventricular fibrillation, or ventricular tachycardia.

Images