Aligned geometric reconstruction based on constant fluoroscopic snapshots

Abstract

To generate an anatomical map.SOLUTION: In one embodiment, a method for generating a three-dimensional (3D) anatomical map includes: applying a trained artificial neural network to (a) a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject, and (b) respective first 3D coordinates of the set of 2D fluoroscopic images, yielding second 3D coordinates of the 3D anatomical map; and rendering to a display the 3D anatomical map responsively to the second 3D coordinates.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 118,047, filed November 25, 2020, the disclosure of which is hereby incorporated by reference herein.

[0002] FIELD OF THE INVENTION FIELD OF THE INVENTION The present invention relates to medical devices, and more particularly, but not exclusively, to medical devices that generate anatomical maps. [Background technology]

[0003] A wide variety of medical procedures involve the placement of probes, such as catheters, within a patient's body. Position sensing systems have been developed to track such probes. Magnetic position sensing is one method known in the art. In magnetic position sensing, magnetic field generators are typically placed at known locations outside the patient's body. Magnetic field sensors within the distal end of the probe generate electrical signals in response to these magnetic fields, and these signals are processed to determine the coordinate position of the distal end of the probe. These methods and systems are described in U.S. Patent Nos. 5,391,199, 6,690,963, 6,484,118, 6,239,724, 6,618,612, and 6,332,089, WO 1996 / 005768, and U.S. Patent Application Publication Nos. 2002 / 0065455, 2003 / 0120150, and 2004 / 0068178. Position can also be tracked using impedance or current-based systems.

[0004] One medical procedure in which these types of probes or catheters have proven extremely useful is in the treatment of cardiac arrhythmias, which, and atrial fibrillation in particular, remain common and dangerous conditions, especially in the aging population.

[0005] Diagnosis and treatment of cardiac arrhythmias involve mapping the electrical properties of cardiac tissue, particularly the endocardium, and selectively ablating the cardiac tissue through the application of energy. Such ablation can stop or modify the propagation of unwanted electrical signals from one part of the heart to another. The ablation process disrupts unwanted electrical pathways by creating non-conductive lesions. Various energy delivery modalities have been previously disclosed for creating lesions, including the use of microwave, laser, and more commonly, radiofrequency energy to create conduction blocks along cardiac tissue walls. In a two-step mapping-then-ablation procedure, a catheter containing one or more electrical sensors is typically advanced into the heart to sense and measure electrical activity at each point within the heart by acquiring data at multiple points. These data are then used to select a target region of the endocardium for this ablation.

[0006] Electrode catheters have been commonly used in medical practice for many years. They are used to stimulate and map electrical activity within the heart and to ablate sites of abnormal electrical activity. In use, an electrode catheter is inserted into a major vein or artery, such as the femoral vein, and then guided into the heart chamber of interest. A typical ablation procedure involves inserting a catheter with one or more electrodes at its distal end into a heart chamber. A reference electrode is typically taped to the patient's skin or may be provided by a second catheter placed in or near the heart. RF (radio frequency) current is applied between the catheter electrode of the ablation catheter and an indifferent electrode (which may be one of the catheter electrodes), and the current flows between the electrodes, i.e., through the medium between the blood and the tissue. The distribution of the current may depend on the amount of electrode surface in contact with the tissue compared to the blood, which has a higher electrical conductivity than the tissue. Tissue heating occurs due to the electrical resistance of the tissue. Sufficient tissue heating can cause cell destruction in the cardiac tissue, resulting in the formation of lesions in the non-conductive cardiac tissue. In some applications, irreversible electroporation can be performed to ablate tissue.

[0007] Electrode sensors within a cardiac chamber can detect far-field electrical activity, i.e., ambient electrical activity occurring away from the sensor, which may distort or obscure local electrical activity, i.e., signals occurring at or near the sensor. Commonly assigned U.S. Patent Application Publication No. 2014 / 0005664 to Govari et al. discloses a method for distinguishing the local component in an intracardiac electrode signal due to tissue in contact with the electrode from far-field contributions to the signal, and explains that a therapeutic procedure applied to the tissue can be controlled in response to the distinguished local component. Summary of the Invention [Means for solving the problem]

[0008] According to an embodiment of the present disclosure, there is provided a method for generating a three-dimensional (3D) anatomical map, the method including applying a trained artificial neural network to (a) a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject and (b) first 3D coordinates of each of the set of 2D fluoroscopic images to yield second 3D coordinates that define the 3D anatomical map, and rendering the 3D anatomical map on a display in response to the second 3D coordinates.

[0009] Furthermore, according to an embodiment of the present disclosure, the set of 2D fluoroscopic images includes only two 2D fluoroscopic images.

[0010] Still further, according to an embodiment of the present disclosure, the set of 2D fluoroscopic images includes an anterior-posterior projection of the body-part and a left anterior-oblique projection of the body-part.

[0011] Additionally, according to an embodiment of the present disclosure, the second 3D coordinates include one of more of the following: a mesh vertex of a 3D mesh; and a 3D point cloud.

[0012] Further, according to an embodiment of the present disclosure, the method includes refining the 3D anatomical map in response to signals received from electrodes of a catheter inserted into a body portion of the living subject.

[0013] Furthermore, according to an embodiment of the present disclosure, the first 3D coordinate and the second 3D coordinate are in the same coordinate space.

[0014] Still further, according to an embodiment of the present disclosure, a method includes training an artificial neural network to generate a 3D anatomical map in response to training data, the training data including a plurality of sets of 2D fluoroscopic images of respective body parts of each living subject, 3D coordinates of each of the sets of 2D fluoroscopic images, and 3D coordinates of each of the plurality of 3D anatomical maps of each body part of each living subject.

[0015] Additionally, according to an embodiment of the present disclosure, a method includes inputting into an artificial neural network a plurality of sets of 2D fluoroscopic images of respective body parts of respective living subjects and respective 3D coordinates of the plurality of sets of 2D fluoroscopic images, and iteratively adjusting parameters of the artificial neural network to reduce a difference between an output of the artificial neural network and a desired output, the desired output including the respective 3D coordinates of the plurality of 3D anatomical maps.

[0016] Further, according to an embodiment of the present disclosure, the method includes generating a plurality of 3D anatomical maps of the training data in response to signals received from electrodes of at least one catheter inserted into a body portion of each living subject.

[0017] Furthermore, according to an embodiment of the present disclosure, each of the multiple sets of 2D fluoroscopic images includes only two 2D fluoroscopic images.

[0018] Still further, according to an embodiment of the present disclosure, the multiple sets of 2D fluoroscopic images include respective anterior-posterior projections and respective left anterior-oblique projections of the respective body parts.

[0019] Also, according to another embodiment of the present disclosure, there is provided a medical system comprising: a fluoroscopic imaging device configured to capture a set of two-dimensional (2D) fluoroscopic images of a body portion of a living subject; a display; and processing circuitry configured to apply a trained artificial neural network to (a) the set of two-dimensional (2D) fluoroscopic images of the body portion of the living subject and (b) first 3D coordinates of each of the set of 2D fluoroscopic images to yield second 3D coordinates of a 3D anatomical map; and, in response to the second 3D coordinates, render the 3D anatomical map on the display.

[0020] Additionally, according to an embodiment of the present disclosure, the set of 2D fluoroscopic images includes only two 2D fluoroscopic images.

[0021] Furthermore, according to an embodiment of the present disclosure, the set of 2D fluoroscopic images includes an anterior-posterior projection of the body part and a left anterior-oblique projection of the body part.

[0022] Further, according to an embodiment of the present disclosure, the second 3D coordinates include one of more of the following: a mesh vertex of a 3D mesh; and a 3D point cloud.

[0023] Still further, according to an embodiment of the present disclosure, the system includes a catheter including electrodes and configured to be inserted into a body portion of a living subject, and the processing circuitry is configured to improve the 3D anatomical map in response to signals received from the electrodes of the catheter.

[0024] Additionally, according to an embodiment of the present disclosure, the first 3D coordinate and the second 3D coordinate are in the same coordinate space.

[0025] Further, according to an embodiment of the present disclosure, the fluoroscopic imaging device is configured to capture a plurality of sets of two-dimensional (2D) fluoroscopic images of each body part of each living object, and the processing circuitry is configured to train the artificial neural network to generate a 3D anatomical map in response to training data, the training data including the plurality of sets of 2D fluoroscopic images of each body part of each living object, 3D coordinates of each of the plurality of sets of 2D fluoroscopic images, and 3D coordinates of each of the plurality of 3D anatomical maps of each body part of each living object.

[0026] Further, according to an embodiment of the present disclosure, the processing circuitry is configured to input, to the artificial neural network, the plurality of sets of 2D fluoroscopic images of each body part of each living subject and the respective 3D coordinates of the plurality of sets of 2D fluoroscopic images, and iteratively adjust parameters of the artificial neural network to reduce a difference between an output of the artificial neural network and a desired output, the desired output including the respective 3D coordinates of the plurality of 3D anatomical maps.

[0027] Still further, according to an embodiment of the present disclosure, the system includes at least one catheter including electrodes and configured to be inserted into the body portion of each living subject, and the processing circuitry is configured to generate a plurality of 3D anatomical maps of the training data in response to signals received from the electrodes of the at least one catheter inserted into the body portion of each living subject.

[0028] Additionally, according to an embodiment of the present disclosure, each of the multiple sets of 2D fluoroscopic images includes only two 2D fluoroscopic images.

[0029] Further, according to an embodiment of the present disclosure, the multiple sets of 2D fluoroscopic images include respective anterior-posterior projections and respective left anterior-oblique projections of the respective body parts.

[0030] Also, according to yet another embodiment of the present disclosure, there is provided a software product including a non-transitory computer-readable medium having stored thereon program instructions that, when read by a central processing unit (CPU), cause the CPU to apply the trained artificial neural network to (a) a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject, and (b) first 3D coordinates of each of the set of 2D fluoroscopic images to yield second 3D coordinates of a 3D anatomical map, and render the 3D anatomical map on a display in response to the second 3D coordinates. [Brief explanation of the drawings]

[0031] The present invention will be understood from the following detailed description taken in conjunction with the accompanying drawings. [Figure 1] 1 is a schematic illustration of a medical system constructed and operative in accordance with an exemplary embodiment of the present invention; [Figure 2] 2 is a flow diagram including steps of a method for training an artificial neural network for use in the system of FIG. 1; [Figure 3] FIG. 2 is a schematic illustration of an artificial neural network trained within the system of FIG. 1; [Figure 4] FIG. 2 is a flow diagram including detailed steps of a method for training an artificial neural network for use in the system of FIG. 1. [Figure 5] FIG. 2 is a flow diagram including steps of a method for applying a trained artificial neural network within the system of FIG. 1. [Figure 6] FIG. 2 is a schematic illustration of a trained artificial network applied within the system of FIG. 1. DETAILED DESCRIPTION OF THE INVENTION

[0032] Overview Prior to the advent of catheter-based cardiac chamber mapping, physicians would rely exclusively on fluoroscopy to guide catheters during surgery. The primary drawback of fluoroscopy is the risk of radiation. However, fluoroscopy does have some advantages in that it provides a view of the various blood vessels and the beating heart.

[0033] Therefore, catheter-based mapping is typically used to map heart chambers by moving a catheter around the chambers and generating a three-dimensional (3D) anatomical map of the heart. Magnetic and / or impedance-based position tracking can be used to track the catheter without the use of fluoroscopy. Nevertheless, fluoroscopy can be used, for example, to accurately introduce a sheath. Additionally, fluoroscopic images can be registered to the magnetic and / or impedance-based position tracking and rendered on a display along with the generated 3D map.

[0034] Although catheter-based mapping has many advantages over fluoroscopy, catheter-based mapping can miss certain features, such as blood vessels, that are seen using fluoroscopy. For example, some people may have three blood vessels associated with lung disease, while others may have four blood vessels associated with lung disease. A physician may simply miss a fourth blood vessel while moving the catheter, producing an anatomical map that is missing the fourth blood vessel.

[0035] Embodiments of the present invention solve the above problem by using a trained artificial neural network (ANN) to generate an initial 3D anatomical map from two two-dimensional (2D) fluoroscopic images of a body part (e.g., a heart chamber) (e.g., an anterior-posterior (AP) projection and a left anterior-oblique (LAO) projection, or any suitable pair of fluoroscopic image projections). The initial 3D anatomical map can then be refined based on catheter-based mapping by moving a catheter around the body part (e.g., a heart chamber).

[0036] In some embodiments, an ANN is trained to generate a 3D anatomical map from two 2D fluoroscopic images (e.g., an anterior-posterior (AP) and left anterior-oblique (LAO) projections, or any suitable pair of fluoroscopic image projections) and the respective 3D coordinates of the 2D fluoroscopic images based on training data. The training data includes (a) a set of 2D fluoroscopic images and the respective 3D coordinates of the 2D fluoroscopic images from each electrophysiology (EP) procedure as input to the ANN, and (b) a carefully acquired catheter-based 3D anatomical map from each EP procedure as the desired output of the ANN. During training of the ANN, the parameters (e.g., weights) of the ANN are corrected so that the output of the ANN approaches the desired output within given limits.

[0037] The coordinates of the fluoroscopic images (used during training and application of the ANN) are typically aligned with a magnetic and / or impedance-based position tracking system so that the 3D anatomical map generated by the ANN can be rendered on a display according to the known alignment. This alignment (i.e., the coordinates of the fluoroscopic images and the corresponding coordinates of the 3D anatomical map) therefore forms part of the training process.

[0038] Once trained, the ANN receives two 2D fluoroscopic images (e.g., anterior-posterior (AP) projection, left anterior-oblique (LAO) projection, or any suitable pair of fluoroscopic image projections) and the 3D coordinates of each of the two 2D fluoroscopic images as input, and outputs a 3D anatomical map in coordinates in the coordinate system of the magnetic and / or impedance-based position tracking system. This 3D anatomical map can be represented by the mesh vertices of a 3D mesh, or a 3D point cloud. This 3D point cloud can be used to generate an encapsulation mesh using a suitable algorithm, such as "marching cubes," a computer graphics algorithm published by Lorensen and Cline in the Proceedings of SIGGRAPH in 1987, for extracting an isosurface polygonal mesh from a three-dimensional discrete scalar field.

[0039] System Description Referring now to FIG. 1, this figure is a schematic diagram of a medical system 10 constructed and operative in accordance with an exemplary embodiment of the present invention. The medical system 10 is configured to perform a catheterization procedure on a living subject's heart 12 and is constructed and operative in accordance with a disclosed embodiment of the present invention. The system includes a catheter 14 that is percutaneously inserted by an operator 16 through the patient's vascular system into a chamber or vasculature of the heart 12. The operator 16, typically a physician, contacts the distal end 18 of the catheter with the heart wall at an ablation target site. Electrical activation maps, anatomical location information, i.e., of the distal end 18 of the catheter 14, and other functional images can then be prepared using processing circuitry 22 located within a console 24 according to any suitable method, such as those disclosed in U.S. Pat. Nos. 6,226,542, 6,301,496, and 6,892,091. One commercially available product embodying elements of system 10 is available as the CARTO® 3 system, commercially available from Biosense Webster, Inc., Irvine, California, USA, which is capable of generating electroanatomical maps of the heart as needed for ablation. This system can be modified by one skilled in the art to embody the principles of the embodiments of the invention described herein.

[0040] Regions determined to be abnormal, for example, by evaluation of the electrical activation map, can be ablated by application of thermal energy, such as by passing radiofrequency current through wires within catheter 14 (or another catheter) to one or more electrodes 21 (for simplicity, only a few electrodes are labeled) at distal end 18, where the radiofrequency energy is applied to the myocardium of heart 12. The energy is absorbed within the tissue, heating (or cooling) it to a point (typically about 60 degrees Celsius) at which the tissue permanently loses its electrical excitability. If successful, this procedure creates non-conducting lesions in the cardiac tissue that interrupt the abnormal electrical pathways causing the arrhythmia. By applying the principles of the present invention to various heart chambers, many different cardiac arrhythmias can be treated.

[0041] The catheter 14 typically includes a handle 20 having suitable controls thereon to enable the operator 16 to steer, position, and orient the distal end 18 of the catheter 14 as needed for mapping and ablation. To assist the operator 16, the distal portion of the catheter 14 contains a position sensor (not shown) that provides signals to a processing circuit 22, which calculates the position of the distal end 18.

[0042] Ablation energy and electrical signals may be transmitted to and from heart 12 via cables 34 to console 24. Pacing and other control signals may be transmitted from console 24 to heart 12 via cables 34 and electrodes 21.

[0043] Wire connections 35 link console 24 with body surface electrodes 30 and other components of the positioning subsystem. As taught in U.S. Patent No. 7,536,218, electrodes 21 and body surface electrodes 30 may be used to measure tissue impedance at the ablation site.

[0044] The console 24 typically houses one or more ablation power generators 25. The catheter 14 may be adapted to deliver ablation energy to the heart using any known ablation technique, such as radiofrequency energy, irreversible electroporation, ultrasound energy, cryotechnology, and laser-generated light energy. Such methods are disclosed in U.S. Patent Nos. 6,814,733, 6,997,924, and 7,156,816.

[0045] Processing circuitry 22 may be an element of a positioning system within system 10 that measures the position and orientation coordinates of catheter 14. In one embodiment, the positioning subsystem includes a magnetic position tracking device that determines the position and orientation of catheter 14 by generating magnetic fields within a predetermined working volume using field generating coils 28 and sensing these fields at catheter 14. The positioning subsystem may employ impedance measurements, for example, as taught in U.S. Pat. Nos. 7,756,576 and 7,536,218.

[0046] Fluoroscopic imager 37 includes a C-arm 39, an X-ray source 41, an image intensifier module 43, and an adjustable collimator 45. A control processor (not shown) may be located within console 24, allowing an operator to control the operation of fluoroscopic imager 37, for example, by setting imaging parameters and controlling collimator 45 to adjust the size and position of the field of view. The control processor communicates with fluoroscopic imager 37 via cable 51 to enable and disable X-ray source 41 or limit its emission to a desired region of the subject by controlling collimator 45, and to acquire image data from image intensifier module 43. An optional display monitor 49 is connected to the control processor, allowing operator 16 to view images generated by fluoroscopic imager 37. If display monitor 49 is not included, fluoroscopic images may be viewed on display 29, either via a split screen or alternating with other non-fluoroscopic images.

[0047] As mentioned above, catheter 14 is coupled to console 24, which allows operator 16 to observe and adjust the functions of catheter 14. Processing circuitry 22 is typically a computer with appropriate signal processing circuitry. Processing circuitry 22 is coupled to drive display 29. The signal processing circuitry typically receives, amplifies, filters, and digitizes signals from catheter 14, including signals generated by the aforementioned sensors and electrodes 21 located distally within catheter 14. The digitized signals are received and used by console 24 and a positioning subsystem to calculate the position and orientation of catheter 14, analyze the electrical signals from electrodes 21, and generate the desired electroanatomical map.

[0048] System 10 typically includes other elements, not shown for simplicity. For example, system 10 may include an electrocardiogram (ECG) monitor coupled to receive signals from one or more body surface electrodes to provide ECG-synchronized signals to console 24. As noted above, system 10 typically also includes a reference position sensor, either an externally applied reference patch attached to the outside of the subject's body, or an internally-placed catheter inserted into heart 12 while maintained in a fixed position relative to the heart 12. Conventional pumps and lines are provided for circulating a fluid through catheter 14 to cool the ablation site.

[0049] The fluoroscopic imager 37 is typically aligned with the coordinate space 31 of the positioning subsystem associated with the field generating coils 28 and the distal end 18 of the catheter 14. Thus, images captured by the fluoroscopic imager 37 can be used in conjunction with the positioning subsystem. For example, a representation of the distal end 18 of the catheter 14 can be rendered on the display 29 superimposed on an x-ray image captured by the fluoroscopic imager 37.

[0050] Reference is now made to Figures 2 and 3. Figure 2 is a flow diagram 100 including steps of a method for training an artificial neural network 52 for use in the system 10 of Figure 1. Figure 3 is a schematic illustration of the artificial neural network 52 trained in the system 10 of Figure 1.

[0051] The fluoroscopic imager 37 (FIG. 1) is configured to capture multiple sets of 2D fluoroscopic images 54 of respective body parts 56 of respective living subjects 58 (block 102). Each of the multiple sets of 2D fluoroscopic images 54 is associated with the image's 3D coordinates 60. For example, each 2D fluoroscopic image may include coordinates identifying at least two given points within the image in coordinate space (e.g., two corners of the image, or a corner and center of the image, or any other suitable point). Alternatively, each 2D fluoroscopic image may include coordinates identifying one given point within the image and the image's orientation within coordinate space. These coordinates may be provided by the controller of the fluoroscopic imager 37 with reference to coordinate space 31 (FIG. 1) of any other coordinate space that is aligned with coordinate space 31. When the fluoroscopic imaging device 37 is fixed relative to the coordinate space 31, the 3D coordinates of the multiple sets of 2D fluoroscopic images 54 do not need to be provided by the controller of the fluoroscopic imaging device 37 for each of the multiple sets of 2D fluoroscopic images 54 when the 3D coordinates of the multiple sets of 2D fluoroscopic images 54 are known to the medical system 10.

[0052] In some embodiments, each of the multiple sets of 2D fluoroscopic images 54 includes only two 2D fluoroscopic images, which are generally orthogonal projections of the body-part 56. In some embodiments, the multiple sets of 2D fluoroscopic images 54 include a respective anterior-posterior projection and a respective left anterior-oblique projection of each body-part 56, as shown in FIG.

[0053] For each set of 2D fluoroscopic images 54, a corresponding anatomical map 62 is generated using a catheter-based method described in further detail herein. At least one catheter (e.g., catheter 14) including electrodes (e.g., electrode 21) is configured to be inserted into the body portion 56 of each living subject 58 (block 104). Processing circuitry 22 is configured to receive signals from the electrodes (e.g., electrode 21) of the catheter (e.g., catheter 14) (block 106). The catheter is carefully moved around the body portion 56 to ensure that the map generated from the catheter's movement is accurate. Processing circuitry 22 (FIG. 1) is configured to generate multiple 3D anatomical maps 62 as training data for the artificial neural network 52 in response to the signals received from the electrodes of the catheter inserted into the body portion 56 of each living subject 58 (block 108). The multiple 3D anatomical maps 62 can be defined relative to respective 3D coordinates 64, which can include mesh vertices of a 3D mesh and / or a 3D point cloud.

[0054] Processing circuitry 22 is configured to train artificial neural network 52 to generate 3D anatomical maps in response to training data (block 110), the training data including (a) a plurality of sets of 2D fluoroscopic images 54 (captured by fluoroscopic imager 37) of each body part 56 of each living subject 58, (b) respective 3D coordinates 60 of the plurality of sets of 2D fluoroscopic images 54, and (c) respective 3D coordinates 64 of a plurality of 3D anatomical maps 62 of each body part 56 of each living subject 58. Each set of 2D fluoroscopic images 54, and each 3D coordinate 60 of that set of 2D fluoroscopic images 54, has an associated 3D anatomical map 62 (with corresponding 3D coordinates 64) captured for each body part 56 of each living subject 58. In other words, the training data includes a set of 2D fluoroscopic images 54, 3D coordinates 60 for each of the set of 2D fluoroscopic images 54, and 3D coordinates 64 for one of a plurality of 3D anatomical maps 62 for each body part of the living subject 58. The body parts 56 may include any suitable body part, for example, a heart chamber. The body parts 56 used in training the artificial neural network 52 are body parts of the same type, for example, a heart chamber.

[0055] Reference is now made to Figure 4, which is a flow diagram including detailed sub-steps of the step of block 110 of Figure 2. Reference is also made to Figure 3.

[0056] A neural network is a network or circuit of neurons, or in the modern sense, an artificial neural network, composed of artificial neurons or nodes. The connections of biological neurons are modeled as weights. Positive weights reflect excitatory connections, while negative values ​​represent inhibitory connections. The inputs are modified by the weights and summed using linear combinations. An activation function can control the amplitude of the output. For example, the allowed range of the output is usually 0 to 1, but can also be -1 to 1.

[0057] These artificial networks can be used in predictive modeling, adaptive control, and other applications, and can be trained through datasets. Self-learning resulting from experience can occur within the network, allowing it to draw conclusions from complex and seemingly unrelated sets of information.

[0058] For completeness, biological neural networks consist of groups of chemically connected or functionally associated neurons. One neuron may be connected to many other neurons, and the total number of neurons and connections in a network can vary widely. Connections, called synapses, are usually formed from axons to dendrites, although dendritic synapses and other connections are also possible. Apart from electrical signaling, there are other forms of signaling resulting from the diffusion of neurotransmitters.

[0059] Artificial intelligence, cognitive modeling, and neural networks are information processing paradigms inspired by the way biological nervous systems process data. Artificial intelligence and cognitive modeling attempt to simulate some of the properties of biological neural networks. In the field of artificial intelligence, artificial neural networks have been successfully applied to speech recognition, image analysis, and adaptive control, and to build software agents or autonomous robots (in computer and video games).

[0060] A neural network (NN) is an interconnected group of natural or artificial neurons that uses a mathematical or computational model based on a connectionist approach to computation for information processing, in the case of artificial neurons, called an artificial neural network (ANN) or simulated neural network (SNN). In most cases, ANNs are adaptive systems that change their structure based on external or internal information flowing through the network. More practically, neural networks are nonlinear statistical data modeling or decision-making tools. They can be used to model complex relationships between inputs and outputs and to find patterns in data.

[0061] In some embodiments, artificial neural network 52 comprises a fully connected neural network, such as a convolutional neural network. In other embodiments, artificial neural network 52 may comprise any suitable ANN. Artificial neural network 52 may include software executed by processing circuitry 22 (FIG. 1) and / or hardware modules configured to perform the functions of artificial neural network 52.

[0062] The artificial neural network 52 includes an input layer 80, where inputs are received, and one or more hidden layers 82, which progressively processes the inputs to an output layer 84, where the output of the artificial neural network 52 is provided. The artificial neural network 52 may include layer weights between the layers 80, 82, 84 of the artificial neural network 52. The artificial neural network 52 operates on data received at the input layer 80 according to the values ​​of the various layer weights between the layers 80, 82, 84 of the artificial neural network 52.

[0063] The layer weights of the artificial neural network 52 are updated during training of the artificial neural network 52 so that the artificial neural network 52 performs the data manipulation task that the artificial neural network 52 is trained to perform.

[0064] The number of layers and layer width within the artificial neural network 52 may be configurable. As the number of layers and layer width increase, the precision with which the artificial neural network 52 can manipulate data according to the task at hand increases. However, a greater number of layers and wider layers generally require more training data, i.e., more training time, and the training may not converge. By way of example, the input layer 80 may include 400 neurons (e.g., to compress a batch of 400 samples), and the output layer may also include 400 neurons.

[0065] Training the artificial neural network 52 is largely an iterative process. One method for training the artificial neural network 52 is described below. The processing circuitry 22 (FIG. 1) is configured to iteratively adjust (block 112) the parameters (e.g., layer weights) of the artificial neural network 52 to reduce the difference between the output of the artificial neural network 52 and a desired output of the artificial neural network 52. The desired output includes the 3D coordinates 64 of each of the plurality of 3D anatomical maps 62.

[0066] The substeps of the step in block 112 are now described below.

[0067] Processing circuitry 22 (FIG. 1) is configured to input the sets of 2D fluoroscopic images 54 of each body part 56 of each living subject 58 and the 3D coordinates 60 of each of the sets of 2D fluoroscopic images 54 to an input layer 80 of artificial neural network 52 (block 114, arrow 70). Processing circuitry 22 is configured to compare the output of artificial neural network 52 with desired outputs, i.e., the corresponding 3D coordinates 64 of each of the plurality of 3D anatomical maps 62 (block 116, arrow 72). The comparison is typically performed using a suitable loss function that calculates the overall difference between all outputs of artificial neural network 52 and all desired outputs (e.g., the 3D coordinates 64 of each of the plurality of 3D anatomical maps 62).

[0068] At decision block 118, processing circuitry 22 (FIG. 1) is configured to determine whether the difference between the output of artificial neural network 52 and the desired output is sufficiently small. If the difference between the output of artificial neural network 52 and the desired output is sufficiently small (branch 120), processing circuitry 22 is configured to save the parameters (e.g., weights) of the trained artificial neural network 52 for use in applying the trained artificial neural network 52 (block 122), as described in more detail with reference to FIGS.

[0069] If the difference is sufficiently small (branch 124), processing circuitry 22 is configured to correct the parameters (e.g., weights) of artificial neural network 52 (block 126) to reduce the difference between the output of artificial neural network 52 and the desired output of artificial neural network 52. In the above example, the minimized difference is the overall difference between all outputs of artificial neural network 52 and all desired outputs (e.g., 3D coordinates 64 of all respective plurality of 3D anatomical maps 62). Processing circuitry 22 is configured to correct the parameters using any suitable optimization algorithm, e.g., a gradient descent algorithm such as Adam optimization. The steps of blocks 114 through 118 are then repeated.

[0070] Reference is now made to Figures 5 and 6. Figure 5 is a flow diagram 200 including steps in a method of applying the trained artificial neural network 52 of the system 10 of Figure 1. Figure 6 is a schematic diagram of the trained artificial neural network 52 being applied in the system 10 of Figure 1.

[0071] The fluoroscopic imaging device 37 (FIG. 1) is configured to capture a set of 2D fluoroscopic images 86 of a body part 88 of a living subject 90 (block 202). In some embodiments, the set of 2D fluoroscopic images 86 includes only two 2D fluoroscopic images. In some embodiments, the set of 2D fluoroscopic images 86 includes an anterior-posterior (AP) projection of the body part 88 and a left anterior-oblique projection (LAO) of the body part 88, or any other suitable projection pair. The body part 88 may be any suitable body part, for example, a heart chamber.

[0072] Processing circuitry 22 (FIG. 1) is configured to apply trained artificial neural network 52 to (a) a set of 2D fluoroscopic images 86 of a body part 88 of living subject 90 and (b) 3D coordinates 96 of each of the set of 2D fluoroscopic images 86 to yield 3D coordinates 92 of a 3D anatomical map 94 (block 204). In some embodiments, the 3D coordinates 96 of the set of 2D fluoroscopic images 86 and the 3D coordinates 92 of the 3D anatomical map 94 are in the same coordinate space and are registered to the coordinate space of the positioning subsystem described above with reference to FIG. 1 (e.g., coordinate space 31 of FIG. 1). The 3D coordinates 92 of the 3D anatomical map 94 may include mesh vertices of a 3D mesh and / or a 3D point cloud.

[0073] Processing circuitry 22 is configured to render 3D anatomical map 94 on display 29 (FIG. 1) in response to 3D coordinates 92 (block 206).

[0074] Catheter 14 (FIG. 1) is configured to be inserted into body portion 88 of living subject 90 (block 208) and moved around body portion 88 to acquire signals via electrodes 21 (FIG. 1) to correct and improve 3D anatomical map 94. Processing circuitry 22 is configured to receive signals from electrodes 21 of catheter 14 (block 210). Processing circuitry 22 is configured to improve 3D anatomical map 94 in response to the signals received from electrodes 21 of catheter 14 (block 212).

[0075] In practice, some or all of the functionality of processing circuitry 22 may be combined within a single physical component, or alternatively, may be implemented using multiple physical components. These physical components may comprise hardwired or programmable devices, or a combination of the two. In some embodiments, at least some of the functionality of processing circuitry 22 may be performed by a programmable processor under the control of suitable software. This software may be downloaded to the device in electronic form, for example, over a network. Alternatively or additionally, this software may be stored on a tangible, non-transitory, computer-readable medium, such as optical, magnetic, or electronic memory.

[0076] The term "about" or "approximately" used herein in connection with any numerical value or range of values ​​indicates an appropriate and suitable dimensional tolerance that enables a portion of a component or a collection of components to function for its intended purpose as described herein. More specifically, "about" or "approximately" may refer to a range of values ​​of ±20% of the recited value; for example, "about 90%" may refer to a range of values ​​of 72% to 108%.

[0077] Various features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination.

[0078] The embodiments described above are cited by way of example, and the present invention is not limited to what has been particularly shown and described in the foregoing specification. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described in the foregoing specification, as well as variations and modifications thereof that would occur to one skilled in the art upon reading the foregoing description, and that are not disclosed in the prior art.

[0079] [Embodiment] (1) A method for generating a three-dimensional (3D) anatomical map, comprising: applying the trained artificial neural network to (a) a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject, and (b) first 3D coordinates of each of the set of 2D fluoroscopic images to yield second 3D coordinates that define a 3D anatomical map; and rendering the 3D anatomical map on a display in response to the second 3D coordinate. (2) The method of embodiment 1, wherein the set of 2D fluoroscopic images includes only two 2D fluoroscopic images. (3) The method described in embodiment 2, wherein the set of 2D fluoroscopic images includes an anterior-posterior projection of the body part and a left anterior-oblique projection of the body part. (4) The method of embodiment 1, wherein the second 3D coordinates include one of more of the following: a mesh vertex of a 3D mesh, and a 3D point cloud. (5) The method of embodiment 1, further comprising improving the 3D anatomical map in response to signals received from electrodes of a catheter inserted into the body portion of the living subject.

[0080] (6) The method of embodiment 1, wherein the first 3D coordinate and the second 3D coordinate are in the same coordinate space. (7) further comprising training the artificial neural network to generate a 3D anatomical map in response to training data, the training data comprising: a plurality of sets of 2D fluoroscopic images of each body part of each living subject; 3D coordinates of each of the plurality of sets of 2D fluoroscopic images; and 2. The method of claim 1, further comprising: respective 3D coordinates of a plurality of 3D anatomical maps of the respective body parts of the respective living subjects. (8) inputting the sets of 2D fluoroscopic images of each body part of each living subject and the respective 3D coordinates of the sets of 2D fluoroscopic images into the artificial neural network; 8. The method of claim 7, further comprising iteratively adjusting parameters of the artificial neural network to reduce a difference between an output of the artificial neural network and a desired output, the desired output comprising the respective 3D coordinates of the plurality of 3D anatomical maps. (9) The method of embodiment 7, further comprising generating the plurality of 3D anatomical maps of the training data in response to signals received from electrodes of at least one catheter inserted into the body portion of each of the living subjects. (10) The method of embodiment 7, wherein each of the multiple sets of 2D fluoroscopic images includes only two 2D fluoroscopic images.

[0081] (11) The method described in embodiment 10, wherein the multiple sets of 2D fluoroscopic images include respective anterior-posterior projections and respective left anterior-oblique projections of each of the body parts. (12) A health care system, a fluoroscopic imaging device configured to capture a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject; The display and A processing circuit, the processing circuit comprising: applying the trained artificial neural network to (a) a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject, and (b) first 3D coordinates of each of the set of 2D fluoroscopic images to produce second 3D coordinates of a 3D anatomical map; and a processing circuit configured to: render the 3D anatomical map on the display in response to the second 3D coordinate. (13) The system of embodiment 12, wherein the set of 2D fluoroscopic images includes only two 2D fluoroscopic images. (14) The system of embodiment 13, wherein the set of 2D fluoroscopic images includes an anterior-posterior projection of the body part and a left anterior-oblique projection of the body part. (15) The system of embodiment 12, wherein the second 3D coordinates include one of more of the following: a mesh vertex of a 3D mesh, and a 3D point cloud.

[0082] (16) The system of embodiment 12, further comprising a catheter having electrodes and configured to be inserted into the body portion of the biological subject, wherein the processing circuitry is configured to improve the 3D anatomical map in response to signals received from the electrodes of the catheter. (17) The system of embodiment 12, wherein the first 3D coordinate and the second 3D coordinate are in the same coordinate space. (18) The fluoroscopic imaging device is configured to capture a plurality of sets of two-dimensional (2D) fluoroscopic images of each body part of each living subject; The processing circuitry is configured to train the artificial neural network to generate a 3D anatomical map in response to training data, the training data comprising: a plurality of sets of said 2D fluoroscopic images of respective body parts of respective living subjects; 3D coordinates of each of the plurality of sets of 2D fluoroscopic images; and The system of embodiment 12, further comprising: respective 3D coordinates of a plurality of 3D anatomical maps of the respective body parts of the respective living subjects. (19) The processing circuit inputting into the artificial neural network the sets of 2D fluoroscopic images of each body part of each living subject and the respective 3D coordinates of the sets of 2D fluoroscopic images; 19. The system of claim 18, configured to iteratively adjust parameters of the artificial neural network to reduce a difference between an output of the artificial neural network and a desired output, the desired output comprising the respective 3D coordinates of the plurality of 3D anatomical maps. (20) The system of embodiment 18, further comprising at least one catheter having electrodes and configured to be inserted into the body portion of each of the living subjects, wherein the processing circuit is configured to generate the plurality of 3D anatomical maps of the training data in response to signals received from the electrodes of the at least one catheter inserted into the body portion of each of the living subjects.

[0083] (21) The system described in embodiment 18, wherein each of the multiple sets of 2D fluoroscopic images includes only two 2D fluoroscopic images. (22) The system of embodiment 21, wherein the plurality of sets of 2D fluoroscopic images include respective anterior-posterior projections and respective left anterior-oblique projections of each of the body parts. (23) A software product including a non-transitory computer-readable medium having stored thereon program instructions, the instructions, when read by a central processing unit (CPU), causing the CPU to: applying the trained artificial neural network to (a) a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject, and (b) first 3D coordinates of each of the set of 2D fluoroscopic images to produce second 3D coordinates of a 3D anatomical map; and rendering the 3D anatomical map on a display in response to the second 3D coordinate.

Claims

1. 1. A health care system comprising: a fluoroscopic imaging device configured to capture a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject; The display and A processing circuit, the processing circuit comprising: applying the trained artificial neural network to (a) a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject, and (b) first 3D coordinates of each of the set of 2D fluoroscopic images to produce second 3D coordinates of a 3D anatomical map; and rendering the 3D anatomical map on the display in response to the second 3D coordinate; and a catheter including electrodes and configured to be inserted into the body portion of the living subject, the processing circuitry configured to refine the 3D anatomical map in response to signals received from the electrodes of the catheter. Healthcare system.

2. The system of claim 1 , wherein the set of 2D fluoroscopic images includes only two 2D fluoroscopic images.

3. The system of claim 2 , wherein the set of 2D fluoroscopic images includes an anterior-posterior projection of the body part and a left anterior-oblique projection of the body part.

4. The system of claim 1 , wherein the second 3D coordinates include one of more of the following: a mesh vertex of a 3D mesh, and a 3D point cloud.

5. The system of claim 1 , wherein the first 3D coordinate and the second 3D coordinate are in the same coordinate space.

6. the fluoroscopic imaging device is configured to capture a plurality of sets of two-dimensional (2D) fluoroscopic images of each body part of each living subject; The processing circuitry is configured to train the artificial neural network to generate a 3D anatomical map in response to training data, the training data comprising: a plurality of sets of said 2D fluoroscopic images of respective body parts of respective living subjects; 3D coordinates of each of the sets of 2D fluoroscopic images; and 2. The system of claim 1, further comprising: respective 3D coordinates of a plurality of 3D anatomical maps of the respective body parts of the respective living subjects.

7. the processing circuitry inputting into the artificial neural network the sets of 2D fluoroscopic images of respective body parts of respective living subjects and the respective 3D coordinates of the sets of 2D fluoroscopic images; 7. The system of claim 6, configured to iteratively adjust parameters of the artificial neural network to reduce a difference between an output of the artificial neural network and a desired output, the desired output comprising the respective 3D coordinates of the plurality of 3D anatomical maps.

8. 7. The system of claim 6, further comprising at least one catheter comprising electrodes and configured to be inserted into the body portion of the respective living subject, wherein the processing circuitry is configured to generate the plurality of 3D anatomical maps of the training data in response to signals received from the electrodes of the at least one catheter inserted into the body portion of the respective living subject.

9. The system of claim 6 , wherein each of the plurality of sets of 2D fluoroscopic images includes only two 2D fluoroscopic images.

10. The system of claim 9, wherein the plurality of sets of 2D fluoroscopic images include a respective anterior-posterior projection and a respective left anterior-oblique projection of the respective body-part.

11. 1. A software product including a non-transitory computer-readable medium having stored thereon program instructions, the instructions, when read by a central processing unit (CPU), causing the CPU to: applying the trained artificial neural network to (a) a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject, and (b) first 3D coordinates of each of the set of 2D fluoroscopic images to produce second 3D coordinates of a 3D anatomical map; rendering the 3D anatomical map on a display in response to the second 3D coordinate; and refining the 3D anatomical map in response to signals received from electrodes of a catheter configured to be inserted into the body portion of the living subject. Software products.

12. A method of operating a system for generating a three-dimensional (3D) anatomical map, the system comprising a processing circuit and a catheter, the method comprising: applying the processing circuitry to (a) a set of two-dimensional (2D) fluoroscopic images of a body part of a living subject, and (b) first 3D coordinates of each of the set of 2D fluoroscopic images to result in second 3D coordinates that define a 3D anatomical map; the processing circuitry rendering the 3D anatomical map on a display in response to the second 3D coordinate. The method further includes the processing circuitry refining the 3D anatomical map in response to signals of the body part of the living subject received from electrodes of the catheter.

13. The method of claim 12 , wherein the set of 2D fluoroscopic images includes only two 2D fluoroscopic images.

14. The method of claim 13, wherein the set of 2D fluoroscopic images includes an anterior-posterior projection of the body-part and a left anterior-oblique projection of the body-part.

15. The method of claim 12 , wherein the second 3D coordinates include one of more of the following: a mesh vertex of a 3D mesh, and a 3D point cloud.

16. The method of claim 12 , wherein the first 3D coordinate and the second 3D coordinate are in the same coordinate space.

17. The method of claim 16, further comprising: training the artificial neural network to generate a 3D anatomical map in response to training data, the training data comprising: a plurality of sets of 2D fluoroscopic images of each body part of each living subject; 3D coordinates of each of the sets of 2D fluoroscopic images; and 13. The method of claim 12, comprising: respective 3D coordinates of a plurality of 3D anatomical maps of the respective body parts of the respective living subjects.

18. The processing circuitry inputting into the artificial neural network a plurality of sets of 2D fluoroscopic images of each body part of each biological subject, and the respective 3D coordinates of the plurality of sets of 2D fluoroscopic images; 18. The method of claim 17, further comprising: the processing circuitry iteratively adjusting parameters of the artificial neural network to reduce a difference between an output of the artificial neural network and a desired output, the desired output comprising the respective 3D coordinates of the plurality of 3D anatomical maps.

19. The method of claim 17, further comprising the processing circuit generating the plurality of 3D anatomical maps of the training data in response to signals received from electrodes of the catheter.

20. 18. The method of claim 17, wherein each of the multiple sets of 2D fluoroscopic images includes only two 2D fluoroscopic images.

21. 21. The method of claim 20, wherein the plurality of sets of 2D fluoroscopic images comprises a respective anterior-posterior projection and a respective left anterior-oblique projection of the respective body-part.

Images