System and method for estimating time to target from image characteristics

Abstract

A system for navigating an interventional instrument in a patient's anatomy receives one or more images including a portion of the patient's anatomy and an interventional instrument disposed within the portion of the anatomy. The system identifies, from the one or more images, a position of the interventional instrument in the portion of the anatomy and anatomical features of the portion of the anatomy. The system further identifies a target region of interest (ROI) in the anatomy. The system predicts, based on the anatomical features of the portion of the anatomy, a time to navigate the interventional instrument from the position of the interventional instrument through the portion of the anatomy to the position of the target ROI.

Description

[Technical Field]

[0001] The following generally relates to endovascular treatment technologies, device tracking technologies, artificial intelligence (AI) technologies, and related technologies. [Background technology]

[0002] Ischemic stroke is an emergency medical condition in which a blood clot or other vascular obstruction impedes blood flow to brain tissue. A common treatment is an endovascular procedure, in which a catheter or intravascular device is inserted, its tip navigated to the site of the vascular obstruction, and used to remove the obstruction. This process of removing the obstruction via endovascular intervention is sometimes referred to as reperfusion, as it effectively restores vascular blood supply to the tissue. While ischemic stroke is an example of such an emergency, similar revascularization procedures may be appropriate for treating obstructions in other critical anatomical structures (e.g., the heart, lungs, etc.). Delaying revascularization in an ischemic stroke or other emergency obstruction situation can result in long-term, sometimes irreversible, disability, or even life-threatening consequences. Navigation from the access point to the target site is not always easy, may require focusing on specific areas of the navigation path, and can take a long time. Monitoring the total navigation time and remaining time in emergency procedures such as ischemic stroke revascularization can better inform decisions such as modifying the device or navigation strategy and is important for successful treatment. For non-emergency procedures, having information about navigation time to the target can help staff prepare the room for the next step, including preparing the appropriate personnel and equipment for the next procedure step.

[0003] Reducing delays in treatment of acute neurovascular conditions such as ischemic stroke significantly increases the likelihood of a favorable procedural outcome and shortens rehabilitation periods (see, e.g., Tapuwa D. Musuka MBChB, Stephen B. Wilton MD, Mouhieddin Traboulsi MD, Michael D. Hill MD, "Diagnosis and management of acute ischemic stroke: speed is critical." CMAJ, September 8, 2015, 187(12)).

[0004] One step in revascularization procedures for stroke treatment is endovascular navigation of the tip of an interventional instrument to a target vessel or region of interest (ROI). This is typically performed under the guidance of a real-time interventional imaging modality, such as X-ray or ultrasound imaging. Interventional imaging typically provides two-dimensional (2D) images, as it can be difficult or impossible to obtain three-dimensional (3D) cross-sectional images in real time or while the physician is positioned close enough to the patient to manipulate the interventional instrument. However, multiview imaging is sometimes used, which can provide 3D image information. In some cases, preoperative 3D images, such as computed tomography (CT) angiography images, may be available, but in emergency situations, there may not be time to acquire such 3D preoperative images. Navigation time can increase significantly if the patient has tortuous anatomy or multiple branches along the navigation path. Furthermore, vessel tortuosity can lead to multiple device exchanges, increasing procedure time and potentially increasing the risk of injury. Therefore, it is important for the physician to maintain precision during intravascular navigation, which is facilitated by zooming in on the real-time interventional image displayed around the distal end of the device to see details related to geometric complexities (e.g., bends, bifurcations) while navigating. Summary of the Invention [Problem to be solved by the invention]

[0005] However, focusing on a small region of interest (ROI) around the distal end of the device can result in losing sight of the overall vasculature and the remaining path to the target. This can also result in losing track of the time required to reach that target, facilitating decisions during the procedure (e.g., changing staff, modifying navigation speed, changing devices, etc.). Looking at a small ROI increases the likelihood that the physician will take the wrong path at a vessel bifurcation.

[0006] Specific improvements are disclosed below to overcome these and other problems. [Means for solving the problem]

[0007] In some embodiments disclosed herein, a system for navigating an interventional device within a patient's anatomy includes a processor and a memory, wherein the processor is configured to: receive one or more images including a portion of the patient's anatomy and an interventional device disposed within the portion of the anatomy; identify from the one or more images a position of the interventional device within the portion of the anatomy and at least one anatomical feature of the portion of the anatomy; identify a target region of interest (ROI) within the anatomy; and predict, based on the at least one anatomical feature of the portion of the anatomy, an amount of time to navigate the interventional device from a position of the interventional device through the portion of the anatomy to a position of the target ROI.

[0008] In some embodiments disclosed herein, a method for navigating an interventional device in a patient's anatomy includes receiving one or more images including a portion of the patient's anatomy and an interventional device positioned within the portion of the anatomy; identifying a position of the interventional device within the portion of the anatomy and at least one anatomical feature of the portion of the anatomy from the one or more images; identifying a target ROI in the anatomy; and predicting a time to navigate the interventional device from the position of the interventional device through the portion of the anatomy to the position of the target ROI based on the at least one anatomical feature of the portion of the anatomy.

[0009] In some embodiments disclosed herein, a non-transitory computer-readable storage medium stores a computer program having instructions that, when executed by a processor, cause the processor to receive one or more images including a portion of a patient's anatomy and an interventional device disposed within the portion of the anatomy, identify from the one or more images a position of the interventional device within the portion of the anatomy and at least one anatomical feature of the anatomy, identify a target region of interest (ROI) in the anatomy, and predict an amount of time to navigate from a position of the interventional device through the portion of the anatomy to a position of the target ROI based on the at least one anatomical feature of the portion of the anatomy.

[0010] In some of these embodiments, the at least one anatomical feature of the portion of the anatomy includes at least one of vascular tortuosity, bifurcation, vascular curvature, small vascular diameter, and vascular bifurcation in the portion of the anatomy. In some of these embodiments, the trained machine learning model is utilized to predict an amount of time to navigate from a location of the interventional device through the portion of the anatomy to a location of a target ROI based on the at least one anatomical feature of the portion of the anatomy. In some of these embodiments, a tip of the interventional device comprises a radiopaque material, and the processor is further configured to identify a location of the interventional device in one or more images based on the radiopaque tip. In some of these embodiments, a path is determined to navigate the interventional device from a location of the interventional device through the portion of the anatomy to a location of the target ROI, and multiple consecutive ROIs are identified along the path, and an amount of time to navigate to each of the multiple consecutive ROIs is predicted.

[0011] One advantage is in reducing delays during endovascular procedures.

[0012] Another advantage resides in monitoring the tip of an intravascular device during an intravascular procedure.

[0013] Another advantage resides in using deep learning to monitor the tip of an intravascular device during an intravascular procedure.

[0014] Another advantage resides in using imaging to monitor the tip of an intravascular device during an intravascular procedure.

[0015] A given embodiment may provide none, one, two, more, or all of the advantages discussed above, and / or may provide other advantages that will become apparent to those skilled in the art upon reading and understanding this disclosure.

[0016] The disclosure may take form in various components and arrangements of components, and in various steps and arrangements of steps. The drawings are only for purposes of illustrating preferred embodiments and are not to be construed as limiting the disclosure. [Brief explanation of the drawings]

[0017] [Figure 1] 1 illustrates a schematic diagram of an intravascular device according to the present disclosure. [Figure 2] 2 illustrates a schematic diagram of a method for performing a vascular treatment using the device of FIG. 1. [Figure 3] 1 illustrates the operation of a machine learning model that receives as input an image containing the known location of the tip of an interventional instrument and the identified location of a target, and outputs an estimated time to the target. [Figure 4] FIG. 4 shows a schematic representation of the visualization displayed by the device of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0018] Referring to FIG. 1 , a system 10 is shown schematically. The system 10 can be, for example, an endovascular system, an endobronchial system, a surgical system, or any other medical system. As shown in FIG. 1 , the system 10 includes an interventional instrument 12 (e.g., a catheter, probe, needle, electrode, etc., shown schematically as a line in FIG. 1 ) configured for insertion into a portion of a patient's anatomy. For example, the interventional instrument may be configured for insertion into a patient's blood vessel V having an occlusion or thrombus C (shown schematically as a dashed line in FIG. 1 ). In some embodiments, the interventional instrument 12 is radiopaque. The interventional instrument 12 includes a tip 14. In some embodiments, the tip 14 is radiopaque to improve imaging of the tip in fluoroscopic imaging. The radiopaque tip 14 may be composed of a different material than the remainder of the interventional instrument 12. For example, the tip 14 may have a short radiopaque wire (e.g., a platinum or nitinol wire) metallurgically bonded (e.g., by welding) to the end of the interventional device 12. Alternatively, in some embodiments, radiopaque markers (not shown), such as platinum tungsten-filled polyurethane bands, may be spaced along the tip 14. In yet another embodiment, the interventional device 12 may be configured to be entirely radiopaque. For example, a wire may be placed along the length of the interventional device and coated with a radiopaque coating, such as a tantalum coating.

[0019] In some embodiments, a clinician may control the movement of the interventional instrument 12 within the blood vessel V. In other embodiments, robotic control of the movement of the interventional instrument 12 is provided. FIG. 1 also shows an embodiment having a robot 16 (shown as a box in FIG. 1 ) operably connected to the interventional instrument 12. The robot 16 may be configured to control the movement of the interventional instrument 12 into, through, and out of the blood vessel V. In some embodiments, a clinician may control the robot using a controller such as a joystick or mouse clicks on a user interface, while in other embodiments, an autonomous control system may be provided that is configured to automatically steer the robot 16.

[0020] FIG. 1 also illustrates an embodiment in which a processing device 18, such as a computer, controls the robot 16 to automatically move the interventional instrument 12 through the blood vessel V. The processing device 18 may also include a server computer, or multiple server computers interconnected to form, for example, a server cluster, cloud computing resources, etc., to perform more complex computational tasks. The processing device 18 may include components such as a processor 20 (e.g., a microprocessor or other hardware processor), at least one user input device (e.g., a mouse, keyboard, trackball, etc.) 22, and a display device 24 (e.g., an LCD display, plasma display, cathode ray tube display, etc.). In some embodiments, the display device 24 may be a separate component from the processing device 18. In some embodiments, the processing device 18 may include two or more display devices.

[0021] The processor 20 is operatively connected to one or more non-transitory storage media 26. The non-transitory storage media 26 may include, by way of non-limiting example, a magnetic disk, RAID, or other magnetic storage medium, a solid-state drive, a flash drive, an erasable read-only memory (EEROM) or other memory, an optical disk or other optical storage device, various combinations thereof, or the like, such as network storage, an internal hard drive of the electronic processing device 18, various combinations thereof, or the like. References herein to a non-transitory medium or media 26 should be interpreted broadly to include a single medium or multiple media of the same or different types. Similarly, the processor 20 may be embodied as a single processor or as two or more processors. The non-transitory storage medium 26 stores instructions executable by the processor 20. The instructions may include instructions for generating a visualization of a graphical user interface (GUI) 28 for display on the display device 24 (see, for example, FIG. 4).

[0022] FIG. 1 also shows an imaging device 30 configured to acquire a temporal sequence of images, or imaging frames 35, of the movement of the interventional instrument 12 (including the tip 14 of the interventional instrument 12). The interventional instrument may also be referred to herein as an interventional device. In the illustrative example of FIG. 1, the imaging device 30 is a fluoroscopic imaging device (e.g., an X-ray imager, a C-arm imager, a CT scanner, etc.), and the radiopaque tip 14 of the interventional instrument 12 is visible under fluoroscopic imaging. The imaging device 30 of FIG. 1 is configured to perform real-time imaging. For example, the imaging device 30 may acquire images at a frame rate of 15 to 60 frames per second (i.e., 15 to 60 fps) in some non-limiting exemplary embodiments.

[0023] The imaging device 30 of FIG. 1 communicates with the processor 20 of the electronic processing device 18. As shown in FIG. 1, the imaging device 30 includes an X-ray imaging device, such as a C-arm imaging device, including an X-ray source 32 and an X-ray detector 34. In other embodiments, the imaging device may include another modality, such as ultrasound (US), computed tomography (CT), magnetic resonance imaging (MRI), or nuclear medicine imaging. In embodiments, the field of view of the imaging device 30 is large enough to encompass at least the tip 14 of the interventional instrument 12 and the vasculature leading to the occlusion or thrombus C. In embodiments including a non-fluoroscopic imaging device, the radiopaque tip 12 may be replaced with a tip of a type that is observable with the selected imaging modality. For example, if ultrasound is used for monitoring, the radiopaque tip 12 may be made of a material that provides good contrast in ultrasound images or may include a marker of such material. Images 35 captured by the imaging device 30 may be stored in a non-transitory storage medium 26.

[0024] 1 is configured to execute process 100, which may be a vascular diagnostic method, a vascular therapeutic method, an endobronchial diagnostic method, an endobronchial therapeutic method, or a surgical method, as described above. Non-transitory storage medium 26 stores instructions readable and executable by processor 2 to perform the disclosed operations (or process 100), including, for example, performing the vascular therapeutic method. In some examples, the method may be performed at least in part by cloud processing.

[0025] 2, with continued reference to FIG. 1, an exemplary method 100 is illustrated generally as a flow chart. To begin method 100, interventional instrument 12 is inserted into blood vessel V by a clinician and / or using robot 16.

[0026] In operation 102 of method 100, imaging device 30 performs interventional imaging during an interventional procedure to acquire a temporal sequence of images 35 (or imaging frames) of the movement of interventional instrument 12 (including tip 14) during the procedure. In some embodiments, this interventional imaging 102 is performed during the interventional procedure to provide visual guidance to a physician or clinician operating interventional instrument 12 or robot 16 to move interventional instrument 12 toward a target region of interest (ROI). In some embodiments, the target ROI is an occlusion or thrombus C within blood vessel V. The temporal sequence of images 35 includes interventional instrument 12 (and tip 14 of interventional instrument 12) and a portion of the anatomical structure in which interventional instrument 12 is located (e.g., a portion of blood vessel V or vasculature leading to the target ROI). In some embodiments, the field of view of the images encompasses both the position of tip 14 and the target ROI, while in other cases the field of view encompasses the position of tip 14 and a portion of the vasculature between the current position of tip 14 and the target ROI, but not the target ROI itself (at least until tip 14 is navigated sufficiently close to the region of interest). The temporal sequence of images 35 is transferred to processing unit 18.

[0027] In operation 104 of method 100, processor 20 identifies a location of interventional instrument 12 within a portion of the anatomical structure in the temporal sequence of images 35 (e.g., within a blood vessel V included in the temporal sequence of images 35). In some embodiments, processor 20 may identify the location of interventional instrument 12 by identifying a location of a tip 14 (e.g., a radiopaque tip) of the interventional instrument within a portion of the anatomical structure in the temporal sequence of images 35.

[0028] In operation 106 of method 100, processor 20 identifies the location of a target ROI within the anatomical structure relative to the location of interventional instrument 12 (e.g., tip 14). In some embodiments, processor 20 may generate a path from the location of tip 14 to the location of the target ROI. It will be understood that the order of operations 104 and 106 can be swapped, or operations 104 and 106 can be performed simultaneously if sufficient computing power is available. The processor may identify anatomical features between the location of interventional instrument 12 and the location of the target ROI.

[0029] In operation 108 of method 100, processor 20 predicts a traversal time for interventional instrument 12 to traverse (pass) a portion of an anatomical structure (e.g., blood vessel V) and reach a target ROI (e.g., tip 14 reaching the target ROI). In some embodiments, processor 20 may determine a path between the location of interventional instrument 12 and the location of the target ROI. In some embodiments, processor 20 may predict the traversal time by estimating a distance of interventional instrument 12 (e.g., tip 14) relative to the target ROI over a series of imaging frames in the temporal sequence of images 35. In some embodiments, processor 20 may predict the traversal time by estimating a velocity of interventional instrument 12 (e.g., tip 14) relative to the target ROI over a series of imaging frames in the temporal sequence of images 35. In some embodiments, the velocity of interventional instrument 12 (e.g., tip 14) relative to the target ROI may be determined by the velocity of robot 16 traversing the path. In some embodiments, processor 20 may predict crossing times by detecting anatomical features of a portion of the anatomical structure (e.g., vessel tortuosity, bifurcations, vessel branches, vessel curvature, small vessel diameter, calcifications, lesions, etc.) and the location of anatomical features within the anatomical structure in a series of imaging frames in the temporal sequence of images 35 in predicting such time. In some embodiments, processor 20 may predict crossing times based on characteristics of the interventional device (e.g., size, shape, material composition, flexibility, etc.). These embodiments may be combined to predict crossing times by any combination of such distance, speed, anatomical features, and device characteristics. In some embodiments, processor 20 calculates at least one statistical measure (e.g., mean, median, standard deviation) of the estimated crossing time, which is calculated for the predicted crossing time and the confidence of the calculated measure(s) generated.

[0030] In some embodiments, for a given image in the temporal sequence of images, processor 20 may repeat operations 106, 108, and 110 for multiple different ROIs. For example, in the temporal sequence of images 35, processor 20 may determine a path from the position of tip 14 to the position of the target ROI and identify multiple consecutive ROIs along that path. Processor 20 may then predict a traversal time for each consecutive ROI along that path and combine each traversal time to predict the traversal time for the entire path from the position of tip 14 to the position of the target ROI. In this manner, processor 20 may estimate the time to traverse various “waypoints” along the path by considering particular anatomical features (e.g., bends, etc.) of the corresponding segment of the path (e.g., between the “waypoint” and the previous “waypoint”), particular characteristics (e.g., shape, etc.) of the interventional device as it traverses the corresponding segment of the path, the particular distance along the corresponding segment of the path, the distance of the corresponding segment of the path, and / or other factors.

[0031] 3 , in some embodiments, the time estimation operation 108 can be performed by utilizing a machine learning model (e.g., a neural network (NN)) 36 stored in the non-transitory storage medium 26 of the electronic processing device 12. The machine learning model 36 may be trained based on historical data, such as historical imaging data and historical patient data (e.g., intravascular imaging data and intravascular patient data), to estimate the traversal time of an interventional device (e.g., the tip 14 of the interventional device) reaching the target ROI. For example, the machine learning model 36 may be trained to associate relationships between features identified from the historical imaging data and / or historical patient data (e.g., distance from the tip position to the target ROI position, anatomical features from the tip position to the target ROI position, locations of anatomical features from the tip position to the target ROI position, device velocity indicated by the historical imaging data, device characteristics, etc.) to the corresponding traversal time for the device to travel from the tip position to the target ROI position. For example, the machine learning model may be trained to predict the traversal time of each segment of the path between the location of the interventional device and the target ROI based on the specific anatomical features of each segment (e.g., vessel curvature) and / or the interventional device's ability to adapt to the anatomical features (e.g., flexibility across vessel curvature). For example, the machine learning model may be trained to associate specific anatomical features with a "slowdown" in the device's navigation, which leads to an increase in traversal time. As shown schematically in FIG. 3 , the trained machine learning model 36 receives as input current images 120 of the procedure, including the interventional device and target ROI 122, and outputs a predicted traversal time for the interventional device 12 to reach the location of the target ROI 124. In some embodiments, the machine learning model 36 may be trained on a temporal sequence of images 35.

[0032] In operation 110, the processor 20 outputs the predicted traversal time, for example, to the display device 24. In some examples, the processor 20 generates a visualization 38 of the path from the position of the interventional instrument 12 to the target ROI and displays the visualization on the display device 24. The visualization 38 may include a visualization of the interventional instrument 12, the target ROI, a schematic path from the tip 13 to the target ROI, one or more measurements (e.g., distance, speed, etc.), anatomical features, device characteristics, etc.

[0033] Referring to FIG. 4 , an illustrative example of such visualization on a graphical user interface (GUI) 28 for display on the display device 24 is shown. In this example, the image is a digital subtraction angiography (DSA) image obtained by acquiring and subtracting images to obtain an image that highlights vascular structures. In some embodiments, the image is acquired using intravascular contrast, and in other embodiments, the image is acquired without intravascular contrast. In FIG. 4 , operations 106, 108, and 110 are repeated for multiple successive ROIs along a path (labeled “Path” in FIG. 4 ) of the interventional instrument 12 from an identified current position of the tip 14 of the interventional instrument 12 (“Current Device Position”) to a target ROI (“Target,” e.g., thrombus C), which is the final ROI of multiple successive ROIs along the path. Each ROI can be considered a “waypoint” along the path to the final target. The set of ROIs is labeled ROI1, ROI2, ..., ROI3, ... N It is written as ROI N is the final target, from the current position of the tip to each ROI i ("i" is an index) i (ROI i ) can then be expressed as ROI i from ROI i+1 The crossing time to i+1 (ROI i+1 )t i (ROI i) in the diagram. In this way, traversal times for traversing various "waypoints" (ROIs) along the route can be calculated or predicted, which can be converted to speed or velocity as distance / time. As shown in FIG. 4, such velocities can be annotated as traversal times in the visualization at the corresponding "waypoints" (optionally represented as "velocity" estimates). As can be seen in FIG. 4, very sharp turns in the vasculature are estimated to require a "very slow velocity," while other portions are labeled as "average velocity" or "slightly slower velocity." Of course, these are exemplary annotations, and other representations of the annotations are contemplated. FIG. 4 also shows the display "Distance to target: 23 mm" output by optional operation 11 of FIG. 2 and the display "Time to target: 30 seconds" output by operation 110 of FIG. 2. This image can be generated relative to the current image of real-time imaging 102 of FIG. 2, so that the visualization of FIG. 4 can be updated in real time (e.g., every few seconds or faster) to provide the physician with up-to-date estimates of time to target and traverse time, as indicated by feedback arrow 11.

[0034] Additionally, if a three-dimensional (3D) image 112 of the vasculature, such as a preoperative 3D computed tomography angiography (CTA) image or a preoperative 3D magnetic resonance angiography (MRA) image, is available, the distance from the tip to the ROI can be estimated in operation 114, and this information can also be displayed in operation 116. To estimate the current device tip position relative to the preoperative 3D image, registration between the preoperative 3D image and the intraoperative image data may need to be performed. If the 3D image is acquired intraoperatively and the device tip is visible in the 3D image, the registration step is not necessary. The estimated tip-to-ROI distance can be used to estimate the time it will take for the tip 14 to reach the ROI in operation 108.

[0035] Generally, imaging 102 is real-time imaging performed during an interventional procedure to provide imaging guidance to a physician guiding the interventional instrument through the vasculature to position tip 14 at a target ROI (e.g., thrombus C). The sequence of images therefore spans some or all of the time the interventional instrument is being inserted. Thus, the above-described operations 104, 106, 108, and 110 (and optionally 114 and 116) are optionally performed on successive images of this image sequence to update the estimated time between insertions of the interventional instrument. This is indicated in FIG. 2 by flow-back arrow 118.

[0036] example

[0037] The operation of system 10 will now be described in more detail. Imaging device 30 may comprise an interventional x-ray imaging system comprised of an x-ray tube 32 configured to generate x-rays and an x-ray detector 34 configured to acquire x-ray images 35. Examples of such systems include fixed monoplane and biplane C-arm x-ray systems, mobile C-arm x-ray systems, etc. X-ray imaging systems can generate both conventional fluoroscopic (x-ray) images and contrast-enhanced fluoroscopic images, including system 10.

[0038] In some embodiments, the location of the target region of interest and / or the location of the interventional device (e.g., tip) may be annotated by a user (e.g., via the touchscreen GUI 28). Alternatively, in other embodiments, the location of the target region of interest and / or the location of the interventional device (e.g., tip) may be automatically annotated by the processor 20. For example, the processor may automatically detect a target ROI (e.g., an aneurysm) in the temporal sequence of images 35 and annotate the detected target ROI. The annotation may be indicated using a bounding box, a centroid, binary segmentation, or the like. The annotation may be generated using any of the methods established in the art (traditional computer vision-based methods or deep learning-based methods), such as thresholding, region growing, template matching, level set, active contour modeling, neural networks (e.g., U-Net), manual or semi-automated methods, etc. Also, if the target location is not visible in the sequence of images, the target location can be provided as a label from a set of known labels (e.g., cerebral aneurysm). The location of the interventional device (e.g., tip 13) can be identified in the image 35 by, for example, segmentation, device tip detection, device tracking (EM tracking), shape sensing processes, neural networks (e.g., U-Net), etc.

[0039] Prior to use, the machine learning model (e.g., NN 36) can be trained with training data. For example, the model may estimate the traversal time for the tip 14 to reach the target ROI based on features identified in previous images of the procedure, such as anatomical features, device characteristics, distance to the target ROI, and device speed. The estimated traversal time is then compared to a ground truth value. The parameters of the model may be adjusted based on the difference between the estimated value and the ground truth value. These operations can be repeated until a stopping criterion is met.

[0040] To train the machine learning model, retrospective navigation data from a relatively large (N>>100) patient population (covering various anatomies, abnormalities, age groups, genders, BMIs, etc.) who have undergone any endovascular procedure may be received at processing unit 18. This retrospective navigation data may include, for example, two-dimensional (2D) angiographic image sequences of the vasculature including the identified device tip location and target ROI location. In some embodiments, both can be input to the model (e.g., NN) via input channels consisting of a binary mask of the device tip and / or target ROI, or a bounding box around the device tip or target ROI, or a Gaussian heat map centered on the device tip or target ROI, etc. Inputs to the model may also be pre- or intra-operative three-dimensional (3D) images of the vasculature, if available, which provide more accurate measurements of path length and the geometry of critical points; any available information that can be reliably collected and influence device navigation (e.g., gender, age, blood pressure, weight, cardiac health, smoking history, family health history, medical history, genomic data, etc.); or any available information about the device (e.g., speed) that can be obtained, for example, from the robotic navigation system 16. In another example, characteristics of the interventional device are considered to train the model. For example, a softer device may have a lower risk of perforation at high velocities but present increased difficulty maneuvering into vessel branches. Additionally, the shape of the tip 14 can affect the time to target (e.g., a “pigtail” guidewire reduces the risk of perforation of the system 10 at high velocities compared to a straight tip, and a curved tip is easier to maneuver into vessel branches than a straight tip).

[0041] The machine learning model (e.g., NN36) may be trained using at least one 2D intraoperative image sequence and any other optional information, such as device velocity provided from any 3D or robotic device manipulation system of the vasculature. In some embodiments, the multiple 2D images input to the model must come from a contiguous sequence (e.g., representing the same task or stage in the same procedure), but can include, for example, regular fluoroscopic and digital subtraction angiography (DSA) images from the same stage, images from different viewing angles (to provide additional 3D context), etc.

[0042] The NN36 may be a convolutional neural network (CNN), a temporal convolutional network (TCN), a recurrent neural network (RNN), a transformer, or any other suitable artificial NN (ANN). RNN-based implementations may use unidirectional or bidirectional LSTM (long short-term memory) architectures, etc. Temporal information as multiple images are input to the NN36 can enable more accurate predictions that are consistent across frames. The NN36 may include several fully connected or convolutional layers with pooling, normalization, dropout, nonlinear layers, etc. Additional information may be incorporated by appending it to the flattened feature layer before applying the nonlinearity (sigmoid, ReLU, etc.).

[0043] The output of the machine learning model is a prediction or estimate of the time to target, for example, a prediction of the time it will take to navigate the interventional instrument 12 along a path from its current location through the anatomy to the target ROI. The machine learning model may output a single output indicating the time to navigate the interventional instrument 12 along the path to the target ROI, or multiple outputs indicating the estimated time to navigate the interventional instrument 12 to multiple discrete points along the path to the target ROI. By training the machine learning model to estimate the time to navigate to several discrete points toward the ROI, the model can be trained to associate features from the images with information regarding the ease and duration of navigation (e.g., which features are associated with consistent speed reductions, which features may indicate scale, which path lengths are associated with time to target, etc.). In some embodiments, the output may be expressed as a percentage when the scale is unknown (i.e., when only 2D information is available) or in seconds (or other units) when the scale is known (i.e., when 3D information is also available).

[0044] The error is calculated by comparing the output produced by the machine learning model with the ground truth value using some loss function (e.g., L1 or L2 loss, Huber loss, log cosh loss, etc.), which is used to perform stochastic gradient descent to optimize the network weights. The ground truth value for time to target can be obtained by evaluating, for each input image frame in the training data, the time remaining until the device tip 14 reaches the target ROI. This can be estimated using the position of the device tip 14, the position of the target along with the frame rate at which the image sequence 35 was captured, anatomical features between those positions, device characteristics, etc.

[0045] Once trained, the machine learning system may be configured to receive at least one 2D angiographic image of an image sequence 35 including the interventional instrument 12 at a current stage of the procedure with a known instrument tip position. The location of the target ROI may be user-annotated or automatically annotated onto the image of the image sequence. Other data may also be received to estimate distance, velocity, anatomical features, device characteristics, etc., related to navigation of the interventional instrument 12.

[0046] In some embodiments, a machine learning model (trained with a dropout layer) may be run multiple times on the same input during inference to produce slightly different outputs (because dropout randomly drops out outputs from a specified number of nodes). These different outputs may be used to calculate the mean and variance in the time-to-target estimates. The variance may be used to indicate the level of confidence in the output (i.e., high variance indicates inconsistent network output and therefore low confidence, while low variance indicates consistent output and high confidence).

[0047] In some embodiments, the estimated time to navigate the device to several discrete points along the path to the ROI may be used to generate additional information by combining the output across frames. For example, by combining the time estimates across frames and calculating how the estimates are changing, an estimated velocity of the device's tip can be calculated. Similarly, evaluating the change in the estimate across frames can indicate the location of potential velocity degradation and the amount of velocity degradation.

[0048] The GUI 28 displays a 2D intraoperative sequence of the vasculature and overlays the estimated time to target either as a numeric value (for a single-value output) or as an annotated route to the target (for an output consisting of various discrete points along the path to the ROI). The route may be annotated with post-processed information (e.g., red where speed slowdown may occur and green otherwise, or red where low confidence is present and green where high confidence is present, etc.). The instantaneous or average velocity of the system 10 in each anatomical region may also be displayed.

[0049] In some embodiments, the machine learning model may be trained to directly output regions of velocity slowing. For example, in addition to estimating the time to target, the model may also generate an output the same size as the input image, with a heat map showing potential locations of velocity slowing (e.g., vessel tortuosity, bifurcation, small vessel diameter, etc.). This information can be derived directly from "ground truth" time-to-target information and can be used during training.

[0050] In some embodiments, if data from robot navigation by the robot 16 is used for training, estimated device velocity can be extracted from the robot 16 and used to calculate other parameters such as time to target and amount of velocity degradation. The network can also be trained based on both manual and robot navigation imagery, as robot navigation data may differ somewhat from manual navigation data. Training on both types of data is likely to produce a more generalizable machine learning model.

[0051] In some embodiments, if the image 35 includes multiple C-arm views at the same time (e.g., data acquired from a biplane system), this data can be used by the machine learning model to calculate a more accurate estimate of the time to the target or the time to a discrete point towards the target, as this data inherently provides more information about the path to the target and therefore increases the machine learning model's reliability in its predictions. The multiple views can be input as separate input channels to the machine learning model, or as separate inputs to a Siamese twin network architecture, in which parallel convolutional layers process the multiple views separately in early layers of the machine learning model and integrate the network weights in subsequent layers to provide a combined output.

[0052] In some embodiments, if the path of the interventional instrument 12 is not defined, the output of the machine learning model may be used to estimate an appropriate path from the current device position to the target ROI, i.e., estimated times to discrete points along the path to the target help inform how to reach the target ROI.

[0053] In some embodiments, instead of just using the device tip 14, the machine learning model uses information from the entire device that is visible in the image 35. This can provide the machine learning model with additional information during training, such as what curvatures in the device are associated with fast and abrupt movements of the device.

[0054] In some embodiments, the machine learning model is trained to directly output a confidence value. This confidence may here be related to the training error. This may allow the machine learning model to learn features in the image that typically lead to higher errors. For example, regions where blood vessels are shortened may typically generate high errors during training due to the ambiguity caused by the shortening, and therefore may be associated with a low confidence value.

[0055] In some embodiments, the post-processed estimates may be customized to each user, for example, the speed reduction estimate may be refined based on average device speed as navigation progresses (e.g., some users may navigate slower than others, either generally or due to a higher patient perforation risk).

[0056] In some embodiments, the output of the machine learning model may be used to control the autonomous robot 16. For example, an identified area of ​​reduced speed may be an area with highly tortuous or narrow vessels that must be carefully navigated and may be used to tell the robot 16 to slow down its movement.

[0057] In some embodiments, if 3D information is also available, the distance to the target can be calculated and this distance is displayed in the GUI 28.

[0058] In some embodiments, a baseline navigation time may be learned for different types of anatomical structures using data from experts. This baseline constitutes the time it takes to navigate to a target through different parts of the anatomy. If the user chooses to receive this information, the time to target can be compared to the baseline. This can help the trainee learn which parts of the anatomy are slow and potentially help isolate which techniques to practice. This can also alert the physician to unexpected behavior if they have been navigating for a while without realizing that expected progress is not being made.

[0059] The present disclosure has been described with reference to preferred embodiments. Modifications and alterations may occur to others upon reading and understanding the foregoing detailed description. It is intended that the exemplary embodiments be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. 1. A system for navigating an interventional instrument within a patient's anatomy, the system comprising: processor and memory, wherein the processor receiving one or more images including a portion of a patient's anatomy and an interventional device positioned within the portion of the anatomy; identifying a location of the interventional device within the portion of the anatomy and at least one anatomical feature of the portion of the anatomy from the one or more images; Identifying a target region of interest (ROI) in the anatomical structure; predicting an amount of time to navigate the interventional instrument from a position of the interventional instrument through the portion of the anatomy to a position of the target ROI based on the at least one anatomical feature of the anatomy; It is configured as follows: system.

2. 2. The system of claim 1, wherein the at least one anatomical feature of the portion of the anatomy comprises at least one of vascular tortuosity, bifurcation, vascular curvature, small vascular diameter, and vascular bifurcation in the portion of the anatomy.

3. The system of claim 1 , wherein the processor is further configured to predict the time based on at least one characteristic of the interventional device, including at least one of a size, a shape, and a material composition of the interventional device.

4. The processor further comprises: determining a path to navigate the interventional device from a location of the interventional instrument through the portion of the anatomy to a location of the target ROI; identifying a plurality of consecutive ROIs along the route and predicting a time to navigate to each of the plurality of consecutive ROIs; The system of claim 1 configured to:

5. The processor further comprises: generating a visualization of the path navigating the interventional instrument from a location of the interventional instrument through the portion of the anatomy to a location of the target ROI; and displaying the generated visualization of the path on a display device. The system of claim 4 configured to:

6. The processor further comprises: displaying an image of the one or more images; generating a graphical representation of the path superimposed on the displayed image; and superimposing on the graphical representation an annotation of the predicted time to navigate to each of the plurality of successive ROIs along the route; The system of claim 5 , configured to:

7. The processor further comprises: calculating at least one statistical measure of said predicted time; generating at least one confidence level for said at least one statistical measure; The system of claim 1 configured to:

8. The processor further comprises: estimating a distance from a position of the interventional instrument through the portion of the anatomical structure to a position of the target ROI based on the one or more images, and predicting the time to navigate based on the distance; The system of claim 1 configured to:

9. The processor further comprises: predicting a velocity of the interventional instrument from a position of the interventional instrument through the portion of the anatomical structure to a position of the target ROI based on the sequence of one or more images, and predicting the time to navigate based on the velocity; The system of claim 1 configured to:

10. 2. The system of claim 1, wherein the processor is configured to utilize a machine learning model trained to predict the time to navigate from a position of the interventional instrument through a portion of the anatomy to a position of the target ROI based on the at least one anatomical feature of the portion of the anatomy.

11. a robot controlled by the processor to move the interventional instrument through the portion of the patient's anatomy at a speed determined based on the predicted time; The system of claim 1 further comprising:

12. an imaging device in communication with the processor and configured to acquire the one or more images; The system of claim 1 further comprising:

13. The system of claim 1 , wherein a tip of the interventional instrument is radiopaque, and the processor is further configured to identify a location of the interventional instrument in the one or more images based on the tip.

14. 1. A method for navigating an interventional instrument within a patient's anatomy, comprising: receiving one or more images including a portion of a patient's anatomy and an interventional device positioned within the portion of the anatomy; identifying a location of the interventional instrument within the portion of the anatomy and at least one anatomical feature of the portion of the anatomy from the one or more images; identifying a target ROI within the anatomical structure; predicting an amount of time to navigate the interventional instrument from a position of the interventional instrument through the portion of the anatomy to a position of the target ROI based on the at least one anatomical feature of the anatomy; A method having the following.

15. 15. The method of claim 14, wherein the at least one anatomical feature of the portion of the anatomy comprises at least one of vascular tortuosity, bifurcation, curvature, small vessel diameter, and vessel bifurcation in the portion of the anatomy.

16. The method of claim 14 , wherein the time is further predicted based on at least one characteristic of the interventional device, including at least one of a size, a shape, and a material composition of the interventional device.

17. determining a path to navigate the interventional device from a location of the interventional instrument through the portion of the anatomy to a location of the target ROI; identifying a plurality of consecutive ROIs along the route and predicting a time to navigate to each of the plurality of consecutive ROIs based on the at least one anatomical feature of the portion of the anatomy; 15. The method of claim 14, further comprising:

18. estimating a distance from a location of the interventional instrument through the anatomical structure to a location of the target ROI based on the one or more images, and predicting a time to navigate based on the distance; The method of claim 14 further comprising:

19. predicting a velocity of the interventional instrument from a position of the interventional instrument through the portion of the anatomical structure to the position of the target ROI based on the sequence of one or more images, and predicting the time to navigate based on the velocity; The method of claim 14 further comprising:

20. When executed by a processor, the processor: receiving one or more images including a portion of a patient's anatomy and an interventional device positioned within the portion of the anatomy; identifying a location of the interventional instrument within the portion of the anatomy and at least one anatomical feature of the portion of the anatomy from the one or more images; identifying a target ROI within the anatomical structure; predicting an amount of time to navigate the interventional instrument from a position of the interventional instrument through the portion of the anatomical structure to a position of the target ROI based on the at least one anatomical feature of the portion of the anatomical structure; A non-transitory computer-readable storage medium having stored thereon a computer program having instructions.

Images