Tracking of elongated devices

Abstract

A system for tracking a coaxial elongate device within an anatomical structure includes a controller having a first interface and a second interface. The coaxial elongate device includes an outer elongate device and an inner elongate device driven by a motor. The first interface is configured to receive images including at least one of the outer elongate device or the coaxial inner elongate device. The second interface is configured to receive encoded data representing the driving of the coaxial inner elongate device by the motor. The controller is configured to receive images including the outer elongate device and at least one of the coaxial inner elongate device retracted within the outer elongate device, receive the encoded data, determine a position of the outer elongate device, and estimate a position of the coaxial inner elongate device retracted within the outer elongate device based on the position of the outer elongate device and the encoded data.

Description

[Technical Field]

[0001] The present invention relates to tracking of elongate devices. [Background technology]

[0002] In endovascular interventions, fluoroscopic imaging is used to independently manipulate coaxial intravascular devices. A coaxial intravascular device includes an inner device, such as a guidewire, and an outer device, such as a catheter. The inner device can be retracted into and advanced from the outer device. The opacity of the catheter makes it difficult to observe the position of the guidewire tip as it is retracted into the catheter. Furthermore, this creates a cavity in the outer device where blood can collect and clot. Furthermore, as the guidewire is extended, there is a risk of damaging the surrounding blood vessel. Summary of the Invention [Problem to be solved by the invention]

[0003] Not knowing the location of the guidewire tip can be particularly problematic if the physician fails to realize that the guidewire remains fully retracted, creating a cavity within the outer device for an extended period of time where blood can collect and clot. Re-advancing the guidewire can result in the formation of a thrombus, which is potentially life-threatening. Additionally, during guidewire exchange, the guidewire can be rapidly advanced within the outer device, advancing it out of the outer device at an undesirable rate and potentially damaging the surrounding blood vessel. For example, advancing the guidewire at an undesirable rate and without information about the location of the guidewire tip may perforate the endothelium. [Means for solving the problem]

[0004] According to one aspect of the present disclosure, a system for tracking a coaxial elongate device within an anatomical structure includes a controller having a first interface and a second interface. The coaxial elongate device includes an outer elongate device and an inner elongate device driven by a motor. The first interface is configured to receive images including at least one of the outer elongate device or the coaxial inner elongate device. The second interface is configured to receive encoded data representing the driving of the coaxial inner elongate device by the motor. The controller is configured to receive images including the outer elongate device and at least one of the coaxial inner elongate device retracted within the outer elongate device, receive the encoded data, determine a position of the outer elongate device, and estimate a position of the coaxial inner elongate device retracted within the outer elongate device based on the position of the outer elongate device and the encoded data.

[0005] According to this aspect of the disclosure, further optional embodiments are disclosed in claims 2 to 15.

[0006] According to another aspect of the present disclosure, a controller for tracking a coaxial elongate device within an anatomical structure includes a memory storing instructions, a processor executing the instructions, a first interface, and a second interface. The first interface is configured to receive images including an outer elongate device with a coaxial inner elongate device driven by a motor. The second interface is configured to receive encoded data corresponding to the driving of the coaxial inner elongate device by the motor. When executed by the processor, the instructions cause the system to receive images including at least one of the outer elongate device and the coaxial inner elongate device, receive the encoded data, determine a position of the outer elongate device, and estimate a position of the coaxial inner elongate device retracted within the outer elongate device based on the position of the outer elongate device and the encoded data.

[0007] According to this other aspect of the present disclosure, further optional embodiments are disclosed in claims 17-18.

[0008] According to another aspect of the present disclosure, a method for tracking a coaxial elongate device in an anatomical structure includes receiving, via a first interface, an image including at least one of an outer elongate device or a coaxial inner elongate device driven by a motor, receiving, via a second interface, encoded data corresponding to the driving of the coaxial inner elongate device by the motor, determining a position of the outer elongate device, and estimating a position of a coaxial inner elongate device retracted within the outer elongate device based on the position of the outer elongate device and the encoded data.

[0009] According to this other aspect of the present disclosure, further optional embodiments are disclosed in claims 20-22.

[0010] The illustrative embodiments are best understood from the following detailed description when read in conjunction with the accompanying drawings. It is emphasized that the various features are not necessarily drawn to scale. In fact, dimensions may be arbitrarily increased or decreased for clarity of discussion. Wherever applicable and practical, like reference numerals refer to like elements. [Brief explanation of the drawings]

[0011] [Figure 1A] 1 illustrates a system for tracking elongated devices according to a representative embodiment. [Figure 1B] 1 illustrates an elongated device for elongated device tracking, according to a representative embodiment. [Figure 2] 1 illustrates a method for tracking an elongated device according to a representative embodiment. [Figure 3] 1 illustrates a user interface for tracking an elongated device according to a representative embodiment. [Figure 4] 10 illustrates an X-ray image for tracking an elongated device according to a representative embodiment. [Figure 5A] 1 illustrates a user interface for tracking an elongated device, according to a representative embodiment. [Figure 5B] 1 illustrates a user interface for tracking an elongated device, according to a representative embodiment. [Figure 6]1 illustrates a computer system on which a method for tracking elongated devices is implemented, according to another representative embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] In the following detailed description, for purposes of explanation and not limitation, exemplary embodiments disclosing specific details are described to provide a thorough understanding of embodiments according to the present teachings. However, other embodiments consistent with the present disclosure that depart from the specific details disclosed herein are within the scope of the appended claims. Descriptions of known systems, devices, materials, methods of operation, and methods of manufacture may be omitted to avoid obscuring the description of exemplary embodiments. Nevertheless, systems, devices, materials, and methods within the purview of those skilled in the art may be within the scope of the present teachings and used in accordance with exemplary embodiments. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. The definitions and explanations of terms herein are indicative of the technical and scientific meaning of the terms as commonly understood and accepted in the art of the present teachings.

[0013] Terms such as first, second, and third may be used herein to describe various elements or components, but it will be understood that these elements or components should not be limited by these terms. These terms are used only to distinguish one element or component from another. Thus, a first element or component discussed below could be referred to as a second element or component without departing from the teachings of the inventive concept.

[0014] As used in the specification and the appended claims, the singular forms of the terms "a," "an," and "the" are intended to include both the singular and the plural forms unless the context clearly dictates otherwise. Additionally, the terms "have," "comprise," and / or similar terms, when used herein, specify the presence of stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0015] Unless otherwise specified, when an element or component is said to be "connected," "coupled," or "adjacent" to another element or component, it is understood that the element or component can be directly connected or coupled to the other element or component, or there may be intervening elements or components. That is, these and similar terms encompass the cases where one or more intermediate elements or components may be used to connect the two elements or components. However, when an element or component is said to be "directly connected" to another element or component, this only encompasses the cases where the two elements or components are connected to each other without any intermediate or intervening elements or components.

[0016] The present disclosure, therefore, is intended to derive, through one or more of its various aspects, embodiments, and / or particular features or subcomponents, one or more of the advantages as particularly set forth below.

[0017] As described herein, visual cues can be incorporated into the fluoroscopic images so that the physician can clearly monitor the position of the guidewire tip within the catheter.

[0018] FIG. 1A illustrates a system 100 for tracking elongated devices according to a representative embodiment.

[0019] 1A is a system for tracking an elongated device and includes components that may be provided together or distributed. System 100 includes an imaging device 101, an elongated device 120, a robot 130, a motor 140, and a display 180. Robot 130 and motor 140 may comprise separate elements or integrated elements or systems. Computer 110 includes a first interface 111, a second interface 112, and a controller 150. Controller 150 includes at least a memory 151 that stores instructions and a processor 152 that executes the instructions.

[0020] The imaging device 101 may comprise an X-ray device or system. The imaging device 101 may be configured to provide a two-dimensional X-ray image including the elongated device 120, including the tip of the elongated device.

[0021] The computer 110 may comprise a workstation, laptop, desktop, dedicated computer, or virtual machine running on a local or remote cloud service. The computer 110 is used to control and / or coordinate the operation of the imaging device 101, the motors 140, the robot 130, and the display 180. A computer that can be used to implement the computer 110 is shown in FIG. 6, although the computer 110 may include more or fewer elements than those shown in FIG. 1 or FIG. 6. The computer 110 includes at least a first interface 111, a second interface 112, and a controller 150. The first interface 111 interfaces the computer 110 with the imaging device 101. The second interface 112 interfaces the computer 110 with the motors 140 and / or the encoders 145. Another interface (not shown or labeled) connects the computer 110 to the display 180. The first interface 141 and the second interface 142 may include ports, adapters, and / or other types of suitable hardware configured to accept cable inputs from cables connecting to the imaging device 101, the motor 140 and / or the encoder 145, and the display 180.

[0022] Controller 150 includes at least memory 151 for storing instructions and processor 152 for executing instructions. In some embodiments, multiple different elements of system 100 of FIG. 1 may include a controller, such as controller 150. Controller 150 may include interfaces, such as a third interface, a fourth interface, a fifth interface, and a sixth interface. One or more interfaces of controller 150 may include a port, a disk drive, a wireless antenna, or other type of receiver circuitry connecting controller 150 to other electronic elements of computer 110 or external to computer 110. Controller 150 may be configured to receive, via first interface 111, images including image data of at least one of an outer elongate device, an inner elongate device retracted within the outer elongate device. One or more of the interfaces of computer 110 or the interfaces specific to controller 150 may include user interfaces such as buttons, keys, a mouse, a microphone, a speaker, a display separate from display 180, or other elements that a user can use to interact with computer 110 or, in particular, controller 150 to input commands and receive output.

[0023] In some embodiments, the interface of the controller 150 may include a user interface used by a user to control the robot 130. The user interface may provide tactile feedback by vibrating when the guidewire crosses an alignment position with the catheter. In some embodiments, another element, such as the robot 130, may vibrate when alignment is near. In other embodiments, an interface on the robot 130 or motor 140 may vibrate or light up when alignment is near.

[0024] Each time the guidewire is retracted into the catheter by more than X amount, audio feedback is added, for example 1 mm. The audio feedback can be a beep, a verbal reading of the mm, or other special sound effect.

[0025] Memory 151 may store an association between the encoded data from encoder 145 and pixel position data along the catheter as the outer device of elongate device 120. Processor 152 may estimate the position of the tip of the guidewire as the inner device of elongate device 120 along the outer device when the inner device is retracted by motor 140. Processor 152 may further generate and transmit overlay information to display 180 to superimpose a (highlighted) indicator on the two-dimensional X-ray image. When the tip of the inner device is fully retracted into the outer device, the superimposed indicator may be superimposed on the two-dimensional X-ray image along the outer device at the estimated position of the tip of the inner device. The superimposed indicator is located at a position on the two-dimensional X-ray image based on the stored association between the encoded data from encoder 145 and pixel position data along the catheter as the outer device of elongate device 120, and the encoded data from encoder 145.

[0026] Controller 150 may directly perform some of the operations described herein and indirectly perform other operations described herein. For example, controller 150 may indirectly control operations, such as by generating and transmitting content to be displayed on display 180. Controller 150 may be configured to provide display information such that display 180 can overlay the outer elongate device to indicate the estimated position of the inner elongate device. Controller 150 may directly control other operations, such as logical operations performed by processor 152 executing instructions from memory 151 based on input received from electronic elements and / or from a user via an interface. Thus, processes performed by controller 150 when processor 152 executes instructions from memory 151 may include steps not directly performed by controller 150. In some embodiments, controller 150 may perform operations by executing instructions received from a cloud, such as a server in a data center.

[0027] The elongated device 120 may comprise an intravascular device, such as a coaxial combination of two intravascular devices. In the description herein, the elongated device 120 is primarily described in the context of a combination of a catheter as an outer device and a guidewire as an inner device. The movement of the guidewire as an inner device within the catheter as an outer device may be controlled by the robot 130 under the power of the motor 140. The controller 150 may be configured to control the movement of at least one of the outer elongated device and the inner elongated device. In some embodiments, the controller 150 may automatically control the speed of the elongated device 120 by instructing the motor 140 to reduce power and / or the robot 130 to slow movement when the guidewire is within the catheter, for example, as the guidewire approaches an alignment point during advancement.

[0028] Robot 130 may comprise a controllable device driven by motors 140 and movable in one or more degrees of freedom. Robot 130 is configured to drive elongated device 120 at least forward and backward as one degree of freedom. Robot 130 may be configured to rotate elongated device 120 as a second degree of freedom. Robot 130 may be configured to move elongated device 120 vertically up and down as a third degree of freedom and left and right as a fourth degree of freedom.

[0029] Motor 140 may comprise an electric motor, a linear motor, a precision stepper motor, or a servo motor ("servo"). In some embodiments, robot 130 and motor 140 may comprise an integrated unit referred to as either or both the robot and / or the motor. Motor 140 is configured to drive robot 130 to drive elongated device 120 in one or more degrees of freedom. A robotic system may include, for example, motor 140 and controller 150, or a combination of robot 130 and motor 140 in addition to controller 150.

[0030] Motors 140 are also provided with encoders 145 that track and encode the movement of robot 130 and elongated device 120 in one or more degrees of freedom. Encoders 145 output encoded data reflecting the amount of movement of robot 130 and elongated device 120 in each degree of freedom.

[0031] The display 180 may be local to the controller 150 or may be remotely connected to the controller 150. The display 180 may be connected to the controller 150 via a local wired interface, such as an Ethernet (Registered trademark) cable, or via a local wireless interface, such as a Wi-Fi connection. The display 180 may interface with other user input devices through which a user can input commands, including a mouse, keyboard, thumbwheel, etc. The display 180 may be a monitor, such as a computer monitor, a display on a mobile device, an augmented reality display, a television, an electronic whiteboard, or another screen configured to display electronic images. The display 180 may include one or more input interfaces, such as those described above, that may connect to other elements or components, and an interactive touchscreen configured to display prompts to the user and collect touch input from the user. In some embodiments, if the guidewire is not within the current field of view, the display 180 may display a warning, such as an arrow or text near the intersection of the catheter and the edge of the two-dimensional x-ray image. The controller 150 may be configured to calculate the approximate distance in pixels from the tip of the outer elongate device to the tip of the inner elongate device. In some embodiments, the display 180 may display the distance of the position of the tip of the guidewire relative to the tip of the catheter, for example, via a numerical display, a separate linear display, or a color overlay of the section of the catheter without the guidewire, i.e., the empty space between the tip of the catheter and the tip of the guidewire.

[0032] Controller 150 may include interfaces, such as a first interface, a second interface, a third interface, and a fourth interface. One or more interfaces may include a port, a disk drive, a wireless antenna, or other type of receiver circuitry that connects controller 150 to other electronic elements. One or more interfaces may include a user interface, such as buttons, keys, a mouse, a microphone, a speaker, a display separate from display 180, or other elements that a user can use to interact with controller 150 to input commands and receive output.

[0033] As described above, system 100 may include a system for tracking a coaxial elongate device, such as elongate device 120. The coaxial elongate device may include an outer elongate device and an inner elongate device driven by a motor, such as motor 140. System 100 may include a computer with a controller, such as computer 110 with controller 150. The computer may include a first interface, such as first interface 111, configured to receive images including image data of at least one of the outer elongate device or the coaxial inner elongate device, and a second interface, such as second interface 112, configured to receive coded data representing actuation of the coaxial inner elongate device by the motor. The controller of system 100 is configured to receive images including at least one of the outer elongate device and the coaxial inner elongate device retracted within the outer elongate device, receive the coded data, determine a position of the outer elongate device, and estimate a position of the coaxial inner elongate device retracted within the outer elongate device based on the position of the outer elongate device and the coded data.

[0034] FIG. 1B illustrates an elongate device for elongate device tracking, according to a representative embodiment.

[0035] 1B, elongate device 120 is shown to include an outer elongate device 124 along with an inner elongate device 122. Outer elongate device 124 may comprise a catheter, and inner elongate device 122 may comprise a guidewire.

[0036] 1B, in some embodiments, elongate device 120 may be tracked without encoder 145. For example, external means for encoding the relative position between the guidewire and catheter may be used. In some embodiments, optical shape sensing may be used. In other embodiments, markers and cameras on the proximal section of elongate device 120 may be used to track movement by robot 130 and motor 140.

[0037] FIG. 2 illustrates a method for tracking an elongated device according to a representative embodiment.

[0038] The method of FIG. 2 may be performed by a system 100 that includes a controller 150.

[0039] In S210, pre-calibration is performed. The pre-calibration may be performed based on timing of calibration of the encoded data when the tip of the guidewire as the inner device and the tip of the catheter as the outer device are aligned in the two-dimensional X-ray image. The association between the encoded data along the catheter as the outer device and the pixel position data may be based on the pre-calibration of the encoded data, which includes determining the encoded data when the two tips are aligned in the image.

[0040] Pre-calibration may be performed using computer 110 and display 180 of FIG. 1 . For example, the user interface to computer 110 may include a button that the user can press when the tip of the guidewire is believed to be at the end of the catheter based on the two-dimensional x-ray image currently displayed on display 180. In some embodiments, when the user presses such a button based on the two-dimensional x-ray image currently displayed on display 180, an offset between the guidewire tip and the catheter tip may be calculated by controller 150. The button may include an interface, such as a third interface, configured to accept input for pre-calibrating the position of the tip of the outer elongate device and the estimated position of the tip of the inner elongate device.

[0041] In some embodiments, controller 150 may perform calibration automatically, without direct user instruction, based on interpretation of the catheter end in a two-dimensional x-ray image on display 180 and based on encoded data relative to the guidewire from encoder 145. The tip's relative position (tip alignment) with respect to encoder 145 may be calibrated each time a new instance of elongated device 120 is introduced. Robot 130 or another element of system 100 may have dedicated hardware for this. Alternatively, x-ray images and image processing may be used to detect the event when the device tip is aligned in the image. In this case, the offset in robot encoder space may be recorded. This may be achieved by monitoring the guidewire extension length in pixels during guidewire retraction or catheter advancement, or both, until the guidewire tip disappears within the catheter. Alternatively, controller 150 may detect when the guidewire is first seen during advancement out of the catheter or when the catheter is retracted.

[0042] In S220, a two-dimensional X-ray image including the image data is generated and transmitted. S220 may be repeatedly performed in parallel with some or all of steps S230 through S270 of FIG.

[0043] In S230, the intravascular device is driven and data corresponding to the driving of the intravascular device is encoded. S230 may be repeatedly executed in parallel with some of S220 and all of the steps from S240 to S270 in FIG.

[0044] At S240, the two-dimensional X-ray image and the encoded data are received. The two-dimensional X-ray image is received at the computer 110 from the imaging device 101 via the first interface 111. The encoded data is received at the computer 110 from the encoder 145 via the second interface 112.

[0045] At S250, a representation of the displacement is output based on the encoded data. This representation may include an indicator alert of guidewire displacement relative to displacement or the absence of displacement. The displacement may be an estimate of movement in one or more degrees of freedom of elongated device 120 based on the number of rotations of motor 140, based on the amount of time motor 140 is driving elongated device 120, based on the speed at which motor 140 is driving elongated device 120, and / or based on any other factors that may be considered by encoder 145. The representation of the displacement may be the amount of displacement in each of the one or more degrees of freedom.

[0046] In S260, the position of the outer intravascular device is determined. Controller 150 may calculate a set of order line segments representing the catheter of elongate device 120 within the two-dimensional X-ray image. The position determined in S260 may be determined based on an image analysis program implemented by controller 150. The association between the coded data received in S240 and the position of the pixel data of the two-dimensional X-ray image received in S240 may be based on pre-calibration.

[0047] In S270, the position of the coaxial inner endovascular device is determined, which may involve several sub-processes, starting with encoded data from encoder 145 to determine robot coordinates from the absolute displacement of the guidewire based on movement by robot 130, then including calibration to pixel coordinates to calculate the approximate distance in pixels from the tip of the device, and then applying the approximate distance in pixels to the catheter line segment of the catheter to locate the tip of the guidewire on the image.

[0048] The determination in S270 may include registering two or more coordinate systems with each other. For example, the robot 130 and motors 140 may reference a first three-dimensional coordinate system, and the imaging device 101 may reference a second three-dimensional coordinate system. The two coordinate systems may be registered with each other to enable transformation of the encoded data relative to the two-dimensional X-ray image in the three-dimensional coordinate system of the imaging device 101.

[0049] The determination in S270 may include applying a trained artificial intelligence model to the images and encoded data to estimate the position of the coaxial inner elongate device. In some embodiments, the association between the encoded data from encoder 145 and pixel position data along the outer device from the image analysis performed by controller 150 is based on a calibration of a scale factor between the image and the encoded data. The calibration may be performed in S210 as described above and may include determining the encoded data when the two tips are aligned in the image. The scale factor is constant across the image when the elongate device 120 is parallel to the detector plane from the two-dimensional x-ray detector; otherwise, the scale factor changes for each position of the catheter and guidewire of the elongate device 120.

[0050] Active calibration, associated with S270, may be performed throughout most or all of the intervention to enhance the user experience and provide improved accuracy of the location of the guidewire tip within the catheter. Active calibration may be performed to calibrate robot-encoded data units from encoder 145 to image units based on two-dimensional x-ray images from imaging device 101. Image analysis algorithms may detect the tip of the catheter within the two-dimensional x-ray images and provide a translation of the location of the catheter tip within the images. The corresponding location of the guidewire tip from the guidewire movement may be determined from the encoded data from encoder 145. Comparison results in a dimensional ratio that can be applied to estimate the location of the guidewire on a segmented representation of the catheter on display 180. The pixel calibration algorithm may be performed in real time and is further described below with reference to FIG. 4.

[0051] In S280, the estimated position of the coaxial inner intravascular device is displayed. When the tip of the guidewire of elongate device 120 is retracted into the catheter of elongate device 120, a highlight or other indicator may be generated and superimposed on the two-dimensional X-ray image, e.g., the location of the guidewire tip, over a representation of the catheter of elongate device 120 in the image. For example, display 180 may output an indication in the image data of the image that the guidewire as the inner elongate device is fully retracted into the outer elongate device. The position of the guidewire tip is established in S270 based on the encoder value in the encoded data output by encoder 145 in S240. Controller 150 may be configured to automatically estimate the position of the inner elongate device when retraction of the inner elongate device into the outer elongate device is detected.

[0052] The display of the guidewire tip (or any other portion) in S280 may be performed at a limited time during the intervention, such as once the guidewire tip is fully retracted into the catheter, or may be performed throughout the intervention. In some embodiments, controller 150 may selectively determine when to display the guidewire tip (or other portion) based on, for example, proximity to the catheter tip (or other portion), so as to display the guidewire tip even if the guidewire tip (or other portion) protrudes slightly from the catheter.

[0053] In some embodiments, the display of the guidewire tip may vary based on the distance to the catheter tip. For example, the guidewire tip may be highlighted using different sizes, shapes, and / or colors based on the distance to the tip. As an example, the highlighting may vary from blue to green based on the distance from tip alignment. As another example, the highlighting may vary from a circular shape at the alignment position to a gradually rectangular shape as the distance increases. A rectangular shape may be used in embodiments where the calibration is relatively coarse, and the rectangular shape may cover the potential area where the tip may be present. In some embodiments, the overlay may vary based on the eigenvalues / eigenvectors of the system covariance matrix. In some embodiments, the circular highlighting may vary from a small diameter at alignment to a larger diameter as the distance increases from alignment.

[0054] FIG. 3 illustrates a user interface for tracking an elongated device according to a representative embodiment.

[0055] User interface 381 in Figure 3 shows the profile of the elongate device being imaged in the anatomy. Visual cues are provided to the physician via user interface 381 labeled B and C. As a result, the physician can clearly monitor the position of the guidewire tip within the catheter to avoid errors. The tip position is highlighted as shown.

[0056] In the first of the user interfaces 381, labeled A, the guidewire is extended beyond the catheter as usual. In the second and third of the user interfaces 381, labeled B and C, the guidewire is retracted into the catheter, and the estimated location of the tip of the guidewire is highlighted by a highlighted visual overlay.

[0057] Alternatively (not shown), the user interface may include two or more (X-ray) two-dimensional images taken from different viewing angles, optionally simultaneously, showing the anatomical structure and catheter and / or guidewire from two or more different angles, One or more of the images may include a superimposed indicator of the tip (or other portion) of the guidewire in accordance with the present disclosure.

[0058] FIG. 4 illustrates an X-ray image for tracking an elongated device according to a representative embodiment.

[0059] User interface 481 in Figure 4 shows an x-ray image with potential paths and positions of elongated devices within the anatomy. In Figure 4, one endovascular device is shown positioned in the aortic arch and descending aorta.

[0060] As mentioned above, active calibration, associated with S270 of FIG. 2, may be performed throughout most or all of the intervention. Active calibration may be performed to calibrate robot-encoded data units from encoder 145 to image units based on two-dimensional x-ray images from imaging device 101. Image analysis algorithms may detect the catheter tip (or other predetermined portion) in the two-dimensional x-ray images and provide a translation of the position of the catheter tip (or other predetermined portion) in the images. The corresponding position of the guidewire tip (or other predetermined portion) from the guidewire movement may be determined from the encoded data from encoder 145. The comparison yields a dimensional ratio, which may be applied to estimate the position of the guidewire on a segmented representation of the catheter on display 180.

[0061] Calibrating the position of the tip (or other predetermined portion) of the guidewire relative to the two-dimensional x-ray image involves two distinct complexities that are addressed by controller 150. The first is that if elongated device 120 is not generally parallel to the imaging plane of the two-dimensional x-ray image, the relative depth of elongated device 120 will change for different positions of elongated device 120. The second is the curvature of the catheter and any bends that need to be taken into account when determining the translation of the guidewire within the catheter.

[0062] In FIG. 4, p0t is the tip of the catheter at time t. The catheter of the elongated device 120 may be described by a list of points p0t, p1t, ..., pnt. The image robot encoder scale factors along the centerline of the vessel are denoted as s0, s1, ..., sk. A transformation scale factor along the centerline of the outer elongated device may be assigned, which provides a variable transformation between encoded data and pixels. The scale factor provides the number of robot encoder counts per pixel, which may vary based on the alignment of the elongated device 120 relative to the imaging plane, particularly in two-dimensional images such as two-dimensional x-ray images.

[0063] To calculate the image position of the guidewire tip, the catheter and guidewire tip are calibrated, and the guidewire can be retracted from the catheter tip in CT counts. From the encoded data, the controller 150 can count the amount of retraction of the inner elongate device relative to the tip of the outer elongate device. The position along the catheter trajectory corresponding to the guidewire tip is then found by a first algorithm. The first algorithm for locating the guidewire tip involves first finding the centerline closest to the first point in a list of points and then adjusting it based on a scale factor. This process can be performed for each point in the list of points, or for a limited set of points closest to the end that specifies the catheter tip. The centerline of the vessel can be made available in advance. Alternatively, the centerline of the vessel in the two-dimensional X-ray image can be dynamically calculated while inserting the catheter. The scale factor along the vessel central axis can be iteratively calculated through a second algorithm.

[0064] The second algorithm uses displacements from the encoded data from encoder 145 to update the scale factor for each point in the list of points along the centerline of the catheter. The second algorithm is dynamically applied as a function of spatial and temporal parameters and is used to create a mapping of the calibration in at least the spatial domain relative to the location of the calibration on the image or within the vasculature.

[0065] The first algorithm may be run continuously during the manipulation of the guidewire during the intervention. Scale factors corresponding to catheter points are initialized as soon as the catheter tip passes through a region of the vasculature, so that the scale factors needed to estimate the position of the guidewire within the catheter are well-defined. Several different schemes may be used to update the scale factors. For example, an advanced approach may consider the direction and rate of change of the encoder to improve estimation insofar as the scale factors tend to be underestimated during push-out due to vessel buckling and overestimated during device retraction due to slack removal.

[0066] In some embodiments, for any algorithm step, the relative width in pixels of at least one elongate device may be used by the system to additionally or alternatively estimate the scaling changes in the image. In particular, the actual width of said at least one device, if known (e.g. in mm), can be mapped to the pixel width of the device in the image to directly estimate the scaling factor, or can be used as supplementary information in the scaling factor algorithm.

[0067] In additional or alternative embodiments, the device may also include opaque reference points (i.e., opaque to imaging) of known size and / or known displacement on the elongate device, and the difference between the actual geometry of these reference points and the imaged geometry may be used to estimate the scaling factors. If the first scaling algorithm lacks sufficient information, the scaling factors may be derived directly from the image and the assumption that the device is placed at the isocenter of the imaging space using standard projective geometry techniques.

[0068] In additional or alternative embodiments, the system is configured to estimate the direction of curvature of the (curved) outer device and use this estimate to refine the scaling factors for the curved sections so that the tip of the inner device is shown in the correct visual position. An external system that estimates the orientation of the curved device relative to the image is used to enhance information about the device's sections in the image. That is, the relative positions of the distal sections of the curved device in-plane and out-of-plane adjust the scaling factors. Certain sections are moving away from the X-ray detector and should have proportionally larger scaling factors than sections of the device positioned relatively closer to the detector (a given physical movement of the device traverses more pixels in the image).

[0069] If a multi-planar X-ray imaging system (such as a bi-plane) is available, a scaling factor algorithm can be used on each image. The user interface can be applied to any of the image sources used in the algorithm, or can be automatically selected as the primary user interface based on the most accurate scaling factor, availability of scaling factors, or user preference. If the multi-planar X-ray imaging system is co-registered, the device position in 3D space can be reconstructed using standard multi-view projection geometry methods. The scaling factor can be back-calculated for the device in the image.

[0070] If the motion drive of the elongated device is slipping, occasional resetting of the absolute position of the device can be done by referring to visually detected events in the image of the inner device moving in and out of the tip of the outer device.

[0071] The controller 150 can determine if the guidewire is retracted based on the robot coordinates. As the guidewire is retracted, a scale factor is applied to the encoder-based retraction distance to yield a retraction length in pixels. This length is used to find the pixel location along the catheter line segment from the tip of the catheter using a Euclidean method. At this location, a glyph / graphic blend with the x-ray image pixel is performed to indicate the estimated guidewire position, as shown in FIG. 3.

[0072] FIG. 3 or FIG. 5 illustrates a user interface for tracking an elongated device according to a representative embodiment.

[0073] In some embodiments based on Figure 3, the display of the guidewire tip may change based on the estimated relative position of the device tip in or out of the image plane within the outer device. For example, the system may be configured so that the color intensity is not the same when the inner device is moving away from or towards the viewer, or the size or shape of the display is changed based on how far it deviates from plan (due to perspective) as it moves within the outer device. The relative in or out of plane of the device is directly proportional to the scaling factor.

[0074] Additional embodiments based on Figure 3 can include overlay graphics or context placed in place of functional information about the inner device within the outer device, for example, articulation points or regions of different stiffness of the inner device, or device-specific markers such as the start and end of a stent or balloon, or other diagnostic and therapeutic device information. The information can be virtual, such as a gradient indicating equal-length sections of the device.

[0075] The user interfaces 581 in Figures 5A and 5B show a side view of the elongated device being imaged within the anatomy. In Figures 5A and 5B, the position of the guidewire is overlaid on the most recent x-ray image, even when the imaging device 101 is off, as shown in the first of the user interfaces 581 labeled "A." When a new acquisition is obtained, an overlay is rendered on the new image, as shown in the second of the user interfaces 581 labeled "B."

[0076] 5A and 5B, the controller 150 is used to estimate the position of the guidewire within the catheter even when X-ray is not enabled. The most recent X-ray image may be used for feedback, and highlighting may be drawn along the segmented device as if fluoroscopy were active. If the catheter is moved, as observed through the encoded data from the encoder 145, a warning may be displayed on the display 180, and / or another two-dimensional X-ray image may be taken to update the image.

[0077] In some embodiments similar to those based on Figures 5A and 5B, the fluoroscopy frame rate may be reduced, for example by a factor of two, when the guidewire is retracted into the catheter, and updates indicating the estimated guidewire tip may be performed at the original frame rate.

[0078] Many variations are within the scope of the above teachings. For example, some elongate devices may include a catheter having some radiolucent segments and some opaque segments, as shown in Figures 5A and 5B. In some embodiments, as the guidewire advances through the catheter in the radiolucent segment, the guidewire tip may be detected in the two-dimensional x-ray image and a scale factor may be stored in a lookup table. As the guidewire enters the x-ray opaque segment, the linear scale factor may be dynamically updated based on the lookup table parameters.

[0079] In some embodiments, more than one intravascular device may be tracked using the system 100 and method of Figure 2. For example, stents, balloons, and other types of intravascular devices may be tracked within a catheter as elongate device 120 using the teachings herein.

[0080] In some embodiments, the robot's movement is enhanced depending on the relative position of each device to the tip. For example, if the inner device is within the outer device and is approaching the tip of the outer device, its speed is limited to a preset value at a certain distance, or to a continuous function based on the distance to the tip. The speed limit can be based on the direction of movement of the inner device within the outer device; for example, retraction into the outer device has a higher speed limit than advancement toward the tip of the outer device.

[0081] FIG. 6 illustrates a computer system in which a method for tracking elongated devices is implemented, according to another representative embodiment.

[0082] 6, computer system 600 includes a set of software instructions that can be executed to cause computer system 600 to perform any of the methods or computer-based functions disclosed herein. Computer system 600 may operate as a standalone device or may be connected to other computer systems or peripheral devices, for example, using network 601. In an embodiment, computer system 600 performs logical processing based on digital signals received via an analog-to-digital converter.

[0083] In a networked deployment, computer system 600 operates in a server capacity, as a client-user computer in a server-client-user network environment, or as a peer computer system in a peer-to-peer (or distributed) network environment. Computer system 600 can also be implemented as or incorporated into a variety of devices, such as a workstation, including a controller, a fixed computer, a mobile computer, a personal computer (PC), a laptop computer, a tablet computer, or any other machine capable of executing a series of software instructions (sequential or otherwise) that specify actions to be performed by the machine. Computer system 600 can be incorporated into or as a device within an integrated system that includes additional devices. In one embodiment, computer system 600 can be implemented using electronic devices that provide voice, video, or data communications. Furthermore, although computer system 600 is shown in the singular, the term "system" is intended to include any collection of systems or subsystems that individually or collectively execute a set or sets of software instructions to perform one or more computer functions.

[0084] As shown in FIG. 6 , computer system 600 includes processor 610. Processor 610 may be considered a representative example of a controller's processor and executes instructions to implement some or all aspects of the methods and processes described herein. Processor 610 is tangible and non-transitory. As used herein, the term "non-transitory" is to be interpreted as a characteristic of a state that lasts for a period of time, rather than as a permanent characteristic of a state. The term "non-transitory" specifically negates ephemeral characteristics, such as carrier waves or signals or other forms of characteristics that exist only temporarily at any time and place. Processor 610 is a product and / or machine component. Processor 610 is configured to execute software instructions to perform the functions described in various embodiments herein. Processor 610 may be a general-purpose processor or part of an application-specific integrated circuit (ASIC). Processor 610 may be a microprocessor, microcomputer, processor chip, controller, microcontroller, digital signal processor (DSP), state machine, or programmable logic device. The processor 610 may be a logic circuit including a programmable gate array (PGA), such as a field programmable gate array (FPGA), or another type of circuit including discrete gate and / or transistor logic. The processor 610 may be a central processing unit (CPU), a graphics processing unit (GPU), or both. Additionally, any processor described herein may include multiple processors, parallel processors, or both. Multiple processors may be included in or combined with a single device or multiple devices.

[0085] As used herein, the term "processor" encompasses an electronic component capable of executing a program or machine-executable instructions. References to a computing device having a "processor" should be interpreted to include more than one processor or processing core, such as a multi-core processor. A processor may refer to a collection of processors within a single computer system or distributed among multiple computer systems. The term computing device should also be interpreted to include a collection or network of computing devices each containing a processor or multiple processors. A program comprises software instructions that are executed by one or more processors, which may be within the same computing device or distributed across multiple computing devices.

[0086] The computer system 600 further includes a main memory 620 and a static memory 630, which communicate with each other and with the processor 610 via a bus 608. Either or both of the main memory 620 and the static memory 630 may be considered representative examples of memory in a controller and may store instructions used to implement some or all aspects of the methods and processes described herein. The memory described herein is a tangible storage medium that stores data and executable software instructions and is non-transitory while the software instructions are stored therein. The term "non-transitory" as used herein is to be interpreted as a characteristic of a state that lasts for a period of time, rather than as a permanent characteristic of a state. The term "non-transitory" specifically negates ephemeral characteristics, such as carrier waves or signals or other forms of characteristics that exist only temporarily at any time and in any place. The main memory 620 and the static memory 630 are manufactured and / or machine components. Main memory 620 and static memory 630 are computer-readable media from which data and executable software instructions can be read by a computer (e.g., processor 610). Each of main memory 620 and static memory 630 may be implemented as one or more of random access memory (RAM), read-only memory (ROM), flash memory, electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disk, removable disk, tape, compact disk read-only memory (CD-ROM), digital versatile disk (DVD), floppy disk, Blu-ray disk, or any other form of storage medium known in the art. These memories may be volatile or non-volatile, secure and / or encrypted, or unsecure and / or unencrypted.

[0087] "Memory" is an example of a computer-readable storage medium. Computer memory is any memory that is directly accessible to a processor. Examples of computer memory include, but are not limited to, RAM memory, registers, and register files. References to "computer memory" or "memory" should be interpreted as possibly including multiple memories. A memory may be, for example, multiple memories within the same computer system. A memory may also be multiple memories distributed among multiple computer systems or computing devices.

[0088] As shown, computer system 600 further includes a video display unit 650, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid-state display, or a cathode ray tube (CRT). In addition, computer system 600 includes an input device 660, such as a keyboard / virtual keyboard or a touch-sensitive input screen, or voice input with voice recognition, and a cursor control device 670, such as a mouse or a touch-sensitive input screen or pad. Computer system 600 optionally includes a disk drive unit 680, a signal generating device 690, such as a speaker or remote control, and / or a network interface device 640.

[0089] In one embodiment, as shown in FIG. 6 , disk drive unit 680 includes computer-readable medium 682 having one or more sets of software instructions 684 (software) embedded therein. The set of software instructions 684 is read from computer-readable medium 682 for execution by processor 610. Furthermore, when executed by processor 610, software instructions 684 perform one or more steps of the methods and processes described herein. In one embodiment, software instructions 684 reside, in whole or in part, in main memory 620, static memory 630, and / or processor 610 during execution by computer system 600. Furthermore, computer-readable medium 682 may contain software instructions 684 or may receive and execute software instructions 684 in response to propagated signals such that devices connected to network 601 communicate audio, video, or data over network 601. Software instructions 684 may be transmitted or received over network 601 via network interface device 640.

[0090] In one embodiment, dedicated hardware implementations, such as application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic arrays, and other hardware components, are configured to implement one or more of the methods described herein. One or more embodiments described herein may implement functionality using two or more specific interconnected hardware modules or devices with associated control and data signals that can be communicated between and through the modules. Thus, the present disclosure encompasses software, firmware, and hardware implementations. Nothing in this application should be interpreted as being implemented or executable solely in software rather than in hardware such as a tangible, non-transitory processor and / or memory.

[0091] According to various embodiments of the present disclosure, the methods described herein may be implemented using a hardware computer system executing a software program. Furthermore, in exemplary, non-limiting embodiments, implementations may include distributed processing, component / object distributed processing, and parallel processing. A virtual computer system process may implement one or more of the methods or functions described herein, and the processors described herein may be used to support a virtual processing environment.

[0092] Thus, elongate device tracking allows for the incorporation of visual cues on fluoroscopic images so that the physician can clearly monitor the position of the guidewire tip within the catheter.

[0093] In this disclosure, some examples have been based on the location of the "tip" of the inner device (and / or outer device), where the invention also includes the location of other predetermined portions of the inner device (and / or outer device) as long as these predetermined portions are clearly defined from the outset (e.g., with respect to a known portion or tip of the inner and / or outer device).

[0094] While elongate device tracking has been described with reference to several exemplary embodiments, it should be understood that the words used are words of description and illustration, rather than words of limitation. Changes may be made within the purview of the appended claims as amended without departing from the scope and spirit of elongate device tracking as presently described and in its aspects. While elongate device tracking has been described with reference to particular means, materials, and embodiments, the elongate device tracking is not intended to be limited to the details disclosed, but rather, the elongate device tracking extends to all functionally equivalent structures, methods, and uses that are within the scope of the appended claims.

[0095] The illustrations of the embodiments described herein are intended to provide a general understanding of the structure of various embodiments. The illustrations are not intended to serve as a complete description of all elements and features of the disclosure described herein. Many other embodiments may be apparent to those skilled in the art upon review of the present disclosure. Other embodiments may be utilized and derived from the present disclosure, such that structural and logical substitutions and changes may be made without departing from the scope of the disclosure. Additionally, the illustrations are merely representational and may not be drawn to scale. Certain proportions within the illustrations may be exaggerated, and other proportions may be minimized. Therefore, the disclosure and the illustrations should be considered illustrative, not limiting.

[0096] One or more embodiments of the present disclosure may be referred to herein, individually and / or collectively, by the term "invention" for convenience only, without any intention to intentionally limit the scope of the present application to any particular invention or inventive concept. Furthermore, while specific embodiments have been illustrated and described herein, it should be understood that subsequent configurations designed to achieve the same or similar purpose may be substituted for the specific embodiment shown. The present disclosure is intended to cover any and all subsequent adaptations or modifications of the various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those skilled in the art upon review of this specification.

[0097] In the foregoing detailed description, various features may be grouped together or described in a single embodiment for the purpose of streamlining the disclosure. This disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter may be directed to less than all features of any of the disclosed embodiments. Accordingly, the following claims are incorporated into the detailed description, with each claim standing on its own as defining separately claimed subject matter.

[0098] The foregoing description of the disclosed embodiments is provided to enable those skilled in the art to practice the concepts described in this disclosure. Accordingly, the subject matter disclosed above should be considered illustrative and not limiting, and the appended claims are intended to cover all such modifications, extensions, and other embodiments that fall within the true scope of the present disclosure. Accordingly, to the maximum extent permitted by law, the scope of the present disclosure should be determined by the broadest permissible interpretation of the following claims, and should not be limited or constrained by the foregoing detailed description.

Claims

1. 1. A system for tracking a coaxial elongate device in an anatomical structure, the coaxial elongate device having an outer elongate device and an inner elongate device driven by a motor, the system comprising: A controller; a first interface configured to receive an image including at least one of the outer elongate device and the inner elongate device; a second interface configured to receive coded data representative of actuation of the inner elongated device by the motor; and The controller receiving, via the first interface, an image including at least one of the outer elongate device and the inner elongate device retracted within the outer elongate device; receiving the encoded data via the second interface; determining a position of the outer elongate device; estimating a position of the inner elongate device retracted within the outer elongate device based on the position of the outer elongate device and the encoded data; It is configured as follows: system.

2. The controller further comprises: a display providing display information overlaid on the outer elongate device to indicate the estimated position of the inner elongate device; It is configured as follows: the inner elongate device is at least partially concealed within the outer elongate device; The system of claim 1 .

3. The system of claim 2 , wherein the estimated location includes an estimated location of a tip of the inner elongate device.

4. the image comprises a two-dimensional x-ray image; the coaxial elongate device comprises an intravascular device; The system of claim 1 .

5. The system of claim 1 , wherein the association between the coded data and the location of pixel data of the image is based on a pre-calibration.

6. a third interface configured to accept inputs for pre-calibrating the position of the tip of the outer elongate device and the estimated position of the tip of the inner elongate device; The system of claim 5 further comprising:

7. The controller further comprises: configured to control movement of at least one of the outer elongate device and the inner elongate device. The system of claim 1 .

8. The system of claim 1 , wherein the position of the inner elongate device is estimated only when the inner elongate device is retracted within the outer elongate device.

9. The controller further comprises: calculating an approximate distance in pixels from the tip of the outer elongate device to the tip of the inner elongate device; The system of claim 1 configured to:

10. The controller further comprises: actively calibrating the encoded data to pixel data of the image; The system of claim 1 configured to:

11. The controller further comprises: identifying the location of the distal end of the outer elongate device within image data of the image; assigning a transformation scale factor along a centerline of the outer elongate device, wherein the transformation scale factor provides a variable transformation between the coded data and pixels; counting the amount of retraction of the inner elongate device relative to the distal end of the outer elongate device from the coded data; identifying a position along the trajectory of the outer elongate device corresponding to the distal end of the inner elongate device; The system of claim 1 configured to:

12. The controller further comprises: outputting an indication that the inner elongate device is fully retracted within the outer elongate device in the image data of the image. The system of claim 1 configured to:

13. The controller further comprises: detecting when the inner elongate device is fully retracted within the outer elongate device in image data of the image; automatically estimating the position of the inner elongate device upon detecting that the inner elongate device has been retracted into the outer elongate device; The system of claim 12 configured to:

14. The controller further comprises: The system of claim 1 , configured to estimate a position of at least one additional elongate device within the outer elongate device.

15. a robotic system having a processor controlling at least one of the motor, the coaxial elongate device, and the coaxial elongate device, the motor providing coded data to the second interface; an imaging device providing the image to the first interface; or at least one of the inner elongate device or the outer elongate device; The system of claim 1 further comprising:

16. 1. A controller for tracking a coaxial elongate device through an anatomy, the controller comprising: a memory for storing instructions; a processor for executing said instructions; a first interface configured to receive an image including an outer elongate device having a coaxial inner elongate device driven by a motor; a second interface configured to receive encoded data corresponding to actuation of the coaxial inner elongate device by the motor; and When executed by the processor, the instructions cause a system including the controller to: receiving the image including at least one of the outer elongate device and the coaxial inner elongate device; receiving the encoded data; determining a position of the outer elongate device; estimating a position of the coaxial inner elongate device retracted within the outer elongate device based on the position of the outer elongate device and the encoded data; controller.

17. When executed by the processor, the instructions cause the controller to: causing a display to provide data on the estimated position of the coaxial inner elongate device superimposed on the outer elongate device; the coaxial inner elongate device is at least partially concealed within the outer elongate device; 17. The controller of claim 16.

18. When executed by the processor, the instructions cause the controller to: applying a trained artificial intelligence model to the images and the encoded data to estimate the position of the coaxial inner elongate device; 17. The controller of claim 16.

19. 1. A method for tracking a coaxial elongate device within an anatomy, comprising: receiving, via a first interface, an image including at least one of an outer elongated device or a coaxial inner elongated device driven by a motor; receiving, via a second interface, coded data corresponding to actuation of the coaxial inner elongate device by the motor; determining a position of the outer elongate device; estimating a position of the coaxial inner elongate device retracted within the outer elongate device based on the position of the outer elongate device and the encoded data; A method having the following.

20. displaying on a display an estimated position of the coaxial inner elongate device superimposed on the outer elongate device, the coaxial inner elongate device being at least partially obscured within the outer elongate device; 20. The method of claim 19 further comprising:

21. The method of claim 19 , wherein the estimated location includes an estimated location of a tip of the coaxial internal elongate device.

22. the image comprises a two-dimensional x-ray image; the elongate device comprises an intravascular device; 22. The method of claim 21.

Images