Elongated device tracking

Abstract

A system for tracking a coaxial elongate device in an anatomical structure includes a controller including a first interface and a second interface. The coaxial elongate device includes an inner elongate device and an outer elongate device driven by a motor. The first interface is configured to receive an image including at least one of an external elongate device or a coaxial internal elongate device. The second interface is configured to receive encoded data representing driving of the coaxial internal elongate device by the motor. The controller is configured to receive an image including at least one of the external elongate devices within which the coaxial internal elongate device is retracted; receiving encoded data; determining the position of the external elongate device; and estimating a position of the coaxial inner elongate device retracted within the outer elongate device based on the position of the outer elongate device and the encoded data.

Description

Background Art

[0001] In endovascular interventions, fluoroscopic images are used to independently manipulate coaxial endovascular devices. The coaxial endovascular device includes an internal device (such as a guide wire) and an external device (such as a catheter). The internal device can be retracted into and advanced from the external device. When the tip of the guide wire is retracted into the catheter, the opacity of the catheter makes it difficult to observe the position of the guide wire tip. In addition, this creates a cavity in the external device where blood can accumulate and clot. Further, when the guide wire is extended, there is a risk of damaging the enclosed blood vessel.

[0002] When a doctor fails to notice that the guide wire is fully retracted, creating a cavity in the external device where blood can accumulate and clot for an extended period of time, not knowing the position of the guide wire tip can be particularly problematic. When the guide wire is advanced again, the clot may embolize, which can be life-threatening. In addition, during guide wire replacement, the guide wire can be rapidly advanced within the external device and has the potential to be undesirably rapidly advanced from the external device and damage the enclosed blood vessel. For example, a guide wire that is undesirably rapidly advanced without knowing the position of its tip may pierce the endothelium. Summary of the Invention

[0003] According to one aspect of the present disclosure, a system for tracking a coaxial elongate device in an anatomical structure includes a controller that includes a first interface and a second interface. The coaxial elongate device includes an internally elongate device driven by a motor and an externally elongate device. The first interface is configured to receive an image including at least one of the externally elongate device or the coaxial internally elongate device. The second interface is configured to receive encoded data representing the driving of the coaxial internally elongate device by the motor. The controller is configured to receive an image including at least one of the externally elongate device with the coaxial internally elongate device retracted within the externally elongate device; receive the encoded data; determine the position of the externally elongate device; and estimate the position of the coaxial internally elongate device retracted within the externally elongate device based on the position of the externally elongate device and the encoded data.

[0004] Additional alternative embodiments are disclosed in claims 2 to 15 according to this aspect of the present disclosure.

[0005] According to another aspect of the present disclosure, a controller for tracking a coaxial elongate device in an anatomical structure includes: a memory storing instructions; a processor executing the instructions; a first interface; and a second interface. The first interface is configured to receive an image including an external elongate device having a coaxial internal elongate device driven by a motor. The second interface is configured to receive encoded data corresponding to the driving of the coaxial internal elongate device by the motor. When run by the processor, the instructions cause the system to: receive an image including at least one of the external elongate device and the coaxial internal elongate device; receive the encoded data; determine the position of the external elongate device; and estimate the position of the coaxial internal elongate device retracted within the external elongate device based on the position of the external elongate device and the encoded data.

[0006] According to this another aspect of the present disclosure, additional alternative embodiments are disclosed in claims 17 to 18.

[0007] According to another aspect of the present disclosure, a method for tracking a coaxial elongate device in an anatomical structure includes: receiving, via a first interface, an image including at least one of a coaxial internal elongate device or an external elongate device driven by a motor; receiving, via a second interface, encoded data corresponding to the driving of the coaxial internal elongate device by the motor; determining the position of the external elongate device; and estimating the position of the coaxial internal elongate device retracted within the external elongate device based on the position of the external elongate device and the encoded data.

[0008] According to this another aspect of the present disclosure, additional alternative embodiments are disclosed in claims 20 to 22. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Example embodiments are best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be emphasized that the various features are not necessarily drawn to scale. In fact, for clarity of discussion, the dimensions may be increased or decreased arbitrarily. Where applicable and practical, like reference numerals refer to like elements.

[0010] Figure 1A A system for tracking an elongate device according to a representative embodiment is shown.

[0011] Figure 1B An elongate device for tracking an elongate device according to a representative embodiment is shown.

[0012] Figure 2 A method for tracking an elongate device according to a representative embodiment is shown.

[0013] Figure 3Shows a user interface for tracking a slender device according to a representative embodiment.

[0014] Figure 4 Shows an X-ray image for tracking a slender device according to a representative embodiment.

[0015] Figure 5A and Figure 5B Shows a user interface for tracking a slender device according to a representative embodiment.

[0016] Figure 6 Shows a computer system according to another representative embodiment, on which a method for tracking a slender device is implemented. Detailed Description

[0017] In the following detailed description, for purposes of explanation and not limitation, representative embodiments disclosing specific details are set forth in order to provide a thorough understanding of embodiments according to the present teachings. However, other embodiments consistent with the present disclosure that depart from the specific details disclosed herein are still within the scope of the claims. Descriptions of known systems, devices, materials, operating methods, and manufacturing methods may be omitted to avoid obscuring the description of the representative embodiments. Nevertheless, systems, devices, materials, and methods within the knowledge of those of ordinary skill in the art are within the scope of the present teachings and may be used in accordance with the representative embodiments. It should be understood that the terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting. The definitions and interpretations of the terms herein supplement the technical and scientific meanings of the terms commonly understood and accepted in the technical field of the present teachings.

[0018] It should be understood that although the terms first, second, third, etc. may be used herein to describe various elements or components, these elements or components should not be limited by these terms. These terms are only used to distinguish one element or component from another. Thus, without departing from the teachings of the inventive concept, the first element or component discussed below may be referred to as the second element or component.

[0019] As used in the specification and claims, the singular forms of the terms "a," "an," and "the" are intended to include both the singular and the plural forms, unless the context clearly dictates otherwise. Additionally, when used in this specification, the terms "comprises" and / or "comprising" and / or similar terms specify the presence of the stated features, elements, and / or components, but do not preclude the presence or addition of one or more other features, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0020] Unless otherwise specified, when an element or component is referred to as "connected to", "coupled to", or "adjacent to" another element or component, it should be understood that the element or component can be directly connected or coupled to the other element or component, or there can be intermediate elements or components. That is, these terms and similar terms cover cases where one or more intermediate elements or components can be employed to connect two elements or components. However, when an element or component is referred to as "directly connected" to another element or component, this only covers the situation where the two elements or components are connected to each other without any intermediate or intervening elements or components.

[0021] Accordingly, through one or more of the various aspects, embodiments, and / or specific features or sub-components of the present disclosure, the present disclosure aims to provide one or more advantages as specifically pointed out below.

[0022] As described herein, visual cues can be incorporated onto fluoroscopic images such that a physician can clearly monitor the position of the tip of a guidewire within a catheter.

[0023] Figure 1A A system 100 for tracking an elongate device according to a representative embodiment is shown.

[0024] Figure 1A The system 100 in is a system for tracking an elongate device and includes components that can be provided together or can be distributed. The system 100 includes an imaging device 101, an elongate device 120, a robot 130, a motor 140, and a display 180. The robot 130 and the motor 140 can include separate elements or integrated elements or systems. The computer 110 includes a first interface 111, a second interface 112, and a controller 150. The controller 150 includes at least a memory 151 that stores instructions and a processor 152 that executes the instructions.

[0025] The imaging device 101 can include an X-ray device or system. The imaging device 101 can be configured to provide a two-dimensional X-ray image including the elongate device 120, which includes the tip of the elongate device.

[0026] The computer 110 can include a workstation, a laptop computer, a desktop computer, a dedicated computer, or a virtual machine running on a local or remote cloud service. The computer 110 is used to control and / or coordinate the operations of the imaging device 101, the motor 140, the robot 130, and the display 180. Figure 6 The computer that can be used to implement the computer 110 is depicted in, but the computer 110 can include more than what is shown in FIG. 1 or Figure 6More or fewer components as depicted. Computer 110 includes at least a first interface 111, a second interface 112, and a controller 150. The first interface 111 docks the computer 110 with the imaging device 101. The second interface 112 docks the computer 110 with the motor 140 and / or the encoder 145. Another interface (not shown or labeled) docks the computer 110 with the display 180. The first interface 141 and the second interface 142 may include ports, adapters, and / or other types of suitable hardware configured to accept cable inputs from cables connected to the imaging device 101, the motor 140 and / or the encoder 145, and the display 180.

[0027] The controller 150 includes at least a memory 151 that stores instructions and a processor 152 that executes the instructions. In some embodiments, a plurality of different components of the system 100 in FIG. 1 may include a controller, such as the controller 150. The controller 150 may also include interfaces, such as a third interface, a fourth interface, a fifth interface, and a sixth interface. One or more interfaces of the controller 150 may include ports, disk drives, wireless antennas, or other electronic components that connect the controller 150 to the computer 110 or other types of receiver circuits external to the computer 110. The controller 150 may be configured to receive, via the first interface 111, an image including image data of at least one of the external elongate devices with the internal elongate device retracted within the external elongate device. The computer 110 or one or more interfaces specific to the controller 150 may also include a user interface, such as buttons, keys, a mouse, a microphone, a speaker, a display separate from the display 180, or other components by which a user may interact with the computer 110 or specifically with the controller 150 (e.g., input instructions and receive outputs).

[0028] In some embodiments, the interface of the controller 150 may include a user interface by which a user controls the robot 130. When the guide wire passes through the alignment position with the catheter, the user interface may provide haptic feedback by vibrating. In some embodiments, another component, such as the robot 130, may vibrate when alignment is approaching. In other embodiments, the interface on the robot 130 or the motor 140 may vibrate or illuminate when alignment is approaching.

[0029] Whenever the guide wire is retracted into the catheter by more than an amount X, for example 1 mm, audio feedback is added. The audio feedback may be a beep, a verbal millimeter reading, or other special effect sounds.

[0030] The memory 151 may store the association between the encoded data from the encoder 145 and the pixel position data along the catheter of the external device that is the elongate device 120. When the internal device is retracted by the motor 140, the processor 152 may evaluate the position of the guidewire tip of the internal device that is the elongate device 120 along the external device. The processor 152 may also generate the overlay information and send it to the display 180 to overlay an (highlighted) indicator on the two-dimensional X-ray image. Once the tip of the internal device is fully retracted into the external device, the overlay indicator may be overlaid on the two-dimensional X-ray image along the external device at the evaluated position of the tip of the internal device. Based on the encoded data from the encoder 145 and the stored association between the encoded data from the encoder 145 and the pixel position data along the catheter of the external device that is the elongate device 120, the overlay indicator is placed at the position on the two-dimensional X-ray image.

[0031] The controller 150 may directly perform some of the operations described herein and may indirectly implement other operations described herein. For example, the controller 150 may indirectly control operations, such as by generating and transmitting content to be displayed on the display 180. The controller 150 may be configured to provide display information such that the display 180 may display the estimated position of the internal elongate device overlaid on the external elongate device. The controller 150 may directly control other operations based on inputs received via the interface from electronic components and / or the user, such as the logical operations performed by the processor 152 executing instructions from the memory 151. Thus, when the processor 152 executes instructions from the memory 151, the process implemented by the controller 150 may include steps not directly performed by the controller 150. In some embodiments, the controller 150 may perform operations by executing instructions received from the cloud, such as from a server in a data center.

[0032] The elongate device 120 may include an intravascular device, such as a coaxial combination of two intravascular devices. In the description herein, the elongate device 120 is mainly described in the context of a combination of a catheter as the external device and a guidewire as the internal device. The movement of the guidewire as the internal device within the catheter as the external device may be controlled by the robot 130 powered by the motor 140. The controller 150 may be configured to control the movement of at least one of the external elongate device and the internal elongate device. In some embodiments, the controller 150 may automatically control the speed of the elongate device 120, such as by instructing the motor 140 to reduce power and / or instructing the robot 130 to slow down when the guidewire approaches the alignment point while the guidewire is advancing and when the guidewire is within the catheter.

[0033] The robot 130 may include a controllable device powered by a motor 140 and movable in one or more degrees of freedom. The robot 130 is configured to drive the elongate device 120 at least forward and backward in one degree of freedom. The robot 130 may also be configured to rotate the elongate device 120 in a second degree of freedom. The robot 130 may also be configured to vertically move the elongate device 120 up and down in a third degree of freedom, and to move the elongate device 120 left and right in a fourth degree of freedom.

[0034] The motor 140 may include an electric motor, a linear motor, a precision stepper motor, or a servo motor (“servo motor”). In some embodiments, the robot 130 and the motor 140 may include an integrated unit referred to as either or both of the robot and / or the motor. The motor 140 is configured to drive the robot 130 to drive the elongate device 120 in one or more degrees of freedom. The robot system may include, for example, the motor 140 and the controller 150, or a combination of the robot 130 and the motor 140 and the controller 150.

[0035] The motor 140 is also equipped with an encoder 145 that tracks and encodes the movement of the robot 130 and thus the elongate device 120 in one or more degrees of freedom. The encoder 145 outputs encoded data reflecting the amount of movement of the robot 130 and thus the elongate device 120 in each degree of freedom.

[0036] The display 180 may be a local display of the controller 150 or may be remotely connected to the controller 150. The display 180 may be connected to the controller 150 via a local wired interface such as an Ethernet cable or via a local wireless interface such as a Wi-Fi connection. The display 180 may interface with other user input devices through which a user may input instructions (including a mouse, a keyboard, a thumbwheel, etc.). The display 180 may be a monitor such as a computer monitor, a display on a mobile device, an augmented reality display, a television, an electronic whiteboard, or another screen configured to display an electronic image. The display 180 may also include one or more input interfaces (such as the above input interfaces that may be connected to other elements or components), and an interactive touch screen configured to display prompts to the user and collect touch inputs from the user. In some embodiments, when the guidewire is not in the current field of view, the display 180 may display a warning, such as an arrow or text, near the intersection of the catheter and the edge of the two-dimensional X-ray image. The controller 150 may be configured to calculate an approximate pixel distance from the tip of the external elongate device to the tip of the internal elongate device. In some embodiments, the display 180 may display the distance of the position of the tip of the guidewire relative to the tip of the catheter, such as via a digital display or a separate linear display or a color overlay of the portion of the catheter without the guidewire (i.e., the blank space between the tip of the catheter and the tip of the guidewire).

[0037] The controller 150 may also include interfaces, such as a first interface, a second interface, a third interface, and a fourth interface. One or more interfaces may include ports, disk drives, wireless antennas, or other types of receiver circuits that connect the controller 150 to other electronic components. One or more interfaces may also include a user interface, such as buttons, keys, a mouse, a microphone, a speaker, a display separate from the display 180, or other components that a user may use to interact with the controller 150 (such as inputting instructions and receiving outputs).

[0038] As described above, the system 100 may include a system for tracking a coaxial elongate device, such as the elongate device 120. The coaxial elongate device may include an outer elongate device and an inner elongate device driven by a motor, such as the motor 140. The system 100 may include a computer having a controller, such as the computer 110 having the controller 150. The computer may include: a first interface, such as the first interface 111, configured to receive an image including image data of at least one of the outer elongate device or the coaxial inner elongate device; and a second interface, such as the second interface 112, configured to receive encoded data representing the drive of the motor on the coaxial inner elongate device. The controller of the system 100 is configured to: receive an image including at least one outer elongate device, with the coaxial inner elongate device retracted within the outer elongate device; receive the encoded data; determine the position of the outer elongate device; and estimate the position of the coaxial inner elongate device retracted within the outer elongate device based on the position of the outer elongate device and the encoded data.

[0039] Figure 1B An elongate device for elongate device tracking according to a representative embodiment is shown.

[0040] In Figure 1B , the elongate device 120 is shown as including an outer elongate device 124 having an inner elongate device 122. The outer elongate device 124 may include a catheter, and the inner elongate device 122 may include a guidewire.

[0041] Although not shown in Figure 1B , in some embodiments, the elongate device 120 may also be tracked without an encoder 145. For example, an external device that encodes the relative position between the guidewire and the catheter may be used. In some embodiments, optical shape sensing may be used. In other embodiments, markers and a camera on the proximal portion of the elongate device 120 may be used to track the movement of the robot 130 and the motor 140.

[0042] Figure 2 A method for elongate device tracking according to a representative embodiment is shown.

[0043] Figure 2The method can be executed by a system 100 including a controller 150.

[0044] At S210, pre-calibration is performed. When the tip of the guide wire as an internal device and the tip of the catheter as an external device are aligned in a two-dimensional X-ray image, pre-calibration can be performed based on the calibration timing of the encoded data. The association between the encoded data and the pixel position data along the catheter as an external device can be based on the pre-calibration of the encoded data, including the determination of the encoded data when the two tips are aligned in the image.

[0045] Pre-calibration can be performed using the computer 110 and the display 180 in FIG. 1. For example, the user interface of the computer 110 can include a button that the user can press when the guide wire tip is considered to be at the end of the catheter based on the two-dimensional X-ray image currently displayed on the display 180. In some embodiments, when the user presses such a button based on the two-dimensional X-ray image currently displayed on the display 180, the controller 150 can calculate the offset between the guide wire tip and the catheter tip. The button can include an interface, such as a third interface, which is configured to accept an input to pre-calibrate the position of the tip of the external elongate device and the estimated position of the tip of the internal elongate device.

[0046] In some embodiments, the controller 150 can automatically perform calibration without direct user instruction, for example, based on interpreting the end of the catheter in a two-dimensional X-ray image on the display 180 and based on the encoded data of the guide wire from the encoder 145. Whenever a new instance of the elongate device 120 is introduced, the relative position of the tip with respect to the encoder 145 (tip alignment) can be calibrated. The robot 130 or another element of the system 100 can have dedicated hardware for this. Alternatively, when the device tips are aligned in the image, X-ray images and image processing can be used to detect the event. Then, the offset in the robot encoder space can be recorded. This can be achieved by monitoring the length of the guide wire protruding in pixels until the guide wire tip disappears into the catheter during or while the guide wire is retracted or the catheter is advanced. Alternatively, the controller 150 can detect when the guide wire is first visible, during catheter extension, or when the catheter is retracted.

[0047] At S220, a two-dimensional X-ray image including image data is generated and sent. S220 can be repeatedly executed in parallel with some or all of the steps from S230 to S270 in Figure 2

[0048] At S230, the intravascular device is driven and data corresponding to the driving of the intravascular device is encoded. S230 can be repeatedly executed in parallel with some S220 and all of the steps from S240 to S270 in Figure 2

[0049] At S240, a two-dimensional X-ray image and encoded data are received. The two-dimensional X-ray image can be received at computer 110 via the first interface 111 from the imaging device 101. The encoded data can be received at computer 110 via the second interface 112 from the encoder 145.

[0050] At S250, a representation of the displacement is output based on the encoded data. The representation can include an indication of warning the displacement or that there is no displacement of the guide wire relative to the displacement. The displacement can be an estimate of the movement of the elongate device 120 in one or more degrees of freedom based on the number of rotations of the motor 140, based on the amount of time the motor 140 drives the elongate device 120, based on the speed at which the motor 140 drives the elongate device 120, and / or based on any other factor that the encoder 145 can consider. The representation of the displacement can be the amount of displacement in each of one or more degrees of freedom.

[0051] At S260, the position of the external intravascular device is determined. The controller 150 can calculate a set of sequential line segments of the catheter of the elongate device 120 in the two-dimensional X-ray image. The position determined at S260 can be determined based on an image analysis program implemented by the controller 150. The association between the encoded data received at S240 and the position of the pixel data of the two-dimensional X-ray image received at S240 can be based on pre-calibration.

[0052] At S270, the position of the coaxial internal intravascular device is determined. The position of the coaxial internal intravascular device can involve a number of sub-processes that start with the encoded data from the encoder 145, to determine the robot coordinates based on the absolute displacement of the guide wire according to the movement of the robot 130, and then include the calibration of the pixel coordinates to calculate the approximate distance in pixels from the tip of the device, and then apply the approximate distance in pixels to the catheter line segments of the catheter to locate the tip of the guide wire on the image.

[0053] The determination at S270 can involve registering two or more coordinate systems with each other. For example, the robot 130 and the motor 140 can refer to a first three-dimensional coordinate system, and the imaging device 101 can refer to a second three-dimensional coordinate system. The two coordinate systems can be registered with each other such that the encoded data can be converted into a two-dimensional X-ray image in the three-dimensional coordinate system of the imaging device 101.

[0054] The determination at S270 can include applying a trained artificial intelligence model to the image and the encoded data to estimate the position of the coaxial internal elongate device. In some embodiments, the association between the encoded data from the encoder 145 and the pixel position data along the outer device from the image analysis performed by the controller 150 is based on the calibration of the scale factor between the image and the encoded data. The calibration can be performed at S210 as described above and can include determining the encoded data when the two tips are aligned in the image. The scale factor is constant across the image when the elongate device 120 is parallel to the detector plane from the two-dimensional X-ray detector, otherwise the scale factor varies for each position of the catheter and guidewire of the elongate device 120.

[0055] Active calibration can be performed relative to S270, but throughout most or all of the intervention procedure, in order to improve the user experience and provide improved accuracy for the position of the guidewire tip within the catheter. Active calibration can be performed to calibrate the robotic encoded data unit from the encoder 145 to the image unit based on the two-dimensional X-ray image from the imaging device 101. The image analysis algorithm can detect the tip of the catheter in the two-dimensional X-ray image and provide the translation of the position of the catheter tip in the image. Based on the encoded data from the encoder 145, the corresponding position of the guidewire tip for the movement of the guidewire can be determined. The comparison yields a size ratio, which can then be applied to estimate the position of the guidewire on the segmented representation of the catheter on the display 180. The pixel calibration algorithm can run in real time and is described in more detail below with respect to Figure 4 More detailed description.

[0056] At S280, the estimated position of the coaxial internal intravascular device is displayed. When the guidewire tip of the elongate device 120 is retracted into the catheter of the elongate device 120, a highlight or other indicator can be generated and superimposed on the two-dimensional X-ray image, for example at the tip position of the guidewire tip and on the display of the catheter of the elongate device 120 in the image. For example, the display 180 can output in the image data of the image an indication that the guidewire as the internal elongate device is fully retracted within the external elongate device. At S270, the position of the guidewire tip is established based on the encoder value in the encoded data output by the encoder 145 at S240. The controller 150 can be configured to automatically estimate the position of the internal elongate device once it detects that the internal elongate device has been retracted into the external elongate device.

[0057] The display of the tip (or any other part) of the guidewire at S280 can be performed within a limited time during the intervention, e.g., once the tip of the guidewire is fully retracted into the catheter, or can be performed throughout the intervention. In some embodiments, the controller 150 can selectively determine when to display the tip (or other part) of the guidewire, e.g., based on proximity to the tip (or other part) of the catheter, such that the tip of the guidewire is displayed even when the tip (or other part) of the guidewire extends slightly from the catheter.

[0058] In some embodiments, the display of the tip of the guidewire can vary based on the distance to the tip of the catheter. For example, variable size, shape, and / or color can be used to highlight the tip of the guidewire based on the distance to the tip. As an example, the highlighting can change from blue to green based on the distance aligned with the tip. As another example, the highlighting can gradually transition from a circular shape at the aligned position to an oval further away. The oval shape can be used in relatively crude embodiments for calibration, and the oval shape can cover the potential area where the tip might be located. In some embodiments, the overlay can vary based on the eigenvalues / eigenvectors of the system covariance matrix. In some embodiments, the circular highlight can change from a small diameter at alignment to a large diameter after alignment.

[0059] Figure 3 A user interface for tracking an elongate device according to a representative embodiment is shown.

[0060] Figure 3 The user interface 381 therein shows aspects of an elongate device imaged in an anatomical structure. Visual cues are provided to the physician via the user interface 381 labeled B and C. Thus, the physician can clearly monitor the position of the guidewire tip within the catheter to avoid errors. As shown, the tip position is highlighted.

[0061] In a first user interface 381 labeled A, the guidewire normally extends beyond the catheter. In a second user interface and a third user interface in the user interface 381 labeled B and C, the guidewire is retracted into the catheter, where the estimated position of the guidewire tip is highlighted by a highlighted visual overlay.

[0062] As an alternative (not shown), the user interface can include two or more (X-ray) two-dimensional images acquired from different perspectives, optionally simultaneously, thus showing the anatomical structure and the catheter and / or guidewire from two or more different angles. According to the present disclosure, one or more of the images can include an overlay indicator of the tip (or other part) of the guidewire.

[0063] Figure 4 An X-ray image for tracking an elongate device according to a representative embodiment is shown.

[0064] Figure 4 The user interface 481 in Figure 4 shows an X-ray image with potential paths and positions of the elongate device in the anatomical structure. In

[0065] As previously described, active calibration can be performed with respect to Figure 2 S270 in

[0066] but throughout most or all of the intervention procedure. Active calibration can be performed to calibrate the robotic encoded data unit from the encoder 145 to the image unit based on the two-dimensional X-ray image from the imaging device 101. The image analysis algorithm can detect the tip (or other predetermined part) of the catheter in the two-dimensional X-ray image and provide the translation of the position of the catheter tip (or other predetermined part) in the image. The corresponding position of the tip (or other predetermined part) of the guidewire from the movement of the guidewire can be determined according to the encoded data from the encoder 145. The comparison yields a size ratio, and then this size ratio can be applied to estimate the position of the guidewire on the segmented representation of the catheter on the display 180.

[0067] In Figure 4 p0t is the tip of the catheter at time t. The catheter of the elongate device 120 can be described by a list of points p0t, p1t, …, pnt. The image robotic encoder scale factors along the (one or more) centerlines of the blood vessel are shown as s0, s1, …, sk. A translation scale factor can be assigned along the centerline of the external elongate device, where the translation scale factor provides a variable translation between the encoded data and the pixels. The scale factor provides the number of robotic encoder counts per pixel, which can vary based on the alignment of the elongate device 120 with respect to the imaging plane, particularly in a two-dimensional image such as a two-dimensional X-ray image.

[0068] To calculate the image position of the guidewire tip, the catheter and the guidewire tip are calibrated, and the guidewire can be retracted from the tip of the catheter as a function of the ct count. The controller 150 can count the amount of retraction of the inner elongate device relative to the tip of the outer elongate device from the encoded data. Then, the position corresponding to the tip of the guidewire along the catheter trajectory is found by a first algorithm. The first algorithm for finding the guidewire tip involves first finding the centerline closest to the initial point in a list of points and then adjusting based on a scale factor. This process can be performed for each point in the list of points or for a limited set of points closest to the end of the specified catheter tip. The vessel centerline can be made available in advance. Alternatively, the vessel centerline in the two-dimensional X-ray image can be calculated dynamically when the catheter is inserted. The scale factor along the vessel central axis can be calculated iteratively by a second algorithm.

[0069] The second algorithm uses the displacement of the encoded data from the encoder 145 and updates the scale factor for each point in the list of points along the catheter centerline. The second algorithm is applied dynamically based on spatial and temporal parameters, and the second algorithm is used to create a calibrated map in at least the spatial domain of the calibration position on the image or in the vasculature.

[0070] The first algorithm can run continuously during the intervention while manipulating the guidewire. Once the tip has passed through that region of the vasculature, the scale factor corresponding to the catheter point is initialized, so the scale factor required to estimate the position of the guidewire within the catheter is well defined. Several different schemes can be used to update the scale factor. For example, an advanced method can consider the direction and speed of the encoder change in order to improve the estimate, because during pushing, the scale factor tends to be underestimated due to the buckling of the vasculature, and during pulling out the device, the scale factor tends to be overestimated due to the removal of slack.

[0071] In some embodiments, for any algorithm step, the system can use the pixel relative width of at least one elongate device to further or alternatively estimate the scaling change in the image. Especially if the actual width of the at least one device (e.g., in mm) is known, it can be mapped to the pixel width of the device in the image to directly estimate the scaling factor, or it can be used as supplementary information in the scaling factor algorithm.

[0072] In additional or alternative embodiments, the device can also include opaque fiducials (i.e., opaque to imaging) of known size and / or known displacement on the elongate device, and the difference between the actual geometry and the imaging geometry of these fiducials can be used to estimate the scaling factor. In cases where the first scaling algorithm lacks sufficient information, the scaling factor can be derived directly from the image and the assumption that the device is located at the isocenter of the imaging space using standard projection geometry techniques.

[0073] In additional or alternative embodiments, the system is arranged to estimate the orientation of the curvature of an (angled) external device and use that estimate to refine the scaling factor for the angled portion to show the tip of the internal device in the correct visual position. An external system that estimates the orientation of the angled device relative to the image is used to enhance information about the portion of the device in the image. That is, the relative in-plane and out-of-plane position of the distal portion of the angled device modulates the scaling factor. A particular portion is moving away from the x-ray detector and should have a proportionally larger scaling factor than a portion of the device that is positioned relatively closer to the detector (a given physical movement of the device will traverse more pixels in the image).

[0074] In cases where a multi-plane x-ray imaging system (such as a bi-plane) is available, the scaling factor algorithm can be used in each image. The user interface can be applied to any image source used in the algorithm or can be automatically selected as the primary user interface based on the most accurate scaling factor or the availability of the scaling factor or user preference. In cases where multi-plane x-ray imaging systems are co-registered, standard multi-view projection geometry methods can be used to reconstruct the device position in 3D space. The scaling factor can be calculated in reverse for the device in the image.

[0075] In cases where the actuation driver of the elongate device is sliding, the absolute position of the device can be occasionally reset by referencing an event in the image when the internal is visually detected entering or leaving the tip of the external device.

[0076] The controller 150 can check whether the guide wire is retracted based on the robotic coordinates. When the guide wire is retracted, a scale factor can be applied to the encoder-based retraction distance to produce a retraction length in pixels. This length is used to find the pixel position along the catheter line segment from the tip of the catheter using the Euclidean method. At this position, a glyph / graphic is blended with the x-ray image pixels to show the estimated guide wire position, as Figure 3 shown.

[0077] Figure 3 FIG. 5 shows a user interface for tracking an elongate device according to a representative embodiment.

[0078] In some embodiments based on Figure 3 the display of the guide wire tip can vary based on the estimated relative position of the tip of the device inside the external device entering and exiting the image plane. For example, the system is arranged such that when the internal device moves away from or towards the viewer, the color intensity is not the same, or the size or shape of the display changes based on how far it is out of plan as it travels inside the external device (to avoid shortening). The relative in-and-out plane of the device is proportional to the scaling factor.

[0079] Based on Figure 3Additional embodiments may include superimposed graphics or backgrounds within an external device located at positions regarding functional information of an internal device, such as joint points or different stiffness regions of the internal device, or device-specific markers, such as the start and end of a stent or balloon, or other diagnostic and therapeutic device information. The information may also be virtual, such as a gradient indicating parts of an isometric device.

[0080] Figure 5A and Figure 5B The user interface 581 in Figure 5A and Figure 5B shows aspects of an elongate device imaged within an anatomical structure. In

[0081] Based on Figure 5A and 5B In embodiments, the controller 150 is used to estimate the position of a guide wire within a catheter even when fluoroscopy is not enabled. The most recent X-ray image can be used for feedback, and highlights can be drawn along the segmented device as if fluoroscopy were active. When the catheter moves, as observed by encoded data from the encoder 145, a warning can be displayed on the display 180, and / or another two-dimensional X-ray image can be taken to update the image.

[0082] In some embodiments similar to those based on Figure 5A and Figure 5B When the guide wire is retracted into the catheter, the fluoroscopy frame rate can be reduced, such as by a factor of 1 / 2, and updates can be performed at the original frame rate to show the estimated guide wire tip.

[0083] Many variations are within the scope of the above teachings. For example, some elongate devices may include a catheter having some X-ray transparent sections and some opaque sections, as shown in Figure 5A and Figure 5B In some embodiments, when the guide wire advances through the catheter within an X-ray transparent section, the guide wire tip can be detected in a two-dimensional X-ray image, and a scale factor can be stored in a look-up table. When the guide wire enters an X-ray opaque section, the linear scale factor can be dynamically updated based on the look-up table parameters.

[0084] In some embodiments, the system 100 and method of Figure 2 can be used to track multiple intravascular devices. For example, according to the teachings herein, stents, balloons, and other types of intravascular devices can be tracked within a catheter as elongate device 120.

[0085] In some embodiments, the robotic movement will be enhanced based on the relative position of each device with respect to the tip. For example, when an inner device is inside an outer device and close to the tip of the outer device, its speed is limited to a preset value at a specific distance, or a continuous function based on the distance to the tip. The speed limit can be based on the direction of movement of the inner device within the outer device. For example, the speed limit for retracting the outer device is higher than the speed limit for advancing towards the tip of the outer device.

[0086] Figure 6 A computer system according to another representative embodiment is shown, on which a method for tracking an elongate device is implemented.

[0087] Refer to Figure 6 , computer system 600 includes a set of software instructions that can be executed to cause computer system 600 to perform any method or computer-based function disclosed herein. Computer system 600 can operate as a stand-alone device or can be connected, for example, using network 601 to other computer systems or peripheral devices. In an embodiment, computer system 600 performs logical processing based on digital signals received via an analog-to-digital converter.

[0088] In a networked deployment, computer system 600 operates as a server or as a client user computer in a server-client user network environment, or as a peer computer system in a peer-to-peer (or distributed) network environment. Computer system 600 can also be implemented as or incorporated into various devices, such as a workstation including a controller, a fixed computer, a mobile computer, a personal computer (PC), a laptop computer, a tablet computer, or any other machine capable of executing a set of software instructions (sequential or otherwise) specifying actions to be taken by that machine. Computer system 600 can be incorporated as a device or integrated within a device, which in turn is integrated within an integrated system including additional devices. In an embodiment, computer system 600 can be implemented using an electronic device that provides voice, video, or data communication. Additionally, although computer system 600 is shown in the singular, the term "system" should also be considered to include any collection of systems or subsystems that individually or jointly execute a set of or multiple sets of software instructions to perform one or more computer functions.

[0089] As Figure 6As shown, computer system 600 includes a processor 610. Processor 610 can be considered a representative example of a processor of a controller and executes instructions to implement some or all aspects of the methods and processes described herein. Processor 610 is tangible and non-transitory. As used herein, the term "non-transitory" should be construed as not being a permanent property of a state, but rather a property of a state that will persist for a period of time. The term "non-transient" specifically negates transient properties such as those of a carrier wave or signal or other forms that exist only temporarily anywhere at any time. Processor 610 is an article of manufacture and / or a machine component. Processor 610 is configured to execute software instructions to perform the functions described in various embodiments herein. Processor 610 can be a general-purpose processor or can be part of an application specific integrated circuit (ASIC). Processor 610 can also be a microprocessor, a microcomputer, a processor chip, a controller, a microcontroller, a digital signal processor (DSP), a state machine, or a programmable logic device. Processor 610 can also be a logic circuit, including a programmable gate array (PGA), such as a field programmable gate array (FPGA), or another type of circuit including discrete gates and / or transistor logic. Processor 610 can be a central processing unit (CPU), a graphics processing unit (GPU), or both. Additionally, any processor described herein can include multiple processors, parallel processors, or both. Multiple processors can be included in a single device or multiple devices, or can be coupled to a single device or multiple devices.

[0090] As used herein, the term "processor" encompasses electronic components capable of executing programs or machine-executable instructions. References to a computing device that includes a "processor" should be construed as including multiple processors or processing cores, as in a multi-core processor. A processor can also refer to a collection of processors within a single computer system or distributed among multiple computer systems. The term computing device should also be construed as including a collection or network of computing devices, each including one or more processors. A program has software instructions that are executed by one or more processors, which can be within the same computing device or can be distributed across multiple computing devices.

[0091] The computer system 600 also includes a main memory 620 and a static memory 630, where the memories in the computer system 600 communicate with each other and with the processor 610 via a bus 608. Either or both of the main memory 620 and the static memory 630 can be considered representative examples of the memory of the controller and store instructions for implementing some or all aspects of the methods and processes described herein. The memories described herein are tangible storage media for storing data and executable software instructions and are non-transitory during the period in which the software instructions are stored therein. As used herein, the term "non-transitory" should be construed as not being a permanent property of a state, but rather a property of a state that will persist for a period of time. The term "non-transient" specifically negates transient properties such as those of a carrier wave or a signal or other forms that exist only temporarily anywhere at any time. The main memory 620 and the static memory 630 are articles of manufacture and / or machine components. The main memory 620 and the static memory 630 are computer-readable media from which a computer (e.g., the processor 610) can read data and executable software instructions. Each of the main memory 620 and the static memory 630 can be implemented as one or more of a random access memory (RAM), a read-only memory (ROM), a flash memory, an electrically programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a removable disk, a magnetic tape, a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a floppy disk, a Blu-ray disc, or any other form of storage medium known in the art. The memory can be volatile or non-volatile, secure and / or encrypted, insecure and / or unencrypted.

[0092] "Memory" is an example of a computer-readable storage medium. Computer memory is any memory directly accessible by a processor. Examples of computer memory include, but are not limited to, RAM memory, registers, and register files. References to "computer memory" or "memory" should be construed as potentially being multiple memories. Memory can be, for example, multiple memories within the same computer system. Memory can also be multiple memories distributed among multiple computer systems or computing devices.

[0093] As shown, the computer system 600 also includes a video display unit 650, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), a flat panel display, a solid state display, or a cathode ray tube (CRT). Additionally, the computer system 600 includes an input device 660, such as a keyboard / virtual keyboard or a touch-sensitive input screen or voice input with voice recognition, and a cursor control device 670, such as a mouse or a touch-sensitive input screen or a keyboard. The computer system 600 also optionally includes a disk drive unit 680, a signal generation device 690 (such as a speaker or a remote control), and / or a network interface device 640.

[0094] In one embodiment, as Figure 6 shown, the disk drive unit 680 includes a computer-readable medium 682 in which one or more sets of software instructions 684 (software) are embedded. The one or more sets of software instructions 684 are read from the computer-readable medium 682 for execution by the processor 610. Further, the software instructions 684 perform one or more steps of the methods and processes described herein when run by the processor 610. In an embodiment, the software instructions 684 reside, at least in part, within the main memory 620, the static memory 630, and / or the processor 610 during operation by the computer system 600. Additionally, the computer-readable medium 682 may include the software instructions 684 or receive and execute the software instructions 684 in response to a propagated signal such that a device connected to the network 601 can transmit voice, video, or data over the network 601. The software instructions 684 may be sent or received over the network 601 via the network interface device 640.

[0095] In an embodiment, a dedicated hardware implementation such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic array, and other hardware components are constructed to implement one or more of the methods described herein. One or more embodiments described herein may be implemented using two or more specific interconnected hardware modules or devices having related control and data signals that may be transferred between and through the modules. Accordingly, the present disclosure encompasses software, firmware, and hardware implementations. Nothing in this application shall be construed as implementing or being implementable using only software rather than hardware such as tangible non-transitory processors and / or memories.

[0096] In accordance with various embodiments of the present disclosure, the methods described herein may be implemented using a hardware computer system that executes a software program. Additionally, in an exemplary non-limiting embodiment, the implementation may include distributed processing, component / object distributed processing, and parallel processing. Virtual computer system processing may implement one or more of the methods or functions described herein, and the processors described herein may be used to support a virtual processing environment.

[0097] Thus, the tracking of the elongate device enables the incorporation of visual cues on the fluoroscopic image such that the physician can clearly monitor the position of the guidewire tip within the catheter.

[0098] In the present disclosure, some examples are based on the position of the "tip" of an internal device (and / or an external device). Now, the present invention also includes the position of other predetermined portions of the internal device (and / or the external device), provided that these predetermined portions are initially clearly defined (e.g., relative to a known portion or tip of the internal device and / or the external device).

[0099] Although the tracking of elongated devices has been described with reference to several exemplary embodiments, it should be understood that the words used are words of description and illustration, not of limitation. Changes may be made within the scope of the claims, as presently stated and modified, without departing from the scope and spirit of the tracking of elongated devices in its aspects. Although the tracking of elongated devices has been described with reference to specific apparatus, materials, and embodiments, the tracking of elongated devices is not intended to be limited to the details disclosed; rather, the tracking of elongated devices extends to all functionally equivalent structures, methods, and uses, such as within the scope of the claims.

[0100] The illustrations of the embodiments described herein are intended to provide a general understanding of the structures of the various embodiments. These illustrations are not intended to be a complete description of all elements and features of the present disclosure described herein. Many other embodiments may be apparent to those of ordinary skill in the art upon reading the present disclosure. Other embodiments may be utilized and derived from the present disclosure, such that structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Additionally, the illustrations are merely representative and may not be drawn to scale. Some of the ratios within the illustrations may be exaggerated while others may be minimized. Accordingly, the present disclosure and the figures are to be considered illustrative and not restrictive.

[0101] One or more embodiments of the present disclosure may be referred to herein, individually and / or collectively, by the term "invention" merely for convenience and are not intended to voluntarily limit the scope of this application to any particular invention or inventive concept. Moreover, although specific embodiments have been shown and described herein, it should be understood that any subsequent arrangement designed to achieve the same or similar purpose may replace the specific embodiments shown. The present disclosure is intended to cover any and all subsequent adaptations or variations of various embodiments. Combinations of the above-described embodiments and other embodiments not specifically described herein will be apparent to those of ordinary skill in the art upon reading the specification.

[0102] To simplify the present disclosure, the foregoing detailed description, various features may be grouped together or described in a single embodiment. The present disclosure should not be construed as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the claims reflect, the inventive subject matter may involve less than all of the features of any of the disclosed embodiments. Accordingly, the claims are incorporated into the detailed description, where each claim independently defines a separately claimed subject matter.

[0103] The foregoing description of the disclosed embodiments is provided to enable those skilled in the art to practice the concepts described in this disclosure. Accordingly, the subject matter disclosed above will be regarded as illustrative and not restrictive, and the claims are intended to cover all such modifications, enhancements, and other embodiments that fall within the true scope of this disclosure. Thus, to the maximum extent permitted by law, the scope of this disclosure will be determined by the broadest permissible interpretation of the claims and shall not be limited or restricted by the foregoing detailed description.

Claims

1. A system for tracking a coaxial elongate device in an anatomical structure, wherein, The coaxial slender device includes an internal slender device and an external slender device driven by a motor, and the system includes a computer, and the computer includes: A controller; A first interface configured to receive an image including at least one of the external slender device and the internal slender device; and A second interface configured to receive encoded data representing the driving of the internal slender device by the motor, wherein the controller is configured to: Receive, via the first interface, an image including at least one of the external slender devices, with the internal slender device retracted into the external slender device; Receive the encoded data via the second interface; Determine the position of the external slender device; and Estimate the position of the internal slender device retracted into the external slender device based on the position of the external slender device and the encoded data.

2. The system according to claim 1, Among them, The controller is further configured to: Provide display information such that a display can show the estimated position of the internal slender device superimposed on the external slender device, wherein the internal slender device is at least partially hidden within the external slender device.

3. The system according to claim 2, wherein The estimated position includes the estimated position of the tip of the internal slender device.

4. The system according to claim 1, wherein, The image includes a two-dimensional X-ray image, and wherein the coaxial slender device includes an intravascular device.

5. The system according to any one of the preceding claims, wherein, The association between the encoded data and the position of the pixel data of the image is based on pre-calibration.

6. The system according to claim 5, further comprising: A third interface configured to accept an input to pre-calibrate the position of the tip of the external slender device and the estimated position of the tip of the internal slender device.

7. The system according to claim 1, wherein The controller is further configured to: Control the movement of at least one of the external slender device and the internal slender device.

8. The system according to claim 1, wherein Estimate the position of the internal slender device only when the internal slender device is retracted into the external slender device.

9. The system according to claim 1, wherein The controller is further configured to: Calculate an approximate pixel distance from the tip of the external slender device to the tip of the internal slender device.

10. The system according to claim 1, wherein, The controller is further configured to: Actively calibrate the encoded data to the pixel data of the image.

11. The system according to claim 1, wherein, The controller is further configured to: Identify the position of the tip of the external slender device in the image data of the image; Allocate a translation scale factor along the centerline of the external slender device, wherein the translation scale factor provides a variable translation between the encoded data and the pixels; Count the retraction amount of the internal slender device relative to the tip of the external slender device according to the encoded data; and Identify the position corresponding to the tip of the internal slender device along the trajectory of the external slender device.

12. The system according to claim 1, wherein The controller is further configured to: Output an indication in the image data of the image that the internal slender device is fully retracted into the external slender device.

13. The system according to claim 12, wherein, The controller is further configured to: Detect in the image data of the image when the internal slender device is fully retracted into the external slender device, and Once the internal elongate device is detected as retracted into the external elongate device, the position of the internal elongate device is automatically estimated.

14. The system according to claim 1, wherein The controller is further configured to: Estimate the position of at least one additional elongate device within the external elongate device.

15. The system according to claim 1, further comprising: At least one of the motor, the robotic system, the imaging device, the internal elongate device, or the external elongate device, wherein the motor provides encoded data to the second interface; the robotic system includes the motor and a processor for controlling at least one of the coaxial elongate devices; and the imaging device is configured to provide the image to the first interface.

16. A controller for tracking coaxial elongate devices in an anatomical structure, comprising: A memory that stores instructions; A processor that executes the instructions; A first interface configured to receive an image including an external elongate device having a coaxial internal elongate device driven by a motor; A second interface configured to receive encoded data corresponding to the driving of the coaxial internal elongate device by the motor, wherein when run by the processor, the instructions cause a system including the controller to: Receive an image including at least one of the external elongate device and the coaxial internal elongate device; Receive the encoded data; Determine the position of the external elongate device; and Estimate the position of the coaxial internal elongate device retracted into the external elongate device based on the position of the external elongate device and the encoded data.

17. The controller according to claim 16, wherein, When run by the processor, the instructions further cause the controller to: Provide data of the estimated position of the coaxial internal elongate device superimposed on the external elongate device to a display, Wherein the coaxial internal elongate device is at least partially hidden within the external elongate device.

18. The controller according to claim 16, wherein, When run by the processor, the instructions further cause the controller to: Apply a trained artificial intelligence model to the image and the encoded data to estimate the position of the coaxial internal elongate device.

19. A method for tracking coaxial elongate devices in an anatomical structure, comprising: Receiving, via a first interface, an image including at least one of a coaxial internal elongate device or an external elongate device driven by a motor; Receiving, via a second interface, encoded data corresponding to the driving of the coaxial internal elongate device by the motor; Determining the position of the external elongate device; and Estimating the position of the coaxial internal elongate device retracted into the external elongate device based on the position of the external elongate device and the encoded data.

20. The method according to claim 19, further comprising: Displaying, on a display, the estimated position of the coaxial internal elongate device superimposed on the external elongate device, wherein the coaxial internal elongate device is at least partially hidden within the external elongate device.

21. The method according to claim 19, wherein The estimated position includes the estimated position of the tip of the coaxial internal elongate device.

22. The method according to claim 21, wherein, The image includes a two-dimensional X-ray image, and Wherein the elongate device includes an intravascular device.

Images