Tracking of interventional medical devices

Abstract

The system includes an interface 193 to an optical shape sensing device 02 having a shape that matches the shape of the interventional medical device 01. The system also includes a controller 190 having a memory 191 that stores instructions and a processor 192 that executes the instructions. The instructions cause the system to identify the shape of the optical shape sensing device 102 using optical shape sensing signals received through the interface 193, identify the interventional medical device 01 in different coordinate spaces of an imaging system that images the interventional medical device 01 in different imaging modes, register the coordinate spaces with each other, and register the interventional medical device 01. The instructions also cause the system to obtain a segmented representation 401 of the interventional medical device 01 and re-register the interventional medical device 01 to the other of the coordinate spaces based on registering the interventional medical device 01 to one of the coordinate spaces using the segmented representation 401.

Description

[Technical Field]

[0001]

[0001] Optical shape detection (OSS) technology is used to provide real-time in-procedure information on the shape and relative position of the interventional medical device during interventional medical procedures. Information from OSS is used to locate and navigate the interventional medical device during interventional medical procedures. OSS is used to determine the shape of the interventional medical device during interventional medical procedures. Followed The method uses light along a multicore optical fiber. The principle involved utilizes distributed strain measurement within the optical fiber using characteristic Rayleigh backscattering or a controlled lattice pattern. The shape along the optical fiber begins at a specific point known as the emission or z=0, and the subsequent shape, position, and orientation are relative to the emission.

[0002]

[0002] In isolation, registration is used to align the coordinate systems of two separate devices and / or systems. For example, registration from an OSS device to an X-ray imaging system is used for conversion from an OSS device to an X-ray imaging system T OX This can be achieved by the following: Registration from an ultrasound imaging system to an X-ray imaging system is a conversion from an ultrasound imaging system to an X-ray imaging system. UX This can be achieved by the following: Registration from OSS devices to ultrasound imaging systems is a conversion from OSS devices to ultrasound imaging systems T OU This can be achieved.

[0003]

[0003] Furthermore, segmentation is used in the medical imaging system to represent the surface of the structure as a three-dimensional model. [Background technology]

[0004]

[0004] Currently, registration between the OSS device and the X-ray imaging system accumulates significant errors, such as when the proximal end of the OSS device (i.e., the end closest to the user) is moved by several centimeters. Correcting these errors requires re-registration between the OSS device and the X-ray imaging system, which then requires two new offset X-ray projections. These additional X-ray projections can disrupt the flow of interventional medical procedures, prolong the duration of the interventional medical procedures, and expose patients and clinicians to further X-ray doses.

[0005]

[0005] Registration of OSS devices with ultrasound imaging systems also accumulates significant errors. Re-registration of OSS devices with ultrasound imaging systems can disrupt the flow of interventional medical procedures and prolong the time required for interventional medical procedures, while image analysis software searches for the latest ultrasound image for the OSS device. To limit the search for OSS devices in ultrasound images, image analysis software requires, for example, user identification of the tip of the OSS device in the ultrasound image to the identification of the OSS device in the ultrasound image. Even with initial constraints, OSS devices must be sufficiently identified and located in ultrasound images for re-registration of OSS devices with ultrasound imaging systems, which disrupts the flow of interventional medical procedures and prolongs the time required for interventional medical procedures. [Overview of the project]

[0006]

[0006] According to one aspect of the present disclosure, a system for tracking the position of an interventional medical device in an interventional medical procedure includes an interface and a controller. The interface tracks the shape of the interventional medical device during the interventional medical procedure. FollowedThe system interfaces with an optical shape detection device that has a shape. The controller includes a memory for storing instructions and a processor for executing instructions. When an instruction is executed by the processor, it causes the system to identify the shape of the optical shape detection device using optical detection signals received from the optical shape detection device via the interface, and, based on the identification of the shape of the optical shape detection device, to identify the intervention medical device in a first coordinate space of a first imaging system that images the intervention medical device in a first imaging mode during an intervention medical procedure. The instruction also causes the system to register the intervention medical device in the first coordinate space, to identify the intervention medical device in a second coordinate space of a second imaging system that images the intervention medical device in a second imaging mode during an intervention medical procedure, and to register the first coordinate space of the first imaging system with the second coordinate space of the second imaging system. The command further causes the intervention medical device to be segmented in the second coordinate space to obtain a segmented representation of the intervention medical device in the second coordinate space, to be registered in the second coordinate space using the segmented representation of the intervention medical device, and to be re-registered in the first coordinate space based on the registration of the intervention medical device in the second coordinate space.

[0007]

[0007] According to another aspect of the present disclosure, a tangible non-temporary computer-readable storage medium stores a computer program. When executed by a processor, the computer program is stored in a system including the tangible non-temporary computer-readable storage medium in the form of an interventional medical device during an interventional medical procedure. FollowedThe command causes the system to identify the shape of an optical shape detection device using an optical shape detection signal received via an interface from an optical shape detection device having a shape, and to identify the intervention medical device in a first coordinate space of a first imaging system that images the intervention medical device in a first imaging mode during an intervention medical procedure, based on the identification of the shape of the optical shape detection device. The command also causes the system to register the intervention medical device in the first coordinate space, identify the intervention medical device in a second coordinate space of a second imaging system that images the intervention medical device in a second imaging mode during an intervention medical procedure, and register the first coordinate space of the first imaging system with the second coordinate space of the second imaging system. The command further causes the system to segment the intervention medical device in the second coordinate space to obtain a segmented representation of the intervention medical device in the second coordinate space, register the intervention medical device in the second coordinate space using the segmented representation of the intervention medical device, and re-register the intervention medical device in the first coordinate space based on the registration of the intervention medical device in the second coordinate space.

[0008]

[0008] According to yet another aspect of the present disclosure, a method for tracking the position of an interventional medical device in an interventional medical procedure is to track the shape of the interventional medical device during the interventional medical procedure. FollowedThe method comprises the steps of: identifying the shape of an optical shape detection device using an optical shape detection signal received via an interface from an optical shape detection device having a shape; and identifying the intervention medical device in a first coordinate space of a first imaging system that images the intervention medical device in a first imaging mode during an intervention medical procedure, based on the identification of the shape of the optical shape detection device. The method also comprises the steps of: registering the intervention medical device in the first coordinate space; identifying the intervention medical device in a second coordinate space of a second imaging system that images the intervention medical device in a second imaging mode during an intervention medical procedure; and registering the first coordinate space of the first imaging system with the second coordinate space of the second imaging system. The method further comprises the steps of: segmenting an interventional medical device in a second coordinate space to obtain a segmented representation of the interventional medical device in the second coordinate space; registering the interventional medical device in the second coordinate space using the segmented representation of the interventional medical device; and re-registering the interventional medical device in the first coordinate space based on the registration of the interventional medical device in the second coordinate space.

[0009]

[0009] Exemplary embodiments will be best understood from the following detailed description when read in conjunction with the drawings in the attached drawings. It should be emphasized that various features are not necessarily drawn to a constant scale. In fact, dimensions are arbitrarily enlarged or reduced for clarity of discussion. Similar reference numerals refer to similar elements, insofar as they are applicable and practical. [Brief explanation of the drawing]

[0010] [Figure 1]

[0010] This figure shows a system for tracking intervention medical devices according to a typical embodiment. [Figure 2A]

[0011] FIG. 1 is a diagram illustrating registration of an interventional medical device with respect to an X-ray imaging system in interventional medical device tracking according to an exemplary embodiment. [Figure 2B]

[0012] FIG. 2 is a diagram illustrating registration of an ultrasound system with respect to an X-ray imaging system in interventional medical device tracking according to an exemplary embodiment. [Figure 2C]

[0013] FIG. 3 is a diagram illustrating registration of an interventional medical device with respect to an ultrasound system in interventional medical device tracking according to an exemplary embodiment. [Figure 3]

[0014] FIG. 4 is a diagram illustrating registration of an interventional medical device with respect to an ultrasound system and an X-ray system in interventional medical device tracking according to another exemplary embodiment. [Figure 4]

[0015] FIG. 5 is a diagram illustrating registration of an interventional medical device with respect to an ultrasound system in interventional medical device tracking according to an exemplary embodiment. [Figure 5]

[0016] FIG. 6 is a diagram illustrating a method for interventional medical device tracking according to an exemplary embodiment. [Figure 6]

[0017] FIG. 7 is a diagram illustrating a method for interventional medical device tracking according to an exemplary embodiment. [Figure 7]

[0018] FIG. 8 is a diagram illustrating a method for interventional medical device tracking according to an exemplary embodiment. [Figure 8]

[0019] FIG. 9 is a diagram illustrating a method for interventional medical device tracking according to an exemplary embodiment. [Figure 9]

[0020] FIG. 10 is a diagram illustrating a computer system on which a method for interventional medical device tracking according to another exemplary embodiment is implemented. DETAILED DESCRIPTION OF EMBODIMENTS

[0011]

[0021] The following detailed descriptions include representative embodiments that disclose specific details for illustrative purposes only, not limitation, to provide a complete understanding of the embodiments described herein. To avoid obscuring the description of the representative embodiments, descriptions of known systems, devices, materials, methods of operation, and methods of manufacture are omitted. Nevertheless, systems, devices, materials, and methods that are within the understanding of those skilled in the art are within the scope of this teaching and will be used in accordance with the representative embodiments. It should be understood that the terminology used herein is solely for the purpose of describing specific embodiments and is not intended to be limiting. Defined terms are in addition to the technical and scientific meanings of defined terms that are generally understood and accepted in the art of this teaching.

[0012]

[0022] Terms such as "first," "second," and "third" are used herein to describe various elements or components, but it will be understood that these elements or components should not be limited by these terms. These terms are used solely to distinguish one element or component from another. Therefore, the first element or component discussed below may be referred to as the second element or component without departing from the teaching of the concepts of the present invention.

[0013]

[0023] The terminology used herein is for the sole purpose of describing specific embodiments and is not intended to be limiting. Where used herein and in the appended claims, singular terms are intended to include both singular and plural forms unless the context clearly indicates otherwise. Furthermore, where used herein, the terms “equipment, includes,” and / or “equipment, includes,” and / or similar terms identify the presence of the described feature, element, and / or component, but do not preclude the presence and addition of one or more other feature, element, component, and / or groups thereof. Where used herein, the term “and / or” includes any and all combinations of one or more of the related enumerated items.

[0014]

[0024] Unless otherwise indicated, when an element or component is described as “connected,” “joined,” or “adjacent” to another element or component, it should be understood that the element or component may be directly connected or linked to the other element or component, or there may be an intervening element or component. That is, these terms and similar terms include cases where one or more intermediate elements or components are used to connect two elements or components. However, when an element or component is described as “directly connected” to another element or component, this includes only cases where two elements or components are connected to each other without any intermediate or intervening elements or components.

[0015]

[0025] Accordingly, this disclosure is intended to illustrate, through its various aspects, embodiments, and / or one or more of its specific features or sub-components, one or more of the advantages specifically set forth below. To provide a complete understanding of the embodiments described herein, rather than limiting them, exemplary embodiments disclosing specific details are provided. However, other embodiments that deviate from the specific details disclosed herein but are not inconsistent with this disclosure remain within the scope of the appended claims. Furthermore, descriptions of well-known apparatuses and methods are omitted so as not to obscure the description of the exemplary embodiments. Such methods and apparatuses are within the scope of this disclosure.

[0016]

[0026] As described herein, interventional medical device tracking utilizes three-dimensional segmentation of interventional medical devices in ultrasound volume and registration of the ultrasound imaging system to the X-ray imaging system. Interventional medical device tracking enables accurate maintenance of OSS device registration to both the ultrasound imaging system and the X-ray imaging system throughout the duration of the interventional medical procedure without repeated X-ray imaging.

[0017]

[0027] Similarly, as described herein, the use of OSS device shapes improves the process of identifying OSS devices in images such as those from ultrasound imaging systems, which in turn improves the registration process described below.

[0018]

[0028] Figure 1 shows a system for tracking interventional medical devices according to a typical embodiment.

[0019]

[0029] Figure 1 shows a tracking system 100. The tracking system 100 includes a controller 190, a console having an interface 193, and a touch panel 196. The controller 190 includes at least a memory 191 for storing instructions and a processor 192 for executing instructions. The controller 190 controls one or more aspects of the methods described herein. The processor 192 retrieves or otherwise receives instructions from the memory 191 via a bus (not shown). Once executed by the processor 192, the instructions cause the controller 190 to implement one or more aspects of the methods shown and described in relation to Figures 5, 6, 7, and 8. The interface 193 provides an interface between the console, including the controller 190, and the optical shape sensing device 102. The interface 193 represents interfaces between elements and components of the tracking system 100. The touch panel 196 includes buttons, keys, and any other touch surfaces that can be used to input instructions from a user to the tracking system 100.

[0020]

[0030] The tracking system 100 also includes a monitor 195, an X-ray imaging system 120, and an ultrasound imaging system 110. The monitor 195 is used to display images from the X-ray imaging system 120 and the ultrasound imaging system 110. In a non-limiting example, the X-ray imaging system 120 performs fluoroscopic imaging during an interventional medical procedure. Similarly, in a non-limiting example, the ultrasound imaging system 110 performs transesophageal echocardiography (TEE) or other forms of ultrasound imaging. The X-ray imaging system 120 performs imaging in a three-dimensional coordinate space centered on the isocenter of the C-arm of the X-ray imaging system 120. The ultrasound imaging system 110 performs imaging in a different three-dimensional coordinate space. The three-dimensional coordinate space of the ultrasound imaging system 110, and the other three-dimensional coordinate space, are registered with the three-dimensional coordinate space of the X-ray imaging system 120, so that the isocenter of the C-arm of the X-ray imaging system 120 becomes the origin of all such registered coordinate spaces.

[0021]

[0031] The tracking system 100 also includes an interventional medical device 101 integrated with an optical shape detection device 102. The optical shape detection device 102 is flexible and adapts to the shape of the interventional medical device 101 during interventional medical procedures. Followed It has a shape. In this specification, references to the intervention medical device 101 also refer to the optical shape detection device 102, insofar as the optical shape detection device 102 is integrated with the intervention medical device 101. On the other hand, references to the optical shape detection device 102 refer to the optical shape detection device 102 independently interfaced to the controller 190 via interface 193 to supply optical shape detection signals generated by the optical shape detection device 102, in particular to the extent that the optical shape detection device 102 is independent of the intervention medical device 101.

[0022]

[0032] The elements and components of the tracking system 100 in Figure 1 may be provided together or distributed. For example, the controller 190, monitor 195 and touch panel 196 may be provided as an integrated computer system separate from the X-ray imaging system 120, ultrasound imaging system 110 and interventional medical device 101. The X-ray imaging system 120, ultrasound imaging system 110 and interventional medical device 101 may be provided separately from each other and may be integrated together by an integrated computer system including the controller 190, monitor 195 and touch panel 196.

[0023]

[0033] The controller 190 includes one or more input interfaces in addition to interface 193. Interface 193 and other input interfaces (not shown) of the controller 190 include cables, adapters, ports, disk drives, wireless communication antennas, and other forms of interfaces specifically used to connect elements and components of the tracking system 100. The input interfaces further connect user interfaces to the controller 190, such as a mouse, keyboard, microphone, video camera, touchscreen display, or other elements or components. The interfaces of the tracking system 100 connect the controller 190 to the monitor 195, the X-ray imaging system 120, and the ultrasound imaging system 110. For example, the controller 190 is connected to the monitor 195 via a local wired interface such as an Ethernet cable, or via a local wireless interface such as a Wi-Fi connection.

[0024]

[0034] Monitor 195 is a computer monitor, a mobile device display, a television, an electronic whiteboard, or another screen configured to display electronic images. Monitor 195 also includes one or more input interfaces, such as those described above, for connecting other elements or components to Monitor 195. Monitor 195 also includes a touchscreen that allows direct input by touch.

[0025]

[0035] In one embodiment, a tracking system 100 tracks an interventional medical device 101 during an interventional medical procedure. An X-ray imaging system 120 is a first imaging system that images the interventional medical device 101 during the interventional medical procedure, and an ultrasound imaging system 110 is a second imaging system that images the interventional medical device 101 during the interventional medical procedure. Instructions stored in memory 191, when executed by processor 192, cause the tracking system 100 to track the location of the interventional medical device 101 during the interventional medical procedure. The process of tracking the interventional medical device 101 includes identifying the shape of an optical shape detection device 102 using an optical shape detection signal received from the optical shape detection device 102 via interface 193. The process in this embodiment also includes identifying the interventional medical device 101 in a first coordinate space of the X-ray imaging system 120 based on the identification of the shape of the optical shape detection device 102. The interventional medical device 101 is then registered in the first coordinate space of the X-ray imaging system 120. The process also includes identifying the interventional medical device 101 in the second coordinate space of the ultrasound imaging system 110. The first coordinate space of the X-ray imaging system 120 is registered with the second coordinate space of the ultrasound imaging system 110. The process for tracking the position of the interventional medical device 101 in this set of embodiments also includes segmenting the interventional medical device 101 in the second coordinate space of the ultrasound imaging system 110 to obtain a segmented representation of the interventional medical device 101 in the second coordinate space. The interventional medical device 101 is then registered in the second coordinate space of the ultrasound imaging system 110 using the segmented representation of the interventional medical device. Subsequently, the interventional medical device 101 is re-registered in the first coordinate space of the X-ray imaging system 120 based on the registration of the interventional medical device 101 in the second coordinate space using the segmented representation.The re-registration of the interventional medical device 101 to the first coordinate space of the X-ray imaging system 120 is performed without requiring further X-ray imaging of the patient. The process performed in this operation is carried out on demand, periodically, or when movement of the interventional medical device 101 exceeding a threshold is detected.

[0026]

[0036] The controller 190 directly performs some of the operations described herein and indirectly performs other operations described herein. For example, the controller 190 directly controls the display of the monitor 195 and indirectly controls imaging by the X-ray imaging system 120 and / or imaging by the ultrasound imaging system 110. Therefore, the process performed by the tracking system 100 when the processor 192 executes instructions from the memory 191 includes steps that are not performed directly by the controller 190.

[0027]

[0037] In another set of embodiments using the tracking system 100, registration is performed using a predetermined shape of the interventional medical device 101. For example, a predetermined shape of the interventional medical device 101 is stored as a template in memory 191 and retrieved from memory 191 for use in exploring the ultrasonic space for the interventional medical device 101. The shape of the interventional medical device 101 is also dynamically obtained from the optical shape detection device 102. In this set of embodiments, the ultrasonic imaging system 110 is the first imaging system, and the X-ray imaging system 120 is the second imaging system. The interventional medical device 101 is registered with the ultrasonic space (first coordinate space) based on the shape of the interventional medical device 101 identified using the optical shape detection signal, and also based on the shape of the interventional medical device 101 identified in the ultrasonic space (first coordinate space). The process of tracking the intervention medical device 101 includes identifying the shape of the optical shape detection device 102 using an optical shape detection signal received from the optical shape detection device 102 via interface 193, and identifying the shape of the intervention medical device 101 in a first coordinate space of an ultrasound system (first imaging system) that images the intervention medical device 101 in a first imaging mode during the intervention medical procedure. The intervention medical device 101 is registered with the ultrasound space (first coordinate space) based on the shape of the intervention medical device 101 identified using the optical shape detection signal, and also based on the shape of the intervention medical device 101 identified in the ultrasound space (first coordinate space). In this set of embodiments, no preliminary registration between the X-ray imaging system, the intervention medical device 101, and the ultrasound imaging system 110 is required in order to register the intervention medical device 101 with the ultrasound imaging system 110 using the known shape of the intervention medical device 101.

[0028]

[0038] Before proceeding to the explanation of Figure 2A, the concepts of registration and segmentation will be explained in more detail below. Registration involves aligning different three-dimensional coordinate systems. In Figure 1, the X-ray imaging system 120, the ultrasound imaging system 110, and the optical shape detection device 102 each have their own three-dimensional coordinate systems. A common three-dimensional coordinate system is provided by aligning different three-dimensional coordinate systems, such as by sharing a common origin and a set of axes. Registration first involves adjusting the origin of one coordinate system to the origin of another coordinate system, and then aligning the axes of that coordinate system with the axes of that other coordinate system. Registration usually involves calculating and applying a transformation matrix based on the observation of common three-dimensional elements in the two coordinate systems.

[0029]

[0039] Segmentation generates representations of the surface of structures, such as anatomical features, and of the interventional medical device 101. The segmented representation consists, for example, of a set of points in three-dimensional (3D) coordinates on the surface of the structure and triangular planar segments defined by connecting groups of three adjacent points, so that the entire structure is covered by a mesh of non-intersecting triangular planes. The three-dimensional model of the interventional medical device 101 is obtained by segmentation. Segmentation also involves segmenting anatomical structures and / or other structures present in the three-dimensional ultrasound volume.

[0030]

[0040] Figure 2A shows the registration of an interventional medical device to an X-ray imaging system in an interventional medical device tracking scenario, according to a typical embodiment.

[0031]

[0041] In Figure 2A, an optical fiber is incorporated into the interventional medical device 201. An example of the optical fiber in Figure 2A is the optical shape detection device 102 in Figure 1. The optical fiber provides the position and orientation of the interventional medical device 201. An example of the interventional medical device in Figure 2A is a catheter with a guidewire. In Figure 2A, the optical fiber is incorporated into the interventional medical device 201 (e.g., the guidewire of the catheter) in the right-hand vascular branch. The interventional medical device 201 with the incorporated optical fiber is superimposed on an X-ray (fluoroscopy) image of a vascular phantom generated by the X-ray imaging system 120.

[0032]

[0042] In Figure 2A, the shape of the optical fiber is identified using an optical shape detection signal received from the optical fiber via an interface such as interface 193. The X-ray imaging system 120 is a first imaging system that generates an X-ray image in a first coordinate space perpendicular to the X-ray imaging system 120. The interventional medical device 201 is registered with the X-ray imaging system 120 by assigning the position of the interventional medical device 201 to the coordinates of the interventional medical device 201 in the first coordinate system based on the X-ray image and the optical shape detection signal. The interventional medical device 201 is registered in the X-ray coordinate space using two X-ray projection images shifted by 30 degrees or more. The operator identifies the tip of the interventional medical device 201 in each X-ray image, and the visible portion of the interventional medical device 201 is automatically detected. The transformation of the interventional medical device 201 and the X-ray coordinate space is determined from the two X-ray projections and reconstructions of the interventional medical device 201 based on the optical shape detection signal from the optical fiber. The interventional medical device 201 is identified in the first coordinate space of the X-ray imaging system 120 during an interventional medical procedure, based on the identification of the shape of the optical shape detection device 102 using an optical shape detection signal from an optical fiber.

[0033]

[0043] Figure 2B shows the registration of an ultrasound system to an X-ray imaging system 120 in an interventional medical device tracking scenario, according to a typical embodiment.

[0034]

[0044] In Figure 2B, registration between the ultrasound imaging system 110 and the X-ray imaging system 120 is achieved by an image fusion platform. An example of an image fusion platform is EchoNavigator. The registration algorithm provided by EchoNavigator is based on acquiring fluorescence fluoroscopy images from the X-ray imaging system 120. The fluorescence fluoroscopy images include the probe head of the ultrasound imaging system 110. As an example, the ultrasound imaging system 110 is a transesophageal echocardiography (TEE) ultrasound system. From the position of the probe head in the X-ray image, the ultrasound space is converted to the X-ray space (T UX ) and the transformation associated with it can be calculated.

[0035]

[0045] Figure 2C shows the registration of an interventional medical device to an ultrasound system in an interventional medical device tracking scenario, according to a typical embodiment.

[0036]

[0046] As shown in Figure 2C, multiple registrations between various coordinate systems can be combined so that three or more coordinate systems are aligned. In Figure 2C, the optical shape detection coordinate system (OSS space) can be registered with the X-ray imaging system coordinate system (X-ray space) using a program executed by the controller 190. The X-ray imaging system coordinate system (X-ray space) can be registered with the local environment (patient space) including the X-ray imaging system 120 using a program such as EchoNavigator executed by the controller 190. Separately, the ultrasound imaging system coordinate system (US space) can be registered with the X-ray imaging system coordinate system (X-ray space) using a program such as EchoNavigator executed by the controller 190.

[0037]

[0047] Assuming that the interventional medical device 101 and the ultrasound imaging system 110 are registered in X-ray space, the interventional medical device 101 can be rendered in the ultrasound imaging system coordinate system (US space) by the transformation outlined in Figure 2C.

[0038]

[0048] Figure 3 shows the registration of an interventional medical device to an ultrasound system and an X-ray system in an interventional medical device tracking, according to another representative embodiment.

[0039]

[0049] In Figure 3, the guidewire 301 is shown in the ultrasound space on the left and in the X-ray space on the right. Since the interventional medical device 101 is registered in both the ultrasound and X-ray spaces, and the ultrasound space is registered with the X-ray space, the images shown in Figure 3 represent the same coordinate system even if the viewpoint differs between the two images.

[0040]

[0050] Figure 4 shows the registration of an interventional medical device to an ultrasound system in an interventional medical device tracking scenario, according to a typical embodiment.

[0041]

[0051] In Figure 4, the reconstruction of the interventional medical device 101 from optical shape detection using the optical shape detection device 102 is shown as the OSS reconstruction 402. The image-based segmentation of the optical shape detection device 102 in ultrasound space is shown as the segmented representation 401. As shown, in the registration process described herein, the OSS reconstruction 402 can be registered with the segmented representation 401 in ultrasound space. OU This associates the current position of the OSS reconstruction 402 with a segmented representation 401 of the position of the interventional medical device 101 in ultrasound space.

[0042]

[0052] In the first set of embodiments described herein, registration is achieved by identifying the position of the interventional medical device 101 in a medical image, such as by user designation, tracking of a sensor integrated at the tip of the interventional medical device 101, or by other means. Three-dimensional segmentation of the interventional medical device 101 in ultrasound images is used to update the registration in the first set of embodiments. Three-dimensional segmentation of tubular interventional medical devices in ultrasound images is achieved by image processing techniques combined with sensor tracking techniques. An example of three-dimensional segmentation of an interventional medical device is described in U.S. Provisional Patent Application No. 62 / 855,013, filed with the U.S. Patent and Trademark Office on 31 May 2019, and the disclosures of this U.S. Provisional Patent Application are incorporated by reference in their entirety. Examples of tubular interventional medical devices that are readily subject to three-dimensional segmentation include guidewires and catheters. An alternative mechanism for identifying interventional medical devices in ultrasound images includes user clicks at the tip position of the interventional medical device in the ultrasound image, and initialization using deep learning with artificial intelligence based on previous instantiations of identifying interventional medical devices in ultrasound images.

[0043]

[0053] The updated registration addresses errors known to occur when optical shape detection is registered in X-ray space and / or ultrasound space. That is, while optical shape detection provides highly accurate reconstruction of the local shape of the interventional medical device 101, it is susceptible to registration misalignment errors due to the accumulation of errors along the length of the interventional medical device 101. For example, while accuracy is very good immediately after registration of the interventional medical device 101 to X-ray space is completed, if the proximal end of the interventional medical device 101 is moved several meters, the registration accumulates significant errors. Using the teachings of the first set of embodiments presented herein, errors can be corrected by re-registration without requiring further X-ray projection and thus without increasing the X-ray dose exposure to the patient and clinician. Furthermore, the registration of the interventional medical device 101 to the X-ray imaging system 120 can be updated continuously, on demand by the clinician (e.g., when the clinician notices an error) or automatically (e.g., when the tracking system 100 detects that the misalignment exceeds a predetermined threshold). By utilizing segmentation of the interventional medical device 101 in the ultrasound space and registration as described herein, the registration of the interventional medical device 101 can be continuously and accurately updated throughout the procedure.

[0044]

[0054] The interventional medical device 101, roughly registered in the X-ray space and the 3D ultrasound space, can automatically maintain a finely tuned registration based on the segmented shape of the interventional medical device 101 in the 3D ultrasound space. The shape of the interventional medical device 101 in the 3D ultrasound space is determined by image processing or deep learning techniques. Therefore, a rigid point-to-point transformation is calculated from the corresponding part of the interventional medical device 101 to the 3D segmentation of the interventional medical device 101 in the ultrasound coordinate system. The precise registration of the interventional medical device 101 can be maintained throughout the procedure by automatically segmenting the interventional medical device 101 in the image and aligning the reconstruction from the optical shape detection device 102.

[0045]

[0055] In the second set of embodiments, registration of the interventional medical device 101 is achieved and updated using a known shape of the interventional medical device 101, which involves a simplified workflow compared to the first set of embodiments. For example, if the shape of the interventional medical device 101 is known, for example, from a template and / or optical shape detection device 102, the shape can be identified in ultrasound coordinate space, and registration between optical shape detection and ultrasound space can be achieved without first registering the interventional medical device 101 with the X-ray imaging system 120. The template of the shape of the interventional medical device 101 is obtained from a library of templates stored in memory, such as memory 191. The template includes a partial template of the shape of the interventional medical device 101, such as a template of the shape of the distal tip of the interventional medical device 101. If the template is a partial template of the shape, the rest of the shape of the interventional medical device 101 is identified based on image analysis software that searches for the rest of the shape of the interventional medical device 101 in a region adjacent to the part of the shape identified in the ultrasound image from the partial template of the shape.

[0046]

[0056] Furthermore, in the second set of embodiments, registration between the ultrasonic coordinate space and the X-ray coordinate space is performed without requiring an X-ray image of the probe head of the ultrasonic imaging system 110. For example, a common shape of the interventional medical device 101 in both coordinate systems is used as a mechanism for registering the two coordinate systems. If the interventional medical device 101 is already registered in the X-ray space, and the interventional medical device 101 is registered in the ultrasonic space using a template of the shape of the interventional medical device 101, the ultrasonic coordinate system is registered with the X-ray coordinate system by calculating the transformation from the segmentation of the interventional medical device 101 in the ultrasonic space to the corresponding interventional medical device 101 in the X-ray space.

[0047]

[0057] In the second set of embodiments, a portion of the shape is used as a constraint in the search for the shape of the interventional medical device 101 in the ultrasound image. Artificial intelligence is applied to analyze the ultrasound image in ultrasound coordinate space. The search is initially constrained by the tip of the interventional medical device 101, and once the tip of the interventional medical device 101 is identified in the search, the artificial intelligence is applied to find the rest of the shape of the interventional medical device 101 based on the characteristics and parameters identified from previous instantiations of the interventional medical device 101 in previous searches of the ultrasound image.

[0048]

[0058] In both the first and second sets of embodiments, metrics are generated to show corrections between the identification of the interventional medical device 101 in various coordinate spaces. For example, the metrics are generated based on a correction between the existing position of the segmented representation of the interventional medical device 101 in ultrasound space and the newly identified position of the interventional medical device 101 in ultrasound coordinates. The correction is an assessment of confidence in the accuracy of identification and is based, for example, on a certain amount of discrepancy between the segmented representation and the proposed newly identified position of the interventional medical device 101 in ultrasound coordinates.

[0049]

[0059] Figure 5 shows a method for tracking interventional medical devices according to a typical embodiment.

[0050]

[0060] In Figure 5, the method begins in S510 by identifying the shape of an optical shape detection device. The optical shape detection device is the optical shape detection device 102 in the embodiment of Figure 1, and includes an optical fiber. The optical shape detection device is identified using the optical shape detection technique described above.

[0051]

[0061] In S520, the method shown in Figure 5 involves identifying the interventional medical device in a first coordinate space. The first coordinate space is the coordinate space of a first imaging system operating in a first imaging mode, for example, an X-ray imaging system 120 operating in X-ray imaging mode. Identification in S520 is performed by a user who specifies the interventional medical device 101 in the X-ray image, or by image analysis software that identifies the interventional medical device 101 in the X-ray image.

[0052]

[0062] In S530, the method shown in Figure 5 involves registering the intervention medical device 101 in a first coordinate space. The registration in S530 is between the intervention medical device 101 and the X-ray space of the X-ray imaging system 120. The registration in S530 is between the optical shape detection device 102 Followed The shape of the interventional medical device 101 is identified in S510 based on its shape. The registration in S520 is also based on the shape of the interventional medical device 101 identified in S520 based on the X-ray image.

[0053]

[0063] In S540, the method shown in Figure 5 involves identifying the interventional medical device 101 in a second coordinate space. The second coordinate space is the coordinate space of a second imaging system operating in a second imaging mode, for example, an ultrasound imaging system 110 operating in ultrasound imaging mode. The identification in S540 is performed by a user specifying the interventional medical device 101 in the ultrasound image, or by image analysis software that identifies the interventional medical device 01 in the ultrasound image. Although not shown in Figure 5, S540 may be performed between S550 and S560 (described below) to register the first coordinate space with the second coordinate space before fine-tuning the registration of the interventional medical device 101 in the second coordinate space (i.e., in the ultrasound space).

[0054]

[0064] In one embodiment, the user identifies the tip of the interventional medical device 101 in the ultrasound image, and the image analysis software limits the search for the rest of the interventional medical device 101 to the area around the designated tip.

[0055]

[0065] In another embodiment, the sensor at the tip of the interventional medical device 101 is a passive ultrasound sensor that responds to radiation from an ultrasound imaging system 110. The sensor-based tracking of the interventional medical device 101 is described in U.S. Provisional Patent Application No. 62 / 855,013, filed with the U.S. Patent and Trademark Office on 31 May 2019, and the disclosures of this U.S. Provisional Patent Application are incorporated by reference in their entirety.

[0056]

[0066] In embodiments using a sensor at the tip of the interventional medical device 101, the image analysis software limits the search of the rest of the interventional medical device 101 to the area around the tip identified from the signal from the sensor. This limitation is based on user identification of the tip of the interventional medical device 101, identification based on the signal from the passive ultrasound sensor, or identification by image analysis software trained by artificial intelligence to recognize the tip of the interventional medical device.

[0057]

[0067] In another embodiment, the interventional medical device 101 performs T from the interventional medical device 101 to the X-ray space OX by transformation, and also performs T from the ultrasound space to the X-ray space UX by transformation, and is coarsely registered to the ultrasound space. Then, the ultrasound image-based device segmentation described below at S560 is continuously calculated throughout the entire acquisition process using the tip of the interventional medical device 101 as a coarse estimation for limiting the search space of the image processing algorithm. The transformation T from the interventional medical device 101 to the ultrasound space OU is calculated for each ultrasound frame, and the registration of the interventional medical device 101 with respect to the ultrasound space is updated continuously or at regular intervals throughout the procedure.

[0058]

[0068] In S550, the first coordinate space is registered with the second coordinate space. The registration in S550 is performed by imaging the head of the ultrasound probe in the ultrasound imaging system 110 using the X-ray imaging system 120. As described above, in some embodiments, S550 may be performed before S540.

[0059]

[0069] In S560, the interventional medical device in the second coordinate space is segmented so as to generate a segmented representation of the interventional medical device. In one embodiment, the segmentation in S560 is initialized by a user who identifies the interventional medical device 101 in the ultrasound image at S540. An image processing algorithm searches for the interventional medical device 101 in the image in the region identified by the user. Rigid transformation T from the interventional medical device 101 to the ultrasound space OU is calculated by rotating / translating the distal portion of the interventional medical device 101, which corresponds to the length of the ultrasound device segmentation, to best match the segmented representation in the ultrasound space. Rigid transformation T OU is then applied to the entire reconstructed length from the optical shape sensing device 102.

[0060]

[0070] In S565, for example, a segmented representation of the interventional medical device 101 is rendered on monitor 195 in Figure 1. The segmented representation of the interventional medical device 101 is superimposed on the ultrasound image and then on the X-ray image.

[0061]

[0071] In S570, the interventional medical device is registered in the second coordinate space of the ultrasound imaging system 110. The registration in S570 is either the initial registration of the interventional medical device 101 in the ultrasound space, or a repeated registration of the interventional medical device 101 to correct an expired previous registration.

[0062]

[0072] In S580, the interventional medical device 101 is re-registered in the first coordinate space of the X-ray imaging system 120. The re-registration in S580 corrects the expired previous registration and does not require further imaging by the X-ray imaging system 120.

[0063]

[0073] In one embodiment, the interventional medical device 101 is segmented in the ultrasound volume based on artificial intelligence from the previous identification of the interventional medical device 101 in the ultrasound image. In this embodiment, user identification or initialization from sensors is not necessarily required to limit the exploration space. Conversion from the interventional medical device 101 to the ultrasound space T OU This is calculated for each ultrasound frame, and the registration of the interventional medical device 101 to the ultrasound space is continuously updated throughout the procedure.

[0064]

[0074] Although not shown in Figure 5, the first coordinate space is re-registered in the second coordinate space by calculating a transformation from the segmented representation of the intervention medical device 101 in the first coordinate space to the shape of the intervention medical device 101 identified in the second coordinate space based on optical shape detection. According to the second set of embodiments described herein, the re-registration is performed without requiring an X-ray image of the probe head of the ultrasound imaging system 110. If the intervention medical device 101 is already registered in the X-ray space and the intervention medical device 101 is already registered in any ultrasound space based on a shape template, the ultrasound space is registered in the X-ray space by calculating a transformation from the segmented representation of the intervention medical device 101 in the ultrasound space to the corresponding section of the intervention medical device 101 in the X-ray space.

[0065]

[0075] Figure 6 shows a method for tracking interventional medical devices according to a typical embodiment.

[0066]

[0076] In the method shown in Figure 6, the selection of the interventional medical device 101 is detected in the second coordinate space of the ultrasound imaging system 110. The selection is the user selection of the tip of the interventional medical device 101.

[0067]

[0077] In S640, the interventional medical device 101 is identified in a second coordinate space. The identification in S640 is based on the selection in S635 and involves using image analysis software to explore the area around where the user selects the rest of the interventional medical device 101.

[0068]

[0078] In S645, the existing position of the interventional medical device 101 is converted to a segmented representation T. OU Calculate the new T. OU Using this transformation, the previous position of the interventional medical device 101 in the ultrasound space can be updated to a new position in the segmented representation.

[0069]

[0079] In S648, the second coordinate space of the ultrasound imaging system 110 is registered with the first coordinate space of the X-ray imaging system based on the segmented representation of the interventional medical device 101 in the second coordinate space of the ultrasound imaging system 110. This registration updates the previous position of the interventional medical device 101 in the X-ray space to account for any errors, for example, caused by the movement of the interventional medical device 101.

[0070]

[0080] The embodiment in Figure 6 is a supplement to the embodiment in Figure 5 and includes functions that complement the functions described with respect to Figure 5.

[0071]

[0081] In the embodiment described in Figure 6 above, the identification of the interventional medical device 101 in the ultrasound space is based on user designation of the tip of the interventional medical device 101. In another embodiment, the known shape of the interventional medical device 101 can be explored in the ultrasound space without requiring knowledge of the shape of the distal tip of the interventional medical device 101. The exploration of the known shape of the interventional medical device 101 is based on information from the optical shape detection signal from the optical shape detection device 102, and conversion T OU This eliminates the need for rough registration of the interventional medical device 101 into the ultrasound space.

[0072]

[0082] Figure 7 shows a method for tracking interventional medical devices according to a typical embodiment.

[0073]

[0083] In Figure 7, at S736, the interventional medical device 101 is identified in the second coordinate space of the ultrasound imaging system 110. The identification at S736 is based on the user selecting the position of the tip of the interventional medical device 101, or on the signal from the passive ultrasound sensor at the tip of the interventional medical device 101.

[0074]

[0084] In S760, the interventional medical device 101 is segmented in the ultrasound image to generate a segmented representation of the interventional medical device 101.

[0075]

[0085] In S770, the interventional medical device 101 is registered in the second coordinate space of the ultrasound imaging system 110. The registration in S770 involves at least transformation T OU Based on.

[0076]

[0086] In S780, the interventional medical device is registered in the first coordinate space of the X-ray imaging system 120. The transformation in S780 is transformation T OX , T UX and T OU It is based on all three of the above.

[0077]

[0087] The embodiment in Figure 7 is a supplement to the embodiment in Figure 5 and includes functions that complement the functions described with respect to Figure 5.

[0078]

[0088] In the description of the embodiment in Figure 7 above, the identification of the interventional medical device 101 in the ultrasonic space is based on user identification or sensor identification of the tip of the interventional medical device 101. In another embodiment, the known shape of the interventional medical device 101 can be retrieved from a template or from information from an optical shape detection signal from an optical shape detection device 102. Template or knowledge-based registration of the shape of the interventional medical device 101 can be used to determine or maintain registration between the ultrasonic space and the X-ray space, in particular when the ultrasonic probe of the ultrasonic imaging system 110 is at an angle that makes detection difficult in the X-ray imaging system 120. Conversion from interventional medical device 101 to ultrasonic space T OU This is known, and the conversion from the intervention medical device 101 to the X-ray space T OX Assuming that is known, the conversion T from the ultrasound space to the X-ray space UX (T UX =inv(T OU )* T OX ) can be roughly determined as follows. Conversion T UX Embodiments that maintain registration by repeatedly updating do not require further exposure to X-rays.

[0079]

[0089] Figure 8 shows a method for tracking interventional medical devices according to a typical embodiment.

[0080]

[0090] In Figure 8, the method begins in S836 by identifying the intervention medical device 101 in a second coordinate space. Identification is performed based on user designation or based on signals from sensors at the tip of the intervention medical device 101. Identification in S836 is also performed continuously, on demand, periodically, or based on the detection of movement of the intervention medical device 101 compared to previous registrations during the intervention medical procedure.

[0081]

[0091] In S837, it is determined whether the position of the interventional medical device deviates from the existing segmented representation by more than a threshold. The determination in S837 is based on the detection of movement of the interventional medical device 101 compared to the previous registration. If the deviation does not exceed the threshold (S837 = No), the method returns to S836; otherwise, it proceeds to S860.

[0082]

[0092] In S860, the interventional medical device 101 is segmented in the ultrasound space.

[0083]

[0093] In S870, the interventional medical device 101 is registered in a second coordinate space based on the segmentation in S860.

[0084]

[0094] In S880, the second coordinate system is re-registered with the first coordinate system. The re-registration in S880 is performed without requiring further exposure to X-rays from the X-ray imaging system 120.

[0085]

[0095] After S880, the process returns to S836. Thus, the process in Figure 8 is recursive, involving repeatedly checking whether the position of the interventional medical device 101 has deviated beyond a threshold from the last existing segmented representation, and correcting the deviation by updating the registration of the interventional medical device 101 in the X-ray space and the ultrasound space.

[0086]

[0096] The embodiment in Figure 8 is a supplement to the embodiment in Figure 5 and includes functions that complement the functions described with respect to Figure 5.

[0087]

[0097] In the embodiment described in Figure 8 above, the identification of the interventional medical device 101 in ultrasound space is also based on knowledge of the tip of the interventional medical device 101. In another embodiment, an irregularly shaped therapeutic device deployed with optical shape detection is delivered via an OSS-enabled delivery sheath or catheter. Known irregularly shaped therapeutic devices are detected in ultrasound space by image-based segmentation or by artificial analysis applied to previous instantiations of identifying the OSS delivery sheath or catheter. Segmentation of irregularly shaped therapeutic devices is used to locate the distal end of the OSS-enabled delivery device in ultrasound space for repeated updates of the registration of the interventional medical device 101 in ultrasound space.

[0088]

[0098] In further embodiments, the registration method described above is automatically triggered. For example, the interventional medical device 101 is continuously segmented in the ultrasound space during background processing. The existing registration of the interventional medical device 101 is updated to best match the interventional medical device 101 in the ultrasound space, either frame by frame, every n frames, or only when a metric representing the correlation and / or offset between the position of the interventional medical device 101 and the position of the segmented representation of the interventional medical device 101 in the ultrasound space exceeds a predetermined threshold.

[0089]

[0099] Furthermore, the registration method described above is triggered on demand. The user interface includes a metric representing the correlation and / or offset between the position of the interventional medical device 101 and the position of the interventional medical device 101 in ultrasound space. The user then selects the "Update Registration" soft button whenever the offset metric exceeds a desired error limit, or whenever the user wishes to update the current registration based on visual inspection.

[0090]

[0100] Furthermore, the user interface provides metrics indicating the success of registration after registration has been performed. The metrics include information regarding the correlation between the shape of the interventional medical device 101 in re-registration and the shape of the interventional medical device 101 in the segmented representation in ultrasound space. Alternatively, the metrics include a confidence level that the correct shape of the interventional medical device 101 was detected.

[0091]

[0101] Figure 9 shows a computer system in which a method for tracking interventional medical devices is implemented according to several representative embodiments.

[0092]

[0102] The computer system 900 in Figure 9 shows a complete set of components for a communication device or computer device. However, the “controller” described herein may be implemented with fewer components than the set of components in Figure 9, such as a combination of memory and a processor. The computer system 900 includes some or all of the components of one or more component devices in a system for tracking interventional medical devices as described herein, but such devices do not necessarily have to include one or more of the elements described for the computer system 900 and may include other elements not described.

[0093]

[0103] Referring to Figure 9, the computer system 900 includes a set of software instructions that can be executed to cause the computer system 900 to perform any of the methods or computer-based functions disclosed herein. The computer system 900 may operate as a standalone device or may be connected to other computer systems or peripheral devices, for example, using a network 901. In an embodiment, the computer system 900 performs logic processing based on digital signals received via an analog-to-digital converter.

[0094]

[0104] In a networked deployment, the computer system 900 operates as a server or client-user computer in a server-client-user network environment, or as a peer computer system in a peer-to-peer (or distributed) network environment. The computer system 900 can also be implemented or incorporated as various devices, such as the controller 190 in Figure 1A, a fixed computer, a mobile computer, a personal computer (PC), a laptop computer, a tablet computer, or any other machine capable of executing a set of software instructions (such as sequential) that specify the actions a machine should perform. The computer system 900 can be implemented as a device in or incorporated into an integrated system that includes additional devices. In one embodiment, the computer system 900 can be implemented using an electronic device that provides voice, video, or data communication. Furthermore, although the computer system 900 is shown singularly, the term “system” includes a collection of systems or subsystems that individually or collectively execute one or more sets of software instructions to perform one or more computer functions.

[0095]

[0105] As shown in Figure 9, the computer system 900 includes a processor 910. The processor 910 is considered a typical example of the processor 192 of the controller 190 in Figure 1, and executes instructions to carry out some or all aspects of the methods and processes described herein. The processor 910 is tangible and non-transient. Where used herein, the term “non-transient” should be interpreted as a characteristic of a state that lasts for a certain period of time, rather than as a permanent characteristic of the state. The term “non-transient” particularly negates instantaneous characteristics such as carrier waves or signals or other forms of characteristics that exist only temporarily at any given time and place. The processor 910 is a manufactured product and / or a mechanical component. The processor 910 is configured to execute software instructions to perform the functions described in the various embodiments herein. The processor 910 may be a general-purpose processor or may be part of an application-specific integrated circuit (ASIC). The processor 910 may also be a microprocessor, microcomputer, processor chip, controller, microcontroller, digital signal processor (DSP), state machine, or programmable logic device. The processor 910 may also be a logic circuit, including a programmable gate array (PGA) such as a field-programmable gate array (FPGA), or another form of circuit including discrete gate and / or transistor logic. The processor 910 may be a central processing unit (CPU), a graphics processing unit (GPU), or both. Furthermore, any processor described herein may include multiple processors, parallel processors, or both. Multiple processors may be contained in a single device or multiple devices, or connected to each other.

[0096]

[0106] As used herein, the term “processor” encompasses electronic components capable of executing programs or machine-executable instructions. References to computing devices including “processors” should be interpreted as including more than one processor or processing core, as in a multi-core processor. A processor may also refer to a collection of processors within a single computer system, or a collection of processors distributed across multiple computer systems. The term “computing device” should also be interpreted as including a collection or network of computing devices, each containing one or more processors. A program consists of software instructions executed by one or more processors located within the same computing device or distributed across multiple computing devices.

[0097]

[0107] The computer system 900 further includes main memory 920 and static memory 930, and the memories of the computer system 900 communicate with each other and with the processor 910 via bus 908. Either or both of the main memory 920 and the static memory 930 are considered typical examples of the memory 191 of the controller 190 in Figure 1B, and store instructions used to carry out some or all aspects of the methods and processors described herein. The memories described herein are tangible storage media that store data and executable software instructions, and are non-transient while the software instructions are stored therein. Where used herein, the term “non-transient” should be interpreted as a characteristic of a state that persists for a certain period of time, rather than as a perpetual characteristic of the state. The term “non-transient” particularly negates the instantaneous characteristics of carrier waves or signals or other forms of characteristics that exist only temporarily at any given time and place. The main memory 920 and the static memory 930 are manufactured products and / or mechanical parts. The main memory 920 and static memory 930 are computer-readable media from which data and executable software instructions can be read by a computer (e.g., a processor 910). The main memory 920 and static memory 930 are implemented as one or more of the following: random access memory (RAM), read-only memory (ROM), flash memory, electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, tapes, compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), floppy disks, Blu-ray disks, or any other form of storage medium known in the art. The memory may be volatile or non-volatile, secure and / or encrypted, or non-secure and / or unencrypted.

[0098]

[0108] "Memory" is an example of a computer-readable storage medium. Computer memory is any memory that is directly accessible to the processor. Examples of computer memory include, but are not limited to, RAM memory, registers, and register files. A reference to "computer memory" or "memory" should be interpreted as potentially referring to multiple memories. Memory may, for example, be multiple memories within the same computer system. Memory may also be multiple memories distributed across multiple computer systems or computer devices.

[0099]

[0109] As shown, the computer system 900 further includes a video display unit 950, such as a liquid crystal display (LCD), organic light-emitting diode (OLED), flat panel display, solid-state display, or cathode ray tube (CRT). Furthermore, the computer system 900 includes an input device 960, such as a keyboard / virtual keyboard, touch input screen, or voice input with voice recognition function, and a cursor control device 970, such as a mouse, touch input screen, or pad. The computer system 900 also optionally includes a disk drive unit 980, a signal generating device 990, such as a speaker or remote control, and / or a network interface device 940.

[0100]

[0110] In one embodiment, as shown in Figure 9, the disk drive unit 980 includes a computer-readable medium 982 in which one or more sets of software instructions 984 (software) are embedded. The set of software instructions 984 is read from the computer-readable medium 982 to be executed by the processor 910. Furthermore, when the software instructions 984 are executed by the processor 910, they perform one or more steps of the methods and processes described herein. In one embodiment, when the computer system 900 executes the software instructions 984, all or part of them reside in the main memory 929, static memory 930 and / or the processor 910. Furthermore, the computer-readable medium 982 includes the software instructions 984 or receives and executes the software instructions 984 in response to propagated signals, thereby allowing devices connected to the network 901 to transmit voice, video, or data through the network 901. The software instructions 984 are transmitted or received through the network 901 via a network interface device 940.

[0101]

[0111] In one embodiment, embodiments of dedicated hardware, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic arrays, and other hardware components, are constructed to implement one or more of the methods described herein. One or more embodiments described herein implement functions using two or more specific interconnected hardware modules or devices, together with relevant control and data signals that can communicate between and through the modules. Thus, this disclosure encompasses embodiments of software, firmware, and hardware. Nothing in this application should be interpreted as being implemented or possible using software alone, rather than hardware such as tangible, non-transient processors and / or memory.

[0102]

[0112] According to various embodiments of this disclosure, the methods described herein are carried out using a hardware computer system that executes a software program. Furthermore, in exemplary, non-limiting embodiments, the implementation may include distributed processing, component / object distributed processing, and parallel processing. Virtual computer system processing may perform one or more of the methods or functions described herein, and the processors described herein may be used to support the virtual processing environment.

[0103]

[0113] Therefore, interventional medical device tracking allows the updated registration to correct the position of the interventional medical device 101. However, interventional medical device tracking is not limited to its application to the specific details described herein, but is rather applicable to further embodiments in which other types of medical imaging systems and interventional medical devices are used.

[0104]

[0114] While interventional medical device tracking has been described with reference to several exemplary embodiments, it should be understood that the language used is descriptive and illustrative, not restrictive. Modifications may be made within the present and amended appendix claims without departing in that manner from the scope and spirit of interventional medical device tracking. While interventional medical device tracking has been described with reference to specific means, materials and embodiments, interventional medical device tracking is not intended to be limited to the disclosed details, but rather extends to all functionally equivalent structures, methods and uses as found in the appendix claims.

[0105]

[0115] The examples of embodiments described herein are intended to provide a general understanding of the structure of various embodiments. The examples are not intended to serve as a complete description of all elements and features of the disclosure described herein. Many other embodiments will be apparent to those skilled in the art upon consideration of this disclosure. Other embodiments may be utilized and derived from this disclosure so that structural and logical substitutions and modifications are made without departing from the scope of this disclosure. Furthermore, the examples are representative only and may not be drawn to scale. Certain proportions in the examples may be exaggerated, while other proportions may be minimized. Therefore, this disclosure and the figures should be considered illustrative and not restrictive.

[0106]

[0116] One or more embodiments of this disclosure are referred to herein individually and / or collectively by the term “invention” solely for convenience and without the intention of voluntarily limiting the scope of this application to any particular invention or inventive concept. Furthermore, it should be understood that while certain embodiments are illustrated and described herein, subsequent arrangements designed to achieve the same or similar objectives may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all subsequent adaptations or variations of various embodiments. Combinations of the above embodiments with other embodiments not specifically described herein will be apparent to those skilled in the art upon consideration of the description.

[0107]

[0117] This abstract of the disclosure is provided in accordance with 37C.FR §1.72(b) of the U.S. Patent Law Enforcement Rules and is filed with the understanding that it is not to be used to interpret or limit the scope or meaning of the claims. Furthermore, in the above “Modes for Carrying Out the Invention,” various features may be grouped together or described in a single embodiment for the sake of streamlining the disclosure. The disclosure is not to be construed as reflecting an intention that the claimed embodiments require more features than those expressly described in each claim. Rather, as the attached claims indicate, the subject matter of the invention may cover fewer features than all of any of the features of the disclosed embodiments. Accordingly, the attached claims are incorporated into the “Modes for Carrying Out the Invention,” and each claim stands alone as defining separately claimed subject matter.

[0108]

[0118] The foregoing description of the disclosed embodiments is provided to enable any person skilled in the art to practice the concepts described herein. Accordingly, the above disclosed subject matter should be considered illustrative rather than restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other practices that fall within the true intent and scope of this disclosure. Therefore, to the maximum extent permitted by law, the scope of this disclosure should be determined by the broadest possible interpretation of the appended claims and their equivalents, and not limited or restricted by the foregoing detailed description.

Claims

1. A system for tracking the position of an interventional medical device in an interventional medical procedure, wherein the system is An interface to an optical shape detection device having a shape that follows the shape of the interventional medical device during the interventional medical procedure, A controller including a memory for storing instructions and a processor for executing the instructions, When the aforementioned instruction is executed by the processor, it will be used to control the system. The optical detection signal received from the optical shape detection device via the interface is used to identify the shape of the optical shape detection device. Based on identifying the shape of the optical shape detection device, the intervention medical device is identified in a first coordinate space of an image of a first imaging system that images the intervention medical device in a first imaging mode during the intervention medical procedure. The interventional medical device is registered in the first coordinate space. During the interventional medical procedure, the interventional medical device is identified in the second coordinate space of the image of the second imaging system, which images the interventional medical device in a second imaging mode. The first coordinate space of the first imaging system is registered with the second coordinate space of the second imaging system. In the second coordinate space, the intervention medical device is segmented, The segmented interventional medical devices are registered in the second coordinate space. A system for re-registering an interventional medical device in the first coordinate space based on registering the interventional medical device in the second coordinate space.

2. The aforementioned system, The optical shape detection device and, The first imaging system is an X-ray imaging system, The second imaging system described above is an ultrasonic imaging system, It further includes, The aforementioned instruction further states to the system: The system according to claim 1, wherein the segmented interventional medical device is rendered in the second coordinate space.

3. The aforementioned instruction further states to the system: The conversion from the interventional medical device identified in the second coordinate space to the segmented interventional medical device in the second coordinate space is calculated. The system according to claim 1, wherein the second coordinate space is registered with the first coordinate space based on the transformation.

4. The aforementioned instruction further states to the system: The system detects the selection of the interventional medical device in the image from the second imaging system, and searches for the interventional medical device in the region of the image adjacent to the selection. The system according to claim 3, wherein the conversion is calculated by converting the distal portion of the interventional medical device in the image to match the corresponding distal portion of the interventional medical device in the segmentation.

5. The aforementioned instruction further states to the system: As a constraint in the image search in the second coordinate space of the system, the tip of the interventional medical device is used in the initial search to repeatedly identify the interventional medical device in the image. In the second imaging mode, the intervention medical device is segmented for each frame of the image, The system according to claim 3, wherein the interventional medical device is repeatedly registered in the second coordinate space.

6. The aforementioned instruction further states to the system: The system repeatedly identifies the intervention medical device in the image of the second coordinate space. The system determines when the position of the interventional medical device deviates by more than a predetermined threshold from the segmented interventional medical device in the second coordinate space. If the position of the intervention medical device deviates beyond the predetermined threshold, the intervention medical device is resegmented. The system according to claim 1, wherein the interventional medical device is re-registered in the second coordinate space, and if the position of the interventional medical device is shifted by more than a predetermined threshold, the second coordinate space is re-registered with the first coordinate space.

7. A tangible, non-temporary, computer-readable storage medium for storing computer programs, wherein, when executed by a processor, the computer program is stored in a system including the tangible, non-temporary, computer-readable storage medium. Using the optical shape detection signal received via the interface, the shape of the optical shape detection device is identified, which has a shape that follows the shape of the interventional medical device during the interventional medical procedure. Based on identifying the shape of the optical shape detection device, the intervention medical device is identified in a first coordinate space of an image of a first imaging system that images the intervention medical device in a first imaging mode during the intervention medical procedure. The interventional medical device is registered in the first coordinate space. During the interventional medical procedure, the interventional medical device is identified in the second coordinate space of the image of the second imaging system, which images the interventional medical device in a second imaging mode. The first coordinate space of the first imaging system is registered with the second coordinate space of the second imaging system. The intervention medical device in the second coordinate space is segmented, Using the segmented interventional medical device, the interventional medical device is registered in the second coordinate space. A tangible, non-temporary, computer-readable storage medium that causes the interventional medical device to be re-registered in the first coordinate space based on registering the interventional medical device in the second coordinate space.

8. The aforementioned computer program further installs the aforementioned system. The intervention medical device is identified in the second coordinate space of the second imaging system. The conversion from the interventional medical device identified in the second coordinate space to the segmented interventional medical device in the second coordinate space is calculated. The computer-readable storage medium according to claim 7, wherein the second coordinate space is registered with the first coordinate space based on the transformation.

9. A method for tracking the location of an interventional medical device in an interventional medical procedure, The steps include: identifying the shape of an optical shape detection device having a shape that follows the shape of the interventional medical device during the interventional medical procedure, using an optical shape detection signal received via the interface; A step of identifying the intervention medical device in a first coordinate space of an image of a first imaging system that images the intervention medical device in a first imaging mode during the intervention medical procedure, based on identifying the shape of the optical shape detection device, The steps include registering the interventional medical device in the first coordinate space, The steps include identifying the intervention medical device in a second coordinate space of an image of a second imaging system that images the intervention medical device in a second imaging mode during the intervention medical procedure, The steps include registering the first coordinate space of the first imaging system with the second coordinate space of the second imaging system, The steps include segmenting the intervention medical device in the second coordinate space, The steps include registering the interventional medical device in the second coordinate space using the segmented interventional medical device, The steps include: registering the interventional medical device in the second coordinate space and then re-registering the interventional medical device in the first coordinate space; A method having

10. The steps include rendering the segmented interventional medical device in the second coordinate space, The steps of repeating the segmentation step, registering the intervention medical device in the second coordinate space, and re-registering the intervention medical device in the first coordinate space, The method according to claim 9, further comprising the above.

11. The method according to claim 10, wherein the steps of segmenting, registering the interventional medical device with respect to the second coordinate space, and re-registering the interventional medical device with respect to the first coordinate space are performed based on user selection of the interventional medical device in the second coordinate space.

12. The method according to claim 10, wherein the repeated step is performed periodically and automatically.

13. The method further includes the step of detecting that the position of the interventional medical device in the image in the second coordinate space has moved from the segmented position of the interventional medical device, The method according to claim 10, wherein the steps of segmenting, registering the intervention medical device with respect to the second coordinate space, and re-registering the intervention medical device with respect to the first coordinate space are performed automatically based on the detection step.

14. The steps include identifying the intervention medical device in the second coordinate space of the second imaging system, A step of calculating a transformation from the interventional medical device identified in the second coordinate space to the segmented interventional medical device in the second coordinate space, The steps include registering the second coordinate space with the first coordinate space based on the transformation, The method according to claim 9, further comprising the above.

15. The method further includes detecting the selection of the interventional medical device in the image from the second imaging system, and searching for the interventional medical device in a region of the image adjacent to the selection. The method according to claim 14, wherein the conversion is calculated by converting the distal portion of the interventional medical device in the image to match the corresponding distal portion of the interventional medical device in the segmentation.

Images