Interventional medical device tracking

Abstract

A system for tracking a position of an interventional medical device (01) comprises an interface (193) and a controller (190). The interface (193) interfaces the system to an optical shape sensing device (102) whose shape conforms to a shape of the interventional medical device (01). The controller (190) comprises a memory (191) storing instructions and a processor (192) executing the instructions. The instructions cause the system to identify a shape of the optical shape sensing device (102) using optical shape sensing signals received from the optical shape sensing device (102) via the interface (193) and to identify a shape of the interventional medical device (01) in a first coordinate space of a first imaging system that images the interventional medical device (01) in a first imaging modality. The instructions further cause the system to register the interventional medical device (01) to the first coordinate space.

Description

Background Technology

[0001] Optical shape sensing (OSS) technology is used to provide real-time in-procedure information on the shape and relative position of interventional medical devices during interventional procedures. Information from OSS is used to locate and navigate the interventional medical device during the procedure. OSS uses light along a multi-core optical fiber that conforms to the shape of the interventional medical device during the procedure. The principle involved utilizes distributed strain measurements within the optical fiber, which employ characteristic Rayleigh backscattering or a controlled grating pattern. The shape along the fiber begins from a specific point (referred to as the emission point or z=0), and subsequent shape, position, and orientation are relative to the emission point.

[0002] Individually, registration is used to align the coordinate systems of two separate devices and / or systems. For example, it can be achieved via a transformation T from an OSS device to an X-ray imaging system. OX This is to complete the registration from the OSS device to the X-ray imaging system. It can be achieved via a transformation from an ultrasound imaging system to an X-ray imaging system. UX This is to complete the registration from the ultrasound imaging system to the X-ray imaging system. It can be achieved via a transformation from the OSS device to the ultrasound imaging system. OU To complete the registration from the OSS equipment to the ultrasound imaging system.

[0003] In addition, segmentation is used in medical imaging systems to represent the surface of a structure as a three-dimensional model.

[0004] Currently, registration between the OSS device and the X-ray imaging system can accumulate significant errors, for example, when the proximal end of the OSS device (i.e., the end closest to the user) moves a few centimeters. Correcting these errors requires re-registration between the OSS device and the X-ray imaging system, which in turn requires two new offset X-ray projections. These additional X-ray projections can disrupt interventional medical procedures, potentially prolonging their duration, and subject patients and clinicians to additional X-ray doses.

[0005] Registration between the OSS device and the ultrasound imaging system can also accumulate significant errors. Reregistration between the OSS device and the ultrasound imaging system can disrupt the interventional procedure and prolong its duration when image analysis software searches for the latest ultrasound images of the OSS device. Image analysis software may require specifying the OSS device in the ultrasound images, for example, by having the user specify the tip of the OSS device in the ultrasound images, to constrain the search for the OSS device in the ultrasound images. Even with initial constraints, the OSS device must be fully identified and located in the ultrasound images for reregistration between the OSS device and the ultrasound imaging system, which disrupts the interventional procedure and prolongs its duration. Summary of the Invention

[0006] According to one aspect of this disclosure, a system for tracking the position of an interventional medical device during an interventional medical procedure includes an interface and a controller. The interface connects the system to an optical shape sensing device, the shape of which is consistent with the shape of the interventional medical device during the interventional medical procedure. The controller includes a memory storing instructions and a processor executing the instructions. When executed by the processor, the instructions cause the system to: identify the shape of the optical shape sensing device using optical shape sensing signals received from the optical shape sensing device via the interface, and identify the shape of the interventional medical device in a first coordinate space of a first imaging system that images the interventional medical device in a first imaging mode during the interventional medical procedure. The instructions also cause the system to register the interventional medical device to the first coordinate space based on the shape of the interventional medical device identified using the optical shape sensing signals and based on the shape of the interventional medical device identified in the first coordinate space.

[0007] According to another aspect of this disclosure, a tangible non-transient computer-readable storage medium stores a computer program. When executed, the computer program causes a system including the tangible non-transient computer-readable storage medium to: identify the shape of an optical shape sensing device using optical shape sensing signals received via an interface, the shape of which remains consistent with the shape of an interventional medical device during an interventional medical procedure; and to identify the shape of the interventional medical device in a first coordinate space of a first imaging system that images the interventional medical device in a first imaging mode during the interventional medical procedure. The computer program further causes the system to register the interventional medical device to the first coordinate space based on the shape of the interventional medical device identified using the optical shape sensing signals and based on the shape of the interventional medical device identified in the first coordinate space.

[0008] According to another aspect of this disclosure, a method for tracking the position of an interventional medical device during an interventional medical procedure includes: identifying the shape of an optical shape sensing device using an optical shape sensing signal received via an interface, the shape of the optical shape sensing device being consistent with the shape of the interventional medical device during the interventional medical procedure; and identifying the shape of the interventional medical device in a first coordinate space of a first imaging system, the first imaging system imaging the interventional medical device in a first imaging mode during the interventional medical procedure. The method further includes registering the interventional medical device to the first coordinate space based on the shape of the interventional medical device identified using the optical shape sensing signal and based on the shape of the interventional medical device identified in the first coordinate space. Attached Figure Description

[0009] When with attachment Figure 1 The best understanding of the exemplary embodiments will be achieved by reading the following description. It should be emphasized that the various features are not necessarily drawn to scale. In fact, dimensions may be arbitrarily increased or decreased for clarity of discussion. Wherever applicable and useful, the same reference numerals refer to the same elements.

[0010] Figure 1 The illustration shows a system for tracking interventional medical devices according to a representative embodiment.

[0011] Figure 2A The illustration shows a registration operation in interventional medical device tracking, where an interventional medical device is registered to an X-ray imaging system, according to a representative embodiment.

[0012] Figure 2B The illustration shows a registration operation of an ultrasound system to an X-ray imaging system in interventional medical device tracking according to a representative embodiment.

[0013] Figure 2C The illustration shows the registration operation of registering an interventional medical device to an ultrasound system in interventional medical device tracking according to a representative embodiment.

[0014] Figure 3 The illustration shows the registration operation of registering an interventional medical device to an ultrasound system and an X-ray system in interventional medical device tracking according to another representative embodiment.

[0015] Figure 4 The illustration shows the registration operation of registering an interventional medical device to an ultrasound system in interventional medical device tracking according to a representative embodiment.

[0016] Figure 5 The illustration depicts a method for tracking interventional medical devices according to a representative embodiment.

[0017] Figure 6 The illustration depicts a method for tracking interventional medical devices according to a representative embodiment.

[0018] Figure 7 The illustration depicts a method for tracking interventional medical devices according to a representative embodiment.

[0019] Figure 8 The illustration depicts a method for tracking interventional medical devices according to a representative embodiment.

[0020] Figure 9 The illustration shows a computer system according to another representative embodiment, on which a method for tracking interventional medical devices is implemented. Detailed Implementation

[0021] In the following detailed description, representative embodiments with specific details disclosed are set forth for purposes of explanation and not limitation, in order to provide a thorough understanding of embodiments according to this teaching. Descriptions of known systems, apparatuses, materials, methods of operation, and methods of manufacture may be omitted to avoid obscuring the description of representative embodiments. Nevertheless, systems, apparatuses, materials, and methods within the scope of this teaching are all within the capabilities of those skilled in the art and can be used according to representative embodiments. It should be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. The defined terms have meanings other than their scientific and technical meanings as commonly understood and accepted in the art of this teaching.

[0022] It should be understood that although the terms first, second, third, etc., may be used herein to describe various elements or components, these elements or components should not be limited by these terms. These terms are only used to distinguish one element or component from another. Therefore, the first element or component discussed below may also be referred to as the second element or component without departing from the teachings of the inventive concept.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. The singular forms of the terms “a,” “an,” and “the” as used in the specification and claims are intended to include both the singular and plural forms unless the context clearly indicates otherwise. Additionally, when used herein, the terms “comprising” and / or “including” and / or similar terms specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof. The term “and / or” as used herein includes any and all combinations of one or more of the associated items listed.

[0024] Unless otherwise stated, when an element or component is said to be "connected to," "coupled to," or "adjacent to" another element or component, it should be understood that the element or component can be directly connected to or coupled to the other element or component, and that intermediate elements or components may be present. That is, these and similar terms cover situations where one or more intermediate elements or components may be used to connect two elements or components. However, when an element or component is said to be "directly connected to" another element or component, this only covers situations where two elements or components are connected to each other without any intermediary or intermediate elements or components.

[0025] This disclosure is therefore intended to provide one or more advantages specifically indicated below through its various aspects, embodiments, and / or particular features or sub-components. For purposes of explanation and not limitation, exemplary embodiments with specific details disclosed are set forth in order to provide a thorough understanding of embodiments pursuant to these teachings. However, other embodiments consistent with this disclosure but departing from the specific details disclosed herein remain within the scope of the claims. Furthermore, descriptions of well-known apparatuses and methods may be omitted to avoid obscuring the description of exemplary embodiments. Such methods and apparatuses are within the scope of this disclosure.

[0026] As described in this article, interventional medical device tracking can utilize the three-dimensional segmentation of the interventional medical device within the ultrasound volume and the registration of the ultrasound imaging system with the X-ray imaging system. Interventional medical device tracking can accurately maintain the registration of the OSS device with the ultrasound imaging system and the X-ray imaging system throughout the entire duration of the interventional medical procedure without repeating X-ray imaging.

[0027] As also described herein, using the shape of the OSS device can enhance the process for identifying the OSS device in images, for example, from an ultrasound imaging system, and this in turn can enhance the registration process described below.

[0028] Figure 1 The illustration shows a system for tracking interventional medical devices according to a representative embodiment.

[0029] Figure 1 The figure illustrates a tracking system 100. The tracking system 100 includes a console having a controller 190, an interface 193, and a touch panel 196. The controller 190 includes at least a memory 191 storing instructions and a processor 192 executing those instructions. The controller 190 controls one or more aspects of the methods described herein. The processor 192 retrieves or otherwise receives instructions from the memory 191 via a bus (not shown). These instructions, when executed by the processor 192, cause the controller 190 to perform actions regarding… Figure 5 , Figure 6 , Figure 7 and Figure 8One or more aspects of the methods shown and described. Interface 193 provides an interface between a console including controller 190 and optical shape sensing device 102. Interface 193 represents an interface between elements and components of tracking system 100. Touch panel 196 includes buttons, keys, and any other touch surfaces capable of inputting commands from a user to tracking system 100.

[0030] The tracking system 100 also includes a monitor 195, an X-ray imaging system 120, and an ultrasound imaging system 110. The monitor 195 can be used to display images from the X-ray imaging system 120 and the ultrasound imaging system 110. As a non-limiting example, the X-ray imaging system 120 can perform fluorescence imaging during interventional medical procedures. Also as a non-limiting example, the ultrasound imaging system 110 can perform transesophageal echocardiography (TEE) or other forms of ultrasound imaging. The X-ray imaging system 120 performs imaging in a three-dimensional coordinate space centered at the isocenter of the C-arm of the X-ray imaging system 120. The ultrasound imaging system 110 performs imaging in another three-dimensional coordinate space. The three-dimensional coordinate space of the ultrasound imaging system 110 and other three-dimensional coordinate spaces can be registered to the three-dimensional coordinate space of the X-ray imaging system 120, such that the isocenter of the C-arm of the X-ray imaging system 120 becomes the origin of all such registered coordinate spaces.

[0031] The tracking system 100 also includes an interventional medical device 101 integrated with an optical shape sensing device 102. The optical shape sensing device 102 may be flexible and may have a shape that flexibly conforms to the shape of the interventional medical device 101 during the interventional medical procedure. In the description herein, references to the interventional medical device 101 also refer to the optical shape sensing device 102, provided that the optical shape sensing device 102 is integrated with the interventional medical device 101. Alternatively, references to the optical shape sensing device 102 may specifically refer to the optical shape sensing device 102 independent of the interventional medical device 101, provided that the optical shape sensing device 102 is independently interfaced with the controller 190 via interface 193 to provide the optical shape sensing signal generated by the optical shape sensing device 102.

[0032] Figure 1The components and parts of the tracking system 100 can be provided together or distributed. For example, the controller 190, monitor 195, and touch panel 196 can be provided as an integrated computer system, separate from the X-ray imaging system 120, ultrasound imaging system 110, and interventional medical device 101. The X-ray imaging system 120, ultrasound imaging system 110, and interventional medical device 101 can be provided separately from each other and can be integrated together via an integrated computer system including the controller 190, monitor 195, and touch panel 196.

[0033] In addition to interface 193, controller 190 may also include one or more input interfaces. Other input interfaces (not shown) of interface 193 and controller 190 may include cables, adapters, ports, disk drives, antennas for wireless communication, and other forms of interfaces specifically designed for connecting elements and components of tracking system 100. Input interfaces may also connect user interfaces (e.g., mouse, keyboard, microphone, video camera, touchscreen display, or other elements or components) to controller 190. The interfaces of tracking system 100 can connect controller 190 to monitor 195, X-ray imaging system 120, and ultrasound imaging system 110. For example, controller 190 may be connected to monitor 195 via a local wired interface (e.g., Ethernet cable) or via a local wireless interface (e.g., Wi-Fi connection).

[0034] Monitor 195 may be a computer monitor, a display on a mobile device, a television, an electronic whiteboard, or another screen configured to display electronic images. Monitor 195 may also include one or more input interfaces, such as those mentioned above that allow other elements or components to be connected to monitor 195. Monitor 195 may also include a touchscreen that enables direct input via touch.

[0035] In one set of embodiments, the tracking system 100 tracks the interventional medical device 101 during an interventional medical procedure. An X-ray imaging system 120 may be a first imaging system imaging the interventional medical device 101 during the interventional medical procedure, and an ultrasound imaging system 110 may be a second imaging system imaging the interventional medical device 101 during the interventional medical procedure. Instructions stored in memory 191, when executed by processor 192, cause the tracking system 100 to track the position of the interventional medical device 101 during the interventional medical procedure. The process of tracking the interventional medical device 101 may include identifying the shape of the optical shape sensing device 102 using optical shape sensing signals received from the optical shape sensing device 102 via interface 193. The process in this set of embodiments may also include identifying the interventional medical device 101 in a first coordinate space of the X-ray imaging system 120 based on the identified shape of the optical shape sensing device 102. The interventional medical device 101 is then registered to the first coordinate space of the X-ray imaging system 120. The process may also include identifying the interventional medical device 101 in a second coordinate space of the ultrasound imaging system 110. The X-ray imaging system 120 is registered to the second coordinate space of the ultrasound imaging system 110. The process of tracking the position of the interventional medical device 101 in this embodiment may further include segmenting the interventional medical device 101 in the second coordinate space of the ultrasound imaging system 110 to obtain a segmented representation of the interventional medical device 101 in the second coordinate space. The segmented representation of the interventional medical device is then used to register the interventional medical device 101 to the second coordinate space of the ultrasound imaging system 110. Subsequently, based on the registration of the interventional medical device 101 to the second coordinate space using the segmented representation, the interventional medical device 101 is re-registered to the first coordinate space of the X-ray imaging system 120. This re-registration of the interventional medical device 101 to the first coordinate space of the X-ray imaging system 120 is performed without requiring additional X-ray imaging of the patient. The process performed in this set of operations can be performed on demand, periodically, or once movement of the interventional medical device 101 is detected to exceed a threshold.

[0036] The controller 190 can directly execute some of the operations described herein, and can indirectly implement other operations described herein. For example, the controller 190 can directly control the display of the monitor 195, and indirectly control the imaging performed by the X-ray imaging system 120 and / or the imaging performed by the ultrasound imaging system 110. Therefore, when the processor 192 executes instructions from the memory 191, the process implemented by the tracking system 100 may include steps that are not directly executed by the controller 190.

[0037] In another set of embodiments using the tracking system 100, registration can be performed using a predetermined shape of the interventional medical device 101. For example, the predetermined shape of the interventional medical device 101 can be stored as a template in memory 191, and the template can be retrieved from memory 191 for searching for the interventional medical device 101 in the ultrasound space. The shape of the interventional medical device 101 can also be dynamically obtained from the optical shape sensing device 102. In this set of embodiments, the ultrasound imaging system 110 can be a first imaging system, and the X-ray imaging system 120 can be a second imaging system. The interventional medical device 101 can be registered to the ultrasound space (first coordinate space) based on the shape of the interventional medical device 101 identified using optical shape sensing signals and based on the shape of the interventional medical device 101 identified in the ultrasound space (first coordinate space). The process for tracking interventional medical device 101 may include: identifying the shape of optical shape sensing device 102 using optical shape sensing signals received from optical shape sensing device 102 via interface 193, and identifying the shape of interventional medical device 101 in a first coordinate space of an ultrasound system (first imaging system) that images interventional medical device 101 in a first imaging mode during the interventional medical procedure. The interventional medical device 101 is registered to the ultrasound space (first coordinate space) based on the shape of interventional medical device 101 identified using the optical shape sensing signals and based on the shape of interventional medical device 101 identified in the ultrasound space (first coordinate space). In this set of embodiments, in order to register interventional medical device 101 to ultrasound imaging system 110 using the known shape of interventional medical device 101, it is not necessary to initially register an X-ray imaging system to interventional medical device 101 and ultrasound imaging system 110.

[0038] To continue describing Figure 2A Previously, the concepts of registration and segmentation will be explained in more detail below. Registration involves aligning different three-dimensional coordinate systems. Figure 1 In this system, the X-ray imaging system 120, the ultrasound imaging system 110, and the optical shape sensing device 102 can each have their own three-dimensional coordinate system. A common three-dimensional coordinate system is provided by aligning the different three-dimensional coordinate systems (e.g., by sharing a common origin and a set of axes). Registration may involve first adjusting the origin of one coordinate system to the origin of another, and then aligning the axes of one coordinate system with the axes of the other. Registration typically involves calculating and applying a transformation matrix based on observations of common three-dimensional elements in the two coordinate systems.

[0039] Segmentation produces a representation of the surface of a structure (e.g., anatomical features and interventional medical device 101). The segmentation representation includes, for example, a set of points in three-dimensional (3-D) coordinates on the surface of the structure, and triangular planar segments defined by connecting adjacent sets of three points, such that the entire structure is covered by a mesh of non-intersecting triangular planes. A three-dimensional model of the interventional medical device 101 is obtained through segmentation. Segmentation may also involve performing segmentation on anatomical structures and / or other structures present in the three-dimensional ultrasound volume.

[0040] Figure 2A The illustration shows a registration operation in interventional medical device tracking, where an interventional medical device is registered to an X-ray imaging system, according to a representative embodiment.

[0041] exist Figure 2A In this process, optical fibers are integrated into the interventional medical device 201. Figure 2A An example of optical fiber is Figure 1 The optical shape sensing device 102 in the middle. The optical fiber provides the position and orientation of the interventional medical device 201. Figure 2A An example of interventional medical device 201 is a catheter with a guidewire. Figure 2A In this process, an optical fiber can be integrated into an interventional medical device 201 in a right-side vascular branch (e.g., integrated into the guidewire of a catheter). The interventional medical device 201 with integrated optical fiber is superimposed on an X-ray (fluorescence) image of a vascular phantom generated by an X-ray imaging system 120.

[0042] exist Figure 2A In this system, the shape of the optical fiber can be identified using optical shape sensing signals received from the optical fiber via an interface (e.g., interface 193). The X-ray imaging system 120 can be a first imaging system that generates X-ray images in a first coordinate space specific to the X-ray imaging system 120. The interventional medical device 201 is registered to the X-ray imaging system 120 by assigning the position of the interventional medical device 201 from the optical shape sensing signals to coordinates of the interventional medical device 201 in the first coordinate system based on the X-ray images. The interventional medical device 201 can be registered to the X-ray coordinate space using two X-ray projection images offset by 30 degrees or more. The operator identifies the tip of the interventional medical device 201 in each X-ray image and automatically detects the visible portion of the interventional medical device 201. Based on the optical shape sensing signals from the optical fiber, the transformation from the interventional medical device 201 to the X-ray coordinate space is determined according to the reconstruction of the two X-ray projections and the interventional medical device 201. The interventional medical device 201 is identified in the first coordinate space of the X-ray imaging system 120 during interventional medical procedures by recognizing the shape of the optical shape sensing device 102 using optical shape sensing signals from optical fibers.

[0043] Figure 2BThe illustration shows a registration operation of an ultrasound system to an X-ray imaging system 120 in interventional medical device tracking according to a representative embodiment.

[0044] exist Figure 2B In this process, registration between the ultrasound imaging system 110 and the X-ray imaging system 120 is performed using an image fusion platform. An example of such an image fusion platform is EchoNavigator. The registration algorithm provided by EchoNavigator is based on a fluorescence image acquired from the X-ray imaging system 120. The fluorescence image contains the probe head of the ultrasound imaging system 110. For example, the ultrasound imaging system 110 could be a transesophageal echocardiography (TEE) system. Based on the pose of the probe head in the X-ray image, a transformation (T0) that associates the ultrasound space with the X-ray space can be calculated. UX ).

[0045] Figure 2C The illustration shows the registration operation of registering an interventional medical device to an ultrasound system in interventional medical device tracking according to a representative embodiment.

[0046] like Figure 2C As shown, it can integrate multiple registrations between different coordinate systems, enabling the alignment of three or more coordinate systems. Figure 2C In this system, a program executed by controller 190 can be used to register the optical shape sensing coordinate system (OSS space) to the X-ray imaging system coordinate system (X-ray space). A program executed by controller 190 (e.g., EchoNavigator) can also be used to register the X-ray imaging system coordinate system (X-ray space) to the local environment (patient space) including the X-ray imaging system 120. Additionally, a program executed by controller 190 (e.g., EchoNavigator) can be used to register the ultrasound imaging system coordinate system (US space) to the X-ray imaging system coordinate system (X-ray space).

[0047] Once the interventional medical device 101 and the ultrasound imaging system 110 have been registered in the X-ray space, it is possible to... Figure 2C The transformations outlined in the diagram are used to plot the interventional medical device 101 in the ultrasound imaging system coordinate system (US space).

[0048] Figure 3 The illustration shows the registration operation of registering an interventional medical device to an ultrasound system and an X-ray system in interventional medical device tracking according to another representative embodiment.

[0049] exist Figure 3 In the image, guidewire 301 is shown in both the ultrasound space on the left and the X-ray space on the right. Because the interventional medical device 101 is registered to both the ultrasound space and the X-ray space, and the ultrasound space is registered to the X-ray space, therefore... Figure 3 The images shown reflect the same coordinate system, even though the perspectives of the two images differ.

[0050] Figure 4 The illustration shows the registration operation of registering an interventional medical device to an ultrasound system in interventional medical device tracking according to a representative embodiment.

[0051] exist Figure 4 In this document, the reconstruction operation of the interventional medical device 101 using the optical shape sensing device 102 based on optical shape sensing is denoted as OSS reconstruction 402. Image-based segmentation of the optical shape sensing device 102 in ultrasound space is denoted as segmentation representation 401. As shown, in the registration process described herein, the OSS reconstruction 402 can be registered to the segmentation representation 401 in ultrasound space. Transform T OU The current position of the OSS reconstruction 402 is associated with the segmented representation 401 of the position of the interventional medical device 101 in the ultrasound space.

[0052] In the first set of embodiments described herein, registration can be accomplished by identifying the location of interventional medical device 101 in medical images (e.g., by user specification, tracking sensors integrated into the tip of interventional medical device 101, etc.). Three-dimensional segmentation of interventional medical device 101 in ultrasound images can be used to update the registration in the first set of embodiments. Three-dimensional segmentation of tubular interventional medical devices in ultrasound can be achieved using image processing techniques incorporating sensor tracking technology. Examples of three-dimensional segmentation of interventional medical devices are explained in U.S. Provisional Patent Application US 62 / 855013, filed May 31, 2019, with the U.S. Patent and Trademark Office, the disclosure of which is incorporated herein by reference in its entirety. Examples of tubular interventional medical devices readily susceptible to three-dimensional segmentation include guidewires and catheters. Alternative mechanisms for identifying interventional medical devices in ultrasound images include initialization by a user clicking on the location of the tip of the interventional medical device in the ultrasound image and deep learning based on prior instantiation of interventional medical device identification in ultrasound images using artificial intelligence.

[0053] The updated registration addresses errors known to occur when optical shape sensing is registered to X-ray and / or ultrasound space. That is, while optical shape sensing provides a highly accurate reconstruction of the local shape of interventional medical device 101, it can be prone to errors in registration misalignment due to the accumulation of errors along the length of interventional medical device 101. For example, while accuracy may be excellent immediately after registration of interventional medical device 101 to X-ray space, significant errors can accumulate if the proximal end of interventional medical device 101 is moved a few centimeters. Using the teachings of the first set of embodiments provided herein, errors can be corrected by re-registration without additional exposure to X-ray projection, thus not increasing X-ray dose exposure to patients and clinicians. Furthermore, the registration operation of interventional medical device 101 to X-ray imaging system 120 can be continuously updated, either upon request by the clinician (e.g., when the clinician notices an error) or automatically (e.g., when tracking system 100 detects misalignment exceeding a predetermined threshold). By utilizing segmentation of the interventional medical device 101 in the ultrasound space and registration as described herein, the registration of the interventional medical device 101 can be continuously and accurately updated throughout the duration of the procedure.

[0054] The interventional medical device 101, which has been roughly registered to X-ray space and three-dimensional ultrasound space, can maintain automatic fine-tuning registration based on the segmentation shape of the interventional medical device 101 in three-dimensional ultrasound. The shape of the interventional medical device 101 in three-dimensional ultrasound is determined via image processing or deep learning techniques. Then, a strict point-to-point transformation from the corresponding portion of the interventional medical device 101 to the three-dimensional segmentation of the interventional medical device 101 in the ultrasound coordinate system is calculated. By automatically segmenting the interventional medical device 101 in the image and aligning it with the reconstruction obtained from the optical shape sensing device 102, accurate registration of the interventional medical device 101 can be maintained throughout the entire process.

[0055] In the second set of embodiments, the known shape of the interventional medical device 101 can be used to achieve and update the registration of the interventional medical device 101, and this can involve a simplified workflow compared to the first set of embodiments. For example, when the shape of the interventional medical device 101 is known, for example according to a template and / or according to the optical shape sensing device 102, the shape can be identified in the ultrasound coordinate space, and registration between the optical shape sensing and the ultrasound space can be performed without first registering the interventional medical device 101 to the X-ray imaging system 120. A template of the shape of the interventional medical device 101 can be obtained from a template library stored in a memory (e.g., memory 191). The template may include a template of a portion of the shape of the interventional medical device 101, for example, a template of the shape of the distal tip of the interventional medical device 101. When the template is a template of a portion of the shape, the remaining portion of the shape of the interventional medical device 101 can be identified by image analysis software searching in the area near the portion of the shape identified in the ultrasound image based on the template of that portion of the shape.

[0056] In addition, in the second set of embodiments, registration between the ultrasound coordinate space and the X-ray coordinate space can be performed without requiring an X-ray image of the probe head of the ultrasound imaging system 110. For example, the common shape of the interventional medical device 101 in both coordinate systems can be used as a mechanism for registering these two coordinate systems. When the interventional medical device 101 has been registered to the X-ray space and can be registered to the ultrasound space using a template of the shape of the interventional medical device 101, the ultrasound coordinate system can be registered to the X-ray coordinate system by calculating the transformation from the segmentation result of the interventional medical device 101 in the ultrasound space to the corresponding interventional medical device 101 in the X-ray space.

[0057] In the second set of embodiments, this portion of the shape can be used as a constraint for the initial search of the shape of the interventional medical device 101 in ultrasound images. Artificial intelligence can be applied to analyze the ultrasound images in ultrasound coordinate space. The search may initially be constrained by the tip of the interventional medical device 101, and once the tip of the interventional medical device 101 is identified in the search, artificial intelligence can be applied to find the remaining portion of the shape of the interventional medical device 101 based on characteristics and parameters identified from previous instantiations of the interventional medical device 101 in previous searches of ultrasound images.

[0058] In both the first and second sets of embodiments, a metric can be generated to demonstrate the correlation between the identification results of the interventional medical device 101 in different coordinate spaces. For example, the metric can be generated based on the correlation between the existing location of the segmented representation of the interventional medical device 101 in ultrasound space and the newly identified location of the interventional medical device 101 in ultrasound coordinates. This correlation can be an estimate of the confidence level regarding the accuracy of the identification and can be based, for example, the amount of difference between the segmented representation and the proposed new identified location of the interventional medical device 101 in ultrasound coordinates.

[0059] Figure 5 The illustration depicts a method for tracking interventional medical devices according to a representative embodiment.

[0060] exist Figure 5 In this method, the process begins at S510: identifying the shape of the optical shape sensing device. The optical shape sensing device can be... Figure 1 The optical shape sensing device 102 in the embodiments may include an optical fiber. The optical shape sensing technology described above can be used to identify the optical shape sensing device.

[0061] At S520, Figure 5 The method includes identifying the interventional medical device in a first coordinate space. The first coordinate space may be the coordinate space of a first imaging system operating in a first imaging mode, for example, the coordinate space of an X-ray imaging system 120 operating in an X-ray imaging mode. Identification at S520 may be by the user specifying the interventional medical device 101 in the X-ray image, or by image analysis software identifying the interventional medical device 101 in the X-ray image.

[0062] At S530, Figure 5 The method includes registering the interventional medical device 101 to a first coordinate space. Registration at S530 can be registration of the interventional medical device 101 to the X-ray space of the X-ray imaging system 120. Registration at S530 can be based on the shape of the interventional medical device 101 identified at S510 according to its shape consistent with the optical shape sensing device 102. Registration at S520 can also be based on the shape of the interventional medical device 101 identified at S520 based on the X-ray image.

[0063] At S540, Figure 5 The method includes identifying the interventional medical device 101 in a second coordinate space. The second coordinate space can be the coordinate space of a second imaging system operating in a second imaging mode, for example, the coordinate space of an ultrasound imaging system 110 operating in an ultrasound imaging mode. Identification at S540 can be either the user specifying the interventional medical device 101 in the ultrasound image, or image analysis software identifying the interventional medical device 101 in the ultrasound image. Although in Figure 5 Not shown, but S540 can be performed between S550 and S560 (described below) to register the first coordinate space to the second coordinate space before the registration of the interventional medical device 101 in the second coordinate space (i.e., in the ultrasound space).

[0064] In one embodiment, a user can identify the tip of the interventional medical device 101 in an ultrasound image, and the image analysis software can constrain the search for the remainder of the interventional medical device 101 to the area around the specified tip.

[0065] In another embodiment, the sensor on the tip of the interventional medical device 101 may be a passive ultrasound sensor that responds to emissions from the ultrasound imaging system 110. Sensor-based tracking of the interventional medical device 101 is described in U.S. Provisional Patent Application US 62 / 855013, filed May 31, 2019, with the U.S. Patent and Trademark Office, the disclosure of which is incorporated herein by reference in its entirety.

[0066] In embodiments using sensors on the tip of the interventional medical device 101, the image analysis software can constrain the search for the remainder of the interventional medical device 101 to the area around the tip identified based on signals from the sensors. This constraint can be based on the user's identification of the tip of the interventional medical device 101, on the identification of signals from a passive ultrasound sensor, or on the identification by image analysis software trained with artificial intelligence to recognize the tip of the interventional medical device.

[0067] In another embodiment, the X-ray can be transmitted from the interventional medical device 101 to the X-ray space via a T-band. OX Transformation and via T from ultrasound space to X-ray space UX A transformation is used to coarsely register the interventional medical device 101 to the ultrasound space. Then, by using the tip of the interventional medical device 101 as a coarse estimate to constrain the search space of the image processing algorithm, device segmentation based on the ultrasound image, as described below at S560, is continuously calculated throughout the acquisition process. The transformation T from the interventional medical device 101 to the ultrasound space can be calculated on each ultrasound frame. OU And throughout the process, the registration of the interventional medical device 101 to the ultrasound space is updated continuously or at fixed intervals.

[0068] At S550, the first coordinate space is registered to the second coordinate space. Registration at S550 can be performed by imaging the head of the ultrasound probe in the ultrasound imaging system 110 using the X-ray imaging system 120. As mentioned above, in some embodiments, S550 can be performed before S540.

[0069] At S560, the interventional medical device is segmented in the second coordinate space to generate a segmented representation of the interventional medical device. In one embodiment, the segmentation at S560 is initialized by the user identifying the interventional medical device 101 in the ultrasound image at S540. The image processing algorithm searches for the interventional medical device 101 in the region identified by the user in the image. The rigid transformation T from the interventional medical device 101 to the ultrasound space is calculated. OU This allows the distal portion of the interventional medical device 101, corresponding to the length of the segmentation result from the ultrasound device, to be rotated / translated to most closely match the segmentation representation in the ultrasound. Then, a rigid transformation T is performed. OU Applied to the entire length based on the reconstruction results from the optical shape sensing device 102.

[0070] At S565, for example at Figure 1 A segmented representation of the interventional medical device 101 is plotted on a monitor 195. The segmented representation of the interventional medical device 101 can be overlaid on an ultrasound image, which in turn can be overlaid on an X-ray image.

[0071] At S570, the interventional medical device is registered to the second coordinate space of the ultrasound imaging system 110. The registration at S570 can be an initial registration of the interventional medical device 101 to the ultrasound space, or it can be a re-registration of the interventional medical device 101 to correct for outdated earlier registrations.

[0072] At S580, the interventional medical device 101 is re-registered to the first coordinate space of the X-ray imaging system 120. The re-registration at S580 corrects outdated earlier registrations and eliminates the need for additional imaging by the X-ray imaging system 120.

[0073] In one embodiment, the interventional medical device 101 is segmented in the ultrasound volume based on prior identification of the device in the ultrasound image using artificial intelligence. In this embodiment, it is not necessarily necessary to constrain the search space based on user identification results or sensor initialization. The transformation T from the interventional medical device 101 to the ultrasound space can be calculated on each ultrasound frame. OU And throughout the process, the registration from the interventional medical device 101 to the ultrasound space is continuously updated.

[0074] Although Figure 5Not shown, but by calculating the transformation from the segmented representation of the interventional medical device 101 in the first coordinate space to the shape of the interventional medical device 101 based on optical shape sensing recognition in the second coordinate system, the first coordinate space can be re-registered to the second coordinate space. According to the second set of embodiments described herein, re-registration can be performed without requiring X-ray images of the probe head of the ultrasound imaging system 110. When the interventional medical device 101 has already been registered to the X-ray space and has already been registered to any ultrasound space based on a shape template, the ultrasound space can be registered to the X-ray space by calculating the transformation from the segmented representation of the interventional medical device 101 in the ultrasound space to the corresponding cross-section of the interventional medical device 101 in the X-ray space.

[0075] Figure 6 The illustration depicts a method for tracking interventional medical devices according to a representative embodiment.

[0076] exist Figure 6 In this method, the selection of the interventional medical device 101 is detected in the second coordinate space of the ultrasound imaging system 110. This selection may be the user's choice of the tip of the interventional medical device 101.

[0077] At S640, the interventional medical device 101 is identified in the second coordinate space. The identification at S640 is based on the selection at S635 and may involve using image analysis software to search for the remaining portion of the interventional medical device 101 in the area surrounding the user-selected location.

[0078] At S645, the transformation T from the current position of the interventional medical device 101 to the segmented representation is calculated. OU By using the new T OU The transformation can update the previous position of the interventional medical device 101 in the ultrasound space to a new position in the segmented representation.

[0079] At S648, the second coordinate space of the ultrasound imaging system 110 is registered to the first coordinate space of the X-ray imaging system based on the segmented representation of the interventional medical device 101 in the second coordinate space of the ultrasound imaging system 110. This registration updates the previous position of the interventional medical device 101 in the X-ray space to account for any errors, such as those caused by movement of the interventional medical device 101.

[0080] Figure 6 The embodiments may be for Figure 5 The embodiments are supplemented, and supplements are included regarding Figure 5 The function described.

[0081] Above Figure 6In the description of the embodiments, the identification of the interventional medical device 101 in the ultrasound space is based on the user's specification of the tip of the interventional medical device 101. In another embodiment, it is possible to search for the known shape of the interventional medical device 101 in the ultrasound space without needing to know the shape of the distal tip of the interventional medical device 101. The search for the known shape of the interventional medical device 101 can be based on information from the optical shape sensing signal from the optical shape sensing device 102, and eliminates the need for information obtained via transformation T. OU The need to roughly calibrate the interventional medical device 101 to the ultrasound space.

[0082] Figure 7 The illustration depicts a method for tracking interventional medical devices according to a representative embodiment.

[0083] exist Figure 7 In step S736, the interventional medical device 101 is identified in the second coordinate space of the ultrasound imaging system 110. Identification at S736 can be based on the user-selected position of the tip of the interventional medical device 101, or via signals from a passive ultrasound sensor at the tip of the interventional medical device 101.

[0084] At S760, interventional medical device 101 is segmented in the ultrasound image to generate a segmented representation of interventional medical device 101.

[0085] At S770, the interventional medical device 101 is registered to the second coordinate space of the ultrasound imaging system 110. Registration at S770 can be based at least on transform T. OU .

[0086] At S780, the interventional medical device is registered to the first coordinate space of the X-ray imaging system 120. The transformation at S780 can be based on all three transformations T. OX T UX and T OU .

[0087] Figure 7 The embodiments may be for Figure 5 The embodiments are supplemented, and supplements are included regarding Figure 5 The function described.

[0088] Above Figure 7In the description of the embodiments, the identification of the interventional medical device 101 in the ultrasound space is based on user or sensor identification of the tip of the interventional medical device 101. In another embodiment, the known shape of the interventional medical device 101 can be retrieved based on a template or information from optical shape sensing signals from the optical shape sensing device 102. Registration between the ultrasound space and the X-ray space can be defined or maintained using a template or knowledge of the shape of the interventional medical device 101, especially if the ultrasound probe of the ultrasound imaging system 110 is at an angle that is difficult to detect in the X-ray imaging system 120. Assuming the transformation T from the interventional medical device 101 to the ultrasound space... OU It is known and the transformation T from interventional medical device 101 to X-ray space OX If it is known, then the transformation T from ultrasound space to X-ray space is possible. UX It is roughly determined to be (T) UX =inv(T OU )*T OX As in other embodiments described in the text, transformation T is updated repeatedly. UX Implementations that maintain registration may not require additional X-ray exposure.

[0089] Figure 8 The illustration depicts a method for tracking interventional medical devices according to a representative embodiment.

[0090] exist Figure 8 In this method, the process begins at S836: identifying the interventional medical device 101 in a second coordinate space. Identification can be based on user-specified information or on signals from sensors at the tip of the interventional medical device 101. Identification at S836 can also be performed on demand, periodically and continuously during the interventional medical procedure, or based on the detection of movement of the interventional medical device 101 relative to a previous registration.

[0091] At S837, it is determined whether the deviation of the position of the interventional medical device from the existing segmentation representation exceeds a threshold. The determination at S837 can be based on the detection of movement of the interventional medical device 101 compared to the previous registration. If no deviation exceeds the threshold (S837 = No), then the method returns to S836; otherwise, it proceeds to S860.

[0092] At S860, the interventional medical device 101 is segmented in the ultrasound space.

[0093] At S870, the interventional medical device 101 is registered to the second coordinate space based on the segmentation at S860.

[0094] At S880, the second coordinate system is re-registered to the first coordinate system. This re-registration at S880 is provided without requiring re-exposure to X-rays from the X-ray imaging system 120.

[0095] After S880, the process returns to S836. Therefore, Figure 8 The process is recursive and may involve repeatedly checking whether the deviation of the position of the interventional medical device 101 from the last existing segmentation representation exceeds a threshold, and correcting such deviation by updating the registration of the interventional medical device 101 to the X-ray space and the ultrasound space.

[0096] Figure 8 The embodiments may be for Figure 5 The embodiments are supplemented, and supplements are included regarding Figure 5 The function described.

[0097] Above Figure 8 In the description of the embodiments, the identification of the interventional medical device 101 in the ultrasound space is again based on knowledge of the tip of the interventional medical device 101. In another embodiment, an irregularly shaped treatment device employing optical shape sensing can be delivered via a delivery sheath or catheter implemented via OSS. Known irregularly shaped treatment devices can be detected in the ultrasound space via image-based segmentation or via previously instantiated manual analysis applied to the identification of the OSS delivery sheath or catheter. The segmentation results of the irregularly shaped treatment device can be used to locate the distal end of the OSS-implemented delivery device in ultrasound to repeatedly update the registration between the interventional medical device 101 and the ultrasound space.

[0098] In additional embodiments, the registration method described above can be triggered automatically. For example, the interventional medical device 101 can be continuously segmented in ultrasound space during background processing. The existing registration of the interventional medical device 101 can be updated to match the interventional medical device 101 most closely in ultrasound space in each frame, every n frames, or only when a metric describing the correlation and / or offset between the location of the interventional medical device 101 and the location of the segmented representation of the interventional medical device 101 in ultrasound space exceeds a predetermined threshold.

[0099] Additionally, the above registration method can be triggered on demand. The user interface may include a metric describing the correlation and / or offset between the position of the interventional medical device 101 and its position in ultrasound space. Then, when the offset metric exceeds the expected error limit, or whenever the user wishes to update the current registration based on visual inspection, the user can select the "Update Registration" soft button.

[0100] Furthermore, after registration has been performed, the user interface can provide a metric describing the success of the registration. This metric may include information about the correlation between the shape of the interventional medical device 101 in re-registration and the shape of the interventional medical device 101 in its segmented representation in ultrasound space. Alternatively, the metric may include a confidence level that the correct shape of the interventional medical device 101 has been detected.

[0101] Figure 9 The illustration shows a computer system according to some representative embodiments, on which a method for tracking interventional medical devices is implemented.

[0102] Figure 9 The computer system 900 illustrates a complete set of components for a communication device or computer device. However, the "controller" as described herein can be used in fewer than [number missing]. Figure 9 This set of components is used for implementation. For example, it can be implemented through a combination of memory and processor. Computer system 900 may include some or all of the components of one or more of the components of the system for tracking interventional medical devices described herein, but any such device may not necessarily include one or more of the components described for computer system 900 and may include other components not described.

[0103] refer to Figure 9 The computer system 900 includes a set of software instructions that can be executed to cause the computer system 900 to perform any of the methods or computer-based functions disclosed herein. The computer system 900 can operate as a standalone device or, for example, be connected to other computer systems or peripheral devices using a network 901. In an embodiment, the computer system 900 performs logic processing based on digital signals received via an analog-to-digital converter.

[0104] In a networked deployment, computer system 900 operates as a server, or as a client computer in a server-client user network environment, or as a peer-to-peer (or distributed) computer system in a peer-to-peer (or distributed) network environment. Computer system 900 can also be implemented as various devices or incorporated into various devices, for example, Figure 1The controller 190, fixed computer, mobile computer, personal computer (PC), laptop computer, tablet computer, or other machine capable of (sequentially or otherwise) executing a set of software instructions specifying actions to be performed by that machine. The computer system 900 can be incorporated as a device or incorporated into a device, which in turn is included in an integrated system including additional devices. In embodiments, the computer system 900 can be implemented using electronic devices that provide voice, video, or data communications. Furthermore, although the computer system 900 is illustrated as a single system, the term "system" should also be considered to include any collection of systems or subsystems that individually or jointly execute one or more sets of software instructions to perform one or more computer functions.

[0105] like Figure 9 As shown, the computer system 900 includes a processor 910. The processor 910 can be considered as... Figure 1 The processor 192 of the controller 190 executes instructions to implement some or all aspects of the methods and processes described herein. The processor 910 is tangible and non-transient. As used herein, the term "non-transient" should not be interpreted as a perpetual state characteristic, but rather as a characteristic of a state that will persist for a period of time. The term "non-transient" specifically denies transient characteristics, such as carrier waves or signals or other forms of characteristics that exist only momentarily at any time and place. The processor 910 is an article of manufacture and / or a machine part. The processor 910 is configured to execute software instructions to perform the functions described in the various embodiments herein. The processor 910 may be a general-purpose processor or a part of an application-specific integrated circuit (ASIC). The processor 910 may also be a microprocessor, microcomputer, processor chip, controller, microcontroller, digital signal processor (DSP), state machine, or programmable logic device. The processor 910 may also be a logic circuit (including a programmable gate array (PGA) such as a field-programmable gate array (FPGA)) or another type of circuit comprising discrete gate and / or transistor logic units. The processor 910 may be a central processing unit (CPU), a graphics processing unit (GPU), or both. Additionally, any processor described herein may include multiple processors, parallel processors, or both. Multiple processors may be included in or coupled to a single device or multiple devices.

[0106] As used herein, the term "processor" encompasses any electronic component capable of running programs or machine-executable instructions. References to computing devices that include "processor" should be interpreted as including more than one processor or processing core, as is the case in a multi-core processor. A processor can also refer to a collection of processors within a single computer system or distributed across multiple computer systems. The term computing device should also be interpreted as including a collection or network of computing devices, each including one or more processors. A program has software instructions that are executed by one or more processors, which may be located within the same computing device or distributed across multiple computing devices.

[0107] The computer system 900 also includes main memory 920 and static memory 930, wherein the memories in the computer system 900 communicate with each other and with the processor 910 via bus 908. One or both of the main memory 920 and static memory 930 can be considered as... Figure 1 A representative example of the memory 191 of the controller 190 is shown, which stores instructions for implementing some or all aspects of the methods and processes described herein. The memory described herein is a tangible storage medium for storing data and executable software instructions, and is non-transient during the time the software instructions are stored therein. As used herein, the term "non-transient" should not be construed as a perpetual state characteristic, but rather as a characteristic of a state that will persist for a period of time. The term "non-transient" specifically denies transient characteristics, such as carrier waves or signals or other forms of characteristics that exist only momentarily at any time and place. Main memory 920 and static memory 930 are articles of manufacture and / or machine parts. Main memory 920 and static memory 930 are computer-readable media from which a computer (e.g., processor 910) can read data and executable software instructions. Each of the main memory 920 and static memory 930 can be implemented as one or more of the following: random access memory (RAM), read-only memory (ROM), flash memory, electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, removable disk, magnetic tape, compact disc read-only memory (CD-ROM), digital versatile disk (DVD), floppy disk, Blu-ray disc, or any other form of storage medium known in the art. The memory can be volatile or non-volatile, secure and / or encrypted, insecure and / or unencrypted.

[0108] “Memory” is an example of a computer-readable storage medium. Computer memory is any memory that a processor can directly access. Examples of computer memory include, but are not limited to, RAM, registers, and register files. The reference to “computer memory” or “memory” should be interpreted as potentially referring to multiple memories. Memory can be, for example, multiple memories within the same computer system. Memory can also be multiple memories distributed across multiple computer systems or computing devices.

[0109] As shown in the figure, the computer system 900 may also include, for example, a video display unit 950 (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED), a flat panel display, a solid-state display, or a cathode ray tube (CRT)). Additionally, the computer system 900 includes an input device 960 (e.g., a keyboard / virtual keyboard, a touch-sensitive input screen, or a voice input unit with voice recognition) and a cursor control device 970 (e.g., a mouse, a touch-sensitive input screen, or a pad). The computer system 900 may also optionally include a disk drive unit 980, a signal generation device 990 (e.g., a speaker or a remote control), and a network interface device 940.

[0110] In an embodiment, such as Figure 9 As shown, the disk drive unit 980 includes a computer-readable medium 982 in which one or more sets of software instructions 984 (software) are embedded. The set of software instructions 984, to be executed by the processor 910, is read from the computer-readable medium 982. Additionally, the software instructions 984, when executed by the processor 910, perform one or more steps of the methods and processes described herein. In embodiments, the software instructions 984 reside wholly or partially in main memory 920, static memory 930, and / or reside wholly or partially in the processor 910 during operation by the computer system 900. Furthermore, the computer-readable medium 982 may include the software instructions 984 or receive and execute the software instructions 984 in response to a propagated signal, causing a device connected to the network 901 to transmit voice, video, or data on the network 901. The software instructions 984 may be sent or received on the network 901 via a network interface device 940.

[0111] In embodiments, dedicated hardware implementations (e.g., application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic arrays, and other hardware components) are constructed to implement one or more of the methods described herein. One or more embodiments described herein may use two or more specific interconnect hardware modules or devices to implement functionality, having associated control and data signals capable of communicating between and through these modules. Therefore, this disclosure covers software, firmware, and hardware implementations. Nothing in this application should be construed as being implemented solely using software and not using hardware such as tangible non-transient processors and / or memory.

[0112] According to various embodiments of this disclosure, the methods described herein can be implemented using a hardware computer system running software programs. Additionally, in exemplary non-limiting embodiments, implementations can include distributed processing, component / object distributed processing, and parallel processing. Virtual computer system processing can implement one or more of the methods or functions described herein, and the processors described herein can be used to support virtual processing environments.

[0113] Therefore, interventional medical device tracking enables updated registration to correct the position of interventional medical device 101. Nevertheless, interventional medical device tracking is not limited to the specific details described herein, but can be applied to additional embodiments using other types of medical imaging systems and interventional medical devices.

[0114] While interventional medical device tracking has been described with reference to several exemplary embodiments, it should be understood that the terms used are descriptive and illustrative, not limiting. Changes may be made within the scope and spirit of the claims, as described herein and with modifications, without departing from the scope and spirit of interventional medical device tracking in its various aspects. Although interventional medical device tracking has been described with reference to specific means, materials, and embodiments, interventional medical device tracking is not intended to be limited to the disclosed details; rather, interventional medical device tracking extends to all functionally equivalent structures, methods, and uses as described within the scope of the claims.

[0115] The embodiments described herein are intended to provide a general understanding of the structure of various embodiments. These descriptions are not intended to constitute a complete description of all elements and features of the disclosure described herein. Many other embodiments will be apparent to those skilled in the art upon review of this disclosure. Other embodiments can be utilized and derived from this disclosure, allowing structural and logical substitutions and changes to be made without departing from the scope of this disclosure. Furthermore, these illustrations are representative only and may not be drawn to scale. Some scales in the illustrations may be enlarged, while others may be minimized. Therefore, this disclosure and the accompanying drawings should be considered illustrative rather than restrictive.

[0116] In this document, for convenience only, the term "invention" may be used individually and / or collectively to refer to one or more embodiments of this disclosure, without intending to limit the scope of this application to any particular invention or inventive concept. Furthermore, while specific embodiments have been illustrated and described herein, it should be understood that any subsequent arrangements designed to perform the same or similar purposes with respect to the specific embodiments shown may be substituted. This disclosure is intended to cover any and all subsequent modifications or variations of the various embodiments. After reading the specification, combinations of the above embodiments, as well as other embodiments not specifically described herein, will be apparent to those skilled in the art.

[0117] This abstract of disclosure is provided in accordance with 37 C. FR § 1.72(b) and is to be understood at the time of filing not to be used for interpreting or limiting the scope or meaning of the claims. Additionally, in the foregoing detailed description, various features may be grouped together or described in a single embodiment for the purpose of simplification. This disclosure should not be construed as reflecting an intention that the claimed embodiments require more features than expressly recited in each claim. Rather, as reflected in the claims, the inventive subject matter may refer to all features of fewer than any of the disclosed embodiments. Therefore, the claims are incorporated into the detailed description, with each claim independently defining a separately claimed subject matter.

[0118] The foregoing description of the disclosed embodiments is provided to enable any person skilled in the art to practice the concepts described in this disclosure. Therefore, the subject matter disclosed above should be considered illustrative rather than restrictive, and the claims are intended to cover all such modifications, enhancements, and other embodiments falling within the true spirit and scope of this disclosure. Accordingly, to the fullest extent permitted by law, the scope of this disclosure will be determined by the broadest permissible interpretation of the claims and their equivalents, and should not be limited by the foregoing detailed description.

Claims

1. A system for tracking the location of an interventional medical device (101) during an interventional medical procedure, comprising: An interface (193) to an optical shape sensing device (102), the shape of which remains consistent with the shape of the interventional medical device (101) during the interventional medical procedure; as well as A controller (190) includes a memory (191) for storing instructions and a processor (192) for executing the instructions, wherein the instructions, when executed by the processor (192), cause the system to: The shape of the optical shape sensing device (102) is identified using the optical shape sensing signal received from the optical shape sensing device (102) via the interface (193); The shape of the interventional medical device (101) is identified in a first coordinate space of a first imaging system, the first imaging system imaging the interventional medical device (101) in a first imaging mode during the interventional medical procedure; The interventional medical device (101) is segmented in the first coordinate space to obtain a segmented representation (401) of the interventional medical device (101) in the first coordinate space. Using the segmentation representation, the interventional medical device (101) is registered to the first coordinate space based on the shape of the interventional medical device (101) identified using the optical shape sensing signal and based on the shape of the interventional medical device (101) identified in the first coordinate space; The interventional medical device (101) is registered to the second coordinate space of a second imaging system, which images the interventional medical device (101) in a second imaging mode during the interventional medical procedure; and Register the first coordinate space of the first imaging system to the second coordinate space of the second imaging system.

2. The system according to claim 1, further comprising: Optical shape sensing device (102); An ultrasound imaging system (110) comprising the first imaging system; as well as X-ray imaging system (120), which includes the second imaging system, and The instructions also cause the system to: The segmented representation (401) of the interventional medical device (101) is plotted in the first coordinate space; and The shape of the interventional medical device (101) is identified in the first coordinate space based on the segmentation representation (401) of the interventional medical device (101) in the first coordinate space.

3. The system according to claim 1, wherein, The shape of the interventional medical device (101) in the first coordinate space is identified based on a template of the shape of the interventional medical device (101).

4. The system according to claim 3, wherein, The template of the shape of the interventional medical device (101) includes a template of a portion of the shape of the interventional medical device (101), and the remaining portion of the shape of the interventional medical device (101) is identified by searching for the remaining portion of the shape of the interventional medical device (101) in an area close to the portion of the shape of the interventional medical device (101) identified according to the template.

5. The system according to claim 4, wherein, The instructions also cause the system to: Calculate the transformation from the interventional medical device (101) identified in the first coordinate space using the optical shape sensing signal to the interventional medical device (101) identified in the first coordinate space based on the shape of the interventional medical device (101) identified in the first coordinate space according to the template.

6. The system according to claim 1, wherein, The shape of the interventional medical device (101) is identified in the first coordinate space using artificial intelligence applied to analyze images in the first coordinate space.

7. The system according to claim 1, wherein, The instructions also cause the system to: The shape of the interventional medical device (101) is repeatedly identified in the image in the first coordinate space of the system using a template of the tip of the interventional medical device (101) as a constraint in the image search. and The interventional medical device (101) is repeatedly registered to the first coordinate space based on the shape of the interventional medical device (101) identified in the image.

8. The system according to claim 1, wherein, The instructions also cause the system to: The interventional medical device (101) is segmented in the first coordinate space to obtain a segmented representation (401) of the interventional medical device (101) in the first coordinate space. A metric is generated based on the correlation between the position of the segmented representation (401) of the interventional medical device (101) and the position of the interventional medical device (101) in the first coordinate space, and When the metric exceeds a predetermined threshold, the interventional medical device (101) is re-registered to the first coordinate space.

9. A tangible, non-transient, computer-readable storage medium for storing a computer program, said computer program causing a system comprising said tangible, non-transient, computer-readable storage medium to function when executed by a processor (192): The shape of the optical shape sensing device (102) is identified using optical shape sensing signals received via interface (193), the shape of which remains consistent with the shape of the interventional medical device (101) during the interventional medical procedure; The shape of the interventional medical device (101) is identified in a first coordinate space of a first imaging system, the first imaging system imaging the interventional medical device (101) in a first imaging mode during the interventional medical procedure; The interventional medical device (101) is segmented in the first coordinate space to obtain a segmented representation (401) of the interventional medical device (101) in the first coordinate space. The interventional medical device (101) is registered to the first coordinate space based on the shape of the interventional medical device (101) identified using the optical shape sensing signal and based on the shape of the interventional medical device (101) identified in the first coordinate space. The interventional medical device (101) is registered to the second coordinate space of a second imaging system, which images the interventional medical device (101) in a second imaging mode during the interventional medical procedure; and Register the first coordinate space of the first imaging system to the second coordinate space of the second imaging system.

10. The non-transient computer-readable storage medium according to claim 9, wherein, The computer program causes the system to also: The first coordinate space is re-registered to the second coordinate space by calculating the transformation from the segmented representation (401) of the device in the first coordinate system to the shape of the optical shape sensing device (102) identified using optical shape sensing in the second coordinate system.

11. The non-transient computer-readable storage medium according to claim 9, wherein, The computer program causes the system to also: A template for obtaining the shape of the interventional medical device (101) is obtained; and The shape of the template is searched in the image in the first coordinate space.

12. The non-transient computer-readable storage medium according to claim 9, wherein, The computer program causes the system to also: The first coordinate space is periodically divided to obtain a segmented representation (401). The registration of the interventional medical device (101) to the first coordinate space using the shape of the interventional medical device (101) identified in the first coordinate space is periodically repeated based on the segmentation.

13. The non-transient computer-readable storage medium according to claim 12, wherein, The computer program causes the system to also: Calculate the correlation between the segmentation representation (401) in the first coordinate space and the interventional medical device (101), and The registration is automatically repeated based on the correlation.

Images