Assists in moving inserted components within an object

Abstract

The present invention relates to assisting a user in moving an insertion element within an object. The three-dimensional (3D) position and 3D local shape of an insertion element are tracked within the object. Based on the tracked 3D position and 3D local shape of the insertion element, an insertion element representation (42'") is generated that represents the insertion element within the object with its 3D position and 3D local shape. In addition, at least two different markings (78) representing indications of the local shape of the insertion element applied along the length of the insertion element representation (42'") are provided. A display image (60") including the insertion element representation (42'") with the different markings (78) is displayed. The markings may, for example, include alternating ring structures (78) having alternating lighter and darker colors and / or different colors for the tip and the rest of the insertion element representation to improve the 3D visualization of the insertion element.

Description

Technical Field

[0001] The invention relates to an assisting device for assisting a user in moving an insertion element within an object, a method for assisting a user in moving an insertion element within an object, and a computer program product for assisting a user in moving an insertion element within an object. The invention also relates to an interventional system for moving an insertion element within an object, comprising the assisting device. Background Art

[0002] During the catheterization procedure, it is important to know the position of the catheter within the internal structure of the person through which the catheter should be navigated. For this reason, a real-time X-ray projection image is usually generated during the catheterization procedure, wherein the physician moves the catheter within the internal structure based on the generated X-ray projection image. However, since the X-ray projection image is only a two-dimensional (2D) image, it is difficult for the physician to accurately grasp the three-dimensional (3D) position of the catheter within the internal structure, and therefore it is difficult to accurately navigate the catheter tip within the internal structure.

[0003] WO2015 / 177012A2 discloses an imaging device for imaging a first object, such as the tip of a catheter within a second object, the second object being, for example, a vascular structure of a person. A three-dimensional representation of the second object including a representation of a surface of the second object and a position of the first object relative to the position of the second object is provided, and a projection of the first object onto the representation of the surface of the second object is determined and shown to a user, such as a physician, on a display. Summary of the invention

[0004] An object of the present invention is to provide an assisting device for assisting a user in moving an insertion element within an object, a method for assisting a user in moving an insertion element within an object, and a computer program product for assisting a user in moving an insertion element within an object, which allow the user to move the insertion element within the object more accurately and faster. A further object of the present invention is to provide an interventional system for moving an insertion element within an object more accurately and faster, comprising the assisting device.

[0005] In a first aspect of the present invention, an assistive device for assisting a user in moving an insertion element within an object is presented. The assistive device comprises a tracking unit, an insertion element representation generating unit and a display. The tracking unit is configured to track a 3D position and a 3D local shape of the insertion element within the object. The insertion element representation generating unit is configured to generate an insertion element representation representing the insertion element within the object with its 3D position and 3D local shape based on the tracked 3D position and the tracked 3D local shape of the insertion element. The insertion element representation generating unit is further configured to provide at least two different markings indicating the local shape of the insertion element applied along the length of the insertion element. The display is configured to display a display image including the insertion element representation, the insertion element representation having the at least two different markings applied along the length of the insertion element representation.

[0006] Since the insertion element representation generation unit is configured to provide at least two different markings representing indications of the local shape of the insertion element applied along the length of the insertion element representation, a visualization can be provided, which allows a user to easily and accurately grasp the 3D shape of the insertion element and thus can allow more accurate and faster movement of the insertion element within the object. In particular, this allows improved visualization compared to shading effects that are not very noticeable for insertion elements that typically have a longitudinal shape, such as catheters and guidewires.

[0007] The markings may include different colors, different opacities, different brightness, different shades, or any other type of marking.

[0008] The marking may be a ring-like structure with a deformed shape. For example, the ring-like structure may be applied along the length of the insertion element representation in the form of alternating lighter and darker colors, lower and higher brightness, lower and higher opacity, etc. The deformed shape of the ring-like structure depends on the local shape of the insertion element. This allows for improved visualization of the local shape of the insertion element.

[0009] The insertion element representation generation unit may be configured to provide the at least two markers as an annular structure having a deformed shape. The deformed shape may be such that for a bend in the local shape of the insertion element, the annular structure appears to be closer on the inner side of the bend and further apart on the outer side of the bend, so as to emphasize the three-dimensional effect of the shape of the annular structure. Alternatively or additionally, the insertion element representation generation unit may be configured to provide an annular structure having a distortion effect for the distortion of the local shape of the insertion element. Providing the markers as an annular structure having a deformed shape allows for improved 3D visualization of the insertion element within the object.

[0010] The auxiliary device may include an object image providing unit for providing an object image showing a view within the object. The tracking unit may be adapted to track the orientation of the insertion element. The insertion element representation generating unit may be configured to generate the insertion element representation with a 3D position, orientation and size based on the object image. The display may be configured to display the display image including the insertion element representation with the at least two different markers applied along the length of the insertion element representation arranged within the object image.

[0011] The object image providing unit may include an image acquisition unit for acquiring an object image, or may be connected to an image acquisition unit for receiving an object image acquired by the image acquisition unit. The image acquisition unit may include a computed tomography (CT) system, a magnetic resonance imaging (MRI) system, an ultrasound (US) system, a nuclear imaging system (such as a positron emission tomography (PET) system or a single photon emission tomography imaging system), or any other image acquisition unit for acquiring an object image. The object image may be a 3D image, a 2D projection image, or any other type of image. The object image may be a CT image, a magnetic resonance (MR) image, a PET image, a US image, or any other type of image acquired using another type of image acquisition unit. The object image providing unit may include a storage unit for storing the object image, such as a computer readable medium in the form of a memory. The object image providing unit may also include a receiving unit for receiving the object image and for providing the received object image. The object image providing unit may also be adapted to provide a 3D model of a view within the object as an object representation based on the object image. For example, at least a surface within the object may be segmented in the object image so as to determine an object surface representation of the surface within the object. The object image providing unit may be adapted to provide a polygonal mesh representing the view within the object as the object representation.Furthermore, the object image providing unit may be adapted to generate the object representation.

[0012] The tracking unit may be adapted to track and provide an orientation of the tip of the insertion element relative to a target element within the object, e.g. a point to which the tip is to be moved. The target element may be selected by a user or identified automatically. For example, the position of the tip may be defined as the distal-most position of a center point at the tip of the insertion element, wherein the tip may be slightly elongated, i.e. extending along a longitudinal axis, and wherein the orientation of the tip may be defined as the orientation of the longitudinal axis of the tip. It is also considered that the orientation of the tip of the insertion element relative to the target element within the object also improves the 3D impression that may be provided to a user moving the insertion element within the object.

[0013] The tracking unit and the object image providing unit can be registered relative to each other. The registration can be performed in several ways, such as 2D-3D registration. The tracking unit can be registered with the target image providing unit by identifying the position of the insertion element in the target image and also by determining the position by the tracking unit. The insertion element representation generation unit can be configured to generate an insertion element representation with 3D position, orientation and size based on the object image. Alternatively, the object image providing unit can be configured to provide an object image showing a view within the object depending on the tracking orientation of the insertion element. This allows the shape and orientation of the insertion element within the visualization object.

[0014] The tracking unit may be adapted to track the position of the insertion element or, for example, the tip of the insertion element relative to the position of a target element within the object, such as a point to which the tip is to be moved. The tracking unit may be adapted to provide the position of the tip of the insertion element relative to the position of the target element by using optical shape sensing and / or by using an electromagnetic sensor and / or by using X-rays. For example, the tracking unit may be adapted to acquire X-ray projection images of the insertion element within the object in at least two different acquisition directions, wherein the tip of the insertion element can be identified in the X-ray projection images, and the position of the tip of the insertion element relative to the target element within the object may be determined based on the position of the identified tip of the insertion element within the X-ray projection images taking into account the respective acquisition directions. The determination of the position of the tip of the insertion element may be registered with the object image in order to provide the position of the tip of the insertion element relative to the position of the target element within the object. For example, the X-ray projection image that may be used to determine the position of the tip of the insertion element may be registered with an object image in which the target element is shown within the object in order to register the determined position of the insertion element with the object image in which the position of the target element within the object is provided. Alternatively, if optical shape sensing and / or electromagnetic sensing are used to determine the position of the tip of the insertion element relative to a target element of the object, the optical shape sensing system and / or the electromagnetic sensing system can be aligned with an object image in which the target element is shown within the object in order to determine the position of the tip of the insertion element relative to the position of the target element within the object.

[0015] The tracking unit may be adapted to calculate a distance between the provided position of the tip of the insertion element and the target element within the object. The calculated distance is preferably a corresponding minimum distance.

[0016] The object image providing unit may be adapted to generate an object image after the insertion element has been introduced into the object, and to provide the generated object image. The object image may be a live image showing target elements within the object and the tip of the insertion element within the object. The object image may be, for example, an X-ray projection image generated by an X-ray projection system such as an X-ray C-arm system. This allows increasing the amount of 3D information that a user can grasp from the resulting features shown on the display, which may allow the user to further improve the accuracy of moving the insertion element within the object.

[0017] The display can be configured to display orientation information of the insertion element relative to the object based on the view within the object, the tracked 3D position of the insertion element within the object, and the tracked orientation. The orientation information can include a thumbnail of the object at the corresponding orientation, a thumbnail of the patient including the object at the corresponding orientation, and / or an angle depending on a reference coordinate system. This allows for improved visualization of the current arrangement of the insertion element within the object. The orientation information can also, for example, indicate the acquisition direction of the view within the object, so as to allow a user to better grasp the orientation of the insertion element relative to the object.

[0018] The object image providing unit may be configured to provide at least two object images with different views within the object. The display may be configured to display at least two display images. The display may be configured to display two or more display images simultaneously, for example, side by side, overlapping each other, one for each eye of a user or in any other manner. Each of the display images may include the insertion element representation having the at least two different markings applied along the length of the insertion element representation arranged within one of the object images. The different views within the object may, for example, include different 2D projection images, different slices of a 3D image, or different viewing angles of a 3D image. This allows different viewing angles of the insertion element within the object to be provided simultaneously to improve the visualization of the current arrangement of the insertion element within the object.

[0019] One of the at least two different markings may be applied at the tip of the insertion element representation. The marking applied at the tip may be different from the marking applied along the rest of the length represented by the insertion element so as to allow the user to identify the orientation of the tip of the insertion element representation. The marking at the tip may be, for example, a color, so that the color at the tip is different from the color applied along the rest of the length represented by the insertion element. The color at the tip may be, for example, white or black or any color that contrasts with the color of the background of the insertion element representation, so that the tip can be clearly identified. This allows improved identification of the orientation of the tip. The application of a different marking at the tip that is different from the marking applied along the rest of the length represented by the insertion element may be combined with other markings, such as applying an alternating ring-like structure along the rest of the length of the insertion element.

[0020] The insertion element may include an optical fiber, a catheter and / or a guidewire, i.e. the auxiliary device may include an optical fiber, a catheter, a guidewire or any combination of an optical fiber, a catheter and a guidewire. The auxiliary device may also include more than one optical fiber, a catheter and a guidewire or any combination thereof.

[0021] The tracking unit may be configured to track the 3D position and 3D local shape of the insertion element by using optical shape sensing. The tracking unit may also be configured to track the 3D position, 3D local shape and orientation of the insertion element by using optical shape sensing. Optical shape sensing may be based on optical frequency domain reflectometry (OFDR). OFDR may be used to extract high-density strain measurements from multiple cores in an insertion element (e.g., an optical fiber included in an insertion element). Strain measurements may be based on light reflected from a continuously written fiber Bragg grating (FBG) or from the inherent Rayleigh scattering features in an optical fiber. The optical fiber may, for example, include a central core and three outer cores, and local shape and position sensing may be performed by measuring axial distortion and curvature along the length of the optical fiber. Alternatively or additionally, the tracking unit may be configured to track the 3D position, 3D local shape and / or orientation of the insertion element by any other tracking method. Other tracking methods may include tracking methods and tracking systems as described in US7772541B2 and US8116601B2.

[0022] The tracking unit may be configured to provide parameter values ​​for different points along the length of the insertion element based on the tracked 3D position and the 3D local shape of the insertion element. The insertion element representation generation unit may be adapted to calculate visualization values ​​for points along the length of the insertion element that depend on the parameter values ​​provided by the tracking unit, and to provide the at least two different markings applied along the length of the insertion element representation based on the visualization values. The tracking unit may be configured to calculate the 3D position for different points along the length of the insertion element based on the x-position, y-position and z-position of the point. The parameter value may include at least one of the following: distortion, strain and curvature values, i.e. the parameter value may include values ​​of distortion, strain and / or curvature or other parameters.

[0023] The tracking unit may be configured to track the physical property values ​​of the insertion element, for example, including the temperature value, pressure value and / or any other kind of physical property value of the insertion element. The insertion element representation generation unit may be configured to provide at least two different superimposed marks along the length of the insertion element representation, the at least two different superimposed marks representing the indication of the physical property value of the insertion element. The superimposed marks may be suitable for being superimposed on the at least two different marks without covering them. The display may be configured to display the display image including the insertion element representation, the insertion element representation having the at least two different marks and the at least two different superimposed marks applied along the length of the insertion element representation. In order for the superimposed marks not to cover the marks, the superimposed marks may be selected according to the marks. For example, if the marks are colors, the superimposed marks may be brightness or shading. If the marks are brightness, the superimposed marks may be, for example, colors. When the marks are ring-shaped structures having alternating lighter and darker colors, higher and lower opacity, higher and lower brightness, etc., the superimposed marks may be, for example, different colors. For example, if the markings along the length of the insertion element representation are ring-like structures with alternating higher and lower brightness, the superimposed markings for the temperature values ​​can be a color map from red to green for high to low temperatures. This allows the physical property values ​​of the insertion element to be provided as well as the local shape of the insertion element for improved 3D visualization while providing the user with additional information of the insertion element.

[0024] In addition, the auxiliary device may include a target element providing unit for providing a target representation of a target element within the object to which the insertion element or the tip of the insertion element should be moved. The display may be adapted to display the display image including the insertion element representation, the insertion element representation having the at least two different marks applied along the length of the insertion element representation and the target representation of the target element. The display image may also include a target representation and an insertion element representation having at least two different marks applied along the length of the insertion element representation arranged within the object image. The target element providing unit may be adapted to automatically select a target element based on a user's command, for example, by providing a description of the target element by naming the target element. Alternatively or additionally, the target element providing unit may be adapted to select a target element based on a manual selection of the user, for example, by marking the target element within the object image, for example, by clicking on the target element, drawing a frame around it, etc. Also showing on the display a representation of the target element to which the insertion element should be moved within the object can improve the accuracy of moving the insertion element within the object. The target element providing unit may be further configured to calculate a path from the tip to the target element that avoids other substances, and to provide a target path representation indicating an indication of a path from the tip to the target element that avoids other substances. The display can be configured to also display the target path. This allows for a continuous and guided interaction with the assisted user, as changes in the position of the substance within the object result in a recalculation of the target path, and the user can follow the target path to move the tip of the insertion element to the target element. The target path can also be followed automatically by the insertion element, for example controlled by a robotic device.

[0025] The invention per se provides an assistive device for assisting a user in moving an insertion element within an object, wherein the assistive device comprises: a tracking unit for tracking the 3D position and orientation of the insertion element within the object; an insertion element representation generation unit for generating an insertion element representation representing the insertion element within the object with its 3D position and orientation based on the tracked 3D position and orientation of the insertion element, wherein the insertion element representation generation unit is configured to provide the insertion element representation with at least two different markings, wherein the tip of the insertion element representation is provided with a marking different from any other marking of the insertion element representation; and a display for displaying a display image comprising the insertion element representation with the tip and its marking. The assistive device may be combined with any aspect presented in the specification and the dependent claims. In particular, the insertion element representation with the tip and its marking may be displayed within an object image.

[0026] In yet another aspect of the present invention, an interventional system for moving an insertion element within an object is presented. The interventional system comprises an insertion device comprising the insertion element and an assistive device according to claim 1 or any embodiment of the assistive device for assisting a user in moving the insertion element within the object.

[0027] The insertion device may include a handpiece connected to the insertion element for controlling the insertion element. The insertion element may include one or more optical fibers, guidewires, catheters and / or sensors. The handpiece may include a user interface for receiving commands from a user. The handpiece may additionally or alternatively include a data processing unit for processing data, for example, sensor data, such as temperature or pressure data. The sensor data may, for example, be received from a sensor included in the insertion element. The handpiece may be connected to an auxiliary device for exchanging data.

[0028] The insertion device can be configured to move an insertion element, such as a catheter, in an object (such as a heart or vascular structure of a human or animal). The insertion element can include a tip at its distal end, such as the tip of a catheter or a needle. The insertion device can be adapted to allow a user (such as a physician) to manually move the insertion element in the object. For example, the insertion device can include a guide wire, a steering line, etc., to allow the user to move the insertion element in the object. The insertion device can also include a robot module, which can allow the user to move the insertion element in the object by using a handpiece (such as a joystick or another input unit of a user interface) for inputting a steering command into the robot system. The insertion element can also be automatically moved by the robot module based on the calculated path of the tip to the target element that avoids other substances.

[0029] In another aspect of the present invention, a method for assisting a user in moving an insertion element within an object is presented. The method comprises:

[0030] - tracking the 3D position and the 3D local shape of the insertion element within the object by a tracking unit,

[0031] - generating, by an insertion element representation generating unit, an insertion element representation representing the insertion element within the object with its 3D position and 3D local shape based on the tracked 3D position and the tracked 3D local shape of the insertion element,

[0032] - providing, by the insertion element representation generating unit, at least two different markings applied along the length of the insertion element representation representing an indication of the local shape of the insertion element, and

[0033] - displaying, by a display, a display image comprising said insertion element representation, said insertion element representation having said at least two different markings applied along a length of said insertion element representation.

[0034] The markings may include different colors, different opacities, different brightness, different shades, or any other type of marking.

[0035] The marking may be a ring-like structure with a deformed shape. For example, the ring-like structure may be applied along the length of the insertion element representation in the form of alternating lighter and darker colors, lower and higher brightness, lower and higher opacity, etc. The deformed shape of the ring-like structure depends on the local shape of the insertion element. This allows for improved visualization of the local shape of the insertion element.

[0036] The method may include the step of tracking the orientation of an insertion element within the object. The method may include the step of providing, by an object image providing unit, an object image showing a view within the object. The method may include the step of generating, by an insertion element representation generating unit, an insertion element representation representing the insertion element within the object based on the tracked 3D position, tracked 3D local shape and tracked orientation of the insertion element with its 3D position, 3D local shape, its orientation and size and based on the object image with its size. The method may include the step of displaying, by a display, a display image comprising an insertion element representation within the object image, the insertion element representation having at least two different markers applied along the length of the insertion element representation. This allows for providing improved 3D visualization.

[0037] One of the at least two different markings may be applied at the tip of the insertion element representation. The marking applied at the tip may be different from the marking applied along the rest of the length represented by the insertion element, so as to allow the user to identify the orientation of the tip of the insertion element. The marking at the tip may, for example, be a color, so that the color at the tip is different from the color applied along the rest of the length represented by the insertion element. The color at the tip may, for example, be white or black or any color that contrasts with the color of the background of the insertion element representation, so that the tip can be clearly identified. This allows improved identification of the orientation of the tip. The application of a different marking at the tip that is different from the marking applied along the rest of the length represented by the insertion element may be combined with other markings, such as applying an alternating ring structure along the rest of the length of the insertion element.

[0038] The present invention itself provides a method for assisting a user in moving an insertion element within an object, wherein the method comprises:

[0039] - tracking the 3D position and orientation of the insertion element within the object by a tracking unit;

[0040] - generating, by an insertion element representation generating unit, an insertion element representation representing the insertion element within the object with its 3D position and orientation based on the tracked 3D position and orientation of the insertion element,

[0041] - providing said insertion element representation with at least two different markings, wherein a tip of said insertion element representation is provided with a marking different from any other marking of said insertion element representation, and

[0042] - displaying by a display a display image comprising the representation of the insertion element with the tip and its marking. The method may be combined with any aspect presented in the description and the dependent claims. In particular, the representation of the insertion element with the tip and its marking may be displayed within the object image.

[0043] In yet another aspect of the present invention, a computer program product for assisting a user in moving an insertion element within an object is presented. The computer program product comprises program code means for causing the assisting device according to claim 1 or any embodiment of the assisting device to perform the steps of the method according to claim 13 or any embodiment of the method when the computer program product is run on the assisting device according to claim 1 or any embodiment of the assisting device.

[0044] In yet another aspect of the present invention, a computer readable medium storing a computer program product according to claim 14 or any embodiment of the computer program product is presented.

[0045] It should be understood that the auxiliary device according to claim 1, the interventional system according to claim 12, the method according to claim 13, the computer program product according to claim 14 and the computer-readable medium according to claim 15 have similar and / or identical preferred embodiments, in particular as defined in the dependent claims.

[0046] It shall be understood that a preferred embodiment of the invention can also be any combination of the independent claims or the above embodiments with the respective dependent claims.

[0047] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In the following figures:

[0049] Figure 1 Schematically and exemplarily, an embodiment of an interventional system for moving a tip of a catheter within a vascular structure is shown, the interventional system comprising an assisting device for assisting a user in moving the tip of a catheter within a vascular structure,

[0050] Figure 2 schematically and exemplarily shows a display of an auxiliary device for displaying two display images,

[0051] Figure 3 schematically and exemplarily showing a display image including an inset element representation having an alternating ring-like structure along the length of the inset element representation,

[0052] Figure 4 schematically and exemplarily showing a portion of an insert element representation having an alternating ring-like structure along the length of the insert element representation,

[0053] Figure 5 A flow chart exemplarily illustrating an embodiment of a method for assisting a user in moving a catheter within a vascular structure is shown. DETAILED DESCRIPTION

[0054] Figure 1 An embodiment of an interventional system 100 is schematically and exemplarily shown, which comprises an auxiliary device 10 , an image acquisition unit 12 and an insertion device 14 .

[0055] The assistance device 10 comprises an object image providing unit 16 , a tracking unit 18 , an insertion element representation generating unit 20 , a target element providing unit 22 , a computer readable medium in the form of a memory 24 , a transceiver 26 , a display 28 and a user interface in the form of an input unit 30 .

[0056] The image acquisition unit 12 includes a control unit 32, a support module in the form of an inspection table 34, a C-arm 36, an X-ray source in the form of an X-ray tube 38, and an X-ray detector 40. The X-ray tube 38 and the X-ray detector 40 are mounted on opposite sides of the C-arm 36, and the C-arm 36 is movable relative to the inspection table 34 so as to allow the image acquisition unit 12 to acquire 2D projection images in different acquisition directions. The control unit 32 controls the image acquisition unit 12 so that it can acquire an object image based on the projection images in different acquisition directions. In this embodiment, the object image is a 2D image. The object image may also be, for example, a 3D image or a slice of a 3D image.

[0057] The insertion device 14 includes an insertion element 42, which includes a catheter, a guide wire and an optical fiber. In this embodiment, the guide wire and the optical fiber are included in the catheter. In other embodiments, the insertion element may include one or more catheters, guide wires and / or optical fibers. In addition, the insertion device 14 includes a handpiece 44 with a user interface 56 and a data processing unit 58. The handpiece 44 allows the user to directly manipulate the insertion element 42 or manipulate the insertion element 42 via the user interface 56. The insertion element 42 is moved using a guide wire (not shown). In other embodiments, the insertion element may include one or more sensors for measuring one or more physical property values, such as temperature values ​​or pressure values ​​along the length of the insertion element. The sensor data can be collected by the data processing unit 58 and provided to the tracking unit 18 of the auxiliary device 10 in the form of physical property values.

[0058] The patient 46 is placed on the examination table 34 and the C-arm 36 is moved so that the image acquisition unit 12 can acquire an object image showing a view within the object in the form of a vascular structure 48 of the patient 46 by emitting X-rays 50 that pass through the vascular structure 48 of the patient 46 and are detected by the X-ray detector 40. The insertion element 42 is inserted into the vascular structure 48 to move the tip 52 of the insertion element 42 to a target element 54 within the vascular structure 48. The target element 54 is manually selected by a user using the user interface 56. In other embodiments, the target element may also be automatically detected and selected in the object image as the target element based on a procedure intended to be performed by using the insertion element within the object.

[0059] The assist device 10 may be used to assist a user in moving the insertion element 42 within the vascular structure 48 by providing better visualization that allows for faster and more accurate movement of the insertion element 42 within the vascular structure 48 .

[0060] The object image providing unit 16 is connected to the control unit 32 of the image acquisition unit 12 and receives the object image from the image acquisition unit 12. The object image providing unit 16 provides an object image showing a view inside the vascular structure 48. In this embodiment, the object image providing unit 16 receives the object images and provides them to other units of the auxiliary device 10. In other embodiments, the object image providing unit 16 can also be connected to another type of image acquisition unit, or include an image acquisition unit, such as a CT system, an MRI system, a US system, a nuclear imaging system such as a PET system or a single photon emission tomography imaging system, or any other image acquisition unit for acquiring object images. The object images generated by the other type of image acquisition unit may include CT images, MR images, PET images, US images, or any other type of images acquired using another type of image acquisition unit.

[0061] The tracking unit 18 is connected to the insertion device 44 and tracks the 3D position, 3D local shape and orientation of the insertion element 42 within the vascular structure 48 based on optical shape sensing. The insertion element 42 includes an optical fiber having multiple cores (in this embodiment, a central core and three outer cores (not shown)), which provide strain measurement data that allows the tracking unit 18 to determine the 3D position, 3D local shape and orientation of the catheter 42 by optical shape sensing. Optical shape sensing can be performed according to the description in US7772541B2, US8116601B2 or according to another known optical shape sensing technology. In other embodiments, the auxiliary device can also track the 3D position and 3D local shape of the insertion element by using optical shape sensing.

[0062] The tracking unit 18 is registered with the object image providing unit 16 by identifying the position of the insertion element 42 in the object image and also by determining this position by the tracking unit 18. In other embodiments, the tracking unit 18 and the object image providing unit 16 may also be registered in any other way.

[0063] The insertion element representation generation unit 20 generates an insertion element representation that represents the insertion element 42 within the vascular structure 48 in its 3D position, 3D local shape, orientation, and size based on the object image and the tracked 3D position, tracked 3D local shape, and tracked orientation of the insertion element 42. In this embodiment, the tracking unit 18 provides parameter values ​​for different points along the length of the insertion element 42 based on the tracked 3D position, tracked local shape, and tracked orientation of the insertion element 42. The insertion element representation generation unit 20 calculates visualization values ​​for points along the length of the insertion element 42 that depend on the parameter values ​​for points along the length of the insertion element 42. The parameter values ​​include distortion, strain, and curvature determined by optical shape sensing. In other embodiments, the parameter values ​​may include one or more of distortion, strain, and curvature. The insertion element representation generation unit 20 provides different markings applied along the length of the insertion element representation in the form of different colors representing an indication of the local shape of the insertion element according to the visualization values. In this embodiment, the tip 52 is white and has a different color compared to another color applied along the length of the insertion element representation. In other embodiments, the insertion element representation generation unit may provide different marks in the form of a ring structure according to the visualization value, the ring structure having a deformed shape that is indicative of the local shape of the insertion element. The ring structure may, for example, be an alternating ring structure of higher brightness and lower brightness, etc. The insertion element representation generation unit may provide the deformed shape of the ring structure so that for a bend in the local shape of the insertion element, the ring structure appears to be closer on the inside of the bend and further apart on the outside of the bend, so as to emphasize the three-dimensional effect of the shape of the ring structure. In addition, the insertion element representation generation unit may provide the ring structure with a distortion effect for the distortion of the local shape of the insertion element.

[0064] In other embodiments, instead of color, other marks can be provided, such as different shadows, different opacities, different brightness, etc. In other embodiments, the insertion element representation providing unit can also provide at least two different marks applied along the length of the insertion element representation according to the visualization value. In other embodiments, physical property values ​​such as temperature values ​​and / or pressure values ​​can be provided by the tracking unit, and the insertion element representation providing unit can provide at least two different superimposed marks indicating the physical property values ​​of the insertion element along the length of the insertion element representation, which are suitable for superimposing on at least two different marks without covering them. In order to make the superimposed mark not cover the mark, the superimposed mark can be selected according to the mark. For example, if the mark is a color, the superimposed mark can be brightness or shadow. In another embodiment, one of the at least two different marks can be applied at the tip of the insertion element, and the mark applied at the tip can be different from the mark applied along the rest of the length of the insertion element representation, so as to allow the user to identify the orientation of the tip of the insertion element.

[0065] The display 28 displays a display image including an insertion element representation having different colors applied along the length of the insertion element representation within the object image, i.e., a white tip and the rest of the length of the insertion element 42 having another color. This allows additional information to be provided to the user and assists the user in moving the insertion element 42 within the vascular structure 48, for example, for moving the tip 52 to the target element 54. Alternatively or additionally, the display can display an insertion element representation having an annular structure having a deformed shape along the length of the insertion element representation. In addition, superimposed markings can be displayed along the length of the insertion element representation. In other embodiments, the display can also display orientation information of the catheter relative to the vascular structure and / or orientation information of the tip of the catheter relative to the target element.

[0066] The target element providing unit 22 may provide a target representation of a target element 54 within the vascular structure 48 to which the tip 52 of the insertion element 42 is to be moved. The target element providing unit 22 may additionally calculate a path from the tip 52 to the target element 54 that avoids other matter, and provide a target path representation that indicates the path from the tip 52 to the target element 54 that avoids other matter. The display 28 may display a target representation that represents the target element 54 and / or the target path representation. For example, the display 28 may virtually project the target representation and the insertion element representation with the markings onto the object image, taking into account the image acquisition geometry used to generate the object image, so as to visualize the 3D spatial relationship between the target element 54 and the tip 52 of the insertion element 42 on the object image.

[0067] The memory 24 stores a computer program product for assisting a user in moving an insertion element 42 within a vascular structure 48. The computer program product includes program code modules that, when the computer program product is run on the assisting device 10, cause the assisting device 10 to perform a method for assisting a user in moving an insertion element 42 within a vascular structure 48 (e.g., Figure 5 The memory 24 may additionally store other data, such as object images and display settings.

[0068] The transceiver 26 allows transmission and reception of data. The transceiver 26 may be connected to a clinical information system for exchanging data (not shown). Alternatively or additionally, the transceiver may connect the interventional system with the Internet.

[0069] The input unit 30 can be used by the user to input commands, such as a start command for starting an auxiliary process or a stop command for stopping an auxiliary process. The input unit 30 can be a keyboard, a computer mouse, a touch screen, etc.

[0070] Figure 2 An auxiliary device (eg, as shown in FIG. 1 ) for displaying two display images 60 and 60 ′ is schematically and exemplarily shown. Figure 1 1 . The display 28 of the auxiliary device 10 is presented. In this embodiment, the object image providing unit provides two object images 62 and 62' with different views inside the vascular structure. The display 28 displays the two display images 60 and 60' side by side so that the user can see two different perspectives inside the vascular structure.

[0071] Display image 60 shows an insertion element representation 42' including a catheter representation 64 and a guidewire representation 66 within the catheter representation 64 (both within object image 62). In addition, a target representation 68 is placed at the height of the patient's lowest kidney within object image 62. The insertion element enters the patient's body via the right iliac bone. An endovascular aneurysm repair (EVAR) device is placed in the abdominal aorta below the renal arteries. Display image 60' shows another perspective, respectively, in which the insertion element representation 42" includes a catheter representation 64' and a guidewire representation 66' within the catheter representation 64', both within object image 62'. The target representation 68' is placed at the height of the lowest kidney, i.e., at the target element.

[0072] In this embodiment, a different color is applied to the tips 52' and 52" of the catheter representations 64' and 64" as compared to the rest of the length of the catheter representations 64' and 64". The tips 52' and 52" are white 70 and 70' to improve identification of the orientation of the tips 52' and 52". In particular, this allows better identification of whether the tips 52' and 52" are pointing towards or away from the viewing angle. Another color 72 and 72' is applied along the rest of the length of the catheter representations 64' and 64". This allows better identification of the local shape of the catheter represented by its catheter representation.

[0073] In this embodiment, the guidewire is inside the catheter.The tips 53 and 53' of the guidewire representations 66 and 66' have another color than the rest of the length of the guidewire representations 66 and 66' to improve identification of the orientation of the tips 53 and 53'.

[0074] In this embodiment, the display 28 furthermore displays orientation information of the insertion element representations 42' and 42" relative to the vascular structure based on the view within the object and the tracked 3D position and the tracked orientation of the insertion element within the vascular structure. The orientation information includes a human body representation 74 and 74' rotated around the longitudinal axis, wherein the zero degree position corresponds to a reference system showing the patient on the examination table in a front view as seen by the user. Furthermore, the orientation information includes rotation angle representations 76 and 76', which provide the degree to which the view within the object is rotated compared to the reference system. The rotation angle depends on the rotation of the C-arm relative to the patient on the examination table. In the display image 60, the rotation angle is 0° and the angulation angle is -1°. In the display image 60', the rotation angle is 50° and the angulation angle is 0°. The orientation information allows for improved visualization.

[0075] Figure 3 Schematically and exemplarily, a display image 60" is shown including an insertion element representation 42'" having an alternating annular structure 78 having a deformed shape along the length of the insertion element representation 42'" within the object image 62". In this embodiment, the annular structure 78 alternates with lighter and darker colors along the length of the insertion element representation 42'"'. In other embodiments, the annular structures may also alternate with brightness, opacity, shading, or any other kind of marking. In yet another embodiment, an overlay marking may be applied along the annular structure along the length of the insertion element representation. Additionally, the tip of the insertion element may have a different marking, such as a color, than the rest of the insertion element representation.

[0076] In this embodiment, the annular structure has a deformed shape so that for a bend in the local shape of the insert element, the annular structures 78 appear to be closer together on the inside of the bend and further apart on the outside of the bend so as to emphasize the three-dimensional effect of the shape of the annular structure 78. In addition, the annular structure 78 shows a distortion effect for the distortion of the local shape of the insert element. This allows for improved visualization of the depth and curvature of the elongated structure of the insert element representation 42".

[0077] Figure 4 In the display image 60'' a portion of the insertion element representation 42'' is schematically and exemplarily shown having an alternating ring structure 78' having a deformed shape along the length of the insertion element representation 42''. Figure 3In the embodiment of the embodiment of the present invention, the ring structure 78' has alternating colors of lighter color and darker color. The shape of the ring structure 78' is deformed so that the ring structures 78' are closer together on the inner side 80 of the bend and farther apart on the outer side 82 of the bend. The difference in the length of the ring structures on the inner side of the bend and the outer side of the bend can be perceived by the user as the depth and curvature of the insertion element representation 42", such as a shower hose. This allows for improved 3D visualization of the insertion element.

[0078] Figure 5 A flow chart exemplarily illustrating an embodiment of a method for assisting a user in moving an insertion element in the form of a catheter within an object in the form of a vascular structure is shown.

[0079] In step 200, the 3D position, 3D local shape and orientation of the catheter within the vascular structure are tracked by a tracking unit. The tracked 3D position, 3D local shape and orientation of the catheter within the vascular structure can be provided as parameter values ​​by the tracking unit. In other embodiments, only the 3D position and 3D local shape of the insertion element within the object can also be tracked by the tracking unit. In yet another embodiment, the tracking unit can track physical property values ​​(such as temperature values, pressure values ​​or any other physical properties) of the insertion element and provide the physical property values.

[0080] In step 210, an object image showing a view within a vascular structure is provided. In this embodiment, a catheter is inserted into the vascular structure so that the object image includes the catheter. In this embodiment, the object image is a 2D projection X-ray image acquired from different acquisition directions. In other embodiments, the object image may also be a 3D image or any other type of image. The object image may also be a CT image, an MR image, a PET image, a US image, or an image acquired using any other type of image acquisition unit. Step 210 is optional. In addition, steps 200 and 210 may be performed in parallel or in an alternating order.

[0081] In step 220, the tracking unit and the object image providing unit are registered relative to each other. The registration can be performed in several ways, such as 2D-3D registration. In this embodiment, the tracking unit is registered with the object image providing unit by identifying the position of the catheter in the object image and also by determining the position by the tracking unit. In other embodiments, any other method for registering the tracking unit with the object image providing unit can be performed in order to register them. Step 220 is optional. In particular, when step 210 is not performed, no registration is required.

[0082] In step 230, an insertion element representation representing the catheter in the vascular structure is generated by the insertion element representation generation unit based on the tracked 3D position, tracked 3D local shape and orientation of the insertion element and based on the object image and its size. In other embodiments, the insertion element representation generation unit may generate the insertion element representation representing the catheter in the vascular structure based on the tracked 3D position and tracked 3D local shape of the insertion element and its 3D position and 3D local shape.

[0083] In step 240, different marks in the form of annular structures with alternating brightness and deformed shapes applied along the length of the insertion element representation are provided by the insertion element representation generation unit, which represent the indication of the local shape of the insertion element, that is, the brightness of the annular structure along the length of the insertion element alternates between lower brightness and higher brightness. In addition, the tip is provided with white, and the rest of the insertion element representation is provided with another color other than white. In addition, orientation information including human body representation and rotation angle representation relative to the reference system is provided. The deformed shape of the annular structure depends on the local shape of the insertion element. In particular, the deformed shape of the annular structure is provided so that for the bending of the local shape of the catheter, it appears to be closer on the inner side of the bending and more separated on the outer side of the bending, so as to emphasize the three-dimensional effect of the shape of the annular structure. The deformed shape is further provided with a distortion effect for the distortion of the local shape of the catheter. The mark can also include opacity, brightness, shadow or any other type of mark. In other embodiments, at least two different marks representing the indication of the local shape of the insertion element applied along the length of the insertion element representation can be provided by the insertion element representation generation unit. In addition, at least two different superimposed marks can be provided.

[0084] In step 250, two display images are displayed by the display unit. Both display images show different perspectives within the vascular structure. Each of the display images includes an insertion element representation having an annular structure and a tip color different from other colors applied along the length of the insertion element representation in the object image. The tip is white to allow improved identification of the orientation of the tip and therefore the orientation of the catheter. In addition, orientation information is displayed in each of the display images. In other embodiments, only one display image or more than two display images may also be displayed. One or more display images may also include an insertion element representation having at least two different marks and optionally two or more superimposed marks applied along the length of the insertion element representation. In other embodiments, the display image may include an insertion element representation having at least two different marks and optionally two or more superimposed marks, and the at least two different marks are applied in one of the object images along the length of the insertion element representation.

[0085] In one embodiment, the insertion element representation has a different color and the tip of the insertion element representation has a different color than the rest of the insertion element representation.A display image including the insertion element representation having a white tip may be displayed within the object image.

[0086] In other embodiments of the method, one or more target elements may be selected by a user, and a target representation of the target element may be provided by a target element providing unit and displayed by a display. In addition, a target path representation may be provided.

[0087] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the claims. The invention can also be used, for example, to assist a user in moving an inserted element within an object, such as a tube system, e.g., a pneumatic tube system.

[0088] In the claims, the words "comprising" and "including" do not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality.

[0089] A single unit or device may fulfill the functions of several items recited in the claims.The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0090] The operations performed by one or several units or devices (such as tracking the 3D position and the 3D local shape of the insertion element within the object by the tracking unit, generating an insertion element representation representing the insertion element within the object with its 3D position and 3D local shape based on the tracked 3D position and the tracked 3D local shape of the insertion element by the insertion element representation generation unit, providing at least two different marks indicating the local shape of the insertion element applied along the length of the insertion element representation by the insertion element representation generation unit, displaying a display image including the insertion element representation by the display, the insertion element representation having at least two different marks applied along the length of the insertion element representation, etc.) may also be performed by any other number of units or devices. These operations and / or controls of the auxiliary device according to the method for assisting a user in moving an insertion element within an object may be implemented as program code modules of a computer program and / or dedicated hardware.

[0091] The computer program may be stored / distributed on suitable media, such as optical storage media or solid-state media provided together with or as part of other hardware, but the computer program may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0092] Any reference signs in the claims should not be construed as limiting the scope.

[0093] The present invention relates to assisting a user in moving an insertion element within an object. The 3D position and 3D local shape of the insertion element are tracked within the object. Based on the tracked 3D position and 3D local shape of the insertion element, an insertion element representation is generated that represents the insertion element within the object with its 3D position and 3D local shape. In addition, at least two different markings are provided that are applied along the length of the insertion element representation to indicate the local shape of the insertion element. A display image including the insertion element representation with different markings is displayed. The markings can, for example, include alternating ring structures having alternating lighter and darker colors and / or different colors for the tip and the rest of the insertion element representation to improve the 3D visualization of the insertion element.

Claims

1. An assisting device (10) for assisting a user in moving an insertion element (42) within an object (48), the assisting device (10) comprising: a tracking unit (18) for tracking the three-dimensional position and the three-dimensional local shape of the insertion element (42) within the object (48), - an insertion element representation generation unit (20) for generating an insertion element representation (42', 42", 42"'; 42"") based on the tracked three-dimensional position and the tracked three-dimensional local shape of the insertion element (42), the insertion element representation representing the insertion element (42) within the object (48) with the three-dimensional position and the three-dimensional local shape of the insertion element, and wherein the insertion element representation generation unit (20) is configured to provide at least two different markings (70, 70', 72, 72'; 78; 78') applied along the length of the insertion element representation (42', 42"; 42"'; 42""), the at least two different markings representing an indication of the local shape of the insertion element (42), the at least two different markings comprising alternating ring-like structures having alternating lighter and darker colors, and - a display (28) for displaying a display image (60, 60'; 60"; 60"') comprising said insertion element representation (42', 42"; 42"'; 42"") having said at least two different markings (70, 70', 72, 72'; 78; 78'), said at least two different markings being applied along said length of said insertion element representation (42', 42"; 42"'; 42"").

2. The auxiliary device (10) according to claim 1, wherein: The insertion element representation generation unit (20) is configured to provide the at least two marks as annular structures (78; 78') having a deformed shape, and: By means of the deformed shape, the annular structure (78; 78') appears closer on the inner side (80) of the bend and further apart on the outer side (82) of the bend for the bend of the local shape of the insert element (42), so as to emphasize the three-dimensional effect of the shape of the annular structure (78; 78'), and / or There is a distorting effect on the distortion of the local shape of the insert element (42).

3. The auxiliary device (10) according to claim 1, wherein: The auxiliary device (10) comprises an object image providing unit (16) for providing an object image (62, 62'; 62”)， wherein the tracking unit (18) is adapted to track the orientation of the insertion element (42) within the object (48), wherein the insertion element representation generation unit (16) is configured to generate the insertion element representation (42', 42", 42"'; 42"") having a three-dimensional position, orientation and size based on the object image (62, 62'; 62"); and wherein the display (28) is configured to display the display image (60, 60'; 60"; 60"') comprising the insertion element representation (42', 42"; 42"'; 42"") having the at least two different markings (70, 70', 72, 72'; 78; 78'), wherein the at least two different markings are applied along the length of the insertion element representation (42', 42"; 42"'; 42"") arranged within the object image (62, 62'; 62").

4. The auxiliary device (10) according to claim 3, wherein: The display (28) is configured to display orientation information (74, 74', 76, 76') of the insertion element (42) relative to the object (48) based on the view within the object (48), the tracked three-dimensional position of the insertion element (42) within the object (48), and the tracked orientation.

5. The auxiliary device (10) according to claim 4, wherein: The object image providing unit (16) is configured to provide at least two object images (62, 62') having different views within the object (48), and wherein the display (28) is configured to display at least two display images (60, 60'), wherein each of the display images (60, 60') includes the insertion element representation (42', 42") having the at least two different markings (70, 70', 72, 72'), the at least two different markings being applied along the length of the insertion element representation (42', 42") arranged within one of the object images (62, 62').

6. The auxiliary device (10) according to claim 1, wherein: One of the at least two different markings (70, 70', 72, 72'; 78; 78') is applied at a tip (52', 52") of the insertion element representation (42', 42"), and wherein the marking (70, 70') applied at the tip (52', 52") is different from the marking (72, 72') applied along the remainder of the length of the insertion element representation (42', 42") so as to allow the user to identify the orientation of the tip (52', 52") of the insertion element representation (42', 42") 7. The auxiliary device (10) according to claim 1, wherein: The tracking unit (18) is configured to track the three-dimensional position and the three-dimensional local shape of the insertion element (42) by using optical shape sensing.

8. The auxiliary device (10) according to claim 1, wherein: The tracking unit (18) is configured to provide parameter values ​​for different points along the length of the insertion element (42) based on the tracked three-dimensional position and the tracked three-dimensional local shape of the insertion element (42), and Therein, the insertion element represents that the generation unit (20) is adapted to: - calculating, for said points along said length of said insertion element (42), a visualization value dependent on said parameter value provided by said tracking unit (18), and - providing the at least two different markings (70, 70', 72, 72'; 78; 78') applied along the length of the insertion element representation (42', 42"; 42"'; 42"") in dependence on the visualization value.

9. The auxiliary device (10) according to claim 8, wherein: The parameter value includes at least one of the following: distortion, strain and curvature value.

10. The auxiliary device (10) according to claim 1, wherein: The tracking unit (18) is configured to track a value of a physical property of the insertion element (42), wherein the insertion element representation generation unit (20) is configured to provide at least two different superimposed markings along the length of the insertion element representation (42', 42", 42"'; 42""), the at least two different superimposed markings representing indications of the physical property values ​​of the insertion element (42), the at least two different superimposed markings being adapted to be superimposed on the at least two different markings (70, 70', 72, 72'; 78; 78') without covering the at least two different markings, and wherein the display (28) is configured to display the display image (60, 60', 60"; 60"') comprising the insertion element representation (42', 42"; 42"'; 42"") having the at least two different markings (70, 70', 72, 72'; 78; 78') and the at least two different superimposed markings, wherein the at least two different superimposed markings are applied along the length of the insertion element representation (42', 42"; 42"'; 42"").

11. The auxiliary device (10) according to claim 1, wherein: The auxiliary device (10) comprises a target element providing unit (22) for providing a target representation (68, 68') of a target element (54) within the object (48) to which the insertion element (42) should be moved, and wherein the display (28) is configured to display the display image (60, 60') comprising the insertion element representation (42', 42") having the at least two different markings (70, 70', 72, 72') and the target representation (68, 68') of the target element, wherein the at least two different markings are applied along the length of the insertion element representation (42', 42").

12. An interventional system (100) for moving an insertion element (42) within a subject (48), wherein: The interventional system (100) comprises: - an insertion device (14) comprising said insertion element (42), and - Assistance device (10) for assisting a user in moving the insertion element (42) within the object (48) according to claim 1.

13. A method for representing an inserted element (42) within an object (48), the method comprising: - generating, by an insertion element representation generating unit (20), an insertion element representation (42', 42", 42"', 42"") based on the three-dimensional position and the three-dimensional local shape of the insertion element (42) within the object (48) tracked by the tracking unit (18), the insertion element representation representing the insertion element (42) within the object (48) using the three-dimensional position and the three-dimensional local shape of the insertion element, - providing by the insertion element representation generation unit (20) at least two different markings (70, 70', 72, 72'; 78; 78') applied along the length of the insertion element representation (42', 42"; 42"'; 42""), the at least two different markings representing an indication of the local shape of the insertion element (42), the at least two different markings comprising alternating ring-like structures having alternating lighter and darker colors, and - displaying by the display (28) a display image (60, 60'; 60"); comprising said insert element representation (42', 42"; 42"'; 42"") having said at least two different markings (70, 70', 72, 72'; 78; 78'); 60"'), said at least two different markings being applied along said length of said insert element representation (42', 42"; 42"'; 42"").

14. A computer program product for assisting a user in moving an insertion element (42) within an object (48), the computer program product comprising a program code module for causing the assisting device (10) to perform the steps of the method according to claim 13 when the computer program product is run on the assisting device (10) according to claim 1.

15. A computer readable medium (26) having stored thereon a computer program product according to claim 14.

Images