Image enhancement based on fiber optic shape sensing

Abstract

The present invention relates to an image processing system 10 having a processor unit 20 configured to receive imaging data associated with an imaging system 40 and optical shape sensing data associated with an optical shape sensing system 50 aligned with the imaging system 40, where the optical shape sensing data can be located within the imaging system, wherein the processor unit is configured to define a region of interest within the imaging data based on the imaging data and / or the optical shape sensing data, and further configured to use the optical shape sensing data as a marker within the region of interest such that the processor unit applies image enhancement of the imaging data to the region of interest based on the received optical shape sensing data.

Description

[Technical Field]

[0001] The present invention relates to a system and method for enhancing views during an intervention for a medical imaging system. [Background technology]

[0002] Minimally invasive interventions can be performed under X-ray guidance. To minimize radiation dose, physicians compromise on image quality. Low signal-to-noise objects in X-ray images, such as stent struts and calcifications, can be enhanced using image integration such as StentBoost, which refers to stent enhancement in the image by showing finer details of the stent struts, while background noise and anatomical structures are faded out. This only works if the images can be correctly registered. This means that markers are identifiable and device movement does not result in out-of-plane overlays. Summary of the Invention [Problem to be solved by the invention]

[0003] It would be advantageous to have improved techniques for image enhancement during interventions. [Means for solving the problem]

[0004] The object of the present invention is solved by the subject matter of the independent claims, further embodiments are incorporated in the dependent claims.

[0005] It should be noted that the aspects and examples of the invention described below also apply to apparatus, methods, computer program elements and computer readable media.

[0006] In a first aspect, an image processing system is provided, the image processing system having a processor unit configured to receive imaging data associated with an imaging system and optical shape sensing data associated with an optical shape sensing system aligned with the imaging system so that the optical shape sensing data can be located within the imaging system, the processor unit configured to define a region of interest in the imaging data based on the imaging data and / or the optical shape sensing data, and further configured to use the optical shape sensing data as a marker within the region of interest so as to apply image enhancement of the imaging data to the region of interest based on the received optical shape sensing data.

[0007] For example, stent boosting improved image quality by identifying markers in an image and using them to perform motion compensation across image frames, which then refined the image quality. This algorithm struggles to extrapolate to more general markers that cannot be predefined when multiple markers are present or when a marker is not visible. Combining FORS with stent boosting can address these challenges by limiting the search area within the image and providing 3D localization of the device.

[0008] In other words, an example of the present invention provides for embedding an optical shape sensing ("OSS") fiber in an interventional instrument and using it as a marker once co-registered with an imaging system, such as an interventional X-ray imaging system, a magnetic resonance system, or an ultrasound imaging system.

[0009] The present invention advantageously provides that during vascular procedures, obtaining high-quality images of therapeutic devices (stents, balloons, endografts, etc.) can be difficult due to their small size, obstruction by other devices in the field of view, motion artifacts, or simply the anatomical structure itself. Furthermore, as devices become smaller and more tissue-like, they also lose some of the features that are visible via imaging.

[0010] Stent boost has been developed to overcome some of these challenges, but knowing the exact location of the guidewire, catheter, stent, or endograft could further improve the image quality of these types of devices. There are two specific cases where this is particularly challenging. i. When there are multiple markers present in the image and it is difficult to identify which marker is associated with the device of interest. ii. When the marker is very difficult to visualize on X-ray (e.g., biodegradable stents). iii. When enhancing objects or structures that do not have known markers (e.g., anatomical structures such as caps or blood vessels). iv. Compensate for out-of-plane motion.

[0011] The present invention advantageously provides that a system or apparatus as defined by claim 1 utilizes, for example, a FORS device, a FORS system, an imaging system (e.g., X-ray or ultrasound) and a controller, an image processing system, and a visualization system. The present invention envisages that the FORS device and the imaging system are co-registered in spatial coordinates.

[0012] One aspect of the present invention provides, for example, how to use position information of a FORS device to improve image quality in another imaging modality (eg, x-ray or ultrasound).

[0013] Aspects of the present invention are based, for example, on the following. · To provide 1) a representation of the reconstructed OSS fiber, and 2) images of the object aligned with each other. i) defining, automatically or via a user interface, a region of interest based on data from, for example, an OSS guidewire, a balloon, a stent, or an endograft tip; Searching and identifying markers close to the FORS data within the region of interest (i.e., only markers placed on the path), which discards non-viable markers. · Applying stent boost to that region of interest based on the identified markers (injection of contrast medium from a series of X-ray images of the same region of interest). · i) and ii) are overlaid with higher contrast in the region of interest as a result of stent boost.

[0014] According to an exemplary embodiment of the present invention, the 3D data of FORS can be used to further filter out frames that are out of plane.

[0015] According to an exemplary embodiment of the present invention, the FORS shape between two markers can be used to assess when there is a shape change to the stent (as opposed to current techniques which consider translation). These frames can be dropped, or FORS can be used to morph the shape of the device to match other frames.

[0016] According to an exemplary embodiment of the present invention, the FORS data itself can be used as a marker, which accelerates the computation time for image integration and reduces the number of false positive markers, thereby improving image quality.

[0017] According to an exemplary embodiment of the present invention, known markers (balloons, stents, guidewire curves, endografts, clips or valve devices, vessel contours, vessel bifurcations...) can be recognized in a series of images for stent boost.

[0018] In this manner, calcification along the vessel and cap morphology of chronic total occlusions may also be visualized, according to an exemplary embodiment of the present invention.

[0019] According to an exemplary embodiment of the present invention, a processor unit is configured to apply image enhancement to a series of contrast-enhanced X-ray images of a region of interest taken by an imaging system.

[0020] According to an exemplary embodiment of the present invention, the processor unit is configured to search for and identify markers with respect to restricting the markers to a subgroup of markers located on a path for an interventional instrument.

[0021] According to an exemplary embodiment of the present invention, the processor unit is configured to define a region of interest based on the position of a balloon, a stent, an endograft, or an interventional device.

[0022] According to an exemplary embodiment of the present invention, the processor unit is configured to use the optical shape sensing data to filter out frames of the imaging system that are out of plane.

[0023] According to an exemplary embodiment of the present invention, the processor unit is configured to use optical shape sensing data between at least two markers to assess an interventional instrument or a change in shape of said interventional instrument.

[0024] According to an exemplary embodiment of the present invention, the processor unit is configured to receive imaging data associated with an imaging system, for example, a computed tomography system, a magnetic resonance imaging system, an ultrasound or optical imaging system, an X-ray imaging system, a medical imaging system, or a diagnostic imaging system.

[0025] According to an exemplary embodiment of the present invention, the processor unit is configured to identify markers in a series of images taken by an imaging system.

[0026] According to an exemplary embodiment of the present invention, a series of images acquired by an imaging system is part of an applied image enhancement on a region of interest.

[0027] In a second aspect, there is provided an imaging system configured to communicate with an apparatus according to the first aspect or any implementation of the first aspect.

[0028] In a third aspect, there is provided an optical shape sensing system configured to communicate with an apparatus according to the first aspect or any implementation of the first aspect, the optical shape sensing system being configured to be aligned with an apparatus according to the second aspect or any implementation of the second aspect.

[0029] In a fourth aspect, a method for embedding fiber optic shape sensing in a medical imaging device is provided, the method comprising the steps of:

[0030] As a first step, a step is performed of receiving imaging data associated with an imaging system and optical shape sensing data associated with an optical shape sensing system aligned with the imaging system so that the optical shape sensing data can be placed within the imaging system using a processor unit.

[0031] As a second step, a step is performed in which a region of interest is defined in the imaging data based on the imaging data and / or the optical shape sensing data, and further, the step of using the optical shape sensing data as a marker within the region of interest such that the processor unit applies image enhancement of the imaging data to the region of interest based on the optical shape sensing data received using the processor unit.

[0032] In accordance with an exemplary embodiment of the present invention, the method further includes applying image enhancement to the series of contrast-enhanced X-ray images of the region of interest taken by the imaging system.

[0033] According to an exemplary embodiment of the present invention, the method further comprises applying marker search and identification with respect to restricting the markers to a subgroup of markers placed on a path for the interventional instrument.

[0034] According to an exemplary embodiment of the present invention, the method further comprises applying marker search and identification with respect to restricting the markers to a subgroup of markers placed on a path for the interventional instrument.

[0035] The above aspects and examples will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0036] Exemplary embodiments are described below with reference to the following drawings: [Brief explanation of the drawings]

[0037] [Figure 1] 1 shows a schematic configuration of an image processing system for embedding fiber optic shape sensing in a medical imaging device according to an exemplary embodiment of the present invention. [Figure 2] 1 illustrates a method for embedding fiber optic shape sensing in a medical imaging device, according to an exemplary embodiment of the present invention. [Figure 3] 1 shows an example of optical shape sensing devices superimposed on a pre-operative CT showing their location within the vasculature, according to an exemplary embodiment of the present invention. [Figure 4] 10 shows an example of an image of a stent boost showing better image quality of a stent according to an exemplary embodiment of the present invention. [Figure 5] 10 illustrates an example of defining local regions for enhancement based on FORS device and therapy device location, according to an exemplary embodiment of the present invention. [Figure 6] 10 illustrates an example of using a local region around a FORS device in combination with a therapy device to restrict the area of ​​an X-ray image for image processing, according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0038] 1 shows a schematic configuration of an image processing system for, for example, embedding fiber optic shape sensing in a medical imaging device, according to an exemplary embodiment of the present invention. The image processing system 10 includes a processor unit 20. The image processing system 10 is configured to be connected to, for example, a display unit 30. The image processing system 10 is configured to be connected to an imaging system 40.

[0039] The image processing system 10 is configured to be connected to an optical shape sensing system 50 aligned with the imaging system 40 .

[0040] The processor unit 20 is configured to receive imaging data associated with the imaging system 40 and optical shape sensing data associated with an optical shape sensing system 50 aligned with the imaging system 40 so that the optical shape sensing data can be located within the imaging system.

[0041] The processor unit 20 is configured to define a region of interest within the imaging data based on the imaging data and / or the optical shape sensing data, and the processor unit is further configured to use the optical shape sensing data as a marker within the region of interest to apply image enhancement of the imaging data to the region of interest based on the received optical shape sensing data.

[0042] 2 illustrates a method for embedding fiber optic shape sensing in a medical imaging device according to an exemplary embodiment of the present invention. The method comprises the following steps:

[0043] As a first step, step S1 is performed of receiving image data associated with the imaging system and optical shape sensing data associated with an optical shape sensing system aligned with the imaging system so that the optical shape sensing data can be placed in the imaging system using a processor unit.

[0044] As a second step, step S2 is performed in which a region of interest is defined in the imaging data based on the imaging data and / or the optical shape sensing data, and further, the processor unit uses the optical shape sensing data as a marker within the region of interest so that image enhancement of the imaging data is applied to the region of interest based on the optical shape sensing data received using the processor unit.

[0045] FIG. 3 shows an example of optical shape sensing devices superimposed on a pre-operative CT showing their location within the vasculature, according to an exemplary embodiment of the present invention.

[0046] According to an exemplary embodiment of the present invention, Fiber Optic RealShape (FORS) uses light along a multi-core optical fiber for device localization and navigation during surgical intervention.

[0047] In accordance with exemplary embodiments of the present invention, the principles involved utilize distributed strain measurements in optical fibers using characteristic Rayleigh backscattering or controlled grating patterns.

[0048] According to an exemplary embodiment of the present invention, a shape along an optical fiber begins at a particular point along the sensor, known as the launch or z=0, and subsequent shape positions and orientations are relative to that point.

[0049] According to an exemplary embodiment of the present invention, an optical shape sensing fiber can be integrated into a medical device to provide live guidance of the device during a minimally invasive procedure.

[0050] According to an exemplary embodiment of the present invention, an integrated fiber provides the position and orientation of the entire device.

[0051] Figure 3 shows the shape-sensing guidewire and shape-sensing catheter used for navigation to the left renal artery superimposed on a preoperative CT image.

[0052] FIG. 4 shows an example of a StentBoost image showing better image quality of a stent according to an exemplary embodiment of the present invention, where enhancement is provided by StentBoost, a tool that improves visualization of the stent relative to the vessel wall.

[0053] According to an exemplary embodiment of the present invention, the stent is enhanced in the image by showing finer details of the stent struts, while background noise and anatomical structures are faded out, allowing for more precise placement of the stent and the ability to immediately correct under-deployment.

[0054] According to an exemplary embodiment of the present invention, StentBoost is used in conjunction with a product to improve the image quality of stents. It locates marker bands of the stent in each image frame, compensates for any motion, and then averages across the image frames to improve image contrast. StentBoost is described in U.S. Patent No. 728,962, entitled "Medical Viewing System and Method for Detecting and Enhancing Structures in Noisy Images."

[0055] According to an exemplary embodiment of the present invention, enhancement or stent boost involves taking a series of X-ray images and locating known markers (e.g., balloon / stent markers) for use in coregistration of the series. This technique fails when there are multiple markers present in the images. In this case, a FORS-enabled guidewire can be used to limit the search range for the markers as they are located along the path of the guidewire. The search range can be: Device type, e.g., narrow search for stents / balloons, broader search for endografts, Estimated FORS error, e.g. as a function of curvature, twist, and length along the device, User-defined search areas Imaging system settings, e.g., pixel resolution, type of imaging protocol, Based on this, the FORS wire circumference can be determined.

[0056] This technique still uses markers, e.g., in the x-ray or ultrasound images, for motion compensation, reducing the accuracy requirements for the FORS device: the FORS device and the x-ray system must be co-registered so that their coordinate systems are aligned.

[0057] FIG. 5 shows an example of defining local regions for enhancement based on FORS device and therapy device location, according to an exemplary embodiment of the present invention.

[0058] Figure 5 shows an example of a FORS GW that can be used to identify the location of stent markers in an X-ray image. The FORS device is a guidewire used in combination with a UniCath hub to define the location of the stent. The area of ​​the stent is used to define the search range for the stent marker.

[0059] When motion compensation is performed using markers within the images, it is still difficult to account for out-of-plane motion. According to an exemplary embodiment of the present invention, the 3D position of the FORS can be used to filter out frames that are out-of-plane and not include them in the averaging, or to correct for scaling effects that out-of-plane motion has.

[0060] According to an exemplary embodiment of the present invention, the FORS shape between two markers can be used to assess when there is a shape change to the stent (as opposed to current techniques which consider translation). These frames can be dropped, or FORS can be used to morph the shape of the device to match other frames.

[0061] According to an exemplary embodiment of the present invention, there are limited markers in the image that capture the device (e.g., a biodegradable stent), in which case the FORS position and shape of one or more nodes can be directly used as a localizer for motion compensation.

[0062] According to an exemplary embodiment of the present invention, the FORS device and the imaging system are co-registered so that their coordinate systems are aligned.

[0063] According to an exemplary embodiment of the present invention, FORS accuracy is the limiting factor in the performance of this strategy. There are several additional approaches that can be used to improve accuracy, especially with the goal of improving performance during stent boost.

[0064] According to an exemplary embodiment of the present invention, FORS accuracy is high immediately after alignment, and therefore the method can include an automatic alignment step (including multiple image projections if necessary) to correct FORS errors before the stent boost algorithm.

[0065] According to an exemplary embodiment of the present invention, FORS relative accuracy is also high compared to absolute accuracy, and therefore relative FORS motion can be used to correct for device movement as opposed to absolute FORS position.

[0066] According to an exemplary embodiment of the present invention, enhancement or stent boost involves taking a series of X-ray images and locating known markers (e.g., balloon / stent markers) for use in coregistration of the series. This technique can be generalized to automatically identify appropriate localizers within the images to be used as markers for motion compensation if the region for stabilization is limited to the relevant portion of the images. In this case, a FORS-enabled device can be used to establish search regions for localizers along the path of the guidewire. This technique then uses these automatically generated localizers within the (e.g., X-ray or ultrasound) images for motion compensation, reducing the accuracy requirements on the FORS device.

[0067] According to an exemplary embodiment of the present invention, the FORS device and the imaging system must be co-registered so that their coordinate systems are aligned.

[0068] The localizer Guidewire curve Markers on the endograft Clip or valve device DSA vessel contour or vessel bifurcation (at the tip of the FORS catheter) Such devices can be both anatomical or device based.

[0069] The system may also have a library of predefined localizers to search for in the vicinity of the device, such as radiopaque marker bands, fenestrations, mitraclips, etc. Alternatively, the system may have a predefined set of typical features (e.g., edges, lines, points) that it automatically finds and identifies in the vicinity of the device.

[0070] FIG. 6 illustrates an example of using a local region around a FORS device in combination with a therapy device to restrict the area of ​​an X-ray image for image processing, according to an exemplary embodiment of the present invention.

[0071] FIG. 6 shows an example of a FORS GW that can be used to identify a search area and then identify localizers near wires on the endograft for use in stabilizing the image.

[0072] The present invention can be applied to many applications such as endovascular (guidewires, catheters, stent sheaths, deployment systems, etc.), endoluminal (endoscopes or bronchoscopes), orthopedic (k-wires and screwdrivers), and non-medical applications.

[0073] In another exemplary embodiment, the present invention is applicable to both Rayleigh (enhanced and standard) and fiber Bragg implementations of shape-sensing fibers, and to both manual and robotic operation of such devices.

[0074] In another exemplary embodiment, the present invention may be applied to any imaging system used in conjunction with FORS, including X-ray, ultrasound, MRI, CT, OCT, IVUS, endoscopy, and the like.

[0075] In another exemplary embodiment, a computer program or a computer program element is provided, characterized in that it is configured to perform, on a suitable system, the method steps of the method according to one of the previous embodiments.

[0076] Thus, the computer program element may be stored on a computing unit that may be part of an embodiment, which computing unit may be configured to perform or direct the execution of the steps of the method described above.

[0077] Furthermore, it may be configured to operate the components of the devices and / or systems described above. The computing unit can be configured to operate automatically and / or to execute user instructions. The computer program may be loaded into the working memory of the data processor. The data processor may thus be configured to execute a method according to one of the aforementioned embodiments.

[0078] This exemplary embodiment of the present invention encompasses both computer programs that use the present invention from the beginning, and computer programs that convert existing programs into programs that use the present invention by means of an update.

[0079] Furthermore, the computer program element may be capable of providing all the steps necessary to fulfill the procedures of the exemplary embodiments of the methods described above.

[0080] According to a further exemplary embodiment of the present invention, a computer readable medium such as a CD-ROM, a USB stick, etc. is presented, the computer readable medium having stored thereon computer program elements, which computer program elements are described by the preceding sections.

[0081] The computer program may be stored and / or distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0082] However, the computer program may also be presented over a network such as the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, the computer program element being configured to perform a method according to one of the aforementioned embodiments of the present invention.

[0083] It should be noted that the embodiments of the present invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method-type claims, and other embodiments are described with reference to apparatus-type claims. However, those skilled in the art will understand from the above and below description that, unless otherwise specified, any combination of features belonging to one type of subject matter, as well as any combination between features relating to different subject matters, is disclosed in the present application. However, all features can be combined to provide a synergistic effect that is greater than the simple sum of the features.

[0084] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered exemplary or explanatory and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the dependent claims.

[0085] In the claims, the word "comprise" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. a processor unit configured to receive imaging data comprising a series of X-ray images acquired by an imaging system and optical shape sensing data associated with an optical shape sensing system aligned with the imaging system; In an image processing system having The processor unit is configured to define a region of interest based on the imaging data and / or the optical shape sensing data, and the processor unit is further configured to use the optical shape sensing data as markers within the region of interest to apply image enhancement to the series of X-ray images of the region of interest based on the received optical shape sensing data, the image enhancement comprising motion compensation in the series of X-ray images, and the markers are are nodes whose positions and shapes are used directly as localizers for co-registration of the series of X-ray images for motion compensation; or establishing regions in which image markers can be searched for in the series of X-ray images to coregistrate the series of X-ray images for motion compensation; Image processing system.

2. 2. The image processing system of claim 1, wherein the processor unit is configured to search for and identify image markers in the imaging data and limit a search range for the image markers to a subgroup of image markers located on a path of an interventional instrument within the region of interest identified by the markers obtained from the optical shape sensing data.

3. 3. The image processing system of claim 1, wherein the processor unit is configured to define the region of interest including a location of a balloon, a stent, an endograft, or an interventional device identified based on the imaging data and / or the optical shape sensing data.

4. The image processing system of claim 1 , wherein the processor unit is configured to use the optical shape sensing data to filter out X-ray images in which the markers are out of plane.

5. 5. The image processing system of claim 1, wherein the processor unit is configured to evaluate a shape change of an interventional instrument in the region of interest using the positions of at least two image markers in the imaging data and the optical shape sensing data.

6. 6. The image processing system of claim 1, wherein the imaging system comprises a computed tomography system, a magnetic resonance imaging system, an ultrasound imaging system, an optical imaging system, an X-ray imaging system, a medical imaging system, or a diagnostic imaging system.

7. In an image processing method, receiving imaging data including a series of x-ray images acquired by an imaging system and optical shape sensing data associated with an optical shape sensing system registered to the imaging system; using a processor unit to define a region of interest in the imaging data based on the imaging data and / or the optical shape sensing data, and further using the optical shape sensing data as markers within the region of interest to apply image enhancement to the series of X-ray images of the region of interest based on the received optical shape sensing data, wherein the image enhancement comprises motion compensation in the series of X-ray images, and the markers include: are nodes whose positions and shapes are used directly as localizers for co-registration of the series of X-ray images for motion compensation; or establishing regions that can be searched in the series of X-ray images to coregistrate the series of X-ray images for motion compensation; A method having the following.

8. The method of claim 7 , further comprising applying the image enhancement to a series of contrast-enhanced X-ray images of the region of interest taken by the imaging system.

9. 9. The method of claim 7 or 8, further comprising searching for and identifying image markers in the imaging data and limiting a search range for the image markers to a subgroup of image markers located on a path of an interventional instrument identified by the markers obtained from the optical shape sensing data.

10. A computer program for controlling an image processing system according to any one of claims 1 to 6, arranged to carry out a method according to any one of claims 7 to 9 when executed by a processor.

11. A computer readable storage medium having instructions which, when executed by a computer, cause the computer to perform the method of any of claims 7 to 9.

Images