Target vessel reconstruction for non-rigid movement compensation

Abstract

The invention relates to a method for motion-compensated reconstruction of a target vessel (13) by - Providing (S1) an initial 3D reconstruction of a volume with the target vessel (13), - Identifying (S2) an image of the target vessel (13) and an image of a secondary vessel (14) different from the target vessel in the first 3D reconstruction, - Forward projection (S3) of the target vessel image (13) and the secondary vessel image (14) onto forward projection images, - Deleting (S4) the image of the second vessel (14) in the forward projection images, - Creating (S5) a secondary 3D reconstruction from the forward projection images with automatic motion compensation. The invention further relates to an image processing device for motion-compensated reconstruction, a computed tomography device, a computer program and a computer-readable storage medium.

Description

[0001] The present invention relates to a method for motion-compensated reconstruction of a target vessel. Furthermore, the present invention relates to an image processing device for motion-compensated reconstruction, a computed tomography device, a computer program, and a computer-readable storage medium.

[0002] Regardless of the grammatical gender of a particular term, persons with male, female or other gender identities are included.

[0003] In interventional oncology, particularly in tumor embolization, the blood vessels supplying the cancerous tissue must be blocked to trigger tumor necrosis. Furthermore, as little healthy tissue as possible should be affected by the blockage to prevent necrosis of this tissue. Contrast-enhanced 3D reconstructions can be used for this purpose. The 3D reconstruction makes it possible to locate the specific vessels that need to be blocked—the so-called feeder vessels.

[0004] To achieve the effects mentioned above, the 3D reconstruction of the contrast-enhanced vessels must be as accurate as possible. A significant obstacle to high-quality vascular reconstruction is non-rigid motion artifacts in the organs, for example, due to motion artifacts. While a rigid movement, for instance, displaces the entire organ, a non-rigid movement involves, for example, movement of the organ itself, i.e., it is location-dependent.

[0005] Highly precise motion compensation for the selection of vessels from high-contrast angiographic images is desirable.

[0006] A common solution is motion-compensated reconstruction, as presented in the article Rohkohl, C., Lauritsch, G., Biller, L., Prümmer, M., Boese, J., & Hornegger, J. (2010); Interventional 4D motion estimation and reconstruction of cardiac vasculature without motion periodicity assumption. Medical image analysis, 14(5), 687-694.

[0007] However, due to the complexity of the 3D motion field, these algorithms often fail with smaller vascular structures.

[0008] In general, motion compensation approaches attempt to maximize an image optimization criterion based on a) the projection space, b) the reconstruction space, or c) a combination of both. A common problem with such optimization algorithms is that different optimal solutions exist for different anatomical regions. For example, bony structures may not move significantly, while soft tissue structures move considerably. Motion compensation can then lead to blurring of the bony structures; or the soft tissue regions may not be adequately compensated because the optimization algorithm focuses on the bony regions. A common approach to mitigating these problems is to define a target volume in which the optimization is performed or to remove bone.

[0009] In the article Unberath et al. (2016). Virtual single-frame subtraction imaging. In Proc. Conf. Imag. Form. X-Ray CT (pp. 89-92), a general framework for material decomposition into single frames in the detector area is outlined. The method comprises a segmentation and a background estimation step, which provides a virtual mask image that can be used for subtraction. In many cases, the material decomposition leads to unclipped difference images, enabling the use of novel motion estimation methods that exploit epipolar consistency conditions.

[0010] Furthermore, the article by Preuhs, A., Manhart, M., Roser, P., Stimpel, B., Syben, C., Psychogios, M., ... & Maier, A. (2020). Deep autofocus with cone-beam CT consistency constraint. In Image Processing for Medicine 2020: Algorithms-Systems-Applications. Proceedings of the Workshop from March 15 to 17, 2020 in Berlin (pp. 169-174). Springer Fachmedien Wiesbaden proposes a novel learning-based approach that can compensate for movements within the acquisition plane. A CBCT consistency constraint has been introduced, which has proven reliable for movements perpendicular to the acquisition plane. Thus, movements both within and outside the plane can be reliably detected, achieving an average artifact suppression of 93%.

[0011] According to the article by Preuhs, A., Berger, M., Bauer, S., Redel, T., Unberath, M., Achenbach, S., & Maier, A. (2018). Viewpoint planning for quantitative coronary angiography. International Journal of Computer Assisted Radiology and Surgery, 13, 1159-1167, coronary angiography assesses the status of myocardial perfusion by analyzing 2D X-ray projections of contrast-enhanced coronary arteries. This is done using a movable C-arm system. Due to the inherent reduction in dimensionality in X-ray projection when the 3D scene is projected onto a 2D image, the viewing angle is crucial to ensure adequate visualization of the artery under investigation and thus enable a reliable diagnosis. An algorithm is presented that calculates optimal viewpoints for the assessment of coronary arteries without requiring 3D models.Optimal viewpoint planning is performed solely on the basis of a single angiographic X-ray image.

[0012] DE 10 2013 217 351 A1 discloses a method for motion compensation of image data of a moving object under investigation, wherein a motion field for correcting motion artifacts is iteratively determined using an image metric. Furthermore, DE 10 2021 201 600 A1 discloses a computer-implemented method for generating a corrected set of imaging parameters from at least one initial set of imaging parameters, wherein the corrected set of imaging parameters is generated based on manipulated projection data sets, which have been created by applying a manipulation operation to at least a subset of initial projection data sets.Furthermore, DE 10 2010 022 791 A1 discloses a method for the three-dimensional representation of a moving structure using a rotational angiography method, wherein a first motion correction is carried out with affine transformations and a further motion correction is carried out with elastic deformations.

[0013] The object of the present invention is to improve the reconstruction of a target vessel that is subject to non-rigid movement.

[0014] According to the invention, this problem is solved by a method, an image processing device, and a computed tomography device as defined in the independent claims. Furthermore, a computer program and a computer-readable storage medium relating to the method are provided. Advantageous embodiments of the invention are described in the dependent claims.

[0015] According to the present invention, a method for motion-compensated reconstruction of a target vessel is provided. The target vessel is, for example, a so-called feeder vessel that supplies a tumor with blood. However, the target vessel can also be a bile duct or the like. This target vessel should be reconstructed with as few motion artifacts as possible.

[0016] First, an initial 3D reconstruction of a volume containing the target vessel is provided. "Providing" can refer to various processes, such as receiving, reading, or acquiring data, for example, acquiring and / or reading data from a computer-readable data storage device and / or receiving data from a data storage unit, particularly a database. This provision makes the data from the initial 3D reconstruction available for the procedure. In other words, the target vessel is spatially represented in the initial 3D reconstruction. Typically, this initial 3D reconstruction is distorted by motion artifacts. Based on this distorted 3D reconstruction, it would be difficult, for example, to insert a catheter into the target vessel.

[0017] Therefore, in a further step, an image of the target vessel and an image of a secondary vessel different from the target vessel are identified in the initial 3D reconstruction (hereinafter, the terms "target vessel" and "secondary vessel" always refer to an "image of the target vessel" and an "image of the secondary vessel," respectively, unless explicitly stated otherwise or the context makes it clear otherwise). Of course, many other secondary vessels may be present in the initial 3D reconstruction. All of these secondary vessels typically interfere with automatic motion compensation algorithms. Therefore, the goal is to remove as many secondary vessels as possible from the volume of the initial 3D reconstruction in order to focus the motion compensation on the target vessel. For this purpose, the target vessel is identified, for example, manually by marking it or automatically, for example, through image processing.Identifying the representation of the target vessel and the representation of the secondary vessel (which differs from the target vessel) can involve annotation and / or segmentation, particularly using threshold values ​​and / or time-intensity curves, of individual pixels in the initial 3D reconstruction. The pixels in the initial 3D reconstruction can, for example, be configured as pixels and / or voxels and have corresponding image values ​​representing the volume containing the target vessel and the secondary vessel.

[0018] In a further step, the target vessel and the secondary vessel are projected forward onto forward projection images. Forward projection can map a 3D object onto a two-dimensional plane, particularly virtually. Depending on the projection direction, different forward projection images of the 3D object can result. Forward projection images are 2D images in a projection image space. Generally, all other secondary vessels are also projected forward along with the primary vessel. The purpose of this forward projection is to make it easier to delete secondary vessels in the two-dimensional projection image space than in the 3D reconstruction. "Deleting the secondary vessel" means that the influence of the secondary vessel in the respective image is reduced below a predetermined limit and, if necessary, minimized. This also means, for example, that...The influence of other organs on the respective pixels is generally not reduced.

[0019] Therefore, in a further step of the process, the secondary vessel is erased from the forward projection images. Naturally, all other secondary vessels can also be erased as far as possible. This erasure is achieved, for example, by reducing the contrast or by painting over the secondary vessel(s). The erasure of the secondary vessel does not need to be complete, but it should be largely achieved. For example, the contrast of the secondary vessel relative to its surroundings should be reduced to at least one-quarter.

[0020] The forward projection images therefore no longer contain the secondary vessels, or at most only as shadows. In a subsequent step, a second 3D reconstruction is created from the forward projection images with automatic motion compensation. The second 3D reconstruction is thus reconstructed from the two-dimensional forward projection images without, or largely without, secondary vessels. During the 3D reconstruction, automatic motion compensation is applied, allowing the compensation algorithm to focus on the target vessel. Consequently, optimal motion compensation can be achieved with regard to the target vessel. The resulting second 3D reconstruction therefore shows the target vessel, in which motion artifacts are generally reduced to a greater degree than in the first 3D reconstruction.

[0021] In one embodiment, it is provided that during forward projection, motion blur of the second vessel occurs in the respective forward projection image. This blur is taken into account during deletion by means of a spatial safety zone around the representation of the second vessel in the forward projection images, in particular a safety margin. Since the second vessel can be blurry in the initial 3D reconstruction due to movement, it is correspondingly also blurry in the two-dimensional forward projection. This blurriness should be considered when deleting the second vessel. Therefore, it is advantageous if, during deletion, the spatial safety zone, i.e., for example, a predetermined area, around the center line of a vessel is deleted. Deletion can be performed manually or automatically, for example, using a suitable algorithm.This spatial safety area ensures that the largest possible proportion of pixels from the secondary vessel is deleted, taking into account a typical range of motion of the secondary vessel.

[0022] In an advantageous embodiment, the deletion of the secondary vessel is performed using an inpainting algorithm. The secondary vessel, or an area around it, is thereby painted over. The inpainting algorithm is thus not used to fill a gap, but rather to deliberately overpaint a detected vessel area. Typically, the area to be filled (here, the secondary vessel area) is filled with pixels whose brightness is matched to that of the surrounding area. This would completely delete the secondary vessel area, including the secondary vessel itself, unless not all pixels of the secondary vessel area were filled using the inpainting method.

[0023] According to an alternative embodiment, the deletion of the second vessel is achieved by providing, for each of the forward projection images, two projection images in the same state of motion but at different projection angles. Vessel structures of the second vessel are extracted from these projection images, and the second vessel is deleted in the respective forward projection image if the extracted vessel structures and the second vessel in that projection image satisfy a consistency condition. Vessel structures can thus be deleted iteratively (e.g., by inpainting or (completely) reducing contrast) if, in different phases of motion, one or more consistency conditions are met with respect to vessel structures of the second vessel that were extracted from other projection images.In a specific case, at least two projection images with the same motion state and a projection angle difference of at least 20 degrees can be obtained for each image (see Preuhs, A. et al. (2018)). These two images can be used to iteratively delete vascular structures that are not included in the projection by extracting them into a new projection image. The goal of the optimization is to maximize the consistency of the newly generated projection image (i.e., the extracted vessels). Preferably, as with the inpainting method, a set of projection images without secondary vessels, and in particular without images of secondary vessels, can be obtained. In the case of tumor embolization, the set of projection images thus no longer contains any non-feeder vessels, and in particular, no images of non-feeder vessels.

[0024] In another embodiment, it is provided that non-rigid movements are compensated during motion compensation. This means, in particular, that location-dependent movements within the 3D reconstruction space can also be compensated.

[0025] Movement compensation is therefore able to eliminate or balance complex movement fields.

[0026] According to another embodiment, the target vessel is an efflux or inflow vessel for a specific region of an object, and the secondary vessel is accordingly a non-efflux or non-inflow vessel for that specific region. For example, the target vessel is an inflow vessel (feeder vessel) to a tumor, while the secondary vessel does not supply the tumor. In the case of a secretory organ, the target vessel can be an efflux vessel and the secondary vessel a non-efflux vessel. For instance, the efflux vessels of the liver and pancreas empty into a common main duct that leads into the small intestine. The pancreatic duct from the pancreas would, in this case, be a non-efflux vessel of the liver.

[0027] Such distinctions between inlet and non-inlet vessels or outlet and non-outlet vessels can, in principle, be made at any vessel branching, because the inlet and outlet always occur there from different directions or regions.

[0028] The above problem is also solved according to the invention by an image processing device for motion-compensated reconstruction of a target vessel with - a storage device for providing an initial 3D reconstruction of a volume with the target vessel, and - a computing facility that is trained to * Identifying the target vessel and a secondary vessel distinct from the target vessel in the initial 3D reconstruction, where the target vessel has a higher contrast (relative to its surroundings) than the secondary vessel, * Forward projection of the target vessel and the secondary vessel onto forward projection images, * Deleting (reducing the contrast of) the second vessel in the forward projection images, * Creating a secondary 3D reconstruction from the forward projection images (without the secondary vessels) with automatic motion compensation.

[0029] The storage device can contain one or more storage elements or modules. It may also have its own processor. The storage device can be implemented locally or distributed across the internet.

[0030] The computing device can be a computer. It typically has one or more processors and possibly its own memory elements. Furthermore, it usually has input and output interfaces.

[0031] The advantages and embodiments described above in connection with the method according to the invention can also be applied analogously to the image processing device according to the invention. Accordingly, the aforementioned method features constitute functional features of the image processing device.

[0032] Advantageously, a computed tomography (CT) scanner can also be provided with such an image processing device. The image processing device processes signals acquired from a detector of the computed tomography scanner.

[0033] The advantages and embodiments described above in connection with the inventive method and / or the inventive image processing device are also transferable, mutatis mutandis, to the inventive computed tomography device. Accordingly, the aforementioned method features constitute functional features of the computed tomography device.

[0034] Furthermore, a computer program can be provided that includes instructions which, when executed by the aforementioned image processing device, cause it to perform the procedure described above. Similarly, a computer-readable storage medium can be provided that includes instructions which, when executed by the aforementioned image processing device, cause it to perform the procedure described above.

[0035] The present invention will now be explained in more detail with reference to the accompanying drawings, which show: Fig. 1 a schematic representation of a computed tomography device; Fig. 2 a flowchart of an embodiment of a method according to the invention; Fig. 3 a layer image from the 3D reconstruction with identified target vessel and Fig. 4 an exemplary forward projection based on the 3D volume set from Fig. 3 with deleted secondary vessels.

[0036] The exemplary embodiments described in more detail below represent preferred embodiments of the present invention.

[0037] Cone beam computed tomography (CBCT) scanners and flat detector angiography systems are frequently used for interventional purposes, e.g. for intraoperative tracking of devices or the guidance of such devices.

[0038] The Fig. Figure 1 shows a schematic representation of a monoplane X-ray system as an example of a computed tomography device. The system includes a C-arm 2 held by a stand 1 in the form of a six-axis industrial or articulated robot. An X-ray source, for example an X-ray tube 3 with a collimator, and an X-ray image detector 4 (the image acquisition unit) are mounted at the ends of the C-arm. The implementation of the X-ray diagnostic device is not limited to the industrial robot; conventional C-arm devices can also be used.

[0039] In the beam path of the X-ray tube 3, a patient 6 or a technical object is positioned on a table 5 of a patient positioning table. A control unit 7 with a computer 8 for image processing is connected to the X-ray diagnostic device. This computer receives and processes the image signals from the X-ray image detector 4 (operating elements are not shown). The X-ray images can then be viewed on displays of a monitor 9. The monitor 9 can be supported by a ceiling-mounted, longitudinally movable, swiveling, rotating, and height-adjustable support system 10 with a boom and a lowerable support arm. The control unit 7 can also include a storage device 11 for the computer 8 to provide, for example, 3D reconstructions and / or 2D projection images.

[0040] A 3D reconstruction of a region of interest in patient 6 (generally referred to as the object) can be obtained from two-dimensional projection images. Extensive motion compensation is usually required. In one embodiment, the reconstruction space and the image space are combined by enforcing consistency between the two spaces. This is achieved by transforming the two spaces into a common space, for example, by backprojecting the projection images into the reconstruction space or vice versa. A motion field is then optimized using a quality metric. Ideally, the motion field reflects the true, non-rigid patient movement when the quality metric converges.

[0041] An improved strategy for motion compensation is proposed, according to which the target volume is reduced to a clinically relevant minimum, thereby improving the results of motion compensation. The input for this method is, for example, a stack of projection images with contrast-enhanced vascular structures. Based on this input, the proposed method can be divided into five steps, which are described below along with Fig. 2 will be described.

[0042] In the first step S1, an initial 3D reconstruction of a volume containing the target vessel is provided, for example, reconstructed from the stack of projection images. If necessary, the initial 3D reconstruction is also provided directly, and the reconstruction may have taken place at an earlier time.

[0043] In a second step, S2, the target vessel and (at least) one secondary vessel different from the target vessel are identified in the initial 3D reconstruction. Specifically, this allows for the identification of the feeding vessels in 3D space (i.e., the vessels of interest). Subsequently, the 3D vascular tree is divided into feeder and non-feeder vessels.

[0044] In a third step, S3, the target vessel and the secondary vessel are projected forward onto forward projection images in the two-dimensional projection space. If necessary, the non-feeder and feeder vessels are identified again in the projection space. For moving vessels, the respective forward projection takes motion blur into account. Therefore, a certain spatial safety margin should be considered during the forward projection.

[0045] In a fourth step, S4, the second vessel is erased from the forward projection images (e.g., by inpainting or (completely) reducing the contrast). For this purpose, the forward-projected vessel margins can be used for an inpainting step. This can be achieved using standard inpainting algorithms, such as those described by Unberath et al. Alternatively, consistency conditions, as described by Preuhs et al. (2020), can be applied. In the latter case, at least two projection images with the same motion state and a projection angle difference of at least 20 degrees must be available for each image (see Preuhs et al. (2018)). The two images are then used to iteratively erase the vessel structures that are not included in the projection by extracting them into a new projection image.The goal of optimization is to maximize the consistency of the newly generated projection image (i.e., the extracted vessels). Both the standard inpainting method and the consistency-based method result in a projection image stack in which all non-feeder vessels have been removed.

[0046] In a fifth step, S5, a secondary 3D reconstruction is created from the forward projection images (excluding the secondary vessels or non-feeder vessels) using automatic motion compensation. Specifically, a motion-compensated reconstruction of the projection stack can be performed, where only the feeder vessels are present as high-contrast structures, and the non-feeder vessels are suppressed (e.g., painted over). The motion compensation used is, for example, a non-rigid motion compensation scheme that utilizes high-contrast structures (see Rohkohl, C. et al.).

[0047] Fig. Figure 3 shows one of many slice images obtained from the initial 3D reconstruction. The image shows a tumor 12 and a feeder vessel or target vessel 13. The target vessel 13 was already identified and, if necessary, marked in the initial 3D reconstruction. This identification or marking is preferably retained in the forward projection. Furthermore, the slice image, which corresponds to a plane from the initial 3D reconstruction, shows sections of many other vessels that are non-feeder vessels and are referred to here as secondary vessels 14. These secondary vessels 14, or secondary vessel sections, can be easily eliminated or deleted in a forward projection image. The secondary vessel sections can, for example, be easily detected or segmented using an image processing algorithm. If necessary, not only the respective secondary vessel section is deleted, but also a safety zone around this section.This ensures that any motion artifacts of the second vessel are also eliminated.

[0048] As mentioned, deletion can be achieved through inpainting. For this, the area of ​​the secondary vessel to be deleted is replaced, for example, by pixels whose brightness matches that of the pixels in the vicinity of the secondary vessel area.

[0049] Fig. Figure 4 now shows a forward projection of a second 3D reconstruction, which is derived from the layer image according to Fig. 3 and many other layer images were created. It shows a vascular system 15 in the volume surrounding the tumor 12. In this spatial representation, a target vessel 13 and a non-feeder vessel (secondary vessel) 14 are shown. The target vessel 13 was identified as the feeder vessel for the tumor 12.

[0050] There are therefore both feeder and non-feeder vessels. After deleting the non-feeder vessels, the motion compensation algorithm can concentrate on the target vessel 13, which generally results in better motion compensation. A second 3D reconstruction thus shows the target vessel 13, usually more sharply than the first 3D reconstruction. This would, for example, make it easier for the physician to insert the catheter into the necessary feeder vessel to embolize tumor 12.

[0051] Advantageously, the optimization of a non-rigid motion estimation can thus be simplified by simplifying the cost function. Normally, all high-contrast features are included in the optimization algorithm. High contrast is defined by a specific percentile of the highest intensities in the reconstructed image. In contrast, according to the invention, the high-contrast image features are restricted by removing those vessels that are not included in the reconstruction. This simplifies the optimization problem and prevents the optimization from drifting, for example, to a local minimum. Furthermore, it ensures that the clinically relevant features are preserved.

Claims

[1] Method for motion-compensated reconstruction of a target vessel (13) by - Providing (S1) an initial 3D reconstruction of a volume with the target vessel (13), - Identifying (S2) an image of the target vessel (13) and an image of a secondary vessel (14) different from the target vessel in the first 3D reconstruction, - Forward projection (S3) of the target vessel image (13) and the secondary vessel image (14) onto forward projection images, - Deleting (S4) the image of the second vessel (14) in the forward projection images, - Creating (S5) a secondary 3D reconstruction from the forward projection images with automatic motion compensation. [2] Method according to claim 1, wherein during forward projection (S3) a motion blur of the second vessel is created, which is taken into account during deletion (S4) with a spatial safety area around the image of the second vessel in the forward projection images. [3] Method according to one of the preceding claims, wherein the deletion (S4) of the image of the second vessel is carried out using an inpainting algorithm. [4] Method according to claim 1 or 2, wherein the deletion (S4) of the image of the second vessel (14) is carried out by providing, for each of the forward projection images, two projection images in the same state of motion at different projection angles, from which images of vessel structures of the second vessel (14) are extracted, and the image of the second vessel (14) in the respective forward projection image is deleted if the extracted images of the vessel structures and the image of the second vessel (14) in the respective projection image satisfy a consistency condition. [5] Method according to claim 4, wherein the image of the second vessel (14) in the respective forward projection image is iteratively deleted by stepwise changing the contrast of the second vessel in the projection images until the consistency condition is met. [6] Method according to one of the preceding claims, wherein non-rigid movements are compensated in the motion compensation. [7] Method according to any one of the preceding claims, wherein the target vessel (13) is an outflow vessel for a specific region of an object (6) and the secondary vessel (14) is an inflow vessel for the specific region or wherein the target vessel (13) is an inlet vessel for a specific region of an object (6) and the secondary vessel (14) is an outlet vessel for the specific region. [8] Image processing device for motion-compensated reconstruction of a target vessel (13) with - a storage device for providing an initial 3D reconstruction of a volume with the target vessel (13), and - a computing facility that is trained to • Identifying an image of the target vessel (13) and an image of a secondary vessel (14) different from the target vessel (13) in the initial 3D reconstruction, • Forward projection of the target vessel image (13) and the secondary vessel image (14) onto forward projection images, • Deleting the image of the second vessel (14) in the forward projection images, • Creating a secondary 3D reconstruction from the forward projection images with automatic motion compensation. [9] Computed tomography device with an image processing device according to claim 8. [10] Computer program comprising instructions which, when the program is executed by an image processing device according to claim 8, cause the device to execute the method according to any one of claims 1 to 7. [11] Computer-readable storage medium comprising instructions which, when the program is executed by an image processing device according to claim 8, cause it to execute the method according to any one of claims 1 to 7.

Images