Target tube reconstruction for non-rigid motion compensation

Abstract

The invention relates to a method for motion-compensated reconstruction of a target tube (13), comprising: providing (S1) a first 3D reconstruction of a volume having the target tube (13); identifying (S2) a mapping of a target tube (13) and a mapping of a second tube (14) different from the target tube in the first 3D reconstruction; forward-projecting (S3) the mapping of the target tube (13) and the mapping of the second tube (14) onto the forward-projected image; deleting (S4) the mapping of the second tube (14) in the forward projection image; a second 3D reconstruction is created (S5) from the forward projection image with automatic motion compensation. The invention further relates to an image processing device for motion-compensated reconstruction as well as to a computed tomography device, to a computer program product and to a computer-readable storage medium.

Description

Technical Field

[0001] This invention relates to a method for motion-compensated reconstruction of a target tube. Furthermore, this invention relates to an image processing apparatus for motion-compensated reconstruction, a computed tomography apparatus, a computer program, and a computer-readable storage medium. Background Technology

[0002] In this patent application, the use of personal names and pronouns does not typically specify a particular gender.

[0003] In interventional oncology, particularly in tumor embolization, it is essential to block the blood vessels supplying cancerous tissue to induce tumor necrosis. Furthermore, the healthy tissue affected by the blockage should be minimized to prevent necrosis of healthy tissue. For this purpose, contrast-enhanced 3D reconstruction can be used. 3D reconstruction allows for the identification of the specific vessels that need to be blocked, the so-called supply vessels (inflow vessels).

[0004] To achieve the above effects, the 3D reconstruction of contrast-enhanced tubes must be as accurate as possible. A major obstacle to high-quality tube reconstruction is non-rigid motion artifacts within organs, such as motion artifacts. Rigid motion, for example, involves displacement of the entire organ, while non-rigid motion, such as movement within the organ, is position-dependent.

[0005] For extracting tubes from high-contrast angiography images, high-precision motion compensation is desired.

[0006] A common solution is motion compensation reconstruction, as described in the article "Interventional 4D motion estimation and reconstruction of cardiac vasculature without motion periodicity assumption" by Rohkohl, C., Lauritsch, G., Biller, L., Prümmer, M., Boese, J., & Hornegger, J. published in the journal Medical imageanalysis (2010, Vol. 14, No. 5, pp. 687-694).

[0007] However, due to the complexity of 3D motion fields, these algorithms often fail in the case of smaller tubular structures.

[0008] Typically, motion compensation methods attempt to maximize an image optimization criterion based on a) projection space, b) reconstruction space, or c) a combination of both. A common problem in these optimization algorithms is that different anatomical regions yield different optimal solutions. For example, bone structures may not exhibit significant motion, while soft tissue structures may. Motion compensation may result in blurred bone structures; or soft tissue regions may not be well compensated because the optimization algorithm focuses on the bone region. Common methods to mitigate these problems include defining a target volume for optimization or removing bone.

[0009] In their 2016 paper "Virtual single-frame subtraction imaging" published in the proceedings of the Image Formation in X-Ray CT conference (pp. 89-92), Unberath et al. outlined a general framework for single-frame material decomposition in the detector region. This method includes segmentation and background estimation steps that provide a virtual mask image that can be used for subtraction. In many cases, material decomposition yields non-isolated difference images, thus allowing the use of novel motion estimation methods that leverage epipolar consistency conditions.

[0010] Furthermore, in the article "Deep autofocus with cone-beam CT consistency constraint" published in 2020 by Preuhs, A., Manhart, M., Roser, P., Stimpel, B., Syben, C., Psychogios, M., ... & Maier, A in *Bildverarbeitung für die Medizin2020: Algorithms–Systeme–Anwendungen. Proceedings des Workshops vom 15. bis17. März 2020 in Berlin* (Springer Fachmedien Wiesbaden, pp. 169-174), a novel learning-based method is proposed that can compensate for motion within the acquisition plane. A CBCT consistency constraint is introduced, which has been proven reliable for motion perpendicular to the acquisition plane. Therefore, both in-plane and out-of-plane motion can be clearly identified, with an average artifact suppression rate of 93%.

[0011] According to the article "Viewpoint planning for quantitative coronary angiography" published in the *International Journal of Computer Assisted Radiology and Surgery* (Vol. 13, pp. 1159-1167) in 2018 by Preuhs, A., Berger, M., Bauer, S., Redel, T., Unberath, M., Achenbach, S., & Maier, A., in coronary angiography, myocardial perfusion status is assessed by analyzing the two-dimensional X-ray projections of the coronary arteries forming a contrast. This is accomplished using a movable C-arm system. Due to the inherent dimensionality reduction of X-rays when projecting a 3D scene onto a 2D image, the viewpoint is crucial for ensuring adequate observation of the examined arteries for reliable diagnosis. To address this, an algorithm is proposed that can calculate the optimal viewpoint for assessing the coronary arteries without requiring a 3D model.

[0012] Optimal viewpoint planning is based solely on a single angiographic X-ray image. Summary of the Invention

[0013] The technical problem to be solved by the present invention is to improve the reconstruction of a target tube subjected to non-rigid motion.

[0014] According to the present invention, this technical problem is solved by the method, image processing apparatus, and computed tomography apparatus according to the invention. Furthermore, a computer program and a computer-readable storage medium with reference to the method are provided. Advantageous extensions of the invention are derived from the description.

[0015] Therefore, according to the present invention, a method for motion-compensated reconstruction of a target tube is provided. The target tube is, for example, a so-called feeder tube through which blood is supplied to the tumor. However, the target tube can also be a bile duct, etc. The target tube should be reconstructed with minimal motion artifacts.

[0016] To this end, a first 3D reconstruction having the volume of the target tube is first provided. The term "provide" can be understood, for example, as receiving, reading, detecting, etc., such as detecting and / or reading from a computer-readable data storage device, and / or receiving from a data storage unit, particularly a database. This provision makes the data for the first 3D reconstruction available to the method. Thus, the target tube is spatially represented in the first 3D reconstruction. Typically, this first 3D reconstruction is affected by motion artifacts. Based on this affected 3D reconstruction, inserting a conduit into the target tube, for example, would be difficult.

[0017] Therefore, in a further step, the mapping of the target tube and the mapping of a second tube different from the target tube are identified in the first 3D reconstruction (the terms "target tube" and "second tube" below should always be understood as "the mapping of the target tube" or "the mapping of the second tube" unless explicitly stated otherwise or required by the context). Of course, in addition to the second tube, there may be many other second tubes in the first 3D reconstruction. All these second tubes will generally interfere with the automatic motion compensation algorithm. Therefore, the goal is to remove as many second tubes as possible from the volume of the first 3D reconstruction so that motion compensation can be focused on the target tube. To this end, the target tube is identified, for example, manually by labeling or automatically, for example, by image processing. Identifying the mapping of the target tube and the mapping of the second tube different from the target tube may include annotation and / or segmentation of the corresponding image points of the first 3D reconstruction, with reference in particular to thresholding and / or temporal intensity curves. The image points of the first 3D reconstruction may, for example, be formed as pixels and / or voxels and have respective image values ​​representing the volume containing the target tube and the second tube.

[0018] Now, in a further step, the target tube and the second tube are forward-projected onto the forward-projected image. Forward projection can virtually map, and represent, the 3D object onto a two-dimensional plane. Depending on the projection direction, different forward-projected images of the 3D object can be obtained. The forward-projected image is a 2D image in the projected image space. Typically, all other different second tubes are also forward-projected along with the second tube. The purpose of this forward projection is that it is easier to remove the second tube in the two-dimensional projected image space compared to in 3D reconstruction. The term "removing the second tube" should be understood as reducing the influence of the second tube in the corresponding image to a predetermined limit and, if necessary, minimizing it. However, this also means that, for example, the influence of other organs on the corresponding pixels is usually not reduced.

[0019] Therefore, in a further step of the method, the second tube is removed from the forward-projected image. Of course, all other second tubes can also be removed here, if possible. Removal is achieved, for example, by reducing contrast or by coating one or more second tubes. The removal of the second tubes does not need to be complete, but should be largely completed. For example, the contrast of the second tubes relative to the surrounding environment should be reduced to at least one-quarter.

[0020] Therefore, the forward-projected image no longer contains the second tube, or at most contains it as a shadow. In subsequent steps, a second 3D reconstruction is finally created from the forward-projected image with automatic motion compensation. Thus, the second 3D reconstruction is reconstructed from a two-dimensional forward-projected image with little or no second tube. Automatic motion compensation is performed during the 3D reconstruction process, where the compensation algorithm can focus on the target tube. Therefore, optimal motion compensation can be achieved for the target tube. The resulting second 3D reconstruction thus shows the target tube, where motion artifacts are generally reduced to a greater extent than in the first 3D reconstruction.

[0021] In one embodiment, motion blur of the second tube in the corresponding forward-projected image is generated during forward projection. This motion blur is taken into account during deletion via a spatial safety region, particularly a safety margin, surrounding the mapping of the second tube in the forward-projected image. Since the second tube may be blurred due to motion in the first 3D reconstruction, it is correspondingly blurred in the two-dimensional forward projection. This blur should be taken into account when deleting the second tube. Therefore, it is advantageous to delete a spatial safety region, such as a predetermined region, around the tube's centerline during deletion. Deletion can be performed manually or automatically, for example, using a suitable algorithm. This spatial safety region ensures that, considering the typical scale of motion of the second tube, as large a portion of the image points of the second tube as possible are deleted.

[0022] In an advantageous embodiment, an inpainting algorithm is used to remove the second tube. Here, the second tube or the area surrounding it is inpainted. The inpainting algorithm is not used to fill gaps, but rather to intentionally cover the identified tube area. Typically, the area to be filled (here, the second tube area) is filled with pixels whose brightness adapts to the surrounding environment of the second tube area. In this way, the second tube area, including the second tube itself, will be completely removed, unless not all pixels in the second tube area are filled using the inpainting method.

[0023] According to an alternative embodiment, the removal of the second tube is achieved by providing two projection images of the same motion state at different projection angles for each forward projection image, extracting the tube structure of the second tube from these two projection images, wherein the second tube in the corresponding forward projection image is removed if the extracted tube structure and the second tube in the corresponding projection image satisfy a consistency condition. Therefore, if one or more consistency conditions regarding the tube structure obtained from other projection images regarding the second tube are satisfied at different motion stages, the tube structure can be iteratively removed (e.g., by inpainting or (completely) reducing contrast). In a specific case, at least two projection images with the same motion state and a projection angle difference of at least 20 degrees can be obtained for each image (see Preuhs, A et al. (2018)). These two images can be used to iteratively remove tube structures that do not flow to the projection by extracting these tube structures into a new projection image. The optimization objective is to maximize the consistency of the newly generated projection images (i.e., the extracted tubes). Preferably, similar to the repair method, a set of projection images without the second tube, and especially without mappings of the second tube, can be obtained. In the case of tumor embolization, this set of projection images no longer includes mappings of non-supply tubes, especially non-supply tubes.

[0024] In another embodiment, it is specified that non-rigid motion is compensated for in motion compensation. This specifically means that position-dependent motion within the 3D reconstructed space can also be compensated. Therefore, motion compensation can eliminate or compensate for complex motion fields.

[0025] According to another embodiment, the target tube is an outflow or inflow tube to a specific region of the object, and the second tube is correspondingly a non-outflow or non-inflow tube to that specific region. For example, the target tube is an inflow tube (supply tube) to the tumor, while the second tube does not supply the tumor. In the case of secretory organs, the target tube can be an outflow tube, and the second tube can be a non-outflow tube. For example, the outflow tubes of the liver and pancreas merge into a common main conduit that leads to the small intestine. In this case, the pancreatic duct from the pancreas would be a non-outflow tube from the liver.

[0026] This distinction between inflow and non-inflow pipes or outflow and non-outflow pipes can, in principle, be made at any pipe branch, because there, inflow or outflow always faces or comes from different directions or areas.

[0027] The aforementioned technical problem is also solved by an image processing device for motion compensation reconstruction of a target tube, the image processing device having:

[0028] - A storage device for providing a first 3D reconstruction with the volume of the target tube, and

[0029] - Computing device, designed for

[0030] * Identify the target tube and a second tube different from the target tube in the first 3D reconstruction, wherein the target tube has higher contrast (relative to its surroundings) than the second tube.

[0031] * Project the target tube and the second tube forward onto the forward projection image.

[0032] *Remove (reduce contrast) the second tube in the forward-projected image.

[0033] * Create a second 3D reconstruction from a forward-projected image (without a second tube) with automatic motion compensation.

[0034] A storage device may have one or more storage elements or storage modules. If necessary, the storage device may also have its own processor. Storage devices can be implemented locally or distributed across the Internet.

[0035] A computing device can be implemented using a computer. Typically, a computing device has one or more processors and, if necessary, its own storage elements. Furthermore, a computing device usually has input interfaces and output interfaces.

[0036] The advantages and variations of embodiments described above in conjunction with the method according to the invention are similarly applicable to the image processing apparatus according to the invention. Therefore, the method features constitute the functional features of the image processing apparatus.

[0037] Advantageously, a computed tomography (CT) device having such an image processing apparatus can also be provided. Here, the image processing apparatus processes signals obtained from the detectors of the computed tomography device.

[0038] The advantages and variations of embodiments described above in conjunction with the method and / or image processing apparatus according to the invention are similarly applicable to the computed tomography apparatus according to the invention. Therefore, the method features constitute the functional features of the computed tomography apparatus.

[0039] In addition, a computer program comprising instructions that, when executed by the image processing device, cause the image processing device to perform the methods described above. Similarly, a computer-readable storage medium comprising instructions that, when executed by the image processing device, cause the image processing device to perform the methods described above. Attached Figure Description

[0040] The invention is further illustrated herein with the aid of the accompanying drawings, in which:

[0041] Figure 1 A schematic diagram of a computed tomography (CT) device is shown.

[0042] Figure 2 A flowchart illustrating an embodiment of the method according to the present invention is shown;

[0043] Figure 3 The image shows a layered image from the 3D reconstruction, with the identified target tube; and

[0044] Figure 4 Showing based on Figure 3 The 3D volume set in the example has had its exemplary forward projection removed from the second tube. Detailed Implementation

[0045] The embodiments described in more detail below represent preferred embodiments of the present invention.

[0046] Cone-beam computed tomography (CBCT) and flat-panel detector angiography systems are commonly used for interventional purposes, such as for intraoperative instrument tracking or guidance.

[0047] Figure 1 A schematic diagram of a single-plane X-ray system is shown as an example of a computed tomography (CT) device. This system has a C-arm 2 held by a support 1 (in the form of a six-axis industrial or articulated robot). At the end of the C-arm is mounted an X-ray radiation source, such as an X-ray radiator 3 with an X-ray tube and collimator, and an X-ray image detector 4 as an image acquisition unit. The implementation of this X-ray diagnostic device does not rely on an industrial robot. Conventional C-arm devices can also be used.

[0048] In the radiation path of the X-ray radiator 3, a patient 6 or a technical object to be examined is placed on the table 5 of the patient support table. A system control unit 7 is connected to the X-ray diagnostic equipment. This system control unit has a computing device 8 for image processing, which receives and processes image signals (operating elements, for example, not shown) from the X-ray image detector 4. The X-ray image can then be viewed on the monitor assembly 9. The monitor assembly 9 can be held in place by a ceiling-mounted, longitudinally movable, swingable, rotatable, and height-adjustable support system 10 with cantilevered and lowerable support arms. The system control unit 7 may also have a storage device 11 for the computing device 8 to provide, for example, 3D reconstruction and / or 2D projection images.

[0049] A 3D reconstruction of the region of interest (ROI) of patient 6 (generally, the object) can be obtained from a 2D projection image. Large-scale motion compensation is typically required. In one embodiment, this is done by combining the reconstruction space and the image space in a way that enforces consistency between the two spaces. This is achieved by transforming the two spaces into a common space, for example, by backprojecting the projection image onto the reconstruction space, or by projecting the reconstruction space onto the projection image. The motion field is then optimized based on a quality metric. Ideally, when the quality metric converges, the motion field reflects realistic, non-rigid patient motion.

[0050] An improved motion compensation strategy is proposed here, which limits the target volume to a clinically relevant minimum, thereby improving the outcome of motion compensation. The input to this method is, for example, a set of projection images of tubular structures with contrast enhancement. Based on this input, the proposed method can be divided into five steps, which are described below. Figure 2 Describe these steps.

[0051] In the first step S1, a first 3D reconstruction of the volume of the target tube is provided, which is, for example, reconstructed from a stack of projected images. Alternatively, the first 3D reconstruction can be provided directly if necessary, and this reconstruction may have been performed at an earlier point in time.

[0052] In the second step S2, the target tube and (at least) a second tube different from the target tube are identified in the first 3D reconstruction. In particular, the input tube (i.e., the tube of interest) can be identified in 3D space. Subsequently, the 3D tube tree is divided into supply tubes and non-supply tubes.

[0053] In the third step S3, the target tube and the second tube are forward-projected onto the forward-projected image in the two-dimensional projection image space. If necessary, the non-supply tube and the supply tube are re-identified in the projection image space. For moving tubes, motion blur is considered in the corresponding forward projection. Therefore, a certain spatial safety zone should be considered during forward projection.

[0054] In step S4, the second tube is removed from the forward-projected image (e.g., by inpainting or (completely) reducing contrast). For this, the edges of the tubes in the forward projection can be used as an inpainting step, for example. This can be achieved using conventional inpainting algorithms, as described by Unberath et al. Alternatively, the consistency condition described by Preuhs, A. et al. (2020) can be applied. In the latter case, for each image, there must be at least two projected images with the same motion state and a projection angle difference of at least 20 degrees (see Preuhs, A. et al. (2018)). These two images are then used to iteratively remove tube structures that do not flow to the projection, by extracting these tube structures into the new projected image. The optimization objective is to maximize the consistency of the newly generated projected image (i.e., the extracted tubes). Both standard inpainting methods and consistency-based methods produce a stack of projected images in which all non-feeding tubes are removed.

[0055] In step S5, a second 3D reconstruction is created from the forward-projected image (without a second tube or non-supply tube) with automatic motion compensation. Specifically, motion-compensated reconstruction of the projection stack can be performed, where only the supply tube exists as a high-contrast structure, while the non-supply tubes are suppressed (e.g., coated). The motion compensation used is, for example, a non-rigid motion compensation scheme using a high-contrast structure (see Rohkohl, C. et al.).

[0056] Figure 3 One of the multiple layer images obtained from the first 3D reconstruction is shown. This mapping shows the tumor 12 and the supply tube or target tube 13. The target tube 13 has been identified and, if necessary, marked in the first 3D reconstruction. Such identification or marking is preferred in the forward projection. Furthermore, this layer image (corresponding to a plane in the first 3D reconstruction) shows segments of numerous other tubes, which are non-supply tubes and are referred to herein as second tubes 14. These second tubes 14 or second tube segments can be effectively eliminated or removed in the forward projection image. Second tube segments can be easily identified or segmented, for example, using image processing algorithms. If necessary, not only the corresponding second tube segment is removed, but also the safe region surrounding that segment is removed. This ensures that any motion artifacts of the second tubes are also eliminated.

[0057] As previously mentioned, the deletion can be performed through inpainting. For this purpose, the second tube region to be deleted is replaced, for example, with pixels whose brightness corresponds to the brightness of pixels in the surrounding environment of the second tube region.

[0058] Figure 4 The forward projection of the second 3D reconstruction is now shown, which is derived from... Figure 3The image was created using layer images and numerous other layer images. This figure shows the ductal system 15 within the volume surrounding tumor 12. The target tube 13 and the non-supply tube (second tube) 14 are shown in this spatial representation. Target tube 13 has been identified as the supply tube for tumor 12.

[0059] Therefore, both supply and non-supply tubes exist. After removing the non-supply tubes, the motion compensation algorithm can focus on the target tube 13, thus generally achieving better motion compensation. Therefore, the target tube 13 shown in the second 3D reconstruction is generally clearer than in the first 3D reconstruction. This makes it easier for physicians, for example, to introduce the necessary supply tubes to embolize the tumor 12.

[0060] Therefore, the optimization of non-rigid motion estimation can be advantageously simplified by simplifying the cost function. Typically, all high-contrast features are part of the optimization algorithm. High contrast is defined by a certain percentile of the highest intensity in the reconstructed image. In contrast, according to the invention, high-contrast image features are limited by removing tubes that do not participate in the reconstruction. This simplifies the optimization problem and prevents the optimization from getting trapped in local minima, for example. Furthermore, it ensures that clinically relevant features are preserved.

Claims

1. A method for motion compensation reconstruction of a target tube (13), comprising: - Provide (S1) a first 3D reconstruction of the volume of the target tube (13), - Identify (S2) the mapping of the target tube (13) and the mapping of the second tube (14) that is different from the target tube in the first 3D reconstruction. - Project the mapping of the target tube (13) and the mapping of the second tube (14) forward (S3) onto the forward projection image. -Delete the mapping of the second tube (14) in the forward projection image (S4), - Create a second 3D reconstruction from the forward-projected image with automatic motion compensation (S5).

2. The method according to claim 1, wherein, Motion blur of the second tube is generated during forward projection (S3), and this motion blur is taken into account during deletion (S4) by the spatial safety area around the mapping of the second tube in the forward projection image.

3. The method according to any one of the preceding claims, wherein, The mapping of the second tube is deleted using the repair algorithm (S4).

4. The method according to claim 1 or 2, wherein, The deletion of the mapping of the second tube (14) (S4) is achieved by providing two projection images of the same motion state at different projection angles for each forward projection image, extracting the mapping of the tube structure of the second tube (14) from the two projection images, and deleting the mapping of the second tube (14) in the corresponding forward projection image if the extracted mapping of the tube structure and the mapping of the second tube (14) in the corresponding projection image satisfy the consistency condition.

5. The method according to claim 4, wherein, By gradually changing the contrast of the second tube in the projected image, the mapping of the second tube (14) in the corresponding forward projected image is iteratively deleted until the consistency condition is met.

6. The method according to any one of the preceding claims, wherein, Compensating for non-rigid motion in motion compensation.

7. The method according to any one of the preceding claims, in, The target pipe (13) is the outflow pipe of a specific area of ​​the object (6), and the second pipe (14) is the inflow pipe of that specific area, or The target pipe (13) is the inflow pipe of a specific area of ​​the object (6), and the second pipe (14) is the outflow pipe of that specific area.

8. An image processing apparatus for motion compensation reconstruction of a target tube (13), comprising: - Storage device for providing a first 3D reconstruction with the volume of the target tube (13), and - Computing device, designed for • Identify the mapping of the target tube (13) in the first 3D reconstruction and the mapping of the second tube (14) that is different from the target tube (13). • Project the mapping of the target tube (13) and the mapping of the second tube (14) forward onto the forward projection image. • Remove the mapping of the second tube (14) from the forward projection image. • Create a second 3D reconstruction from the forward-projected image with automatic motion compensation.

9. A computed tomography apparatus having the image processing apparatus according to claim 8.

10. A computer program product comprising instructions that, when executed by an image processing apparatus according to claim 8, cause the image processing apparatus to perform the method according to any one of claims 1 to 7.

11. A computer-readable storage medium comprising instructions that, when executed by an image processing apparatus according to claim 8, cause the image processing apparatus to perform the method according to any one of claims 1 to 7.

Images