Reconstruction method, data processing unit and medical imaging device

Abstract

For a particularly simple 3D reconstruction, a method for reconstructing a 3D volume of a moving examination object is provided, comprising the following steps: providing a data set with a plurality of projection images which was generated during a rotation run of a recording system of an X-ray device around the examination object, selecting at least a first projection image of the data set, providing a generalized 3D model of the examination object, generating a personalized 3D model of the examination object from the generalized 3D model using the first projection image, selecting a second projection image of the data set, registering the second projection image with the personalized 3D model of the examination object, and reconstructing a 3D volume of the examination object on the basis of the first and at least one further registered projection image.

Description

[0001] The invention relates to a method for reconstructing a 3D volume of a moving examination object according to patent claim 1, a data processing unit for carrying out such a method according to patent claim 12 and a medical imaging device according to patent claim 13.

[0002] To obtain 3D volumetric images of an examination subject—a patient or a patient's body part or organ—a multitude of projection images are generated while an X-ray imaging system rotates around the examination subject. So-called cone beam CT (CBCT) or DynaCT images can then be reconstructed into a 3D volumetric image. Due to the relatively long acquisition time of such a rotation scan, patient movement, e.g., due to breathing movements, heartbeat, and unconscious movement, is a major problem, particularly in imaging of the spine, internal organs, and neurological images, and severely impairs the quality of the image.

[0003] Motion compensation or motion correction is often performed to compensate for patient movement. This generally involves an initial image reconstruction using well-known reconstruction approaches, followed by image-based motion compensation via backprojection or re-registration. However, this approach often fails due to excessive patient movement during the acquisition process. Initializing the motion-compensated image reconstruction is particularly difficult in cases of significant patient movement.

[0004] The website https: / / www.cs.cornell.edu / projects / bigsfm / describes an approach to create a 3D model using machine learning methods based on a large number of images of an object or a building (solving the global SfM translations problem).

[0005] From the article by Amirhossein Saedpanah et al, “Geometrical Self-Calibration of CBCT Systems”, 12th Conference on Industrial Computed Tomography, Fürth, Germany (iCT 2023), it is known to perform object-related self-calibrations of CBCT images.

[0006] It is an object of the present invention to provide a method for reconstructing a moving examination object, which enables stable motion compensation even in the case of strong movements. Furthermore, it is an object of the invention to provide a device suitable for carrying out the method.

[0007] The object is achieved according to the invention by a method for reconstructing a 3D volume of a moving examination object according to patent claim 1, a processing and calculation unit for carrying out such a method according to patent claim 12 and a medical imaging device according to patent claim 13. Advantageous embodiments of the invention are each the subject of the associated subclaims.

[0008] The method according to the invention for reconstructing a 3D volume of a moving examination object comprises the following steps: providing a data set with a plurality of projection images, which was generated during a rotation run of a recording system of an X-ray device around the examination object, selecting at least a first projection image of the data set, providing a generalized 3D model of the examination object, generating a personalized 3D model of the examination object from the generalized 3D model using the first projection image, selecting a second projection image of the data set, registering the second projection image with the personalized 3D model of the examination object, and reconstructing a 3D volume of the examination object on the basis of the first and at least one further registered projection image, thus in particular the first,of the second and possibly further projection images.,

[0009] The invention focuses primarily on stabilizing the initialization of a 3D reconstruction. This does not necessarily require the initial motion correction to be very precise. By using a generalized 3D model of the examination subject, which is then personalized using an initial projection image, a basis for further optimization can be created, ensuring the fundamental success of reconstructing the 3D volume of the examination subject.

[0010] In the article by Karthik Shetty, Annette Birkhold, et al, “HOOREX: Higher Order Optimizers for 3D Recovery from X-Ray Images”, Workshop on Machine Learning for Multimodal Healthcare Data at conference, Saturday, July 29, Hawaii, USA, a 3D digital twin model of a patient’s vertebrae is generated using a single 2D projection image of the patient’s vertebrae. This is an adaptive approach in which a neural network is trained to generate a skeletal model by fusing a regression-based method with an optimization-based method. First, a regression method is used as a priori knowledge learned by a network due to its ease of use. Subsequently, the corresponding reprojection error is iteratively optimized for a better 2D fit.

[0011] The basic idea of ​​the invention is to use an identical or similar approach with a generalized 3D model and a first projection image, from which a personalized 3D model is generated, e.g., using a trained function, as a first draft for a reconstruction. Subsequently, the movements on additional projection images are corrected by registering at least one or more additional projection images. The projection images, corrected for motion by registration, can then be reconstructed into a 3D volume image. This makes it possible to guarantee a stable reconstruction method even in the case of significant movements of the examination subject, which can then be optimized. This ensures improved diagnosis and treatment for the patient.

[0012] The method is suitable not only for moving examination subjects, but also for poorly or insufficiently calibrated acquisition trajectories of the X-ray system when rotating around the examination subject. This method can also improve reconstruction when the X-ray system rotates around the patient with insufficient accuracy.

[0013] According to a further embodiment of the invention, the following additional steps are performed before the reconstruction step: updating the personalized 3D model of the examination subject using the second projection image, selecting at least one additional projection image from the data set, and registering the at least one additional projection image with the updated personalized 3D model of the examination subject. Reconstruction can then be performed using the at least three projection images used. Thus, the personalized 3D model is further refined (personalized) using the already registered second projection image or, if necessary, additional projection images, so that further optimizations and quality improvements of the 3D volume can be achieved based on at least three projection images.

[0014] According to a further embodiment of the invention, the steps of updating the personalized 3D model, selecting another projection image, and registering the next projection image are repeated prior to the reconstruction step until a termination criterion is met. The use of a further iterative method until a termination criterion is met allows for the automation of quality improvement. A termination criterion can be selected as needed, for example, such that sufficient projection images are motion-corrected to perform a high-quality reconstruction, but no more than necessary, thus simultaneously keeping the computational effort low.

[0015] According to a further embodiment of the invention, the termination criterion is formed by a predetermined threshold regarding the number of projection images or by a registration parameter. The number of projection images (or a certain percentage of recorded projection images) can be predetermined, for example, such that the image quality of the reconstructed 3D volume reaches a certain quality level. A registration parameter can mean, for example, that all projection images that exceed or fall below a predetermined degree of movement are used, and other projection images are not used. It can also be provided that only projection images with sufficiently spaced angulations (projection angles) are used.

[0016] According to a further embodiment of the invention, the generalized 3D model is formed from a shape model. Shape models are generally known, such as the statistical shape model (SSM), the statistical shape intensity model (SSIM), or the Skinned Multi-Person Linear model (SMPL), so they can be used for simple application of the method. Shape models of bone structures are known, for example, from the article by Shetty, K., Birkhold, A., Jaganathan, S., Strobel, N., Egger, B., Kowarschik, M., and Maier, A.: "Boss: Bones, organs and skin shape model." Computers in Biology and Medicine, Volume 165, Article 107383, October 2023.

[0017] According to a further embodiment of the invention, all projection images are selected and registered one after the other. This allows for a complete and optimal reconstruction using the entire data set of projection images.

[0018] According to a further embodiment of the invention, the projection images are selected randomly or based on an algorithm. An algorithm can specify which boundary conditions must be met. For example, the selection can be based on which projection image is most similar to the 3D model or which deviates most from the 3D model. It is also possible, for example, to select projection images whose projection angles differ by a certain angle, or, for example, every 2nd, 5th, or 10th projection image.

[0019] According to a further embodiment of the invention, prior to reconstruction, the already registered projection images are re-registered with the most current personalized 3D model. This allows for further improvement of the motion correction of the already registered projection images, thus resulting in an even better quality of the 3D volume to be reconstructed. Additionally, all remaining, not yet used projection images can be registered with the current personalized 3D model, and then a reconstruction can be performed based on all projection images.

[0020] According to a further embodiment of the invention, the personalized 3D model is generated by applying at least one pre-trained machine learning function, and / or the personalized 3D model is updated by applying at least one pre-trained machine learning function. In particular, the pre-trained function was generated by merging a regression-based method with an optimization-based method.

[0021] The invention further comprises a data processing unit for carrying out a method described above and a medical imaging device having a data processing unit, wherein the medical imaging device has a recording system for recording a data set with a plurality of projection images during a rotation run around the examination object.

[0022] The invention and further advantageous embodiments according to the features of the dependent claims are explained in more detail below with reference to schematically illustrated embodiments in the drawings, without thereby limiting the invention to these embodiments. They show: Fig. 1 a sequence of steps of a method for reconstructing a 3D volume of a moving object under investigation; Fig. 2 shows a further sequence of steps of a method for reconstructing a 3D volume of a moving object under investigation; Fig. 3 shows a further sequence of steps of a method for reconstructing a 3D volume of a moving object under investigation; and Fig. 4 a medical X-ray machine to carry out the procedure.

[0023] When reconstructing 3D volumes from a data set containing multiple projection images acquired during a rotation of a recording system around an examination subject, problems often arise due to patient or organ movements. Particularly with large movements, this can lead to a reconstruction producing a 3D volume of insufficient quality or even making reconstruction impossible. The present method for reconstructing a 3D volume of a moving examination subject solves this problem by using a 3D model that is initially personalized using a single projection image. This enables motion correction of additional projection images, thus guaranteeing a stable reconstruction. Fig. 1 shows the steps of the procedure.

[0024] In a first step 10, a data set containing a plurality of projection images acquired from different projection directions relative to the examination object is provided. Provision can occur, for example, by retrieving a previously acquired data set from a memory or database, or by directly acquiring a data set using an X-ray device. The data set can be, for example, a CBCT (cone-beam CT) image within the scope of a rotation trajectory. In this case, projection images are generally acquired over an angular range of at least 180° around the examination object. The X-ray device can be, for example, a C-arm X-ray device.

[0025] In a second step 11, a first projection image is selected from the data set. The first projection image can be selected randomly or according to an algorithm. For example, the first projection image can be the first projection image recorded or a projection image from a first, previously defined projection direction. It can also be a projection image of particularly high image quality, with a specific contrast, specific image content, or another specified property.

[0026] In a third step 12, a generalized 3D model of the examination object (e.g., the entire patient, a section, a body part, or an organ of the patient) is provided. This can be a shape model, such as a statistical shape model (SSM) or a statistical shape intensity model (SSIM), or in the case of bones, a BOSS or OSSA (see above).

[0027] In a fourth step 13, a personalized 3D model of the object under investigation is generated from the generalized 3D model, using the first projection image. For this purpose, a pre-trained machine learning function can be used, for example, the method presented in the article "HOOREX: Higher Order Optimizers for 3D Recovery from X-Ray Images" (cited above) or a similar procedure.

[0028] In a fifth step 14, a second projection image from the data set is then selected. The first projection image can also be selected randomly or according to an algorithm. Preferably, the projection angle of the second projection image differs significantly from the projection angle (>5° or, better, >10°) of the first projection image to enable a meaningful subsequent 3D reconstruction.

[0029] In a sixth step 15, the second projection image is registered with the personalized 3D model of the examination object. For this purpose, a known method for 2D-3D registration can be used, for example, using machine learning, as in Unberath, Mathias, et al., “The impact of machine learning on 2D / 3D registration for image-guided interventions: A systematic review and perspective.” Frontiers in Robotics and AI 8 (2021): 716007, or using an algorithm as in Y. Lei and Y. Zhang, “An improved 2D-3D medical image registration algorithm based on modified mutual information and expanded Powell method,” 2013 IEEE International Conference on Medical Imaging Physics and Engineering, Shenyang, China, 2013, pp. 24-29.

[0030] Subsequently, in a seventh step 16, a 3D volume is reconstructed from the registered projection images, in this example from the first and the second projection image, since these have now undergone a motion correction through registration.

[0031] In the Fig. In the simplest case shown in Figure 1, only two projection images are used for a reconstruction. Since the image quality is significantly higher for a 3D reconstruction with three or, even better, multiple projection images, even more projection images are generally selected. In the simplest case, the additional selected projection images are also registered to the personalized 3D model and, following registration of all (selected or all) images, are reconstructed into a 3D volume.

[0032] A further refinement of the method can be achieved by updating the personalized 3D model using the second or each subsequent projection image after registration. This can again be performed in the same way as the original personalization of the general 3D model, for example, using the HOOREX method described above and a trained function.

[0033] In the Fig. 2 shows the steps of another method. The first four steps 10 to 13 are initially identical to the steps from Fig. 1 is identical. In an eighth step 24, another projection image is then selected. The selection of the additional projection image can also be random or carried out according to an algorithm. In a ninth step 25, the additional projection image is then registered with the personalized 3D model of the examination object. For this purpose, a known method for 2D-3D registration can again be used. Subsequently, in a tenth step 17, a termination criterion (or several termination criteria) is queried. Such a termination criterion can, for example, be a predetermined threshold with regard to the number of projection images, i.e., when a certain number or a certain percentage of the projection images have already been used. It can also be provided to check the image quality and, after using the projection images that meet a certain quality criterion (e.g.regarding image sharpness, contrast, movement, etc.). The termination criterion can also include a registration parameter, for example, a predetermined degree of reprojection errors or convergence of the registration algorithm. If the termination criterion (or the termination criteria) is not met, the personalized 3D model is updated in an eleventh step 18 using the additional projection image (for example, as previously described using the method and a trained function) and then returned to the eighth step 24, the selection of another projection image. This loop is repeated until a termination criterion is met. If a termination criterion is met, the seventh step 16, a reconstruction of all previously registered projection images into a 3D volume, is carried out.

[0034] In the Fig. Figure 3 shows a further method in which, prior to reconstruction, a registration of the remaining, not yet used projection images with the current personalized 3D model is performed in a twelfth step 26. Alternatively or additionally, the previously registered projection images can also be re-registered with the current personalized 3D model.

[0035] In addition, other methods for movement correction can also be performed, either before the procedure begins or before the respective reconstruction.

[0036] In the Fig.4 shows a medical X-ray device 30 controlled by a system controller 33, comprising a data processing unit 31 for carrying out the steps of the method described above. The X-ray device 30 has an acquisition system, e.g., a C-arm 32 with an X-ray source 36 and an X-ray detector 37, which is designed to generate a plurality of projection images, e.g., for a CBCT image, of an examination object 35 within the framework of a rotation trajectory. The data processing unit 31 is designed to select projection images, to generate a personalized 3D model from a generalized 3D model using at least one projection image, e.g., using trained functions, to register projection images with the personalized 3D model, and to carry out reconstructions.

[0037] As an alternative to a recording, a previously recorded data set can also be provided and used, for example, using a provisioning unit. A data storage device 34 can be provided for storing a data set, intermediate or final results of the method, or the generalized or personalized 3D model.

[0038] Overall, an approach is demonstrated that enables an initial (and not necessarily precise) motion correction of the individual projections, despite significant patient movements. Subsequently, an initial corrected image reconstruction can be performed, on which other known methods for more precise motion compensation can then be applied. Even with 3D images with very strong and poorly modelable movements, initialization can be performed with this method and then iteratively refined using other state-of-the-art methods.

[0039] Regardless of the grammatical gender of a particular term, persons with male, female or other gender identities are included.

[0040] The invention can be briefly summarized as follows: For particularly good image quality in 3D volume images impaired by patient movements, a method for reconstructing a 3D volume of a moving examination object is provided, comprising the following steps: providing a data set with a plurality of projection images, which was generated during a rotation run of an acquisition system of an X-ray device around the examination object, selecting at least a first projection image of the data set, providing a generalized 3D model of the examination object, generating a personalized 3D model of the examination object from the generalized 3D model using the first projection image, selecting a second projection image of the data set, registering the second projection image with the personalized 3D model of the examination object,and reconstruction of a 3D volume of the object under investigation based on the registered projection images. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] https: / / www.cs.cornell.edu / projects / bigsfm

[0004] Amirhossein Saedpanah et al, “Geometrical Self-Calibration of CBCT Systems”, 12th Conference on Industrial Computed Tomography, Fürth, Germany (iCT 2023

[0005] Karthik Shetty, Annette Birkhold, et al, „HOOREX: Higher Order Optimizers for 3D Recovery from X-Ray Images“, Workshop on Machine Learning for Multimodal Healthcare Data at conference, Saturday, July 29, Hawaii, USA

[0010] Shetty, K., Birkhold, A., Jaganathan, S., Strobel, N., Egger, B., Kowarschik, M., Maier, A.: „Boss: Bones, organs and skin shape model“. Computers in Biology and Medicine, Volume 165, Article 107383, Oktober 2023

[0016] HOOREX: Higher Order Optimizers for 3D Recovery from X-Ray Images

[0027] Unberath, Mathias, et al. „The impact of machine learning on 2d / 3d registration for image-guided interventions: A systematic review and perspective.“ Frontiers in Robotics and AI 8 (2021): 716007

[0029] Y. Lei and Y. Zhang, „An improved 2D-3D medical image registration algorithm based on modified mutual information and expanded Powell method,“ 2013 IEEE International Conference on Medical Imaging Physics and Engineering, Shenyang, China, 2013, pp. 24-29

[0029]

Claims

[1] Method for reconstructing a 3D volume of a moving object under investigation, comprising the following steps: • Provision of a data set with a large number of projection images, which was generated during a rotation run of an X-ray device's recording system around the object under examination, • Selection of at least a first projection image of the data set, • Providing a generalized 3D model of the object under investigation, • Creating a personalized 3D model of the object under investigation from the generalized 3D model using the first projection image, • Selection of a second projection image of the data set, • Registration of the second projection image with the personalized 3D model of the object under examination, and • Reconstruction of a 3D volume of the object under examination based on the first and at least one further registered projection image. [2] A method according to claim 1, comprising the following additional steps before the reconstruction step: • Updating the personalized 3D model of the examination object using the second projection image, and • Selecting at least one further projection image of the data set and registering the at least one further projection image with the updated personalized 3D model of the object under examination. [3] Method according to claim 1 or 2, wherein, prior to the reconstruction step, a repetition of the steps of updating the personalized 3D model, selecting a further projection image and registering the further projection image is carried out up to a termination criterion. [4] Method according to one of the preceding claims, wherein the generalized 3D model is formed by a shape model. [5] Method according to one of the preceding claims, wherein all projection images are selected and registered one after the other. [6] Method according to one of the preceding claims, wherein the selection of the second and / or further projection images is random or based on an algorithm. [7] Method according to one of the preceding claims, wherein the selection of the second and / or further projection images is carried out according to which projection image is most similar to the 3D model or which deviates most from the 3D model. [8] Method according to claim 3, wherein the termination criterion is formed by a predetermined threshold with regard to the number of projection images already registered, or by a registration parameter. [9] Method according to one of the preceding claims, wherein a re-registration of the already registered projection images with the current personalized 3D model is carried out before the reconstruction. [10] Method according to one of the preceding claims, wherein the personalized 3D model is generated by applying at least one pre-trained machine learning function and / or the updating of the personalized 3D model is performed by applying at least one pre-trained machine learning function. [11] The method of claim 10, wherein the pre-trained function was generated by fusing a regression-based method with an optimization-based method. [12] Data processing unit (31) which is designed to carry out a method according to one of claims 1 to 11. [13] A medical imaging device (30) comprising a data processing unit (31) according to claim 12, wherein the medical imaging device (30) has a recording system for recording a data set with a plurality of projection images during a rotation run around the examination object (35).

Images