CBCT simulation for training AI-based CT-to-CBCT registration and CBCT segmentation

Abstract

The invention relates to a computer-implemented method for generating a simulated cone beam computed tomography image based on a computed tomography image. The computed tomography image is transformed into an attenuation coefficient of the represented tissue, and the computed tomography image is forward projected to the projection image based on predetermined scanner parameters of the simulated cone beam computed tomography scanner. After artificial noise representing noise detected by the simulated cone beam computed tomography scanner is added to the projection image, the projection image is back-projected using a predetermined reconstruction algorithm to generate a simulated cone beam computed tomography image of the object. The invention further relates to a method for generating training data for training an artificial intelligence module on the basis of a simulated image, and to a method for registering a computed tomography image to a cone beam computed tomography image, and a method of segmenting a cone beam computed tomography image using an artificial intelligence module trained using training data including simulated cone beam computed tomography images.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for generating a simulated cone-beam computed tomography image based on a computed tomography image, a computer-implemented method for generating training data for training an artificial intelligence module to register a cone-beam computed tomography image to a computed tomography image, a computer-implemented method for registering a computed tomography image to a cone-beam computed tomography image, a computer-implemented method for segmenting a cone-beam computed tomography image, a data processing device, a computer program, and a computer-readable storage medium. Background Art

[0002] Computed tomography (CT) to cone-beam CT (CBCT) image registration and intraoperative C-arm cone-beam CT (CBCT) image segmentation are recurring tasks in many medical applications and image-guided interventions. For example, diagnostic CT scans must be aligned with intraoperative CBCT scans to map interventional plans to the patient coordinate system for radiation therapy or to guide percutaneous needle interventions. Furthermore, automated segmentation of anatomical structures in cone-beam CT (CBCT) images is a prerequisite for many interventional applications. For example, the renal parenchyma and surrounding organs at risk must be delineated for radiation therapy or to guide percutaneous needle interventions. Segmentation of CBCT images is known to be challenging due to the limited field of view and the presence of image artifacts in CBCT images. In recent years, AI-based CT to CBCT registration methods and AI-based segmentation methods have become increasingly popular, as they have been shown to outperform or at least be comparable to traditional non-AI-based methods in terms of speed and accuracy. AI-based training methods can be categorized as supervised and semi-supervised.

[0003] However, while supervised training algorithms allow for direct optimization of the target deformation, the true transformation is often unknown and therefore unavailable during training. Consequently, these techniques are generally applicable only to single-modality registration tasks where a synthetic transformation is the ground truth. As a remedy, semi-supervised methods can search for the target deformation by formulating several conditions (such as its smoothness) and optimizing these conditions. However, formulating meaningful conditions can be challenging. Furthermore, training such algorithms requires a large number of available training image pairs specific to the application at hand. Therefore, training such algorithms requires a large amount of appropriate training data. Access to large datasets is one of the main challenges in developing AI-based algorithms. For CT images, large public databases (which may be partially longitudinal) exist that can be used to train neural networks. However, this is not the case for CBCT data, as abdominal CBCT is not the norm for clinical diagnostic imaging, and patients cannot be unnecessarily exposed to radiation. Furthermore, to achieve comparable accuracy and robustness, more data is typically required than for CT data to compensate for the increased variability of cone-beam computed tomography image data, such as positioning, limited field of view, non-quantitative modality, and image artifacts. Therefore, there is a need for improvement in this area.

[0004] JERICHO O'CONNELL et al.: “FastCAT: Fast Cone Beam CT (CBCT) Simulation,” ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY, ITHACA, NY 14853, February 16, 2021 (2021-02-16), XP081880000, describes a fast cone-beam CT simulator that uses precomputed Monte Carlo CBCT phantom-specific scatter and detector response functions to reduce simulation time for CBCT imaging.

[0005] CAO YUZHU et al.: “CDFRegNet: A cross-domain fusion registration network for CT-to-CBCT image registration,” COMPUTER METHODS AND PROGRAMS INBIOMEDICINE, ELSEVIER, AMSTERDAM, NL, volume 224, July 15, 2022 (2022-07-15), XP087151346, describes a cross-domain fusion registration network for CT-to-CBCT image registration using an edge-guided attention module and a cross-domain attention module.

[0006] Therefore, the inventors of the present invention have discovered that it would be advantageous to have a method for solving these problems while retaining the advantages of each method. Summary of the Invention

[0007] The object of the present invention is to provide an improved method which provides training data with higher accuracy and offers the possibility to train AI-based algorithms with more robust models.

[0008] The objects of the invention are solved by the subject-matter of the independent claims, wherein further embodiments are incorporated in the dependent claims.

[0009] The described embodiments similarly relate to a computer-implemented method for generating a simulated cone-beam computed tomography image based on a computed tomography image, a computer-implemented method for generating training data for training an artificial intelligence module to register a cone-beam computed tomography image to a computed tomography image, a computer-implemented method for registering a computed tomography image to a cone-beam computed tomography image, a computer-implemented method for segmenting a cone-beam computed tomography image, a data processing apparatus, a computer program, and a computer-readable storage medium. The further described embodiments can be combined in any possible manner. Although not described in detail, different combinations of the embodiments may produce synergistic effects.

[0010] Furthermore, it should be noted that all embodiments of the present invention relating to the methods can be performed in the order of the steps described, however, this is not the only and necessary order of the steps of the method. Unless expressly stated otherwise below, the methods presented herein can be performed in another order of the steps disclosed without departing from the corresponding method embodiments.

[0011] According to a first aspect of the present invention, a computer-implemented method for generating a simulated cone-beam computed tomography (CBCT) image based on a computed tomography (CT) image is provided. The method comprises the steps of receiving data representing a computed tomography image of an object. The data may be acquired directly by and transmitted from a computed tomography scanner, or downloaded from a cloud storage, or acquired from a DICOM station, or in any other possible manner. The computed tomography image comprises a volume of the object, the volume of the object being divided into a plurality of voxels, each voxel containing tissue properties of the tissue of the object expressed in Hounsfield units (HU). The method further comprises the steps of transforming the Hounsfield units of the computed tomography image voxels into attenuation coefficients, receiving predetermined scanner parameters of a simulated cone-beam computed tomography scanner, and forward projecting the computed tomography image into a projection image based on the scanner parameters of the simulated cone-beam scanner computed tomography. The method further comprises the steps of adding artificial noise to the projection image, the artificial noise being representative of noise detected by the simulated cone-beam computed tomography scanner; back-projecting the projection image using a predetermined reconstruction algorithm to generate a simulated cone-beam computed tomography image of the object; and providing the simulated cone-beam computed tomography image of the object.

[0012] Therefore, we propose a method for physically simulating CBCT images from CT scans to develop a new registration method for CT-to-CBCT registration and to train a neural network for CBCT segmentation. This approach alleviates the high training data requirements, allowing already trained CT models to more easily adapt to new data from cone-beam computed tomography scanners. Since CT data is more readily available than CBCT data in clinical practice, the proposed solution will help address the data scarcity issue. Therefore, by providing an algorithm that uses CT scans to generate realistic simulations of CBCT data, the data scarcity issue for training CBCT protocols can be addressed. This allows the creation of CBCT datasets covering a wide range of anatomical variations, scan positions, fields of view, and parameter settings by leveraging widely available CT data sources. Leveraging this method and simulated CBCT data, supervised or unsupervised training of CT-to-CBCT registration algorithms and AI-based segmentation of CBCT images can be employed.

[0013] Therefore, one of the core elements of the present invention is an algorithm that simulates a realistic CBCT scan based on a given CT scan. Notably, since the image generation process for CBCT images is known, any annotations present in the CT images (e.g., via voxel-based segmentation) can be transferred to the CBCT images, as no additional annotation is required.

[0014] An image simulation method for simulating data from one imaging modality to another is provided. A detailed description of the key elements is provided below. The simulation algorithm physically simulates a CBCT scan based on a CT scan. This involves receiving a computed tomography image of an object acquired by a computed tomography scanner. The object can be a patient, human, or animal, and the acquired image can depict at least a portion of the object or patient. Alternatively, the term "object" can be understood to describe only a portion of the patient, for example, when the acquired image provides a limited field of view. The object can also be an anatomical region of interest or the location of the imaging system, as CBCT typically cannot capture the entire object. Therefore, the object must be interpreted extensively. The computed tomography image depicts the object volume divided into a plurality of voxels, each of which contains a tissue property of the object's tissue expressed in Hounsfield units. For example, a tissue property can be a measure of the tissue's opacity with respect to X-ray radiation of a specific wavelength. By identifying tissue types such as bone, calculus, soft tissue, blood, and contrast agent, the Hounsfield units of a computed tomography image voxel can be converted to a corresponding tissue-specific attenuation coefficient for that tissue. This attenuation coefficient can be a function of the energy of the X-ray radiation.

[0015] After receiving predetermined scanner parameters for a simulated cone-beam computed tomography scanner, a CT image is forward projected based on the scanner parameters of a simulated CBCT scanner. These scanner parameters may include, for example, the position of the scanner relative to the patient, rotation angles, scattering characteristics, beam hardening effects, or detector imperfections. Next, artificial noise is added to the projected image in the projection space. This noise may represent electronic noise in the simulated CBCT scanner detector or crosstalk between adjacent detector channels. Furthermore, the noisy forward projected image is back-projected using a given reconstruction algorithm and scanner parameters specific to the simulated CBCT scanner (e.g., resolution). Because the field of view of cone-beam CT is typically different (and typically smaller) than that of CT, the reconstruction algorithm may include a cropped field of view. The reconstruction algorithm may be, for example, DCS or FDK. The back-projected simulated CBCT image may be provided to a user or used for further processing. Therefore, the present invention proposes the use of physical simulations of CBCT images from CT scans. These images can be used to develop new supervised registration methods for CT-to-CBCT registration or to train AI-based CBCT image segmentation.

[0016] As shown in the figure, a wide set of parameters are available. All parameters can be selected to cover a wide range of scanners or protocols, or to optimize the application for one specific protocol and / or reconstruction algorithm.

[0017] In an embodiment of the present invention, the method further comprises the step of modifying the tissue characteristics of the subject's tissue. Thus, the modified tissue classification may include the addition of kidney stones or their components, or the increase of spinal bone density. Furthermore, the contrast agent in the CT image may be assigned a blood classification to simulate images acquired without the contrast agent.

[0018] In a further embodiment, a spectral image-guided therapy system similar to a spectral CT scanner could be introduced. Several kidney stones with widely varying chemical compositions are known to exhibit unique spectral CT signatures. CBCT acquisitions that mimic the anatomically natural location of such stones would allow for the development of proof points and appropriate scanning and reconstruction protocols, which is faster and more comprehensive than actual in vivo scanning.

[0019] Thus, typical application-specific artifacts that prevent correct segmentation or registration can be integrated into the simulation. For example, in an intra-procedural CBCT volume, the outline of the kidney may be corrupted by streak artifacts generated by high iodine concentrations in the urinary outflow tract. The simulation of physical effects and their current (imperfect) corrections produce typical, realistic CBCT volumes that can be used for supervised training of AI-based methods. Thus, AI-based algorithms can be trained to determine chemical composition from CT, CBCT, or PET (positron emission tomography) data. Key applications of this technology can be demonstrated.

[0020] In an embodiment of the present invention, scanner parameters include at least two tube peak voltages for a simulated cone-beam computed tomography (CBCT) scanner. Thus, multi-energy forward projections of two or more hypothetical tube peak voltages can be implemented in the simulation to achieve spectral or dual-energy CBCT simulations. CT image forward projections based on scanner parameters for a simulated CBCT scanner can be multi-energy. Thus, scanner parameters can include detection parameters, such as multi-energy X-ray radiation, in addition to position, rotation angle, scatter, beam hardening effects, or detector defects. In addition to or as an alternative to simulating different tube peak voltage (kVp) settings for the X-ray source, the detector can be modeled as energy selective.

[0021] In an embodiment of the present invention, the computed tomography image is a fan-beam computed tomography image or a cone-beam computed tomography image. Therefore, simulation of CBCT images can be performed based on images acquired using a conventional fan-beam computed tomography scanner or based on images acquired using a cone-beam computed tomography scanner. This allows for the utilization of widely available conventional CT image data, or provides training data for AI-based CBCT-to-CBCT registration, for example, for interventional success control and outcome monitoring.

[0022] According to another aspect of the present invention, a computer-implemented method for generating training data for training an artificial intelligence module to register a cone-beam computed tomography image to a computed tomography image is provided. The method comprises receiving a first computed tomography image representing an object, generating data representing a second computed tomography image of the object, wherein the second computed tomography image differs from the first computed tomography image in that a transformation is applied to the second computed tomography image, the transformation comprising deformation, distortion, rotation, and / or cropping of the object, and generating data representing the transformation. The method further comprises the steps of generating a simulated cone-beam computed tomography image based on one of the first and second computed tomography images according to the method of any of the preceding embodiments, wherein the data representing the computed tomography image comprises the first and / or second computed tomography images, and generating a set of training data comprising a simulated cone-beam computed tomography image based on one of the first and second computed tomography images, the other of the first and second computed tomography images, and the data representing the transformation. In addition, a set of training datasets is also provided.

[0023] Therefore, using the training data provided by the proposed method, given two images I and J and a ground truth transformation T, a registration method can be trained to predict the transformation T':=y(I,J) by minimizing the loss function L(T,T'). Therefore, one of images I and J is preferably a CT image, while the other is preferably a CBCT image. Therefore, data representing a first computed tomography image and a second computed tomography image must be provided. The first and second computed tomography images may need to be images of the same subject. In alternative embodiments, it is sufficient for the first and second computed tomography images to be images of corresponding body parts from different patients. However, the first and second CT images need to represent at least similar body parts of the subject, preferably the same body part of the same patient acquired with a similar field of view. For training purposes, a transformation between the first and second CT images may be required. The transformation can be a deformation, distortion, rotation, and / or translation of the object or image, and / or a cropping of the field of view of the image. The field of view of cone-beam CT is typically different from, and typically smaller than, that of CT. In addition to the patient being rotated, translated, etc., the CBCT imager may be in a different position from the CT gantry. With these two CT images, and data representing the underlying transformation, one of the first and second CT images can be used as the basis for generating a simulated cone-beam computed tomography image according to the method described above. A set of training data generated using the method according to the present invention includes the generated simulated cone-beam computed tomography image, as well as the CT image of the two CT images that was not used to generate the simulated CBCT image, and data representing the transformation as the ground truth. An AI-based registration algorithm can be trained to predict the transformation T' given an image I (the first CT image), a simulated CBCT image based on the second CT image as image J, and the ground truth transformation T by minimizing a loss function L(T,T').

[0024] Based on this, two main approaches can be followed to train an AI-based registration algorithm, depending on the target application and data availability. There may be at least two strategies to collect information to provide a set of training data sets, in particular a first CT image and a second CT image, with their respective transformations T as ground truth. For example, in a first embodiment, the true transformation T is calculated by registering two CT images I and J as a first CT image and a second CT image. For example, these can be two longitudinal scans or an inspiratory / expiratory image pair of the same object. The transformation T can be determined by applying traditional registration methods to images I and J. After using one of the images I and J as the basis for generating a simulated CBCT image, a set of training data can be provided. The training data set includes a simulated CBCT image generated based on one of the CT images I and J, the other of the CT images I and J, and data representing the transformation provided by the registration. Therefore, an AI-based algorithm can be trained to generate a simulated CBCT image based on I and J. CBCT And T as the ground truth, predict T' by minimizing the loss function L(T,T').

[0025] In an alternative embodiment, an artificial transformation T is applied to transform the first CT image, thereby generating a second CT image. One of these CT images is then used as the basis for generating a simulated CBCT image. Since the transformation is artificially applied to the CT image, the data representing the transformation is known. Therefore, an AI-based algorithm can be trained to generate a simulated CBCT image based on image I as the first CT image and image J = I. T CBCT To estimate T', we apply an artificial transformation T based on the image I and the known data representing the transformation T to generate a simulated CBCT image.

[0026] In an embodiment of the present invention, the step of generating data representing a second computed tomography image of the subject includes receiving data representing a second computed tomography image of the subject. In this case, there are scan pairs comprising a true transformation, wherein multiple pairs of CT scans are considered, such as pre-operative and post-operative scans, longitudinal scans at different time points, or pairs of inhalation / expiration of a patient. The pairs of CT scan images can be acquired from the same patient at different time points or under different circumstances. However, inter-subject registration can even be learned using scan pairs obtained from different patients, where the scan pairs depict similar regions of the patient's body.

[0027] In an embodiment of the invention, the step of generating data representing the transformation comprises registering the first computed tomography image to the second computed tomography image. Thus, the real transformation T is calculated by registering two CT images I and J. For example, they can be two longitudinal scans of the same subject or an inspiration / expiration pair. The algorithm can then be trained to generate the transformation based on I and J. CBCT, and the calculated transformation T as the ground truth to predict T'. This situation depends heavily on the accuracy of the underlying registration algorithm, which sets a lower bound on the expected accuracy. However, for many applications, CT-CT registration has been shown to be very accurate and is generally easier than CT-CBCT registration.

[0028] In an embodiment of the present invention, registration of the first computed tomography image to the second computed tomography image is performed using a conventional or AI-based registration algorithm. Thus, the ground truth transformation can be determined by registering the scans using an existing (e.g., conventional) registration algorithm. Alternatively, a conventional AI-based registration algorithm can be applied to register the two CT images.

[0029] In an embodiment of the invention, the step of generating data representing a second computed tomography image of the object comprises applying an artificial transformation to data representing the first computed tomography image of the object thereby generating data representing the second computed tomography image of the object.

[0030] In this scenario, a first CT image acquired using a single CT scan is available, and an artificial transformation is performed on it, which deforms the CT image to provide a second CT image. Prior to CBCT simulation, a random artificial transformation can be used to deform the CT scan. Depending on the application, different transformation models can be chosen to generate the random transformations. For simple, coarse alignment of CT and CBCT scans, an affine transformation can be chosen. In this case, the trained network will predict the parameters defining the transformation, such as rotation angles, translation vectors, etc. If more complex deformations between the images are expected, a dense transformation field can be generated using, for example, radial basis functions or biophysical models, for example, to learn the registration of inspiratory and expiratory scans. In certain embodiments, the transformation T may not be randomly selected, but rather derived from a previous registration with any other (intra-patient or inter-patient) dataset. In this way, the domain of T will include the actual transformation, but the requirements for available data are higher. The generation of simulated CBCT images can be performed using any CT image. In this case, collecting training data is easy, as only a single scan is required. However, the applicability depends on the transformation model. Furthermore, the training data may only contain spatial deformations between scans, without any longitudinal changes. Before generating the simulated CBCT image, the CT image I can be transformed by applying an artificial transformation T to derive the image J: Image J = I T CBCT :=(I°T) CBCT . An AI-based algorithm can be trained to estimate T' based on I, J, and T.

[0031] In an embodiment of the invention, the step of generating data representing the transformation comprises receiving data representing the artificial transformation.Thus, since the data representing the artificial transformation is itself known, there is no need to determine the transformation separately.

[0032] According to another aspect of the present invention, a computer-implemented method for registering a computed tomography image to a cone-beam computed tomography image is provided. The method comprises the steps of receiving data representing a computed tomography image of an object, receiving data representing a cone-beam computed tomography image of the object, and determining, using an artificial intelligence module, a transformation required to register the computed tomography image to the cone-beam computed tomography image, wherein the artificial intelligence module is trained using training data generated by the method according to any of the preceding embodiments. The method further comprises the steps of registering the computed tomography image to the cone-beam computed tomography image based on the determined transformation, and providing the computed tomography image registered to the cone-beam computed tomography scan.

[0033] The method is applicable to all applications involving CT-to-CBCT registration, such as for image-guided lung, liver, or kidney interventions, and provides a very fast and reliable method for registering images in near real time. In a supervised registration approach, given two images I and J and a ground truth transformation T as a set of training data, an artificial intelligence module can be trained to predict the transformation T':=y(I,J) by minimizing a loss function L(T,T'). At least one of images I and J is a simulated CBCT image generated by the simulation method described above.

[0034] According to another aspect of the present invention, a computer-implemented method for segmenting a cone-beam computed tomography image is provided. The method comprises the steps of: receiving data representing a cone-beam computed tomography image of an object acquired by a cone-beam computed tomography scanner;

[0035] Segmenting the cone-beam computed tomography image using an artificial intelligence module, wherein the artificial intelligence module is trained using training data, the training data comprising a plurality of simulated cone-beam computed tomography images generated according to the method of any of the preceding embodiments, and providing a segmented cone-beam computed tomography image.

[0036] Thus, AI-based segmentation can be trained to segment anatomical structures in CBCT images using simulated CBCT data generated using the simulation method described above.

[0037] In an embodiment of the present invention, the training data includes a plurality of computed tomography images acquired using a computed tomography scanner and / or a plurality of cone-beam computed tomography images acquired using a cone-beam computed tomography scanner.

[0038] The final training dataset for the preferred supervised training can have different compositions. In the first composition, only simulated CBCT data is used. In this most basic embodiment, a set of CT images is collected, and one or more CBCT scans are simulated for each CT scan using different parameter settings (such as scanner positioning, noise level, etc.). In this way, a model specific to the (simulated) CBCT data is trained. In the second composition of the training dataset, CT images and simulated CBCT images are used. In this embodiment, the original CT scans can also be used during training. In this way, a fairly general CT-CBCT model is trained that is capable of segmenting both CT and (simulated) CBCT data. In the third composition of the training dataset, CT images, simulated CBCT images, and real CBCT data are used. Additionally, the real CBCT data can be used to incorporate data attributes that cannot be covered by the simulation, such as specific types of image artifacts, equipment, or surgical scenarios.

[0039] Additionally or alternatively, simulated CBCT data can be used to pretrain a model for a specific task, which can then be refined by training the model in several further stages using clinical CBCT data for domain adaptation. Compared to a CT-only pretrained model, the model must only adapt to specific image properties of the CBCT data, such as the characteristic signal-to-noise ratio, while other properties, such as the limited field of view, are already accounted for during pretraining. Experiments have shown that in this way, fewer clinical images are required to achieve suitable segmentation quality.

[0040] The method can be applied to all applications involving CBCT segmentation, such as for image-guided lung, liver or kidney interventions.

[0041] In embodiments of the present invention, the artificial intelligence module is trained using training data using a supervised or semi-supervised training algorithm. Depending on the target application and data availability, two main approaches can be used to train AI-based registration or segmentation algorithms, namely supervised or semi-supervised.

[0042] According to another aspect of the present invention, a data processing device is provided, comprising a processor configured to execute the steps of the method according to any one of the aforementioned embodiments.

[0043] According to another aspect of the present invention, a computer program is provided comprising instructions that, when executed by a computer, cause the computer to perform the steps of the method according to any one of the preceding embodiments. The computer program element may be executed on one or more processing units that are instructed to perform the method according to any one of the preceding embodiments.

[0044] According to another aspect of the present invention, there is provided a computer-readable storage medium comprising instructions, wherein when the instructions are executed by a computer, the computer is caused to perform the steps of the method according to any one of the aforementioned embodiments.

[0045] Therefore, the benefits provided by any of the above aspects also apply to all other aspects, and vice versa.

[0046] In summary, the present invention relates to a computer-implemented method for generating a simulated cone-beam computed tomography (CBCT) image based on a CBCT image. In some embodiments, the CBCT image is transformed into an attenuation coefficient of a representative tissue and forward-projected onto a projection image based on predetermined scanner parameters of a simulated CBCT scanner. After adding artificial noise representing noise detected by the simulated CBCT scanner to the projection image, the projection image is back-projected using a predetermined reconstruction algorithm to generate a simulated CBCT image of the object. The present invention also relates to a method for generating training data for training an artificial intelligence module based on the simulated image, a method for registering a CBCT image to a CBCT image, and a method for segmenting a CBCT image using an artificial intelligence module trained using training data including the simulated CBCT image.

[0047] One of the advantages of the present application is that the algorithm of the present invention can simulate CBCT images directly from CT images.

[0048] Another advantage is that simulation of Generative Adversarial Network (GAN) simulations can be omitted, which allows to speed up the simulation process and improve the simulation quality.

[0049] Another advantage may be that it would be possible to simulate CBCT images in near real time, thereby eliminating lags in the simulation.

[0050] These advantages are non-limiting and other advantages may be envisioned within the context of the present application.

[0051] The above aspects and embodiments will become apparent and clarified based on the exemplary embodiments described below. Exemplary embodiments of the present invention will be described below with reference to the following drawings: BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A block diagram of a computer-implemented method for generating a simulated cone-beam computed tomography image based on a computed tomography image according to an embodiment of the present invention is shown.

[0053] Figure 2 Example images of simulated CBCT scans generated based on CT scans using different parameter settings are shown.

[0054] Figure 3 A block diagram of a computer-implemented method for generating training data for registering cone-beam computed tomography images to computed tomography images to train an artificial intelligence module is shown in accordance with an embodiment.

[0055] Figure 4a and 4b Block diagram showing two strategies for generating training datasets and training AI-based CT-to-CBCT registration algorithms.

[0056] Figure 5 A block diagram of a computer-implemented method for registering a computed tomography image to a cone-beam computed tomography image according to an embodiment is shown.

[0057] Figure 6 A block diagram of a computer-implemented method for segmenting a cone-beam computed tomography image is shown according to an embodiment.

[0058] Figure 7 Automatic kidney segmentation of clinical CBCT scans is shown.

[0059] Reference Signs List

[0060] 110 computed tomography images

[0061] 111 First Computed Tomography Image

[0062] 112 Second Computed Tomography Image

[0063] 120 simulated cone-beam computed tomography images

[0064] 130 objects

[0065] 140 Transformation

[0066] 141 Determined Transformation

[0067] 150-segment cone-beam computed tomography image

[0068] 200 Artificial Intelligence Modules DETAILED DESCRIPTION

[0069] Figure 1 A block diagram illustrates a computer-implemented method for generating a simulated cone-beam computed tomography (CT) image based on a CT image according to an embodiment of the present invention. The method includes step S110 of receiving data representing a CT image of an object acquired by a CT scanner, the CT image including a volume of the object, the volume of the object being divided into a plurality of voxels, each voxel including a representation of a tissue characteristic of tissue of the object expressed in Hounsfield units, and step S120 of transforming the Hounsfield units of the voxels of the CT image into corresponding attenuation coefficients. The method also includes step S130 of receiving predetermined scanner parameters of the simulated cone-beam CT scanner; step S140 of forward projecting the CT image into a projection image based on the scanner parameters of the simulated cone-beam CT scanner; and step S150 of adding artificial noise to the projection image, the artificial noise being a representation of noise detected by the simulated cone-beam CT scanner. The method further includes step S160 of back-projecting the projection image using a predetermined reconstruction algorithm to generate a simulated cone-beam computed tomography image of the object, and step S170 of providing the simulated cone-beam computed tomography image of the object.

[0070] Figure 2 Example images of simulated CBCT scans 120 of a subject 130 generated based on a CT scan 110 using different parameter settings are shown. The top image is a CT image acquired using a computed tomography scanner. The bottom three images show simulated CBCT images 120 based on CT image 110, generated using a method according to the present invention. The image on the left is a simulated CBCT image generated with a restricted field of view, while the image in the middle contains additional limited-angle artifacts. The image on the right shows additional iodine beam hardening artifacts due to the filling of the urine outflow tract with contrast agent.

[0071] Figure 3A block diagram illustrates a computer-implemented method for generating training data for training an artificial intelligence module, wherein the training data is used to register a cone-beam computed tomography image to a computed tomography image, according to an embodiment. The method includes step S210 of receiving a first computed tomography image representing an object acquired by a computed tomography scanner, and step S220 of generating data representing a second computed tomography image of the object, wherein the second computed tomography image differs from the first computed tomography image in that a transformation is applied to the second image, wherein the transformation includes deformation, distortion, rotation, and / or cropping of the object's field of view, and step S230 of generating data representing the transformation. The method further comprises, in step S240, generating a simulated cone-beam computed tomography image based on one of the first computed tomography image and the second computed tomography image according to the method of any of the aforementioned embodiments, and, in step S250, generating a set of training data, the set of training data comprising the simulated cone-beam computed tomography image based on one of the first computed tomography image and the second computed tomography image, the other of the first computed tomography image and the second computed tomography image, and data representing the transformation. Furthermore, the set of training data is provided in step S260.

[0072] Figure 4a and 4b A block diagram showing two strategies for generating training datasets and training AI-based CT to CBCT registration algorithms. Figure 4a In the simulation, an artificial transformation T is applied to transform the image I into I T , execute I T CBCT :=(I°T) CBCT To generate simulated CBCT images. Two images, I and I T CBCT It is input into the convolutional neural network CNN as a training algorithm based on I and I T CBCT Estimate T'. For supervised learning, the loss function is determined based on T and T'.

[0073] exist Figure 4b In this example, two images I and J are provided and the true transformation T is determined. In this example, image I is used to generate a simulated CBCT image I CBCT . Two images, I CBCT and J are input to the convolutional neural network CNN, which is trained as a training algorithm based on J and I CBCT To estimate T'. For supervised learning, the loss function is determined based on T and T'.

[0074] Figure 5 A block diagram of a computer-implemented method for registering a computed tomography image to a cone-beam computed tomography image according to an embodiment is shown. The method comprises step S310 of receiving data representing a computed tomography image of an object acquired by a computed tomography scanner, step S320 of receiving data representing a cone-beam computed tomography image of an object acquired by a cone-beam computed tomography scanner, and step S330 of determining, using an artificial intelligence module, a transformation required to register the computed tomography image to the cone-beam computed tomography image, wherein the artificial intelligence module is trained using training data generated by a method according to any of the aforementioned embodiments. The method further comprises step S340 of registering the computed tomography image to the cone-beam computed tomography image according to the determined transformation, and step S350 of providing the computed tomography image registered to the cone-beam computed tomography scan.

[0075] Figure 6 A block diagram of a computer-implemented method for segmenting a cone-beam computed tomography image according to an embodiment is shown. The method comprises step S410 of receiving data representing a cone-beam computed tomography image of an object acquired by a cone-beam computed tomography scanner, step S420 of segmenting the cone-beam computed tomography image using an artificial intelligence module, wherein the artificial intelligence module is trained using training data comprising a plurality of simulated cone-beam computed tomography images generated according to the method of any of the aforementioned embodiments, and step S430 of providing a segmented cone-beam computed tomography image.

[0076] Figure 7 Automated kidney segmentation of a clinical CBCT scan is shown. The left image 150 shows the segmentation results of an algorithm trained only on CT data, which failed to correctly segment the kidney of subject 130 in the CBCT image, as can be seen in the dark edges in the lower portion of the image. The black arrows indicate dark edges, which are used to visualize the segmentation results for subject 130's kidney. The right image 150 shows the segmentation results of an algorithm trained on both CT and simulated CBCT data, resulting in significantly better performance, even though no clinical CBCT data was used in this training process. In this image, the dark edges cover the entire kidney region. In our experiments, the CT-only model achieved a Dice of 0.49 when applied to a set of 45 CBCT images, while the CT / simulated CBCT model achieved a Dice of 0.65.

[0077] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description should be considered illustrative or exemplary rather than restrictive. The present invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention by studying the drawings, the disclosure, and the appended claims.

[0078] In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A computer-implemented method for generating a simulated cone-beam computed tomography image (120) based on a computed tomography image (110); the method comprising the following steps: receiving data representing a computed tomography image (110) of a subject (130), the computed tomography image comprising a volume of the subject (130), the volume of the subject being divided into a plurality of voxels, wherein the voxels comprise a representation of tissue properties of tissue of the subject expressed in Hounsfield units; transforming the Hounsfield units of the computed tomography image (110) voxels into attenuation coefficients; receiving predetermined scanner parameters for simulating a cone-beam computed tomography scanner, forward projecting the computed tomography image (110) into a projection image based on the scanner parameters of the simulated cone-beam computed tomography scanner; adding artificial noise to the projection image, the artificial noise being representative of noise detected by the analog cone-beam computed tomography scanner; Back-projecting the projection image using a predetermined reconstruction algorithm to generate a simulated cone-beam computed tomography image (120) of the object; and The simulated cone-beam computed tomography image of the object is provided (120).

2. The method according to claim 1, wherein The method further comprises the step of altering the tissue properties of the tissue of the subject (130).

3. The method according to any one of claims 1 or 2, wherein: The scanner parameters include at least two tube peak voltages of the simulated cone-beam computed tomography scanner.

4. The method according to any one of claims 1 to 3, wherein: The computed tomography image (110) is a fan-beam computed tomography image or a cone-beam computed tomography image.

5. A computer-implemented method for generating training data for training an artificial intelligence module (200) to register a cone-beam computed tomography image to a computed tomography image, the method comprising the steps of: receiving data representing a first computed tomography image (111) of an object (130); generating data representing a second computed tomography image (112) of the object (130), wherein the second computed tomography image (112) differs from the first computed tomography image (111) in that a transformation (140) is applied to the second computed tomography image (112), the transformation (140) comprising a deformation, and / or distortion, and / or rotation, and / or translation, and / or cropping of the field of view of the object (130); generating data representing the transformation (140); The method according to any one of claims 1 to 4, generating a simulated cone-beam computed tomography image (120) based on one of the first computed tomography image (111) and the second computed tomography image (112), wherein the data representing the computed tomography image (110) comprises the first (111) computed tomography image and / or the second (112) computed tomography image; generating a set of training data, the set of training data comprising the simulated cone-beam computed tomography image (120) based on one of the first computed tomography image (111) and the second computed tomography image (112), the other of the first computed tomography image (111) and the second computed tomography image (112), and data representing the transformation (140); and The set of training data sets is provided.

6. The method according to claim 5, wherein: The step of generating data representing a second computed tomography image (112) of the object (130) includes receiving data representing the second computed tomography image (112) of the object (130) acquired by a computed tomography scanner.

7. The method according to claim 6, wherein: The step of generating data representing the transformation (140) comprises registering the first computed tomography image (111) to the second computed tomography image (112).

8. The method according to claim 7, wherein: The registration of the first computed tomography image (111) to the second computed tomography image (112) is performed using a conventional registration algorithm or an AI-based registration algorithm.

9. The method according to claim 5, wherein: The step of generating data representing a second computed tomography image (112) of the object (130) comprises applying an artificial transformation (140) to the data representing the first computed tomography image (111) of the object (130), thereby generating data representing the second computed tomography image (112) of the object (130).

10. The method according to claim 9, wherein: The step of generating data representing the transformation (140) includes receiving data representing the artificial transformation (140).

11. A computer-implemented method for registering a computed tomography image to a cone-beam computed tomography image, the method comprising the steps of: receiving data representing a computed tomography image (110) of an object; receiving data representing a cone-beam computed tomography image of the object; determining, using the artificial intelligence module (200), a transformation (141) required to register the computed tomography image to the cone-beam computed tomography image, wherein the artificial intelligence module (200) is trained using training data generated by the method according to any one of claims 5 to 10; registering the computed tomography image to the cone-beam computed tomography image according to the determined transformation (141); and The computed tomography image is provided in registration with the cone-beam computed tomography.

12. A computer-implemented method for segmenting a cone-beam computed tomography image, the method comprising the steps of: receiving data representing a cone-beam computed tomography image of a subject acquired by a cone-beam computed tomography scanner; segmenting the cone beam computed tomography image using an artificial intelligence module (200), wherein the artificial intelligence module (200) is trained using training data generated by the method according to any one of claims 5-10 or using training data comprising a plurality of simulated cone-beam computed tomography images generated by the method according to any one of claims 1 to 4; and A segmented cone-beam computed tomography image (150) is provided.

13. The method according to claim 12, wherein: The training data includes a plurality of computed tomography images acquired using a computed tomography scanner and / or a plurality of cone-beam computed tomography images acquired using a cone-beam computed tomography scanner.

14. The method according to any one of claims 11 to 13, wherein: The artificial intelligence module (200) is trained using the data training using a supervised or semi-supervised training algorithm.

15. A computer program comprising instructions which, when said program is run by a computer, cause said computer to perform the steps of the method according to any one of claims 1 to 14.

Images