Generating synthetic electron density images from magnetic resonance images

Abstract

To provide a conversion device operable to perform a learning-based method of generating a synthetic electron density image (sCT) of an anatomical portion based on one or more magnetic resonance, MR, images.SOLUTION: A method is processing-efficient and capable of producing highly accurate sCT images irrespective of misalignment in the underlying training set. The conversion device (20) receives and installs a machine-learning model (22) trained to predict coefficients of an image transfer function (24). The conversion device then receives a current set of MR images (MRI) of the anatomical portion, computes current coefficients ([C]) of the image transfer function by operating the machine-learning model on the current set of MR images (MRI), and computes a current sCT image of the anatomical portion by operating the current coefficients ([C]), in accordance with the image transfer function, on the current set of MR images (MRI).SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates generally to the field of medical imaging, and more particularly to techniques for generating synthetic electron density images from magnetic resonance images, for example for use in external radiation therapy planning. [Background technology]

[0002] Background technology In conventional planning of radiation therapy, e.g., for cancer, computed tomography (CT) images are acquired of a patient's anatomy and used as a basis for optimizing the treatment plan. CT or "CT scan" refers to a medical imaging technique that measures the attenuation of radiation, typically X-rays, as it passes through an object from various angles and reconstructs a 2D or 3D image (a "tomogram") of the object based on the measured attenuation. The resulting CT image values ​​thus represent how the object responds to radiation and can be directly converted to electron density, e.g., so-called Hounsfield units (HU), which can be used to calculate the radiation dose.

[0003] Treatment planning also requires imaging the target region for radiation therapy, e.g., one or more tumors with a small margin. It is often necessary to image so-called organs at risk (OARs), which are sensitive and essential organs that must be exposed to minimal radiation. A drawback of CT is that soft tissue structures are difficult to separate in CT images. Therefore, imaging using CT images alone inevitably involves guesswork and large margins. To alleviate this problem, modern radiation therapy often includes MRI scans as an adjunct. MRI (Magnetic Resonance Imaging) is an imaging technique that uses strong magnetic fields, magnetic field gradients, and radio waves to generate 2D or 3D cross-sectional images of internal organs and structures. MRI does not contain ionizing radiation and provides excellent soft-tissue contrast, allowing for more accurate target and structure imaging. However, because MR images do not contain electron density information, radiation treatment planning traditionally relies on a combined CT / MRI workflow. This workflow is costly, time-consuming, and causes unnecessary discomfort to the patient. Moreover, CT itself imposes a significant radiation dose on the patient, which limits its repeated use on patients.

[0004] To overcome these shortcomings, MRI-only workflows have been proposed in which MRI data are processed by algorithms to artificially generate electron density images, known as "pseudo-CT," "surrogate CT," or "synthetic CT (sCT)," thereby eliminating the need for CT scans. One major challenge in this situation is the lack of correspondence between individual pixel intensities in MR images and the associated radiation attenuation properties of tissues. MRI intensity is instead correlated with the tissue's proton density and magnetic relaxation. This leads to ambiguity between multiple tissues in MR images. For example, bone and air both appear dark in MR images, even though they have significantly different radiation attenuation properties.

[0005] Known techniques for sCT can be roughly divided into three categories: atlas-based approaches, segmentation-based approaches, and learning-based approaches.

[0006] Atlas-based approaches use a database of pre-matched MR and CT images, referred to as MR and CT atlases or MR and CT templates. In this case, an MR atlas from the database is registered by being transformed with the incoming new MR images, and then a similar transformation is applied to the combined CT atlas to generate an sCT image. Atlas-based approaches are generally quite time-consuming.

[0007] A segmentation-based approach is exemplified in the article "A dual-model HU conversion from MRI intensity values ​​within and outside a bone segment for MRI-based radiotherapy treatment planning of prostate cancer" by Korhopronen et al., published in Med. Phys. 41, 011704 (2014). Conversion models for soft and bone tissues are determined by comparing bulk intensity values ​​within clearly defined regions of interest (ROIs) in MR and standard CT images, respectively, e.g., by regression analysis. The incoming MR image is then manually segmented, and individual conversion models are applied to each segment to convert the MR image into an sCT image. This approach relies on tissue segmentation, which can be difficult to perform automatically with high accuracy on MR images. Furthermore, dividing the anatomical structure into several tissue types results in a rough approximation of the actual anatomical structure.

[0008] Learning-based approaches involve training and using machine learning methods to directly translate from a first medical imaging modality to a second. A technique for training a convolutional neural network (CNN) to translate MR images into sCT images is disclosed in the article "MR-based synthetic CT generation using a deep convolutional neural network method" by Xiao Han, published in Med. Phys., 44(4), pp. 1408-1419 (2017). A similar disclosure of direct mapping between images is presented in International Publication No. WO 2018 / 048507. In fact, this technique has been well known in the scientific community since before 2017. For example, a technique for training a convolutional neural network (CNN) to convert MR images into PET images is disclosed in the article "Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis" by Li et al., published in Med Image Comput Comput Assist Interv. 2014;17(0 3):305-312. Also, a method for using machine learning techniques to convert MR images into estimated CT images is disclosed in the article "Estimating CT Image from MRI Data Using Structured Random Forest and Auto-context Model" by Huynh et al., published in IEEE Trans Med Imaging. 2016;35(1):174-183, and in the article "Generation of brain pseudo-CTs using an undersampled, single-acquisition UTE-mDixon pulse sequence and unsupervised clustering" by Su et al., published in Med. Phys. 42(8), August 2015.

[0009] Learning-based approaches typically use a large training set of matched MR and CT images to train an artificial neural network (ANN) or similar to generate sCT images based on incoming MR images. During training, the ANN fits an unstructured nonlinear function to each pixel based on a finite neighborhood (receptive field) around the pixel, based on the entire training set. This means that small errors in matching between MR and CT images in the training set will result in the loss of fine details in the sCT. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] International Publication No. 2018 / 048507 [Non-patent literature]

[0011] [Non-Patent Document 1] “A dual model HU conversion from MRI intensity values ​​within and outside a bone segment for MRI-based radiotherapy treatment planning of prostate cancer”, Korhopronen et al., Med.Phys.41,011704 (2014) [Non-patent document 2] “MR-based synthetic CT generation using a deep convolutional neural network method”, Xiao Han, Med.Phys., 44(4), pp. 1408-1419 (2017) [Non-patent document 3] “Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis”, Li et al., Med Image Comput Comput Assist Interv.2014;17(0 3):305~312 [Non-patent document 4] "Estimating CT Images from MRI Data Using Structured Random Forest and Auto-context Model," Huynh et al., IEEE Trans Med Imaging. January 2016;35(1):174-183 [Non-patent document 5] “Generation of brain pseudo-CTs using an undersampled, single-acquisition UTE-mDixon pulse sequence and unsupervised clustering”, Su et al., Med.Phys.42(8), August 2015 Summary of the Invention [Problem to be solved by the invention]

[0012] Summary of the Invention SUMMARY OF THE INVENTION It is an object of the present invention to at least partially overcome one or more limitations of the prior art.

[0013] A further problem is to provide a technique for generating a synthetic electron density image based on MR images of an anatomical region.

[0014] A still further challenge is to provide such a technique that can be automated.

[0015] Another challenge is to provide a technique capable of producing synthetic electron density images with a high level of detail and accuracy. [Means for solving the problem]

[0016] One or more of these objects, as well as further objects that may become apparent from the following description, are at least partly achieved by a computer-implemented method and apparatus for generating a synthetic electron density image of an anatomical site, a machine learning model, a computer-implemented method for providing a trained machine learning model, and a computer-readable medium as set forth in the independent claims, embodiments of which are specified by the dependent claims.

[0017] Further objects, as well as features, aspects and technical advantages of the present invention will become apparent from the following detailed description, the appended claims and the drawings.

[0018] Embodiments of the present invention will now be described in more detail with reference to the accompanying schematic drawings. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a block diagram of an exemplary MRI-only imaging system. [Figure 2] FIG. 1 is a block diagram of an imaging system according to some embodiments of the present disclosure. [Figure 3A] FIG. 3 is a block diagram of a system for training a machine learning model used in the imaging system of FIG. 2. [Figure 3B] FIG. 3B is a detailed block diagram of a training device in the system of FIG. 3A. [Figure 3C] FIG. 3B illustrates training data for use in the system of FIG. 3A. [Figure 4A] FIG. 1 shows a simplified example of MR and CT images used for training and the resulting sCT images generated by prior art techniques. [Figure 4B] 1A-1C show simplified examples of MR and CT images used for training and the resulting sCT images generated by methods according to some embodiments of the present disclosure. [Figure 4C] 1A-1C show simplified examples of MR and CT images used for training and the resulting sCT images generated by methods according to some embodiments of the present disclosure. [Figure 5] FIG. 1 is a block diagram of an imaging system according to some embodiments of the present disclosure. [Figure 6] 1 is a flowchart of a method for generating an sCT image from an MR image according to some embodiments of the present disclosure. [Figure 7] FIG. 7 is a functional block diagram of one implementation of the method in FIG. 6. [Figure 8A] 7 is a flowchart of a method for training a machine learning model for subsequent use in the method of FIG. 6. [Figure 8B] FIG. 8B is a functional block diagram of an implementation of the method in FIG. 8A. [Figure 8C] FIG. 8B is a functional block diagram of an implementation of the method in FIG. 8A. [Figure 9] FIG. 1 is a block diagram of an apparatus capable of implementing methods for generating sCT images and / or training machine learning models according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0020] DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS Embodiments of the present invention will now be described in more detail below with reference to the accompanying drawings, in which some, but not all, embodiments of the present invention are shown. Indeed, the present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like numbers refer to like elements throughout.

[0021] It will also be understood that, wherever possible, any advantage, feature, function, device and / or operational aspect of any of the embodiments of the present invention described and / or contemplated herein may be included in any other embodiments of the present invention described and / or contemplated herein, and / or vice versa. Additionally, wherever possible, any term expressed in the singular herein is intended to include the plural, and vice versa, unless expressly stated otherwise. As used herein, "at least one" shall mean "one or more," and these phrases are intended to be interchangeable. Thus, although the phrases "one or more" or "at least one" are also used herein, the indefinite article singular shall mean "at least one" or "one or more." In this specification, unless the context otherwise requires to express an essential linguistic or extralinguistic meaning, the term "comprise" or variations thereof, such as the third person singular or the present continuous tense, is used in an inclusive sense, i.e., to specify the presence of stated features, but not to exclude the presence or addition of further features in various embodiments of the invention. In this specification, a "set" of several items is intended to imply comprising one or more items.

[0022] Furthermore, although terms such as "first," "second," etc. may be used herein to describe various elements, it should be understood that these elements are not limited by these terms. These terms are merely used to distinguish one element from another. For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of the present invention. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0023] As used herein, a "synthetic electron density image" refers to any type of image that is computationally generated to include signal values ​​("intensity values") directly related to electron density. Such an image may depict, replace, or complement an electron density image generated by computed tomography (CT) using radiation of any wavelength range, including but not limited to X-ray radiation. For simplicity in the following description, a synthetic electron density image will be referred to as an "sCT image."

[0024] As used herein, "MR image" refers to any type of image produced by a device configured to detect nuclear magnetic resonance (NMR) signals, such as an MR scanner of any configuration. In the figures, MR images are represented by "MRI."

[0025] As used herein, an "image" can be two-dimensional (2D) or three-dimensional (3D). A 3D image corresponds to a stack of spatially adjacent 2D images, commonly referred to in the field of tomographic imaging as an "image stack." Each 2D image corresponds to a cross-sectional slice of the object being imaged.

[0026] As used herein, a "pixel" is an image element and is associated with a pixel value. In a 2D image, the location of a pixel can be defined, for example, by an (x, y) value in a 2D regular grid having a fixed location relative to the respective 2D image. In the case of a 3D image, a pixel is also known as a "voxel," and the location of a pixel can be defined, for example, by an (x, y, z) value in a 3D regular grid having a fixed location relative to the respective 3D image. As used herein, a pixel can correspond to the smallest individual element or group of such elements in an image.

[0027] As used herein, a "matrix" is an array of any dimension. An array contains elements that can be assigned individual values. For example, a matrix can have two or three dimensions. The term "matrix" is sometimes referred to as a "tensor" in the context of this disclosure.

[0028] As used herein, "machine learning model" refers to any type of predictive model that can be trained to predict a numerical value, including, but not limited to, artificial neural networks (ANNs), such as deep learning networks, recurrent neural networks, and the like, as well as support vector machines (SVMs) and Bayesian networks.

[0029] In the following description, a machine learning model is denoted by MLM, where a trained MLM is denoted by the subscript T(MLM T ), and untrained MLMs are indicated by an asterisk (MLM*). In this context, the process of running a trained MLM on new input data to produce results is commonly referred to as "inference."

[0030] Embodiments of the present invention generally relate to the field of generating composite electron density information from one or more MR images. Forming composite electron density information can be particularly useful for MR-based radiation therapy planning. However, it should be recognized that it can be used in other applications as well. For example, the composite electron density information can be used as attenuation information for a positron emission tomography (PET) camera or a single photon emission computed tomography (SPECT) camera. The composite electron density can also constitute proton stopping power information for use in proton therapy planning.

[0031] The embodiments are based on the insight that applying an "optimized" image transfer function to one or more MR images could improve the level of detail and accuracy in the composite electron density image compared to existing techniques. The embodiments are also based on the insight that a suitably trained machine learning model can provide such an "optimized" image transfer function, and thus this function can be trained to parameterize the image transfer function for an incoming MR image set, such that a composite electron density image is obtained by applying the parameterized image transfer function to the MR image set.

[0032] Thus, unlike traditional learning-based approaches, the trained machine learning model does not directly output a synthetic electron density image, but rather parameterizes an image transfer function that is tuned to generate a synthetic electron density image when acted upon the input to the machine learning model. Also, as further illustrated below, the machine learning model can be trained to parameterize the image transfer function without requiring segmentation of the incoming MR image, thereby increasing the versatility of the technique and enabling it to be automated and therefore performed without subjective human input.

[0033] FIG. 1 shows a schematic diagram of an MRI-only imaging system including an MRI device 10 configured to generate one or more MR images of a patient's anatomy (not shown). The MRI device 10, commonly known as a scanner, is well known in the art and will not be further described. A conversion device 20 is configured to receive and process the MR images and generate and output sCT images. In this case, the sCT images can be analyzed together with the MR images for radiation therapy planning, as exemplified in the Background section. Alternatively or additionally, the sCT images can be analyzed for any other purpose in which they replace CT images that would have been acquired by a conventional CT device.

[0034] 2 is a more detailed diagram of an MRI-only imaging system according to one embodiment. In the illustrated example, a transform device 20 is configured to retrieve MR images from a storage device or storage 100 that stores a plurality of MR images [MRI] generated by an MR device 10. The transform device 20 includes a coefficient generator 21 that generates a trained machine learning model MLM T , 22 operates on the incoming MR image to generate and output a current set of coefficients [C] of a predefined image transfer function ITF. The ITF is a parametric function that takes one or more pixel values ​​in the MR image as inputs (independent variables) and generates pixel values ​​in the sCT image (dependent variables). As used herein, "coefficients" refer to any unknown values ​​needed to complete the ITF so that the dependent variable can be calculated based on the independent variables. In other words, coefficients are applied to parameterize the ITF. According to one embodiment, the ITF is a function of k·I p +m, where I p is the pixel value in the incoming MR image, and k,m are coefficients. An affine ITF has been found to provide a reasonable compromise between computational complexity and accuracy of the resulting sCT image. In general, however, the ITF can be any suitable function, including but not limited to polynomial functions of any degree.

[0035] In the following example, it is assumed that the same ITF is applied to all pixels in an incoming MR image, but the coefficients may vary between pixel locations within the MR image. Thus, the above-mentioned affine ITF can be represented by K·MRI+M, where K and M are matrices that define the coefficients applied to individual pixel values ​​in the MR image MRI. Thus, for this example, the coefficient set [C] includes a K matrix and an M matrix.

[0036] It should be noted that the conversion device 20 can operate on two or more MR images to generate an sCT image. According to one embodiment, two or more MR images can be generated during one acquisition sequence of the MR device 10 to highlight different tissues of the patient, for example, so-called Dixon images that separately highlight fat and water. Such MR images are known in the art as "multi-channel" MR images. According to another embodiment, different acquisition sequences of the MR device 10 can generate MR images with different image contrasts, for example, so-called T1-weighted and T2-weighted images. Any two or more MR images from one or more acquisition sequences can be processed together by the conversion device 20 to generate an sCT image.

[0037] Transformation device 20 further includes an image transformer 23 configured to receive [C] from coefficient generator 21 and apply [C] to parameterize ITF 24. Image transformer 23 is further configured to receive a current MR image, i.e., the same MR image processed by coefficient generator 21, and apply the now parameterized ITF to the MR image to generate a current sCT image. In general, with a parameterized ITF, image transformer 23 can be thought of as applying the current coefficients [C] to the current MR image in accordance with the ITF.

[0038] FIG. 3A shows a trained machine learning model MLM for use in the conversion device 20 of FIG. T A training system for generating a trained machine learning model (MLM) is shown. TThe machine learning model includes a training device 30 configured to retrieve matched MR and CT images from a storage device 200 for use in generating a training set. As shown, the storage device 200 stores a plurality of MR images [MRI] acquired by an MRI device 10 and a corresponding plurality of real CT images [CT] acquired by a conventional CT device 12. These images [MRI], [CT] constitute a so-called training set. FIG. 3C illustrates such a training set 210, which includes N matched MR and CT images. However, as can be appreciated from the foregoing, it is also contemplated that more than one MR image, e.g., a multi-channel MR image, may be matched to a CT image in the training set and thus used to train the machine learning model. Furthermore, although FIG. 3C illustrates the training set 210 as including matched 2D images, the training set 210 could equally well include matched 3D images. Indeed, it is currently believed that improved performance can be achieved by training a machine learning model on the matched 3D images with respect to the level of detail and accuracy of the sCT images generated by conversion device 20 (FIG. 2), because the machine learning model is thereby trained to also consider structure in the z-direction of the MR and CT images. When such a trained machine learning model is applied, improved performance of conversion device 20 can be achieved for either 2D or 3D MR images.

[0039] The training device 30 can operate on a large number of matched MR and CT images acquired from one or more patients. Typically, the machine learning model can be trained on tens or even hundreds or more matched 2D or 3D images. The images in the training set can be acquired for selected anatomical regions of the patient, such as the head, torso, abdomen, extremities, etc., so that the machine learning model can be trained to operate specifically on MR images acquired from the selected anatomical region.

[0040] FIG. 3B is a more detailed diagram of training device 30 according to one embodiment. According to the illustrated example, training device 30 is configured to retrieve a current data set consisting of matched MR and CT images, e.g., from storage device 200 (FIG. 3A). Training device 30 includes a coefficient predictor 31 having a machine learning model MLM* 32 to be trained. Coefficient predictor 31, which corresponds to coefficient generator 21, is configured to modify model parameters of MLM* in response to feedback data FB* (see below) and output a predicted coefficient set [C]*. Training device 30 further includes an image transformer 33, which corresponds to image transformer 23 and is configured to receive and apply [C]* to parameterize an ITF 34 identical to ITF 24 in transformer 20. Image transformer 33 is further configured to receive MR images of the current data set and apply the currently parameterized ITF to the MR images to generate a current predicted sCT image, denoted sCT* in FIG. 3B. The prediction evaluator 35 is configured to apply a correction algorithm to generate feedback data FB*. According to one embodiment, the correction algorithm operates to optimize a cost function that receives as input the sCT* and CT images of the current data set. As will be further illustrated below, the cost function can operate on further input data, such as [C]* or portions thereof.

[0041] According to the embodiment of FIG. 3B, the training device 30 automatically trains the MLM* based on the training set to learn the parameters or weights of the MLM*. The parameters can be iteratively trained according to an optimality criterion defined, for example, by a cost function. Specifically, the training device 30 can be considered to train the MLM* until a similarity criterion is met between the CT image, which can be considered as a reference image, and the sCT* image. The training device 30 then prepares the trained machine learning model MLM* for later incorporation into the transformation device 20 (FIG. 2). T can be output (Figure 2).

[0042] According to one non-limiting example, the machine learning model is a neural network for deep learning, such as a convolutional neural network (CNN). Such a neural network includes a stack of separate layers configured to transform inputs into outputs. These layers can differ in input size, output size, and the relationship between inputs and outputs for the layers. Each layer can be connected to one or more upstream and downstream layers in the stack of layers. For deep learning, convolutional neural networks typically have two or more stages of nonlinear feature transformation.

[0043] 4A-4C, we will now provide a simplified qualitative discussion of some of the technical advantages of generating and using parameterized ITFs in accordance with the above description.

[0044] FIG. 4A is intended to illustrate the prior art of direct inter-image mapping from MR images to sCT images using a trained neural network, e.g., as described in the Background section. At the top of FIG. 4A, matched MR and CT images are shown, each containing individual pixels p1 and p2, with exemplary signal values ​​("intensity values"). Both the MR and CT images exhibit an inverted-L structure, where pixel value 1 occurs against a background of pixel value 0 in the MR image (top left), and pixel value 1000 occurs against a background of pixel value 100 in the CT image (top right). A slight horizontal misalignment of one pixel is introduced between the MR and CT images. For example, if a neural network is trained based on multiple matched MR and CT images containing small misregistrations, as represented in Fig. 4A by three additional instances of matched MR and CT images, the trained neural network may generate the sCT image shown at the bottom of Fig. 4A based on the MR image shown at the top left. From the signal value of pixel p3 in the sCT image, the inverted-L structure is significantly blurred due to the misregistration, as indicated by the thick dashed line.

[0045] According to one embodiment shown in Figures 4B-4C, a machine learning model was trained based on a number of matched MR and CT images of the type shown in Figure 4A, i.e., with slight misalignment with each other, to generate the coefficients K and M of ITF = K·MRI + M. According to Figure 4B, MR images representing the inverse L structure were used to generate the coefficients K and M of the trained machine learning model MLM. T , and the model outputs [C] in the form of coefficient matrices or maps, i.e., K and M matrices. Each matrix contains elements pK, pM, which hold coefficient values ​​("pixel coefficients") and correspond to individual pixels p1 in the MR image. Thus, the MLM Tgenerates K and M coefficients for each pixel p1 in the incoming MR image. According to FIG. 4C, the ITF is parameterized by the K and M matrices and applied to the same MR image input in FIG. 4B, resulting in the sCT image shown on the right. The pixel value of each pixel p3 in the sCT image is calculated by applying the coefficient values ​​of corresponding elements pK and pM in the K and M matrices to the corresponding pixel value p1 in the MR image according to the ITF. From the signal value of pixel p3 in the sCT image, the inverted-L structure is reproduced in the sCT image without blurring, as indicated by the thick line. As can be seen, the generation of sCT images is robust to misregistration between images in the training set. In other words, the parameterized ITF enhances the propagation of details from the incoming MR image to the sCT image.

[0046] The embodiment in FIGS. 4B-4C uses the MLM algorithm to generate [C] using the same coefficient values ​​for all pixels in the incoming MR image. T The present invention is simplified in that it is trained on a matrix of coefficients [C]. Applicant has found that significantly improved results can be obtained by allowing the coefficient values ​​to vary spatially within the image, i.e., between adjacent pixels in [C]. This means that during training, the machine learning model MLM* is enabled to adapt the coefficient values ​​of the ITF to corresponding pixel values ​​in the matched MR and CT images. To further stimulate the ITF's efforts to propagate detail from the MR image to the sCT image, it can be advantageous to apply constraints to the local variations in the coefficient values ​​within the individual coefficient matrices. When the MLM* is trained without such constraints, the resulting MLM* will exhibit a large variation in coefficient values ​​even as long as the coefficient values ​​of adjacent elements of the individual coefficient matrices are largely independent of each other. Tmay produce a coefficient matrix with high variance, which is inconsistent with the physical environment in which the MR images are taken and with the known properties of MR images. It is well known that MR images may have artifacts in the form of slowly varying background components of image inhomogeneity ("bias fields") and non-standard variations in the MR image intensity grayscale. By way of a non-limiting example, a slowly varying bias field may have a spatial scale of approximately 10 cm between its maximum and minimum values. Therefore, artifacts may cause the same tissue to produce different intensity values ​​in different parts of the MR image. Applying a local variance constraint in training ensures that the MLM is trained such that the individual coefficient values ​​vary slowly between matrix elements in the individual coefficient matrices, approximately corresponding to the image inhomogeneity between pixels p1 in the incoming MR image due to artifacts. T produces [C]. As a result, the parameterized ITF can at least partially compensate for image inhomogeneities in the incoming MR images when producing sCT images.

[0047] For example, local variation constraints can be set such that the coefficient coefficient (CV) of the coefficient values ​​in an individual coefficient matrix is ​​significantly smaller than the CV of the intensity values ​​in a typical MR image, where the CV is determined within a local region of predefined extent around each element / pixel in the coefficient matrix / MR image. For example, this local region can include all pixels / elements within a one-pixel distance from the individual element / pixel, e.g., neighboring pixels in the x, y, and z directions.

[0048] The local variation constraints can be set differently between coefficient matrices, or they can be different within a coefficient matrix, i.e., they can vary between matrix elements (pixel positions).Furthermore, the local variation constraints may not be applied to all coefficient matrices.

[0049] In general, the parameterized ITF can be viewed as a regression function that is fitted pixel-by-pixel between the MR and CT images in the training set to generate individual pixel values ​​in the sCT image based on the corresponding pixel values ​​in the incoming MR image.

[0050] The preceding discussion demonstrates that embodiments of the present invention can reduce the effects of misregistration between MR and CT images in a training set, thereby achieving a high level of detail and accuracy in the resulting sCT images. Furthermore, the images used to train the MLM* or the MLM T There is no need for pre-segmentation of the input MR images, and therefore both training and generation of the sCT can be fully automated without sacrificing accuracy, although pre-segmentation may be used if desired.

[0051] Another technical advantage is that sCT images can be generated with the same pixel resolution as the incoming MR images. In conventional techniques that use neural networks to perform direct image-to-image mapping from MR images to sCT images, in order for the neural network to operate efficiently, the neural network must be trained on MR images and CT images resampled to a standardized pixel size. The standardized pixel size may be determined by the pixel size of the training set or by hardware constraints, because a smaller standardized pixel size requires more computing resources and further limits the field of view and / or depth of the neural network. Therefore, in conventional techniques, the incoming MR images may need to be resampled to the standardized pixel resolution before estimation in the trained neural network, and the resulting sCT images are generated with the standardized pixel size. If the incoming MR images have a pixel size smaller than the standardized pixel size, i.e., more detail, this additional level of detail will be irreversibly lost in the resampling process. According to an embodiment of the present invention, the machine learning model is decoupled from the resulting sCT image and instead generates the current [C]. This allows the pixel size of the current [C] to be changed, for example, by interpolation, before the ITF is parameterized and applied to the incoming MR image, without or only slightly affecting the ability of the parameterized ITF to transfer the details of the incoming MR image to the sCT image, especially when the parameterized ITF is a slowly varying function. In the case of the affine ITF example given above, K and M can be interpolated to increase the pixel resolution of the parameterized ITF by adding one or more element values ​​between existing elements in the respective coefficient matrices K and M. Any interpolation function can be used, including, but not limited to, a linear function.In this disclosure, the process of increasing pixel size (and thus decreasing pixel resolution) is referred to as "downsampling," and the process of decreasing pixel size (and thus increasing pixel resolution) is referred to as "upsampling."

[0052] A corresponding embodiment is illustrated in FIG. 5, where the pixel resolution of the incoming MR images MRI is determined by a trained machine learning model MLM T 5, the pixel resolution of the MR image is higher than the predefined pixel resolution of the MRI image. To adapt the pixel resolution of the incoming MR image to the predefined pixel resolution, a downsampling device 21A is arranged upstream of the coefficient generator 21. The coefficient generator 21 operates on the downsampled MR image to generate a set of coefficients of the predefined resolution. In the case of FIG. 5, the downsampled MR image and the corresponding set of coefficients are respectively S and [C] S It is represented by [C] S To match the resolution of the coefficients [C] to the pixel resolution of the incoming MR image, an upsampling device 21B is placed upstream of the image converter 23. The image converter 23 then operates on the upsampled coefficient set [C] to generate an sCT image with the same pixel resolution as the incoming MR image.

[0053] As can be seen from the above, the set of coefficients generated by coefficient generator 21 can be upsampled or interpolated to any desired resolution before being input to image converter 23 without substantial penalty to the level of detail in the resulting sCT image. Tcan be configured to operate on a predefined or standardized pixel size, which is significantly larger than the pixel size used in traditional direct image-to-image mapping with neural networks. The ability to train and operate on larger pixels frees up computing resources. Thus, for a given computing resource, embodiments of the present invention enable the deployment of machine learning models with larger fields of view and deeper layers, thereby further improving the performance of sCT image generation.

[0054] The technique illustrated in Figure 5 can similarly be implemented in the training device 30 of Figure 3B, for example by placing a downsampling device upstream of the coefficient predictor 31 and an upsampling device upstream of the image converter 33, which then operates on the upsampled [C]* to generate an sCT* image with pixel resolution equivalent to the incoming MR image.

[0055] It should be noted here that, for example, the resolution of the incoming MR image can be increased by MLM. T It is equally possible to upsample this MR image to match the resolution of the input MR image, and to downsample the set of coefficients [C] to generate an sCT image at any desired resolution, for example, at the same pixel resolution as the incoming MR image. In general, a first resampling can be performed on the incoming MR image and a second resampling can be performed on the set of coefficients, where the first and second resamplings can differ, for example, by changing the resolution in different directions.

[0056] An exemplary method for generating an sCT image according to one embodiment is shown in Figure 6. This method can be implemented, for example, by the conversion device 20 of Figure 2 or Figure 5. In step 601, a trained machine learning model MLMT In step 602, a current MR image is received, for example, from an MRI device (see 10 in FIG. 2) or from storage (see 100 in FIG. 2). The current MR image can be stored in computer memory.

[0057] In optional step 603, the current MR image may be preprocessed. Such preprocessing is known in the art and may include normalization with respect to any one of registration, resolution, and intensity value distribution. Alternatively or additionally, step 603 may include the above-mentioned resampling of the MR image to a predefined pixel resolution. Step 603 may also include removing or masking certain portions within the MR image. Step 604 optionally preprocesses the current MR image to generate a trained machine learning model MLM T This will generate a trained machine learning model MLM T calculates the coefficient set [C] for the ITF. Thus, referring to Figures 4B-4C, step 604 can substitute the current pixel coefficients in elements corresponding to pixels in the current MR image into coefficient matrices K, M. The coefficient set [C] can then be stored in computer memory.

[0058] In optional step 605, the set of coefficients [C] is post-processed, for example, by upsampling and / or by low-pass filtering as described above. In step 606, a current sCT image is calculated by applying the resulting set of coefficients [C] to the current MR image according to the (therefore parameterized) ITF. In optional step 607, the current sCT image can be post-processed, for example, by removing or masking certain portions, signal enhancement, noise reduction, etc. In step 608, the current sCT image is output, optionally together with the current MR image, for example, for storage in memory and / or for display.

[0059] As already mentioned above, the current sCT can be calculated based on two or more MR images that can be generated for different settings of the MRI device, e.g., for different sequences or contrasts. As is known in the art, further MR images can be generated by combining multiple MR images, e.g., by calculating the sum or difference between multiple MR images. According to the exemplary embodiment shown in FIG. 7, the coefficient generator 21 uses a machine learning model MLM trained to calculate a coefficient set for a four-channel MR image. T In FIG. 7, the MR images of the various channels are represented by A1 to A4 and are depicted as 3D images, i.e., individual stacks of 2D images. According to one non-limiting example, the A1 image can be acquired to enhance fat, the A2 image can be acquired to enhance water, the A3 image can be calculated as A1+A2, and A4 can be calculated as A1-A2. According to the illustrated example, the coefficient generator 21 outputs one K matrix per channel and one M matrix common to all channels. The K matrices for the various channels are represented by K1 to K4. Since the incoming MR images are 3D images, the resulting K and M matrices are also 3D matrices. Instead of one common M matrix, the coefficient generator 21 can be configured to output one M matrix per channel. The image converter 23 then applies the K and M matrices to the images A1 to A4 according to the affine ITF to generate sCT images, which are also 3D images.

[0060] 7 is given as a non-limiting example only. For the affine ITF example, the trained machine learning model MLM T Generally, Nin incoming MR images are taken and Nout coefficient matrices are calculated.

number

number

[0061] For example, an affine ITF acting on a coefficient matrix is

number

number

number

[0062] FIG. 8A illustrates an exemplary method for training a machine learning model MLM* according to one embodiment. This method may be implemented, for example, by the training device 30 in FIGS. 3A-3B. As will be appreciated from the foregoing, this method is performed to learn model parameters of the MLM* for a particular ITF based on training images including single-channel or multi-channel MR images of an anatomical region of a patient and corresponding CT images obtained by CT scanning of the same or similar anatomical region of the patient. The output of this method is referred to as a trained machine learning model MLM*. T is.

[0063] In step 801, a training set (see 210 in FIG. 3C) consisting of matched MR and CT images is input and stored in memory (see 200 in FIG. 3A). In step 802, the MLM* is initialized, which may include an iteration index, initialization of starting parameters for the MLM*, etc. Steps 803-808 may be performed repeatedly until one or more predefined stopping criteria are met (e.g., until the iterative process converges according to predefined criteria). In step 803, training images are selected, e.g., randomly, from the training set, where the training images include matched MR and CT images. Step 803 may also include preprocessing of the training images. Step 804 calculates a predicted coefficient set [C]* according to the current model parameters of the MLM* for the training images. Step 805 calculates the sCT* image defined by the ITF when parameterized by [C]* and applied to the MR images in the training images. Step 806 evaluates the cost function. For example, the cost function may determine the difference between the CT and sCT* images in the training images. Such difference may correspond, for example, to an error map indicating the difference in pixel values ​​between the two images. This difference may be quantified by any known regression loss function, including, for example, calculations of mean absolute error (MAE), mean squared error (MSE), Huber loss, log-cosh loss, etc. Step 807 determines or updates the MLM* model parameters based on the evaluation in step 806, for example, by optimizing (minimizing or maximizing) the cost function with respect to the MLM* model parameters. In the case of a neural network embodiment, the model parameters may include weights and offset values, as is known in the art. Step 807 may use any conventional update algorithm, including, but not limited to, backpropagation, stochastic gradient descent (SGD), adaptive moment estimation (Adam), BFGS, Rprop, etc.

[0064] Step 808 determines whether a stopping criterion has been met. Various stopping criteria can be used, such as a predetermined maximum number of iterations or a predetermined quality measure given, for example, by a cost function or other measure of difference between the sCT* and CT images. If step 808 determines that the stopping criterion has not been met, step 803 can be returned to. Otherwise, the MLM* is considered to be trained and the training method proceeds to step 809, which prepares the MLM* for later use by the generation method in FIG. 6. T Store or output.

[0065] As mentioned above, we apply variability constraints during training to the trained machine learning model MLM. T It has been found advantageous to suppress large pixel-by-pixel variance within the individual coefficient matrices generated by [C]*. Constraints can be applied at various training stages. According to one embodiment, the constraints can be embedded in the MLM* and thus applied during step 804. According to another embodiment, the constraints are applied as low-pass filtering of [C]* in step 804. According to yet another embodiment, the constraints are included in the cost function and thus applied in step 806. It should be appreciated that constraints can take many different forms. According to one embodiment, the constraints act to limit the variability within local regions of predefined ranges around individual pixels / elements in the individual coefficient matrices. The predefined ranges can be determined by testing.

[0066] Since training is an autonomous process, it is desirable to mitigate undesired results, which is achieved by applying constraints. Without constraints, there is a risk that the MLM* will be trained to suppress details from the MR image in the sCT image generated by applying the parameterized ITF.

[0067] According to one non-limiting example, the constraint is implemented as a total variance (TV) penalty in the cost function. The TV penalty can be calculated in each dimension for each coefficient matrix. For an example where the local region extends one pixel in each dimension of the 3D matrix B, the TV penalty can be defined as follows:

number

[0068] According to one embodiment, the MLM* can be trained with respect to the affine ITF defined in Equation 2 by including a penalty term ρ in the cost function.

number

[0069] For an embodiment in which the cost function involves calculating the MAE on the error map, Equation 2 and Equation 4 can be combined to form the following cost function L:

number

[0070] An exemplary embodiment of the training method of FIG. 8A is shown in FIG. 8B, which shows a training apparatus operating on single-channel 2D MR images and configured corresponding to FIG. 3B. The coefficient predictor 31 operates on the MR images to generate [C]* in the form of predicted K* and M* matrices (step 804). The image transformer 33 operates on the MR images with an ITF parameterized by K* and M* to generate a predicted sCT* image (step 805). The prediction evaluator 35 in FIG. 3B corresponds to the cost function calculator 35A and model parameter updater 35B in FIG. 8B. The cost function calculator 35A calculates the mean absolute error (MAE), K*(TV K ) TV penalty and M*(TV M ) and calculate the cost function terms including the TV penalty. K and TV M to become the current value of the cost function (step 806). The model parameter updater 35B calculates updated values ​​of the model parameters of MLM* to reduce the current value of the cost function in subsequent iterations (step 807).

[0071] 8C shows an alternative embodiment of the training device 30. This embodiment assumes that a target coefficient set has been pre-computed, for example, by running another machine learning model (not shown) on a target set of matched MR and CT images, or by other types of computational or estimation processes. The target coefficient set is denoted [C] in FIG. TAR, which may include, for example, one or more target K and M matrices. The target device 30 in FIG. 8C operates on a training set of MR images. As with the embodiment of FIG. 3B, the target device 30 includes the same coefficient predictor 31 that calculates a predicted coefficient set [C]*, but does not include the image transformer 33. Furthermore, the prediction evaluator 35 converts [C]* to [C] TAR and based on the evaluated cost function generates feedback data FB* which causes coefficient predictor 31 to update the model parameters of the MLM*. In contrast to the method in Figure 8A, training device 30 in Figure 8C implements a corresponding method, but in which step 805 is omitted and at least steps 806-807 are modified. Training device 30 in Figure 8C can also apply the above-mentioned constraints to, for example, the cost function.

[0072] In general, any or portions of the methods described herein can be implemented in a processing device by a combination of software and hardware circuitry, or solely by customized hardware circuitry. FIG. 9 is a block diagram of an exemplary structure of such a processing device 90. According to FIG. 9, the processing device 90 includes control circuitry 91 responsible for the overall operation of the processing device 90. As shown, the control circuitry 91 may include a processor 92, which may include a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, a microprocessor, an ASIC, an FPGA, or any other dedicated or general-purpose processing device, or any combination thereof. The processor 92 may execute instructions 94 stored in a separate memory, such as memory 93, and / or in an internal memory (not shown) of the control circuitry 91, for the purpose of controlling the operation of the processing device 90. The instructions 94, when executed by the processor 92, may cause the processing device 90 to perform any or portions of the methods described herein. As shown in FIG. 9, the memory 93 may also store data 95 for use by the processor 92. Depending on the implementation, the data 95 may include, for example, a memory 93, a memory 94 ... T ,MLM*,[MRI],[CT],[C] PRE The memory 93 may include one or more of the data items described above and / or illustrated in the drawings, such as, for example, a buffer, a flash memory, a hard disk drive, removable media, a volatile memory, a nonvolatile memory, a random access memory (RAM), or other suitable devices. According to an exemplary configuration, the memory 93 includes nonvolatile memory for long-term data storage and volatile memory that serves as system memory for the control circuitry 91. The memory 93 may exchange data with the control circuitry 91 via a data bus. Associated control lines and an address bus may also be provided between the memory 93 and the control circuitry 91. The memory 93 is considered a non-transitory computer-readable medium.

[0073] As shown in FIG. 9, instructions 94 can be provided to the processing unit 90 on a computer-readable medium 90A, which can be a tangible (non-transitory) product (e.g., magnetic media, optical disk, read-only memory, flash memory, etc.) or a propagated signal.

[0074] While the present invention has been described in connection with what are presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

[0075] Additionally, although the figures depict operations in a particular order, this should not be understood as requiring such operations to be performed in the particular order or sequential order shown, or as requiring all of the operations shown to achieve desired results. In certain environments, parallel processing may be advantageous.

[0076] Furthermore, all of the methods, devices, embodiments, examples, and aspects described above are equally applicable to generating sCT images based on raw images acquired by medical imaging devices using imaging modalities other than MRI, such as cone-beam computed tomography (CBCT), positron emission tomography (PET), single-photon emission computed tomography (SPECT), magnetic particle imaging (MPI), etc. It is also contemplated that an sCT image may be generated based on two or more raw images acquired by at least two different imaging modalities. According to one embodiment, a computer-implemented method for generating a synthetic electron density image of an anatomical region may include the following steps: receiving a machine learning model trained to predict coefficients of an image transfer function; receiving a current set of raw images of the anatomical region; calculating current coefficients of the image transfer function by applying the machine learning model to the current set of raw images; and calculating a current synthetic electron density image of the anatomical region by applying the current coefficients to the current set of raw images according to the image transfer function. At least one original image in the set of original images may be acquired using an imaging modality other than MRI, such as any one of CBCT, PET, SPECT, and MPI.

[0077] Below, a series of items are listed to summarize some aspects and embodiments of the invention disclosed in the foregoing description.

[0078] Item 1: 1. A computer-implemented method for generating a synthetic electron density image (sCT) of an anatomical site, comprising: A machine learning model (MLM) trained to predict the coefficients of the image transfer function T ) receiving step (601); receiving (602) a current set of magnetic resonance images of the anatomical region; The machine learning model (MLM TCalculating (604) the current coefficients ([C]) of the image transfer function by applying [C] to the current set of MR images; calculating (606) a current synthetic electron density image (sCT) of the anatomical region by applying the current coefficients ([C]) to the current set of MR images in accordance with the image transfer function; A method comprising:

[0079] Item 2: The machine learning model (MLM) is used to predict the coefficients. T ) to achieve a similarity criterion between a reference image (CT) and a synthetic electron density image (sCT) generated by applying the coefficients to an MR image corresponding to the reference image (CT) according to the image transfer function. Item 1. The computer-implemented method of item 1.

[0080] Item 3: the current coefficients ([C]) include one or more current pixel coefficients for each pixel location in the current MR image set; 3. The computer-implemented method of claim 1 or 2.

[0081] Item 4: The machine learning model (MLM T ) to allow the one or more current pixel coefficients to vary between pixel locations; Item 3. The computer-implemented method of item 3.

[0082] Item 5: the one or more current pixel coefficients vary between pixel locations; Item 3. The computer-implemented method of item 3 or 4.

[0083] Item 6: The machine learning model (MLM) is adapted to vary the one or more pixel coefficients between the pixel locations while respecting constraints on the variation of pixel coefficients within predefined regions around each of the pixel locations. T ) training, 6. The computer-implemented method of any one of items 3 to 5.

[0084] Item 7: calculating (606) the current composite electron density image (sCT) includes applying the one or more current pixel coefficients to one or more pixel values ​​at each of the pixel locations in the current MR image set in accordance with the image transfer function to generate corresponding pixel values ​​of the current composite electron density image (sCT); 7. The computer-implemented method of any one of items 3 to 6.

[0085] Item 8: Calculating (604) the current coefficients ([C]) includes substituting current pixel coefficients into one or more coefficient matrices (K, M) at matrix elements corresponding to pixels in each of the MR images. 8. The computer-implemented method of any one of items 3 to 7.

[0086] Item 9: calculating (606) the current composite electron density image (sCT) includes applying the one or more coefficient matrices (K, M) to pixel values ​​in the current MR image set element-by-element according to the image transfer function; Item 9. The computer-implemented method of item 8.

[0087] Item 10: The step of calculating the current coefficients ([C]) (604) is carried out by the machine learning model (MLM T) on the current MR image set, the method further comprising a second resampling of the one or more coefficient matrices (K,M) before applying the one or more coefficient matrices (K,M) element-wise to the pixel values ​​in the received current MR image set according to the image transfer function; Item 10. The computer-implemented method of item 9.

[0088] Item 11: The first resampling allows the resolution of the current MR image set to be adjusted to the resolution of the trained machine learning model (MLM). T ) to a resolution expected by the second resampling, and the second resampling changes the resolution of the one or more coefficient matrices (K, M) to a target resolution equal to the resolution of the current MR image set. Item 11. The computer-implemented method of item 10.

[0089] Item 12: the first resampling is downsampling and the second resampling is upsampling; 12. The computer-implemented method according to item 10 or 11.

[0090] Item 13: the image transfer function is a polynomial function; 13. The computer-implemented method of any one of items 1 to 12.

[0091] Item 14: the image transfer function is an affine function; 14. The computer-implemented method of any one of items 1 to 13.

[0092] Item 15: The machine learning model (MLM T ) includes artificial neural networks (ANN), 15. The computer-implemented method of any one of items 1 to 14.

[0093] Item 16: A computer-readable medium comprising computer instructions (94) that, when executed by a processor (92), cause the processor (92) to perform a method according to any one of items 1 to 15.

[0094] Item 17: 1. An apparatus for generating a synthetic electron density image (sCT) of an anatomical region, the apparatus including a processor (92), the processor (92) comprising: A machine learning model (MLM) trained to predict the coefficients of the image transfer function ([C]) T ) and receiving a current set of magnetic resonance images of the anatomical region; The machine learning model (MLM T ) on the current MR image set to generate current coefficients ([C]) of the image transfer function (ITF); generating a current synthetic electron density image (sCT) of the anatomical region by applying the current coefficients ([C]) to the current MR image set in accordance with the image transfer function; The apparatus is configured to:

[0095] Item 18: a magnetic resonance imaging device (10) configured to generate a current MR image of an anatomical region; Item 18. The apparatus according to item 17, which is arranged to receive the current MR image; A system including:

[0096] Item 19 A machine learning model that is trained to predict the coefficients ([C]) of an image transfer function to achieve a similarity criterion between a reference image (CT) and a synthetic electron density image (sCT) generated by applying the coefficients ([C]) to an MR image corresponding to the reference image (CT) according to the image transfer function.

[0097] Item 20: the coefficients ([C]) include one or more pixel coefficients for each pixel location in the MR image; the machine learning model is trained to allow the one or more pixel coefficients to vary between pixel locations; Item 19. The machine learning model described in item 19.

[0098] Item 21: and further trained to vary the one or more pixel coefficients between the pixel locations within each of the MR images while adhering to constraints on the variation of pixel coefficients within a predefined region around each of the pixel locations. The machine learning model described in item 20.

[0099] Item 22: A machine learning model (MLM) trained to predict coefficients of an image transfer function for use in generating a synthetic electron density image (sCT) of an anatomical region as a function of a set of magnetic resonance MR images of the anatomical region. T ), comprising: (a) initializing the machine learning model (MLM*); (b) acquiring training data including one or more MR images and one or more reference images (CT); (c) applying the machine learning model (MLM*) to the one or more MR images to generate predicted coefficients of the image transfer function ([C]*); (d) applying the predicted coefficients ([C]*) to the one or more MR images in accordance with the image transfer function to generate one or more predicted composite electron density images (sCT*); (e) evaluating a predefined cost function based on at least the reference image (CT) and the one or more predicted composite electron density images (sCT*); (f) determining model parameter values ​​for the machine learning model (MLM*); (g) repeating steps (b) through (f) for said model parameter values ​​until a predefined criterion is met; (h) the machine learning model (MLM T ) and outputting model parameter values ​​for A method comprising:

[0100] Item 23: A machine learning model (MLM) trained to predict coefficients of an image transfer function for use in generating a synthetic electron density image (sCT) of an anatomical region as a function of a set of magnetic resonance MR images of the anatomical region. T ), comprising: (a) initializing the machine learning model (MLM*); (b) the pre-calculated target coefficients of the image transfer function ([C] TAR ) and (c) applying the machine learning model (MLM*) to one or more MR images to generate predicted coefficients of the image transfer function ([C]*); (d) calculating a sum of at least the predicted coefficients ([C]*) and the pre-calculated target coefficients ([C] TAR ) evaluating a predefined cost function based on (e) determining model parameter values ​​for the machine learning model (MLM*); (f) repeating steps (b) through (e) for said model parameter values ​​until a predefined criterion is met; (g) the machine learning model (MLM T ) and outputting model parameter values ​​for A method comprising:

[0101] Item 24: the predicted coefficients ([C]*) comprise one or more predicted pixel coefficients for each pixel location in the one or more MR images; The method further includes allowing the machine learning model (MLM*) to vary one or more pixel coefficients between the pixel locations. 24. The computer-implemented method of item 22 or 23.

[0102] Item 25: applying a constraint on the variability of pixel coefficients within a predefined region around each of said pixel locations; Item 25. The computer-implemented method of item 24.

[0103] Item 26: applying the variability constraints by the machine learning model (MLM*), by the predefined cost function, or by processing the predicted coefficients ([C]*); Item 26. The computer-implemented method of item 25.

[0104] Item 27: A computer-readable medium comprising computer instructions (94) that, when executed by a processor (92), cause the processor (92) to perform the computer-implemented method of any one of items 22 to 26.

Claims

1. 1. A computer-implemented method for generating a synthetic electron density image (sCT) of an anatomical site, comprising: A machine learning model (MLM) trained to predict the coefficients of the image transfer function Ｔ ) (601); receiving 602 a current set of magnetic resonance images of the anatomical region; The machine learning model (MLM Ｔ Calculating (604) current coefficients ([C]) of the image transfer function by applying calculating (606) a current synthetic electron density image (sCT) of the anatomical site by applying the current coefficients ([C]) to the current MR image set according to the image transfer function; The method includes:

2. The machine learning model (MLM) is used to predict the coefficients. Ｔ ) to achieve a similarity criterion between a reference image (CT) and a synthetic electron density image (sCT) generated by applying the coefficients to an MR image corresponding to the reference image (CT) according to the image transfer function.

2. The computer-implemented method of claim 1.

3. the current coefficients ([C]) include one or more current pixel coefficients for each pixel location in the current MR image set; 3. A computer-implemented method according to claim 1 or 2.

4. The machine learning model (MLM Ｔ ) to allow the one or more current pixel coefficients to vary between pixel locations; 4. The computer-implemented method of claim 3.

5. the one or more current pixel coefficients vary between pixel locations; 5. A computer-implemented method according to claim 3 or 4.

6. The machine learning model (MLM) is adapted to vary the one or more pixel coefficients between the pixel locations while respecting a constraint on the variation of pixel coefficients within a predefined region around each of the pixel locations. Ｔ ) training, A computer-implemented method according to any one of claims 3 to 5.

7. and computing (606) the current synthetic electron density image (sCT) includes applying the one or more current pixel coefficients to one or more pixel values ​​at each of the pixel locations in the current MR image set in accordance with the image transfer function to generate corresponding pixel values ​​of the current synthetic electron density image (sCT). A computer-implemented method according to any one of claims 3 to 6.

8. Calculating (604) the current coefficients ([C]) includes substituting current pixel coefficients into one or more coefficient matrices (K,M) in matrix elements corresponding to pixels in each of the MR images; A computer-implemented method according to any one of claims 3 to 7.

9. and computing (606) the current synthetic electron density image (sCT) comprises applying the one or more coefficient matrices (K,M) to pixel values ​​in the current MR image set element-wise according to the image transfer function.

9. The computer-implemented method of claim 8.

10. The step of calculating the current coefficients ([C]) (604) is carried out by the machine learning model (MLM Ｔ ) on the current MR image set, The method further includes a second resampling of the one or more coefficient matrices (K,M) prior to applying the one or more coefficient matrices (K,M) element-wise to the pixel values ​​in the received current MR image set according to the image transfer function.

10. The computer-implemented method of claim 9.

11. The first resampling increases the resolution of the current MR image set by the trained machine learning model (MLM Ｔ ) to a resolution expected by the second resampling, and the second resampling changes the resolution of the one or more coefficient matrices (K, M) to a target resolution equal to the resolution of the current MR image set.

11. The computer-implemented method of claim 10.

12. the first resampling is downsampling and the second resampling is upsampling; 12. A computer-implemented method according to claim 10 or 11.

13. The image transfer function is a polynomial function or an affine function. A computer-implemented method according to any one of claims 1 to 12.

14. The machine learning model (MLM Ｔ ) includes artificial neural networks ANN, A computer-implemented method according to any one of claims 1 to 13.

15. 1. A computer-implemented method for generating a synthetic electron density image (sCT) of an anatomical site, comprising: A machine learning model (MLM) trained to predict the coefficients of the image transfer function Ｔ ) (601); receiving 602 a current set of raw images of the anatomical region; The machine learning model (MLM Ｔ Calculating (604) current coefficients ([C]) of the image transfer function by applying calculating (606) a current synthetic electron density image (sCT) of the anatomical region by applying the current coefficients ([C]) to the current set of raw images in accordance with the image transfer function; The method includes:

16. A computer readable medium comprising computer instructions (94) which, when executed by a processor (92), cause the processor (92) to perform a method according to any one of claims 1 to 15.

17. 1. An apparatus for generating a synthetic electron density image (sCT) of an anatomical site, the apparatus including a processor (92), the processor (92) comprising: A machine learning model (MLM) trained to predict the coefficients of the image transfer function ([C]) Ｔ ) and receiving a current set of magnetic resonance images of the anatomical region; The machine learning model (MLM Ｔ ) on the current MR image set to generate current coefficients ([C]) of the image transfer function (ITF); generating a current synthetic electron density image (sCT) of the anatomical region by applying the current coefficients ([C]) to the current MR image set in accordance with the image transfer function; The apparatus is configured to:

18. a magnetic resonance imaging device (10) configured to generate a current MR image of an anatomical site; 18. The apparatus of claim 17, arranged to receive the current MR image; A system including:

19. A machine learning model that is trained to predict the coefficients ([C]) of an image transfer function to achieve a similarity criterion between a reference image (CT) and a synthetic electron density image (sCT) generated by applying the coefficients ([C]) according to the image transfer function to an MR image corresponding to the reference image (CT).

20. The coefficients ([C]) include one or more pixel coefficients for each pixel location in the MR image; the machine learning model is trained to allow the one or more pixel coefficients to vary between pixel locations; 20. The machine learning model of claim 19.

21. and further trained to vary the one or more pixel coefficients between the pixel locations within each of the MR images while adhering to a constraint on the variation of pixel coefficients within a predefined region around each of the pixel locations.

21. The machine learning model of claim 20.

22. A machine learning model (MLM) trained to predict coefficients of an image transfer function for use in generating a synthetic electron density image (sCT) of an anatomical region as a function of a set of magnetic resonance MR images of the anatomical region. Ｔ 1. A computer-implemented method for providing a method for detecting a plurality of stimuli, comprising: (a) initializing the machine learning model (MLM*); (b) acquiring training data including one or more MR images and one or more reference images (CT); (c) applying the machine learning model (MLM*) to the one or more MR images to generate predicted coefficients of the image transfer function ([C]*); (d) applying the predicted coefficients ([C]*) to the one or more MR images in accordance with the image transfer function to generate one or more predicted synthetic electron density images (sCT*); (e) evaluating a predefined cost function based on at least said reference image (CT) and said one or more predicted synthetic electron density images (sCT*); (f) determining model parameter values ​​for the machine learning model (MLM*); (g) repeating steps (b) through (f) for said model parameter values ​​until a predefined criterion is met; (h) the machine learning model (MLM Ｔ ) outputting model parameter values ​​for The method includes:

23. A machine learning model (MLM) trained to predict coefficients of an image transfer function for use in generating a synthetic electron density image (sCT) of an anatomical region as a function of a set of magnetic resonance MR images of the anatomical region. Ｔ ), comprising the steps of: (a) initializing the machine learning model (MLM*); (b) pre-calculated target coefficients of the image transfer function ([C] ＴＡＲ ) and (c) applying the machine learning model (MLM*) to one or more MR images to generate predicted coefficients of the image transfer function ([C]*); (d) calculating a coefficient matrix ([C]*) based on at least the predicted coefficients ([C]*) and the pre-calculated target coefficients ([C] ＴＡＲ ) evaluating a predefined cost function based on (e) determining model parameter values ​​for the machine learning model (MLM*); (f) repeating steps (b) to (e) for said model parameter values ​​until a predefined criterion is met; (g) the machine learning model (MLM Ｔ ) outputting model parameter values ​​for The method includes:

24. the predicted coefficients ([C]*) comprise one or more predicted pixel coefficients for each pixel location in the one or more MR images; The method further includes enabling the machine learning model (MLM*) to vary one or more pixel coefficients between the pixel locations.

24. A computer-implemented method according to claim 22 or 23.

25. applying a constraint on the variation of pixel coefficients within a predefined region around each of said pixel locations; 25. The computer-implemented method of claim 24.

26. applying the variability constraints by the machine learning model (MLM*), or by the predefined cost function, or by processing the predicted coefficients ([C]*); 26. The computer-implemented method of claim 25.

27. A computer readable medium comprising computer instructions (94) which, when executed by a processor (92), cause the processor (92) to perform a computer-implemented method according to any one of claims 22 to 26.

Images