Producing MRI images of the liver without contrast enhancement
A predictive model using supervised learning generates MRI images of the liver without contrast enhancement, addressing the challenge of differentiating liver lesions and blood vessels by training on contrast-enhanced images, enhancing image clarity and reducing patient discomfort and acquisition time.
Patent Information
- Application Number
- JP2022521396
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-10-08
- Filing Date
- 2020-10-05
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2040-10-05
AI Technical Summary
Existing MRI techniques struggle to distinguish between liver lesions and blood vessels in images generated during the hepatocellular contrast phase due to the inability to differentiate contrast enhancement caused by contrast agents, leading to unclear differentiation between the two.
A method using supervised learning and a predictive model, such as an artificial neural network, to generate MRI images of the liver without contrast enhancement by training on contrast-enhanced images, allowing differentiation between blood vessels and healthy liver cells based on patterns learned from contrast-enhanced images.
Enables clear differentiation between liver lesions and blood vessels in MRI images without the need for contrast agents, reducing patient discomfort and shortening MRI acquisition time by utilizing self-learning algorithms to predict non-contrast-enhanced images from contrast-enhanced data.
Smart Images

Figure 0007587575000001 
Figure 0007587575000002 
Figure 0007587575000003
Abstract
Description
[Technical field]
[0001] The present invention deals with the generation of artificial MRI images of the liver. The subject of the present invention is a method, a system and a computer program product for generating MRI images of the liver without contrast enhancement. [Background technology]
[0002] Magnetic resonance imaging, or MRI for short, is an imaging technique used especially in medical diagnostics to visualize the structure and function of tissues and organs inside the human or animal body.
[0003] In MRI, the magnetic moments of protons in the examined object are aligned in a basic magnetic field, resulting in a macroscopic magnetization along the longitudinal direction. This macroscopic magnetization is subsequently deflected from its rest position by the incident radiation of a radio frequency (HF) pulse (excitation). The return from the excited state to the rest position (relaxation), i.e. the magnetization dynamics, is subsequently detected as a relaxation signal using one or more HF receiving coils.
[0004] For spatial encoding, rapidly switching magnetic gradient fields are superimposed on the basic magnetic field. The captured relaxation signals, or the detected and spatially resolved MRI data, initially exist as raw data in spatial frequency space and can be transformed into real space (image space) by a subsequent Fourier transformation.
[0005] In native (non-contrast) MRI, tissue contrast is generated by different relaxation times (T1 and T2) and proton densities.
[0006] T1 relaxation represents the transition of the longitudinal magnetization to the equilibrium state, where T1 is the time required to reach 63.21% of the equilibrium magnetization prior to resonant excitation. T1 is also called the longitudinal relaxation time, or spin-lattice relaxation time.
[0007] Similarly, T2 relaxation represents the transition of the transverse magnetization to an equilibrium state.
[0008] MRI contrast agents work by altering the relaxation times of the structures they entrap. Two distinct classes of substances can be distinguished: paramagnetic and superparamagnetic. Both classes of substances contain unpaired electrons that induce a magnetic field around individual atoms or molecules.
[0009] Superparamagnetic contrast agents produce significant T2 shortening, whereas paramagnetic contrast agents produce primarily T1 shortening. A shortened T1 time results in an increase in signal intensity on T1-weighted sequences, whereas a shortened T2 time results in a decrease in signal intensity on T2-weighted sequences.
[0010] The effect of the contrast agent is indirect, since the contrast agent itself does not emit a signal, but merely affects the signal intensity of hydrogen protons in its vicinity.
[0011] In T1-weighted images, paramagnetic contrast agents cause areas that contain the contrast agent to appear brighter (higher signal) than areas that do not contain the contrast agent.
[0012] In T2-weighted images, superparamagnetic contrast agents cause areas that contain the contrast agent to appear darker (lower signal) than areas that do not contain the contrast agent.
[0013] Both higher and lower signal imaging results in contrast enhancement.
[0014] One example of a superparamagnetic contrast agent is iron oxide nanoparticles (SPIO, superparamagnetic iron oxide).
[0015] Examples of paramagnetic contrast agents are gadolinium chelates such as gadopentetate dimeglumine (trade name Magnevist®, etc.), gadobenate meglumine (trade name Multihance®), gadoteric acid (Dotarem®, Dotagita®, Cyclolux®), gadodiamide (Omniscan®), gadoteridol (Prohance®), and gadobutrol (Gadovist®).
[0016] Extracellular, intracellular, and intravascular contrast agents can be distinguished according to their diffusion patterns within tissue.
[0017] Gadoxetic acid-based contrast agents are characterized by their specific uptake in liver cells (hepatocytes), their enrichment in functional tissues (parenchyma) and contrast enhancement in healthy liver tissue. Cysts, metastatic cancers and most hepatocellular carcinoma cells no longer function like normal liver cells and do not or only poorly uptake the contrast agent, are not enhanced and are therefore identifiable and localizable.
[0018] Examples of gadoxetic acid based contrast agents are described in US Pat. No. 6,039,931A and are commercially available, for example, under the trade names Primovist® and Eovist®.
[0019] The contrast enhancing effect of Primovist® and Eovist® is provided by the stable gadolinium complex Gd-EOB-DTPA (gadolinium ethoxybenzyl diethylenetriaminepentaacetic acid, sodium gadoxetate). DTPA forms a complex with the paramagnetic gadolinium ion that has extremely high thermodynamic stability. The ethoxybenzyl radical (EOB) is a mediator of hepatobiliary uptake of the contrast agent.
[0020] Primovist® can be used for the detection and characterization of tumors in the liver. The blood supply to healthy liver tissue is achieved mainly via the portal vein, while the hepatic artery supplies most primary tumors. After a bolus of contrast agent is injected intravenously, it is accordingly possible to observe the time delay between the increase in signal in the healthy liver parenchyma and in the tumor.
[0021] In the case of contrast enhancement achieved by Primovist® in the distribution phase, what is observed are typical perfusion patterns that provide information characterizing the lesion. Depicting the behavior during wash-in and wash-out, as well as the vascularization, helps characterize the type of lesion and determine the spatial relationship of the tumor and blood vessels.
[0022] In T1-weighted images, Primovist® results in clear signal enhancement in healthy liver parenchyma 10-20 minutes after injection (during the hepatic contrast phase), whereas lesions containing no or only few hepatocytes, e.g., metastatic or moderately to poorly differentiated hepatocellular carcinoma (HCC), appear as darker areas.
[0023] However, in the hepatic contrast phase, blood vessels are also visible as dark areas, which means that in the MRI images generated in the hepatic contrast phase, it is generally not possible to distinguish between liver lesions and blood vessels on the basis of contrast alone. Liver lesions and blood vessels can be distinguished using an MRI image without contrast enhancement otherwise caused by a contrast agent, for example in conjunction with a further MRI image of the dynamic phase (where the blood vessels are highlighted). However, if, for example, a shortened MRI image acquisition method is used for the examined subject, for example a contrast agent has already been administered for a certain period of time before the acquisition of the MRI image in order to directly acquire an MRI image during the hepatic contrast phase, and an MRI image of the dynamic phase is created after a second administration of the contrast agent, it is no longer possible to create an MRI image without contrast enhancement (native MRI image) with the same MRI image acquisition process. Summary of the Invention [Problem to be solved by the invention]
[0024] The present invention addresses this problem by the subject matter of the independent claims. Preferred embodiments of the invention are presented in the dependent claims, the description and the drawings. [Means for solving the problem]
[0025] In a first aspect, the present invention provides a method for producing ... composition comprising the steps of: - receiving at least one first MRI image of the subject showing the liver or a portion of the liver of the subject, the blood vessels in the liver being depicted with contrast enhancement due to a contrast agent; - receiving at least one second MRI image of the same subject showing the same liver or the same part of the liver, healthy liver cells being visualized with contrast enhancement due to a contrast agent; - sending the received MRI images to a prediction model, the prediction model being trained using supervised learning to predict one or more MRI images showing the liver or part of the liver of the subject without contrast enhancement caused by the contrast agent based on MRI images showing the liver or part of the liver of the subject, where blood vessels in the liver are visualized with contrast enhancement due to the contrast agent, and based on MRI images of the same liver or the same part of the liver of the same subject, where healthy liver cells are visualized with contrast enhancement due to the contrast agent; - receiving from the prediction model one or more predicted MRI images showing the liver or a portion of the liver of interest, without contrast enhancement caused by a contrast agent; - displaying and / or outputting one or more predicted MRI images and / or storing one or more predicted MRI images on a data storage medium; The present invention provides a method comprising:
[0026] The present invention further comprises: A receiving unit; A control calculation unit; An output unit; Including, the control and computing unit is configured to prompt the receiving unit to receive at least one first MRI image of the subject, the first MRI image showing a liver or a portion of the liver of the subject, the blood vessels in the liver being depicted with contrast enhancement due to a contrast agent; the control and computing unit is configured to prompt the receiving unit to receive at least one second MRI image of the subject showing the same liver or the same part of the liver, healthy liver cells being depicted with enhanced contrast due to the contrast agent; - the control and computing unit is configured to predict one or more MRI images based on the received plurality of MRI images, the one or more predicted MRI images showing the liver or a portion of the liver of the subject without contrast enhancement caused by a contrast agent; the control and computing unit is configured to display the one or more predicted MRI images and prompt the output unit to output or store the one or more predicted MRI images in a data storage medium; Provide a system.
[0027] The present invention further provides a computer program product comprising a computer program loadable into the memory of a computer, the computer program causing the computer to: - receiving at least one first MRI image of the subject showing the liver or a portion of the liver of the subject, the blood vessels in the liver being depicted with contrast enhancement due to a contrast agent; - receiving at least one second MRI image of the same subject showing the same liver or the same part of the liver, healthy liver cells being visualized with contrast enhancement due to a contrast agent; - sending the received MRI images to a prediction model, the prediction model being trained using supervised learning to predict one or more MRI images showing the liver or part of the liver of the subject without contrast enhancement caused by the contrast agent based on MRI images showing the liver or part of the liver of the subject, where blood vessels in the liver are visualized with contrast enhancement due to the contrast agent, and based on MRI images of the same liver or the same part of the liver of the same subject, where healthy liver cells are visualized with contrast enhancement due to the contrast agent; - receiving one or more predicted MRI images showing the liver or a portion of the liver of the subject, without contrast enhancement caused by a contrast agent, output from the prediction model; - displaying and / or outputting one or more predicted MRI images and / or storing one or more predicted MRI images on a data storage medium; Encourage them to do so.
[0028] The present invention further provides a method for the use of a contrast agent in an MRI modality, the MRI modality comprising: - administering a contrast agent which diffuses in the liver of the subject; - generating at least one first MRI image showing the liver or a portion of the liver of the subject, the blood vessels in the liver being depicted with enhanced contrast due to a contrast agent; - generating at least one second MRI image showing the same liver or the same part of the liver, in which healthy liver cells are visualized with enhanced contrast due to the contrast agent; - sending the generated MRI images to a predictive model, the predictive model being trained using supervised learning to predict one or more MRI images showing the liver or a portion of the liver of the subject without contrast enhancement caused by the contrast agent based on MRI images showing the liver or a portion of the liver of the subject, where blood vessels in the liver are visualized with contrast enhancement due to the contrast agent, and based on MRI images of the same liver or the same portion of the liver of the same subject, where healthy liver cells are visualized with contrast enhancement due to the contrast agent; - receiving one or more predicted MRI images showing the liver or a portion of the liver of the subject, without contrast enhancement caused by a contrast agent, output from the prediction model; - displaying and / or outputting one or more predicted MRI images and / or storing one or more predicted MRI images on a data storage medium; Includes.
[0029] Further provided is a contrast agent for use in an MRI procedure, the MRI procedure comprising: - administering a contrast agent which diffuses in the liver of the subject; - generating at least one first MRI image showing the liver or a portion of the liver of the subject, the blood vessels in the liver being depicted with enhanced contrast due to a contrast agent; - generating at least one second MRI image showing the same liver or the same part of the liver, in which healthy liver cells are visualized with enhanced contrast due to the contrast agent; - sending the generated MRI images to a predictive model, the predictive model being trained using supervised learning to predict one or more MRI images showing the liver or a portion of the liver of the subject without contrast enhancement caused by the contrast agent based on MRI images showing the liver or a portion of the liver of the subject, where blood vessels in the liver are visualized with contrast enhancement due to the contrast agent, and based on MRI images of the same liver or the same portion of the liver of the same subject, where healthy liver cells are visualized with contrast enhancement due to the contrast agent; - receiving one or more predicted MRI images showing the liver or a portion of the liver of the subject, without contrast enhancement caused by a contrast agent, output from the prediction model; - displaying and / or outputting one or more predicted MRI images and / or storing one or more predicted MRI images on a data storage medium; Includes.
[0030] Additionally, a kit is provided that includes an imaging agent and a computer program product according to the present invention. [Brief description of the drawings]
[0031] [Figure 1] Schematic representation of the temporal profile of the concentration of the contrast agent in the hepatic artery (A), hepatic vein (P), and healthy liver tissue (L). [Diagram 2] 1 illustrates a schematic of an example of an abbreviated MRI image acquisition procedure. [Diagram 3] 1 shows a schematic diagram of a preferred embodiment of a system according to the present invention; [Figure 4] 1 shows, diagrammatically and exemplarily, one embodiment of the method according to the invention. [Diagram 5] Further embodiments of the present invention are exemplarily and diagrammatically illustrated. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0032] The present invention will be described in more detail below without distinguishing between the several subjects of the invention (method, system, computer program product, use, imaging agent used, kit) - instead, the following descriptions are intended to apply equally to all subjects of the invention (method, system, computer program product, use, imaging agent used, kit) regardless of the context in which they appear.
[0033] In the present specification or claims, when steps are described in a sequential order, this does not necessarily mean that the invention is limited to the order described. On the contrary, the steps may be performed in a different order or in parallel with each other, except when one step is built upon another step, i.e., the built upon step necessarily requires subsequent execution (as will become evident in each individual case). Thus, the order described is a preferred embodiment of the invention.
[0034] The present invention generates one or more artificial MRI images of the liver or liver portion of an examined subject, the MRI images showing the liver or liver portion without contrast enhancement caused by a contrast agent. The artificial MRI images are generated based on a plurality of MRI images, all of which were recorded with contrast enhancement by a contrast agent. The artificial MRI images can be generated using a self-learning algorithm and mimic an MRI image of the liver or liver portion of an examined subject without contrast enhancement caused by administration of a contrast agent.
[0035] "The examination object is typically a living organism, preferably a mammal, and most preferably a human.
[0036] A portion of the examination object is subjected to a contrast-enhanced magnetic resonance imaging examination. The "examination region", also called image volume (field of view, FOV), is in particular the volume that is imaged with a magnetic resonance image. The examination region is typically defined by a radiologist, for example on an overview image (localizer). It is self-evident that the examination region can alternatively or additionally also be defined automatically, for example based on a selected protocol. The examination region includes at least a portion of the liver of the examination object.
[0037] The examination region is brought into the base magnetic field.
[0038] A contrast agent that diffuses within the examination region is administered to the subject, preferably intravenously (e.g., into a vein in the arm) as a bolus using a dose based on body weight.
[0039] "Imaging agent" is understood to mean a substance or mixture, the presence of which modifies the signal in magnetic resonance measurements. Preferably, the imaging agent brings about a shortening of the T1 and / or T2 relaxation times.
[0040] Preferably, the imaging agent is a hepatobiliary imaging agent, such as Gd-EOB-DTPA or Gd-BOPTA.
[0041] In a particularly preferred embodiment, the contrast agent is a substance or mixture having gadoxetic acid or a gadoxetate salt as the contrast enhancing active substance. A very particular preference is given to the disodium salt of gadoxetic acid (disodium Gd-EOB-DTPA).
[0042] The examination area is subjected to an MRI procedure, during which an MRI image is generated (measured) that shows the examination area at the examination stage.
[0043] The measured MRI images can exist as two-dimensional images showing a section through the object. The measured MRI images can exist as a bundle of two-dimensional images, each image of the bundle showing a different section. The measured MRI images can exist as three-dimensional images (3D images). To make the illustration easier, the invention is described at some points in this specification on the basis of the existence of two-dimensional MRI images, but it is not desired to limit the invention to two-dimensional MRI images. It is clear to the skilled person how the individual descriptions can be applied to bundles of two-dimensional images and to 3D images (see, for example, "M. Reisler, W. Semmler: Magnetresonanztomographie [Magnetic resonance imaging], Springer Verlag, 3rd edition, 2002, ISBN: 978-3-642-63076-7").
[0044] After intravenous administration of a hepatobiliary contrast agent in the form of a bolus, the agent first reaches the liver via the arteries, which are visualized as contrast-enhanced in the corresponding MRI images. The phase in which the hepatic arteries are visualized as contrast-enhanced in the MRI images is called the "arterial phase". This phase begins immediately after the administration of the contrast agent and usually lasts for 15-25 seconds.
[0045] The contrast then reaches the liver via the hepatic veins. The contrast in the hepatic artery is already decreasing, while the contrast in the hepatic veins is reaching its maximum value. The phase in which the hepatic veins are visualized with contrast enhancement in the MRI image is called the "venous phase". This phase can already begin during the arterial phase, and these phases can overlap. Usually, this phase begins 20-30 seconds after intravenous administration and usually lasts for 40-60 seconds.
[0046] The venous phase is followed by a "late phase" in which the contrast in the hepatic arteries becomes even smaller, the contrast in the hepatic veins becomes similarly smaller, and the contrast in healthy liver tissues becomes increasingly greater. This phase begins 70-90 seconds after administration of the contrast agent and usually lasts for 100-120 seconds.
[0047] The arterial, venous, and late phases are collectively referred to as the "dynamic phases."
[0048] 10-20 minutes after injection, the hepatobiliary contrast agent produces a clear signal enhancement in healthy liver parenchyma. This phase is called the "hepatobiliary phase." The contrast agent is only slowly cleared from liver cells, and as a result, the hepatobiliary phase can last for more than 2 hours.
[0049] The aforementioned phases are described in more detail, for example, in the following documents: J. Magn. Reson. Imaging, 2012, 35(3): 492-511, doi:10.1002 / jmri.22833; Clujul Medical, 2015, Vol. 88 no. 4: 438-448, DOI: 10.15386 / cjmed-414; Journal of Hepatology, 2019, Vol. 71: 534-542, http: / / dx.doi.org / 10.1016 / j.jhep.2019.05.005).
[0050] In this specification, a "first MRI image" refers to an MRI image in which blood vessels are identifiable, which are depicted with contrast enhancement due to a contrast agent. Preferably, the at least one first MRI image is at least one MRI image measured in the dynamic phase. Specific settings are applied in each case to the at least one MRI image measured in the arterial phase, the venous phase, and / or the late phase. Very specific settings are applied in each case to the at least one MRI image measured in the arterial phase, the venous phase, and the late phase. Preferably, the at least one first MRI image is a T1-weighted representation.
[0051] When using a paramagnetic contrast agent, blood vessels are characterized by high signal intensity in the at least one first MRI image due to contrast enhancement (high signal representation). Those (contiguous) structures in the first MRI image that have a signal intensity within an experimentally ascertainable range can therefore be assigned to blood vessels. This means that with the at least one first MRI image there is information about where blood vessels are depicted in the MRI image or which structures in the MRI image can be attributed to blood vessels (arteries and / or veins).
[0052] In this specification, a "second MRI image" refers to an MRI image showing the examination area in the hepatic imaging phase. During the hepatic imaging phase, healthy liver tissue (parenchyma) is visualized with contrast enhancement. In this way, those (contiguous) structures in the second MRI image that have a signal intensity within an experimentally verifiable range can be assigned to healthy liver cells. Thus, the at least one second MRI image contains information about where healthy liver cells are visualized in the MRI image or which structures in the MRI image can be attributed to healthy liver cells. Preferably, the at least one second MRI image is a T1-weighted representation.
[0053] The acquisition of MRI images of the liver dynamic phase and hepatic contrast phase extends over a relatively long period of time. During said period, movements by the patient should be prevented in order to minimize movement artifacts in the MRI images. Long-term restrictions on movements can be uncomfortable for the patient. Therefore, what is established herein is a shortened MRI image acquisition procedure, in which a contrast agent is already administered to the subject prior to the acquisition of MRI images for a certain period of time (i.e., 10-20 minutes) in order to allow direct acquisition of MRI images during the hepatic contrast phase. Then, the MRI images of the dynamic phase are acquired in the same MRI image acquisition process after a second administration of the contrast agent. As a result, the MRI dwell time of the patient or subject is clearly shorter compared to a conventional MRI image acquisition process. Therefore, the invention preferably comprises recording at least one MRI image of the liver or a part of the liver in a hepatic imaging phase after a (first) administration of a first contrast agent to the examination subject, and recording at least one further MRI image of the same liver or the same part of the liver in a dynamic phase after administration of a second contrast agent or a second administration of the first contrast agent to the same examination subject. The first contrast agent is in this context a paramagnetic contrast agent in the hepatobiliary system. Also, the second contrast agent can be a paramagnetic extracellular contrast agent.
[0054] The "first MRI image" and the "second MRI image" are sent to a predictive model that is configured to predict, based on the plurality of received MRI images, one or more MRI images that show the subject's liver or a portion of the liver without contrast enhancement caused by a contrast agent.
[0055] In this context, the term "prediction" means that an MRI image showing the liver or part of the liver of an examination subject without contrast enhancement caused by a contrast agent is calculated using an MRI image showing the same examination area with contrast enhancement caused by a contrast agent.
[0056] The predictive model is preferably created using a self-learning algorithm in a supervised machine learning process. Learning is achieved using training data that includes a large number of MRI images of the liver or part of the liver of the subject in the dynamic phase and the hepatic enhancement phase. In addition, training data is preferably also used that is created from MRI images of the same liver or part of the liver of the same subject without contrast enhancement, i.e. without administration of a contrast agent.
[0057] Self-learning algorithms generate statistical models based on training data during machine learning. This means that examples are not simply memorized, but the algorithm "recognizes" patterns and regularities in the training data. The algorithm is therefore also capable of evaluating unknown data. Validation data can be used to check the quality of the evaluation of unknown data.
[0058] The self-learning algorithm is trained using supervised learning, i.e., dynamic and hepatic contrast-enhanced MRI images are continuously passed to the algorithm to inform it which non-contrast-enhanced MRI images are associated with these contrast-enhanced MRI images, and the algorithm learns the relationship between the contrast-enhanced and non-contrast-enhanced MRI images in order to predict one or more non-contrast-enhanced MRI images for a contrast-enhanced MRI image.
[0059] Self-learning algorithms trained using supervised learning have been widely described in the prior art (see, for example, C. Perez: Machine Learning Techniques: Supervised Learning and Classification, Amazon Digital Services LLC - Kdp Print Us, 2019, ISBN 1096996545, 9781096996545).
[0060] Preferably, the predictive model is an artificial neural network.
[0061] Such an artificial neural network includes at least three layers of processing elements: a first layer having input neurons (nodes), an Nth layer having at least one output neuron (node), and (N-2) hidden layers, where N is a natural number greater than 2.
[0062] The input neurons serve to receive the digital MRI image as input. Typically there is one input neuron for each pixel or voxel of the digital MRI image. There may be additional input neurons for additional inputs (e.g. information about the examination region, about the examination subject, and / or about the conditions prevailing when the MRI image was generated).
[0063] In such a network, the output neurons serve to generate a third artificial MRI image for the first and second MRI images. The processing elements of the layer between the input and output neurons are connected to each other in a predetermined pattern with predetermined connection weights.
[0064] Preferably, the artificial neural network is a so-called Convolutional Neural Network (abbreviated as CNN).
[0065] Convolutional neural networks can process input data in the form of matrices. This allows the use of digital MRI images depicted as matrices (e.g., width x height x color channels) as input data. In contrast, a regular neural network, for example in the form of a multi-layer perceptron (MLP), requires a vector as input, i.e., to use an MRI image as input, and the pixels or voxels of the MRI image must be fed out in a long chain in succession. As a result, a regular neural network cannot, for example, recognize an object in an MRI image independently of the object's location in the MRI image. The same object at a different location in the MRI image would have a completely different input vector.
[0066] A CNN essentially consists of alternating layers of filters (convolutional layers) and aggregation (pooling layers), and finally one or many layers of "normal" fully connected neurons (densely / fully connected layers).
[0067] When analyzing sequences (a series of MRI images), space and time can be treated as equivalent dimensions and processed, for example, via 3D folds, as shown in Baccouche et al. (Sequential Deep Learning for Human Action Recognition; International Workshop on Human Behavior Understanding, Springer 2011, pages 29-39) and Ji et al. (3D Convolutional Neural Networks for Human Action Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 221-231). Furthermore, it is possible to train different networks, one for time and one for space, and then combine the features at the end, as described in Karpathy et al. (Large-scale Video Classification with Convolutional Neural Networks; Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014, pages 1725-1732) and Simonyan & Zisserman (Two-stream Convolutional Networks for Action Recognition in Videos; Advances in Neural Information Processing Systems, 2014, pages 568-576).
[0068] Recurrent neural networks (RNNs) are a family of so-called feedforward neural networks that contain feedback connections between layers. RNNs allow for the modeling of continuous data by sharing parameter data with different parts of the neural network. The architecture for RNNs involves recursion, where the current value of a variable represents the influence of that variable on its own value at a future time point, because at least a portion of the output data from the RNN is used as feedback to process subsequent inputs in the sequence.
[0069] More details can be found in the prior art (see, for example, "S. Khan et al.: A Guide to Convolutional Neural Networks for Computer Vision, Morgan & Claypool Publishers 2018, ISBN 1681730227, 9781681730226").
[0070] Neural networks can be trained, for example, using backpropagation. In this regard, what is required of the network is to map a given input vector to a given output vector as reliably as possible. The mapping quality is described by an error function. The goal is to minimize the error function. In backpropagation, an artificial neural network is taught by modifying the connection weights.
[0071] In the trained state, the connection weights between the processing elements contain information about the relationship between the contrast-enhanced MRI images of the dynamic and hepatic contrast phases and non-contrast-enhanced MRI images that can be used to predict one or more MRI images showing the examination region without contrast enhancement, calculated using only the contrast-enhanced MRI images of the examination region.
[0072] Cross-validation can be used to classify the data into a training dataset and a validation dataset. The training dataset is used to train the network weights using backpropagation. The validation dataset is used to check the accuracy of predictions, which can then be used to apply the trained network to a set of unknown MRI images.
[0073] As already indicated, further information about the test subject, further information about the test area, and / or further information about the test conditions can also be used for training, validation, and prediction.
[0074] Examples of information about the test subject include sex, age, weight, height, medical history, the effects of medications already taken, time and amount of intake, blood pressure, central venous pressure, respiratory rate, serum albumin, total bilirubin, blood glucose, iron content, lung capacity, and the like, which may be obtained, for example, from a database or electronic medical record of the patient.
[0075] Examples of information about areas of examination are underlying disease, surgery, partial resection, liver transplant, liver iron, fatty liver, and the like.
[0076] The received MRI images may be subject to retroactive motion correction before the images are fed to the predictive model. Such motion correction ensures that pixels or voxels of a first MRI image represent the same region of examination as corresponding pixels or voxels of a second MRI image downstream in time. Motion correction methods are described in the prior art (see, for example, EP3118644, EP3322997, US20080317315, US20170269182, US20140062481, EP2626718).
[0077] The invention provides a system allowing the method according to the invention to be carried out.
[0078] Although the aforementioned units may be considered as components of a single computer system, the aforementioned units may also be considered as components of multiple separate computer systems connected together via a network for transmitting data and / or control signals from one unit to another.
[0079] A "computer system" is a system for electronic data processing that processes data using programmable algorithms. Such systems typically include a "computer," which is a device that includes a processor that performs logical operations, and may also include peripheral devices.
[0080] In computer technology, a "peripheral" refers to any device that is connected to a computer and helps control the computer and / or acts as an input / output device. Examples are monitors (screens), printers, scanners, mice, keyboards, drives, cameras, microphones, speakers, etc. Internal ports and expansion cards are also considered peripherals in computer technology.
[0081] Today's computer systems are often divided into desktop PCs, portable PCs, laptops, notebooks, netbooks and tablet PCs, as well as so-called mobile devices (e.g. smartphones), and all these systems can be used to implement the present invention.
[0082] Input to the computer system is accomplished by means of input means such as, for example, a keyboard, mouse, microphone, touch-sensitive display, and / or the like.
[0083] The system according to the invention is configured to receive at least one first MRI image with contrast-enhanced hepatic contrast phase and at least one second MRI image with contrast-enhanced dynamic phase and to generate (predict, calculate) based on these data and optionally further data one or more MRI images without contrast enhancement showing the examination region, i.e. the liver or a part of the liver.
[0084] The control and computing unit serves to control the receiving units, coordinate the flow of data and signals between the various units, and process and generate the MRI images. There may be multiple control and computing units.
[0085] The receiving unit serves for receiving MRI images. The MRI images can be transmitted, for example, from a magnetic resonance system or can be read from a data storage medium. The magnetic resonance system can be a component of the system according to the invention. However, the system according to the invention can also be considered as a component of a magnetic resonance system.
[0086] At least one first MRI image, at least one second MRI image, and optionally further data are transmitted from the receiving unit to the controlling and computing unit.
[0087] The control and computing unit is configured to predict, based on the contrast-enhanced MRI images of the dynamic and hepatic contrast phases, one or more MRI images showing the examination region without contrast enhancement. Preferably, loadable into the memory of the control and computing unit is a prediction model used to calculate the MRI images without contrast enhancement. The prediction model has been created (trained) using a self-learning algorithm, preferably by supervised learning.
[0088] Via the output unit, the at least one predicted MRI image can be displayed (eg on a monitor), output (eg by means of a printer) or stored on a data storage medium.
[0089] A further embodiment of the invention relates to a method for the use of or the use of a contrast agent in an MRI procedure, the MRI procedure comprising: - administering a contrast agent which diffuses in the liver of the subject; - generating at least one first MRI image showing the liver or a portion of the liver of the subject, the blood vessels in the liver being depicted with enhanced contrast due to a contrast agent; - generating at least one second MRI image showing the same liver or the same part of the liver, in which healthy liver cells are visualized with enhanced contrast due to the contrast agent; - sending the generated MRI images to a predictive model, the predictive model being trained using supervised learning to predict one or more MRI images showing the liver or a portion of the liver of the subject without contrast enhancement caused by the contrast agent based on MRI images showing the liver or a portion of the liver of the subject, where blood vessels in the liver are visualized with contrast enhancement due to the contrast agent, and based on MRI images of the same liver or the same portion of the liver of the same subject, where healthy liver cells are visualized with contrast enhancement due to the contrast agent; - receiving one or more predicted MRI images showing the liver or a portion of the liver of the subject, without contrast enhancement caused by a contrast agent, output from the prediction model; - displaying and / or outputting one or more predicted MRI images and / or storing one or more predicted MRI images on a data storage medium; Includes.
[0090] In a preferred variant, at least one "second MRI image" is generated after a (first) administration of a first contrast agent to the examined subject, and at least one "first MRI image" is generated after a second administration of the first contrast agent to the same examined subject or after an administration of a second contrast agent, meaning that the "second MRI image" as defined above is generated temporally before the "first MRI image" as defined above.
[0091] The invention will be described in more detail below with reference to figures, but there is no desire to limit the invention to the features or combinations of features shown in the figures. FIG. 1 shows a schematic representation of the temporal profile of the concentration of the contrast agent in the hepatic artery (A), the hepatic vein (P) and in healthy liver cells (L). The concentration is depicted in the indicated areas (hepatic artery, hepatic vein and liver cells) as a function of time t in the form of signal intensity I during a magnetic resonance measurement. After an intravenous bolus injection is made, the concentration of the contrast agent first rises in the hepatic artery (A) (dashed curve). The concentration passes through a maximum value and then falls. The concentration in the hepatic vein (P) rises more slowly than in the hepatic artery and reaches its maximum value later (dotted curve). The concentration of the contrast agent in the liver cells (L) rises slowly (continuous curve) and reaches its maximum value at a much later time (not depicted in FIG. 1). Several characteristic time points can be defined. At time TP0 the contrast agent is administered as a bolus intravenously. At time TP1 the concentration (signal intensity) of the contrast agent in the hepatic artery reaches its maximum value. At time TP2 the curves of signal intensity in the hepatic artery and in the hepatic vein intersect. At time TP3, the concentration (signal intensity) of the contrast agent in the hepatic vein passes through its maximum value. At time TP4, the curves of signal intensity in the hepatic vein and liver tissue intersect. At time TP5, the concentration in the hepatic artery and vein has decreased to a level that no longer produces visible contrast enhancement.
[0092] Figure 2 shows a schematic example of an abbreviated MRI image acquisition procedure. In the abbreviated MRI image acquisition procedure, a contrast agent is first administered (1). After a certain waiting period, e.g. 10-20 minutes, the examination subject is guided to the MRI (2). The MRI process is then started and an MRI of the liver or a part of the liver is first performed in the hepatic contrast phase (3). After that, a further intravenous bolus is administered to the examination subject (4) and an MRI of the liver or a part of the liver is performed immediately afterwards in the dynamic phase.
[0093] 3 shows a schematic diagram of a preferred embodiment of the system according to the present invention. The system (10) includes a receiving unit (11), a control and calculation unit (12) and an output unit (13).
[0094] The control and computing unit (12) is configured to prompt the receiving unit (11) to receive at least one first MRI image of the subject showing the liver or a portion of the liver of the subject, wherein blood vessels in the liver are depicted with enhanced contrast due to the contrast agent.
[0095] The control and computing unit (12) is further configured to prompt the receiving unit (11) to receive at least one second MRI image of the subject showing the same liver or the same part of the liver, where healthy liver cells are depicted with enhanced contrast due to the contrast agent.
[0096] The control and computing unit (12) is further configured to predict, based on the plurality of received MRI images, one or more MRI images showing the liver or a portion of the liver of the subject without contrast enhancement caused by the contrast agent.
[0097] The control and computing unit (12) is further configured to display the at least one predicted MRI image and prompt the output unit (13) to output or store the at least one predicted MRI image in a data storage medium.
[0098] FIG. 4 shows a schematic and exemplary embodiment of a method according to the present invention. The method (100) comprises: - receiving (110) at least one first MRI image of the subject showing a liver or a portion of the liver of the subject, the blood vessels in the liver being depicted with contrast enhancement due to a contrast agent; - receiving (120) at least one second MRI image of the same subject showing the same liver or the same part of the liver, healthy liver cells being visualized with contrast enhancement due to the contrast agent; - sending (130) the received plurality of MRI images to a prediction model, the prediction model having been trained using supervised learning to predict one or more MRI images showing the liver or a portion of the liver of the subject without contrast enhancement caused by the contrast agent based on MRI images showing the liver or a portion of the liver of the subject, where blood vessels in the liver are visualized with contrast enhancement due to the contrast agent, and based on MRI images of the same liver or the same portion of the liver of the same subject, where healthy liver cells are visualized with contrast enhancement due to the contrast agent; - receiving (140) from the prediction model one or more predicted MRI images showing the liver or a portion of the liver of interest, without contrast enhancement caused by a contrast agent; - displaying and / or outputting one or more predicted MRI images and / or storing one or more predicted MRI images on a data storage medium (150); Includes.
[0099] A further embodiment of the invention is exemplarily and diagrammatically shown in Fig. 5. A first MRI image (1) is provided showing the liver or a part of the liver of an examination subject, in which blood vessels are depicted with contrast enhancement (signal enhancement) due to a contrast agent.
[0100] A second MRI image (2) is provided showing the same liver or the same part of the liver as in the first MRI image, with healthy liver tissue (parenchyma) being contrast-enhanced (signal-enhanced) due to the contrast agent.
[0101] A first MRI image (1) and a second MRI image (2) are sent to a prediction model (PM).
[0102] The prediction model (PM) is configured to generate, based on the first MRI image (1) and the second MRI image (2), a third MRI image (3) representative of an MRI image without contrast enhancement caused by a contrast agent.
[0103] The predictive model is preferably created using a self-learning algorithm in a process of supervised machine learning using a training data set that includes a large number of first, second and related third MRI images, the third MRI images actually recorded prior to administration of, for example, a first intravenous bolus of contrast agent.
[0104] Self-learning algorithms generate statistical models based on training data during machine learning. This means that examples are not simply memorized, but the algorithm "recognizes" patterns and regularities in the training data. The algorithm is therefore also capable of evaluating unknown data. Validation data can be used to check the quality of the evaluation of unknown data.
[0105] The self-learning algorithm is trained using supervised learning, i.e., the first and second MRI images are passed to the algorithm and it is informed which third MRI images are associated with the particular first and second MRI images, and the algorithm then learns the relationships between the MRI images in order to predict (calculate) the third MRI image for unknown first and second MRI images.
[0106] Self-learning algorithms trained using supervised learning have been widely described in the prior art (see, for example, C. Perez: Machine Learning Techniques: Supervised Learning and Classification, Amazon Digital Services LLC - Kdp Print Us, 2019, ISBN 1096996545, 9781096996545).
[0107] Preferably, the predictive model is an artificial neural network.
[0108] Such an artificial neural network includes at least three layers of processing elements: a first layer having input neurons (nodes), an Nth layer having at least one output neuron (node), and (N-2) hidden layers, where N is a natural number greater than 2.
[0109] The input neurons serve to receive the digital MRI image as input. Typically there is one input neuron for each pixel or voxel of the digital MRI image. There may be additional input neurons for additional inputs (e.g. information about the examination region, about the examination subject, and / or about the conditions prevailing when the MRI image was generated).
[0110] In such a network, the output neurons serve to generate a third MRI image relative to the first and second MRI images.
[0111] The processing elements of the layer between the input neurons and the output neurons are connected to each other in a predetermined pattern with predetermined connection weights.
[0112] Preferably, the artificial neural network is a so-called Convolutional Neural Network (abbreviated as CNN).
[0113] Convolutional neural networks can process input data in the form of matrices. This allows the use of digital MRI images depicted as matrices (e.g., width x height x color channels) as input data. In contrast, a regular neural network, for example in the form of a multi-layer perceptron (MLP), requires a vector as input, i.e., to use an MRI image as input, and the pixels or voxels of the MRI image must be fed out in a long chain in succession. As a result, a regular neural network cannot, for example, recognize an object in an MRI image independently of the object's location in the MRI image. The same object at a different location in the MRI image would have a completely different input vector.
[0114] A CNN essentially consists of alternating layers of filters (convolutional layers) and aggregation (pooling layers), and finally one or many layers of "normal" fully connected neurons (densely / fully connected layers).
[0115] More details can be found in the prior art (see, for example, "S. Khan et al.: A Guide to Convolutional Neural Networks for Computer Vision, Morgan & Claypool Publishers 2018, ISBN 1681730227, 9781681730226").
[0116] Neural networks can be trained, for example, using backpropagation. In this regard, what is required of the network is to map a given input vector to a given output vector as reliably as possible. The mapping quality is described by an error function. The goal is to minimize the error function. In backpropagation, an artificial neural network is taught by modifying the connection weights.
[0117] In the trained state, the connection weights between the processing elements contain information about the relationship between the contrast-enhanced MRI images of the dynamic and hepatic contrast phases and the non-contrast-enhanced MRI images that can be used to predict one or more MRI images showing the examination region without contrast enhancement, calculated using only the contrast-enhanced MRI images of the same examination region.
[0118] Cross-validation can be used to classify the data into a training dataset and a validation dataset. The training dataset is used to train the network weights using backpropagation. The validation dataset is used to check the accuracy of the predictions, and the trained network can be applied to a number of unknown MRI images with the validation dataset.
Claims
1. - receiving at least one first MRI image of the subject, showing a liver or a part of the liver of said subject, said at least one first MRI image being contrast-enhanced due to a contrast agent in said liver or in said part of said liver; - receiving at least one second MRI image of the subject showing the liver or said portion of the liver, healthy liver cells being visualized with enhanced contrast due to a contrast agent; sending the at least one received first MRI image and the at least one received second MRI image to a predictive model, the predictive model being trained using supervised learning to predict one or more MRI images showing the liver or the portion of the liver of the subject without contrast enhancement caused by a contrast agent based on the third MRI image showing the liver or the portion of the liver of the subject, where the blood vessels in the liver or the portion of the liver are visualized with contrast enhancement due to a contrast agent, and based on the fourth MRI image of the liver or the portion of the liver of the subject, where healthy liver cells are visualized with contrast enhancement due to a contrast agent; - receiving from said prediction model said one or more predicted MRI images showing said liver or said portion of said liver of said subject, without contrast enhancement caused by a contrast agent; - displaying and / or outputting said predicted MRI image(s) and / or storing said predicted MRI image(s) on a data storage medium; The method includes:
2. 2. The method of claim 1, wherein the at least one first MRI image is a T1-weighted representation of the liver or the portion of the liver in a dynamic phase after administration of a paramagnetic contrast agent in the hepatobiliary system.
3. The at least one first MRI image is (i) an MRI image showing the liver or the portion of the liver of the subject in the arterial phase; (ii) an MRI image showing the liver or the portion of the liver of the subject in the venous phase; (iii) an MRI image showing the liver or the portion of the liver of the subject in a late phase; The method according to claim 2, wherein
4. 4. The method according to any one of claims 1 to 3, wherein the at least one second MRI image is a T1-weighted representation of the liver or said portion of the liver in a hepatic imaging phase after administration of a paramagnetic contrast agent or a paramagnetic extracellular contrast agent in the hepatobiliary system.
5. 5. The method according to claim 2, wherein the at least one second MRI image with a T1-weighted representation of the liver or said part of the liver is recorded in a hepatic imaging phase after a first administration of a paramagnetic contrast agent in the hepatobiliary system to the examination subject, and the at least one first MRI image with a T1-weighted representation of the liver or said part of the liver is recorded in a dynamic phase after a second administration of a paramagnetic contrast agent or a paramagnetic extracellular contrast agent in the hepatobiliary system to the examination subject.
6. The method according to any one of claims 1 to 5, wherein the contrast agent is a substance or mixture having gadoxetic acid or a gadoxetate salt as a contrast enhancing active substance, preferably the disodium salt of gadoxetic acid.
7. The method according to any one of claims 1 to 6, wherein the test subject is a mammal, preferably a human being.
8. The method according to any one of claims 1 to 7, wherein the predictive model is an artificial neural network.
9. A receiving unit; A control calculation unit; An output unit; Including, the control and computing unit is configured to prompt the receiving unit to receive at least one first MRI image of the subject, showing a liver or a part of a liver of the subject, blood vessels in the liver or in the part of the liver being depicted with contrast enhancement due to a contrast agent; the control and computing unit is configured to prompt the receiving unit to receive at least one second MRI image of the subject, showing the liver or the portion of the liver, healthy liver cells being visualized with enhanced contrast due to the contrast agent; the control and computing unit is configured to predict one or more MRI images based on the received at least one first MRI image and the received at least one second MRI image, the predicted one or more MRI images showing the liver or the portion of the liver of the subject without contrast enhancement caused by a contrast agent; the control and computing unit is configured to display the predicted MRI image(s) and prompt the output unit to output or store the predicted MRI image(s) in a data storage medium; system.
10. A computer program product including a computer program loadable into a memory of a computer, the computer program causing the computer to: - receiving at least one first MRI image of the subject, showing a liver or a part of the liver of said subject, said at least one first MRI image being contrast-enhanced due to a contrast agent in said liver or in said part of said liver; - receiving at least one second MRI image of the subject showing the liver or said portion of the liver, healthy liver cells being visualized with enhanced contrast due to a contrast agent; sending the at least one received first MRI image and the at least one received second MRI image to a predictive model, the predictive model being trained using supervised learning to predict one or more MRI images showing the liver or the portion of the liver of the subject without contrast enhancement caused by a contrast agent based on the third MRI image showing the liver or the portion of the liver of the subject, where the blood vessels in the liver or the portion of the liver are visualized with contrast enhancement due to a contrast agent, and based on the fourth MRI image of the liver or the portion of the liver of the subject, where healthy liver cells are visualized with contrast enhancement due to a contrast agent; receiving the one or more predicted MRI images showing the liver or the portion of the liver of the subject, without contrast enhancement caused by a contrast agent, output from the prediction model; - displaying and / or outputting said predicted MRI image(s) and / or storing said predicted MRI image(s) on a data storage medium; A computer program product that prompts the user to execute
11. A computer program product according to claim 10, wherein the computer program causes the computer to perform one or more of the steps recited in claims 1 to 7.
12. 1. A method for using a contrast agent in an MRI system, the MRI system comprising: - administering said contrast agent which diffuses in the liver or in a part of the liver of the subject; - generating at least one first MRI image showing the liver or said portion of the liver of the examined subject, in which blood vessels in the liver or said portion of the liver are depicted with enhanced contrast due to the contrast agent; - generating at least one second MRI image showing the liver or said portion of the liver, healthy liver cells being visualized with enhanced contrast due to the contrast agent; sending the generated at least one first MRI image and the generated at least one second MRI image to a predictive model, the predictive model being trained using supervised learning to predict one or more MRI images showing the liver or the portion of the liver of the subject without contrast enhancement caused by a contrast agent based on the third MRI image showing the liver or the portion of the liver of the subject, where the blood vessels in the liver or the portion of the liver are visualized with contrast enhancement due to a contrast agent, and based on the fourth MRI image of the liver or the portion of the liver of the subject, where healthy liver cells are visualized with contrast enhancement due to a contrast agent; receiving the one or more predicted MRI images showing the liver or the portion of the liver of the subject, without contrast enhancement caused by a contrast agent, output from the prediction model; - displaying and / or outputting said predicted MRI image(s) and / or storing said predicted MRI image(s) on a data storage medium; A method for using a contrast agent, including
Citation Information
Patent Citations
Magnetic resonance imaging apparatus
JP2002165775A
Use of contrast agents in the manufacture of diagnostic agents for visualization of the intestinal lumen
JP2003519200A
Medical image diagnostic apparatus and method using liver angiographic image, and program
JP2011092677A
Hepatic function diagnostic apparatus, MRI machine, and hepatic function diagnostic method
JP2011167408A
Image processing device, and method and program for controlling image processing device
JP2017064370A
